A fuel cell humidifier includes a membrane module accommodating a humidifying membrane, a first cap coupled to a first side of the membrane module and supplying a supply air to the humidifying membrane, and a second cap coupled to a second side of the membrane module and discharging a humidified supply air from the humidifying membrane to a cathode of a fuel cell stack. An exhaust gas inlet is coupled to the membrane module and provides an exhaust gas discharged from the stack and an exhaust gas outlet is coupled to the membrane module and discharges a dehumidified exhaust gas, which passes through the membrane module, to an exhaust system. A valve provides selective communication between a first region, where the supply air flows, including the first and second caps and a second region, where the exhaust gas flows, including the exhaust gas inlet and the exhaust gas outlet.