A gas turbine engine comprises a main compressor section having a high pressure compressor with a downstream most end, and more upstream locations. A turbine section has a high pressure turbine. A first tap taps air from at least one of the more upstream locations in the main compressor section, passes the tapped air through a heat exchanger and then to a cooling compressor. The cooling compressor compresses air downstream of the heat exchanger. A second tap taps air from a location closer to the downstream most end than the location(s) of the first tap. The first and second tap mix together and are delivered into the high pressure turbine. An intercooling system for a gas turbine engine is also disclosed.