A power management controller tracks the idle state of a compute unit and compares the tracked idle state with a first threshold. If the tracked idle state is above the first threshold a power state of the compute unit is limited to a low power state so that the power state does not rise due to activity that occurs in low utilization scenarios. The tracked idle state is compared to a second threshold and if the tracked idle state is below the second threshold, indicating that the compute unit is not in a low utilization scenario, a limit on the power state is removed and the power state of the compute unit is allowed to rise.