A method of handling an uplink (UL) timing of a timing advance (TA) group for a mobile device in a wireless communication system is disclosed. The method comprises maintaining the UL timing of the TA group, if a time alignment timer of the TA group does not expire; and stopping maintaining the UL timing of the TA group, when at least one secondary cell in the TA group is deactivated and the time alignment timer of the TA group does not expire, wherein the at least one secondary cell is configured to the mobile device by a network of the wireless communication system.