A semiconductor memory device has a plurality of banks in which operations are performed for the banks in accordance with a command supplied from an external controller. The semiconductor memory device comprises a latch circuit for latching a bank selection signal indicative of a bank which was precharged last among the banks before execution of an auto refresh command for sequentially refreshing the banks, and a refresh control circuit, responsive to the execution of the auto refresh command, for controlling the order in which the banks are refreshed according to the auto refresh command such that a bank which is selected by the latched selection signal is refreshed last among the banks.