A semiconductor memory device includes a substrate, a plurality of first control gate electrodes, a plurality of second control gate electrodes, first to second select gate electrodes, first to second gate electrodes, a bit line, first to second semiconductor pillars, and a controller. The controller applies a first potential to the first select gate electrode, a third potential lower than the first potential to the second select gate electrode, a second potential to the first gate electrode and the second gate electrode, a selecting potential not less than the third potential to one of the plurality of the first control gate electrodes, and an unselecting potential higher than the selecting potential to other than the one of the plurality of first control gate electrodes in a reading operation.