A firearm having a grip panels and backstrap retention system is provided. A latch plate is disposed adjacent to the backstrap side of the frame having a pair of side edges having rearward facing protuberances thereon and is moveable perpendicular to the magazine well axis from an engagement position towards the trigger side of the frame to a disengagement position towards the backstrap side of the frame. A latch plate spring biases the latch plate to the disengagement position. Each grip panel has a forward grip undercut to receive corresponding forward frame undercuts. Each forward grip undercut engages the corresponding forward frame undercut by seating each grip panel in its grip offset surface and translating each grip panel toward the trigger side of the frame. Each grip panel further has a rearward grip undercut facing the frame to receive a corresponding undercut latch plate protuberance. When the latch plate is in the engagement position, the latch plate protuberances engage the rearward grip undercuts.