A pattern inspection apparatus includes a column to scan a substrate on which a pattern is formed, using multi-beams composed of a plurality of electron beams, a stage to mount the substrate thereon and to be movable, a detector to detect secondary electrons emitted from the substrate because the substrate is irradiated with the multi-beams, and a drive mechanism to move the detector in order to follow movement of the stage.