An arithmetic device includes an imaging section adapted to take an image of the measurement object, a relative phase calculation section adapted to calculate a relative phase, and a projection control section adapted to perform a first pattern projection and a second pattern projection, and the relative phase calculation section adapted to obtain a grayscale image of a measurement object on which light is projected due to the first pattern projection and a grayscale image of the measurement object on which light is projected due to the second pattern projection from the imaging section to calculate the relative phase using the grayscale images obtained.