An image processing apparatus includes an image input unit that receives time-series images, and a candidate area tracking unit that extracts a line segment in a specific direction, sets an obstacle candidate area in a vicinity of the detected line segment, and searches the vicinity of the line segment extracted in each frame, thus tracking the obstacle candidate area with low computational cost. Using the tracking result of tracking a group of three or more candidate areas, a plane assumption verifier determines to which of the preset planes these candidate areas belong. On the basis of the result by the plane assumption verifier, an obstacle detector detects an obstacle and estimates the position of the obstacle.