The invention provides a motion prediction method. First, a plurality of motion vector predictors is obtained to be included in a candidate set for motion prediction of a current unit of a current frame. Whether the current frame is a non-reference frame which is not referred to by other frames for motion prediction is then determined. When the current frame is not the non-reference frame, any motion vector predictor corresponding to a previously coded frame is removed from the candidate set, and a motion vector of the current unit is predicted according to the motion vector predictors of the candidate set.