History based motion vector prediction is disclosed. In one example method of video processing, initializing, for a conversion between a first region of video and a bitstream of the first region, a first history motion vector predictor (HMVP) table associated with the first region based on at least one of stored multiple HMVP tables, the multiple HMVP tables being used to store motion information associated with regions coded prior the first region; and performing the conversion by using the initialized first HMVP table which includes at least one motion candidate.