Devices, systems and methods for digital video processing, which includes intra mode coding based on history information, are described. In a representative aspect, a method for video processing includes selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, a first intra prediction mode based on at least a first set of history intra coding information that includes statistical information of a set of intra prediction modes, and performing the conversion based on the first intra prediction mode.