US Patent 11936917 Processing of filler data units in video streams

A method of video processing includes performing a conversion between a video and a bitstream of the video according to a rule, wherein the rule specifies that a sub-bitstream extraction process is implemented to generate a sub-bitstream for decoding, wherein the sub-bitstream extraction process is configured to extract, from the bitstream, a sub-bitstream with a target highest temporal identifier, and wherein, the rule specifies that, during the extracting, upon removing a video coding layer (VCL) network abstraction layer (NAL) unit, filler data units and filler supplemental enhancement information (SEI) messages in SEI NAL units that are associated with the VCL NAL unit are also removed.