In one example, a device for coding (e.g., encoding or decoding) video data includes a memory configured to store video data, and a video coder configured to code a value for a syntax element that indicates a number of sub-layers of a bitstream for which hypothetical reference decoder (HRD) parameters are coded, wherein the value indicates that the number of sub-layers for which HRD parameters are coded is less than a maximum number of sub-layers indicated by a video parameter set (VPS) of the bitstream, code HRD parameters for the number of sub-layers as indicated by the value for the syntax element, and process the bitstream using the HRD parameters.