A video coder can be configured to receive in a video parameter set, one or more syntax elements that include information related to hypothetical reference decoder (HRD) parameters; receive in the video data a first sequence parameter set that includes a first syntax element identifying the video parameter set; receive in the video data a second sequence parameter set that includes a second syntax element identifying the video parameter set; and, code, based on the one or more syntax elements, a first set of video blocks associated with the first parameter set and second set of video blocks associated with the second parameter set.