A technique of extracting hierarchies for multilabel classification. The technique can process a plurality of labels related to a plurality of documents, using a clustering process, to cluster the labels into plurality of clusterings representing a plurality of classes. The technique classifies the documents and predicts a plurality of performance characteristics, respectively, for the plurality of clusterings. The technique selects at least one of the clusterings using information from the performance characteristics and adds the selected clustering into a resulting hierarchy.