A plurality of metrics records, including some records indicating metrics for which anomaly analysis has been performed, is obtained. Using a training data set which includes the metrics records, a machine learning model is trained to predict an anomaly analysis relevance score for an input record which indicates a metric name. Collection of a particular metric of an application is initiated based at least in part on an anomaly analysis relevance score obtained for the particular metric using a trained version of the model.