Patent attributes
An information retrieval system is described for retrieving a list of documents such as web pages or other items from a document index in response to a user query. In an embodiment a prediction engine is used to predict both explicit relevance information such as judgment labels and implicit relevance information such as click data. In an embodiment the predicted relevance information is applied to a stored utility function that describes user satisfaction with a search session. This produces utility scores for proposed lists of documents. Using the utility scores one of the lists of documents is selected. In this way different sources of relevance information are combined into a single information retrieval system in a principled and effective manner which gives improved performance.