Patent attributes
The present invention relates to a method of accurately evaluating a system summary of a document based on one or more predefined units. The method includes dividing the document and the system summary of the document into the one or more predefined units, sentences and phrases, for example. For each of reference summaries for the document, the method generates an oracle. The oracle is a partial set of units that meet a length limitation and maximize a score generated by an evaluation function for a partial set of the units of the document based on the reference summary. The method further includes determines, based on the generated oracle, scores of the respective units included in the set of the oracle. The method further includes determining the score of the system summary based on the score of the system summary unit.
