US Patent 11294884 Annotation assessment and adjudication

Embodiments relate to an intelligent computer platform to construct ground truth data. A document is subjected to NLP and grammatical elements of the document are identified. The identified grammatical elements are annotated and one or more annotations are assigned to the elements, with each annotation having corresponding meta-data. The assigned annotations are assessed for accuracy and a score associated with the meta-data is adjusted. The annotations are selectively re-assigned with respect to the identified grammatical elements responsive to the adjusted score. Ground truth data is constructed and the selectively re-assigned annotations are assigned to corresponding and identified grammatical elements.