US Patent 11880648 Automatic semantic labeling of form fields with limited annotations

Embodiments provide systems, methods, and computer storage media for extracting semantic labels for field widgets of form fields in unfilled forms. In some embodiments, a processing device accesses a representation of a fillable widget of a form field of an unfilled form. The processing device generates an encoded input representing text and layout of a sequence of tokens in a neighborhood of the fillable widget. The processing device uses a machine learning model to extract a semantic label representing a field type of the fillable widget in view of the encoded input. The processing device causes execution of an action using the semantic label.