US Patent 10949174 Automatic classification of user interface elements

Techniques are disclosed relating to classifying user interface elements of existing user interfaces. This may include for example, storing information specifying known metadata values for a plurality of metadata fields and indications of relationships between ones of the known metadata values and a plurality of types of visible user interface elements. The techniques also include determining respective metadata values for a plurality of visible elements of a graphical user interface, where the metadata values are included in user interface code that specifies the plurality of visible elements. The disclosed techniques also include, based on the stored indications of relationships and the determined metadata values, scoring ones of the plurality of visible elements to generate score values for each of the plurality of types of visible elements. Finally, the disclosed techniques include, based on the scoring, classifying the plurality of visible elements according to the plurality of types of visible elements and storing information specifying the classified elements.