Patent attributes
Aspects of the present disclosure provide systems, methods, and computer-readable storage media that support creation and execution of automated sequences of user interface (UI) interactions. To facilitate creation and execution of automated sequences of UI interactions, an automation engine is provided and includes a model configured to capture image data when creating the sequence of UI interactions. The model may also be used during replay of the sequence of UI interactions. For example, the model may be used during replay of the sequence of UI interactions to locate UI elements corresponding to the UI interactions, or to perform pre- and/or post-validation of action execution. The automation engine may also provide processes to enable location of dynamic content, such as UI elements that may be presented in different or unexpected locations, and processes to address complex UI elements, such as data grids, tree views, and drawings (e.g., CAD drawings).
