US Patent 10146785 Operator-guided application crawling architecture

A system for automated acquisition of content from an application includes a guide tracker module, a link extraction controller, and a scraper. The guide tracker module monitors interaction of an operator with an executing instance of the application and records a set of guides. Each guide includes a recorded sequence of user interface interactions concluding at a respective ultimate state of the application. The link extraction controller, for each guide of the set of guides, selectively identifies additional states of the application that correspond to the respective ultimate state and adds the additional states corresponding to the respective ultimate state and the respective ultimate state to a state list. The additional states and the respective ultimate state are all directly reachable from a common penultimate state of the application. Each entry in the state list designates a state and a path of user interface interactions to arrive at the state.