US Patent 7328430 Method for analyzing data and performing lexical analysis

A system and method provide the ability to construct lexical analyzers on the fly in an efficient and pervasive manner. The system and method split the table describing the automata into two distinct tables and splits the lexical analyzer into two phases, one for each table. The two phases consist of a single transition algorithm and a range transition algorithm, both of which are table driven and permit the dynamic modification of those tables during operation. A third ‘entry point’ table may also be used to speed up the process of finding the first table element from state 0 for any given input character.