US Patent 7890978 Generic data processing engine

A generic data processing engine is operable to receive a format definition and process data formatted according to the definition, without use of formatting information in the data. The format definition includes a description of the syntax of the format, and a description of the semantics of the format. The syntax and semantics may be described in the same language or in different languages, and the engine is configured to produce an internal representation of the syntax and semantics. The engine receives queries and uses them together with the internal representation to set masks for the filters. The filters apply the masks to the data and return filtered data to the engine, which may forward a portion of the filtered data to applications, store a portion of the filtered data, set new masks based on a portion of the filtered data, or modify the existing masks based on a portion of the filtered data. The filters may also be configured to return filtered data directly to applications, bypassing the engine.