US Patent 10013422 Incremental data processing

Event logs in a video advertisement insertion network are processed to remove duplicate entries. One or more ad servers are continuously generating new event entries and writing them to a database. The entries are randomized such that generated time contiguous entries are distributed over multiple storage locations, thereby facilitating resource scaling and a uniform use of storage and computing resources. The distributed entries are read from the storage locations using sequential reads of chunks of the entries and processed to remove duplicate entries.