Patent attributes
A method is provided for the recovery of an instance of a job manager running on one of a plurality of nodes used to execute the processing elements associated with jobs that are executed within a cooperative data stream processing system. The states of the processing elements are checkpointed to a persistence mechanism in communication with the job manager. From the checkpointed processing element states, the state of each distributed job is determined and checkpointed. Processing element states are also checkpointed locally to the nodes one which the processing elements are running. Following a failure of the job manager, the job manager is reinstantiated on one of the nodes. The recovery instance of the job manger obtains state data for processing elements and jobs from the persistence mechanism and constructs an initial state for jobs and processing elements. These initial states are reconciled against the current states of the processing elements and adjustments are made accordingly. Once the job and processing element states are reconciled, the system is returned to normal operation.
