US Patent 10528818 Video scene analysis system for situational awareness

Described is a video scene analysis system. The system includes a salience module that receives a video stream having one more pairs of frames (each frame having a background and a foreground) and detects salient regions in the video stream to generate salient motion estimates. The salient regions are regions that move differently than dominant motion in the pairs of video frames. A scene modeling module generates a sparse foreground model based on salient motion estimates from a plurality of consecutive frames. A foreground refinement module then generates a Task-Aware Foreground by refining the sparse foreground model based on task knowledge. The Task-Aware Foreground can then be used for further processing such as object detection, tracking or recognition.