Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for machine learning for home understanding and notification. In one aspect, a method includes obtaining reference videos from a camera within a home, determining from the reference videos that a particular person routinely leaves the home with a particular object at a particular time of day, determining from a sample video from the camera within the home that the particular person appears to be leaving the home without the particular object at the particular time of day, and in response, providing a notification regarding the particular object.