Enclosed are embodiments for navigation with drivable area detection. In an embodiment, a method comprises: receiving a point cloud from a depth sensor, receiving image data from a camera; predicting at least one label indicating a drivable area by applying machine learning to the image data; labeling the point cloud using the at least one label; obtaining odometry information; generating a drivable area by registering the labeled point cloud and odometry information to a reference coordinate system; and controlling the vehicle to drive within the drivable area.