US Patent 11854230 Physical keyboard tracking

In one embodiment, a method includes the steps of capturing an image from a camera viewpoint, the image depicting a physical keyboard, detecting one or more shape features of the physical keyboard depicted in the image by comparing pixels of the image to a predetermined shape template, the predetermined shape template representing visual characteristics of spaces between keyboard keys, accessing predetermined shape features of a keyboard model associated with the physical keyboard, and determining a pose of the physical keyboard based on comparisons between (1) the detected one or more shape features of the physical keyboard and (2) projections of the predetermined shape features of the keyboard model toward the camera viewpoint.