The shape and orientation of rigid or nearly rigid moveable bodies are determined using a shape characterization. Sensors capture a plurality of representations of different perspectives of the body that are analyzed to determine a bounding volume of the body. The shape of the body is determined from the bounding volume. The position of the body is determined using tracking devices that sense the position of the body. The bounding volume and position information are combined to define the shape and orientation in space of the body, and in particular the position of a point of interest on the body.