US Patent 8364486 Speech understanding method and system

A speech recognition system includes a mobile device and a remote server. The mobile device receives the speech from the user and extracts the features and phonemes from the speech. Selected phonemes, supplemental non-verbal information not based upon or derived from the speech, and measures of uncertainty are transmitted to the server, the phonemes are processed for speech understanding remotely or locally based, respectively, on whether or not the audible speech has a high or low recognition uncertainty, and a text of the speech (or the context or understanding of the speech) is transmitted back to the mobile device.