A constraint-based speech recognition system for use with a form-filling application employed over a telephone system is disclosed. The system comprises an input signal, wherein the input signal includes both speech input and non-speech input of a type generated by a user via a manually operated device. The system further comprises a constraint module operable to access an information database containing information suitable for use with speech recognition, and to generate candidate information based on the non-speech input and the information database, wherein the candidate information corresponds to a portion of the information. The system further comprises a speech recognition module operable to recognize speech based on the speech input and the candidate information. In an exemplary embodiment, the manually operated device is a touch-tone telephone keypad, and the information database is a lexicon encoded according to classes defined by the keys of the keypad.