Search CALO Publications:

Author and Title
Edward C.Kaiser. SHACER: a Speech and Handwriting Recognizer. Workshop Proceedings of the Seventh International Conference on Multimodal Interfaces (ICMI 2005), Workshop on Multimodal, Multiparty Meeting Processing, Oct. 7, 2005, Trento, Italy.

Abstract
Within the task domain of a multi-party, multimodal meeting focused on the creation of a whiteboard schedule chart, we have designed and implemented a general method of aligning handwriting and speech for capturing out-of-vocabulary terms, dynamically enrolling them in the system’s recognition modules, and then using them to improve subsequent tracking and recognition. Our approach involves the use of an ensemble of
syllable and phoneme recognizers for speech whose output is integrated with redundantly delivered handwriting recognition. We refer to our conceptual framework as Multimodal Out-Of-Vocabulary Recognition (MOOVR — pronounced mover). Within that framework this paper describes our Speech and HAndwriting reCognizER module (SHACER — pronounced shaker), which observes human-to-human spoken and handwritten interactions, analyzes them off-line and contributes improved
recognitions to a record of the meeting in the form of a project schedule. We examine an example meeting and show how our technique corrects four of five label recognition errors including implicitly discovering the semantics of a handwritten abbreviation.

Download