Search CALO Publications:

Author and Title
Edward C. Kaiser, P. Barthelmess, X. Huang and D. Demirdjian. A Demonstration of Distributed Pointing and Referencing for Multimodal Collaboration Over Sketched Diagrams. Workshop Proceedings of the Seventh International Conference on Multimodal Interfaces (ICMI 2005), Workshop on Multimodal, Multiparty Meeting Processing, Oct. 7, 2005, Trento, Italy.

Abstract
Groups who are not co-located but still need to collaborate on acommon task face the problem of reduced access to the richmultimodal communicative context that they would have if they were collaborating face-to-face. We present a demonstration system that allows participants at remote sites to collaborate in building a project schedule via sketching on multiple distributed whiteboards. We show how participants can be made aware of naturally occurring pointing gestures that reference diagram constituents as those gestures are performed by remote participants. Our system fuses multimodal inputs from pen, speech and 3D gestures, coupled with the dynamic construction of a semantic representation of the interaction, anchored on the sketched diagram, to provide feedback that overcomes some of the intrinsic ambiguities of distant pointing gestures. We also demonstrate how a distributed awareness of the meaning of shared symbols on the whiteboard, like handwritten abbreviations, can be both learned dynamically as the meeting progresses and communicated in unobtrusive ways to remote participants.

Download