Multimodal Spatial Querying: What People Sketch and Talk About
Isolde Schlaisich and
Max Egenhofer 1st International Conference on Universal Access in Human-Computer Interaction, New Orleans, LA,
C. Stephanidis (ed.), pp. 732-736, August 2001.
Abstract
People want to use geographic information systems (GISs) while
spending little time for learning how to use the system. This trend
increases the need for easy-to-learn and intuitive user interfaces.
In order to make GISs easier to use for untrained users, we
investigate an interaction method that mimics the natural
communication between people and allows users to formulate queries.
In human-human interaction people often communicate about space by
talking and simultaneously drawing a freehand sketch; therefore,
this combination of modalities is attractive for describing spatial
scenes when querying a GIS. To assess the usefulness of this
approach, we have conducted a human-subject experiment in which
subjects were asked to draw spatial queries and talk at the same
time. We analyzed what modality subjects preferred when they had
the choice of sketching and talking. We found that subjects gave
complementary information through the two modalities, indicating
that sketching cannot replace the verbal descriptions, and vice
versa. We also found redundant information in both modalities,
which provides a solid foundation for integrating elements from the
two modalities.
PDF
Isolde Schlaisich and
Max Egenhofer