Academic
Publications
MODALITY THEORY: SUPPORTING MULTIMODAL INTERFACE DESIGN

MODALITY THEORY: SUPPORTING MULTIMODAL INTERFACE DESIGN,Niels Ole Bernsen

MODALITY THEORY: SUPPORTING MULTIMODAL INTERFACE DESIGN   (Citations: 14)
BibTex | RIS | RefWorks Download
Modality theory addresses the following general problem of mapping task domain information into interactive multimodal interfaces: given any particular set of information which needs to be exchanged between user and system during task performance in context, identify the input/output modalities which constitute an optimal solution to the representation and exchange of that information. This paper proposes a research agenda for modality theory and presents two steps towards its implementation. The first is a generative taxonomy of output modalities covering the media of graphics, sound and touch. The second is a methodology for carrying out information-mapping in design practice. It is argued that modality theory may provide useful support to contemporary designers of interactive human- computer interfaces who have begun to use a rapidly increasing number of different, and often alternative input/output modalities for the expression and exchange of information between systems and their users.
Cumulative Annual
    • ...Interactions can take place in several modes, for instance a text modality or a manipulation modality (here called 'continuous', described as 'analog' in Niels Ole Bernsen's Modality Theory [29])...

    Bert Bongerset al. Towards a Multimodal Interaction Space: categorisation and application...

    • ...At the same time, theoretical frameworks, such as modality theory [4, 5], TYCOON [6], CARE [7] and others constitute a solid basis for multimodal system design, but ignore context of use modeling and issues of rapid application design...

    Jung-hyun Kimet al. A Pattern Recognition-Based Multi-Modal HCI Using 3 Sensory Channels

    • ...At the same time theoretical frameworks, such as modality theory [3,4], TYCOON [19],...
    • ...We have already analysed literature on modality theory [3,4,5,6,31] and context of use analysis [8,9], both presented in [24]...

    Andreas Ratzkaet al. A Pattern-based Methodology for Multimodal Interaction Design

    • ...This notion of equivalence vs. specialization [12] or equivalence vs. assignment [10] can be applied to interaction modalities in general but also to single modality properties, such as the ones covered by modality theory [3], [4]...
    • ...The first subsection describes modality theory developed by Bernsen [3], [4]...
    • ...Bernsen introduced a comprehensive taxonomy of output modalities [3], [4]...

    Andreas Ratzka. Combining Modality Theory and Context Models

    • ...For example, information content i s characterised o r classified as linguistic or non-linguistic, analogue or non-analogue, arbitrary or non-arbitrary, static or dynamic, etc [13], by the data types, properties of relational structure, arity, user information seeking goal, etc [14], by the intrinsic property, class property, set property, etc [9], etc...

    Constantine Stephanidiset al. Decision making in intelligent user interfaces

Sort by: