Designing for Gesture and Tangible Interaction. Mary Lou Maher

       ABSTRACT

      Interactive technology is increasingly integrated with physical objects that do not have a traditional keyboard and mouse style of interaction, and many do not even have a display. These objects require new approaches to interaction design, referred to as post-WIMP (Windows, Icons, Menus, and Pointer) or as embodied interaction design.

      This book provides an overview of the design opportunities and issues associated with two embodied interaction modalities that allow us to leave the traditional keyboard behind: tangible and gesture interaction. We explore the issues in designing for this new age of interaction by highlighting the significance and contexts for these modalities. We explore the design of tangible interaction with a reconceptualization of the traditional keyboard as a Tangible Keyboard, and the design of interactive three-dimensional (3D) models as Tangible Models. We explore the design of gesture interaction through the design of gesture-base commands for a walk-up-and-use information display, and through the design of a gesture-based dialogue for the willful marionette. We conclude with design principles for tangible and gesture interaction and a call for research on the cognitive effects of these modalities.

       KEYWORDS

      embodied interaction, gesture interaction, tangible interaction, interaction design, cognitive effects

       Contents

       Preface

       Acknowledgments

       1 Introduction

       1.1 Embodiment

       1.2 Affordance

       1.3 Metaphor

       1.4 Epistemic Actions

       1.5 This Book

       2 Tangible Interaction Design

       2.1 What is a Tangible Interaction?

       2.1.1 Tangible Keyboard

       2.1.2 Tangible Models

       2.2 Why is Tangible Interaction Interesting?

       2.2.1 Space-Multiplexed Input and Output

       2.2.2 Concurrency

       2.2.3 Strong Specific Devices

       2.2.4 Spatially Aware Computational Devices

       2.2.5 Spatial Reconfigurability of Devices

       2.3 What Are Key Design Issues for Tangible User Interfaces?

       2.3.1 Designing Interplay of Virtual and Physical

       2.3.2 Selecting from Multiple Gestures and Actions

       2.3.3 Crossing Disciplinary Boundaries

       2.3.4 The Lack of Standard Input and Output Interaction Models

       3 Designing for Tangible Interaction

       3.1 Designing the Tangible Keyboard

       3.1.1 Context of Use

       3.1.2 Design Goals

       3.1.3 Technology Selection

       3.1.4 Design Methods

       3.1.5 Interaction Model

       3.1.6 Design Heuristics

       3.2 Designing Tangible Models

       3.2.1 Context of Use

       3.2.2 Design Goals

       3.2.3 Technology Selection

       3.2.4 Design Methods

       3.2.5 Interaction Model

       3.2.6 Design Guidelines

       4 Gesture-based Interaction

       4.1 What Is a Gesture-based Interaction?

       4.1.1 Gesture Commands: Walk-Up-and-Use Information Display

       4.1.2 Gesture dialogue: The Willfull Marionette

       4.2 Why is Gesture-based Interaction Interesting?

       4.2.1 Public Walk-Up-and-Use Displays or Objects

       4.2.2 Remote Actuators, Sensors, and Displays

       4.2.3 Sociable Multiple-User Interaction

       4.2.4 Gesture Communication

       4.3 What Are Key Design Issues for Gesture-based Interaction?