Designing for Gesture and Tangible Interaction. Mary Lou Maher
4.3.1 Considering How to Engage People
4.3.2 Designing Clear and Consistent Visual Feedback
4.3.3 Selecting Ergonomic Gestures
4.3.4 Designing Discoverable Gestures
4.3.5 Inaccurate Gesture Recognition
5 Designing for Gesture Interaction
5.1 Designing the Walk-Up-and-Use Information Display
5.1.1 Context of Use
5.1.2 Design Goals
5.1.3 Technology Selection
5.1.4 Design Methods
5.1.5 Interaction Model
5.1.6 Design Heuristics
5.2 Designing the Willful Marionette
5.2.1 Context of Use
5.2.2 Design Goals
5.2.3 Technology Selection
5.2.4 Design Methods
5.2.5 Interaction Model
5.2.6 Design Guidelines
6 Looking to the Future: Research Challenges
6.1 Design Challenges for Embodied Interaction
6.1.1 Understanding Embodied Actions as User Input
6.1.2 Designing Gestures with Effective Visual Feedback
6.1.3 Understanding Sensing Technology
6.2 Research Challenges for Embodied Interction
6.2.1 Gesture Design
6.2.2 Multidisciplinary Research
6.2.3 Finding Appropriate Contexts of Use
6.2.4 Limited Scope of Representation
6.2.5 Cognitive and Physical Fatigue
Preface
This book reflects on the significance and design of embodied interaction with digital information and artifacts. We are experiencing a transition from traditional modes of interacting with computing devices in which we are typically sitting still and moving our fingers on a keyboard to large body movements to effect changes and engage with digital information and artifacts. As designers we are challenged not only by the increasing range of technologies that enable interaction design, but also by the range and focus on human-centered design methodologies. Engineering design starts with requirements derived from human needs but necessarily has a focus on the design of the system so that it satisfies those requirements and optimizes performance of the system. In contrast, in interaction design, the human is inherent to the system and therefor the focus remains on human needs, desires, and abilities throughout the design process. Along with this focus on people during the design of interactive systems, there is an opportunity to move beyond human factors and physical considerations to consider the social and cognitive effects of alternative designs. These opportunities create a new era in interaction design that includes such things as designing gestures and a stronger focus on physical and digital affordances and metaphors. This book is a starting point for understanding the significance of this transition and is a harbinger for future interaction designs in which large body movements are the basis for interaction. Not only is embodied interaction creating new modalities for interaction, it is also redefining the focus of good interaction design by moving away from efficiency and productivity as the basis for interaction design toward the inclusion of creativity and social interaction in the goals for new designs.
Mary Lou Maher and Lina Lee
January 2017
Acknowledgments
This book is a reflection on our collaboration with many colleagues whose views and ideas are tightly woven within our understanding of tangible and gesture interaction. After many years of collaboration, it is hard to untangle our thoughts from the engaging discussions about tangible and gesture interaction. We acknowledge our colleagues here as an integral part of our ability to produce this book. Our understanding of tangible interaction design and our design examples, Tangible Keyboard and Tangible Models, were influenced by Tim Clausner, Mijeong Kim, Alberto Gonzalez, and Kaz Grace. Our understanding of gesture interaction design and our design examples, walk-up-and-use information display and the willful marionette, were influenced by collaboration with our artists in residence, Lilla LoCurto and Bill Outcault, and our colleagues Kaz Grace and Mohammad Mahzoon. Lilla and Bill are the artisits that imagined, created, and built the willful marionette.
Completing this book was possible only with the patience of our families. Our special thanks go to the babies born during this writing project and the many sleepless nights we experienced. We particularly acknowledge the birth of Jessica, Kei, Iris, Matthew, and Erin during the time we were writing this book.
The tangible interaction research reported in this book was partially supported by NSF Grant IIS-1218160: HCC: Small: Designing Tangible Computing for Creativity. The artist-in-residence program that enabled our contribution to the willful marionette was supported by the College of Computing and Informatics and the College of Arts and Architecture at UNC Charlotte. We acknowledge the support of the College of Computing and Informatics at UNC Charlotte for funding graduate students and equipment in the InDe Lab.
Mary Lou Maher and Lina Lee
January 2017
CHAPTER 1
Introduction
In this book we explore the design issues for embodied interaction design, and specifically for tangible and gesture interaction. This book describes engaging ways of interacting with tangible and gesture-based interactive systems through four different examples as vehicles for exploring the design issues and methods relevant to embodied interaction design. In recent years, new interaction styles have emerged. Researchers in human-computer interaction (HCI) have explored an embodied interaction that seeks to explain bodily action, human experiences, and physicality in the context of interaction with computational technology (Antle et al., 2009, 2011; Klemmer et al., 2006). Dourish (2004) set out a theoretical foundation of embodiment. The concept of embodiment in tangible user interfaces (TUIs) describes how physical objects may be used simultaneously as input and output for computational processes. Similarly, in gesture-based interaction the concept of embodiment recognizes and takes advantage of the fact that humans have bodies, and people can use those bodies when interacting with technology in the same ways they use them in the natural physical world (Antle et al., 2011). This is an attractive approach to interaction design because it relates to our previous experience and makes it easier to learn new systems.
The success of interaction design depends on providing appropriate methods for the task at hand that improve discoverability and learnability. Designers should consider the user’s mental model based on previous experience when defining how the user can interact with the system, and then give the user clues about expected behavior before they take action. Giving feedback to the users is important to make it clear how the user completes