An in-the-wild study of learning mid-air gestures to browse hierarchical information at a large interactive public display

Abstract

This paper describes the design and evaluation of our Media Ribbon, a large public interactive display for browsing hierarchical information, with mid-air gestures. Browsing a hierarchical information space is a fundamental form of interaction. Designing learnable mid-air gestures is a current challenge for large display interaction. Our in-the-wild evaluation draws on 41 days of quantitative log data, with 4484 gestures detected, and qualitative data from 15 interviews, and associated video. We explored: whether our design enabled people to learn the gestures; how our tutorial and feedback mechanisms supported learning; and the effectiveness of support for browsing hierarchical information. Our contributions are: (1) design of large public display for browsing of hierarchical information; (2) with its gesture set; (3) insights into the ways people learn and use this interface in our context; and (4) guidelines for designing learnable mid-air gestures.