Escapement: A Tool for Interactive Prototyping with Video via Sensor-Mediated Abstraction of Time - Nicholas, Molly Jane; Marquardt, Nicolai; Pahud, Michel; Henry Riche, Nathalie; Romat, Hugo; Collins, Christopher; Ledo, David; Kadekodi, Rohan; Chandramouli, Badrish; Hinckley, Ken (CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023)
We present Escapement, a video prototyping tool that introduces a powerful new concept for prototyping screen-based interfaces by flexibly mapping sensor values to dynamic playback control of videos. This recasts the time dimension of video mock-ups as sensor-mediated interaction. This abstraction of time as interaction, which we dub video-escapement prototyping, empowers designers to rapidly explore and viscerally experience direct touch or sensor-mediated interactions across one or more device displays. Our system affords cross-device and bidirectional remote (tele-present) experiences via cloud-based state sharing across multiple devices. This makes Escapement especially potent for exploring multi-device, dual-screen, or remote-work interactions for screen-based applications. We introduce the core concept of sensor-mediated abstraction of time for quickly generating video-based interactive prototypes of screen-based applications, share the results of observations of long-term usage of video-escapement techniques with experienced interaction designers, and articulate design choices for supporting a reflective, iterative, and open-ended creative design process.