A Dynamic AI System for Extending the Capabilities of Blind People - Grayson, Martin; Thieme, Anja; Marques, Rita; Massiceti, Daniela; Cutrell, Ed; Morrison, Cecily (CHI 2020 Extended Abstracts, 2020)
We introduce an advanced computer vision-based AI system that offers people with vision impairments (VI) dynamic, in-situ access to information about the location, identity and gaze-direction of other people nearby. Our AI system utilizes the camera technology of a head-worn HoloLens device, which captures a near 180˚ field-of-view surrounding the person who is wearing it. Captured images are then processed by multiple state-of-the-art perception algorithms whose outputs are integrated into a real-time tracking model of all people that the system detected. Users can receive information about those people acoustically (via spatialized audio) using a wrist-worn input controller. Having such dynamic access to information, through AI system interactions, enables people with VI to: develop their communication skills; more easily focus on others; and be more confident in their social interactions. Thus, our work explores how AI systems can serve as a useful resource for humans, helping expand their agency to develop new or extend existing skills.
DOI: https://doi.org/10.1145/3334480.3383142