Rodolfo Cossovich, Steve Hodges, Jin Kang, and Audrey Girouard. 2023. Co-designing new keyboard and mouse solutions with people living with motor impairments. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '23). Association for Computing Machinery, New York, NY, USA, Article 98, 1–7.
https://doi.org/10.1145/
With this project, we aim to shed more light on the experiences of people with motor impairments with computers and highlight the importance of providing a wider variety of customizable assistive technology offerings. We co-designed solutions with people benefitting from solutions to make their computer interfaces more accessible.
The goal of this project was to contribute to the nascent literature that investigates ways to improve the ways that people with mobility impairments can become more productive with their computers. We also researched how the design of the co-design activities can elicit meaningful dialogue and empathy and how incorporating the Jacdac platform can support brainstorming between researchers, designers, and people with motor impairments. The findings of this project are under revision to be published at the 25th Conference on Computers and Accessibility.
According to the World Health Organization (WHO), around 15 % of the world's population- approximately one billion people- live with a disability. While our understanding of the types and causes of disabilities continues to improve, the incidence rate of disabilities is expected to increase due to our aging global population[4]. Digital technologies can help people with disabilities, so we focused on the ways that people that have a motor impairment can use prototyping technologies to make their own computers more accessible.
We collected data through an initial online survey, recruiting people with various physical disabilities and backgrounds from different geographic areas across China (n = 42). We ran a series of semi-structured follow-up interviews (n = 9) at people’s homes, meeting each interviewee in their living space to understand their computer and smartphone usage better and the way they can use different input methods.
We used thematic analysis[1] to analyze the data, highlighting some current pitfalls of software-based accessibility features, an unfulfilled desire for multimodal input, and the sometimes unorthodox use of additional human-interface devices.
From these insights, we designed and facilitated co-design workshops to prototype solutions that can effectively improve access to the computer. We selected three individuals with physical disabilities in their upper limbs, along with 25 designers and engineers, to co-design their own solutions using physical prototypes.
Our team utilized the Jacdac platform[2], which is an open-source hardware/software platform that allows everyone to create custom electronic solutions from various hardware devices with standardized PCB-based edge connectors and cables. This platform emerged as a valuable tool to aid the prototyping process and facilitate collaboration among diverse stakeholders. The activities we designed were informed by co-creation workshops focused on popularizing technology and the DIY (do it yourself) culture[3][5], leaving valuable lessons about how designers can integrate stakeholders into their process of brainstorming, prototyping, solution development, and user testing.
Acknowledgments
NYU Shanghai Interactive Media Arts undergraduate students contributed to this project with different roles researching, facilitating workshops, and analyzing data. Microsoft supported this research project through its Artificial Intelligence for Accessibility (AI4A) Grant program. MustardTek and IDEO China provided resources at different stages of the study, and Zhijun Fu from China’s Disabled Persons Federation (Shanghai Pudong local branch) supervised all data collection.
References:
[1] Virginia Braun and Victoria Clarke. 2012. Thematic analysis. In APA Handbook of Research Methods in Psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological., Harris Cooper, Paul M. Camic, Debra L. Long, A. T. Panter, David Rindskopf, and Kenneth J. Sher (Eds.). American Psychological Association, Washington, 57–71. https://doi.org/10.1037/13620-004
[2] James Devine, Michal Moskal, Peli De Halleux, Thomas Ball, Steve Hodges, Gabriele D’Amone, David Gakure, Joe Finney, Lorraine Underwood, Kobi Hartley, Paul Kos, and Matt Oppenheim. 2022. Plug-and-play Physical Computing with Jacdac. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (Sept. 2022), 1–30. https://doi.org/10.1145/3550317
[3] Amy Hurst and Jasmine Tobias. 2011. Empowering individuals with do-it-yourself assistive technology. In
the Proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility. ACM, Dundee Scotland, UK, 11–18. https://doi.org/10.1145/2049536.2049541
[4] Hugo Nicolau and Joaquim Jorge. 2012. Elderly text-entry performance on touchscreens. In Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Boulder Colorado USA, 127–134. https://doi.org/10.1145/2384916.2384939
[5] Dorina Rajanen and Mikko Rajanen. 2019. Co-creation of a Safety Culture in Digital Fabrication. In
Proceedings of the FabLearn Europe 2019 Conference
Fig. 1. Participant P4 using their right hand to operate a smartphone (left) and laptop (right). P4 only has partial control of their right hand, with limited to no finger dexterity, and their left hand can only hold objects. They can use their right-hand index finger for short periods of time but with discomfort, so they mostly use their knuckles to touch the screen Similarly, they use knuckles with a partcially closed, fist-like right hand to operate their keyboard laptop.
Fig. 2. Participant P2 distinctively uses two different groups of muscles for moving a mouse vs. clicking buttons and keys. The left image shows the palm of the hand being used to position the mouse via a whole-arm movement, whereas the right image shows how the index finger movement is used to operate a key. P2 explained how careful they were when clicking because they found small mistakes very time-consuming to correct; a particular challenge was moving an on-screen cursor to the correct place to edit erroneous text.
Fig. 3. Due to spinal cord injuries, both participants P5 and P7 have limited mobility with their fingers in both hands. P5 can only move the position of their right arm with the help of their left hand, as seen in the left picture. P5 primarily uses their left thumb for pressing keyboard keys since other fingers lack sufficient strength to consistently press the keys without exerting significant effort. P7 types with both hands but using mostly the joint of their little finger.
Fig. 4. Participants P1 and P9 reported problems using keyboard shortcuts. P1 (left) has limited mobilty on their left hand, which makes key combinations such as Ctrl + Shift + M difficult. P2 (right) has missing fingers, making pressing Ctrl + C difficult with only one hand. Both participants were aware of the accessibility feature StickyKeys, but they thought was not useful for them.
Fig. 5. Participant P8 can move their right hand fluently, while the left hand has limited mobility. Due to a lack of finger strength, they push their entire hand closer to the keyboard and as a result the other fingers come in contact with the keyboard resulting in accidental key presses. Their solution to utilize the left hand is to fix the position by pressing the palm against the desk and utilizing the tip of the index finger to press the keyboard keys. In the picture, they show a custom accessory that improves their typing speed by providing a rigid accessory to press the desired keyboard key without their other fingers coming into contact with the keyboard.
Fig. 6. Visual results of Phase 3 - Session 1 - Part One: Brainstorming
Fig. 7. P3 along the different sessions of co-design workshops: Paper prototyping (left), using electronics to build a rapid prototype (center), and testing the solution (right)
