Project Star Catcher: Translating Constraint Induced Movement Therapy into Immersive Virtual Reality
Updated: Aug 13, 2020
Immersive virtual reality gaming has the potential to motivate individuals post-stroke to perform intensive repetitive task-based therapy, and it can be combined with motion capture as a way to track therapy compliance and progress.
This project explores the design and preliminary evaluation of an immersive virtual reality game, titled “Project Star Catcher,” for those with weakness on one side of their upper bodies. The game mechanic is adapted from constraint-induced therapy, an established therapy method where users are asked to use the weaker arm by physically binding the stronger arm. Our adaptation innovates from physical to psychological binding by providing a dynamic reward system that promotes the use of the weaker arm. Players are rewarded by scoring points when performing a rehabilitative motion to catch falling stars in an immersive, cosmic virtual reality.
Project Star Catcher was developed for HTC Vive, and preliminary evaluation was performed by users with developmental and physical disabilities. The results suggest that users are generally compliant with the game rules of using the weaker arms, but qualitative observations revealed that there are noticeable variations in user strategies for playing Project Star Catcher. The results of this project include a set of design considerations for rehabilitation games for those with developmental and physical disabilities.
Research Questions:
Can games through immersive Virtual Reality (iVR) be used to successfully improve the accuracy, affordability, and accessibility of physical therapy?
Has modern head-mounted mounted display systems like HTC Vive and Oculus Rift surpassed systems like the Cave Automated Virtual Environment for exercise games?
“When you have a stroke you want to escape into another reality, and this helps you do that. I think that’s a good thing, because if you are not mobile in reality, you can escape into a world where you are mobile, and that would feel like a positive thing”
- Stroke Survivor and User of Project Star Catcher, 2017, Cabrillo College Stroke and Disability Learning Center
Specific Media Making Goals:
The goal of this project was to design and evaluate an immersive virtual media environment designed to translate the physical therapy theory behind Constraint-Induced Movement Therapy [Marc and Taub 2008] from physical constraint into psychological constraint with runtime feedback capabilities. This project evaluated VR for rehabilitation, specifically through the use of the HTC Vive for upper limb rehabilitation of users with hemiparesis. To inform this study, we playtested several immersive systems, including Microsoft Hololens, Microsoft Kinect, Oculus Rift, Playstation VR, and HTC Vive. We concluded that HTC Vive provides the most high-resolution data capture regarding the head and upper limb movements as well as the most accurate 6-DOF motion capture capabilities at the time of the study [Staff 2017]. To inform of media domain design, four questions were focused on:
Are participants with hemiparesis able to use the controllers during gameplay?
Can the game mechanics result in improved adherence and compliance than the adherence of conventional mCIT as reported in the papers?
Is the game understandable and enjoyable by the target population?
What does the data inform about user gameplay strategies?
The intent of mCIT is to constrain the stronger limb to encourage the use of a weaker limb. Therapists concluded that the benefits outweigh the risk of using a psychological constraint in VR instead of a physical constraint. While they did not expect a VR game to cause injury to patients, because gameplay can be done seated and physical interaction with the game is limited to a lightweight headset and controllers, the therapists said there is risk that the VR rehabilitation game potentially will not be effective in encouraging the use of the weaker limb and may not be enjoyed by patients, especially if they have never experienced VR before. Physical constraint, therapists commented, could feel demeaning and makes some patients feel like they lose independence, so they may resist treatment, especially in the beginning, before they start to feel the benefits of mCIT. Patients must go to a clinic and are dependent on being strapped into their restraint by a therapist, and the restraint itself could feel claustrophobic or unconformable. The therapists believed that psychological constraints through game rules and an immersive experience would not feel like losing independence; instead, the patients maintain dignity and feel as if they chose to play the game. Patients can even measure their own progress by tracking game scores and session durations. Thus we expected that the use of psychological constraint for a gamified mCIT protocol would be equivalent or more successful than using physical constraints.
To understand the media domain over the course of one month, we ran five semi-structured focus group sessions with Hope Services (four sessions at UCSC) and Cabrillo College Stroke and Disability Learning Centers (one session at Cabrillo). This involved the various users as a group trying out every minigame Valve The Lab game for HTC Vive. The Lab includes a series of eight minigames ranging from first and third-person control games centered around slingshots, longbows, invader style spaceship games, 360 camera tours, human medical scans, solar system models, robot repairs, and fantasy shop viewings. Most users tended to resonate the most with fantasy and non-worldly experiences that were only available through VR, such as visiting outer space and a new fantasy world. From such, we went through the process of choosing the game domain that appeals to most patients. We defined the following characteristics for our game:
The game domain was chosen about stars and galaxies.
The game provides immediate feedback about users' task performances through data captured using motion tracking sensors, which allows patients and therapists to see the performance in real-time.
The game encourages the use of hemiparesis-affected limbs through adopting the principles of mCIT into motivating stimuli through a mixture of score, haptic, audio, and visual feedback.
From such, Project Star Catcher was born. Project Star Catcher incentivizes game participants to use their hemiparesis-affected arm through catching falling stars in a cosmic VR environment while collecting real-time user data. Different audio/visual/haptic stimuli were iteratively tested through the Unity Game Engine and HTC Vive VR system through a pilot study and playtesting sessions with Hope Services as further discussed in Elor et al. 2018, TACCESS. Design improvements were made based on the pilot studies to increase the difference of stimuli for star color, speed, and catches, which showed improvement in compliance. A modular runtime logfile system was developed in Unity using C# to collect player behavior and motion capture by utilizing Microsoft .NET I/O Framework to produce CSV data at 90Hz refresh rate and sampling unity in-game data. An additional parsing and visualizing pipeline implemented through Mathworks Matlab 2017 by utilizing the statistical and visualization toolboxes. The resulting system was enacted as a task-based behavioral playground that could be dynamically adapted to each user's state in physical rehabilitation by a research evaluator to explore compliance, adherence, and player strategy using VR based modified Constraint-Induced Movement therapy.
Through sessions with a mixture of post-stroke adults and adults with a developmental disability, a compliance rate increase of 40% was determined. These findings can be found in Elor et al. 2018, SMARTCOMP with design guidelines and iterative testing methodology in Elor et al. 2018, ACCESS. Because of this system's modular nature, we are performing additional studies with Project Star Catcher to compare the efficacy of immersive mediums between Cave Automated Virtual Environments vs. Head-Mounted Displays for both impaired and non-impaired users. Additionally, we are performing biofeedback studies with a brainwave, heart rate, and galvanic skin response testing to design machine learning intent models to infer gameplay with Project Star Catcher. This modular system enacts a behavioral playground that is flexible from studying VR for Therapy, physical task-based analysis, and runtime adaptive stimuli.
Related Media and Demos Videos:
Virtual Reality for Physical Therapy Goals
Games for Health & Rehabilitation: Project Star Catcher CITRIS Demo
Cave Automated Virtual Environment: Project Star Catcher Demo
Contributions and Collaberators:
Independent Contributions
Lead and independent developer, prototype designer, and user testing evaluator for project star catcher.
Lead data analyst in game-play behavior and runtime motion capture data from unity.
Contributions of Co-authors
Professor Sri Kurniawan: Advisor, Mentor, and guidance of HCI related protocols and testing.
Professor Mircea Teodorescu: Advisor, Mentor, and guidance of system engineering related architecture and framework.
Related Publications:
Related Media: