INTRODUCTION

People who have intellectual and developmental disabilities (IDD) face many more challenges in their lives each day than those without these disabilities. Students in the Mason LIFE program at George Mason University (Fairfax, VA, USA) [1] face educational as well as employment challenges, [2] because they struggle with learning social skills and situational awareness. Tasks and interactions that are not difficult for people without disabilities could potentially cause those with IDD to derail their entire day and need more assistance than usual. Specifically, when students with IDD are faced with an unfamiliar situation, they may not know how to react, creating stress for them and possibly eliciting a negative reaction. Our long-term goal is to design assistive technology solutions that can be leveraged to help acquaint IDD students with stressful scenarios and thus train them, in a controlled environment, on how to react properly.

This project involves creating a Virtual Reality (VR) game, which places students in a stressful real-life scenario and allows them to practice their decision-making skills. The VR environment creates a safe space in which students can develop their social skills and increase their situational awareness, so they can train themselves to be more comfortable with problems they face in the real world.

The VR environment will be created with a Google Daydream [3] VR headset that the student will wear while being seated in a swivel chair. The student will always be stationary in order to prevent injury. The game itself will be created with Unity and 360° video footage in order to make the game as immersive as possible. Each event in the scenario will have multiple choices that the student can make using the controller provided with the Google Daydream system. Each choice made by the student will influence the next scenario shown. This path will continue until the student experiences an entire module. This method allows the student to either accomplish the task with decisions that would result in a positive outcome, or achieve a non-ideal result, but because they are in a safe environment, there will be no real consequence. The program will also inform the monitoring teacher of the progress the student makes, allowing for the teacher to either assist the student or be able to see a positive growth in the type of decisions made.

The purpose of this program is to create a game that allows students with IDD to practice making real-life difficult choices that helps the students become more confident when faced with a stressful situation in their everyday lives.

DESIGN

The VR game is developed for Google Daydream with the Unity game engine. A Google Chromecast is used to project the student’s in-game view onto an external monitor so staff can watch what the students are seeing. The game features 360° video footage to create an immersive and realistic environment for students. The 360° video footage is analyzed in MATLAB to track moving objects and place invisible “hotspots” for student interaction. The hotspots allow the students to interact with the surrounding 360° video in a natural way. The Daydream game itself is projected to an external monitor through Google Chromecast so a staff worker can view the game that the student is playing. Students will remain seated in a swivel chair for the duration of the game in order to maximize safety while also allowing them to experience the 360° in-game environment. All in-game content is made to be accessible to students with intellectual disabilities, so in-game text is minimized, and any text has a voiceover which reads the text aloud to the student.

A microcontroller (MSP430) is used to develop a help button which students can use to pause the game and alert nearby staff to help them take off the Daydream equipment. The help button contains a gyroscope and an accelerometer. The gyroscope and accelerometer system record each student’s movement to an SD card within the Daydream enabled phone via a Bluetooth module. The gyroscope and accelerometer data taken while students play the game will be analyzed to monitor focus. The button unit itself is wireless and is powered by a USB-rechargeable Lithium Ion Polymer battery. The game itself will be evaluated through focus groups of MasonLIFE students and staff members to get their feedback on the game’s usefulness. The goal is that students playing this game will be able to generalize the social skills learned in the game to real-life situations.

GAME

During the game, the user will be presented with a stressful scenario that could be faced in their everyday lives. These scenarios were decided by one of the main supervisors of the MasonLIFE program who focuses on behavioral studies. She helped the team decide on the scenario as well as breakdown each step of the game, including the options that will be provided and the response or consequence of each option chosen. The game created includes the scenario that the student goes to the local grocery store that they shop at each week and need to find one specific item. However, the item will be out of stock, and the student needs to choose an appropriate way to act.

The student will be able to look around the store and see different types of people to talk to and ask for help, or the option to simply leave the store, which is the worst option out of all of them. Out of the people in the aisle that are available to talk to there will be a grocery store worker that will be the best choice, a friendly customer that will be the second best choice, and a customer that is busy that will be the least ideal choice. Each person in the aisle provides an interaction with the student, and the way they respond depends on whom they are, as well as how the student decides to talk to them.

The student is presented with choices of varying levels of politeness, and the people in game will reciprocate. This allows the student to learn that even if they go up to the right person for help, if they do not treat them respectfully, they may not receive the response they are looking for.

Each possible choice the student can make has a point value of 1 for the poorest choice, 2 for a decent choice and 3 for the best choice. Corrective feedback is given to the student, and, in order to teach students, the student will be forced to try the scenario again until they make the best decision. At the end of each game, the points will be averaged out over the number of choices made and a final score will be given to the supervisor. This allows for the supervisors to see a progression in the types of choices that are made by the students each time the game is used.

TESTING

Three focus groups will be conducted with a 3-5 students per group from the Mason LIFE program selected by their supervisors (total n=12). Students are selected based on ability to give honest and helpful feedback about the system. During the focus groups, the students will play the VR game. Their response times, scores and movement, as measured by the accelerometer and gyroscope, will be recorded to be analyzed. After they finish the game, they will answer questions about the game itself and their understanding of how to use it. These questions will be asked in a Likert Scale format, and the same set of questions will be asked at each focus group in order to be able to see progress made in the usability of the game and make adjustments until it works the best way possible for the MasonLIFE students to use.

We expect that the data we obtain from the gyroscope and accelerometer will show that students are consistently moving throughout the course of gameplay. If students are not moving through the course of gameplay, the game may not be engaging enough. However, if the gyroscope and accelerometer data show that students are moving too quickly and suddenly, it may indicate that the game is too distracting or overwhelming. The response times from students should likely be under three minutes to ensure the student can play the entire game within ten minutes. If response times are well over three minutes, it may indicate the game is too hard or unintuitive. We also expect students are able to play the game without additional instructions. We will investigate the faculty response to the game and student experience through surveys and adapt the game design to address specific requests.