Lego Robot Control via a Speech Generating Communication Device for Play and Educational Activities
Kim D. Adamsa,b , Julie Yanthab , Albert M Cookb
a Glenrose Rehabilitation Hospital and b Faculty of Rehabilitation Medicine, University of Alberta
Edmonton, Alberta
ABSTRACT
Children have utilized switch adapted infrared controllers to manipulate play objects with inexpensive Lego™ robots in order to learn new and demonstrate existing skills. Since some speech generating devices (SGDs) have infrared output, the opportunity arises for children to have an integrated communicative and robotic control system using their own SGD and access method. The purpose of this case study was to evaluate the feasibility of a twelve year old girl with cerebral palsy using an integrated communicative and robot control system for educational activities. She performed various activities, including numeracy and social studies goals. The integrated system offered the participant a means for manipulative, cognitive, and communicative skill demonstration and development. The participant accessed educational materials and engaged in active learning of the curriculum content. Her teachers were pleased with the intervention, reporting that the participant demonstrated her abilities and connected with the curriculum and other students.
KEYWORDS
Assistive robot, Lego, AAC, play, education, integrated system
BACKGROUND
Typically developing children learn physical, cognitive, linguistic and social skills through play. Children who have physical or communicative limitations may have fewer opportunities for play, resulting in less opportunity for skill development [1]. Assistive robots have been used by children with disabilities to manipulate objects to learn new and demonstrate existing skills [2-5]. Recent robot studies have utilized a switch adapted infrared (IR) controller to allow play activities using inexpensive LegoTM robots[6]. Since children can control Lego robots using the built-in infrared (IR) output on some speech generating devices (SGDs) they can have an integrated system which they control with their own communication device and access method. This way, children do not have to turn away from play to communicate or vice versa [7]. Assistive robots have been used to give children access to educational activities [8-10] and an integrated communication and robotic control system could facilitate more active participation in the curriculum.
PURPOSE
The purpose of this study was to evaluate the feasibility of a child with a disability to use their SGD to control a Lego robot for educational activities. Operational goals for the SGD and robot and educational goals were evaluated in one-on-one sessions. Goals for robot programming and interactions with classmates were evaluated in an integrated classroom.
Our hypothesis was that after achieving operational competence in robot control via an SGD, the participant will achieve increased attainment of educational goals. Numeracy and social studies goals are presented here.
METHOD
Study Design and Participant
This was a case study with a 12 year old girl who has cerebral palsy with severe physical and communication limitations. She uses a SGD with which she had approximately seven months experience and accesses the device with two switch step row-column scanning, with one switch on each side of her head.
Goal Attainment Scaling (GAS) has been recommended for measuring assistive technology intervention effectiveness [12, 13]. GAS is a criterion referenced, objective measure that allows the identification of multiple, individualized goals for each participant [11]. Numeracy and writing goals were targeted for the participant since she had very limited independent experience in these areas. GAS scaling was developed by the investigators, the special education teacher and members of a local assistive technology team. The he participant approved the final goals and scaling. The numeracy goal score was scaled according to the numbers the participant demonstrated understanding of (See Table 1). An initial writing goal was set for the participant to use her SGD to generate journal entries about using the robot. However, upon learning that the participant’s social studies class was studying Ancient Greece, the team decided the participant could create a movie of a Greek myth as her project for the class. Instead of writing about the robot, she would generate the narration of the myth by writing it in her own words. Writing was to be done during social studies class with the educational assistant (EA). The EA was provided with some sample writing supports (e.g., sentence frames). Acting out and saying the script was to be done during the robot training sessions with the investigators. The social studies goal score was scaled according to the level of writing support required (See Table 1).
| GAS Scale | Number range |
Level of writing support |
|---|---|---|
-2 (initial status) |
11 through 15 |
copying samples |
-1 (somewhat below expected) |
16 through 20 |
using sentence frames |
0 (expected status) |
21 through 25 |
reviewing sentence frames prior to activity and receiving verbal cues to help generate them |
+1 (somewhat above expected) |
26 through 30 |
reviewing sentence frames prior to activity, but independently generating them |
+2 (much above expected) |
greater than 30 |
independently generating sentences |
Materials
Figure 1: Robot arm (left) and car-like roverbot (right) (Click for larger view) Two robots were built from the Lego MindstormsTM kit: a car-like roverbot and a robot arm (Figure 1). The robots were controlled by IR with direct commands to individual motors or by a program of a sequence of movements (circles and long lines in this study).
The SGD used was the VanguardTM II, with UnityTM 45 Full vocabulary set, Version 4.06 Nov 29, 2006. Two Jelly BeanTM switches were attached with VelcroTM hook and loop fasteners to the wheelchair headrest. The Language Activity MonitorTM was used to log language and IR commands used during the sessions. A new SGD page was created for the robot commands and a Lego remote control unit was used to train the IR signals into the SGD. The participant had the ability to press and hold her switches, and she often used this strategy to control the robots. The SGD’s auto repeat rate was set to 0.7s.
Investigators prompted the participant to use vocabulary already present on the device, as well as programmed some additional vocabulary on the robot page (e.g. “It’s not working”, “This is fun”, “This is boring”).
Procedure
The participant received training and used the robot for functional activities for 12 sessions over 14 weeks. The robotic intervention progressed with the participant using an increasing number of robot functions to complete more complex tasks. The activities that the participant performed are shown in Table 2.
| Draw flowers using circles and lines |
| Follow straight pathway as game with 2 word sentences about numbers |
| Follow square pathway |
| Follow long zig zag pathway as game, roll dice with robot arm via switch |
| Follow short zig zag pathway as game, roll dice with robot arm via switch Follow a maze Connect dot-to-dot drawing of a bat with numbers 1-12 |
| Follow loop pathway as game, roll dice with robot arm via SGD |
| Orient and place puzzle pieces, square shape |
| Orient and place puzzle pieces, rectangular shape |
| Connect dot-to-dot drawing of a spider web with numbers 1-12 (twice) |
| Narrate and act the parts in a Greek myth, Part 1 |
| Narrate and act the parts in a Greek myth, Part 2 |
RESULTS
Numeracy
The participant played board games (i.e. enlarged pathways of various shapes) by using her device to answer questions or to roll the dice with the robot arm to determine how far she could move her “marker”, i.e. the roverbot. Occasionally her classmates observed and gave encouragement. The participant generally seemed to know how many moves to make, understood turn taking, and teased the investigators by trying to take extra turns.
Enlarged dot-to-dot drawings were completed using a pen attached to the roverbot. The participant was able to connect the numbered dots (one to 12) with prompting (the investigators crossed out the numbers on a sheet of paper as she found them). Another method for prompting involved placing the number of blocks next to each numbered dot. This way, the participant maneuvered from smaller to bigger piles of blocks.
The puzzle task engaged orientation and geometry skills. The participant verbally chose a puzzle piece that was placed on top of the roverbot. The participant was instructed to take it to the correct location, and spin the piece to the correct orientation Each puzzle piece had either a line of beetles or worms (she loved bugs) going straight, or turning a corner. She was instructed to put the pieces in the shape of a square, but required a square guide in order to begin. She became quite efficient with this task and placed the last puzzle piece exactly in the right position and orientation.
Although these activities gave the participant experience using the robot for math tasks, the study ended before the tasks could be adapted to give her experience with numbers in her goal range. Hence, as written, the final GAS score for numeracy would be -2.
Social Studies
The team suggested using the myth “Theseus and the Labyrinth” because it leant itself well to robot action such as maneuvering the roverbot through a maze. The participant did not write the narration of the myth on her own, due to lack of time and low interest as reported by the EA, hence the GAS goal could not be scored as originally scaled (according to level of support). However, the story was uploaded into the notepad on the participant’s SGD, so she could narrate by stepping through the story sentence by sentence and she still acted out and said the script. She moved the roverbot (Theseus) and the robot arm (Minotaur) through their positions and also generated novel utterances for the part of Theseus. The investigators moved the other characters and props. The participant demonstrated innovation by creating a new ending to the story.
The movie was created by splicing the narration and acting into the proper sequence and omitting portions of time when the participant was scanning to the robot commands or vocabulary. It took two 90 minute sessions to perform the narration and acting, and the resulting movie was five minutes long.
Teacher and Classmate Comments
When the participant showed the Greek myth movie to her class, one classmate said “I wish I did that with my robot”. The teacher was pleased with the participant’s progress and said “it was demonstrated to everyone who works with her that she has so much potential despite her disabilities” and “computers and robotics are very motivating and interesting to her peers so it is a way of actively connecting her with the curriculum and the other students in a unique and fun way.”
DISCUSSION
The integrated communicative and robotic control system appears to be feasible for learning and demonstrating manipulative, cognitive, and communicative skills in educational activities. The participant was able to functionally manipulate the items used in the numeracy activities (her marker for the board game, the felt pen for the drawings, and the puzzle pieces) and she was very accurate at stopping on the place markers (x’s drawn on a table) for Thesesus in the myth. Too many goals were set for the participant in the overall study so the numeracy activities did not receive much attention. However, the participant was able to demonstrate her cognitive understanding of the numbers one through 12 and she got three dimensional experience with numbers and geometry. Having the participant act out the Greek myth provided a salient learning experience and likely helped her make some meaningful connections with the material. Whereas the participant previously lost interest after a few minutes in writing the myth with the EA, she was very attentive for two 90 minute sessions while creating the movie. In general, the participant is someone who uses much non-verbal communication but uses her SGD only when prompted. The integrated system provided the opportunity to communicate during the activities, particularly in generating novel utterances for Theseus in the myth. Controlling the robot for the numeracy activities was cognitively demanding, so there was little unprompted communication, but she did spontaneously tease the investigators that their spider looked like a sun. Social skills were demonstrated by independently ensuring that Theseus was face-to-face with other characters when he was about to speak to them. This point is particularly interesting because she relies on others to propel her manual wheelchair, thus she has never had the opportunity to orient herself face-to-face for a conversation. She demonstrated her sense of humor when she drew circles around all of the numbers on the spider’s web dot-to-dot puzzle after an investigator joked that an accidental circle looked like a flattened bug.
In addition to the innovative new ending to the myth, there were many instances where the participant demonstrated other skills during these activities. For instance, she learned how robot programs work by exploration and discovery when she carefully observed the results of pressing the stop button during program execution, and she independently learned to press and hold her switch for repeated direct control movements of the robot. The participant exerted her independence by expressing a preference of using the robot arm to roll the dice instead of using an external switch or the random number generating function on her SGD. Also, she did not want the narrator’s voice to be different from the one she usually uses on the SGD.
Although the participant’s GAS scores did not increase, effectiveness of the integrated communication and robotic control system intervention was demonstrated in other ways. It was difficult to write appropriate GAS scaling in this case, but valuable information was obtained regarding intervention protocol and goal setting for future studies.
REFERENCES
- Musselwhite, C.R., Adaptive Play for Special Needs Children. 1986, San Diego, CA: College-Hill Press.
- Cook, A.M., et al., School-Based Use of a Robotic Arm System by Children With Disabilities. IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 2005. 13(4).
- Cook, A.M., et al., Development of a robotic device for facilitating learning by children who have severe disabilities. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2002. 10(3): p. 178-187.
- Kronreif, G., et al. PlayROB – Robot-Assisted Playing for Children with Severe Physical Disabilities. in Proceedings of the 2005 IEEE 9th International Conference on Rehabilitation Robotics. 2005. Chicago, IL, USA.
- Topping, M., An Overview of the Development of Handy 1, a Rehabilitation Robot to Assist the Severely Disabled. Journal of Intelligent and Robotic Systems, 2002. 34: p. 253-263.
- Cook, A.M., et al. Using Lego Robots to Estimate Cognitive Ability in Children who have Seere Disabilities. in RERC-ACT State of the Science Conference. 2007. Broomfield, Colorado.
- Light, J.C. and K.D.R. Drager, Improving the Design of Augmentative and Alternative Technologies for Young Children. Assistive Technology, 2002. 14: p. 17-32.
- Eberhart, S.P., J. Osborne, and T. Rahman, Classroom Evaluation of theArlyn Arm Robotic Workstation. Assistive Technology, 2000. 12: p. 132-143.
- Harwin, W., A. Ginige, and R. Jackson, A Robot Workstation for Use in Education of the Physically Handicapped. Ieee Transactions on Biomedical Engineering, 1988. 35(2): p. 127-131.
- Howell, R. and K. Hay, Software-Based Access and Control of Robotic Manipulators for Severely Physically Disabled Students. Journal of Atrificial Intelligence in Education, 1989. 1(1): p. 53-72.
- Kiresuk, T.J., A. Smith, and J.E. Cardillo, Goal attainment scaling: Applications, theory and measurement, ed. T.J. Kiresuk, A. Smith, and J.E. Cardillo. 1994, Hillsdale, NJ: Erlbaum.
- Schlosser, R.W., Goal attainment scaling as a clinical measurement technique in communication disorders: a critical review. Journal of Communication Disorders, 2004. 37: p. 217–239.
- Ottenbacher, K.J. and A. Cusick, Goal attainment scaling as a method of clinical service evaluation. American Journal of Occupational Therapy, 1990. 44(6): p. 519-25.
Lego is a trademark of the Lego Group, http://www.lego.com/
Mindstorms is a trademark of the Lego Group, http://www.lego.com/
Vantage is a trademark of Prentke Romich Company, http://www.prentrom.com/
Unity is a trademark of Semantic Compaction Systems, http://www.minspeak.com/
Jelly Bean is a trademark of Ablenet, Inc., http://www.ablenetinc.com/
VELCRO is a registered trademark of Velcro Industries B.V, http://www.velcro.com/
Language Activity Monitor (LAM) is a trademark of Prentke Romich Company, http://www.prentrom.com/
Corresponding Author:
Kim Adams
3-48 Corbett Hall
Faculty of Rehabilitation Medicine
University of Alberta
Edmonton AB T6G 2G4
(780) 492-0309 voice
kdadams@ualberta.ca