RESNA 28th Annual Conference - Atlanta, Georgia
Sally Fennema-Jansen, Roger O. Smith, and Dave L. Edyburn, University of Wisconsin-Milwaukee, Milwaukee, WI; Mary Binion, Ohio Center for Autism and Low Incidence
The Student Performance Profile (SPP) is an instrument developed by the Assistive Technology Infusion Project (ATIP) of the Ohio Department of Education in collaboration with the UW-Milwaukee ATOMS Project (Assistive Technology Outcomes Measurement System) team. The SPP requires respondents to rate the contribution to progress of ten different interventions, including assistive technology (AT) devices and AT services. In addition, the respondents rated the current ability on the student's Individualized Education Plan (IEP) goals. The rating on the goals and the rating on the contribution of each intervention were used to calculate a score representing the relative contribution of AT devices to student progress. A comparison was made between the scores representing the relative contribution of AT devices to student progress prior to receiving the new AT and after using the AT for 8 months. The AT device contribution to progress increased significantly.
Assistive technology, outcome, school, measurement
Although the need for AT devices and services must be considered for each student receiving special education, research on the outcomes of AT in the schools is limited (1). One challenge to measuring AT in the schools is that students receive a variety of interventions concurrently. The need for research to isolate the impact of various forms of technology is emphasized by Jutai, Rigby, Ryan and Stickel (2), who had a particular concern regarding the relative psychosocial impact of device use. The approach used in this study has its roots in the work of Christiansen and Smith who identified six general approaches to improving the performance of individuals with disabilities: 1) remediate the impairment; 2) teach the individual to compensate for the impairment; 3) provide assistive technology; 4) redesign the activity; 5) redesign the environment; 6) use a personal assistant (3, 4, 5). Later, Smith recognized two additional pre-intervention approaches: health promotion and universal design (1, 6). By adapting and expanding the intervention approaches to address the specific needs of the educational setting, the framework for the Student Performance Profile was developed.
The Student Performance Profile was developed as a part of the Ohio ATIP, which provided AT devices to students throughout Ohio using a grant application process to award funding. Using the web-based Student Performance Profile – Pre (SPP-Pre), the project collected data from the team contact person before the student began using the new technology. Eight months following the introduction of the assistive technology, the contact person provided information on AT outcomes by completing the Student Performance Profile – Post (SPP-Post).
One section of the Student Performance Profile attempts to isolate the contribution of AT by requiring respondents to rate the amount of contribution to progress of ten different intervention approaches: 1) natural development; 2) compensation for the impairment; 3) adaptation of tasks; 4) redesign of instructional environment; 5) performance expectations changed; 6) general education; 7) related and support services; 8) personal assistance; 9) AT devices; and 10) AT services. This study addresses the need to measure outcomes in the school environment, including the formidable challenge of isolating the outcomes attributable to assistive technology, by analyzing data collected by the Ohio ATIP.
Did the assistive technology contribute to student progress? Did progress attributable to assistive technology vary by disability, area of need, or by the amount of time the AT was used?
The contact person was responsible for completing the SPP online data collection forms. Outcome data had been reported on 1760 students at the time of the cutoff date for inclusion in this study. Students with a variety of disabilities from across the state of Ohio are represented in this group. The Student Performance Profile –Pre and the Student Performance Profile – Post required the respondent to rate the current ability on IEP goals on a scale of 0 – 100. In addition, they rated the amount that the ten different interventions contributed to the student’s progress on a scale of 0 - 10. The percent of the total for all interventions that was attributed to assistive technology devices was multiplied by the ability rating on the relevant IEP goals. A repeated measures analysis of variance was conducted to compare these scores prior to and following the implementation of the new AT. Two-way within-subjects analyses of variance were also conducted to examine differences by disability, by the area of need for which the assistive technology was used, and by the amount of time that the AT was used. Each of the analyses was completed three times, because each student could select up to three areas of need that would be addressed by the technology. These groups are referred to as Group Need 1, Group Need 2, and Group Need 3.
For each of the groups, the results of the analysis of variance indicated a significant difference between the amount that AT devices contributed to progress on the goals prior to the assistive technology intervention and after using the technology for 8 months. Using Wilks’ Lambda (Λ) as the criterion, for Group Need 1, the main effect (the pre to post condition) was significant, Λ =.570 F(1, 1003)=755.530, p<.001, partial ή 2=.430; for Group Need 2, Λ =.747 F(1, 759)=257.480, p<.001, partial ή 2=.253; and for the third Group, Need 3, Λ=.868, F(1, 620)=94.429, p<.001, partial ή 2=.132.
For all three groups, disability was a significant main effect, indicating that the percent of AT contribution to ability on goals varies by disability. This analysis was conducted three times, once for each area of need that was selected for the student. For the first area of need, the between-subjects factor, disability, was significant, F(8, 1003)=4.584, p<.001, partial ή 2=.035; for Group Need 2, F(6, 759)=4.079, p<.001, partial ή 2=.031; and for Group Need 3, F(6, 620)=5.790, p<.001, partial ή 2=.053.
For two of the three groups, the area of need main effect was significant, indicating that the percent of AT contribution to ability on goals varies by the area of need addressed by the AT. For the first area of need, the between-subjects factor, area of need, was significant, F(6, 1094)=2.636, p=.015, partial ή 2=.014. Area of need was also significant for Group Need 3, F(6, 699)=3.304, p=.003, partial ή 2=.028. Results for Group Need 2 were not significant, F(6, 856)=.803, p=.567.
The amount of time that the technology was used was rated by answering a series of questions that addressed the duration and frequency of use of the devices. The values for these responses were multiplied to determine the amount of time per month that the technology was used. The minutes per month were then divided into 6 equal groups and two-way analysis of variance was conducted to compare pre to post AT device contribution to goals across the 6 levels of time that the AT was used. For all of the groups, the main effect, the amount of time the AT was used was significant, indicating that the percent of AT contribution to ability on goals varies by amount of time the technology is used. For the first area of need, F(5, 671)=16.560, p<.001, partial ή 2=.110; for the second group, F(5, 552)=11.789, p<.001, partial ή 2=.096; and for Group Need 3, F(5, 438)=9.264, p<.001, partial ή 2=.096.
The section of the SPP that required respondents to rate the relative contribution of a variety of interventions identified significant differences in the contribution of assistive technology devices prior to and following the use of new AT. Results of a repeated measures analysis of variance revealed that the AT contribution to progress on IEP goals increased from pre to post intervention for all disabilities. The relative contribution of AT to students’ progress on IEP goals increased as the AT was used more. This study only addressed the relative contribution AT devices, and therefore, is considered an underestimate of the total contribution of AT, which would include both AT devices and AT services. The results of this study provide confirmation that the method used to determine the relative contribution of a given intervention has potential, and should be researched more extensively. This approach could be used to study the relative contribution of a variety of interventions. A limitation of this study is that the data was collected as a part of a large grant. Because of this, the respondents would likely be biased to respond in a positive manner when asked about the impact of the assistive technology. Future research is needed to compare the relative contribution of each of the ten interventions that were rated by the respondents. In addition, comparing the results on this measure to other measures of outcome would help to validate the results. Having more than one team member complete the SPP would provide insights into the reliability and validity of the instrument as well. Using the same instrument when grant funding is not an issue would remove some of the bias toward positive responses regarding the impact of AT.
This work is supported in part by the National Institute on Disability and Rehabilitation Research, grant number H133A010403. The opinions contained in this paper are those of the grantee and do not necessarily reflect those of the NIDRR and U.S. Department of Education.
Sally Fennema-Jansen
Research Consultant
University of Wisconsin – Milwaukee
P.O. Box 413
Milwaukee, Wisconsin, 53201
phone: 414-229-5100
email: sfennema@uwm.edu