RESNA's Exam Development process is led by psychometric consultants and guided by the input of assistive technology subject matter experts. Decisions regarding exam content distribution and exam passing scores are made by committee and are based on data provided by practitioners in the field. The processes, outlined below, promote an exam that is fair, reliable, valid, and legally defensible.
Job Analysis Study
Development of the current ATP exam was initiated in 2008/2009 with a job analysis study which was facilitated by psychometric consultant, Knapp and Associates. A panel of seven subject matter experts identified the knowledge and skills typically demonstrated by assistive technology professionals with basic competence. After feedback was solicited from a wider group of subject matter experts, the knowledge and skills outline was then validated through a survey of practitioners. The resulting survey data yielded the test blueprint, which specifies how many exam questions are dedicated to each task in the Exam Outline.
Item (Test Question) Writing
RESNA then formed another panel of subject matter experts to write a bank of test questions to represent the content areas outlined on the test blueprint. The test items were then reviewed and revised as necessary.
Cut-Score Study
After the test was assembled with new items in accordance with the blueprint, about 100 candidates piloted the new exam. Knapp and Associates then facilitated a cut-score study to determine the passing score for the exam. A panel of subject matter experts rated the difficulty level of each test item in respect to basic competency, and then the panelists' aggregated ratings data were analyzed to identify the range of statistically acceptable cut-off scores. After reviewing pilot testers' exam scores in relation to the identified cut-score range, RESNA's Professional Standards Board approved a passing cut-off score.
Later, as new questions were developed and pretested, psychometric consultant Prometric constructed new exam forms that were statistically equated to be equivalent in difficulty to the exam analyzed in the Cut-Score study.