TEST ITEM CONSTRUCTION AND VALIDATION: DEVELOPING A STATEWIDE ASSESSMENT FOR AGRICULTURAL SCIENCE EDUCATION

Abstract

In 2001, the New York State Board of Regents approved the New York State Regents Career and Technical Education Policy. Through the process of program approval, career and technical education (CTE) programs can provide students greater flexibility in attaining graduation credits in the areas of math, science, English and/or social studies for students completing such programs. The policy also states that individual career and technical completers can receive a technical endorsement on their Regents diploma. Part of the process for CTE program approval is to administer a technical assessment certifying that students meet current industry standards. The impetus for this study was addressing the need of one secondary career and technical education program, agricultural science education, which does not have a statewide exam. Currently, the Department of Education at Cornell University in collaboration with New York Agricultural Education Outreach is in the process of developing a statewide exam for use as the technical assessment to certify students. Once approved, this exam will meet the technical assessment requirement for program approval. This study focuses on the written multiple-choice portion of the statewide exam, specifically two aspects of developing an exam, item construction and item validation. Based on criterion-referenced test construction procedures two of the nine sections of the exam were developed, animal systems and plant systems. The results of this study outline a process for developing and validating items. They highlight some of the benefits and disadvantages faced when developing test items for a diverse audience without the aid of a testing institute. Further consideration is given to procedures used to validate test items, specifically expert judgment and analytical data. The results from this study provide guidance to test developers related to aligning items to content, writing and editing items, and revising items.