Student assessment has always played an important role in vocational education, and recent changes in assessment practices may hold great promise for vocational educators. A RAND study was conducted to examine alternative forms of assessment from the perspective of vocational educators. Two products resulted. The first is this report, which describes a variety of assessment alternatives, reviews examples from extended case studies, and discusses criteria for choosing among the alternatives. This report should be of interest to vocational educators at the state and local level, particularly those responsible for decisions about the form and use of assessment systems. The second product is a set of training materials to help vocational educators make effective decisions about assessments. These materials are to be offered as a supplement to Getting to Work: A Guide for Better Schools (Rahn et al., 1995), a recent National Center for Research in Vocational Education (NCRVE) training package.

The purpose of this study was to evaluate the utility of nontraditional forms of assessment for vocational education. This issue is particularly important at the present time because both vocational education and educational assessment are undergoing significant changes. In education, enrollment in high school vocational courses is dropping, the nature of vocational students is changing, and employers are calling for applicants who possess skills different from those of the past. In assessment, new forms of constructed-response measures, including performance tasks, portfolios, and senior projects, are gaining popularity, and assessment is being used more prominently as a policy tool. The confluence of these factors makes this an opportune time to take a careful look at the potential value of alternative forms of assessment for vocational education.

For our study, which ran from 1994 to 1996, we selected six cases that reflected a wide range of assessment options, with particular emphasis on operational programs using constructed-response measures. In each case, we conducted a critical review of the assessment system based on descriptive materials provided by the program, research literature, telephone interviews, and, in all but two cases, a one- to two-day site visit. Our investigations focused on program definition, implementation, and administration; the quality and feasibility of the assessments; and the potential usefulness of the assessment approach for vocational educators. Most, but not all, of our cases were drawn from vocational education. They include the Career-Technical Assessment Program, the Kentucky Instructional Results Information System, the Laborers-Associated General Con-tractors environmental training and certification programs, the National Board for Professional Teaching Standards certification program, the Oklahoma Department of Vocational-Technical Education competency-based testing program, and the Vocational/Industrial Clubs of America national competition.

By comparing these assessment systems, we were able to identify a number of important design and implementation considerations that should inform decisions about the use of alternative assessments in vocational education (and, more broadly, assessment planning in any educational context). First, the user must clarify the purpose of the assessment because the intended use (e.g., improving learning and instruction, certifying mastery) affects the choice of assessment method. Second, the user must understand the nature of the skills being assessed. Different cognitive demands (e.g., factual knowledge, integrated problem solving) may be better measured with one type of assessment than with another. Third, the user should become familiar with the range of alternative assessment options available. We identify four categories of assessments that may be of interest to vocational educators: written assessments, portfolios, performance tasks, and senior projects. Fourth, the user should understand the advantages and disadvantages of each assessment strategy. We describe several dimensions along which alternative assessments may differ, the two most important being quality (which includes reliability, validity, and fairness) and feasibility (which includes cost, time, complexity, and credibility). We discuss the quality and feasibility of the assessment types identified. Fifth, the user should be aware of certain other issues involved in designing assessment systems that might be relevant to a particular program or setting. We review the relative advantages of single versus multiple measures, low versus high stakes, standardization versus adaptability, embedded versus stand-alone assessments, single versus multiple purposes, and voluntary versus mandatory participation.

Finally, we illustrate how vocational educators can use this review to think about the usefulness of alternative assessments for a particular situation. We describe a procedure for evaluating the utility of alternative assessments that includes analyzing the purposes of the assessment and the skills to be assessed, considering alternative types of assessment, reviewing their quality and feasibility, and considering other factors identified in the case studies. This approach is illustrated by two scenarios that describe assessment problems faced by vocational educators and show how the results of this study can contribute to their resolution.

Stecher, B. M., Rahn, M. L., Ruby, A., Alt, M. N., Robyn, A., & Ward, B. (1997, July). Using alternative assessments in vocational education. Berkeley, CA: National Center for Research in Vocational Education.

Download the report (PDF)