Building Academic Skills in Context: Testing the Value of Enhanced Math Learning in CTE Pilot Study
This report describes the conduct and outcomes of an experimental pilot study conducted in Spring 2004 to develop and test a model that aimed to enhance career and technical education (CTE) instruction with the mathematics that is already embedded in the curricula of six occupational areas.
Math is abundant in the CTE curriculum, but it is largely implicit to both teachers and students. The impetus for the study is that many high school students, particularly those in enrolled in CTE courses, do not have the math skills necessary for today’s jobs or college entrance requirements. This research project was aimed at using an authentic context for teaching math skills. Preparation for the study began in the summer of 2003 with the nationwide recruitment of teacher-participants. CTE teachers who were interested in participating were required to identify teachers of mathematics who was willing to work with them during the course of the study. From a total of 274 CTE teachers who applied to participate, 114 were randomly assigned to and participated in the experimental group, whereas the other 122 served as controls.
In the fall of 2003, the experimental CTE teachers and their math-teacher partners attended a professional development workshop for their occupational area. At these workshops, the CTE–math teacher teams identified the mathematical concepts in the curricula of the CTE teachers and developed lessons to provide explicit instruction in these concepts. The lessons were required to incorporate the following elements:
- “pull out” the mathematics found in the CTE context
- assess students’ math understanding
- work through the pulled-out example
- identify the underlying math concept, using math vocabulary
- work through similar examples and generic examples
- check for understanding
- have students create other examples, both from CTE and from traditional math
In the second half of the 2003–2004 school year, the experimental teachers taught the lessons developed by the group of teacher teams in the fall workshop for their occupational area. Pretesting with one standardized mathematics test (TerraNova) was conducted prior to the first lesson, and posttesting was conducted after all had been taught. Three different types of math posttests were administered: another form of TerraNova, ACCUPLACER, and WorkKeys®.
Classes were randomly divided so only one third of each class took each one of these tests. In addition, students in each of the six occupational areas took a posttest that assessed their knowledge and skills in that area. These tests were administered to determine whether or not the instruction time used for enhancing math was detrimental to the learning of the CTE content.
Both quantitative and qualitative data were collected and analyzed to assess fidelity of the treatment and to gain understanding about the teacher experiences during implementation of the math-enhancement model. Teacher surveys, interviews, and focus groups were conducted. Math teachers were asked to meet with the CTE teachers after the lessons had been taught and submit debriefing reports. Additionally, each teacher was observed once during the semester by a member of the research team, and instructional artifacts were collected from each classroom.
Random assignment was made at the classroom, and not the individual student, level; the unit of analysis was the classroom. Despite random assignment, the pretest yielded significant differences in the average math scores of the experimental and control classrooms in two of the six occupational areas, but not overall. Because of these differences, the pretest was used as a covariate in analyzing the posttest scores. These analyses showed a significant difference (p < .10) in favor of the experimental group on the ACCUPLACER test (mean effect size = .20).
Analyses of the six separate sites on the three math posttests found that 14 of the 18 differences favored the experimental group. The nonparametric sign test indicates that such a pattern has a probability of less than .04. Examining sites separately, two of the six sites had significant differences in favor of the experimental group on ACCUPLACER (Site A, effect size = .32; and Site C, effect size = .46). Site C also had a significant difference on WorkKeys (effect size .40).
These improvements in math achievement did not come at the cost of lower scores on the tests of occupational skills and knowledge. At the classroom level, there were no significant differences between the experimental and control groups on these tests.
A review of the formative evaluation data assembled as part of the pilot study led to a number of changes in preparation for the full-year study (2004–2005 school year). These included revisions in the seven-element pedagogic model and in the amount and kind of math support provided to the CTE instructors. The revisions in the model emphasized more bridging between the CTE and mathematics vocabularies and increased attention to how the embedded math is represented in traditional math instruction. Increased math support was provided through additional extended professional development workshops, cluster meetings of small groups between the workshops, Web sites with resources for each of the occupational areas, and are porting system for monitoring the collaboration between the CTE and math instructors.
The full-year study was implemented in five of the original six occupational areas. Pretesting was conducted in the fall of 2004 and posttesting was conducted in late spring 2005. The report of the full-year study will be available at the end of 2005.
Stone, J. R., III, Alfeld, C. Pearson, D., Lewis, M. V., & Jensen, S. (2005, September). Building academic skills in context: Testing the value of enhanced math learning in CTE (Pilot study). St. Paul, MN: National Research Center for Career and Technical Education, University of Minnesota.