Team Teaching Practices Affect Value-Added Measurements

February 12, 2012

Schools can be closed. Compensation for teachers and principals can be affected.

Classroom Value-Added analysis metrics are now being applied in a variety of ways, ranging from simple school and classroom level reporting to high-stakes decisions. As school districts and states use these classroom value-added estimates in such high-stakes contexts it’s important to fully understand any limitations in the links between student achievement and teacher performance.

Linking performance data of students to their teachers might seem like something that district data systems automatically and routinely do, yet in most cases doing so is a real a challenge. That’s because most school student information systems were not designed to account for team-teaching approaches.

Jeffery Watson, a researcher with WCER’s Value Added Research Center, explains that most states and districts lack the capacity to measure the effects of team teaching. Their data systems don’t account for innovative models of instructional organizational and can’t verify such student-teacher linkages.

Team teaching occurs when more than one teacher plans and delivers instruction to a student or group of students. Team-teaching practices include these instructional methods: Pull-out instruction is when a student is removed from a classroom to receive alternative instruction from another faculty member. Push-in instructiondescribes instructional services that are provided by a mainstream teacher for children who otherwise receive most of their instruction in a resource room, or self-contained special education room.

The validity of student-teacher linkage data and classroom-level value-added estimates depends on two things: the degree to which team teaching and innovative practices occur, and the degree to which a district is able to systematically record classroom-level practices.

If value-added methods do not account for these practices when attributing student growth to individual classrooms, educators who engage in team-teaching and multiple instruction will be less likely to buy in to, and trust, value-added estimates.

A recent study by Jeffery Watson and colleagues assessed a large urban school district’s student information system to determine whether and how student-teacher linkage data matched the reality of schools and classrooms. The VARC team assessed

• The degree to which teachers were correctly associated with courses,
• The degree to which student rosters were accurate, and
• The extent to which team teaching and additional instructional services occurred. 

The study focused on nine schools in this urban district. The schools varied in terms of size, grades served, and location within the district. But all served at least two grade levels in which students took the state standardized test.

The study found three ways that errors in student-teacher linkage data are introduced:

• Incorrect class rosters: About 2.5% of all student records were associated with incorrect courses, although this varied by school.
• Incorrect teacher-course associations: Incorrect teacher-course assignments were observed to corrupt more student records (7.6%) than incorrect student rosters. That’s because a single misplaced teacher-course association resulted in incorrect student-teacher linkages for all students assigned to that course.
• Failure to capture team-teaching instructional practices: Team teaching was observed in all schools and it accounted for errors in 4.1% to 30.2% of student records validated by this study. Most teachers reported sharing between 50% and 75% of instruction with another teacher.

Tracking Team-Teaching

Study findings suggest that student-teacher linkage data from the student information system made fairly accurate connections of students and teachers to courses. However, the student information system did not appear to adequately track team teaching, including pull-out and push-in services. There was considerable variation in linkage accuracy rates across the schools.

Summary

For the nine schools included in this study, much of the data (74.2%) in the district’s student information system validly reflected how teachers delivered instruction to students. But student-teacher linkage data quality was inconsistent among schools and was likely driven by characteristics of the schools themselves. For example, in schools where teachers use team teaching there’s more likelihood of error in student information system data. Overall, for the small sample included in this study, the accuracy of student teacher data varied between 54.7% and 92.5%.

Watson says that school districts implementing classroom-level value-added estimates should carefully consider how team teaching could affect the validity of their estimates. If team teaching introduces bias into value-added estimates then it will be important for districts to verify when team teaching occurs, and for economists to develop methods that can take into account information on team teaching.

Watson says districts and states should not consider implementing high-stakes policies based on student-teacher linkages without at least analyzing the quality of those linkages, including team teaching. If a data quality assessment is conducted, a validation processes may be required before stakeholders will accept the validity of classroom level estimates of value-added.

Watson says the findings from this pilot study provide evidence that warrants larger scale analyses of student-teacher linkage data. Next steps for his research include replicating the current study using an online verification process for an entire district so that more information about these factors can be gathered.