Value-Added Measures (VAMs) and Instructor Effectiveness: Unpacking the Debate

Teachers significantly impact student learning, yet there is no consensus among researchers, policymakers, and administrators about how and why teachers' instruction promotes student achievement. While there are many ways to link teacher instruction to student achievement, one family of methods -- Value-Added Measures (VAMs) -- has generated national headlines (such as this article about Los Angeles teachers). Despite VAMs' presence in the news and in many schools, their complexity and proposed uses have caused controversy and confusion. Therefore, I hope to clarify how researchers define VAMs, describe how critics argue against them, and catalyze conversations among teachers, administrators, and policymakers.

VAMs and Their Proposed Uses

Researchers generally define VAMs as statistical models designed to measure a teacher's influence -- the "value" added or subtracted -- on student achievement over time. Typically, student achievement is defined using standardized test scores. To determine a teacher’s value, researchers make two primary calculations:

1. They compare how students actually performed on tests to their predicted performance given the test scores of previous students in a teacher's class.
2. They evaluate how teachers with similar VAM measurements impact student test scores over time.

Thus, researchers using VAM models claim to both isolate the value of an individual teacher and show which teachers add more value over time as compared to similar peers (for a more in-depth discussion, see Vamboozled, and Sean Corcoran and Dan Goldhaber’s Value Added and Its Uses: Where You Stand Depends on Where You Sit).

Supporters of VAMs propose two primary uses for such models. First, VAMs can be used as professional development tools, helping teachers target areas for growth. Second, school and district leaders can use VAMs to make workforce decisions -- recognizing and rewarding effective teachers and denying tenure and dismissing the lowest-performing teachers, according to Corcoran and Goldhaber.

3 Primary Critiques of VAMs

While supporters of VAMs believe that such measures are beneficial, critics argue that the models and their proposed uses are flawed. Here, I highlight three primary critiques (see The Economic Policy Institute’s 2010 report, Problems with the Use of Student Test Scores to Evaluate Teachers for more).

1. Teacher "value" ratings from year to year

Researchers have found that VAM measurements of a teacher are "unstable," meaning that the "value" of a teacher changes drastically from class to class and from year to year. For example, in a study of five large urban districts, researchers examined what happened to teachers' VAM scores over a few years. They found that, of the teachers who were ranked in the top 20 percent of effectiveness in the first year, "fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40 percent." In other words, teachers who are effective one year might be significantly less effective the next year according to VAM scores. Critics argue that if a teacher is highly effective one year, it's hard to imagine how they could become much less effective the next year.

2. Standardized tests as measures of student learning and teacher performance

While standardized test scores are typically the measure of student learning used by VAM researchers, studies show that test scores are subject to numerous factors that teachers do not control, according to Audrey Amrein and David Berliner (PDF). Such factors include the influence of students' other educators (such as teachers and tutors), class size, the quality of the curriculum, and various socioeconomic elements. Thus, critics argue that VAMs cannot "isolate" an individual teacher's effect on student test scores.

3. Using VAMs to make high-stakes personnel decisions

In 2014, the American Statistical Association (PDF), the National Association of Secondary School Principals, and researchers Morgan Polikoff and Andrew Porter expressed concerns about the use of VAMs when making high-stakes employment decisions about teachers. The reports argue that using VAM measurements when considering incentives or punitive actions could lead to inaccurate personnel decisions. In addition, talented teachers might avoid working in high-needs schools where test scores are low, or could leave the profession entirely.

VAMs and Teachers

Debates about VAMs demonstrate that teacher voices are often excluded from conversations about effective instruction and student learning. I propose that when teachers analyze test scores and VAM measures to unpack student achievement, they can have three conversations:

1. Teachers can define for themselves "what counts" as effective instruction by asking: "What evidence do I have that my students are learning? What does my analysis of evidence indicate about my current teaching, and how should I change my instruction in the future to better serve students?"


2. Teachers can work with colleagues to form collaborative learning communities that collectively unpack evidence of student learning, including test scores, and can plan for effective teaching. For an example, read in Science Teacher about the science teachers who systematically improved their practice through analyzing student work.


3. Teachers can talk with administrators to determine how student test scores fit into the overall picture of evidence for student learning. Ideally, such conversations can help teachers and administrators develop a shared vision of teaching and learning goals.

Such conversations can hopefully add teachers' voices to important discussions about instructional and learning goals.

Does your school district use VAMs? If so, please describe your experience with them.