Assessment Subscribe to RSS

Silent Statistics: Student-Performance Data Misses the Most Important Outcomes

| Anthony Cody

Recently, consultants who were reviewing the data systems the California Department of Education uses to track student performance interviewed me. I have had to wrestle with how I feel about the whole process, because unfortunately, I think the emphasis on data has not been the boon to students and educators that was promised. But as someone with a background in science, I have an inherent love of data. More information is always better than less. So how can it be that data could do harm?

What happens if we apply a medical analogy to our schools? We have implemented an evidence-based model whereby we use standardized tests -- in reading and math -- to measure the vital signs of our patients. When the signs are weak -- the scores are low -- we have prescriptive remedies, such as using scripted curricula, reteaching lessons, and spending extra time on core skills in reading and math. These remedies usually have the effect of improving the vital signs, which we take as an indication that the process was successful.

Science Friction

However, disturbing new pieces of data seem to indicate some unintended side effects to our remedies. Part of my job is to work with elementary school teachers on a grant for ways to improve their science instruction. The biggest problem we have had this year is that they simply do not have time to teach science.

The teachers in low-scoring Program Improvement schools have the hardest time: They must prepare weekly schedules listing how much time they intend to spend on each subject, and they are expected to average two and a half hours of reading and writing instruction each day, plus an hour and a half of math.

When you factor in an early dismissal on Wednesdays, this schedule leaves them about thirty minutes a week for science. But according to many of these teachers, they're likely to lose even that science time to reteaching a writing skill, so they may end up teaching no science for many weeks. Thus, the first detrimental side effect is the loss of time for subjects the tests do not emphasize: science, history, art, and physical education. Our data systems do not measure performance in these subjects very much, and by ignoring them, we are causing their systematic decline in our schools.

But even when we do teach science, the remedies for improving test scores can negatively affect it. This week, teachers at a meeting told me that their students were enjoying their science activities, but when they asked the students to read the science textbook, the kids rebelled. They associated reading in a textbook with the scripted reading curriculum, which they hate.

For these students, reading has become an onerous task, rather than a joyful thing. As the son of a bookseller and as an avid reader, that gives me great pause. If we teach reading, but in the process rob it of any joy, haven't we done more harm than good? And because the love of reading is not measured by any test, how do we even know what damage the scripted curriculum has done?

The Dropout Disconnect

This issue got me thinking about other ways our current data systems may be missing crucial pieces of the puzzle.

The biggest missing piece is the dropout rate, which seems to be climbing, especially in urban schools where we have most heavily used these remedies. The trouble here is that when the least successful students leave the schools, the average test scores actually rise. A student who drops out is akin to a patient in a hospital dying rather than being healed. But although hospitals usually trust doctors to provide diagnoses that go beyond the vital signs and know why patients die, our dropouts leave us quietly, and we are left with scant evidence to explain their departure.

The research on why these students are dropping out is limited. I have some ideas, however, based on my eighteen years of teaching in an Oakland middle school. I think students drop out when they are unable to feel successful in school and when they cannot connect success in school to a vision of their own future. They drop out when they are bored because they have become disengaged from the pursuit of knowledge.

How does that disconnection occur -- and how can we repair it? Has the growing use of scripted curricula and the emphasis on test scores actually increased this alienation? This, it seems to me, is the most urgent question we face. Like the love of reading, this is not something measured directly by any test. Because there are no systems to measure student engagement, and because we don't ask or trust teachers to diagnose the reasons kids drop out, all we have is the indirect evidence for the rising rate.

In medicine, systematic efforts are in place to consider all the possible side effects of remedies. In our schools, however, we seem to have fixated on a measurement system and implemented remedies with little regard for possible side effects. We are watching those test scores like hawks, and dosing our students with the "proven" remedies. But the teachers with whom I work are reporting a lot of side effects not reflected by the test scores.

The ultimate side effect, the student who drops out, is completely removed from our data set and thus not even measured. Do half of our patients have to actually die before we realize the medicine is not working?

Please share your thoughts.

see more see less

Comments (12)

Comment RSS
Anne Jolly (not verified)

Former 8th grade science teacher

Was this helpful?

Amazingly accurate insights, Anthony! At this time in our education history we are sacrificing what matters for the sake of what is showing (test scores}.

I love your medical analogy. Teachers are the ultimate "doctors" who must write the prescriptions for the patients (students). They need to use practices that cure the problem rather than treat the symptoms. As a thought . . . would you like the head of the hospital to write your prescriptions, or your doctor? And thank goodness doctors are free to use the practices best for the patients rather than follow a generalized scripted approach. Shudder!

What can we do? Many teachers are plunging ahead doing what they know to be best for the kids. Not enough though . . . not enough.

Judy M. Austin (not verified)

Student-performance data misses important outcomes

Was this helpful?

I share some of these same concerns. Here is what I feel is happening in my own classroom: As my focus is shifted to test scores (End of Course tests, the High School Graduation Tests, the Advanced Placement test, and -- coming next year -- common benchmark tests), I can feel my emphasis shifting from teaching active reading and from cultivating a community of writers to what is specificially tested. My struggle involves this disconnect: I believe that the skills we teach in high school English classes are the life skills of reading, thinking, writing, research, thinking (Yes, thinking primarily!). How does that translate to a standardized test? How does that translate to our standardized tests that seem to more often involve testing on a computer than pencil and paper? If I work with my students to help them move to a higher level of reading, I work with active reading skills: annotating, summarizing, questioning the text. These habits of mind do not translate well to the format of the test: the computer screen. On that test, the students tend to do a cursory "quick read" and then guess what seems to be a reasonable answer. There is no "wrestling" with text that we want astute readers to be able to do. If we work together to improve voice and style and diction and support in writing, how well does that translate to selecting a multiple choice answer on a standardized test? At times, I have the feeling that we may lose the heart and soul of education: the development of the whole person, a thinking, questioning person who can seek out truth on his own. In his place may be a surface level reader who guesses well on multiple choice tests and who can select the right answer ABOUT writing-- but who in reality may not have the life skills of reading and writing that he needs.

see more see less