George Lucas Educational Foundation
Subscribe to RSS

Silent Statistics: Student-Performance Data Misses the Most Important Outcomes

Anthony Cody

Science Coach and mentor, Oakland, California
  • Facebook
  • Twitter
  • Pinterest
  • Share

Recently, consultants who were reviewing the data systems the California Department of Education uses to track student performance interviewed me. I have had to wrestle with how I feel about the whole process, because unfortunately, I think the emphasis on data has not been the boon to students and educators that was promised. But as someone with a background in science, I have an inherent love of data. More information is always better than less. So how can it be that data could do harm?

What happens if we apply a medical analogy to our schools? We have implemented an evidence-based model whereby we use standardized tests -- in reading and math -- to measure the vital signs of our patients. When the signs are weak -- the scores are low -- we have prescriptive remedies, such as using scripted curricula, reteaching lessons, and spending extra time on core skills in reading and math. These remedies usually have the effect of improving the vital signs, which we take as an indication that the process was successful.

Science Friction

However, disturbing new pieces of data seem to indicate some unintended side effects to our remedies. Part of my job is to work with elementary school teachers on a grant for ways to improve their science instruction. The biggest problem we have had this year is that they simply do not have time to teach science.

The teachers in low-scoring Program Improvement schools have the hardest time: They must prepare weekly schedules listing how much time they intend to spend on each subject, and they are expected to average two and a half hours of reading and writing instruction each day, plus an hour and a half of math.

When you factor in an early dismissal on Wednesdays, this schedule leaves them about thirty minutes a week for science. But according to many of these teachers, they're likely to lose even that science time to reteaching a writing skill, so they may end up teaching no science for many weeks. Thus, the first detrimental side effect is the loss of time for subjects the tests do not emphasize: science, history, art, and physical education. Our data systems do not measure performance in these subjects very much, and by ignoring them, we are causing their systematic decline in our schools.

But even when we do teach science, the remedies for improving test scores can negatively affect it. This week, teachers at a meeting told me that their students were enjoying their science activities, but when they asked the students to read the science textbook, the kids rebelled. They associated reading in a textbook with the scripted reading curriculum, which they hate.

For these students, reading has become an onerous task, rather than a joyful thing. As the son of a bookseller and as an avid reader, that gives me great pause. If we teach reading, but in the process rob it of any joy, haven't we done more harm than good? And because the love of reading is not measured by any test, how do we even know what damage the scripted curriculum has done?

The Dropout Disconnect

This issue got me thinking about other ways our current data systems may be missing crucial pieces of the puzzle.

The biggest missing piece is the dropout rate, which seems to be climbing, especially in urban schools where we have most heavily used these remedies. The trouble here is that when the least successful students leave the schools, the average test scores actually rise. A student who drops out is akin to a patient in a hospital dying rather than being healed. But although hospitals usually trust doctors to provide diagnoses that go beyond the vital signs and know why patients die, our dropouts leave us quietly, and we are left with scant evidence to explain their departure.

The research on why these students are dropping out is limited. I have some ideas, however, based on my eighteen years of teaching in an Oakland middle school. I think students drop out when they are unable to feel successful in school and when they cannot connect success in school to a vision of their own future. They drop out when they are bored because they have become disengaged from the pursuit of knowledge.

How does that disconnection occur -- and how can we repair it? Has the growing use of scripted curricula and the emphasis on test scores actually increased this alienation? This, it seems to me, is the most urgent question we face. Like the love of reading, this is not something measured directly by any test. Because there are no systems to measure student engagement, and because we don't ask or trust teachers to diagnose the reasons kids drop out, all we have is the indirect evidence for the rising rate.

In medicine, systematic efforts are in place to consider all the possible side effects of remedies. In our schools, however, we seem to have fixated on a measurement system and implemented remedies with little regard for possible side effects. We are watching those test scores like hawks, and dosing our students with the "proven" remedies. But the teachers with whom I work are reporting a lot of side effects not reflected by the test scores.

The ultimate side effect, the student who drops out, is completely removed from our data set and thus not even measured. Do half of our patients have to actually die before we realize the medicine is not working?

Please share your thoughts.

Was this useful?

Anthony Cody

Science Coach and mentor, Oakland, California

Comments (12) Sign in or register to comment Follow Subscribe to comments via RSS

LJ Fleming's picture
Anonymous (not verified)

Anthony Cody's "Silent Statistics" resonated with me. I would agree with all that he wrote. I think his concerns related to too little time spent teaching other than to the test, students rebelling when asked to read, and increasing drop-out rates are well-founded.

Cody's medical analogy and Anne Jolly's asking if we would like our doctor or the head of the hospital to write our prescriptions are both accurate. All these remedies do have side effects, but we are ignoring them. Meanwhile, we are pushing the teacher who knows the student best out of the equation in order to follow and use more and more standard prescriptions.

Becky Malone's picture
Anonymous (not verified)

I too share your concerns. As a fourth grade teacher, I consider myself a foundation builder for future life skills my students will need to become successful citizens. Testing phobia is endangering us as a society of learners. Reading is the skill needed to pursue other interests and being able to pass a multiple choice test stifles the development of the whole person.

Jacob's picture
Anonymous (not verified)

I enjoyed reading your article. As a student pursuing a degree in education I feel that test scores are way over emphasized. Teacher are then teaching what is required and lose creativity. Students are taught concrete material and no longer realize what they are learning and why they are actually learning it.

Nathan H's picture
Anonymous (not verified)

After reading Anthony's article and currently taking an assessment class where we are discussing formative and summative assessments of English Language Learns, I have a major concern. My concern is how we communicate a student's growth and development to our stakeholders (students, parents, tax paying public, state and federal policy makers). For our students it is easy to express to them the amount of growth they have made over a period of time, but for the other stakeholders, it is more difficult. How can we easily and effectively communicate this to the other stakeholders? A report card? A standardized, state mandated, NCLB driven assessment score? I have seen that these two measurements are not always the same, and sometimes, not even close to each other.
Because of this wild difference in what is being communicated to parents and state/federal policy makers I am struggling to find or develop some form for communication to all stakeholders that can show the growth toward the goal that doesn't say "You are growing so much, but you have an F/non-proficient grade at this level.

Jennifer's picture
Anonymous (not verified)

Nathan, I completely agree with what you have described here about the gap between what we see in the classroom and what the test results show. Having just received our state mandated testing scores last week, I'm feeling the backlash this week. Just today, I received an email from a parent of one of my special needs students. This student is very intelligent but suffers from severe ADHD even on high doses of medication. She has shown such significant progress this year that at her recent IEP meeting, her special ed. teacher and I decided to phase her out of special ed. for reading with the hope that she would be able to sufficiently participate in reading with her class next school year. This particular student missed passing the state test by one point and now mom is concerned that she won't be successful next year in the regular education setting. So much weight has been placed on this state exam that simply explaining to her that one number is not the indication of absolute success was not enough to ease her concerns. I found myself wondering how I could convince her that her child's progress can't truly be measured by this test. The test doesn't see what I see every day. The test doesn't watch her read with beautiful expression and ease. The test doesn't look at the lively story board she created to demonstrate her understanding of the story. How do we encourage students to continue the great work when all of their hard work is overshadowed by one failing score?

Jennifer Wisz's picture
Anonymous (not verified)

I teach second grade, and unfortunately my experience is that this need for data, data, data is killing our students. My students can't just read for the sake of reading and enjoying. They have to read so I can test how many words per minute they can read. They have to read so I can test their reading level. They have to test their reading comprehension with a 52 question reading test three times a year. When is reading fun? I do my best to promote reading as an opportunity to learn, and experience joy, laughter, etc., but when everywhere we turn reading is used to accumulate data it's hard to convince seven year olds that joy can actually be found in books.

I also recognize the loss of experimentation and wonder that comes along with science. I am guilty of only finding the time to complete three science units this year which sadly is more than many of my colleagues. I love for my students to explore and truly become involved in their learning, but when you're left with about fifteen minutes a day for this how do you make it work? With the end of the year upon us, I asked my students to write about their second grade year sharing what they loved most, what they wish they could have done more of, etc. One of my students referred to our Matter unit, where my students completed an indepth exploration of the properties of solids, liquids, and gases and shared their findings in their own unique compilation, as the best part of second grade, and he wished we could have "done more Science." Mrs. Wisz does too, honey. Mrs. Wisz does too!

Frank's picture
Anonymous (not verified)

Wonderful article!

I believe the biggest problem lies with the state standards and what I will call "age based expectations". Once a child's birth date is known, the standards, curriculum and high stakes state assessment are all established - based only on the grade level (which for all practical purposes is indicative of age).

Districts and state BOE's can talk all day long about Vygotsky and meeting the needs of "all students", but reality can not be denied. Students arrive in schools with different levels of competency and preparation, and expecting teachers to "differentiate the instruction to meet the needs of all students" has proven to be an enormously underestimated task.

Again, the main problem is a denial of Vygotsky's major contribution- each learner's zone of proximal development or ZPD- by marching students in lock step- by grade level, through the curriculum year after year. Imagine how crazy it would be if colleges worked the same way. Where else in a learning environment is age so critical to the content taught? Would a piano teacher use a new students age to decide where to proceed with instruction? Would a golf instructor decide where to concentrate the instruction based upon a clients age? yet in education we ask teachers to do so.

How sad for so many.

Jose Lara's picture
Anonymous (not verified)

So what is the alternative to testing?

Ellen's picture

I teach high school social science and have to revive the love of reading in my students who were fed a steady diet of Open Court for the past 8 -10 years. I do that through an extensive SSR library and watching them move through stages of literary appreciation. It takes a few months for them to realize I won't quiz them on their reading. I may require a response, but their grade is based on simply participating.
It looks like the drive to push back data is coming but the forces on the other side are very powerful.

kenny mikey's picture
kenny mikey
Comedic Educator and Motivator

So, what's the disconnect? You specifically mention that kids dropout when they're unable to feel successful or get bored. YES! A thousand times over, YEEEESSS!

I think this is a MAJOR problem for schools in general. Too often learning isn't FUN. And kids learn best - heck , EVERYBODY learns best - when they're having fun! Remember Schoolhouse Rock? Those short pieces were fun and you probably learned more in 3 minutes than your learned by traditional instruction in 3 hours in school.

That's why we create the videos we create. We have an award winning FUN series called "This is ONLY a Test!" that helps reduce test stress. And, guess what - they're short, 3 minute videos. We were inspired by Schoolhouse Rock, for sure.

Yea, I have a company, but don't take this as spam. The reason we even created our product was because kids were flipping out about these tests! We're trying to make a difference. Our true passion is getting kids excited about reading. Our motto is "When you Read, Every Day is an Adventure!" But, we saw all the test stress when NCLB was signed in 2002. We know how to make learning fun, so we thought we could make stress reduction fun too. We've been working with schools for years and we're now on the verge of putting out a new series that's for HOME use. PARENT involvement is crucial to a child's success, so we wanted to create something that parents could watch with their kids that would be fun and MOTIVATING.

We're actually doing a free webinar this week about how parents can help reduce test stress in their kids. If you want to see what it's about, go to

I think FUN is underrated and undervalued in schools. We're ever learning, fun-having creatures - we're the otters of the universe! And when the right stimulation is put in front of us, we absorb information like a sponge.

Sign in to comment. Not a member? Register.