Facebook
Edutopia on Facebook
Twitter
Edutopia on Twitter
Google+
Edutopia on Google+
Pinterest
Edutopia on Pinterest Follow Me on Pinterest
WHAT WORKS IN EDUCATION The George Lucas Educational Foundation

Where Standardized Tests Fail: Subjectivity and Imprecision Tend to Prevail

Exams are better at gauging the big picture than at evaluating students.
Grace Rubenstein
Former senior producer at Edutopia
Related Tags: Assessment,All Grades
Credit: Gregory Cherin

Today's standardized assessments can be useful for spotting big trends or gauging the effectiveness of state programs overall. However, when used in high-stakes accountability, as the sole indicator of an individual student's achievement or the quality of a single school or school district, these tests can be imprecise. Creating and scoring such tests is complex. Here are some of the steps in the testing process where subjectivity prevails and inaccuracies arise:

Test Making

  • Content selection: If the state sets too many standards, teachers won't be able to cover them all and will have to guess which are on the test. If test makers include too few questions on any given skill, the results may not truly show how well a student can perform it.
  • Ambiguous questions: Particularly for multiple-choice questions, a child may be able to make a plausible, even creative, argument for choosing one of the "incorrect" answers, but the format doesn't allow the child to explain.
  • Setting the difficulty level: This determination, typically based on educators' and officials' opinions, is naturally subjective. To select final questions, test makers often try them out on students, which works only insofar as the trial-run group accurately represents the students who will ultimately take the test.
  • Year-to-year comparison: To prevent cheating, states typically ask test makers to create new questions every year. Test makers must then perform the tricky business of trying to ensure that the exams are equally difficult so that scores can be compared like apples to apples.

Test Taking

  • Test preparation: The teaching of test-taking strategies may favor some students and keep their scores from reflecting what they actually know.
  • Distractions: Whether internal or external, distractions such as test anxiety, personal problems, lack of sleep, a sick classmate, or a broken air conditioner can distort students' scores.

Test Scoring

  • Mechanical or human error: Mistakes may occur in setting the answer key, feeding answer sheets into scoring machines, marking answers right or wrong, or other steps in the process.
  • Cut scores: These cutoff points for passing and advanced scores are based partly on educators' and officials' judgment, so they're subjective. Also, given the natural imprecision of scores explained in this chart, a student's score may fall below the cutoff point for failing even if she is knowledgeable enough to pass -- and vice versa.
Grace Rubenstein is a senior producer at Edutopia.

Comments (1)Sign in or register to postSubscribe to comments via RSS

Anonymous's picture
Anonymous (not verified)

What is the basis of the standardized exam. Is it logic, proofs or what? There must be a framework that is used by the examiners that makes it so hard? Must we study logic and proofs to beat them at their own game for both children and adults. They are not fair. I am trying to find the framework of these exams and study what makes the standardized test work. Please some one tell me if you know. thx

blog Games in Education: Teacher Takeaways

Last comment 2 weeks 15 hours ago in Game-Based Learning

blog Focus on the Process and Results Will Follow

Last comment 2 weeks 2 days ago in School Leadership

Discussion Intelligent people are happier

Last comment 3 weeks 2 days ago in Assessment

Discussion Why Your Kid's Grades Wont Matter: Part Two

Last comment 4 weeks 13 hours ago in Assessment

blog Growth Mindset: A Driving Philosophy, Not Just a Tool

Last comment 1 week 5 days ago in Teaching Strategies

Sign in and Join the Discussion! Not a member? Register to join the discussion.