What Works in Public Education

How to Design Better Tests for Students

By Ben Johnson

7/2/09
Print Forward Share Comments(13) Comment RSS

I am taking a class with Cisco Networking this summer, and we are doing the equivalent of a semester of a high school course in one week. We spend two hours listening to a lecture of the key points, and then we take a test. In order to continue in the program, we have to pass the test with an 85 percent effort. Talk about high stakes!

When I was taking the tests, I noticed something. Because we often studied the chapter the night before, you would think the tests would be easy for us, but some of the questions on the test are designed to trick you into selecting the wrong answer. So even though I'm well versed in the material, and it is fresh in my mind, I still have yet to ace one of the tests. (Grrr.)

Although I am passing them reasonably well -- some just barely -- it is frustrating to not get it perfect. The fact that others are struggling too, helps, but the one student in the class who gets 100 percent gets open congratulations and a silent "I'll catch you yet!"

Anyway, this got me thinking. If we want to test a student's knowledge, shouldn't we just straight out ask the question? Is it necessary to throw in distractors, misleading answers, and ones close to the right answer, but not quite? Is that fair?

Hold that thought. I remember taking an assessment-design class in college and saying to myself, "What teacher is going to have time to make all these test questions and design a scientific pretest and posttest for every unit?" I have since discovered the answer to this question: I didn't make time, nor do many other teachers.

Another question, then: If valid assessment is so important, how do you do this? Before I answer that, let's go back to my other thought.

Differentiate Test Questions

Distractors, misleading answers, and trick questions are important in establishing level of difficulty; we can't get rid of them! Remember, we have to differentiate our instruction, so why not our tests, too?

Another reason is that there are two types of tests: summative and formative. One of the safety nets of my Cisco class is that our instructor gives us three chances to take the test. After taking each exam, we can look at the right answers and figure out what we did wrong and then take it again.

This is the main characteristic of formative assessment -- a chance to reflect, and then try again. When I go back and figure out how I blew the question, I actually learn more. It sticks in my brain better!

Back in my college assessment-design class, I learned that there need to be some easy questions, some challenging questions, and some hard questions on each test. A sophisticated teacher will assign different point values for each. A more sophisticated teacher will make sure each question is also aligned with a state standard (but that is a conversation for another day).

Anyway, the struggling student will most likely get the easy questions correct, while the advanced student will be challenged with the hard ones. Both feel that the test has made them stretch, and both can feel success in the questions they answered correctly.

Collaborate with Colleagues

How can we find time to create these tests? The way to find the time to do this is simple (unless you teach in a small school where you are the only teacher at your grade level and are willing to give up your summer vacation time to redo all of your tests by mining the textbooks and the Internet). You work with your teacher peers and come up with the tests -- pretests and posttests -- together. You share ideas, and you design a better mousetrap -- I mean test -- as a group.

Now if you consider the sophistication mentioned above, you all will be able to gather valuable student information that helps you compare teaching performance and predict student performance on state testing.

Some other time-saving tools for designing tests are turning to question-item banks, using Scantron answer sheets, and employing online test-taking tools. The key factor is that if a test is made with the collaboration of colleagues, it will surely be a better product, more useful, and, overall, less time consuming for the teacher.

We can't expect students to spend several hours studying each night and be excited about taking tricky tests, but we can challenge them with well-designed and useful assessments so they can learn and experience successes from meeting the challenges. Please share with us how you like to challenge and excite your students with difficult exams and quizzes.

0
was this helpful?
Terry Smith
Posted on 7/07/2009 9:02am

Creating Fair Tests & Assessments

Thank for this article - the topic has troubled me for many years of elementary teaching. In my class, I consider a fair assessment to be straightforward, addressing clearly what we have been discussing and learning. There are no trick questions, no game-playing, no red-herrings. At the same time, I do not test in a remember-and-replay-back to the teacher method. Students should be prepared to synthesize ideas, apply concepts, and know important details and connections and to do this in speaking and writing and other forms of expression such as audio or video or blogging. The idea of reflecting on what has been done, examining it, then pursuing the concept again makes learning more comfortable - the idea of testing begins to soften.

The standardized tests that my students take are full of "trick" questions - exercises that both frustrate the student and waste the student's time.

0
was this helpful?
Ben Johnson (author)
Posted on 7/09/2009 3:45am

Softer testing

Terry:

Some people might call teacher current testing procedures "loosy goosy," but softer is a more politically correct term. Teachers create easy tests for a number of reasons, none of them acceptable. Unconsciously (or maybe on purpose) a flat or easy test is given to make the teacher look good as an educator. Another more philanthropic reason is for the students to have a successful educational experience so they do not turn off to education. Lack of teacher time might be another reason for flat tests. But the most disturbing reason is that it takes more work to come up with a test that differentiates and is aligned to standardized student learning objectives.

This raises the specter of teacher accountability. Administrators look closely at teacher instruction (or they should). Who looks at teacher testing? Probably no one. I am not suggesting that the principal turn into the testing czar but, if no one looks at the day to day testing quality, then the quality will probably follow a bell curve, most of it being mediocre. The only solution is to have related content area teachers design their tests together. It keeps everybody honest and inserting peer review into the testing arena brings up the quality of the tests dramatically (assuming that all the teachers want to have better tests--that is where the administrator provides motivation). Ultimately we want tests to test at all levels easy, medium and hard and we want to be able to look at the results of the test and know what standard is not being met by our teaching. Now if you give a student such a test and in all the classroom learning experiences you never tried to trip the students up, then most likely the student will not do well on test. Just as in athletics, the practice has to be as hard as the real thing.

My recent experience was a positive one, though difficult. The trick questions made me change how I studied for the material. Just a cursory first read was not enough because I found that the important details kept slipping through. I found that I actually had to take notes while I read, and then review my notes before class. That extra level of effort made all the difference.

Good luck with this and do not get discouraged.

Best Regards,
Ben Johnson
San Antonio, Texas

0
was this helpful?
Donna Jorgensen
Posted on 7/22/2009 9:58am

Assessment and Testing issues

Thank you for this timely posting. I have intitial certification teacher candidates currently struggling with what assessment needs to look like. They want to do a good job, but the very idea of creating pre-during-post assessments of their students' learning and growth is daunting. This is a short tidbit that I can share with my candidates to help them to see that we all have this struggle and that working together with a clear understanding of our goals will help. Maybe tonight they will have a glimmer of hope that they, too, can survive the assessment pressures and make their way through the forest of confusion.

0
was this helpful?
Ben Johnson (author)
Posted on 7/23/2009 4:56pm

Glimmer of Hope

Donna:

Thanks so much for your kind words. I am glad such folk as yourself are engaged in training our teachers to be. Please share with them that yes it takes work to create aligned and effective tests, but the results will show up in more precise teaching and learning (working smarter, not harder- or as Covey would say- sharpen the saw once in while). Good luck to you in helping them make sense of the forest of confusion.

Ben Johnson
San Antonio,TX

0
was this helpful?
Dawn Bloom
Posted on 7/25/2009 6:14am

Testing, Professionalism & Brain Research

Assessment is just as important as our teaching practices. It is no wonder that we want to design tests that will monitor our students' progress as well as our own. Being an educator means being professional and true to ourselves. While it is always nice to see high test scores, it is not always professional to skew those scores in our favor. It does a huge injustice to our students and our professional behavior is then compromised.
Just as our teaching methods need to conform to every learning style, so must our assessments. Research on how the brain learns lends itself to teaching all types and styles of learners. We must present our material in so many differing styles so that we do not discriminate against any of our learners. Research has also shown that when we hear and see information multiple times and in differing ways, then we are better able to remember it. We make connnections with prior learning. As educators create tests to assess student learning, we must keep this research in the backs of our minds that remember that we design instruction to meet the needs of every learner and we must design assessments to meet the needs of every student.

0
was this helpful?
Dawn Bloom
Posted on 7/25/2009 6:18am

Assessment, Professionalism, & Research

Assessment is just as important as our teaching practices. It is no wonder that we want to design tests that will monitor our students' progress as well as our own. Being an educator means being professional and true to ourselves. While it is always nice to see high test scores, it is not always professional to skew those scores in our favor. It does a huge injustice to our students and our professional behavior is then compromised.
Just as our teaching methods need to conform to every learning style, so must our assessments. Research on how the brain learns lends itself to teaching all types and styles of learners. We must present our material in so many differing styles so that we do not discriminate against any of our learners. Research has also shown that when we hear and see information multiple times and in differing ways, then we are better able to remember it. We make connnections with prior learning. As educators create tests to assess student learning, we must keep this research in the backs of our minds that remember that we design instruction to meet the needs of every learner and we must design assessments to meet the needs of every student

0
was this helpful?
Ben Johnson (author)
Posted on 7/25/2009 6:43pm

Real testing

Dawn:

You bring up some good points. Professionalism in testing. What does that mean? Well, it means that rather than teaching teaching teaching and then testing, we pretest, teach and post test, pretest, teach and post test and then pretest, teach and post test. The tests must be created before we teach (no this is not teaching to a test--it is knowing precisely what your students will be needing to know before you even start teaching).

Although I agree with you in some parts of what you say, I feel compelled to help clarify your thinking. 20 years ago, we probably could have gotten away with the shotgun differentiate for everybody approach. But today, I think the scales are decidedly tipped with the large majority of students who learn best bodily kinesthetically. That does not mean that they do not have to learn how to learn auditorily or visually, but it means that we are teaching and testing using methods that do not match the student preference. So, if we are smart, we will teach to their strengths (again, not ignoring the other methods of learning). Why not also test to their strengths? There are many ways to find out if a person has learned, not all of them have to be paper and pencil. As a Spanish teacher, I knew when the students had understood a request in Spanish because they actually did something I could see. Parate por favor! (stand up please) If they stood up, then they knew what I had said. Along with written tests, I did verbal tests too. We therefore, need to teach and test, heavy on the bodily kinesthetics.

Hope that helps.

Ben Johnson
San Antonio, Texas

0
was this helpful?
Kylene
Posted on 7/28/2009 8:50am

Testing difficulties in music

I find this topic particularly interesting as a music educator, for many times my co-teachers and I struggle to find ways in which to assess and test our students. Challenged by our administration to make courses more rigorous, my colleagues and I have been working to design assessments that are appropriate for all types of learners. Also, some of the principals have asked to see more “paper and pencil” assessments from the music department, which leads into discussions of some of the issues mentioned above, particularly, how to best differentiate the test questions.

The subjective nature of music can be difficult to properly express on paper for younger students, while older students may have more knowledge, but be unable to actually produce some techniques on their instrument. Additionally, some of the classes I teach consist of two to four grade levels at a time, therefore the different age levels of students naturally fall into different categories of knowledge, skill, and ability, making testing even more challenging.

My colleagues and I differ in opinion on how to best assess abilities of knowledge and skill. Some music teachers feel that if a student is able to demonstrate a skill on their instrument, then they have met the requirements. However, other colleagues, including myself, feel that this type of assessment does not show if a student truly understands the concept of the particular skill and why it is necessary.

Perhaps a combination of these types of assessments would be best, however, we have not reached a balance in which we feel is appropriate. In efforts to hold students to high standers, I will continue to work ideas to strengthen the use of testing materials in my classroom. I look forward to discussing some of the ideas mentioned above with my colleagues so that we might challenge our students, as well as ourselves, as we work to help students achieve success in music.

0
was this helpful?
Ashley
Posted on 7/29/2009 10:01am

I think this is a very interesting topic for discussion. I don't believe in trying to trick students on tests. What's the point? There are other ways to help students come into higher levels of thinking, but tricking them only does just that...tricks them. I do like giving pretests, teaching, and then giving a post test. It makes the student responsible for his own learning. They see what they need to know when they take a pretest and they are challenged to learn it before they have to take a post test.

0
was this helpful?
Danna Bonney
Posted on 7/29/2009 7:36pm

Creating Fair Tests and Assessments

This topic is troubling by nature, due to the reality that creating fair tests and assessments must address the range of learning and teaching styles. I agree that fair assessments must be straightforward, clear, free from tricks and rich in opportunities for multiple expression of learning. Consistent and frequent reflection on what has been taught and learned, along with extension into analyzing and applying concepts, is critical. So, what do we do with the time-wasting, high stakes standardized tests we throw at our students? I'm interested in the concept of e-portfolios that appears to be slowly arriving on the standardized testing scene. Do you see this as something that will pick up and gain momentum in the near future? I am puzzled why more states don't adopt this practice as a valid assessment tool.

Post a comment

Sign in or create an account now, or after you post.

Sign In

Thanks for your comment. It will be posted once you've signed in to your account. Please sign in here
Not yet a member of the Edutopia community? Create an Account

Create an Account

Almost there! As soon as your account is created, your new comment will be posted.
Mollom CAPTCHA (play audio CAPTCHA)
By creating an account, you agree to Edutopia's terms of use.

Edutopia.org Blogs

Advertisement

@edutopia on Twitter Edutopia on Facebook RSS feed link

Advertisement