Learning By the Numbers: The Answers Are in the Details
Teachers — and entire school districts — are taking a closer look at test scores and other data to learn how to improve their schools and their students’ overall achievement.
Professor Victoria Bernhardt leads a session for educators on how to build a meaningful school portfolio.
For two and a half weeks, the consultant sat in a Grand Rapids, Michigan, school district office and pored through ninth graders' records from kindergarten on. With a statistician's eye, she graphed and charted the numbers that make up a student's school career: test scores, attendance rates, grades. She looked at demographics and school processes as well.
When officials in the Northview Public Schools got a look at the charts, they noticed something interesting: The ninth graders who still struggled with reading were also the students who had missed the most school in first grade, sometimes thirty or forty days in a 180-day school year.
Ninth graders were already getting remedial reading help, but the new data provided an opportunity to prevent younger students from the same reading fate as the high school freshmen. The district began more extensive screening of elementary and middle school students likely to suffer academically because of high absenteeism. Teachers, counselors, and principals followed up by working closely with parents -- in some cases making home visits -- to make sure the children got to school. "Individual accommodations to assist students were made, and efforts were targeted at the root of the problem," says Maureen Grey, Northview's director for instruction.
A Growing Trend
Northview represents a growing trend among educators toward taking a closer look at test scores and other data for the purpose of determining what's working and what's not for individual students, for classrooms, for schools, and for districts.
"We may disagree with testing," says David Rapaport, a social studies teacher at Bret Harte Middle School in San Jose, California, referring to many educators' qualms about the high-stakes nature of today's school testing -- in which a single exam can determine student promotion and graduation or a teacher's remuneration. "But we put those differences aside" when using scores and other data to improve the academic program, Rapaport says. Test scores are "one moment in time," adds Principal Milly Powell of the Erikson Academy, another San Jose public school. "We look at everything."
Taking a lead from business, these school officials have found that use of a wide range of data can be a formidable ally in guiding district policies, school curriculum, and individual student learning plans.
Basing Instruction on Facts
Many districts look to Victoria Bernhardt, a professor of professional studies in education at California State University at Chico, to get them started on the data-gathering and -analyzing path. Bernhardt, who is also executive director of Education for the Future, has used her background in statistics to show school officials how numbers can replace instruction by hunch with instruction by fact. It is an assessment form that can be applied at the individual student, classroom, school, and district levels.
Bernhardt, author of numerous books about improving schools through use of data, says data tell educators "whether they have a vision, whether they have a target, whether everybody's moving in the same direction." In addition, she says, use of data keeps them focused on reform that is working but may not be showing dramatic results. "Schools that do not use data tend to leave their school improvement efforts within two years," she says.
A number of educators are starting the data-analysis process at the district level with the intention -- when sufficient information is available -- of working down to the classroom level so that teachers can look at individual students' records over time, determine strengths and weaknesses, and base lesson plans for particular students on their long-term records.
Access to computerized data warehouses like those from TetraData Corporation, which assemble and compare data in minutes rather than weeks or months, will make it easier for such assessment to be of more benefit in helping individual students. TetraData can computerize every kind of data from demographic background to test scores to courses taken and then create graphs and tables sorted in a variety of ways, such as by gender, economic status, or ethnicity. It can make comparisons of different student groups to other classes and schools and to each other.
What's Inside a School Portfolio
The ideal school portfolio, says Bernhardt, includes data on demographics, student learning, school processes, and parent and student perceptions of school climate and academics. If teachers had the information they needed about every individual student -- a rarity -- it would contain multiple assessments, including grades, results of norm-referenced tests and criterion-referenced tests, writing samples, oral language samples, and classroom assessments. The information would also be consistent from year to year and would follow students over a long period of time, preferably their entire school career.
Administrators in the San Jose Unified School District have supplied schools with a variety of data, including the results of parent and student surveys, and they have encouraged principals and teachers to use the charts and tables to improve the school program.
Elaine Farace, principal at San Jose's Bret Harte Middle School, says she believes action taken as a result of analyzing data has resulted in better overall student performance in math, reading comprehension, and writing. Student performance is monitored every six weeks. If weaknesses are found, action is immediately taken.
The Power of Surveys
For example, survey answers showed that students at Bret Harte Middle School gave their school a rating higher than both county and national averages when it came to a caring school community. Still, staff and parents realized the results of the survey indicated a need for improvement in both student-to-student and adult-to-student relationships. The school quickly called a meeting of parents, staff, and students to develop programs to teach kids how to treat each other with respect and kindness, to improve adult and student interactions, and to support parents in teaching their children good social skills.
Another Bret Harte survey showed that students were home alone after school for up to six hours. The school started a sports and club program and entered into a partnership with the city of San Jose to open a teen center. "So we have supervision from 7:45 in the morning to 7 at night," says Farace.
"I think the beauty of the way we use data here is that it isn't just stuff on a piece of paper," says Bret Harte math teacher Bobby Csaplar. "We look at it and take action on it." When Csaplar noticed that her students weren't doing well on probability questions, she quickly incorporated probability into lessons on ratio and proportion.
Homework centers, writing across the curriculum, and intensive reading instruction for those scoring below the 25th percentile also have resulted from assessing data regularly at Bret Harte. One result, says Farace, is rising test scores in reading and math.
The students are involved in the improvement process, too. At Erikson, Principal Powell regularly meets individually with students and gently goes over test scores and other data to give them a more concrete picture of how they are doing. "I'll go, 'Where are we trying to go? Here's where you are. Are you there yet? What kinds of things can you do to get there? Do you think you could move [the numbers to show improvement]?'" The student typically answers in the affirmative, and then Powell asks what actions might be taken -- after-school tutoring, more reading at home? "We were real worried that it might not be positive, but the kids loved it," Powell says.
Surprise Findings
Marcy Lauck, supervisor of continuous improvement programs for the San Jose district, says data can help schools find information they didn't even know they were seeking. In Bernhardt's book, The School Portfolio Toolkit, Lauck notes that leaders at Castillero Middle School in San Jose decided to attend a portfolio workshop to aid in applying for a grant. "We left the workshop firmly convinced that, although we still wanted to apply for the grant, our commitment was to the data-based, continuous improvement process at the heart of the school portfolio," she writes.
Castillero leaders then decided to analyze the effectiveness of the school's academic, visual, and performing arts magnet program through data. They were confirmed in their belief that students "fully engaged" in the arts program would demonstrate more academic achievement than "less engaged" students. But the data, which included a breakdown of achievement by mainstream, special education, and English-language learners, also showed a huge achievement gap between Hispanic and white students, no matter how engaged in the arts.
"We readjusted our focus," Lauck writes. "The arts are still embedded in all that we do, but we have made a commitment to the achievement of high standards in reading by all of Castillero's students in all content areas."
Meanwhile, in Grand Rapids, Northview Public Schools are about to take the big step of organizing their data through TetraData to provide schools with understandable and speedy computerized data compilation and analysis. "Our goal is to make data analysis a teacher tool that will enable us to make instructional decisions about students in a timely way," says Northview's Grey. She says the district wants teachers to have all data about their students as close as their computer desktop. With such accessibility, teachers can create student instructional programs based on concrete knowledge about everything from individual learning styles to reading strengths and weaknesses.
"We are so excited about the potential and the future," enthuses Grey. "Once you taste this excitement, there's no going back."