Toward Genuine Accountability: The Case for a New State Assessment System
A in-depth discussion on revamping state assessment.
Wiggins’ blueprint for state assessment would provide students with timely and useful feedback on how to improve their work, something the author believes current statewide accountability systems fail to do.
Imagine if basketball season ended in a special test, on the last day of the year, in which the players -- and coaches -- did not know in advance which drills they would be asked to do. Imagine further that they would not know which shots went in the basket until months later. Imagine further that statisticians each year invented a different (and secret) series of such "tests of basketball." Finally, imagine a reporting system in which the coach and players receive the scores -- long after the season has ended -- without knowing exactly which drills were done well and which were not.
The inevitable then happens (since these new basketball test results would be reported in the newspaper). Coaching becomes distorted in a nervous effort to address the test's form and content. Coaches stop worrying about complex performance (i.e., real games) entirely, to concentrate on having students practice drills -- at the expense of student engagement and genuine learning.
Who would improve at the game under these conditions?
Yet, this is what in many states is called accountability of schools when applied to academic standards and state testing. As the analogy suggests, it is really the illusion of accountability. Current tests provide woefully sketchy and delayed feedback, on tasks that do not reflect real achievement. Current approaches to testing and reporting, in fact, unwittingly cause impoverished, not rich and creative, "teaching to the test" items. As a feedback system our current tests are a failure, in other words, regardless of the needed attention to results and standards they cause.
There is a better way: an assessment system, based on common-sense principles about how people improve and are motivated to improve. A more responsive system based on helpful feedback to improve learning. A system that makes local work and teacher judgement more central to state accountability. A system designed to provide incentives for school renewal and built-in professional development each year for all teachers. A system that will inspire more creative teaching instead of more fearful compliant behavior.
My blueprint for state assessment would accomplish eight distinct tasks at the heart of genuine (vs. illusory) accountability. It would:
- measure student performance against state standards in credible, user-friendly ways, where how we test and the content we test have greater fidelity to instructional aims and state standards;
- provide teachers and students with timely, effective, and helpful feedback, to enable progress toward meeting standards;
- ensure that teachers of all grade levels and subject areas work as a team to meet standards responsibly and responsively;
- provide parents with user-friendly and helpful information about how their students are doing now, what the long-term trends are, and guidance on how parents can help students improve performance;
- be minimally intrusive (by not over-relying on time-consuming, one-shot, tests with no value for current teaching and learning);
- constantly strengthen and offer incentives for high-quality local student assessment;
- provide incentives for local districts to continually improve student achievement;
- enable policymakers at the state and local level to know how students are doing in reference to all the state standards, and have confidence in the results.
A Proposed State Performance System
I do not profess to have all the answers about the "what" and the "how." Necessary details await future inquiry, discussion, experimentation, and ownership of the plan by all key constituencies. The key assumption is that local assessment should be a key feature of any statewide accountability plan. The cornerstone of the system would be a Student Standards Folder, a collection of evidence in relation to State Standards, to be scored against common criteria and performance standards on a yearly basis by regional teams of educators. The work contained in the Folder would include:
- Test data from state standardized tests in literacy and numeracy.
- Test data from district-mandated national standardized tests.
- Results from locally-scored state-approved writing prompts and performance assessment tasks. The tasks and prompts would be drawn from a state database of assessments, administered locally by educators at any time during the school year.
- Results from all relevant locally-designed assessments.
The proposed system requires a very different view of professional responsibility, not only extensive training. The proposed plan -- indeed, any plan worthy of being called a comprehensive assessment system -- can only be realized if the job of teaching is defined as requiring various non-contact days given over explicitly to student assessment, and if school schedules and policies are designed to make such work possible.
Rather than thinking of "professional development" as a series of random days devoted to in-service training, we must redefine the job of teaching to include scoring student work and adjusting teaching in light of analysis of results -- for which time is allocated.
The other pressing need is for adequate resources at the state level to ensure that a state Web site offers comprehensive guidance in how the system should operate, a library of user-friendly print and video resources on how to improve assessment, and detailed instructions on how staff can interpret Folder results usefully.
More generally, the state needs to get more into the business of providing models of exemplary assessment than in merely calling for local districts to figure out local assessment on its own. We call for a blue-ribbon committee that represents all major state constituencies to develop the full blueprint for the system we have only sketched.
Underlying Principles of Any Credible and Effective State Assessment System
A set of five guiding principles underlie our proposal. These principles serve as useful criteria, in other words, against which the specifics of this or any educational assessment system -- should be judged:
1. A good accountability system does more than audit performance. It is deliberately designed to improve performance.
Though commonsensical, this principle exposes the weakness of all current state testing systems. Because the principle implies that students, teachers, and administrators must receive timely, on-going, and user-friendly feedback, in response to credible performance challenges.
Focused and accountable teaching requires on-going assessment of the core aim of schooling: whether students can wisely use and reflect upon, not just recall, knowledge in simulations of complex adult intellectual tasks. Only by ensuring that the assessment system models genuine performance; in other words, will student achievement and teaching be improved over time? And only if the assessment system holds all teachers responsible for results can the system improve (as opposed to high-stakes testing in four of the twelve years of schooling).
2. Assessment must be credible if genuine reform is to occur.
Any effective assessment plan must be credible to all key constituencies. It must therefore provide "triangulation" of local and state data, robust and helpful feedback to educators on clearly worthwhile tasks, and intelligible information to laypersons.
Genuine accountability also requires credible assessment tasks -- work that more directly reflects the language of the standards and the reality of adult life. More real-world assessment and assessment more faithful to good instruction are key to getting beyond local excuses for test results.
3. "Local is better." "Trust but verify" must be the motto of an effective state assessment system.
Local is better in all walks of life. Standards are always upheld or not at the local level, day in and day out. The state can neither afford to assess each student on all important pieces of work, all year long, nor is it wise for the state to do so.
An effective state assessment system thus focuses resources and policy on ensuring that local assessment becomes more sophisticated, rigorous, and self-correcting. That goal is best accomplished by putting more authority, not less, in local hands; while also ensuring, however, that local assessors meet standards for assessment and intervention based on results, and where they have incentives to care about state standards and good local assessment.
Our proposed system is built upon this logic: trust teams of educators regionally with the responsibility of scoring work -- for all subject areas. Make results public, and framed in terms of standards. Then, verify local and regional scoring through a variety of audit systems. Students, teachers, parents, and board members can thus have confidence that the local community will not be surprised by state assessment results.
4. An effective assessment plan must build local capacity in high-quality assessment, not merely externally test once a year.
An assessment system should improve the quality of local tests, standards, grading, and reporting. Teachers cannot respond in an informed, timely, and effective manner to test results if they are left out of the assessment design and scoring loop.
5. State accountability must be designed to instigate local creativity and greater local control over standard-upholding, not a fearful compliance mentality.
Two cardinal principles of the Quality movement in business over the past two decades (as articulated by W. Deming) are "Drive out fear" and "No quotas." The point is to ensure that staff is driven by the right incentives to understand their job is continuous improvement, and to be rewarded for creative solutions to making progress toward meeting standards.
By contrast, teachers now avoid creative teaching in fear of test score declines. The secrecy at the heart of "secure" testing ensures that dread, not imagination, will drive teaching. By failing to provide genuine incentives for the improvement of local performance, we further promote the on-going atrophy of site-level accountability and high-quality local assessment.
The five principles help show why current state accountability tests are insufficient to improve student and teacher performance. Current state plans quite properly put local educators on notice about local performance against state standards. But audit testing, in a few grades at year's end, using secret tasks, is inherently unable to improve teaching and learning day in and day out in all classrooms, as my proposal is designed to do and any attempt at fair and comprehensive reform must do.