The Testing Obsession
CAMBRIDGE, MASS. — With California leading the way, the nation is going through a frenzy of testing its public school students. Never before have so many students been given so many formal standardized tests. The college-bound student can expect to take the PSAT, the SAT and an assortment of achievement and advanced placement tests; college-bound or not, students also take state-mandated tests, as well as nationally normed instruments like the Iowa Test of Basic Skills or the Stanford 9. The stakes for students, faculty, administrators and even politicians have never been higher. Admission to college, cost of student housing, jobs and promotions for those in the teaching profession, and election to office--all hinge on whether the all-important scores have gone up, or up enough. But few have even posed the central question: What is the relation between test scores and a quality education?
The testing frenzy is a response to a per ceived problem. Over the last 20 years, Americans have become convinced that U.S. public schools are not doing a good enough job and, in particular, are failing their most disadvantaged students. Whether schools are absolutely worse off than in 1950 is not the issue; nor is there evidence that disadvantaged students were ever well-served. Yet, most observers would agree that today’s schools do not adequately prepare students for the knowledge economy that all developed nations compete in. So a predictable sequence has unfolded: 1) Create or resurrect instruments by which student performance can be assessed. 2) Attach high stakes to these performances. 3) Reward high test scores and punish those deemed responsible for their absence.
It is by no means easy to tell whether this strategy is working. The good news is, there are early reports of performance improvements in nearly every region. This is not surprising. Once a high-stakes test-measuring instrument has been revealed, the minds of everyone--students, teachers, parents, and the media--are wonderfully concentrated. Indeed, this phenomenon recently occurred in California, where most schools reported improvement on the Stanford 9 and on the Academic Performance Index that is derived from it.
But don’t start dancing in the streets yet. The experience of the last decade offers numerous cautionary notes. First, improvement based on sheer familiarity with a test is a onetime occurrence. Then, too, results are typically restricted to a particular test, or even to a particular version of a test. This happened in Chicago, when new versions of the same test resulted in system-wide regressions in scores. Finally, and most important, improvements on specific tests are often not reflected on more generic instruments. This was observed when the vaunted improvement on the Texas Assessment of Academic Skills was not matched by comparable improvements on the National Assessment of Educational Progress, a federal exam. Similarly, irrespective of local improvements in mathematics scores, international comparisons in mathematics and science consistently paint a bleak picture of the competence of American schoolchildren.
So what to do? It has been quipped that the best solution for indifferent test scores is to test even more, as if taking the temperature of a sick person repeatedly would in itself improve their health. A testing frenzy feeds on itself. But test scores in themselves should not be the goal of schooling; nor should practicing for the test be a primary activity for students. Rather, improved test scores should be a valuable but incidental index of a good education. If an effective education is taking place, then one should be able to administer a wide variety of tests from one year to the next and encounter consistently good performance. But no jurisdiction would dare to carry out such an experiment, because, deep down, policymakers know that higher scores are more likely to be the result of “teaching to version X of test Y” than the dividend of generally strong preparation.
But, a randomly chosen politician might respond, “We spend huge amounts of federal, state and local tax dollars on public education, and that amount has increased in recent years. Unless we don’t care whether or not that money is wasted, it is essential that we hold schools--teachers, students, and perhaps parents--accountable for student learning.”
There is nothing wrong in calling for accountability. The crucial question concerns its manner. In this country, we have made the mistake of equating academic accountability with testing--typically, the short-answer, machine-scored test. But many other forms of accountability exist.
In earlier times in the United States, teams of informed “inspectors” visited schools and evaluated them. This procedure is still followed in some independent private schools. U.S. colleges and universities, the envy of the world, do not depend on standardized testing of students; visiting committees evaluate specific departments or the school as a whole, and the subsequent careers of graduates are scrutinized by observers.
The Coalition of Essential Schools, a network of 1,000 high schools in the United States, spurns standardized instruments; instead, it advocates that students exhibit their work in front of knowledgeable teachers and experts, with graduation contingent on a certain number of successful exhibitions. Certification of teachers--for example, by the National Board for Professional Teaching Standards--rates of graduation, acceptance at four-year colleges, student involvement in community service, science fairs or debating clubs are other possible indices of accountability. Perhaps most important, a school should never be held accountable by a single metric; the best indices of accountability are pluralistic and suited to the context of a particular school in a particular community.
The goal of quality education requires an entirely different approach. We must begin with a vision of what it means to be an educated person; the means of assessment should follow from, rather than dictate, the ways in which we educate students. The educated person is one who can think well in terms of the major disciplines, in particular, who displays historical, scientific, mathematical and artistic understanding. Understanding is not the same as knowing lots of facts and figures. Rather, understanding in a discipline entails knowing how to make sense of a phenomenon that is unfamiliar but that can nonetheless be illuminated if one knows how to make sense of documents (history), conduct a controlled experiment (science), analyze a situation quantitatively (mathematics) or qualitatively (literature and the arts).
Should one adopt such a vision, the means of assessment should be performance-based. That is, students are given an unfamiliar example (an item from today’s news, data from a new experiment, a recently completed work of art) and are then required to make sense of it. Their score might be determined in part by their ability to indicate the steps that they would take, were they initially stymied. As Socrates taught us thousands of years ago, part of being educated is knowing what you don’t know and how to bootstrap yourself to understanding.
The United States is too vast to be satisfied with a single education vision. Indeed, California alone is too large. We could never come up with a collective education vision that would satisfy Jesse Helms, Jesse Jackson and Jesse Ventura. Our test-crazed society attempts to deal with this problem by ducking altogether the question of vision. This is a mistake.
We should develop perhaps a half-dozen educational pathways, each aligned to a particular vision of the educated human being. The pathway of “disciplinary understanding” described above would be one. As with current standardized tests, the results of assessments would be made public. But instead of the focus falling on numbers, whose meanings are obscure even to psychometricians, the focus would fall on actual knowledge that is cherished and the ways that knowledge has been exhibited by flesh and blood students. In France, questions on the baccalaureate examination--and they are not multiple choice--are published on the front page of the newspaper, and the public argues about them. This represents a more sensible state of affairs than the current numbing tables of numbers in our newspapers.
The limits of the current testing frenzy should sooner or later become clear, and the focus will eventually shift to the kinds of minds that we value and the best ways to fashion them. There will certainly be a place for testing--or, better, assessment--and families will gravitate toward the pathways that reflect their value system. The idea of a handful of pathways represents a meaningful midpoint between a single system, which involves too many compromises, and a plethora of charter or voucher choices that would be impossible to sustain and monitor.
The biggest cost of a system geared single-mindedly to test scores is that we virtually never hear any public discussion of what it means to use your mind well, to understand, to appreciate, to create knowledge, to be an educated human being. And so students can properly draw the conclusion that we do not care about these values.
Winston Churchill once remarked that the American people can always be counted on to do the right thing--after they have exhausted every other alternative. It is time for those who do care to make known their concerns and visions--and perhaps meet the challenge set by Churchill.
