‘Leading the Nation’--Where?

Berger is a middle school English teacher in Vermont, which pioneered the use of student “portfolios” to judge how schools are doing. The idea is that packages of essays and math problems completed by fourth- and eighth-graders in their regular classes give a better picture than standardized tests. But finding a consistent way to grade such work has been difficult, and the state recently reintroduced standardized tests to augment portfolios in comparing students’ work to statewide standards. Berger wrote this commentary for Education Week.

*

It’s no secret that public schools today lack credibility. We sail from one fad to the next on a tidal wave of jargon, trailing the wreckage of our latest “instructional innovation” behind us, and then we wonder why regular people find it tough to believe we know what we’re doing.

Take portfolios, for instance. Here in Vermont they tell us we’re “leading the nation.” If so, the news isn’t good for the rest of the country.

What exactly are portfolios? They’re folders where kids keep their work. If this sounds like less than an education breakthrough, that’s only because it is less than an education breakthrough.

Not that there’s anything wrong with keeping portfolios. Plato probably carried one to Socrates’ classroom. But boosters today promote portfolios as “accurate measures of educational performance,” offering “credible,” “standardized” data. Unfortunately, portfolio scoring simply doesn’t work.

Portfolios are scored according to rubrics, which detail theoretically “objective” standards and criteria. In Vermont, writing is judged in five “dimensions”--Purpose, Organization, Details, Voice and finally, Grammar, Usage and Mechanics. It is awarded one of four scores in each category--Extensively, Frequently, Sometimes or Rarely. These scores translate into a four-point scale. Innovators naturally insist that portfolio 4.0s and 3.0s have nothing to do with old-fashioned 4.0s and 3.0s--also known as A’s and Bs.

Suppose you’re trying to rate a piece’s Details. If you think the details are “explicit,” you score it an Extensively. If they’re only “elaborated,” of course, then you just give it a Frequently. Rating Organization requires that you distinguish between problems that “affect unity or coherence” (a Sometimes) and those that “make writing difficult to follow” (a Rarely). Scoring Voice is even easier. All you have to do is detect the difference between a “distinctive” tone, an “effective” tone, an “attempt” at an “appropriate” tone and an “appropriate” tone that’s “not evident.” Next you can decide if the piece contains errors that “may distract the reader,” or errors that “interfere with understanding.”

This is “objective”?

By the way, portfolio folks promote all this on the grounds that parents “like the idea of clearly defined standards.” Unfortunately, most teachers--and most regular people--have a hard time deciding, with statistical reliability, whether they’re being “distracted” or their understanding is being “interfered” with.

In order to “calibrate” their judgment with that of the “experts,” teachers score sets of writing samples called benchmarks. Inconveniently, Vermont teachers statewide traditionally disagree with experts’ benchmark scores more often than they agree with them. Even the “experts” commonly disagree with each other. Which scores are the “right” scores? You tell me.

The Rand Corp.--the state’s former assessment consultant--found initial Vermont scoring reliability “so low that most planned uses of data had to be abandoned.” Rand described scattered improvements in succeeding years as “slight” and “trivial,” with some reliability figures actually declining. Rand’s final report cited only “limited evidence from other programs that reliable scoring of writing portfolios is practical.”

Portfolio debacles aren’t limited to Vermont. A report commissioned by Kentucky’s Legislature blasted that state’s portfolio-based system as “seriously flawed,” concluding that “the public is being misinformed” about statewide results and “misled about the accomplishments of individual students.”

*

Portfolio programs are also proving far costlier than anticipated. In the words of one Rand analyst, the “early infatuation” with portfolios was “unrealistic.”

It’s certainly wrong to misrepresent teacher support for portfolio keeping as an endorsement of portfolio assessment. You can use portfolios in your classroom with your kids without ever engaging in the pseudo-objective scoring, the voluminous record-keeping, and the ghastly expense in both money and classroom time.

Portfolios were devised to provide “meaningful, useful data.” They don’t. That’s why as assessment tools they should be abandoned.

The last thing our schools need is more sound and fury, signifying nothing. Addiction to fashion and blindness to folly won’t build anyone’s confidence in public education.