Clueless ‘science’
On television shows such as the “CSI” series, forensic science is high-tech, heroic and always right. The National Academy of Sciences released a long-anticipated report Wednesday on the real world of forensic science -- and the news is disturbing and downright ugly. Laboratories are woefully underfunded, and much of what passes for forensic “science” does not meet even minimal scientific standards. Yet, when forensic scientists testify in court, they often are embarrassingly overconfident about their findings.
The academy’s report, commissioned by Congress and the result of years of investigation by a distinguished panel of independent experts, does not mince words in its indictment of the scanty research foundation upon which much forensic science now stands. The report asserts that “the current situation” is “seriously wanting, both because of the limitations of the judicial system and because of the many problems faced by the forensic science community.” It also calls for the urgent creation of a federal agency devoted to encouraging research and to providing much-needed regulation and oversight.
Put simply, although many kinds of forensic testimony -- involving handwriting identification, fingerprint evidence and ballistics, for example -- are enormously persuasive to a typical jury, they do not meet the basic requirements of good science.
But we don’t need to wait for a new agency to make the necessary changes. Judges, who preside over the presentation of this evidence, need to exercise their role as gatekeepers to protect the integrity of our criminal justice system by requiring higher standards for forensic science when it’s used as legal evidence.
What did the study’s authors identify as some of the most significant problems?
Bias: Doctors testing a new medicine are -- appropriately -- not told which patients receive placebos and which get the test medication, because that knowledge might unconsciously bias their behavior and observations. But forensic scientists are frequently exposed to information that can potentially foster bias. Crime laboratories have failed to create adequate procedures for making testing “blind.”
Error rates: Most of the forensic disciplines lack good information about how often practitioners make mistakes, a basic requirement of any science. Not knowing the error rate is bad enough, but some experts consistently testify under oath that their technique has an error rate of zero, an inherently preposterous claim.
No one really knows just how often document examiners incorrectly analyze handwriting samples, how frequently arson investigators get the cause of a fire wrong or how often forensic odontologists misidentify bite marks. Yet anecdotal information and research suggest that errors are disturbingly frequent. University of Virginia law professor Brandon Garrett’s 2008 study of the first 200 convicted defendants exonerated by DNA evidence, for example, found that faulty forensic science testimony was second only to erroneous eyewitness identification as a cause of miscarriages of justice.
Over-claiming: Science deals in probabilities, not certainty. The only forensic science that makes regular use of formal probabilities is DNA profiling, in which experts testify to the probability of a match. None of the rest of the traditional pattern-identification sciences -- such as fingerprinting, ballistics, fiber and handwriting analysis -- currently has the necessary statistical foundation to establish accurate probabilities. Yet, instead of acknowledging their imperfect knowledge, fingerprint experts, for example, routinely testify that they can identify a specific person’s prints to the exclusion of all other people in the world with 100% certainty.
In 2004, the FBI, often said to have the nation’s best crime lab, wrongly identified Oregon attorney Brandon Mayfield as a terrorist based on an erroneous fingerprint match. It eventually admitted its error, and the government had to pay him $2 million. There are hundreds of less adequate labs across the country. How much confidence can we have in them?
Structural independence: Here in Los Angeles, the city crime laboratory is part of the Police Department, and the county’s lab reports to the sheriff. These kinds of arrangements are typical. But when the police and prosecutors pay and supervise the scientists, it stands to reason that the scientists may have difficulty establishing their independence.
The courts have almost entirely turned a deaf ear to these arguments, essentially giving forensic science and its practices a free pass, simply because they’ve been part of the judicial system for so long. Meanwhile, scandals continue to come to light across the nation involving error and even fraud in labs.
The findings in the National Academy of Sciences report should spur judges to require higher standards. At a bare minimum, judges should immediately prohibit experts from testifying to impossibilities such as “an error rate of zero” or asserting that they are capable of making 100% certain identifications.
In other cases, judges would be well advised to throw out forensic science altogether -- not forever, but until adequate research establishes, for example, that the conventional wisdom about evidence of arson is empirically valid, or until fingerprint and ballistics experts provide adequate proof that their real-world error rate is reasonably low. Courts should require forensic experts to back up their testimony with empirical evidence that they can do what they claim to be able to do.
We want and need forensic science in our legal system, but we have to be able to trust it. The forensic science community has been, at best, wary of, and often downright hostile to, serious inquiry into its strengths and limitations, especially by objective external researchers.
But if judges raised their standards and limited or excluded forensic evidence that didn’t meet them, that fortress mentality would inevitably change. This much-needed research would probably reveal that forensic science is not as perfect as its practitioners have often claimed. But when forensic science rests on an appropriate scientific foundation, it will be far more deserving of our confidence. Our system of justice demands no less.
