Color codes are bad for measuring terror threats, experts tell Homeland Security

Five years ago, the U.S. Department of Homeland Security dropped its color-coded terror threat index developed after the 9/11 attacks amid widespread confusion and ridicule. So what did it do when tasked by Secretary Jeh Johnson in 2014 to measure security along the country’s borders?

Agency staff proposed another system of reds, yellows and greens.

The Institute for Defense Analyses, a consulting firm, was hired by the agency to review the idea and found the index simplistic and misleading, noting that colors were a “disaster” for communicating terror threats.

“DHS should learn from its own history and avoid repeating this error,” the consultants said in its 53-page report.

Homeland Security’s proposal was never made public, nor was the consulting firm’s $90,000 review. A copy of the report was obtained by the Associated Press, and when AP asked the agency whether it would move ahead with the index, spokeswoman Gillian Christensen said: “Currently, there are no color-coded border security indexes or metrics being considered by the Department of Homeland Security.”

The chairman of the Senate Homeland Security and Governmental Affairs Committee criticized the color codes Friday while also emphasizing a need for better measurements.

“DHS spent $90,000 on a question we already know the answer to,” said Sen. Ron Johnson, a Wisconsin Republican who has introduced a bill for border metrics. “Measuring the security across our borders is complex and requires sophisticated and consistent metrics — not a series of colors.”

It was one of the latest attempts by the government to come up with a way to measure border security and help the public understand whether the billions of dollars devoted to it each year are being spent wisely.

In 2010, Homeland Security ended a five-year experiment of measuring miles under “operational control,” in which the Border Patrol was likely to capture illegal crossers. It reported that only about 40% of the borders were controlled in 2010, providing ammunition to those who argue the border is porous.

Then in 2013, Johnson’s predecessor, Janet Napolitano, abandoned plans for what was called the Border Conditions Index, which would have relied on various economic, crime and law enforcement data.

See more of our top stories on Facebook >>

John Sandweg, who was senior counselor to Napolitano, said there was internal consensus about what made up the index but not about how much weight to give each factor.

Sandweg, who advised the secretary when the color terror-alert system was dropped, said he doesn’t see the value of a similar color-coded approach to the border. “It seems to me like an oversimplification of a very complex problem,” he said.

The consultants hired by Johnson agreed.

“The red/yellow/green formulation, while intuitively attractive and easy to understand, will open the Department to charges that it is manipulating a complex problem in an effort to be seen as responding to public concerns,” the report said.

When the report was completed in June, California and New Mexico/West Texas were green (low risk) during the previous quarter, Arizona was yellow (medium risk), and South Texas was red (high risk). The consultants said that reality was more nuanced.

“A new set of metrics should work against this simplistic perception rather than reinforcing it. Instead, the new index does the opposite, by reporting the level of border security in just three large baskets, two of which (red and yellow) are likely to be seen by the public as evidence of a border ‘not controlled,’” the report said.

The consultants identified other problems. A color index might lead reporters with an appetite for eye-catching headlines to produce misleading stories of an out-of-control border. And Homeland Security relations with Congress could be further strained, with administration officials constantly having to defend their color choices.

For example, a West Texas congressman would demand to know why his district is rated low risk when voters tell him the opposite. A South Texas congressman would want lots more money if the administration acknowledges his district is high risk, the report said.

Now, the most closely watched public indicator is Border Patrol apprehensions, released annually. The number fell to a 44-year low last year, a figure cited by those who argue the border is relatively secure. But there is broad agreement that the apprehension tally gives an incomplete picture, just as a police department’s arrest count doesn’t fully reflect how safe a city is.

The color-coded index would have relied on 12 indicators for land borders and seven for maritime borders, each one weighted under a formula that produces reds, yellows and greens. Those indicators ranged from the number of border crossers with known or suspected terrorist ties to marijuana seizures.

The consultants’ report speaks favorably about developing a “dashboard” of key numbers, as many police departments do. Homeland Security recently began publishing the percentage of illegal border crossers it says are caught or turned back — 80% in the 2015 fiscal year — and has developed other measures, including the percentage caught who are repeat offenders.

Rep. Martha McSally, an Arizona Republican on the House Homeland Security Committee, said Friday that current measures fall short and that a threat assessment is a key first step toward building trust.

ALSO

Assistant sheriff is stepping down after overseeing major changes in L.A. County jails

In Owens Valley, they’re skeptical of Angelenos bearing gifts, including new artwork

Kamala Harris in driver’s seat at California’s first U.S. Senate debate