What sentiment analysis can and cannot tell us about the GOP race

With all the interest in the analysis of social sentiment – feelings expressed in the ever-expanding public arena online – how much can we really know about what people are feeling? Especially if the work is being done by computer algorithm, rather than human interpretation?

As more news outlets incorporate sentiment analysis into coverage of politics, its accuracy and usefulness has been met with skepticism from some who study linguistics and social media. In other cases, the criticism has been more biting. Micah L. Sifry, the creator of the website Techpresident, described a new Facebook/Politico analysis as “total bunk” in a post this week.

The Times is taking a look at the work being done by San Francisco-based Kanjoya. The company is using an algorithm to assess emotional language that was derived from the Experience Project, which has collected more than 14 million personal stories, with their associated emotions, since launching in 2007.

MOOD METER: Track social sentiment about the candidates

Kanjoya has pointed that algorithm at tweets about the four remaining GOP hopefuls, Newt Gingrich, Ron Paul, Mitt Romney and Rick Santorum. They chose to assess joy, surprise, anger, sadness and fear, which they described as among the most interesting of the more than 120 emotions they track.

“The way that we see it is more and more of the conversation and how people are expressing themselves is on Twitter, “ said Moritz Sudhof, an engineer with Kanjoya. “It’s an incredibly valuable source of what millions of people are doing and there’s no way a human can go and read all of what people are saying, We provide that in a human consumable format. We know there’s a lot of value in how people are reacting to what’s going on.”

But there are many things to keep in mind, said Moritz and company founder Armen Berjikly who concede that language is difficult to interpret correctly. They said their research has found people agree about the intent of statements about 80% of the time.

Roughly 40% of the tweets they are capturing qualify as emotional in their algorithm. And the emotions classified as “anger” or “joy” may not be what they seem at first. For example, there could be a substantial amount of “joy” around a candidate that could express happiness at their decline, rather any intent to support them at the polls.

Here’s a look at tweets Kanjoya determined did not show emotion:

* Video: Gingrich vs. Juan Williams on the “food-stamp president” https://t.co/GIjFNrXL #tcot #sgp

* Gingrich, Rivals Hit Romney Hard in GOP Debate https://t.co/UUoSVZjq

* Gingrich: ‘I Would Be Delighted’ With a Santorum Endorsement https://t.co/KC1b3PUB

* Ron Paul’s New Hampshire showing tells us something important https://t.co/el4NCdFC

And some tweets to which Kanjoya assigned emotions:

* “Joy”: RON PAUL IS TRENDING! IT PROVES A MESSAGE OF PEACE AND LIBERTY IS POPULAR! #PresidentPaul #RonPaul2012 #SCprimary

* “Anger”: Facts: #Romney got filthy rich by killing corps & destroying jobs. He pays less in taxes than YOU do. As POTUS he’ll help the 1%. #p2 #tcot

* “Sadness”: And Gingrich’s “Reagan Conservative” line is getting stale. He’s more like a Dede Scozzafava conservative #tcotfear:

* “Fear:” With politicians like these, who needs terrorists? #RonPaul

* “Surprise:” WOW! Just in 2005 alone, #Santorum got $483 million in earmarks for Pennsylvania for 872 projects https://t.co/CkSv6zIT #sctweets #teaparty

The Times will be reporting more on the sentiment being analyzed by Kanjoya – looking at how the company is interpreting Twitter reaction to Gingrich’s surge in the polls, among other developments.

MORE MOOD METER REPORTS:

Gingrich speaks and “surprise” about his victory grows on Twitter

South Carolina votes; Newt Gingrich’s win fuels emotional tweets