New Search Engine Calls Up Resentment at Yahoo

Jackie Jahosky’s online lighting shop is just the kind of business Yahoo Inc. expected to help with its new search engine. Instead, the Internet giant has managed to rankle the Long Island entrepreneur, and plenty of others like her.

Yahoo this week rolled out a plan to keep its search engine up to date by charging commercial websites a fee to index them more frequently. The Sunnyvale, Calif., company said the approach, known as paid inclusion, would produce more accurate search engine results and boost its revenue.

Some webmasters and small-business owners are complaining bitterly. They say Yahoo has a financial incentive to boost paying customers in search results and they fear they’ll be forced to pony up if they want to stay even with deep-pocketed competitors.

“It’s going to be totally out of reach for me to pay for this program,” said Jahosky, who runs Specialty-Lights.com out of her home in Ronkonkoma, N.Y.

To participate, she would have to pay Yahoo $49 a year, plus 30 cents each time someone reached her website through the search portal. When she told a Yahoo executive speaking at a search engine conference in New York, “I just don’t like it,” the audience responded with applause, according to attendees.

Yahoo insists the new program won’t affect the integrity of its Internet search results. To the contrary, the company says its search engine will become more relevant because it will include more websites and update them more often.

“We wouldn’t be going out and causing a rumpus if we didn’t think this was creating a much better product,” said Tim Cadogan, Yahoo’s vice president of search. “All we’re doing is presenting the most relevant results.”

Paid inclusion is a cornerstone of Yahoo’s strategy as it battles archrival Google Inc., the leading search provider, over how users navigate the Internet.

Analysts say Yahoo, which dumped Google’s search technology last month in place of its own, is showing an unapologetic willingness to be known as a commercially oriented search engine. Meanwhile, Google is spinning itself as the keeper of a pure search engine untainted by advertising dollars.

Both Yahoo and Google sell clearly marked ads alongside regular search results. But Larry Page, co-founder and president of closely held Google, said users probably wouldn’t realize that some websites may have bought their way into the index through paid inclusion. After all, he said, if it didn’t boost a website’s ranking, then why would a webmaster pay for it?

“Webmasters aren’t stupid,” Page said. “It’s either going to be a commercial failure, or it’s going to affect search results.”

Internet advertising is big business -- firms are expected to buy $3.9 billion worth of online ads this year, growing to $8.9 billion by 2007, according to investment bank Piper Jaffray. Of Yahoo’s $1.63 billion in sales last year, $1.2 billion came from “marketing services,” which include all components of its ad business. The firm had net income of $237.9 million last year.

Search companies use programs that “crawl” the Internet for websites to pull into their index. Then they use complex algorithms to select the results they think are the most relevant to a user’s query.

But Yahoo’s paid inclusion program, which it calls Site Match, lets companies pay to feed updated data into its index every 48 hours instead of waiting weeks to be crawled. Yahoo collects up to 30 cents each time someone clicks through to a website that paid for inclusion. To consumers, the sites that pay for the service are indistinguishable from the ones that don’t.

Yahoo’s Cadogan said paid inclusion would help searchers find the most recent information on the Internet. For example, Yahoo’s search engine will bring up auctions that are in progress, rather than auctions that ended weeks ago.

But in online message boards and Web logs, Web developers expressed concern about Yahoo’s apparent conflict of interest.

“Everybody’s saying, ‘What is holding back Yahoo from giving these sites a boost?,’ because there’s a financial benefit to them sending traffic to these websites,” said Pierre Zarokian, president of Submit Express Inc., a Glendale firm that helps companies sell through search engines.

LookSmart and other search engines began letting paid inclusion influence their results several years ago, drawing complaints from consumer groups. In June 2002, the Federal Trade Commission told search providers that paid inclusion could aid consumers by improving results, but warned the firms to mark results boosted by payments.

“As long as they’re doing what they say they’re doing,” Yahoo won’t run afoul of the agency’s guidelines, said Heather Hippsley, assistant director of the FTC’s advertising practices division.

If that changes, Yahoo will face not only FTC scrutiny, but it also will lose users, said Safa Rashtchy, an analyst with Piper Jaffray. But if done right, he said, paid inclusion could make Yahoo’s search results better and bring in $100 million a year for the company.

“I think it makes sense for Yahoo to take this approach,” Rashtchy said. “Paid inclusion, if done properly -- and that’s a big if, because some didn’t do it properly -- will actually improve the relevancy of search results.”

On Tuesday, after Yahoo introduced Site Match, Ask Jeeves Inc. said it would discontinue its paid inclusion program that included a click-through fee. The Emeryville, Calif.-based search company found that mixing the direct data streams from paying companies with its index of crawled websites skewed the results -- sometimes ranking websites higher than they should have been, other times lower.

“It’s very difficult for any search engine, not just ours, to rank apples and oranges using the same scoring mechanism,” said Jim Lanzone, Ask Jeeves’ vice president of product management.

The more companies that joined the program, the more trouble Ask Jeeves had returning fair rankings, he said.

Paid inclusion “doesn’t work,” Lanzone said. But he added: “We’re not saying Yahoo is evil.”

Yahoo said that it had avoided Ask Jeeves’ technical problems and that it felt confident paid inclusion helped consumers and businesses connect.

As for Jahosky’s complaint that small firms will find it harder to stand out among big businesses, that just means search engines are growing to resemble more traditional forms of media, said John Battelle, who co-founded Wired magazine and now runs the site Searchblog.

“At the end of the day,” he said, “those with more money to spend on getting their message out get their message out louder.”