BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Google Saves The News!

This article is more than 10 years old.

In 2007, two years into the launch of the Huffington Post, cofounder Jonah Peretti coined a term for news sites that disguise how little investment actually goes into most of their content: the mullet strategy. Named after a much-mocked hairstyle that's short and neat in front but long and unkempt in back, mullet strategists maintain a spiffy, well-groomed front page they can show to advertisers while serving most of their actual page views on a constellation of low-quality discussion boards, sexed-up celebrity news bits and user- or auto-generated content.

But the mullet could soon get clipped. A change in the way Google ranks news stories in its ubiquitous search system not only threatens to devalue the lucrative dark arts of increasing online readership--from search engine optimization to aggregation and content farming--but also promises a boost for the besieged newspaper industry, which Google helped to destroy in the first place.

That's not how Google sees it, of course. Matt Cutts, the engineer charged with reworking Google's search algorithm, articulates his mission differently. "Does the user end up happier when they land on the page of the result we show?" he asks. "That's the rough mental model we use."

There's arguably no more important person in the news business right now than Cutts. As the head of Google's antiwebspam team, he's rewriting the complex set of rules for ranking Web pages based on their relevance to a given query, which is the company's DNA, its single most important product and a trade secret more closely guarded than the recipe for Coke. In the past couple years the algorithm has been bedeviled by companies that use tricks to drive their pages to the top of Google's results. Sometimes these pages are straight-up spam--disguised bait for porn sites, for instance. Many others are the work of so-called content farms, which enlist armies of low-paid writers to crank out articles on the cheap and skim profits selling equally inexpensive ads. The result was clutter. "It didn't violate Google's guidelines, and yet it was kind of unsatisfactory, like fast food when what you want is a real meal," says Danny Sullivan, editor-in-chief of Search Engine Land, which chronicles the business.

The algorithm update is Google's answer to the problem. And it could turn out to be a whole lot more. In recent years CEO Eric Schmidt and others have repeatedly emphasized the need for a thriving press, both as a matter of public interest and of Google's self-interest. For Google to make money by connecting users with useful material, creating it needs to be profitable. "Google benefits when there's an ecosystem that produces quality content," says Krishna Bharat, distinguished research scientist and creator of Google News.

But the same migration of readers to the Internet that turned Google into a $185 billion (market cap) Goliath has been disastrous for the traditional media, especially for newspapers, whose reporting is still the primary engine driving the industry, from broadcast TV to talk radio to blogs. Collectively, U.S. newspapers have seen their advertising revenue fall by nearly 50% since 2005, to $25.8 billion last year. Digital revenues, while growing, aren't doing so at the rate required to replace losses from print, forcing most papers to cut staff and quality. Others have been driven out of business altogether, including major dailies in Seattle, Denver, Tucson and Honolulu.

There have, of course, been winners, digital pure plays unencumbered by the expenses of legacy publishers. They've succeeded by attracting large audiences while keeping editorial costs low, often by rereporting or commenting on the journalism produced at great cost by newspapers and other "old media" companies--"aggregating," in the parlance of the trade. A 2010 study by the Pew Research Center found that 80% of stories linked to in blogs came from just four sources: the BBC, CNN, the New York Times and the Washington Post . The Huffington Post, which recently sold itself to AOL for $315 million, is the most famous practitioner of this strategy, but there are plenty of others, from Gawker to Boing Boing to Mashable.

With its update, Google is essentially putting a thumb on the scale to favor traditional media, though Cutts says this isn't the intent. Cutts says only that the new algorithm directs searchers to sites that offer reporting, research and in-depth analysis; it discourages practices like "keyword-stuffing," a form of writing that renders articles highly visible to search engines but unreadable to the average English speaker. Having a lot of low-quality pages hanging off your site will also drag down your overall ranking--even for pages that feature the kind of content Google wants to promote. "It sort of pushes the Web toward ‘Be more useful,'" he says, echoing Google's unofficial motto, "Don't be evil."

Google News, which accounts for roughly a billion of the 4 billion clicks the company sends to news sites each month, takes this philosophy even further by paying close attention to how credible its users consider various news outlets relative to each other. "Do they seek a source out, or do they sidestep it?" says Bharat. His team is also experimenting with factoring into its formula whether a particular article "represents a substantial amount of effort" by its author or is just a rewrite job. "We want to be sure we drive traffic to sources who do original journalism," says Bharat. "We value originality."


Not everyone is a fan. "The big debate is how much should search results reflect the editorial investment in creating original scoops and original content, and how much should search results serve the consumer," says Peretti, who now runs Buzzfeed, a site dedicated to "viral" content.

The problem with traditional print journalism, he says, is it often comes in a form that's indigestible to the typical Web reader, who might be surfing between meetings at work or on his smartphone. He may not want to read the New Yorker's 24,000-word investigation into the Church of Scientology; he wants the 200-word summary with choice outtakes he can find on Gawker or Huffpo. "If they just looked at the data and said, ‘We want to give the best experience to the consumer,' that might not be sending them to the New Yorker piece," he says of Google. A spokesman says Huffington and AOL "have not seen a decline in traffic since the Google tweaks." Still, both Huffpo and Gawker are retooling to emphasize scoops over recaps. "Aggressive news-mongering trumps satirical blogging," Gawker owner Nick Denton declared in November.

For Google to change the rules in favor of publishers that devote more investment to each story is good news to an outfit like the New York Times, which spends an estimated $200 million per year on news gathering. "We applaud this effort they're making to improve the results," says Martin Nisenholtz, the Times' head of digital operations. He says there's early, anecdotal evidence that Times stories are ranking higher under the new algorithm.

It's even more salutary for local and regional papers, which typically depend on Google for 25% to 50% of their traffic, according to Ken Doctor, an analyst who studies the economics of publishing. (The Times gets less than 15% of its traffic from Google referrals, says Nisenholtz.) Those papers have been struggling to compete with startups like Examiner.com, a content-farm-like operation whose keyword search rankings have dropped by 79%, according to analytics firm Sistrix. Examiner CEO Rick Blair says the site's traffic fluctuates and the algorithm change "has enhanced the visibility of some of our best and most appealing content."

Welcome as the changes may be for newspapers, however, they won't do much for the papers' bottom lines, says Doctor. "It's of low marginal benefit, since they can't even monetize all the [advertising] inventory they have right now," he says. "A 5% increase in traffic might yield a 1% or 2% increase in digital revenue."

No one is disputing that Google--which offers a whole suite of tools to publishers to help them make money online, including its AdSense ad network and its new One Pass subscription system--is sincere about wanting to strengthen the news business. But there are less altruistic motives at work, too. Digital reading is increasingly shifting from PC-based browsers to mobile devices, including tablets, and to apps designed specifically for those devices. Nearly half of Americans now consume at least some of their news on a tablet or smartphone, according to the Pew Research Center. Compared with the promiscuous clutter of the open Web--where Google lives, for the most part--the news app is a highly curated experience. "If Google is facing Apple in an increasingly quality-oriented app world, not a browser world, then you have to do something to move up on the value scale," says Doctor, adding: "It seems to me that part of Google's strategic thinking is that it is becoming a more quality-oriented news world."

There's also the threat of social networks, especially Facebook, which now counts more than 50% of the U.S. population as members and accounts for a growing share of referrals to news sites. Google has been experimenting with ways of integrating social discovery into search, most recently with a "+1" option that lets searchers recommend content to friends in the manner of Facebook's "Like" button. "They're terrified of Facebook and, to some extent, Twitter," says Peretti. Like the app environment, social discovery tends to bring users face-to-face with higher-quality content. "People share things they're proud to have their friends think they like," says Peretti. "Sharing a New Yorker article or a story on the top ways of helping people in Japan makes you look like a good person in front of your friends, whereas on Google you search privately. People behave in less noble ways when they think they're anonymous."

Cutts, for his part, waves off talk of competitive concerns. "There's room for interesting social discovery, and there's room for search discovery," he says. All he and his fellow engineers are trying to do, he says, is make their users happy. And while they have many tools at their disposal for doing so, in the end the most valuable one may be their own judgment about what is and isn't worth reading. "We're trying to encode the intuition that a tech-savvy user would have," he says. "These algorithms aren't written by magic."

Special Offer: Free Trial Issue of Forbes