Google just updated its algorithm. The Internet will never be the same (2024)

If you've ever typed "air purifier reviews" into Google, you were probably looking for the kind of content you'll find on HouseFresh.com. The site was started in 2020 by Gisele Navarro and her husband, based on a decade of experience writing about indoor air quality products. They filled their basem*nt with purifiers, running rigorous science-based tests and writing articles to help consumers sort through marketing hype.

HouseFresh is an example of what has been a flourishing industry of independent publishers producing exactly the sort of original content Google says it wants to promote. And indeed, soon after the website's launch, the tech giant started showing HouseFresh at the top of search results. The website grew into a thriving business with 15 full-time employees. Navarro had big plans for the future.

Then, in September 2023, Google made one in a series of major updates to the algorithm that runs its search engine.

"It decimated us," Navarro says. "Suddenly the search terms that used to bring up HouseFresh were sending people to big lifestyle magazines that clearly don't even test the products. The articles are full of information that I know is wrong."

The second Google algorithm update came in March, and it was even more punishing. HouseFresh's thousands of daily visitors dwindled to just hundreds. "We just got absolutely crushed," Navarro says. Over the last few weeks, HouseFresh had to lay off most of its team. If nothing changes, she says, the website is doomed.

A spokesperson for Google tells the BBC that the company only launches changes to Search after rigorous testing confirms that the shift will be helpful for users, and that the company gives website owners help, resources and opportunities for feedback on their Search rankings.

Google stands firm in its position that the changes will be a benefit to the web, and changes to the Search algorithm are just the start. Last week, Google CEO Sundar Pichai stood in front of a crowd at the company's annual developer conference and announced one of the most significant moves in the search engine's history. Going forward, Pichai said, Google Search would provide its own AI-generated answers to many of your questions, a feature called "AI Overviews" that's already rolled out to users in the United States. "The result is a product that does the work for you," Pichai said. "Google Search is generative AI at the scale of human curiosity."

AI Overviews are just one of a slew of dramatic changes Google has made to its core product over the past two years. The company says its recent effort to revamp Search will usher in an exciting new era of technology and help solve many of the issues plaguing the web. But critics say the opposite may be true. As Google retools its algorithms and uses AI to transition from a search engine to a search and answer engine, some worry the result could be no less than an extinction-level event for the businesses that make much of your favourite content.

One thing is certain: Google's work is about to have a profound impact on what many of us see when we go online.

Over the last two years, updates meant to make Search more “helpful” devastated many website owners who say they follow Google’s best practices. (Source: Semrush) (Credit: BBC)

The changes came about because Google recognises the web has a problem. You've seen it yourself, if you've ever used a search engine. The Internet is dominated by a school of website building known as "search engine optimisation", or SEO, techniques that are meant to tune articles and web pages for better recognition from Google Search. Google even provides SEO tips, tools and advice for website owners. For millions of businesses that rely on the mechanisations of the Search machine, SEO can be an unavoidable game.

The trouble is SEO can be abused. Enterprising website owners realised you can sometimes make more money by making content designed to please Google's algorithms, rather than the human beings it's ostensibly designed to serve.

Google's efforts to address this issue aren't always successful. If you've ever been frustrated by what comes up when you search for something like "Best Sneakers for Women", you know the issue. Often, the results for popular search terms are crowded with websites that contain very little useful information, but tonnes of ads and links to retailers that earn publishers a share of profits. What's often lost is what you're probably looking for when you open up Google: information from people who are knowledgeable and passionate about their topic.

Google's war on spammy Search results has ramped up. In 2022, the company issued a "Helpful Content Update" to its algorithm meant to weed out content created solely for the purpose of ranking higher on Search. Google issued a subsequent update in September, 2023, and a third algorithm tweak in March of this year. Google says the result is "45% less low-quality, unoriginal content in search results". It could be viewed as a wild success.

"Our recent updates aim to connect people with content that is helpful, satisfying and original, from a diverse range of sites across the web," a Google spokesperson tells the BBC. "As we work to improve Search, we're continuing to focus on sending valuable traffic to sites and supporting a healthy, open web."

For better or worse, Google Search dictates the shape of the web. Some online publishers say the company wields that power irresponsibly (Credit: Serenity Strull/BBC/Getty Images)

But the updates had some surprising consequences as well. For example, data from the analytics tool Semrush suggests that the website for New York Magazine lost 32% of its Google Search traffic in the past six months, while GQ.com shrank 26%. The data indicates Urban Dictionary, a wildly popular crowdsourced dictionary of English language slang, dropped some 18 million page views, amounting to more than half its Search traffic. OprahDaily.com was down nearly 58%. (Semrush is an industry-standard tool, but its numbers are third-party estimates that may differ from a website’s internal measurements. Semrush data only measures traffic coming from Google Search, specifically.)

A New York Magazine spokesperson said these findings were incomplete and didn't reflect the company's internal analysis. Representatives for GQ, Oprah Daily and Urban Dictionary did not respond to requests for comment by the time this article was published. However, experts and more than half a dozen media executives and websites owners told the BBC the broad trends in the data are all too real.

In place of these sites, there's one platform you'll be seeing much, much more of: Reddit. According to Semrush, Reddit saw a surge that amounted to a 126% growth in traffic from Google Search. The company is already feeling the benefit. Reddit just announced its first quarterly earnings since becoming a publicly traded company in March 2024. Its revenue totals $243m (£191m), up an eye-watering 48% from the year prior.

"The increase in traffic Reddit is seeing is unprecedented on the Internet," says Lily Ray, vice president of SEO strategy and research at the marketing agency Amsive, and a celebrity in the world of SEO. "Cooking content, adult content, video games, gardening, fashion, everything is all just Reddit."

A representative for Reddit declined to comment.

Reddit isn't the only winner after Google's recent algorithm updates. Semrush data shows that other user-generated sites such as Quora and Instagram saw similarly astronomical rises, and there were impressive spikes at LinkedIn and Wikipedia as well. In one sense, Google was just following a trend. Over the past few years, swaths of savvy internet users started adding the word "Reddit" to the end of their web searches in the hopes it would bring up people sharing their honest opinions, as opposed to websites trying to game Google's system. It's something Google's public liaison for search, Danny Sullivan, has noted.

"We've found that people often want to learn from others' experiences, and so we surface content from hundreds of forums and other communities across the web," a Google spokesperson says. "Our agreement with Reddit absolutely did not include ranking its content higher on Search."

But Google results are a zero-sum game. If the search engine sends traffic to one site, it has to take it from another, and the effects on the losers in this Reddit equation are just as dramatic. "Google's just committing war on publisher websites," Ray says. "It's almost as if Google designed an algorithm update to specifically go after small bloggers. I've talked to so many people who've just had everything wiped out," she says.

A number of website owners and search experts who spoke to the BBC said there's been a general shift in Google results towards websites with big established brands, and away from small and independent sites, that seems totally disconnected from the quality of the content.

The change was instant for Daniel Hart, editor-in-chief of the UK-based entertainment news site Ready Steady Cut. "After Google's September update, our traffic halved immediately, and it's only gotten worse. We've just been blitzed by the Reddit stuff in particular, but we're also being replaced by spam websites that are stealing our content. It makes no sense," Hart says. In the months following, the lost income forced Ready Steady Cut to reduce its team of 20 writers and editors down to just four, Hart says.

A Google spokesperson said the company's recent updates have dealt a major blow to spammy, unoriginal content, and Google keeps a close watch on evolving abusive practices that lead to low-quality information in Search.

After Google's updates, the company provided tips to website owners and said there was a path to recovery. Hart says the site hired consultants and pivoted to focus on Google's recommendations, spending sleepless nights working to update the site. After almost a year, none of it helped. "I wasted the last eight months of my life trying to follow Google's advice." he says. "Google claims they want content from people with first-hand experience and helpful context, and we're a massive example of that. It's just heartbreaking."

A few examples show how recent changes to Google Search had startling effects on a wide variety of websites. (Source: Semrush) (Credit: BBC)

(Note: Matthew Keys, publisher of the tech and media news site The Desk, tells the BBC his website "suffered from tremendous traffic loss" due to Google’s updates, but the drop was closer to 75-80%, not the 98.9% estimate reported by Semrush.)

But the biggest offence, according to the website owners and content creators who spoke to the BBC, is the AI generated responses.

Google argues its AI overviews in search results will be a boon to websites. Liz Reid, Google's head of search, wrote in a blog post that the company's AI search results actually increase traffic that Google sends to websites. "AI Overviews get more clicks than if the page had appeared as a traditional web listing for that query," Reid wrote. "As we expand this experience, we’ll continue to focus on sending valuable traffic to publishers and creators." However, the company hasn't shared any of the data backing up that claim, and many website owners and industry experts worry the opposite effect is just as likely.

Katie Berry, owner of the cleaning advice website Housewife How-Tos, assumes users will just end their searches if Google's AI answers questions for them. The AI search results "answer questions superficially, and often incorrectly, so people don't visit my site," Berry says. According to Berry, her site's traffic fell 70% after the 2022 Google update and dropped even further after Google started testing its new AI. "My site had more traffic in its first months of existence than it gets now, even though my rankings have not changed all that much," she says.

Others, such as travel writer David Leiter, say the potential consequences are especially problematic because they feel Google's AI is outright stealing their content.

For example, Leiter says a search for "Best Slot Canyons Near Las Vegas" used to bring up an article on his website, World Travel Guy. However, a search earlier this week brought up an AI-generated response at the top of the page instead.

"Google has replaced my article with the giant AI Overview box, and it spits out an answer that is mostly wrong," Leiter says. "The first four places it lists are not even slot canyons. A slot canyon is a specific type of canyon with a narrow passageway, but the AI doesn't understand that. It's just listing random canyons and even a walking trail instead." The AI Overview did include a link to Leiter's article, but only if you took the time to click a tiny arrow at the bottom of the result. Leiter says he doesn't believe he'll get more traffic than he used to as the top search result. In either case, it's a small consolation. Leiter says Google's recent algorithm updates erased 95% of his traffic.

Google acknowledges that AI tools may provide inaccurate information, but says it's constantly working to improve results. A Google spokesperson says AI Overviews are generally taken from multiple webpages, not single sources, and the responses are designed to highlight relevant links. The spokesperson says publishers can use a special tag on their webpages to control whether or not AI Overviews includes a link to their sites. However, once an AI model scrapes your content, it may be impossible to remove that data.

Media executives aren't the only ones questioning Google's control of the internet. Google is simultaneously battling numerous antitrust lawsuits against different parts of the company's sprawling £1.7tn ($2.2tn) business. The company is currently awaiting a decision in a lawsuit brought by the US Department of Justice accusing Google of running an illegal monopoly in the search engine industry. If the tech giant loses the case, the penalties could range from massive fines to a forced break-up of the company.

Google, which controls more than 90% of the worldwide search business, argues that the company's success stems solely from the fact that it makes superior products. A Google spokesperson says the company faces "immense competition" and people have many choices about how they search for information online.

"I understand that Google doesn't owe us or anyone else traffic," says Navarro, of HouseFresh. "But Google controls the roads. If tomorrow they decide the roads won't go to an entire town, that town dies. It's too much power to just shrug and say, 'Oh well, it's just the free market,'" she says.

"I might just try working in the offline world, just pack it all up and tend a shop somewhere," says Navarro. "Maybe it was naive to think we could succeed just by making great original content that people want to read."

* This article was updated on Monday 27 May to add further information on The Desk's traffic drop.

--

For timely, trusted tech news from global correspondents to your inbox, sign up to the Tech Decoded newsletter, while The Essential List delivers a handpicked selection of features and insights twice a week.