'A Disorderly Picture' of AI Summaries, Suggesting Google Acknowledges Issues
During its Google I/O conference a few weeks ago, Google announced their new AI-assisted search feature, promising users they could let Google "do the Googling for you." This new feature, known as AI Overviews, recently went live in the US on May 14th. However, it turned out to be a mess as AI suggested users put glue in pizza sauce, eat rocks, and use a "squat plug" during workouts (I won't elaborate on the last one).
While some of these examples might be photoshopped online, others were confirmed by the Aussiedlerbote team. They found that Google actually advised using Elmer's glue in pizza. If you search for these answers today, you're likely to see the message "No AI overview is available for this search."
This isn't the first time Google's AI searches have steered people in the wrong direction. When the beta version of AI Overviews, known as Search Generative Experience, was released in March, users noticed the AI was directing them to websites known for spreading malware and spam.
It appears that the AI isn't adept at distinguishing a sarcastic post from a genuine one. Since Google's AI seems to favor Reddit for answers, this isn't surprising. Reddit is home to a lot of humor, and Google's AI doesn't seem to have a sense of humor.
Some of the examples of AI screw-ups include users being advised to use glue in their pizza and use "squat plugs" while exercising (in case you were wondering, "squat plug" is an old joke on Reddit's exercise forums).
These incidents are minor, but they're just a few examples of the issues with AI Overviews. There's a problem with the AI mistakenly quoting articles from The Onion as facts instead of jokes. This makes sense if you consider the internet is full of jokes that would be terrible advice when taken seriously.
Google's AI search results do provide a source for most of their claims, but you need to click through to check its accuracy. This could be an issue if the AI's claims are the first thing you see in a Google search, which can be troublesome if taken at face value without verifying them.
Despite some of these AI screw-ups being worth a laugh, they can be dangerous in other situations. For instance, outdated advice on how to treat a rattlesnake bite or incorrect mushroom identification tips could have fatal consequences.
Google is attempting to deflect responsibility for these inaccuracies by adding a disclaimer at the end of AI Overviews: "Generative AI is experimental." It's unclear if this tactic will stand up in court if someone gets hurt due to an AI Overview suggestion.
There are many more examples of AI Overviews messing up, ranging from mistaking "Air Bud" for a real movie to labeling Barack Obama as a Muslim. Regardless, it's clear that the first search result you see on Google is more unreliable now than it was with just sponsored ads.
It's also worth noting that AI Overviews seem to be much less prominent in search results now than they once were. I tried searching for common advice and facts like "how to make banana pudding" and "who are the last three US presidents," and I didn't find any AI Overviews. Either Google is being cautious or they're changing their AI's behavior in response to recent backlash.
One popular way to fool AI is by clicking on Google's new web filter or appending "udm=14" to the search URL.
It seems something has changed up in Google's search game. In the top left corner for mobile users or the top right for desktop users, you'll now spot a beaker icon. Click on it, and get ready to explore the Search Labs page, which will boast a prominent card highlighting AI Overviews. If you don't see the beaker, sign up for Search Labs from the provided link. Once you're there, you'll find a demo for AI Overviews, neatly wrapped up with a lively "Try an example" button. This button unveils a series of low-pressure answers that exhibit the feature at its finest. Right below, sit three other "try" buttons. Two of these buttons are no longer sporting AI Overviews, instead taking you to a regular search results page, with the example prompts tagged onto your search bar. However, they're not shown answers from Gemini.
This raises concerns about Google's selection of AI Overview examples. If even Google is hesitant about its chosen samples, it's only logical to assume that they shouldn't be the initial highlights a user sees when seeking assistance from Google.
Critics of AI Overviews might assert that these AI-generated responses are just an extension of the knowledge panels Google uses, where they present media quotes without sending users to the initial page. But the spotlight on knowledge panels isn't entirely positive either.
Google's - Uhm - Lucky Day
On May 14, the same exact day AI Overviews made their way into our lives, Google Liaison Danny Sullivan showcased his unwavering support for the web filter - a new feature that emerged alongside AI Overviews but garnered significantly less applause. The web filter switches off both AI and knowledge panels, and is the secret sauce behind the trendy udm=14 hack. Turns out some folks prefer the classic ten blue links.
This sequence of events vaguely resembles a conversation from more than a decade ago, where Google significantly reduced the visibility of the "I'm feeling lucky" button. Before AI Overviews and knowledge panels became the norm, this feature served as a prototype. It took a leap of faith in the excellence of Google's first search result, sending users directly there rather than giving them the opportunity to view all the results.
The potential for a search to fall victim to malware or misinformation was present back then, yet the true reason "I'm Feeling Lucky" breathed its last breath was its inability to attract adequate user participation. Accounting for a scant 1% of searches, the button was losing Google precious advertising revenue by whisking clients away from the search results page before they even had the chance to glance at any adverts. (Unsurprisingly, you can still use "I'm Feeling Lucky," but only on desktop, and only if you navigate past your auto-completed search suggestions.)
Will AI Overviews share the same fate as "I'm Feeling Lucky" anytime soon? It’s unlikely, given the estimated $100 million Google has already invested in AI technology. But for now, it's safe to say AI Overviews are as inconspicuously placed on the site as Google's least-recognized feature. Not only does this hint that users aren't enthusiastic about these AI-generated suggestions, but it also implies they don't want Google to do the searches for them.
Read also:
The problems with Google's AI Overviews are increasingly evident, as Google themselves acknowledge. For instance, the AI suggested users use glue in pizza sauce and advocate for the use of "squat plugs" during workouts, causing amusement but also raising concerns. (tech, google ai overview problems)
Just days after Google launched AI Overviews in the US, some users reported encountering AI that directed them to malicious websites. This issue resembles the earlier problems with Google's Search Generative Experience, which also struggled to distinguish between sarcasm and facts. (tech, google ai overview problems)
