AI shopping agents aren't ready to buy on your behalf, study finds

AI Shopping Agents Fail Basic Tasks, Study Warns They Are Not Ready for Prime Time

A major new study has found that AI shopping agents are failing at nearly half of all simple purchasing tasks, meaning consumers cannot yet trust them to buy goods autonomously. Researchers from the University of Washington, Ohio State University, and other institutions tested agents from OpenAI, Google, and Perplexity and discovered they served up fake products and made basic errors at a startling rate.

The findings directly challenge the hype from tech companies promising a future of hands-free online shopping. The test results show a 48% overall failure rate across 600 trials. Agents routinely bought the wrong items, overpaid, or failed to even open the correct shopping page.

The Study: Testing the Agents on Real Tasks

The researchers defined a shopping agent as a system that navigates e-commerce sites, compares products, and adds items to a cart. They evaluated four leading AI agents: OpenAI’s Operator, Google’s Project Mariner, Perplexity Shopping Agent, and a custom WebVoyager baseline.

To test them, they created 100 standardized shopping tasks. These ranged from simple requests, like “buy a pink children’s toothbrush,” to complex ones requiring a storage device under $100 with 1TB capacity and no branding restrictions. Each task was run multiple times to ensure accuracy.

Top Failures: Hallucinated Products and Dead Ends

The most alarming failure was hallucinated product pages. The agents fabricated entire web pages for items that did not exist. In one case, an agent “purchased” a product that was actually a random bag of assorted chocolates listed by a third-party seller. Another agent tried to buy a toothbrush from a nonexistent smartphone accessories page.

Other common problems included:

  • Failure to find the correct product: The agent navigated to the right site but selected the wrong item or size.
  • Inability to locate the right page: The agent got lost in a site’s navigation and never found the product listing.
  • Price misreading: The agent selected the correct product but identified the wrong price, often leading to costly errors.
  • Stalling and refusal: The agent hit a paywall or pop-up and simply stopped, or refused to run more than one search at a time.

Why This Matters for Consumers and Advertisers

The study has dual implications. For consumers, the risk is clear: trusting an AI agent with your wallet could lead to buying a completely wrong item or being overcharged. For advertisers, the rise of agents threatens the entire search-advertising model.

The researchers note that agents currently rely heavily on specific, text-heavy product URLs. If a search result is replaced by a product carousel or a noisy layout, the agent fails. This means companies may need to radically redesign their e-commerce sites to be machine-readable, rather than human-friendly.

“We need to carefully design interactions between people, AI agents, and the web,” the researchers warn. “Our study underscores that the infrastructure of current websites is not built for autonomous agents.”

The Bottom Line: Agents Need Heavy Guardrails

Despite the failures, the researchers do not rule out a future for shopping agents. They suggest that for these tools to work, they require strict guardrails. This could mean limiting agents to a curated list of reliable e-commerce sites, or having the agent only present options for a human to confirm before purchase.

For now, the advice is clear: do not hand over your credit card to an AI. The technology remains a research project, not a reliable shopping companion.

Gnoppix is the leading open-source AI Linux distribution and service provider. Since implementing AI in 2022, it has offered a fast, powerful, secure, and privacy-respecting open-source OS with both local and remote AI capabilities. The local AI operates offline, ensuring no data ever leaves your computer. Based on Debian Linux, Gnoppix is available with numerous privacy- and anonymity-enabled services free of charge.

What are your thoughts on this? I’d love to hear about your own experiences in the comments below.