The Rise of Agentic Commerce and Shopify Catalogs
The mechanics of online retail are undergoing a quiet transformation. When customers use digital agents to find products, the path from search query to checkout no longer relies on traditional web browsing habits. Shopify's introduction of Shop MCP changed the architecture of automated retail by giving AI agents direct access to over a billion products without requiring an API key. While this open connection lets agents build carts and complete transactions seamlessly, it introduces a critical decision point that store owners must understand: a product list is not a purchase.
When an agent retrieves search results from a massive product feed, something must select a single item to buy. Because modern shopping agents run on language models, the underlying model itself often makes that ultimate choice. A recent test involving three distinct Claude models—Haiku 4.5, Sonnet 5, and Opus 5—fed identical product lists into each model to observe how they choose. The results demonstrate that the winning product depends heavily on which language model powers the consumer's agent, exposing hidden preferences that digital marketers cannot afford to ignore.
Why Different AI Models Never Agree on a Winner
Running identical product searches through the system revealed striking inconsistencies between models. Across multiple product categories, the three tested versions of Claude rarely agreed on the top choice. While individual models remained internally consistent—tending to pick the same item repeatedly during fifty separate runs—they frequently contradicted each other. For instance, when searching for water bottles, Haiku and Sonnet favored one brand, whereas Opus consistently selected another. Furthermore, the catalog's native ranking mattered far less than anticipated, with models frequently bypassing the top-ranked item entirely.
This divergence occurs because every language model approaches ambiguity differently. When given no specific consumer history, budget, or complex prompt constraints, models rely on their internal heuristics to resolve uncertainty. For store owners tracking organic visibility, this reality complicates optimization efforts. Ranking high in a catalog feed guarantees visibility to the agent, but it offers no assurance that the model will select the item for purchase.
The Heavy Reliance on Review Counts and Phantom Metrics
A closer inspection of model reasoning reveals an overwhelming preference for high review counts. In searches for backpacks, coffee beans, and water bottles, Haiku and Sonnet repeatedly selected products boasting the highest volume of reviews, explicitly citing those figures in their decision rationale. However, this reliance introduces unpredictable anomalies when feed data diverges from reality.
In the water bottle test, the catalog displayed an astronomical review count that did not match the brand's live storefront page. Without any means to verify the accuracy of the number, the models accepted the catalog's inflated figure at face value, using it as the primary justification for their selection. Meanwhile, Opus ignored the massive review count entirely, opting instead for a different brand that it deemed a safer default choice. This dynamic suggests that optimizing for generative search requires extreme diligence regarding the hygiene and accuracy of product feeds fed into agentic channels.
The Hidden Influence of List Position and Safety Biases
Beyond raw review counts, positioning within the returned list plays a surprisingly powerful role. When researchers moved the catalog's top-ranked product to the very bottom of a twenty-item list, all three models experienced a sharp drop in selection frequency. Interestingly, none of the models mentioned list position in their written reasoning, demonstrating a silent positional bias that mirrors traditional search engine tendencies.
Opus exhibited a distinct psychological pattern, labeling its choices as 'safe' in a vast majority of its runs. Given minimal instructions, Opus constructed a hypothetical average consumer, favoring general-purpose items over niche variants. This tendency highlights how AI architectures default toward mainstream options when specific constraints are missing.
Action Plan for E-Commerce Brands
As agentic commerce expands, merchants must adapt their optimization strategies beyond standard keyword targeting. Ensuring feed integrity is paramount, as models directly parrot catalog summaries and review figures. Store owners should audit their product feeds regularly to eliminate phantom metrics or contradictory review counts. Additionally, providing explicit product attributes, such as precise use cases, budget categories, and compatibility details, helps steer models away from generic defaults and toward specific product advantages.