All You Need To Know About Cloudflare’s Agent Readiness Score via @sejournal, @slobodanmanic

Agent-readiness crossed from concept to measurable infrastructure this week. On April 17, as Cloudflare Agents Week extended into its sixth day, the company shipped isitagentready.com, a public scanner that scores any website on how prepared it is for AI agents. Paste a URL, get a score, see which checks passed and which failed, read AI-generated guidance on how to improve. For the first time, the agent-legibility conversation moved from “is my website ready for agents” as a gut feeling to “my website scored X out of 100 in these five categories, here are the failing signals.”

The Agent Readiness Score is a real shift. It is also a structurally misleading tool if you stop reading after the composite number.

I ran the scan on this website (nohacks.co) and scored 33 out of 100, Level 2 “Bot-Aware.” The robots.txt passed. The sitemap passed. The AI bot rules in robots.txt passed. Content Signals passed. Then the score collapsed across categories where a content-only blog genuinely doesn’t need what the scanner checks for. More on that in a minute.

First, the context. Cloudflare has been shipping agent-facing infrastructure all week. The Agent Readiness Score arrived alongside Agent Memory, Shared Dictionaries, Redirects for AI Training, an LLM compression technique called Unweight, and a feature-flag tool called Flagship built for AI-generated code. Four days earlier, they shipped Project Think (a new Agents SDK), and OpenAI matched it within hours with their own Agents SDK. I wrote about that in The Agent Runtime Wars Started This Week. The readiness scanner is the logical next piece: If runtimes are the new browser layer, website owners need a way to test whether their website is legible to that layer. Cloudflare shipped the tester.

The question this article answers is narrower: What does the scanner actually check, what should you do with your score, and where is the scoring structurally misleading enough that the number by itself leads you astray?

What Cloudflare Shipped: Scanner, API, And An MCP Endpoint Agents Can Call On You

The scanner is at isitagentready.com. Paste any URL, pick a website type (All Checks, Content Site, or API/Application) to scope which signals get scanned, hit Scan. The scanner fetches the homepage and a handful of well-known paths, runs a set of checks against each, and returns a scored report with pass/fail markers, status codes, response bodies, and AI-generated guidance on what to fix.

The scanner is also available in three other ways:

  • Integrated into Cloudflare Radar, so the same checks run alongside Radar’s existing URL analysis.
  • Exposed programmatically via the Cloudflare URL Scanner API for automation.
  • Available as a stateless MCP server at /.well-known/mcp.json, so any MCP-compatible agent can call the scan as a tool and reason over the result

That last one is worth sitting with for a moment. Cloudflare shipped an agent-readiness scanner that agents themselves can call to audit websites before deciding how to interact with them. The scanner checks whether your website is ready for agents, and any agent can invoke it to decide how to interact with you before arriving. The measurement and the measured are starting to share the same surface.

Back to the practical question. What exactly does it check?

16 Checks, 5 Categories: What The Scanner Actually Tests

The scanner groups its checks into five categories. Here is what each one looks for, grouped by what the check actually means in practice.

Discoverability (3 Checks)

Whether the website publishes the basic metadata an agent needs to find what is where.

  • robots.txt exists. The classic crawl-policy file. An agent that follows robots.txt needs it to exist and parse.
  • sitemap.xml exists. Either declared via a Sitemap directive in robots.txt or available at the standard path. An agent that wants to enumerate pages uses the sitemap.
  • Link headers (RFC 8288). HTTP Link headers pointing to canonical, alternate, or related resources. Useful for agents that parse responses rather than HTML.

Content (1 Check)

  • Markdown for Agents. Content negotiation. The scanner sends Accept: text/markdown and checks whether the website returns Markdown instead of HTML. This is Cloudflare’s own proposal rather than an IETF spec, though the mechanism (HTTP content negotiation via the Accept header) is standard. Real agent runtimes prefer Markdown because it is cheaper to tokenize and easier to parse than HTML. Some early movers (Cloudflare itself, a handful of docs websites) support Markdown content negotiation; most websites do not.

Bot Access Control (3 Checks)

  • AI bot rules in robots.txt (RFC 9309). Whether robots.txt contains directives for AI-specific user agents (GPTBot, ClaudeBot, PerplexityBot, etc.).
  • Content Signals in robots.txt. An emerging spec for expressing per-URL access rules inside robots.txt. Parsed as User-agent: * followed by Content-signal: directives. Adoption is minimal right now.
  • Web Bot Auth request signing. HTTP message signatures at /.well-known/http-message-signatures-directory that let agents prove their identity cryptographically. This is the Agent Name Service side of things, Cloudflare shipped with GoDaddy earlier in Agents Week. Adoption is almost zero outside Cloudflare’s own properties.

API, Auth, MCP & Skill Discovery (6 Checks)

  • API Catalog (RFC 9727). A machine-readable index of a website’s API endpoints at /.well-known/api-catalog.
  • OAuth / OIDC discovery (RFC 8414). Standard OAuth 2.0 authorization server metadata at /.well-known/oauth-authorization-server and /.well-known/openid-configuration.
  • OAuth Protected Resource (RFC 9728). A website declaring which endpoints are OAuth-protected and how to authenticate.
  • MCP Server Card (SEP-1649). A Model Context Protocol server advertising its capabilities at /.well-known/mcp/server-card.json. SEP-1649 is a draft proposal inside the MCP spec process.
  • Agent Skills index. A list of agent-callable skills at /.well-known/agent-skills/index.json. Also emerging.
  • WebMCP (Experimental). An in-page JavaScript API registering agent-callable tools via navigator.modelContext. The scanner uses headless browser rendering to detect whether the website registers any WebMCP tools on page load.

Commerce (3 Optional Checks, Not Scored On Non-Commerce Websites)

  • x402 payment protocol. HTTP 402 Payment Required infrastructure for agent-native payments.
  • UCP profile (Universal Commerce Protocol). Google’s merchant-metadata standard at /.well-known/ucp.
  • ACP discovery document (Agentic Commerce Protocol). At /.well-known/acp.json.

The Commerce category is flagged “optional” on non-commerce websites. The scanner detects whether any ecommerce signals are present and, if not, displays the commerce checks for informational purposes without counting them in the score.

That last design detail matters. It is evidence Cloudflare anticipated exactly the problem the rest of this article is about.

Nohacks.co Scored 33/100, Level 2 Bot-Aware

I ran the scan on nohacks.co. The result was 33 out of 100, Level 2 “Bot-Aware.”

The Agent Readiness Score report for nohacks.co, scanned on 2026-04-18. Composite: 33/Level 2 “Bot-Aware.” Category breakdown: Discoverability 67 (2/3), Content 0 (0/1), Bot Access Control 100 (2/2), API, Auth, MCP & Skill Discovery 0 (0/6). Commerce checks not scored (no ecommerce signals detected). Image Credit: Slobodan Manic

A note on that number: After the first scan, I added Content Signals directives to robots.txt, which moved Bot Access Control from 50 to 100 and pulled the composite up eight points from an initial 25. Every other category below is unchanged from the first scan. I’ll come back to the Content Signals fix and why I made it at the end of this section.

Here is what drove each category score:

  • Discoverability: 67. robots.txt and sitemap.xml passed. Link headers failed because this website does not emit Link: headers in its responses.
  • Content: 0. Markdown content negotiation is not configured. The website returns HTML regardless of the Accept header.
  • Bot Access Control: 100. Both scored checks passed. AI bot rules in robots.txt (I have explicit rules for AI user agents) and Content Signals in robots.txt (I added these after the first scan). Web Bot Auth request signing is listed in this category as an informational check, but not counted toward the 2/2.
  • API, Auth, MCP & Skill Discovery: 0. All six checks failed. No API Catalog. No OAuth discovery. No OAuth Protected Resource metadata. No MCP Server Card. No Agent Skills index. No WebMCP tools on the page.
  • Commerce: not scored. nohacks.co has no e-commerce. The Commerce checks all failed, but the category is correctly excluded from the composite score.

That is a 33 on a scanner built by the company I most trust to understand where the agent-ready web is going. I consider this website reasonably well-designed for agents. The robots.txt is clean and explicit. The content is server-rendered, machine-readable HTML with clean semantic structure. The sitemap is current. The URLs are stable. If you asked me a week ago whether this website was agent-ready, my answer would be somewhere between “mostly yes” and “for what it needs to do, yes.”

And yet: 33, Level 2.

The scanner is measuring what it says it is measuring. The composite score, by itself, is still the wrong number to optimize for.

One note on the Content Signals fix, because it’s relevant to the Goodhart argument later in this article. Content Signals is a Cloudflare proposal with almost no deployment beyond Cloudflare-aligned crawlers. I debated adding it for exactly the score-chasing reason this article warns about. I decided it was defensible for two reasons. First, the fix is declarative, not decorative. The directives state real policy about what should happen with my content, and the statement has meaning even if the spec fails. That is different from adding an empty MCP Server Card to satisfy a scorer. Second, for a website that writes about agent-readiness specifically, publicly declaring content policy is editorial practice regardless of which crawler respects it. The fix was one commit to public/robots.txt and the directives are readable by any human curious enough to check.

Same Website Scores 33 Or 67 Depending On The Preset You Select

On the All Checks preset, nohacks.co scores 33 out of 100, Level 2 “Bot-Aware.” On the Content Site preset, same website, same day, different scan configuration, it scores 67, still Level 2 “Bot-Aware.” Nearly double the composite number. The 34-point gap is the difference between two scan configurations of the same scanner, not a difference between two websites.

Here is what the Content Site preset changes in the scan configuration:

The Content Site preset unchecks every item in the API/Auth/MCP/Skill Discovery category, every item in the Commerce category, and Web Bot Auth in Bot Access Control. Six scored checks remain: three Discoverability (robots.txt, Sitemap, Link headers), one Content Accessibility (Markdown negotiation), two Bot Access Control (AI bot rules, Content Signals). Image Credit: Slobodan Manic

Running that preset on nohacks.co produced this result:

Nohacks.co under the Content Site preset: 67 / Level 2 “Bot-Aware.” Four of six scored checks pass. The two failing checks are Link headers (a fix I have not deployed yet) and Markdown content negotiation (not configured). Both are real shipping signals that agent runtimes benefit from today. Image Credit: Slobodan Manic

Four of six scored checks pass. The two failures are unambiguous remediation targets: Link headers via HTTP response configuration, Markdown content negotiation via origin or CDN response logic. Both ship against real agent-runtime behavior today. Neither is a proposal-stage format that will only maybe become a standard. This is the honest reading of nohacks.co’s agent-readiness state: two specific, actionable gaps.

The Correct Toggle Is Hidden, And The Default Score Is Wrong

The scanner is doing its job. It knows a blog does not need an MCP Server Card. It knows a podcast archive does not publish an API catalog. The Content Site preset is not cosmetic. It removes irrelevant checks and gives a content website an accurate reading against standards that actually apply.

The problem is that the preset is hidden. When a user lands on isitagentready.com and pastes a URL, the default scan is All Checks. The Site Type toggle that would switch to Content Site or API/Application lives inside a Customize dropdown that most users will never open. The user clicks Scan, reads the composite number, takes a screenshot, shares it. The shareable number, the one that travels on social media, the one competitors compare across, is the All Checks composite.

For a content website that runs the default scan without reading individual checks, the composite is structurally too low. The 33 on nohacks.co is wrong for the kind of website nohacks.co is. The 67 from the Content Site preset is the accurate reading. Two numbers from the same scanner on the same website. The accurate number is behind a dropdown. The wrong number is on the front page.

Any web professional who runs the scanner and plans to share the score anywhere public needs to open Customize, select the preset that matches their website type, and re-run before sharing. Without that step, the public score will understate the website’s actual agent-readiness, and the gap between the shared number and the accurate number will be larger for content websites than for API websites (which are closer to the All Checks baseline). Read the individual checks. Do not share a composite until you know which preset produced it.

For the record: the 67 is bothering me. I am going to go get the 100. I know exactly what the Goodhart section below is about to warn against, and I am going to do it anyway. Two fixes stand between me and the 100. Both are five-minute jobs. Both map to real agent-runtime behavior (Link headers for discovery, Markdown content negotiation for efficient agent parsing), so at least the motivation is legitimate and not pure score-chasing. That caveat is also exactly what score-chasers say. Public scores are a gravitational field. Even the person writing a long article about their unreliability ends up orbiting.

Agent Readiness Measures Delivery, Not Message

Every category the Agent Readiness scanner tests is about delivery: discoverability, content negotiation, bot access, API discovery, commerce protocols. None tests the quality of the message itself.

The scanner never asks whether your headlines are clear, whether your product descriptions persuade, whether your content answers the query well, whether your writing is any good. Those are SEO and CRO questions. They occupy the discipline of making the message better. The Agent Readiness Score occupies a different discipline entirely. It asks whether an agent can fetch your content, parse the format it arrives in, authenticate against your endpoints, call your functions, pay for your outputs.

That is the distinction that matters. Classical web optimization (SEO, CRO) is about what you say and how persuasively you say it. Agent-readiness is about how you deliver what you say to a non-human reader. Two websites can publish word-for-word identical content. One serves it as server-rendered HTML with semantic markup, responds to Accept: text/markdown, exposes structured data, returns predictable response codes. The other serves it as a JavaScript-rendered single-page application with no content negotiation and an inconsistent error surface. The message is identical. The delivery is different. The agent-readiness score will be different. And it will be right to be different, because the delivery is what the agent interacts with.

This is also why agent-readiness fixes tend to be orthogonal to SEO and CRO work. You can improve an agent-readiness score without rewriting a single word of your content. You can also have world-class SEO content that scores a 10 on the agent-readiness scanner because none of your delivery pipeline was designed for machine consumers. SEO and CRO work on the content layer. Agent-readiness works on the transport and protocol layer. They are adjacent but not the same craft, and treating them as the same is the mistake that turns an agent-readiness project into a content-rewrite project and misses the actual fix.

The people who will do well over the next several years are the ones who stop arguing about which discipline matters more and start recognizing they occupy different layers of the stack.

3 Goodhart Risks Built Into The Agent Readiness Score

Goodhart’s law says that when a measure becomes a target, it stops being a good measure. The Agent Readiness Score is well-designed, but it is also now a public, shareable, compared number, which produces three predictable behavioral failures in the wild.

The first risk is that website owners will optimize for the number rather than for real agent behavior. Add an MCP Server Card that points nowhere because the scanner wants one. Publish an Agent Skills index with no actual skills. Ship a WebMCP tool that does nothing just to pass the detection check. The score goes up, and nothing changes for real agent runtimes visiting the website.

The second risk is that consultancies will start selling “Agent Readiness Score optimization” as a service, selling the score rather than the underlying architecture. The history of SEO gives us a century of data on how this plays out. PageRank became a target, and a decade of link-spam economy grew up around it. Core Web Vitals became a target, and a generation of performance-theater optimizations followed. The Agent Readiness Score is a better-designed metric than either of those were at launch, but the same gravity applies.

The third risk is that the scanner’s inclusion of emerging standards as scored signals will accelerate the adoption of those standards past the point where they are ready to carry real traffic. The scanner checks for llms.txt, a proposed format for exposing website content to language models. Llms.txt is not a ratified standard, has no governing body, and has competing proposals for how it should be structured. Including it as a scored signal gives it weight it has not earned in the ecosystem. A website owner looking to fix a failing check is the marginal adopter who tips a proposal into a de facto standard before the spec work is done.

None of these failure modes are hypothetical. They are how every public measurement score in the history of the web has played out. The Agent Readiness Score is better than most because Cloudflare is honest about what it is, because the per-check detail is available right alongside the composite number, and because the Commerce category correctly excludes itself on non-commerce websites. That honesty is a feature worth protecting. Website owners and the consultancy industry will be tempted to treat the composite number as the target anyway.

Do not do this.

6 Weekend Fixes That Map To Real Agent Runtimes

Six actions for a web professional running the scanner the weekend of its launch, ordered from highest-leverage to lowest:

  1. Run the scan on your website. It takes about 30 seconds. Note the score and open the detailed report. The detail is where the signal is.
  2. Fix the failing checks that ship against real agent runtimes today. These are the ones whose absence measurably hurts your website for agents visiting it right now:
    • robots.txt. If missing, add one. If present, make sure it contains specific rules for AI user agents (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, etc.).
    • sitemap.xml. If missing, generate one and link it from robots.txt. Keep it current.
    • Markdown content negotiation. Configure your origin or CDN to return text/markdown when the Accept header requests it. Cloudflare’s own AI Crawl Control has first-class support for this. Other providers require custom server logic.
    • Structured data. Ship schema.org JSON-LD for the content types your website publishes (Article, Product, Organization, BreadcrumbList). This is not a scored check, but it is the highest-leverage fix for citation behavior across every agent runtime currently deployed.
  3. Treat the proposal-stage formats as a watch list, not a checklist. llms.txt, Content Signals in robots.txt, Web Bot Auth, API Catalog, MCP Server Card, Agent Skills, WebMCP, ACP, UCP are all real working standards in some sense. They are not shipping against real agent-runtime behavior at scale yet. Watch them. Implement them when your stack has a reason to, not because the scanner flags them.
  4. Ignore the composite number in your own tracking. Track individual check outcomes over time. A website that goes from 3 of 5 real-runtime checks passing to 5 of 5 has measurably improved, even if the composite score barely moved because the 10 proposal-stage checks still fail.
  5. Re-scan after changes. The scanner is fast, free, and available via the URL Scanner API if you want to script regression checks into your deployment pipeline.
  6. Skip the consultancies selling Agent Readiness Score optimization. The work is straightforward enough that a half-day audit and a focused remediation sprint will beat any packaged service.

The scanner is the tool. The work is still the work.

Vendor-Specific Scanners Are Coming: Track What Every Scanner Tests

The Agent Readiness scanner is standards-list-shaped: a set of checks against a fixed list of protocols and formats, some ratified (RFC 8288 Link headers, RFC 9309 robots.txt rules, RFC 8414 OAuth discovery, RFC 9727 API Catalog, RFC 9728 OAuth Protected Resource), some emerging proposals (MCP SEP-1649, WebMCP, Content Signals, Web Bot Auth, x402, UCP, ACP, llms.txt). The next thing that happens in the ecosystem is predictable: Other vendors will ship their own scanners against their own preferred lists. The overlap will be significant because most of the ratified standards are uncontroversial. The divergence will be in which proposals each vendor scores for.

That divergence is where the agent-readiness measurement story gets interesting. A Cloudflare scanner that checks for Web Bot Auth and UCP is making a bet. A Google scanner, if it ships, would check for some of the same things and some different ones (Google has UCP, does not have Web Bot Auth). A Perplexity scanner would check for yet another set. Website owners would see different scores from different scanners on the same website. The composite number, already not trustworthy, becomes vendor-specific.

The signal worth tracking is which checks show up in every scanner that ships. Those are the de facto standards. The checks that only show up in Cloudflare’s scanner are Cloudflare’s bets. Some will win. Most will not.

This is the pattern that made me comfortable publishing an article about a Cloudflare tool on the day it shipped. The Agent Readiness Score is real. The thesis behind it (agent-readiness is a measurable property) is the right thesis. The specific scorecard is version one of something that is going to have dozens of versions, each reflecting its vendor’s bets. Web professionals should engage with the version-one scorecard, fix what it correctly flags as real, watch what it flags as emerging, and keep their own running list of which checks survive across every scanner that ships in the next six months.

That running list is the real agent-readiness standard. The composite score is the marketing layer.

Run the scan. Read the report. Fix what matters. Watch what might.

More Resources:


This post was originally published on No Hacks.


Featured Image: RobinRmD/Shutterstock

3 Unrelated Stories About AI & Writing Tell The Same Story via @sejournal, @gregjarboe

I stumbled upon three separate articles about writing and AI in the same week, each from a completely different angle, and all describing the same thing.

A novelist turned MIT writing lecturer confronting students who outsourced their essays to AI. A new Graphite study showing AI-generated articles now make up roughly half of all new content on the web and have plateaued there. And fresh data from The Accountancy Partnership showing that half of freelance creatives say rising stress is affecting their work, as client budgets for human creative services shrink.

One data point is a fact. Two is a coincidence. Three is a trend.

When read together, these articles formed an argument that every SEO professional, content marketer, and creative freelancer should take seriously, acknowledging the content divide that is happening and asking, “Which side are you on?”

The First Story: What Happens When Students Outsource The Struggle

On May 10, Micah Nathan, a novelist and MIT lecturer in fiction and non-fiction writing, published a piece in The Guardian about confronting his creative writing students over their AI use. The confession session that followed, he wrote, became one of the most productive teaching moments of his eight years at MIT.

His key insight wasn’t about academic honesty. It was about what writing actually does. “Writing isn’t just the production of sentences,” he told his students. “It’s the training of endurance by way of sustained attention. It’s a way of learning what one thinks by attempting to say it. An LLM can reproduce the appearance of that activity, but it can’t replace it, because the value lies not only in the object produced but in the transformation that occurs during its making.”

He described AI prose as “faultily faultless, icily regular, splendidly null,” borrowing Tennyson’s description of a beautiful but empty face, producing what he called “simulacra of thought, generated via pattern recognition learned from millions of human-penned words, rooted in no particular experience by no particular person.”

Insightful readers, he argued, feel that emptiness even if they can’t articulate it.

For SEO professionals, this is not an abstract literary concern. It is a precise description of the content quality problem that Google’s helpful content systems have been trying to solve since 2022. The signal Google is hunting for is exactly what Nathan identifies as the thing AI cannot produce – evidence of a mind actively grappling with a specific problem from a specific experience. Pattern recognition learns from what humans wrote. It cannot replicate why they wrote it. 

→ Read More: Why Great Content Is No Longer Enough & What Beats It In AI Search

The Second Story: The Feared Takeover Hasn’t Happened – Yet

On May 15, Megan Morrone reported for Axios on new data from digital marketing agency Graphite, which analyzed 55,400 online articles and listicles published between January 2020 and March 2026, running each through three AI-detection tools. The finding was more nuanced than most AI content coverage has been about the share of primarily AI-generated content, which has held near 50% for more than a year and appears to have plateaued.

The feared takeover hasn’t materialized. AI content briefly surpassed human-authored content in late 2024, but the two have stayed roughly equal since.

The important caveat Morrone included is that many articles are no longer written purely by humans or AI. A human may use AI for outlining, drafting, rewriting, or editing, making the line genuinely blurry. Dan Klein, a UC Berkeley professor and AI model CTO, flagged the feedback loop risk. Once models train heavily on AI-generated content, the internet could become a machine that produces low-quality content that trains models that produce more low-quality content.

For SEO professionals, the plateau is reassuring and cautionary in equal measures. The volume panic was overstated. But the quality dilution problem is real and growing, and it creates the same opportunity Nathan identified from the other direction. In a web that is roughly half AI-generated content, content that carries genuine human experience and specific expertise becomes more differentiating, not less.

→ Read More: AI Platform Founder Explains Why We Need To Focus On Human Behavior, Not LLMs

The Third Story: The People Producing This Content Are Under Serious Stress

On May 13, Emma Hull at The Accountancy Partnership directly emailed me data from a new report on creative freelancers across PR, marketing, performing arts, graphic design, photography, and adjacent industries. Half of freelance creatives (50.7%) say rising stress levels are affecting their work. Half (50.2%) say client budget cuts are the biggest challenge they faced in 2025. Over two in five (43.3%) believe AI will negatively affect their sector. Nearly half regularly work unpaid hours each week.

Lee Murphy, Managing Director at The Accountancy Partnership, put it plainly: “Creative work is often closely linked to marketing budgets and discretionary spending. When businesses begin tightening costs, creative services can sometimes be one of the first areas to see reduced investment.”

The irony embedded in these three numbers together is worth reflecting on. Clients are cutting budgets for human creative work at the same time AI is generating roughly half the content on the web, while a professor at MIT is documenting the specific cognitive cost that outsourcing the writing process extracts from anyone who does it, whether a student or a professional.

The freelancers under the most pressure are the ones most tempted to use AI to produce more content faster to compensate for lower rates. The content they produce that way becomes part of the 50% that is indistinguishable from machine output. And content that is indistinguishable from machine output is exactly what the Graphite data and Google’s quality systems are training users and algorithms to discount.

→ Read More: Relying Too Much On AI Is Backfiring For Businesses

What The Pattern Actually Means

The three stories, read together, describe a market in the process of bifurcating. On one side sits high-volume, low-differentiation content produced quickly, priced cheaply, and increasingly hard to distinguish from AI output, regardless of who generated it. On the other sits content that carries specific expertise, direct experience, and the editorial judgment that Nathan’s students were trying to skip past. Content that takes longer, costs more, and is increasingly the only kind that earns meaningful search visibility and reader trust.

This is not a new argument in SEO. What is new is the empirical clarity with which three independent sources from three entirely different disciplines – literary education, web content analysis, and freelance labor economics – are all pointing at the same conclusion in the same week.

Shelley Walsh made the point in her recent Search Engine Journal piece on scaling AI content that the commodity versus non-commodity divide is where the real strategic question lives. The three stories above are evidence that the divide is already here, already measurable, and already affecting people’s livelihoods.

The writers who understand this, and produce accordingly, are the ones who will still have work worth doing when the budget cycles turn again.

More Resources:


Featured Image: SvetaZi/Shutterstock

LLM Guidance Doesn’t Transfer The Way SEO Guidance Did via @sejournal, @DuaneForrester

For roughly two decades, the SEO discipline operated on a quiet assumption that turned out to be one of its most valuable features. Guidance from one search engine traveled. If Google said sitemaps mattered, Bing said sitemaps mattered. If Bing said structured data deserved real effort, Google said the same. Practitioners optimized for Google with reasonable confidence that the work would carry across the other engines, and most of the time it did. That portability was not luck. It was the product of a structurally large overlap layer that the major search engines had jointly built, brick by brick, over twenty years.

That world doesn’t exist in LLM-land. The major providers train on different corpora, run different crawlers under different policies, route different queries through different retrieval systems, and apply different alignment processes that shape the final response in ways the upstream signals can’t predict. Guidance from any one provider, including Google’s guidance about its own Gemini products, is one data point. Practitioners carrying the SEO habit forward, the habit of treating one engine’s guidance as roughly the whole map, will optimize confidently for one platform and miss the others.

Sidebar: As I was finalizing this piece, Google published fresh guidance on optimizing for their generative AI features. Their framing is explicit: from Google Search’s perspective, optimizing for AI search is still SEO. That framing is accurate for Google Search. It does not extend to ChatGPT, Claude, Perplexity, or any other LLM, and that is precisely the trap this article is about.

The Shared Standards That Made SEO Guidance Portable

The era of portable guidance was built on actual collaboration, not coincidence. The Sitemaps protocol became the joint property of Google, Yahoo, and Microsoft in November 2006, when the three engines formally agreed to support a common protocol at version 0.90, building on Google’s earlier Sitemaps 0.84 from June 2005. Five years later, on June 2, 2011, the same three engines launched Schema.org, with Yandex joining shortly after, to create a common vocabulary for structured data markup. That was the announcement that got made on stage at SMX Advanced. I was on the Bing team at the time, and what struck me then is what still matters now. The engines were competitors, but they had decided that a shared vocabulary served them all. Webmasters got one set of rules. The web got cleaner data. The engines got better signals. Everybody won.

The pattern repeated with robots.txt, the 1994 convention that became RFC 9309 at the IETF in 2022, formalizing what every serious crawler already honored. And it repeated again, more recently, with IndexNow, the protocol Microsoft Bing and Yandex launched in October 2021. IndexNow is now supported by Bing, Yandex, Naver, Seznam, and Yep. Google has tested the protocol since 2021, but has not adopted it.

That overlap layer is exactly why Google’s guidance felt safe to follow, even if you cared about Bing traffic. The signals the engines used were not identical, but the inputs they accepted, the protocols they honored, and the standards they advertised were. Optimization had a shared substrate.

Where The LLM Stacks Actually Diverge

The LLM environment doesn’t have a shared substrate of comparable size. The differences are not cosmetic, and they are not temporary. They are baked into how the systems are built.

Start with training data. OpenAI has signed disclosed licensing deals with News Corp worth up to $250 million over five years, Axel Springer at roughly $13 million per year, Reddit at an estimated $70 million per year, plus the Financial Times, Condé Nast, Hearst, Vox Media, The Atlantic, the Associated Press, Le Monde, and others. Google has its own Reddit deal, estimated at $60 million per year, granting real-time data API access. Anthropic has not publicly disclosed equivalent publisher licensing deals, and that undisclosed status is itself the practitioner-facing point. The corpora that fed these models, and that continue to refresh them, are not the same documents. Practitioners cannot know what any given provider has paid for and what it hasn’t.

The crawler infrastructure diverges next. OpenAI runs three separate bots: GPTBot for training, OAI-SearchBot for search indexing, and ChatGPT-User for user-initiated retrieval. Anthropic runs three of its own: ClaudeBot for training, Claude-SearchBot for search, and Claude-User for user-initiated retrieval. Perplexity runs PerplexityBot and Perplexity-User. Google introduced Google-Extended in September 2023 as the user-agent that controls whether Google can use a site’s content to train Gemini, separate entirely from the Googlebot that handles traditional search indexing. There is no single AI user-agent. Every provider requires a separate rule, and the rules don’t translate cleanly across providers because the bots don’t do equivalent jobs in equivalent ways.

The retrieval architectures diverge structurally. ChatGPT has historically used Bing’s index as its primary web search source, and that connection appears to still be primary, though OpenAI continues to build out additional infrastructure alongside it. Perplexity built its retrieval system on a Vespa-based pipeline that treats documents and sub-document chunks as first-class retrievable units. Google’s Gemini uses Google’s own index plus Knowledge Graph grounding. Claude uses Brave Search as a retrieval partner. Same query, four different retrieval systems, four different views of which sources exist and which sources are worth surfacing.

Then comes the alignment layer, which is where SEO had no equivalent at all. After a model is trained on its corpus, providers run post-training to shape how the model actually behaves: tone, refusal patterns, format, safety posture, what counts as a good answer. OpenAI’s primary approach has been RLHF, or Reinforcement Learning from Human Feedback, where human raters score model outputs and the model learns to produce highly rated responses. Anthropic developed Constitutional AI, which trains models to critique and revise their own outputs against a written set of principles. These methodologies produce demonstrably different behavior in the final products. The same retrieved content, fed into two models aligned by two methodologies, can yield two materially different responses about the same brand.

When One Provider’s Guidance Demonstrably Fails To Port

The clearest single example of guidance that doesn’t port is llms.txt. Jeremy Howard of Answer.AI proposed the file in September 2024 as a markdown manifest, placed at a site’s root, that would guide LLMs to the most important content. The proposal got picked up across the SEO community. Yoast built a generator. Agencies added llms.txt creation to their service catalogs. Conference speakers declared it essential.

As of mid-2026, no major LLM provider has confirmed they consume the file. Not OpenAI. Not Anthropic. Not Google. Server-log analyses across hundreds of thousands of domains show major AI crawlers don’t routinely request /llms.txt at all. Google’s John Mueller publicly compared it to the deprecated meta keywords tag. Gary Illyes confirmed at Search Central Live in July 2025 that Google does not support llms.txt and is not planning to.

I’ve written about this elsewhere, so I won’t repeat the technicalities here. What matters for this argument is the structural lesson. Schema.org succeeded because three engines built it together and then enforced it together. Llms.txt was proposed by one researcher, picked up by tooling vendors, and ignored by the platforms it was supposed to serve. The shared-standards model that gave SEO its portable guidance is not available to LLM practitioners at the same scale, because the platforms are not building the standards together. They are building their own pipelines.

The Gemini Inversion

The cleanest illustration of how far guidance portability has degraded sits inside one company. Google publishes its own SEO documentation at Search Central, the canonical guidance the industry has followed for two decades. Those documents emphasize traditional ranking signals, E-E-A-T, content quality, technical accessibility, and structured data. That guidance is still useful for Google Search itself.

Google also makes Gemini, the model that powers AI Overviews and Google’s separate AI Mode surface. And the citation behavior of those surfaces does not appear to track the guidance the same company publishes for its own search results.

In late 2024, roughly three-quarters of pages cited in AI Overviews also ranked in Google’s top 12 for the same query. By early 2026, after Google upgraded AI Overviews to Gemini 3 in January, Ahrefs analyzed 4 million AI Overview URLs and found that only 38% of cited pages also appeared in the top 10 for the same query. A separate BrightEdge analysis put the overlap closer to 17%. SE Ranking’s post-upgrade work found that Gemini 3 replaced approximately 42% of the domains previously cited under earlier model versions and generates 32% more sources per response.

The gap widens further when you look at Google’s AI Mode, which is a separate conversational surface that runs on the same Gemini family. Semrush data shows AI Mode and AI Overviews reach semantically similar conclusions 86% of the time, but cite the same URLs only 13.7% of the time. Only 14% of AI Mode citations rank in Google’s traditional top 10.

It appears, so far, that the canonical relationship has shifted. Google’s published SEO guidance is still the cleanest path to ranking in Google Search. But that ranking is no longer a reliable proxy for being cited by Google’s own AI surfaces. The same guidance, the same content, the same domain, can produce three meaningfully different outcomes across Google Search, AI Overviews, and AI Mode, even though all three live inside the same company. The old playbook of following the search engine’s guidance and trusting that the engine’s other surfaces would behave consistently does not appear to be delivering the same returns it used to.

What Still Ports, And Why It’s Smaller Than It Looks

A universal layer does survive. Crawler accessibility still matters across every provider. Primary-source factual content still wins more citations than aggregator restatement. Clean retrievable structure still helps every system understand what a page is about. Presence on the high-authority sources that all major LLMs disproportionately cite, Wikipedia, YouTube, Reddit, major news outlets, still functions as a force multiplier across platforms. Earning visibility on those sources gives content a chance to surface in any LLM that draws on them.

But the universal layer is much smaller than it was in the SEO era. Qwairy’s analysis of 118,000 AI responses across ChatGPT, Perplexity, Google AI Mode, and Claude found that only 11% of cited domains appeared across multiple platforms. The other 89% were platform-specific. A brand that wins citations on Perplexity may be largely invisible on Claude. A brand that’s a regular reference on ChatGPT may not show up in AI Overviews at all. The same content can be the right answer for one system and the wrong answer for the system next to it.

What This Means For The Work

The practical implication is not abandoning all hope. It is that practitioners need to stop treating any single LLM provider’s guidance as the universal map and start treating it as one input among several. Read what every major provider publishes about their own systems. Test your visibility across platforms, not just on the platform you happen to use most. Treat divergence as the default and overlap as the exception, not the other way around.

This is not how SEO worked, and the difference matters. The old reflex was to optimize for Google and trust the portability. The new reality is that following one LLM’s guidance, even Google’s guidance about Gemini, will leave you optimized for a slice of the landscape and potentially blind to the rest. The discipline is being rebuilt on platform-specific work that didn’t exist in the SEO era, and the practitioners who recognize that first are going to spend the next two years setting the standards everyone else follows.

The overlap has shrunk. You now have more work than ever to accomplish.

If you have thoughts on where the divergence between providers is sharpest in your own work, reach out directly. I’d genuinely like to hear what’s showing up in the data.

More Resources:


This post was originally published on Duane Forrester Decodes.


Featured Image: Rawpixel.com/Shutterstock; Paulo Bobita/Search Engine Journal

Microsoft Clarity Now Shows Grounding Queries Behind AI Citations via @sejournal, @TaylorDanRW

When Microsoft Clarity made AI citations available to all users, it opened up a new playground for SEOs to harvest AI visibility data. Finally, we can see the exact “grounding queries” an AI engine uses to pull our content.

It raises a massive question because this is a Microsoft tool: Are the insights useless if your audience doesn’t touch the Bing ecosystem?

Microsoft Clarity Grounding Queries

When you ask Copilot a question, it translates your words into simple search terms called grounding queries to find facts on the web before it answers. You can use this data to improve your own website and content.

  • Finding gaps where your content does not match what the AI searches for.
  • Simplifying pages that the AI reads but does not link to.
  • Using these simple layouts to help your Google search results.

Copilot Vs. Gemini

Both Copilot and Gemini use retrieval-augmented approaches. Instead of generating answers using only pre-trained parameters, they dynamically query external search indexes to retrieve real-time data, which they then use as context to ground their final responses.

Feature Microsoft Copilot Google Gemini
Structure Uses a query translator, Bing index search, and OpenAI models to write the final text. Uses a query translator, Google Search, and Google’s Gemini models to write the final text.
Pulling Sources Uses the Bing index and Microsoft Graph to scan web pages, emails, and Microsoft 365 files. (With permissions enabled) Uses Google Search and Google Workspace to scan web pages, Google Drive files, and Gmail. (With permissions enabled)
Synthesising Answers Focuses on direct answers. It uses structured lists, tables, and bullet points to show facts quickly. Focuses on creative, conversational answers. It is built to handle text, images, and code at the same time.

Does Ranking In Bing Matter?

Yes (Correlation).

One of my websites was doing extremely well in Copilot, with over 36,000 citations across all queries. Now, Clarity doesn’t give you the prompts/queries themselves, but it does give you the Grounding queries (grounding queries and key phrases used to retrieve your site’s content).

Image from author, May 2026

My website has a history, running for years with a previous domain merged in 2019, and boasts over 1,000 articles. Given that Google barely sends traffic, and third-party SEO tools often label it as spam due to non-English backlinks (it covers search engines like Baidu, CocCoc, SwissCows, attracting an international audience), I never expected 36,000 citations.

So, why the Copilot love? I took the 147 grounding queries and tracked their rank in Google and Bing.

Image from author, May 2026

Of the 147 queries, Bing ranked all but 6, with the majority in traffic-driving positions (top 20). Google didn’t rank a single one.

So, If This Is Heavily Dependent On Bing Indexing, Is Clarity’s Data Useful Outside Of The Bing/Microsoft Ecosystem?

Because this is a Microsoft tool, the backend data feeding this dashboard is primarily capturing how your site is cited across Microsoft’s AI surfaces (like Copilot and Bing generative search).

It is not giving you a direct window into how OpenAI’s ChatGPT (using its own search), Google Gemini, or Perplexity are citing your links, because those platforms do not share their internal grounding logs with Microsoft.

And historically, we as an industry have been neglectful of Bing.

Even though the data collection source is skewed toward Microsoft’s AI engine, the insights themselves are highly transferable to your broader, platform-agnostic AI optimization strategies.

Can We Assume Other LLMs Retrieve Data In The Same Way?

AI engines, whether Google Gemini or Microsoft Copilot, use similar RAG frameworks to fetch data.

If the Bing ecosystem flags that a specific page on your site has a high “Share of Authority” for a complex query, it means that page is structured perfectly for AI consumption (clear tables, bullet points, direct answers). Data suggests that you can replicate that formatting across your site to appeal to Google Gemini as well.

However, this can be argued against as other research suggests that the similarity between LLMs is dependent on positional biases, and some may use the SDSR method rather than RAG.

Researchers in SEO have also found that ChatGPT has started to use Google Search as a fallback, when it was initially Bing.

In Summary

If your audience doesn’t touch the Microsoft ecosystem, this dashboard won’t give you a perfect 1-to-1 reflection of your total AI traffic, but it doesn’t make the data useless.

What grounding queries reveal is how AI systems distill user intent into retrievable search terms. That process is broadly consistent across platforms, even when the underlying indexes differ. A page earning citations in Copilot is doing something right structurally with clear answers, well-scoped topics, content aligned with how AI engines translate questions into queries. The Bing dependency tells you where the data comes from. The structural patterns tell you something more transferable.

The gap data is equally instructive. Pages your site ranks for in Bing that never appear as grounding queries signal a mismatch. Either the content isn’t structured for AI retrieval, or the topic isn’t one AI engines are actively grounding answers around.

Treat Clarity’s Citations dashboard as a useful proxy or “lab environment” and window into how LLMs read, slice/chunk, and credit your website’s content. Even if Copilot isn’t your primary AI traffic source, the patterns it surfaces are worth paying attention to.

More Resources:


Featured Image: Prostock-studio/Shutterstock

Google Shares First AI Mode Usage Data After One Year via @sejournal, @MattGSouthern

Google released a report detailing how people use AI Mode in the U.S., drawing on internal Search data and Google Trends to map search behavior one year after launch.

The report, published alongside Google I/O 2026 announcements, said that AI Mode has surpassed 1 billion monthly active users globally. Queries have more than doubled every quarter since launch.

How Query Behavior Is Changing

The report states that the average AI Mode search is three times longer than a traditional search. Both short and long queries are increasing in AI Mode, with users having conversations and asking longer questions.

Follow-up queries in AI Mode rose over 40% monthly in the U.S. More than one in six AI Mode searches are multimodal, using voice, images, or video. Image-based searches are up over 40% month-over-month since launch.

Top keywords include “information,” “identify,” “find,” “explain,” and “summarize.” Common first words are “what,” “how,” “I,” “is,” and “can,” with “I” especially notable, which may suggest people treat AI Mode more like a conversation than a traditional search.

What People Search For

Google grouped AI Mode search topics into five categories: Explore, Decide, Learn, Create, and Do. The top 10 topics include creative content, media, education, fashion, food, health, tech, travel, productivity, and development.

Brainstorming queries increased 30% faster than overall AI Mode queries since launch, with searches for “where to,” “where should I,” and “ideas for” also rising, per Google Trends.

Planning-related queries grew 80% faster over six months, with decision questions starting with “which” increasing 40%, especially “which of” and “which one.”

Shopping And Local Behavior

Shoppers start with traditional search, then move to AI Mode for deeper inquiry, especially in electronics, books, apparel, health and beauty, and automotive.

In AI Mode, store-related questions focus on “near me,” replacement parts, financing-related dealership searches, online options, and stock.

Top retail concerns include price, location, color, brand, and availability. For restaurants, users seek kid-friendly options, views, bars, vegan or vegetarian choices, and outdoor seating.

Creative And Educational Use

AI Mode’s image creation queries have more than tripled since early 2026, with users mainly requesting photos, quizzes, logos, stories, and code, as well as editing photos, documents, videos, messages, and code.

For education, top subjects include math, Spanish, history, English, and biology, while professional development searches focus on Security+, black belt, Network+, bar exam, and real estate license.

Why This Matters

The data shows AI Mode users are searching in ways that don’t map cleanly to traditional keyword patterns. Queries are longer, conversational, and increasingly multimodal. Follow-up conversations are growing, and planning and decision queries are among the strongest growth signals in the report.

If query length and follow-ups keep growing, that means thin content faces a different competition than conversational answers to multi-part questions.

Looking Ahead

Google released this report the same week it announced Gemini 3.5 Flash as the new default model in AI Mode, redesigned the Search box, and previewed search agents for this summer.

The keyword and query data covers May 2025 to April 2026 and comes from a random, unbiased sample of Google searches. The Trends data measures search interest as a share of AI Mode searches, not total query volume. AI Mode Trends data is not publicly available on trends.google.com.

Mt. Stupid Has A Pricing Page via @sejournal, @pedrodias

“There is now ample evidence, collected over the last few years, that AI systems are unpredictable and difficult to control.” That’s Dario Amodei in January, writing about the technology his company sells.

Compare with what’s on your LinkedIn timeline this week. Here’s the script: Schema markup ensures AI engines parse your content. The first sentence of every section must be the answer. Optimize for chunk-level retrieval. There’s a 13% citation lift available if you do X, a 2.8x conversion improvement if you do Y.

It’s one of the cleanest patterns going right now, and the industry has elected not to notice. The people closest to these systems are increasingly cautious about claims of control. The people furthest from it are increasingly certain they know how it works … they’ve cracked it. That gradient runs the wrong way.

What The People Who Built It Actually Say

Anthropic published its main interpretability research post in May 2024. It opens:

“We mostly treat AI models as a black box: something goes in and a response comes out, and it’s not clear why the model gave that particular response instead of another.”

Anthropic, writing about its own model, two years ago.

Things haven’t gotten more confident since. Neel Nanda, who runs Google DeepMind’s mechanistic interpretability team, gave an interview to 80,000 Hours in September 2025 in which the headline finding was that the most ambitious version of mech interp is probably dead. He doesn’t see a realistic world where the discipline delivers “the kind of robust guarantees that some people want from interpretability.” Worth re-reading.

The person whose job is to read AI minds is publicly conceding that the project, as originally conceived, won’t get there.

At NeurIPS 2024, Ilya Sutskever, co-founder of Safe Superintelligence and formerly chief scientist at OpenAI, accepted his Test of Time award and used the platform to say something the room wasn’t expecting from him:

“The more it reasons, the more unpredictable it becomes.”

Sutskever’s career is essentially the scaling hypothesis with a face on it. Hearing him say the next phase produces less predictable outputs is itself an admission.

Now scroll back to your timeline. The gradient is Dunning-Kruger redrawn at an industry scale: Mt. Stupid with a pricing page, and the valley of calibration where the actual work happens.

Image Credit: Pedro Dias

What The People Selling It Actually Say

A practitioner posts a four-pillar framework for “Technical GEO.” A consultant guarantees inclusion in AI Overviews. An agency markets a 13% lift in citation likelihood, derived from data the agency itself produced about the agency’s own prescriptions. A widely shared post promises that maintaining a 300-character paragraph limit dictates how a vector database chunks your content. A vendor claims a 78% “share of model.” A senior figure in your inbox describes a 2.8x improvement in conversion from being cited in SGE.

The vocabulary is deterministic: “ensures,” “guarantees,” “dictates,” percentages precise to the decimal, frameworks confidently named. None of it sounds anything like the language the people who built these systems use when describing how the systems behave.

This is the part I keep getting stuck on. The consultants are confident about the tactics they’ve measured against themselves. Run the same playbook on a few clients, watch some metric move, call it evidence. No control groups, no pre-registered hypotheses, no measurement of what the tactic is actually claimed to change. That’s the bar a real test has to clear; everything else has been confirmation in costume. The problem is the confidence level, which is wrong by an order of magnitude regardless of whether the underlying tactic does anything. The same model that Anthropic publicly says it cannot fully account for is being optimized against by people who confidently claim to know exactly what they’re doing.

Either Anthropic has been suspiciously modest in public, or somebody else is suspiciously certain.

When Somebody Tests

On Monday, last week, Ahrefs published a study by Louise Linehan and Xibeijia Guan with a title that should ideally be impossible: We Tracked 1,885 Pages Adding Schema. AI Citations Barely Moved.

The methodology is the kind of work you would expect to be standard, if the discipline cared about standards. 1,885 pages that added JSON-LD schema between August 2025 and March 2026. 4,000 matched control pages. Citation changes measured 30 days before and 30 days after the schema was added, across Google AI Overviews, Google AI Mode, and ChatGPT. Difference-in-differences on the matched groups.

The finding: No meaningful uplift in citations on any platform. AI Overviews actually showed a small but statistically significant decline. The report notes the odds of a gap that large being chance are roughly 1 in 2,500. The schema-makes-LLMs-understand-your-content thesis, tested at scale against a controlled baseline, did not survive the test.

This is the empirical confirmation of the technical case I made a week ago in The Whole Point Was the Mess: that LLMs read unstructured language, and that schema-and-chunking prescriptions are reasoning about an architecture that doesn’t exist. From first principles, two weeks ago. From controlled measurement, last Monday.

It is worth sitting with that. The dominant prescriptive category in the entire GEO playbook has been empirically falsified under controlled conditions, by a vendor with a substantial audience, in the open. And the frameworks keep selling.

Then Google Itself Answered

On May 15, 2026, Google published official documentation on optimizing for generative AI features in search. The page mythbusts the GEO prescriptions in writing: llms.txt files aren’t needed; chunking content isn’t required; rewriting content for AI systems isn’t necessary; special schema markup isn’t required; pursuing inauthentic mentions doesn’t help. The framing is unusually direct for a Google developer page:

“Many suggested ‘hacks’ aren’t effective or supported by how Google Search actually works.”

Google names Answer Engine Optimization and Generative Engine Optimization by their full terms and rejects the playbook outright.

Image Credit: Pedro Dias

That is the search engine the consultants claim to be optimizing for, telling its own developer audience that the optimizations don’t work. From first principles, two weeks ago. From controlled measurement, last Monday. From Google itself, last Friday. Three independent sources of the same answer, all within a fortnight. All ignored by the people selling the opposite.

The Cost Of Asking

This is where the diagnosis stops being polite.

Confident claims compound on these platforms in a way that skeptical corrections don’t. The difference is in who pays. Posting a confident claim costs you nothing. It gets engagement, builds an audience, generates inbound, makes the slide deck look forward-looking. If it turns out to be wrong, nothing happens. By the time anyone notices, everyone’s moved on to the next acronym.

Posting the correction costs you. It picks a fight. It marks you as a contrarian, or worse, as somebody who doesn’t get it. On LinkedIn, where most of this happens, it works against your professional brand. The algorithm will not reward it. The original poster owns the comment section and can ignore your methodology question while engaging with the congratulatory replies. Your reply lives in a collapsed thread.

There’s a specific move worth naming here. Ask a GEO consultant to explain, in plain terms, what their methodology actually does, what mechanism it acts on, what would count as evidence, what would falsify it. The response escalates into jargon. “Vector-space alignment.” “T1 query optimisation.” “Chunk-level semantic retrieval.” Real terms from machine-learning research, glued into combinations that sound rigorous and resist plain-language verification. The pattern works because it can. Asking “what does that actually mean” looks naive, and observers without the specific technical knowledge can’t tell which combinations are real and which are improvised on the spot.

Read the comments on any high-engagement GEO post. Fifteen replies in, 12 are agreements or “here’s another skill to add to your list.” Two or three offer diplomatically-framed skepticisms: “I would love to see more data,” or “the list is right, but…” The author engages substantively with the philosophical objection because pushing back against “this is too technical” is easy. The methodological objection, that the prescribed skills produce confident speculation without a measurement layer underneath, gets the politest burial.

What this adds up to is gaslighting at industry scale. The people reading the technology correctly get positioned as the ones who haven’t caught up; the prescriptions that controlled tests just falsified get sold as forward-looking. GEO has worked out how to make calibration look like the deficiency.

A recent X experiment captured the dynamic outside SEO. Someone posted a Monet painting and claimed it was AI-generated, asking the replies to explain its inferiority to a real Monet. Hundreds responded, confidently cataloging the “AI tells.” Flat brushwork, soulless composition, no cohesion, no soul. They were analyzing a Monet. The frame determined what they saw.

Screenshot from X, My 2026

The original post, where a lot of the initial replies have now been deleted.

Screenshot from X, May 2026

It’s the same trick. Vocabulary substitutes for substance; framing activates confirmation bias before any examination begins; the performance of analysis becomes what’s purchased rather than the analysis itself; “this is X” arrives before anyone checks whether it is. Once the frame is set, the analysis follows.

So the people most equipped to push back, the practitioners who’ve actually tried to test things, the technical SEOs who know what schema does and doesn’t do, the ones who can spot a fabricated lift number from across the room, stay quiet.

The result, on the timelines the C-suite reads, is a one-sided market.

The cost falls on the people who buy the claim. Clients pay for schema audits the Ahrefs study just falsified. Junior practitioners build careers on methodologies that won’t survive a controlled test. And the discipline burns credibility it will need later, when traditional search displaces further, and SEOs are expected to sit in rooms with engineering teams who’ve just spent two years watching the field confidently mis-call the technology.

Knowledge advances by trying to disprove your hypothesis, not confirm it. GEO does the opposite, runs studies designed to validate what it’s already selling. If the professionals claiming this expertise won’t even try to falsify themselves, who do we expect to believe us?

The Absence Is The Data

Strip the discourse, and what remains is the absence.

A serious technical field watches a controlled test contradict its dominant prescriptions, and the prescriptions keep selling. At that point, asking whether the prescriptions are wrong stops being the interesting question. That has been answered. The harder question is what’s wrong with a field that watches and doesn’t correct.

Same with the gradient. When the people who built the systems hedge and the people optimizing for those systems guarantee, asking who’s right stops being interesting. The researches and builders are right. Nobody who has worked on inference attribution thinks otherwise. The harder question is why the field lets the guarantees travel unchallenged.

The honest answer is that the incentives don’t pull toward correction. Confidence sells in ways caution can’t. The reportable framework wins the budget; the sensible assessment loses. And hedged language doesn’t fit on a pricing page where a guarantee fits perfectly.

None of this needs villains. The market for attention rewards confidence over calibration, every time.

You can keep watching the gradient run the wrong way. Or you can read what it actually is: an industry standing on Mt. Stupid, charging for the view.

More Resources:


This post was originally published on The Inference.


Featured Image: Roman Samborskyi/Shutterstock

Google Brings AI Content Verification To Search via @sejournal, @MattGSouthern

Google is expanding its SynthID verification tools to Search today, with Chrome support planned over the coming weeks. Users will be able to check the origin of images through Search features such as Lens, AI Mode, and Circle to Search.

The company is also launching an AI Content Detection API on Google Cloud, initially available to a group of trusted partners. Several companies are bringing SynthID watermarking to their AI-generated content, according to a blog post by Laurie Richardson, VP of Trust & Safety, and Pushmeet Kohli, Chief Scientist at Google Cloud and VP at Google DeepMind.

SynthID Verification In Search & Chrome

Google said it is expanding SynthID verification to Search today and plans to bring it to Chrome over the coming weeks.

Users can check whether an image was made with AI through features like Lens, AI Mode, and Circle to Search. You can ask questions like “Is this made with AI?” or “Is this AI generated?” to get verification results.

SynthID verification was already available in the Gemini app for images, video, and audio. It works by embedding imperceptible digital watermarks into AI-generated content.

C2PA Content Credentials

Google is also adding verification for C2PA Content Credentials, an industry standard for recording how media was created and modified.

The C2PA verification feature is rolling out in the Gemini app starting today and will roll out to Search and Chrome in the coming months.

AI Content Detection API

Google is launching a new AI Content Detection API on Google Cloud’s Gemini Enterprise Agent Platform, available to select partners. The API is a Google Cloud offering that Google says can detect AI-generated content made by Google and other popular models.

The API can help businesses evaluate and manage media across their platforms. Use cases include sorting feeds, preventing insurance fraud, fact-checking, and labeling synthetic media.

Initial partners include Shutterstock, Snap, Avid, Fox Sports, and Canva.

Industry Adoption Of SynthID

Companies including OpenAI, Kakao, and ElevenLabs are bringing SynthID technology to their AI-generated content. Google has open-sourced its SynthID text watermarking technology and partnered with NVIDIA to watermark AI-generated video from NVIDIA’s Cosmos models.

Meta, a fellow C2PA Steering Committee member, will start labeling camera-captured media with Content Credentials on Instagram. This means photos and videos shot on Pixel phones will be recognized and labeled on Instagram as camera-captured media.

Why This Matters

Google has been developing content-provenance tools since it first introduced SynthID in 2023. At that time, the technology was limited to select Google Cloud customers and was limited to images. The expansion to Search and Chrome moves verification from a specialized tool into surfaces where people encounter content every day.

The AI Content Detection API opens a different use case. Publishers and platforms that need to check whether content was made with AI will be able to access that capability through Google Cloud.

Searchers can already check image context through features like “About this image,” which Google expanded to Circle to Search and Lens in 2024. The SynthID verification adds a layer that checks for watermarks embedded at the point of creation, rather than relying on metadata that can be stripped.

The broader industry adoption of SynthID is worth watching. If more AI-generated media carries SynthID watermarks, Google’s verification tools have a wider base of content to check against. But SynthID only detects content watermarked with SynthID. Content from AI tools that don’t use it may not be identified through SynthID verification.

Looking Ahead

C2PA Content Credentials verification will come to Search and Chrome in the coming months. Google didn’t share specific timelines for broader availability of the AI Content Detection API beyond its initial partner group.


Featured Image: FOTOGRIN/Shutterstock

Inside AI Citation: Proven Strategies To Get Your Brand Cited via @sejournal, @lorenbaker

When customers ask AI a question, only a handful of sources get cited in the answer.

Which content signals does AI evaluate when selecting sources to cite?

Is your brand’s content structured to be one of them?

This is no longer a technology question; it is a brand and content strategy question. Find out exactly what earns AI citations.

Register above to watch the full on-demand session.

Learn How AI-powered Search Generates Answers

In this SEO webinar, Wayne Cichanski, VP of Search & Site Experience at iQuanti, unpacked how AI systems generate answers and what determines whether your brand’s content earns a place in them.

You’ll Learn:

  • How AI retrieval works: Understand the mechanics behind how AI-powered search selects and cites content, so you know exactly what you’re optimizing for.
  • AI citation signals: Identify the topical authority and brand trust signals that determine whether your content earns a place in AI-generated answers.
  • Practical content strategies that drive citation: Walk away with specific, practical tactics for creating and restructuring content that increases your brand’s AI visibility.

From topical authority to content structure and brand trust signals, you’ll learn the mechanics of AI retrieval into clear implications for performance marketers and digital leaders.

Register above to get actionable, practitioner-level strategies for building the topical authority and content structure that AI systems reward with citations.

👆 Register above to watch the recording on your schedule.

It Works Until It Doesn’t: AI Content Strategies That Backfire via @sejournal, @lilyraynyc

Over the past few years, I’ve watched AI content creation tools rapidly gain adoption across the SEO/GEO industry. These tools offer the promise of leveraging AI to automate content creation, reduce headcount, cut costs, and scale output.

As someone who has spent the last decade helping companies recover from Google algorithm updates, my spidey senses started tingling the minute I heard the pitches for many of these tools. Even before AI was part of the conversation, Google already had a long history of reducing the visibility of automated content in its search results.

Despite recent advancements in the quality of AI outputs, I’ve remained skeptical that publishing AI-generated or AI-assisted content at scale can drive sustained performance in Google’s search results. This is especially true now, given how Google updated its ranking systems in recent years specifically to demote overly optimized, SEO-driven content.

Over the past several months, I have been monitoring more than 220 websites that were publicly identified, either by themselves or by their AI content vendors, as customers of various AI content creation, automation, and scaling platforms. These tools fully write articles, assist with writing them, or use AI automations and workflows to support content creation. Many of these tools also now focus on driving visibility, mentions, and citations in AI search responses (AEO/GEO).

I wanted to analyze what happens after the claims of big wins.

A consistent pattern emerged across the 220+ sites I’ve been monitoring, and I believe it is concerning enough to be worth writing about: it works, until it doesn’t.

Below, I will share some of the trends I am observing, plus a variety of common SEO/GEO approaches I believe may be causing declines in organic search (and consequently, AI search) visibility. As a reminder, what is dangerous for SEO can also be dangerous for AI search, largely because of RAG.

The State of AEO/GEO Report Conductor 2026

Methodology & Disclaimers

Before we dive in, it’s important to set the stage with my approach and provide some important disclaimers.

This analysis is based on third-party SEO measurement data: organic traffic estimates and organic page count time series data from Ahrefs, corroborated against the Sistrix Visibility Index data to confirm broader visibility patterns. Top-traffic URLs were identified using Ahrefs’ top-pages export. Where I describe URL patterns or percentage changes, I am quoting directly from these third-party tools as of May 2026.

The dataset covers more than 220 client domains tracked across the publicly published customer-stories pages of over a dozen AI content platforms. For many of these sites, I narrowed the analysis to a specific subfolder where the AI-assisted content had been published, either identified directly in the case study itself or inferred from a sharp increase in new pages around the time of the case study’s publication.

The analysis, conclusions, and recommendations throughout this piece reflect my own professional opinions based on more than a decade of helping companies recover from Google algorithm updates. Other SEO/GEO practitioners may disagree with my findings and approaches, and individual sites and strategies will always have their own context.

3 Important Disclaimers About This Data:

First, these are third-party estimates, not first-party analytics. They are well-validated tools in the SEO industry, but they are not perfect measurements of organic search performance.

Second, the traffic declines described here could reflect many factors, including but not limited to algorithmic adjustments by Google, on-site changes by the site operators themselves, off-site competitive dynamics, brand changes, acquisitions, seasonality, and changes to internal site architecture. I am not asserting that any AI content tool directly caused any traffic outcome described in this piece. I am describing a correlation observed across many listed sites that share similar content patterns and organic traffic trajectories.

Third, vendors and specific domains are deliberately not named here. The pattern is the story, not the specific actors. Any resemblance to a specific company, vendor, or case study is incidental to the broader pattern described.

What The Data Shows: Rapid Growth Before A Steep Decline

If there is one thing the data makes clear, it is this: scaling content production with AI is not a low-risk strategy for organic search. It can produce real short-term gains in both SEO and AI search (LLMs use search engines), but across this dataset, those gains have rarely held. In many cases, the eventual loss has exceeded the initial peak.

Across the group of 220+ sites and subfolders I analyzed:

  • 54% lost 30% or more of their peak organic traffic.
  • 39% lost 50% or more.
  • 22% lost 75% or more.

Within those declines, a recurring trajectory appears: a rapid growth in organic pages over six to 12 months; an organic traffic peak within roughly three to six months of the content peak; and then a steep decline in traffic that erases most of the gain (and frequently drops below the prior baseline) within the following year.

Image Credit: Lily Ray

Most of these traffic drops took place after the case studies were published (which also makes me wonder whether the case studies themselves could be contributing to the declines). In the example below, the case study was published in January 2025, indicated by the the black star below:

Image Credit: Lily Ray

I am also continuously monitoring changes to organic page growth and organic traffic to these sites and subfolders over time. Looking at the updated data, a substantial number of these brands appear to have substantially reduced their content footprints in 2025 and 2026, often removing, redirecting, or 410’ing many of the same pages featured as success stories in published case studies. This could explain the recent drop in pages (yellow line) shown in the above screenshot (and potentially, the corresponding increase in organic search traffic).

In many cases, these case studies remain published to this day, but the pages they reference do not.

The Familiar Rank & Tank Playbook

When a site starts seeing traffic drops due to sitewide content quality issues, it’s rarely a gentle decline. As Glenn Gabe refers to it, a better label would be “Mount AI”: steep growth, followed by a similarly shaped drop-off in organic traffic, once Google’s systems have gathered enough signals to identify what is going on.

Below are several examples of case study sites that used AI to scale content creation and saw massive drops in organic traffic after their case studies were published:

Image Credit: Lily Ray
Image Credit: Lily Ray
Image Credit: Lily Ray
This site’s decline started during the unconfirmed “self-promotional listicle Google update” in January 2026, which I also wrote about on my Substack (Image Credit: Lily Ray)

This pattern is consistent across industries, including cybersecurity, travel, marketing, SaaS, healthcare, B2B services, crypto, and consumer goods, and it shows up across vendors.

The shape of the line in the chart is similar to trajectories we have seen among many sites affected by Google’s algorithm updates in recent years. It is the same boom-bust cycle the SEO industry has watched repeatedly in different forms, accelerated this time by the speed at which AI tools have enabled site owners to scale content.

The SEO Industry Just Went Through This

What is hard to overstate is just how recently the SEO industry watched a near-identical cycle play out. Many SEOs and site owners are still licking their wounds from a brutal round of Google updates and new spam policies that obliterated many sites’ traffic a few years back.

In September 2023, Google launched the Helpful Content Update, the most aggressive crackdown it had done in years against content that, according to its announcement, “feels like it was created for search engines instead of people.”

Roughly six months later, in March 2024, it followed up with the longest core update in Google’s history, which Google states was designed to “reduce unhelpful, unoriginal content in search results by 45%.” Across two consecutive update cycles, Google’s stated target was the same thing: content produced at scale, regardless of whether the production method was human, AI, or a combination of both.

Alongside the March 2024 update, Google formalized a new spam policy called “Scaled Content Abuse,” explicitly naming the practice it was working to suppress: generating many pages to manipulate search rankings, regardless of authorship.

The SEO industry is still working through the collateral damage from those updates, including significant losses for many small publishers, some of whom were publishing original, human-written content but used excessive SEO frameworks that the updates likely flagged. The casualty list also included some publishers who had partnered with ad networks and other emerging tools offering AI content creation and scaling as a service.

Many sites affected by the HCU haven’t recovered to this day, despite their enormous efforts. I spent significant time in 2024 working and speaking with many site owners trying to dig themselves out of that hole.

Having spent hundreds of hours analyzing and presenting about those two major updates, I can say that the content I am seeing published with many of these new AI tools often looks and feels a lot like the exact type of content that was wiped off the map with these 2023 and 2024 Google updates.

8 Recurring Content Patterns That Are Risky For SEO And AI Search

So, what types of content am I seeing published by companies using AI tools to build articles that I believe are ultimately risky for SEO? I believe the answer lies in page templates that aim to influence SEO rankings, AI search responses, and/or citations in AI search, but are highly formulaic and easily repeatable by competitors.

What starts as a genuine approach to try to build helpful content (and score a mention/citation) ends up being an easily detectable footprint by Google when enough sites are publishing similar pages, and the index becomes flooded with tens or hundreds of thousands of these similar pages, which is easier than ever to do using AI.

This is exactly what Google means when it talks about writing for search engines, not humans.

Reviewing top-traffic URLs across the declining domains, eight distinct content templates appear repeatedly. Most sites seeing declines in the analysis use some combination of at least three or four. The most aggressive ones use all eight. Typically, affected sites also have hundreds or thousands of these articles, which amplifies the problem and generally leads to steeper traffic losses.

1. Comparison Pages At Scale

Pattern: /blog/[product-A]-vs-[product-B] published at scale across most reasonable head-to-head matchups in a category. Observed across the dataset for product-vs-product pairings, framework-vs-framework pairings, and, in at least one case, concept-vs-concept pairings unrelated to the publisher’s actual business.

2. The “What Is X” Glossary

Single-term, single-question pages designed to be cited by AI engines. Pattern: /resources/what-is-[term] or /glossary/[term]. Observed across the dataset, including programmatic glossaries scaled across multiple languages from a single source template. Scaling translations with AI and without human review can also frequently lead to sitewide content quality issues.

3. The “Best [X] For [Y]” Listicle

The most familiar AI-content template, with origins in the affiliate-content era. This pattern was observed across the dataset in both broad-category and narrow-niche variants.

4. The Self-Promotional Listicle

A variant of No. 3 in which the publisher is itself a competitor in the category being ranked, and frequently lists itself as the No. 1 best among competitors. These pages generally lack real evidence that the company genuinely tested all of the competitors in the list, which is recommended by Google for review pages.

I wrote about this “listicle” page template causing SEO/GEO issues in February 2026, when I found that many companies publishing dozens, hundreds, or even thousands of self-promotional listicles saw extreme traffic drops beginning on the same day (approximately Jan. 21, 2026). This pattern was observed across multiple sites in the dataset, most aggressively in B2B services.

5. The Competitor-Vs-Alternatives Page

Pattern: /blog/[competitor-brand]-alternatives, or, in the more programmatic form, dedicated landing pages built for every named competitor in a category. This approach was observed extensively across the dataset, including one case where the majority of a site’s top traffic pages were dedicated to individual competitor brand names.

6. Programmatic Location And Language Scaling

This is one of the oldest tricks in the SEO book, and one that I’ve seen sites get in trouble for with algorithm updates for at least 10 years. The approach: Use one template multiplied across every geography or language a search engine will index, with very little unique content per local landing page.

In many cases, the company publishing these pages often does not have real brick-and-mortar locations in each of the neighborhood/city/state pages they are targeting.

This page type was observed across the dataset including state-by-state content, country-by-country service pages, and the multilingual programmatic glossaries described above.

7. The FAQ Farm

Each page answers exactly one question. Pattern: /faq/[full-question]. Designed for extraction by AI engines: a clear question in the URL, the answer in the first paragraph, bullet points in the body, schema markup at the bottom.

The problem? This approach creates a lot of low-quality content and baggage for the site when implemented at scale. Scaling FAQs was also observed extensively across the dataset, including in industries where the templated tone was a noticeable mismatch with the publisher’s brand context.

Here is a screenshot of my March 2024 Amsive article advising against the same exact thing:

Image Credit: Lily Ray

It’s also worth noting that just last week, Google announced it was deprecating FAQ Rich Results, which I believe might have something to do with this new influx of FAQ schema aimed at trying to earn citations and mentions in AI search.

8. Off-Topic Content Published At Scale

Publishing off-topic content, with no apparent connection to the publisher’s actual business, at high volumes, is one of the fastest ways to get in trouble with search engine algorithms. This was also a huge problem during the Helpful Content Update and March 2024 Core Updates, when many sites were experimenting with publishing off-topic content, like funny quotes, jokes, baby names, horoscopes, and other high-volume articles that weren’t actually topically relevant for the publisher.

This method was used across multiple sites in the dataset, including pieces on entertainment topics on a services platform, lists of names and jokes, social-media memes on B2B websites, and historical or biographical content on business-focused sites.

The Late January 2026 Unconfirmed Google Update

A secondary pattern appears in the data around late January 2026: a wave of sites with explicitly GEO-optimized, self-promotional listicles, plus other risky SEO/GEO approaches, saw organic traffic declines between 40% and 95% over the January-April 2026 window.

A large B2B company’s blog subfolder hit by the unconfirmed late-January 2026 Google update. (Image Credit: Lily Ray)

Google did not announce or confirm an update by name in January 2026, but at least 40 sites I identified saw a negative trend beginning around Jan. 20, 2026. In many cases, the impact was isolated to the company’s blog or other subfolder containing a lot of new SEO-driven content. My analysis found that some of these companies were scaling dozens, hundreds, or even thousands of these self-promoting listicles, in which they named their own company the No. 1 best when compared to competitors.

I suspect this adjustment on Google’s end was just the start of Google (and likely the LLM providers building on top of search) beginning to demote this type of content in search results, and it appears that the impact was greater than just the listicles themselves. For affected sites, the entire blog or subfolder containing these articles often also saw declines. In other cases, the impact was carried over across the full domain.

How To Use AI Content Tools Safely

I do believe there is a way to use AI content tools safely, and a way for these tools to support the creation of high-quality content. The tools themselves are not the problem, but the implementation can be. I believe these tools should be used and overseen by experienced SEO professionals who understand the landscape of content approaches that Google has grown extremely sophisticated at penalizing and demoting over the past 10+ years. The problem often stems from a “set it and forget it” approach, or when the goal is to scale as many pages as quickly as possible without human review.

Using AI content tools for research, organization, content briefs, pulling in proprietary company data and insights, and more can be invaluable for speeding up the content creation process. But when articles are simply published “for SEO/GEO” without consideration of the risks involved with search engine ranking systems, the well-intentioned content can actually backfire for both SEO and AI search.

To perform well, I recommend that any AI-assisted content should still demonstrate E-E-A-T, add original or unique information above and beyond what is offered by competing pages (information gain), and consider being transparent about the use of AI to create the content (which is recommended by Google).

The Bottom Line

If there is one takeaway from monitoring these 220+ sites over the past several months, it’s that the playbooks being sold as “AI-first SEO” or “GEO-optimized content at scale” look remarkably similar to the playbooks that got sites flattened by the Helpful Content Update and the March 2024 Core Update. The packaging is new, but the pattern is not.

Across the dataset, the brands still growing are generally the ones whose content does not match the eight templates above. Many brands that scaled into those templates are the ones now removing pages, redirecting subfolders, and taking other steps to try to mitigate recent losses in traffic.

If you’re currently evaluating an AI content vendor, or running a program in-house, here are a few practical questions I think are worth asking before publishing another page:

  • Does this page actually exist because a real customer or reader needs it, or because a search engine or LLM might cite it?
  • Could a competitor publish a near-identical version of this page tomorrow using the same prompt?
  • Would I be comfortable if Google, a journalist, or my own customers saw the full list of URLs in this subfolder?
  • Is the article inherently biased, and if so, is the page transparent with users about those biases?
  • Is there any first-party data, expertise, or original perspective on this page that isn’t available on the first ten results already ranking for the query?

None of this means AI content tools are unusable. They can be genuinely useful for research, briefs, internal data synthesis, and accelerating workflows where a human expert is still in the loop. The trouble starts when the goal becomes volume, or when the people closest to the content stop reviewing what is going out the door.

The SEO industry has already lived through this cycle once in the last few years. The sites that came out of it best were the ones that prioritized quality, originality, and topical focus over scale. I expect the same to be true of this cycle, and I’ll keep tracking the data as it plays out.

The State of AEO/GEO Report Conductor 2026

More Resources:


This post was originally published on Lily Ray NYC Substack.


Featured Image: Stokkete/Shutterstock

Anthropic’s Infrastructure Crisis – What It Means for Marketers & SEO Pros via @sejournal, @gregjarboe

On May 6, 2026, Anthropic CEO Dario Amodei walked out onto a stage at his company’s developer conference in San Francisco and said something you almost never hear from a tech CEO: Growth is the problem.

Anthropic had planned for a 10-fold expansion. What it got was 80-fold growth in Q1, on an annualized basis. Revenue has crossed $30 billion, up from $9 billion at the end of 2025. The company is weighing a funding round at a reported $900 billion valuation – which, if it closes at those terms, would likely surpass OpenAI’s most recent post-money valuation of $852 billion. And yet, as Amodei told the audience that day, “I hope that 80-times growth doesn’t continue because that’s just crazy and it’s too hard to handle.”

He wasn’t being falsely modest. Demand for Claude has already created what Anthropic described as “inevitable strain on our infrastructure,” hitting reliability and performance during peak hours. Hours before Amodei took the stage, the company announced a deal with SpaceX – which, earlier this year, merged with xAI, the company behind the Grok AI models, now rebranded SpaceXAI – to take over the entire compute capacity at the Colossus 1 data center in Memphis, giving it access to more than 300 megawatts of capacity and 220,000 Nvidia GPUs.

The detail worth noting: xAI and Anthropic are direct competitors at the model layer. The fact that Grok’s infrastructure is now running Claude’s workloads is the clearest signal yet of how constrained high-end compute capacity has become. That’s a bridge built under emergency conditions, not a planned expansion.

So, why should SEO professionals, content marketers, and entrepreneurs care about Anthropic’s infrastructure problems? Because this story is actually about something much bigger than one company scrambling for server capacity.

This Has Happened Before

In 2011, I read I’m Feeling Lucky: The Confessions of Google Employee Number 59 by Douglas Edwards, who was Google’s first director of marketing and brand management. That’s when I learned how close Google came to buckling under its own success in the early days.

In late 1999, Edwards wrote, “Google began accelerating its climb to market domination. The media started whispering about the first search engine that actually worked, and users began telling their friends to give Google a try. More users meant more queries, and that meant more machines.” Then the machines became impossible to get. A global shortage of RAM hit at the worst possible moment, and Google’s system, as Edwards put it, “started wheezing asthmatically.”

That infrastructure crisis drove decisions that shaped the web for the next two decades. Google started filtering duplicate content – even non-malicious versions like printer-friendly pages – because every redundant page required adding hardware without improving user experience. The constraint shaped the product. The product shaped SEO.

Anthropic’s compute crisis is the same dynamic, playing out 25 years later at a different scale. The question isn’t whether they’ll solve it. They will. The question is what decisions they’ll make under pressure, and how those decisions will reshape the products that millions of marketers depend on.

What The Data Actually Shows

When I went looking for what this growth moment means for practitioners, I found the headlines and the data pointing in surprisingly different directions.

Rand Fishkin recently shared findings from the Datos State of Search Q1 2026 report, which draws on clickstream data from tens of millions of real devices. His summary was pointed: AI is disrupting traditional search – no, the data doesn’t show that. AI tools are growing faster than traditional search in absolute terms – no, traditional search is still outpacing AI tool growth on an absolute basis. AI Mode in Google is huge – no, it’s still under 0.2% share, growing but still small. ChatGPT is pulling away from Claude – actually, no. Claude is closing the gap, Gemini holds the number two spot and is growing, and ChatGPT has plateaued since September 2025.

These are not the narratives that get clicks. They are, however, what the data says.

At the same time, I went to Think with Google and worked through its report, “The Rise of the Super-Empowered Consumer,” which tells a different part of the same story. Some of what’s in there deserves more attention than it’s getting. AI Overviews is used by over 2 billion people, and users report making decisions faster and with more confidence. AI Mode now has over 75 million daily active users, with nearly 1 in 6 queries using voice or images. Queries in AI Mode run three times longer than traditional searches, and sessions are becoming more conversational. Google Lens handles over 25 billion visual searches every month. Shoppers are 2.3 times more likely to use Google Search than ChatGPT for purchase decisions, and 40% of consumers who use Google AI Mode for shopping say they’re using ChatGPT less as a result.

Two different pictures of the same moment. Both accurate. Neither is complete on its own.

The Takeaway For Practitioners

The AI industry is generating a firehose of information, and most of it gets consumed at the headline level. A company announces 80-fold growth, and people read it as a story about AI winning. Fishkin publishes data showing traditional search still outpacing AI tools in absolute volume, and people read it as a story about AI losing. Google publishes a consumer report showing AI Overviews reaching 2 billion users, and people read it as confirmation that SEO is dead.

None of those readings are wrong. All of them are incomplete.

The strategic value isn’t in reading the news. It’s in following the thread further – downloading the Datos report, working through the Google consumer study, checking the CNBC article against the Cryptopolitan analysis of what the Anthropic-SpaceX deal actually signals about the infrastructure war playing out between the major AI companies.

Google’s early infrastructure crisis produced lasting decisions about duplicate content that practitioners are still navigating. Anthropic’s current one will produce decisions about rate limits, model availability, enterprise pricing, and compute allocation that will shape how Claude-powered tools perform for the marketers and developers using them. Those decisions are already being made.

The practitioners who understand the context those decisions come from will be better positioned than those who only read the headline.

More Resources:


Featured Image: Anton Vierietin/Shutterstock