--- title: "PLG for the Agent Era, Part 2: Is Your Site Ready for AI?" description: "SEO got you found for 25 years. But when an AI agent lands on your site to evaluate your product, can it actually read what you sell? Five new benchmarks measure agent-readiness, and even the companies writing them don't score well." date: 2026-06-03 author: Kat Laszlo canonical: https://tansohq.com/blog/is-your-site-ready-for-ai --- # PLG for the Agent Era, Part 2: Is Your Site Ready for AI? By [Kat Laszlo](https://www.linkedin.com/in/katrinalaszlo/) · June 3, 2026 In Part 1, I showed that 73% of B2B buyers already use AI for vendor research, and 69% end up choosing a different vendor than they planned. Agents are shaping who wins. The question is whether they can actually read your site. --- ## SEO got you found. Can agents read what they find? For 25 years, SEO was how you got discovered. Meta tags, sitemaps, clean URLs, schema.org markup. AI agents still use all of that (sitemaps and structured data show up in every readiness benchmark), but a new layer is forming on top. Agents also care about things that didn't exist two years ago. Whether your content is token-efficient, whether your server can respond with markdown, and whether your capabilities are exposed programmatically. This new layer is called AEO, or Agentic Engine Optimization. Whether it matters yet is an open question, but the frameworks are here. --- ## What agents actually look for The most basic new standard is `llms.txt`, a manifest file like robots.txt but for LLMs. It links to your most important pages in a format optimized for context windows. Short descriptions, direct URLs, organized by topic. Where robots.txt tells crawlers what to avoid, `llms.txt` tells agents what to read first. `AGENTS.md` goes further, declaring what an agent can *do* on your site. Permissions, capabilities, tool endpoints. If `llms.txt` is a table of contents, `AGENTS.md` is an instruction manual. Then there's **markdown serving**. HTML boilerplate (nav bars, cookie banners, JavaScript) burns tokens. Markdown is 3–5x more efficient. If your server can detect an agent via `Accept` headers or User-Agent strings and respond with markdown instead of HTML, the agent gets to your actual content faster and cheaper. Finally, **capability signaling**. Do you expose `agents.json`, MCP endpoints, or machine-readable manifests? These tell an agent what it can do programmatically on your site. That's the difference between "I found this product" and "I can try this product." --- ## Five benchmarks, five opinions The AEO space is already fragmenting. Five major benchmarks have emerged, and each checks different things. | Framework | What it checks | | --- | --- | | agentic-seo | Discovery, content structure, token economics, capability signaling, UX bridge | | Cloudflare | Discoverability, bot access, API/MCP/A2A protocols, commerce readiness | | Fern | llms.txt quality, link resolution, markdown parity, content negotiation | | Vercel | Agent reachability, discoverability, markdown serving, HTML friendliness | | AgentGrade | MCP, payment protocols, identity, content negotiation, OpenAPI | Every framework checks for `llms.txt`, but only Fern validates whether its links actually resolve. Cloudflare and AgentGrade check for MCP endpoints; agentic-seo doesn't. Vercel cares about markdown serving; AgentGrade cares about payment protocols. No single benchmark covers everything. --- ## Measuring it: one scan, every score Running all five benchmarks manually, across different scoring scales, is tedious enough that nobody does it. So I built `aeo-ready` to run them in parallel and prioritize the most common recommendations. When I first scanned tansohq.com, the site had solid traditional SEO (robots.txt, sitemap, schema.org markup) but almost nothing for the agent layer. The agentic-seo benchmark scored 24 out of 100. Content structure, token economics, and capability signaling were all zeros. The scanner prioritizes recommendations by how many benchmarks flag them. The first pass handled the highest-leverage fixes. Adding `llms.txt` and `AGENTS.md`, configuring markdown content negotiation in middleware, and expanding `llms.txt` coverage to include blog posts and tools. That alone moved agentic-seo from 24 to 90. The second pass went deeper. Creating `.md` files for all content pages, adding agent User-Agent detection to middleware, injecting `` across every HTML page, and adding MCP server card and API catalog endpoints. Current score: 86 out of 100 across all five benchmarks. The remaining failures are things I chose not to fix. Optional identity protocols (WebFinger, DID documents), infrastructure standards that don't apply to a content site (payment headers, structured JSON errors), and a handful of markdown parity issues. The easy wins for any site are the discovery files (`llms.txt` and `AGENTS.md`) and markdown support (serving pages as `.md` cuts token cost by 3–5x). --- ## How the big sites score If your score feels low, context helps. Even the companies writing these standards don't score well on them. Sometimes not even on their own frameworks. | Site | agentic-seo | Cloudflare | Fern | Vercel | AgentGrade | | --- | --- | --- | --- | --- | --- | | tansohq.com | 90 | 5/5 | 84 | 73 | 81 | | Stripe | 17 | 1/5 | 83 | 68 | 65 | | Cloudflare | 20 | 3/5 | 85 | 85 | — | | Supabase | 20 | 4/5 | 84 | 71 | 83 | | Vercel | — | 2/5 | 74 | 58 | 59 | Scanned June 3, 2026 via aeo-ready. Stripe leads on Fern but scores a 17 on agentic-seo. Cloudflare gets 3/5 on its own framework. Vercel scores a 58 on its own spec. If the companies defining these standards haven't optimized for them, it raises the obvious question. --- ## Does any of this actually matter? The honest answer: it depends on what you're optimizing for. ### The case against If you're hoping `llms.txt` will boost your visibility in ChatGPT or Perplexity results, the data isn't encouraging. An analysis of 62,000+ AI bot requests over 90 days found that only 84 went to `llms.txt` ... 0.1% of AI bot traffic. Google's John Mueller has called markdown pages "a stupid idea." No major AI provider has publicly committed to reading `llms.txt` for search citations. Only 10% of sites have adopted it after 18 months of industry conversation. The frameworks themselves have problems. Cloudflare's scanner includes commerce and API checks by default, so content sites score artificially low. Vercel's own evaluation found that `AGENTS.md` outperformed skills in agent tasks, and skills weren't even invoked 56% of the time. The model just didn't bother to look them up. There's also a site-type problem. A content site loses points for missing payment protocols, and an API product loses points for missing blog-style markdown. The scores mix up "not relevant" with "not ready." ### The case for AI search citations and agent workflows are two different things. AEO probably won't help agents *find* you, but readability matters once they arrive. Cloudflare's own testing showed that agent-optimized docs consumed 31% fewer tokens and got correct answers 66% faster. When an agent is evaluating your product for a buyer, that's the difference between a clear recommendation and getting skipped. Forrester's 2026 Buyers' Journey Survey of 18,000 buyers found that AI and conversational search are now the most meaningful source of vendor research, outranking vendor websites and sales reps. 55% of buyers use AI to compare vendors. When those tools compare you against competitors, they're parsing structured data, not reading marketing copy. A landing page that persuades humans won't show up in a vendor analysis report if an agent can't extract the pricing, specs, and capabilities. Meanwhile, the same `llms.txt` doing nothing for ChatGPT search is doing real work in the agentic layer, where Claude Code, Cursor, and Windsurf fetch context and choose tools on behalf of developers. Anthropic explicitly recommends `llms.txt` in its Writing for Agents guidance. Chrome's Lighthouse 13.3 added an Agentic Browsing audit that checks for it. Netlify reports that 80% of new signups are now agents. --- ## My take AEO won't help agents find you. But when one lands on your site to evaluate your product, readability determines whether it recommends you or moves on. The bar is low: only 10% adoption after 18 months. The cost is near zero: the discovery files take an afternoon, markdown support takes a day. And the evidence says readability matters once agents arrive. If you're building a developer-facing product, being in the top 10% on agent readiness positions you for a web that's still forming. If you're a local bakery, skip it. --- ## What's next Whether or not AEO drives discovery today, the next question is what happens when an agent does reach your product. How agents find and evaluate your product, what they look for, and how to measure whether you're showing up. What it actually takes to let an agent sign up, try your product, and buy it, without a human touching a browser. Onboarding, auth, purchasing, and account management for the agent web. *[Kat Laszlo](https://www.linkedin.com/in/katrinalaszlo/) is co-founder of Tanso. This post also appears on [katrinalaszlo.com](https://katrinalaszlo.com/blog/is-your-site-ready-for-ai).* ### Want to price with confidence? Book a quick call and we'll walk you through it.