Superintelligence Doesn't Read Sitemaps: How Onchain Diary Got Discovered by AI Engines

Superintelligence Doesn't Read Sitemaps: How Onchain Diary Got Discovered by AI Engines
TL;DR

On September 29 the White House signed an executive order retiring 'artificial intelligence' from the federal vocabulary — agencies will now say 'Super Intelligence' or 'SI,' and the government 'will not acknowledge' the old terms. We're less interested in the naming fight than in the question underneath it: how do these systems actually discover things? Our Web3 safety site sat in Google's 'crawled, not indexed' limbo for months — 272 URLs submitted, 26 clicks in four weeks — while ChatGPT and Perplexity traced it, read it, and cited it, without us submitting anything. This post is the full playbook: a two-tier llms.txt, an MCP server built from static JSON shards, and a JSON-LD entity graph that stitches our sites into one machine-readable entity. Roughly one day of work.

On Tuesday, September 29th, the White House signed an executive order with a title that sounds like a paperback from the airport rack: Inaugurating the Era of Super Intelligence. The substance is a rebrand. Federal agencies are directed to use “Super Intelligence” — “SI” for short — everywhere they used to say “artificial intelligence”: official sites, reports, press releases, policy documents. The order instructs the executive branch to “not acknowledge the usage of” AI, and starts a 60-day clock to draft a federal definition of the new term. There was a CEO lunch. “The word super is the best word of all, and it’s the simplest,” the President explained.

We’re not a policy shop, and this post isn’t about the order. What struck us is the symmetry. Washington spent a week arguing about what to call these systems. We spent the same week documenting something more consequential: how these systems — whatever you call them — actually find things. Because they don’t read sitemaps. And that quietly rewrites what “discoverable” means.

Here’s the traffic report that started it.

The site is Onchain Diary, our Web3 safety education property: 106 English articles, 43 Chinese articles, 232 glossary terms, bilingual, all original. We did everything the SEO textbooks say. Canonical URLs. Sitemaps submitted to Search Console. Clean HTML, fast pages, structured data, internal links. Google’s response: months in “crawled — not indexed” limbo, 272 URLs submitted, 26 clicks in four weeks.

Meanwhile, something odd was happening in chat windows. Ask ChatGPT about onchaindiary and it didn’t just find the site — it traced from the site to the founder’s personal page, from there to UZEN Labs, and from there surfaced our entire product line. Nobody submitted anything to anybody. No sitemap ping, no indexing API, none of the rituals. The machines found us on their own, along a path we hadn’t consciously built.

That contrast is the whole story of this post. The gatekeeper changed, and most of the playbook we’d internalized didn’t transfer.

What traditional SEO misses now

Classic optimization assumes a crawler with a queue. You feed it sitemaps, you sculpt meta tags, you earn backlinks so it ranks you higher. That still works — for crawlers. Bing sends us real traffic every week; Google is the one sulking.

But answer engines — the chat surfaces where more and more questions now start — behave differently in three specific ways:

  1. They walk entities, not links. A language model resolves “onchaindiary” as a thing in the world: who runs it, what it publishes, what else that entity is connected to. The citation follows the entity, not the URL.

  2. They read interfaces, not pages. A chat assistant answering a question will happily call an MCP endpoint and query your content directly. It never renders your beautifully optimized H1.

  3. They cite verifiable factual blocks. Definitions, step-by-step procedures, numbers with sources. Not landing pages. Your content gets quoted when it’s quotable, in the structural sense.

None of that is served by sitemap files. So we built for the machines that actually read us.

Layer one: llms.txt, generated at build time

The simplest piece. Two plain-text manifests served at the root:

  • /llms.txt — a compact index: what the site is, what’s in it, the key entry points. Roughly 50 links.
  • /llms-full.txt — the full inventory. Every article, every glossary term, every hub page, grouped by section.

The detail that matters: both are generated at build time from the same content collections that render the site. There’s no separate file to maintain, so the manifest can never rot. Publish an article, and the manifest updates in the same deploy. We’ve all seen the llms.txt files that were hand-written once in a burst of enthusiasm in 2025 and never touched again. That failure mode is structural, and build-time generation deletes it.

Layer two: an MCP server made of static JSON

This is the layer that felt like a superpower.

The idea: instead of hoping an AI assistant can parse our HTML, give it a proper query interface. We already knew MCP — Model Context Protocol, the emerging standard for letting assistants call external tools. So we built one for the site.

The architecture is deliberately boring:

  • At build time, a script compiles all content — every article and glossary term, both languages — into static JSON shards. 316 items at last count.
  • A Cloudflare Pages Function serves the MCP protocol on top of those shards. Four tools, all read-only: search, read_article, read_glossary, list_content.
  • No authentication. No database. No server to keep warm. It’s a static file with a protocol accent.

The cost profile is the point. The shards are built once per deploy and served from the edge like any static asset. The function is thin — protocol translation, nothing else. There is no per-request infrastructure bill, no cold-start anxiety, and nothing to monitor. If the site is up, the MCP server is up.

And the payoff is asymmetric. An AI assistant that finds the endpoint can answer questions from your content — search the glossary, pull an article, cite the source — without scraping a single page. You’ve gone from “parseable” to “queryable.” That’s a category jump, not an increment.

Layer three: the entity graph

The quiet layer. The one that turned out to matter most.

Every site in our portfolio carries a JSON-LD block, but the blocks aren’t isolated — they’re stitched into one graph. The Organization node for Onchain Diary declares:

  • an @id anchored on the main domain,
  • founder pointing at the founder’s personal site,
  • parentOrganization pointing at UZEN Labs,
  • a sameAs array linking the Paragraph publication, the X account, the GitHub repos, the short-link domain — every surface where this entity also lives.

The two domains do the same dance in reverse. The machine-readable takeaway an AI engine extracts is unambiguous: these are not two sites that mention each other. They are one entity with several addresses.

That’s what ChatGPT walked. Site → founder → lab → product line. Each hop was a sameAs or a relationship edge we’d declared in a few kilobytes of JSON. When people say “entity SEO,” this is the concrete version: not a buzzword, just graph edges, declared once, readable forever.

We did not invent this technique. What surprised us was the ratio — the discovery chain cost an afternoon of JSON-LD, and it’s the layer the AI engines demonstrably followed.

What the probes showed

We run periodic probes — same brand questions, across the engines, logged in a spreadsheet. The results after the stack went live:

  • ChatGPT resolves onchaindiary and traces the full entity chain — site, founder, lab, portfolio — in one answer.
  • Perplexity finds and cites the content directly.
  • Google, unchanged. Still sulking. (Bing, for the record, remained the top referrer among classic engines the whole time.)

The honest framing: we did not “win SEO.” We won something adjacent and increasingly valuable — the definition of who we are, inside the answers machines give when nobody clicks a result at all.

What we learned

sameAs is the new backlink. A backlink tells a crawler your page matters. A sameAs declaration tells a language model that two things are one thing. As engines resolve entities instead of ranking pages, the equivalence edge does work a hyperlink can’t.

MCP is RSS for machines. RSS let content travel beyond the site twenty years ago. MCP lets content be queried beyond the site today. If you published an RSS feed because you wanted to be readable everywhere, the logic for publishing an MCP endpoint is identical — the consumers just changed.

Static beats dynamic on AI surfaces. Build-time generation did more for AI discoverability than any runtime cleverness. The manifests can’t drift, the shards can’t 500, and every deploy ships a consistent snapshot of everything.

One channel sulking is no longer a verdict. The months-long index limbo would have been a death sentence under the old playbook. With Bing sending steady traffic and AI engines tracing the graph, Google’s silence went from existential to annoying. Diversification used to mean social and search. Now it means search, answers, and entities.

The whole stack is embarrassingly cheap. Two build-time generators. One thin function. A few JSON-LD blocks. A day of focused work, and the entity graph was the hardest part — not because it’s technically hard, but because you have to actually decide what your entity is.

The copy-paste part

When we rebuilt the short-link domain onchaindiary.org on October 1st, the entire stack shipped on day one — the entity graph, the sitemap, the llms.txt tier, the MCP proxy. Not because the relaunch needed it, but because this is now our default factory settings for any new property. It’s cheaper than remembering to add it later.

The playbook, compressed: manifest for the readers, endpoint for the askers, graph for the resolvers. Then leave the sitemap for whoever still wants it.

A postscript on names. By late November, the 60-day clock in the executive order runs out, and the U.S. may have a federal definition of “Super Intelligence” on the books. Maybe the industry adopts the rebrand. Maybe everyone keeps saying AI, the way everyone kept saying “weblog” right up until nobody did. From where we sit, the acronym is the least durable part of the stack. The constants are the machines that answer questions, the entity graphs they walk, and the interfaces they query.

llms.txt doesn’t care what you call it.

Frequently Asked Questions

What is the 'Super Intelligence' executive order?

Signed September 29, 2026, the order directs U.S. federal agencies to use 'Super Intelligence'/'SI' instead of 'artificial intelligence'/'AI' in official materials, and starts a 60-day process to draft a federal statutory definition. It doesn't require private companies to rename anything — for now, SI in a federal document means what the rest of the industry still calls AI.

Does this replace traditional SEO?

No. Google and Bing still send the majority of search traffic, and crawlers still need sitemaps, canonicals, and clean HTML. The AI layer is additive: it wins you citations inside chat answers, which increasingly precede the click. We run both stacks in parallel.

What is llms.txt?

A plain-text manifest, served at /llms.txt, that lists your site's content for AI engines and answer engines — think robots.txt, but written for language models instead of crawlers. We serve two tiers: a compact index and a full inventory, both generated at build time so they can never drift from the actual site.

How does the MCP server work?

At build time, every article and glossary term is compiled into static JSON shards — 316 items at last count. A Cloudflare Pages Function serves the MCP protocol on top: four read-only tools (search, read_article, read_glossary, list_content), no auth, no database, no cold-start penalty. Any AI assistant that speaks MCP can query the entire site.

What did it cost to build all this?

The llms.txt generator is two build-time scripts. The MCP server is one Pages Function over static files. The entity graph is a few JSON-LD blocks. All of it rides infrastructure we already paid for. The honest total is one focused day of work, most of which went into the entity graph.