AIScan Site

AI agent readiness scanner ·

[AIScan Site @AIScanSite]

Paste a link in ChatGPT, Claude or Mistral and the fetch never shows in Google Analytics. A 23-tool test found it only in server logs.

Filter for ChatGPT-User, Claude-User, MistralAI-User, and keep key content in HTML: none ran JS.

#AIReadiness #TechSEO
https://t.co/AFUVXW5Pis

[AIScan Site @AIScanSite]

Google is opening Search and Gemini to outside apps. Its Connected Apps Platform plugs in via a cloud-hosted MCP server, registered in Google Connected Console. US preview, waitlist.

Run a service? Your MCP server is now a front door.

#AIReadiness #MCP
https://t.co/PUb673nKzH

[AIScan Site @AIScanSite]

Agents that send Accept: text/markdown can get Markdown from Cloudflare's edge while browsers still get HTML.

Flip one zone setting or add a rule for one path. Then test: a missing page must not return a Markdown 200.
https://t.co/9VJn4Kvzea

#AIReadiness #Cloudflare

[AIScan Site @AIScanSite]

A WordPress developer's own site scored C, 55/100 in AIScan.

Solid basics: 1,133 words server-rendered, JSON-LD, sitemap, RSS feed. The gap is agent-specific: robots names 0 AI bots, there's no llms file, and markdown requests get HTML.

See the scan: https://t.co/ShjxwBuXnw

[AIScan Site @AIScanSite]

One h1, a title, a meta description, one JSON-LD block, all in the HTML your server returns.

Ghost writes three for you. Lovable hands all four to an audit panel. Replit's SEO rating only half checks them. Who writes what:
https://t.co/DhlAYj6TWW

#AIReadiness #TechnicalSEO

[AIScan Site @AIScanSite]

A no-cookie "what is my IP" tool has held A+, 100/100 in AIScan across public scans since September 1.

What keeps it there: rules for 15 AI bots, markdown on request, a Link header to its OpenAPI and API catalog, plus a JSON feed.

See the scan: https://t.co/qqaDl2MWLr

[AIScan Site @AIScanSite]

A freelance web developer's portfolio scored F, 29/100 in AIScan.

It has 778 words of server-rendered text. But the robots and llms files both 404, there's no sitemap, and markdown requests get HTML. Agents find words, then a dead end.

See the scan: https://t.co/bzVN9qkYpW

[AIScan Site @AIScanSite]

JSON-LD in a Google Tag Manager tag? A plain HTTP fetch of your page finds zero schema blocks. Googlebot renders it; crawlers that skip JavaScript see nothing.

Move it into your template, about 10 min:
https://t.co/dzw5DpcqzQ

#AIReadiness #StructuredData

[AIScan Site @AIScanSite]

Google's favicon doc, updated Oct 27: Googlebot must crawl your home page and follow any redirects to the page that sets the favicon, and Googlebot-Image must fetch the icon. Block none of them.

Check your redirects and robots rules.

https://t.co/c13eUs56pf

#AIReadiness #SEO

[AIScan Site @AIScanSite]

A crawler naming Roblox in its user agent took 145.5 GB of 3D files from Open3DLab's sites, which say it ignored their robots rules and terms.

Our take: robots rules are a request, not a lock. Enforce blocks at the firewall.

https://t.co/MCoNt7nsfv

#AIReadiness #AICrawlers

[AIScan Site @AIScanSite]

Walmart set rules for AI shopping agents: identify yourself, and the fast lane is a beta MCP gateway. @eric_seufert says big platforms favor their own agents.

Most stores won't get a gateway. Agents will read your pages, so make them readable.

https://t.co/SOYxljChZQ

[AIScan Site @AIScanSite]

Agree with @jonoalderson: a separate "LLM version" of your site is cloaking with extra steps.

Make the page people already see readable to AI crawlers: allowed in robots rules, real text in the HTML, not hidden behind JS.

Check yours: https://t.co/vpLFuN16zZ #AIReadiness

[Jono Alderson @jonoalderson]

Cloaking is back in SEO fashion. Everyone's busy serving dedicated content to LLMs.

Yawn. Know what's more interesting? Cloaking Google using nothing but CSS.

P.S. Don't try this at home.

https://t.co/xywhqvaj6B

[AIScan Site @AIScanSite]

Redirecting dead URLs to your homepage doesn't fix a 404. It creates a soft 404.

A made-up URL must return 404, with a body that names your site and links somewhere. Fixes for Astro, Lovable, Webflow, Wix and more:
https://t.co/EPD5kAje8p

#AIReadiness #TechnicalSEO

[AIScan Site @AIScanSite]

A free normal map generator on Next.js scored C, 51/100 in AIScan.

The content is solid: 3,860 words server-rendered, a 16 KB llms file and schema markup. But robots names 0 AI bots, and plain http doesn't upgrade to https.

See the scan: https://t.co/jSAMj1TkhW

[AIScan Site @AIScanSite]

A random ETag is worse than none. It looks present, so a presence check passes, and no crawler ever gets a 304.

Hash the final body, honour If-None-Match, return an empty 304. Tested on nginx, Apache, Node, PHP and Cloudflare:
https://t.co/E1iH7mla3B

#AIReadiness #WebPerf

[AIScan Site @AIScanSite]

An AI video studio for concept teasers went from A, 82/100 to A+, 95/100 in AIScan.

What moved it: markdown on request for agents, plus a Link header that points them to its llms file and sitemap. It has held A+ since early October.

See the scan: https://t.co/lpGBsNp8nT

[AIScan Site @AIScanSite]

A developer's travel portfolio, built in Vue, scored F, 29/100 in AIScan.

Without JavaScript, AI crawlers see just 7 words. Robots names no AI bots, there's no llms file, and markdown requests get HTML.

See the scan: https://t.co/ThJgwYA4Qm

[AIScan Site @AIScanSite]

61.6% of the "AI crawler" hits in our logs came from 22 sources rotating 10–15 bot names. Not one got a page.

If GPTBot shows up at 3 a.m., the user agent is the least reliable thing about it.

63 days of our logs:
https://t.co/ATGLUPFU6d

#AIReadiness #AICrawlers

[AIScan Site @AIScanSite]

New in Google's spam guide (Oct 26): "Only index content meant for Search." Tools that publish shared pages or generated web apps should noindex link-shared pages by default.

On an app builder? Check what Search can index.

https://t.co/Pc5pNkGCN5

#AIReadiness #SEO

[AIScan Site @AIScanSite]

AI Overviews now show on 80%+ of Google searches, up from ~50% in June 2025 (Semrush, SE Ranking, seoClarity data via @rustybrick).

Our take: the summary comes first now. Keep key facts in plain, crawlable HTML.

https://t.co/DDNGr80vNX

#AIReadiness #AIOverviews