The llms.txt and Schema Checklist: What Your Site Actually Needs
What to check in robots.txt, structured data, and your sitemap, plus what llms.txt can and cannot do for search.
If we’re working on your site’s SEO, I want to know that people and search tools can reach the useful pages. That’s where I start.
A special file isn’t going to make up for a site that barely explains what your business does.
Check that search crawlers can get in
Open your robots.txt and check the rules. Also check that your hosting or firewall isn’t blocking the crawlers you want to allow.
One important distinction: search crawling and AI training are different things. OpenAI uses OAI-SearchBot for search and GPTBot for training. You can make separate choices about them. OpenAI explains those controls here.
For Google Search, check Googlebot access, whether the page is indexable, and your Search Console settings. Don’t assume that changing Google-Extended turns Google search visibility on or off.
I’d also check:
- Important pages return a normal success response.
- There isn’t an accidental noindex tag.
- Each page has a sensible canonical URL.
- Navigation and other useful pages link to it.
- The XML sitemap lists the pages you actually want indexed.
- The important words are in the page, not just inside an image.
Is llms.txt worth adding?
I keep a short one on this site. It’s an optional summary for systems that choose to use it.
But let’s be clear about its limits. Google says llms.txt does not improve or hurt Google Search visibility. It does not replace your pages or your sitemap. That’s in Google’s own guidance.
If you have one, keep it accurate. Outdated prices, missing services, and old project links don’t become useful just because they’re in a text file.
Use structured data that fits the page
Structured data describes information in a consistent format. I use it to make the site’s business, services, articles, and navigation explicit.
The details still need to be true and visible on the page.
Business and service information
Use the type that fits the business. For example, Organization can describe the company, while Service describes a particular service. Don’t invent a public street address or reviews just to fill a field.
Keep names, service areas, contact details, and offers consistent with what customers can actually read.
Articles and authors
For a guide, I want the title, original publication date, any genuine update date, and the author to match the visible page. The author should link to a useful profile. Google’s article documentation explains the supported properties.
Changing a date on every rebuild doesn’t make an old article new.
What about FAQs?
Useful answers still belong on a site. But Google discontinued FAQ rich results in May 2026. Don’t buy a service on the promise that FAQ markup will put extra answers under your search listing. See Google’s documentation updates.
I keep FAQs because customers ask the questions, not because they are a shortcut.
Check the result after a rebuild
I’d validate the markup, compare it with the visible page, and check the URLs again after a redesign or migration. The Rich Results Test checks supported Google features; it isn’t a complete SEO audit.
My free AI search grader can flag some basic setup issues. Its score is a checklist score, not a prediction of whether an AI will recommend you.
If the technical side is a mess, send me the site. We can work through it. The full AI search guide covers the content side too.
Written by Ian
Keep reading
-
Google AI Overviews: How to Show Up in the Box That Is Eating Your Clicks
The AI summary at the top of Google is taking clicks that used to reach your site. Here is how it picks sources, and how to get cited inside it.
-
GEO vs SEO: What Actually Changes When the Answer Replaces the List
Generative engine optimization is the acronym everyone is selling. Here is what genuinely changes, what stays the same, and what is repackaged hype.
-
Why ChatGPT Recommends Some Businesses and Ignores Others
Practical checks for your business information, search access, and online reputation when exploring how you appear in ChatGPT search.
Want this handled for you?
Send me your site and tell me what you want to improve. I'll take a look and we can talk about where to start.
Book a free call