Schema Markup, llms.txt, and AI Crawlers: What a Local Business Needs to Know

The short answer: Make sure AI search crawlers can reach your site, put your key content in plain HTML, and add accurate structured data (schema markup) for your business, services, and FAQs. An llms.txt file is optional: it's a proposed convention, not an official standard, and there's little proof that major assistants rely on it. Decide deliberately which AI bots you allow, and don't block the ones that let you be found.

Why technical access comes first

An assistant can't recommend what it can't read. Sometimes the reason a business never shows up is mundane: a security setting blocks AI crawlers, a robots.txt rule is too broad, or the important content only appears after JavaScript runs. Content quality and reviews won't help until the door is open.

Think of three layers:

1. Access: can crawlers reach your pages?
2. Readability: can they read the content once there?
3. Meaning: can they tell what the content is about, and who you are?

Layer 1: Access (robots.txt, firewalls, and AI bots)

Your robots.txt file tells crawlers which parts of your site they may visit. AI companies run different bots for different purposes, and it helps to understand the difference:

• Search and retrieval bots fetch pages so an assistant can show or cite them in answers. Blocking these can reduce the chance you're named or linked.
• Training bots collect content to train future models. Whether to allow these is a business decision about your content, and it's separate from being found in search-style answers.
• User-triggered fetchers visit a page when a person asks an assistant to read it.

Bot names and behavior change, so check each provider's current documentation before editing. In general, a local business that wants to be recommended usually wants search and retrieval bots allowed. Decide about training bots separately.

Also check places other than robots.txt:

• Firewalls, CDNs, and security plugins that block unfamiliar bots by default.
• Login or cookie walls that hide content.
• Server logs. They show whether AI bots actually visit. Ask your web host or developer to check.

Layer 2: Readability

Many AI crawlers don't run JavaScript the way a browser does. If your services, prices, or service area load only after scripts run, a crawler may see an empty page.

• Put key text in the HTML itself: service descriptions, areas served, licensing, contact details, and FAQs.
• Use real headings and short paragraphs.
• Avoid placing essential information only in images or PDFs.
• Keep pages fast and mobile-friendly.
• Maintain an up-to-date XML sitemap.

Layer 3: Meaning (schema markup)

Schema markup is structured data, usually written as JSON-LD, that labels the facts on a page in a machine-readable way. Used accurately, it helps systems connect your website to the business entity behind it. Useful types for a local service business include:

• LocalBusiness (or a more specific subtype such as Plumber or Electrician): name, address, phone, hours, service area, logo, and links to your official profiles.
• Service: individual services and their descriptions.
• FAQPage: questions and answers that appear on the page.
• Review and AggregateRating: only where you follow Google's guidelines on review markup.
• Organization and Person: for company and owner credentials.

Rules of thumb:

• Mark up only what's visible on the page. Hidden or misleading markup can backfire.
• Match your other listings exactly. Name, address, and phone should be identical to your Google Business Profile. See becoming a trusted entity.
• Validate it. Use Google's Rich Results Test and the Schema.org validator.
• Don't expect schema alone to cause recommendations. It clarifies meaning. It doesn't replace reviews and independent mentions.

What about llms.txt?

llms.txt is a proposed plain-text file, placed at the root of a website, that points AI systems to a curated summary of your most important pages. It's an idea worth knowing about, but several points are worth being honest about:

• It is a proposed convention, not an official web standard.
• Major AI providers have not confirmed that they use it for ranking or citations, and independent testing has found little evidence that it changes outcomes.
• It isn't a substitute for robots.txt and doesn't control access.

If your site platform makes one easy to publish, it costs little and does no harm. But it shouldn't come before the fundamentals above. A clear "About" page and well-organized service pages do more good.

A short technical checklist

1. Confirm robots.txt isn't blocking search and retrieval bots you want.
2. Check your firewall, CDN, and security plugins for bot blocking.
3. View your pages with JavaScript disabled to see what a crawler sees.
4. Add and validate LocalBusiness, Service, and FAQ schema.
5. Keep your sitemap current and submitted.
6. Make sure business details match across your site and listings.
7. Review server logs for AI crawler visits.

Beyond your own website

Even a perfectly configured site is just one source. That's why we also publish accurate business details to AI-crawlable mirrors, and why content should be written as answers. See answer engine optimization.

Frequently asked questions

Should I block AI crawlers? It depends on the bot's purpose. Blocking search and retrieval bots can reduce your chances of being cited. Training bots are a separate choice about your content.

Do I need llms.txt? No. It's optional and unproven. Fix access, readability, and structured data first.

Will schema markup get me recommended? Not by itself. It helps systems understand your pages, but reviews, consistency, and independent mentions do most of the work.

Can my web developer handle this? Usually yes. Share this checklist.

Start with the main guide: AI Visibility for Local Service Businesses.

Start with the main guide: AI Visibility for Local Service Businesses.