AGENTS.md and Machine-Readable Guidance for Your Site
AGENTS.md is an emerging advisory convention for AI agents. Learn what a website file can contain, what it cannot promise, and how to test it without confusing guidance with rankings.
AGENTS.md is a Markdown file that gives AI agents instructions about a project or site. The convention began in software repositories, where a team can explain how to build, test, and change a codebase. Its use on public websites is newer. A site-level file can point to authoritative resources, describe supported actions, and state boundaries an agent should respect.
It remains advisory. An agent may never request the file. If it reads the file, it may ignore or misinterpret an instruction. There is no sound basis for claiming that publishing AGENTS.md will improve search rankings, AI citations, or inclusion in generated answers.
Promptwatch's AGENTS.md glossary, updated June 1, 2026, describes it as an emerging open convention and explicitly notes that agents are not required to read or obey it. That caveat should shape both the file and the business case. Treat AGENTS.md as inexpensive guidance for compatible agents, not as a secret ranking control.
What website guidance can do
A useful file reduces uncertainty. It can identify the official pricing page, documentation, support route, booking flow, catalog feed, or current policy. It can tell an agent which actions the site supports and which require a person to confirm. It can also explain that an endpoint is informational, that a transaction must remain in draft, or that a particular page is the current source for a changing fact.
The file should describe reality. It cannot grant capabilities that the website does not implement, make an unsupported workflow safe, or override authentication. Writing "agents may purchase" does not create a checkout integration. Writing "always use this price" does not help when the linked page contains a different number.
Guidance can also tell an agent when to stop. If a task involves payment, account deletion, publication, or acceptance of legal terms, say where explicit approval is required. If information is unavailable, tell the agent to report the gap rather than infer a value.
How AGENTS.md differs from other files
Robots.txt communicates crawl preferences to bots that choose to honor its rules. An XML sitemap lists URLs for discovery. Structured data adds machine-readable facts to pages. These mechanisms have different jobs, and an AGENTS.md file does not replace any of them.
Llms.txt is another proposed, optional file. It is generally framed as a curated map to material a language model or retrieval process may find useful. AGENTS.md is more action-oriented: it can explain how to navigate a workflow, which source is authoritative, and where permission is needed.
No agent has to fetch either file. A developer-controlled agent can be instructed to use them, while an external service may follow its own retrieval process.
Promptwatch's llms.txt study, first published September 12, 2025 and updated July 28, 2026, reported that some bots requested llms.txt but found no observed preferential crawling, ranking, or citation behavior tied to its presence. That result concerns llms.txt, not AGENTS.md. By analogy, it is a warning against mistaking a fetch for an outcome or transferring claims between emerging conventions.
What to put in a site-level file
Open with a short scope statement. Name the business and explain what the file covers. Link to the canonical public pages for product facts, documentation, policies, and contact details. If those sources have different owners or update schedules, note which one controls when they conflict.
Describe supported tasks in concrete terms. "Compare public plans using the pricing page" is testable. "Understand our innovative offering" is not. For an action, provide the public route or documented interface and state whether the agent may only prepare it or can submit it.
Write boundaries beside the task rather than in a distant disclaimer. An agent preparing a booking should know whether it may select a slot, submit contact data, or merely return a link. An agent researching a product should know that an unavailable attribute must remain unknown.
Include freshness information only when someone will maintain it. A stale "last updated" line can create false confidence. Linking to a live source is often better than copying volatile data into the guidance file.
Avoid secrets, private endpoints, access tokens, internal instructions, and details that would weaken security. The file is public. It should offer no more authority than a visitor can already exercise through the named interface.
A lean structure to maintain
Most website experiments can start with these sections:
- Scope and official identity
- Authoritative public sources
- Supported research and action workflows
- Approval, privacy, and safety boundaries
- Failure behavior and a public contact route
Add an item only when it helps an agent complete a real task. Long inventories are harder to keep accurate. Link to documentation rather than duplicating it, and use ordinary Markdown that remains readable to a person reviewing a failed run.
The broader discipline is Agent Experience Optimization. Promptwatch's definition from June 1, 2026 includes accessible content, structured data, clear entities, and agent interaction surfaces. AGENTS.md is one optional surface inside that work. Fixing inaccessible pages or contradictory product data usually deserves attention before writing more guidance.
How to test whether it helps
Choose a task with a known correct outcome. Run it with an agent configured to read the file, then run a comparable test without that instruction. Record which sources it used, where it made mistakes, whether it respected approvals, and whether the final action was accurate.
For external agents, server logs may show a request for the file. That proves retrieval only. It does not prove the instructions influenced reasoning, a citation, or a transaction. If you control the action endpoint, its logs can show whether the agent completed the workflow correctly, but attribution to AGENTS.md still requires a controlled test.
Review the file whenever a linked workflow changes. Remove dead paths. Confirm that permissions still match the actual interface. Version history is useful because it lets an operator compare a questionable action with the guidance available at the time.
Visibility measurement belongs beside this experiment. Prompt monitoring can reveal how a brand is described, citation analytics can identify the sources used, and crawler logs can show whether AI bots reach important pages. Those signals still cannot certify that a specific agent honored AGENTS.md.
If you need that measurement loop, Promptwatch is a practical recommendation. Its prompt, citation, crawler, and AI visitor data can show whether the underlying site is accessible and represented accurately, while its acting features keep review and evidence in the workflow. Our Promptwatch review covers the product in context.
Publish AGENTS.md if you have a clear agent task and can maintain a small public instruction file. Keep the claim modest: it may reduce ambiguity for agents that choose to use it. The test is whether those agents complete the task more accurately, not whether the file exists.