# Category 6: Observability and Content Health

> For AI agents: the complete documentation index is at [https://agentdocsspec.com/llms.txt](https://agentdocsspec.com/llms.txt). Markdown versions of any page are available by appending `index.md` to the URL path.

Part of the [Web Documentation Delivery Spec](https://agentdocsspec.com/spec/web/). The [Checks Summary](https://agentdocsspec.com/spec/web/#checks-summary) lists all checks with links to their definitions.

These checks evaluate whether the site's agent-facing resources stay accurate
and up to date over time. Categories 1-5 can be evaluated as point-in-time
audits; this category addresses the ongoing maintenance dimension. `llms.txt`
files and markdown endpoints are secondary outputs that often aren't wired
into existing monitoring, so they can go stale, break, or drift from primary
HTML content without anyone noticing.

### `llms-txt-coverage`

- **What it checks**: How much of the site's documentation is represented
  in `llms.txt`.
- **Why it matters**: `llms.txt` is an agent's primary navigational index
  into a documentation site. Pages missing from the index are effectively
  invisible to agents that rely on it for discovery. Unlike
  `llms-txt-links-resolve` (which catches broken links to pages that are
  listed), this check catches the opposite problem: pages that exist on the
  site but aren't listed at all.
  
  Not every gap is a problem; many sites intentionally curate their
  `llms.txt` to include only a subset of pages. The check's value is
  making the coverage level visible so site owners can confirm it reflects
  their intent.
- **Result levels** (based on coverage of sitemap doc pages, excluding
  non-doc pages like blog posts, pricing, and login pages):
  - **Pass**: `llms.txt` links cover >=95% of the site's primary pages.
  - **Warn**: `llms.txt` links cover 80-95% of primary pages (some live
    pages are missing).
  - **Fail**: `llms.txt` links cover <80% of primary pages (missing large
    sections of the documentation).
  These thresholds are defaults that assume the site intends `llms.txt` to
  mirror the sitemap. Sites that intentionally curate their `llms.txt`
  should adjust thresholds to match their intent (see Notes below).
  Implementations should allow thresholds to be configured.
- **Recommended action**:
  - **Warn**: Review missing pages. If they should be in `llms.txt`, add
    them. If they are intentionally excluded, adjust the coverage threshold
    or add them to an exclusion list so the check reflects your intent.
  - **Fail**: If unintentional, regenerate `llms.txt` from your sitemap or
    build pipeline. If intentional, lower the threshold or set it to 0 to
    make the check informational.
- **Automation**: Heuristic. Compare links in `llms.txt` against a sitemap
  or crawled page list; flag pages present in the sitemap but absent from
  `llms.txt`. Implementations should support exclusion patterns that
  remove known-intentional gaps from the sitemap before calculating
  coverage.
- **Notes**: Not every sitemap page belongs in `llms.txt`. Sites
  intentionally exclude content for good reasons: changelog and
  release notes archives that would bloat the file, older product versions
  that aren't relevant to current development, API reference pages that
  aren't useful in markdown form, or directory pages that just link to
  other pages already listed. This is legitimate curation, not drift.

  The check should accommodate three use cases through configurable
  thresholds and exclusion patterns:

  - **Full parity**: The site intends `llms.txt` to mirror the sitemap.
    Default thresholds (95/80) apply; no exclusions needed.
  - **Curated**: The site intentionally includes only a subset of pages.
    Set thresholds to 0 to make the check informational. It still reports
    coverage percentage and lists what's missing, but never warns or fails.
  - **Hybrid**: The site wants strict coverage but with known exclusions.
    Exclusion patterns remove intentional gaps from the sitemap before
    calculating coverage; remaining pages are held to the default
    thresholds.

  The definition of "primary pages" in the denominator requires judgment.
  Implementations should document how they construct the URL pool from the
  sitemap and what filtering they apply.

### `markdown-content-parity`

- **What it checks**: Whether markdown versions of pages contain the same
  substantive content as their HTML counterparts.
- **Why it matters**: When markdown is generated separately from HTML (rather
  than being the source that HTML is built from), the two can drift. A site
  might update an HTML page but forget to regenerate the markdown version,
  leaving agents with outdated instructions or code examples. This is
  particularly insidious because agents that receive the markdown version
  have no signal that a newer HTML version exists.

  However, in some cases, content divergence may be intentional. Some sites
  intentionally serve different content to different audiences, providing
  agent-optimized markdown alongside human-optimized HTML. In those cases,
  the divergence is deliberate. The check's value is surfacing it so site
  owners can confirm it reflects their intent.
- **Result levels** (based on the percentage of content segments in the
  HTML version that are missing from the markdown version, after
  normalizing whitespace, case, and formatting):
  - **Pass**: <5% of content segments missing (or page has fewer than 10
    segments, which is too small to produce meaningful parity scores).
  - **Warn**: 5-20% of content segments missing (minor differences:
    formatting variations, navigation elements present in one but not the
    other).
  - **Fail**: >=20% of content segments missing (substantive content
    differences: missing sections, outdated code examples, or different
    instructions between the two versions).
  These thresholds are defaults that assume the site intends markdown to
  mirror HTML. Sites that intentionally serve different content per audience
  should adjust thresholds to match their intent (see Notes below).
  Implementations should allow thresholds to be configured.
- **Recommended action**:
  - **Warn**: Review pages with minor differences. If they are formatting
    variations that may affect agent comprehension, fix them. If they
    reflect intentional audience segmentation, adjust thresholds or
    configure the check to account for it.
  - **Fail**: If unintentional, agents receiving the markdown version are
    getting outdated or incomplete content. Regenerate markdown from source
    or fix the build pipeline. If intentional, lower the threshold or set
    it to 0 to make the check informational.
- **Automation**: Heuristic. Fetch both versions, extract text content from
  HTML (strip tags), and compare key sections (headings, code blocks,
  paragraph content) for meaningful differences. Minor formatting
  differences should be ignored. If the HTML contains audience-segmentation
  tags (see Notes), implementations should strip tagged content before
  comparing so that intentionally excluded content does not count as
  missing.
- **Notes**: Sites where markdown is the source format and HTML is generated
  from it are less likely to have parity issues, but the check is still
  valuable as a safety net for build pipeline failures.

  **Audience segmentation.** Some documentation platforms use HTML tags to
  control what content appears in each version. For example, a platform
  might tag certain content as agent-only (included in markdown but not
  rendered in HTML) or human-only (rendered in HTML but excluded from
  markdown). Platforms like Fern and Mintlify have implemented this
  pattern. When the HTML contains recognized audience-segmentation tags,
  implementations should account for them before comparing: content
  explicitly tagged for one audience should not count as missing from the
  other.

  The spec does not define a standard set of segmentation tags or
  prescribe which vendor conventions to recognize. Implementations should
  document which tag conventions they support, and vendors or site owners
  who want their conventions recognized can contribute them to
  implementations directly.

  As with `llms-txt-coverage`, the check should accommodate sites at
  different points on the mirrored-to-curated spectrum:

  - **Mirrored** (default): Markdown should match HTML. Default thresholds
    apply.
  - **Segmented**: The site uses audience-segmentation tags to control
    per-version content. The check strips tagged content before comparing;
    remaining shared content is held to the default thresholds.
  - **Curated**: The site intentionally serves different content with no
    tag-level signal. Set thresholds to 0 to make the check informational.

  **Dynamically generated pages.** Pages built from data (catalogs, model
  listings, compatibility matrices) can diverge between representations
  without anyone deciding they should, because the HTML and markdown
  variants are rendered by different pipelines with different defaults. In
  one observed production case, a catalog's HTML showed 98 items while its
  markdown variant listed 102: the HTML applied a default filter the
  markdown dump didn't, and the markdown was additionally paginated. For
  pages with repeated structure, implementations should compare item counts
  between representations, and should distinguish the likely causes when
  counts differ: a default filter on the dynamic view (divergent by
  configuration), pagination on either side (divergent by windowing, see
  `single-fetch-completeness`), or staleness (one representation generated
  from older data). Each has a different owner and fix.

### `cache-header-hygiene`

- **What it checks**: Whether `llms.txt` and markdown endpoints have cache
  headers that allow timely updates.
- **Why it matters**: Aggressive caching on agent-facing resources means
  that even after a site owner updates their `llms.txt` or markdown content,
  agents (and intermediary CDNs) may continue serving stale versions for
  hours or days. Conversely, no cache headers at all leads to ambiguous
  behavior where different CDN providers apply their own defaults. For
  resources that are relatively small and infrequently fetched, short cache
  lifetimes with revalidation are appropriate.
- **Result levels**:
  - **Pass**: Cache headers allow timely updates (e.g., `max-age` under
    3600, or uses `must-revalidate` with `ETag`/`Last-Modified`).
  - **Warn**: Moderate caching (1-24 hours) that could delay updates.
  - **Fail**: Aggressive caching (over 24 hours) with no revalidation
    mechanism, or no cache-related headers at all (ambiguous behavior).
    An exception: responses that lack `Cache-Control` and `Expires` but
    include `ETag` or `Last-Modified` should pass, since these validation
    headers enable conditional revalidation by browsers and CDNs even
    without explicit cache directives.
- **Recommended action**:
  - **Warn**: Updates to `llms.txt` or markdown content may take hours to
    propagate. Consider reducing cache lifetimes for these resources.
  - **Fail**: Set `max-age` under 3600 or add `must-revalidate` with
    `ETag`/`Last-Modified` so content updates reach agents promptly.
- **Automation**: Full. Inspect `Cache-Control`, `Expires`, `ETag`, and
  `Last-Modified` response headers.

### Ongoing Monitoring Recommendations

The three checks above can be run as one-time audits, but they're most
valuable when run on a schedule. This section offers non-normative guidance
on integrating agent-facing resources into existing monitoring workflows.

**Include `llms.txt` and markdown endpoints in uptime monitoring.** These
resources should be monitored alongside your primary documentation site. A
200 response from your docs homepage doesn't guarantee that `/llms.txt` or
`.md` URL variants are also healthy. Add them to whatever uptime tool you
already use (Pingdom, Uptime Robot, Checkly, etc.) as separate check targets.

**Set up alerting for response time degradation.** If your `llms.txt` or
markdown endpoints start responding slowly, agents may time out before
receiving content. This is especially relevant for dynamically generated
markdown (as opposed to static files), where a backend issue could cause
latency spikes that don't affect the HTML site.

**Run coverage and parity checks on a schedule.** Rather than treating
`llms-txt-coverage` and `markdown-content-parity` as one-time audits, run
them weekly or on every deploy. A CI check that compares `llms.txt` link
coverage against the sitemap can catch missing pages before they reach
production.

**Monitor for silent failures.** A 200 response with empty content, a
generic error message, or a login page is worse than a clean 404, because
agents will try to extract information from the response. Check that
`llms.txt` and markdown responses contain expected content markers (e.g., an
H1, a minimum character count) rather than just checking for a 200 status
code.