Skip to content

Publish only the stable docs version in llms-full.txt #413

Description

@bgravenorst

Summary

llms-full.txt is generated from the current (docs/) tree, which is the pre-release development version served under /development/. It should contain only the stable documentation — the version served at the site root.

Do not hardcode a version number. Stable is whichever version lastVersion points to in docusaurus.config.js, and that value changes after every Web3Signer release.

Current behaviour

docusaurus-plugin-llms is configured with docsDir: "docs". In this site:

  • docs/ is the unreleased current version (path: "development").
  • The stable version is lastVersion (today 26.7.0) and lives in versioned_docs/version-<lastVersion>/.
  • HTML at / and the per-page .md export (scripts/copy-md-to-build.js) already follow lastVersion.

llms-full.txt does not. Agents that ingest https://docs.web3signer.consensys.io/llms-full.txt can treat unreleased behaviour as current.

llms.txt is produced by the same plugin from the same tree, so it has the same version skew.

Why this is needed

For a signing service, citing pre-release docs as current is worse than a ranking issue. llms-full.txt is the corpus advertised to AI crawlers. It should match the stable docs a human gets at /.

After each Web3Signer release the docs freeze a new version and bump lastVersion. A fix that hardcodes 26.7.0 (or any other version string) will be wrong on the next release.

What is needed

  • Generate llms-full.txt from the stable docs tree only — the same source copy-md-to-build.js uses (versioned_docs/version-<lastVersion>/).
  • Derive that version from lastVersion in docusaurus.config.js (fallback: first entry in versions.json when lastVersion is unset), not from a hardcoded release number.
  • Keep this aligned when lastVersion is bumped after a release; no extra manual step beyond the usual versioning workflow.
  • Apply the same source to llms.txt if it still reads docs/, so the index and the full corpus do not disagree.
  • Leave /development/ HTML reachable from the version dropdown. This issue is only the LLM corpus files.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    DocOpsRelated to documentation system, tools and CIbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions