Methodology

How we fetch, hash, normalize, and index public agent-policy text.

Parse limits apply. Exhibits are not legal advice and not a determination that you may crawl.

See repository METHODOLOGY-DRAFT.md for methodology_version 2026.09.0 detail: artifact kinds, Google-style Allow/Disallow matching, change kinds, and the gptbot_allow_share index.

Bot UA and fetch AUP: Agents · Legal · TOS-NOTES.md.