Loading UtilityForge…
GENERATOR FLOOR
Compose a robots.txt file with a configurable allow-all stance, one disallow path per line, an optional sitemap URL, and an optional crawl-delay directive.
WHAT THIS DOES
Runs entirely in your browser.
/tools/robots-txt-generatorrobots-txt-generatorv10.0.0LOCAL · LOCAL RUNIMPLEMENTED — This workbench is functional in the current release.Presets for this workbench
No preset yet. A preset stores the values in this form — never the result — and only for workbenches that run entirely in your browser.
Collections
Saved on this device only. Turn on sync to carry them to another device.
BATCH / ONE VALUE PER LINE
Up to 200 lines, one at a time, in this browser. Each line takes the place of disallow and everything else stays as you set it. Nothing is uploaded, and the row limit is not exceeded silently.
METHOD / LOCAL RUN
The output is a deterministic concatenation of User-agent blocks, Disallow rules, an optional Sitemap line, and an optional Crawl-delay line.
Boundary: Bots honor robots.txt voluntarily. The output does not enforce access, replace server-side access control, or substitute for authentication or network policy.
The calculation or transformation runs in your browser using the values entered above. No input is sent to UtilityForge for this tool.
EXAMPLES / LOCAL RUN FIXTURES
FAQ
Bots honor robots.txt voluntarily. The output does not enforce access, replace server-side access control, or substitute for authentication or network policy.
No. This published tool runs locally in your browser. UtilityForge does not send the values entered in this workbench to a server.
No. This workbench runs in the page: Allow all by default, Disallow paths (one per line), Sitemap URL (optional), Crawl delay (seconds) are read by the code your browser already downloaded, and UtilityForge receives neither the values nor the result.
Yes, and it is enforced rather than advertised: Disallow paths (one per line) (10,000 characters) and Sitemap URL (optional) (500 characters). The limit is a browser input limit on a tool that runs on this device, so nothing is truncated after being sent anywhere.
RELATED TOOLS
LOCAL — Runs entirely in your browser.
COMMON USE CASES / SOURCE NOTE
Last reviewed:
Source information: Local text concatenation; no crawler is contacted and no file is written outside the browser.