LabmillBot

LabmillBot reads the public documentation of research computing and HPC systems (for example partitions, storage and software lists), so that researchers get correct advice for the systems they use. It reads only public pages, never signs in, and keeps facts and short quotes with a link to the source.

How it behaves

• It obeys robots.txt.
• It makes at most one request every few seconds to a site, and fewer when a site answers slowly.
• It identifies itself with the user agent LabmillBot and a link to this page.

Contact

Questions or problems with the crawler: bot@labmill.org. Abuse reports: abuse@labmill.org.

Opt out

To exclude your site, add this to your robots.txt:

User-agent: LabmillBot
Disallow: /