WHIBot
WHIBot is the crawler Wood Heat Index uses to read advertised firewood prices from small-business websites, as part of the market-data collection that supplements our modeled county estimates with real observations. It identifies itself on every request as:
WHIBot/1.0 (+https://woodheatindex.com/bot/)
This page is the destination that address points to.
What it fetches
WHIBot only visits public pages that a firewood-vendor business published to advertise its own prices, drawn from a small, hand-curated registry — never a marketplace, a login-walled page, or anything requiring authentication. It reads prices and delivery terms as facts; it never copies or republishes photos or prose.
How it behaves
- Honors
robots.txt, checked fresh on every visit, under its own name — never a spoofed identity. - Paces itself to no more than one request every two seconds, and fetches at most about ten pages from any one site.
- Visits a given site at most monthly, timed ahead of our regular model rebuild.
- Backs off immediately and permanently from any site that returns a rate-limit or access-denied response — it does not retry.
What happens to the data
Nothing WHIBot collects is published automatically. Every row a fetch turns up is reviewed by a person before it can affect any published estimate, exactly like a reader-submitted price — and it is always labeled by its real source (the vendor's own name and page), never mistaken for a reader submission.
Opting out
If you'd rather WHIBot not visit your site, either add a rule to your own robots.txt:
User-agent: WHIBot
Disallow: /
or just tell us: [email protected]. Either way, we'll mark your site excluded the same day.