NewsFactsBot

If you run a news site and found NewsFactsBot in your logs, this page explains what it is and how to control it.

What it is

NewsFactsBot is the crawler behind NewsFacts. It reads public article pages to keep a record of how published stories change over time. It identifies itself with this user agent:

Mozilla/5.0 (compatible; NewsFactsBot/0.1; +https://newsfacts.com/bot)

What it fetches

• Your public RSS feed or news sitemap, about every 2 minutes, with conditional requests where your server supports them.
• Article pages listed there, typically a few dozen times over 30 days per article, most of them in the first hours.
• No more than two requests to your site at the same moment.

It never logs in, never bypasses a paywall, and never fetches images, scripts or ads.

robots.txt

NewsFactsBot obeys robots.txt, re-reading it daily. It follows a group addressed to NewsFactsBot if you have one, and otherwise your * rules. To stop it collecting your site entirely:

User-agent: NewsFactsBot Disallow: /

Contact the bot's operator

Questions, a rate concern, or a correction to the record? Send a note and we'll reply.