Feeder is an RSS service that crawls feeds periodically to check for new updates. Feeder has custom software and servers to do this, and if you are a site owner you might see our User-agent pop up in your logs.
We only ever crawl feeds that our users have actively subscribed to. The Feeder crawler acts as an agent on behalf of those users, so it doesn't check robots.txt.
Feeder's requests say feeder.co in their User-agent. These are the two you're most likely to see:
User-agent: Feeder/1.0 (feeder.co/fetcher)
User-agent: Mozilla/5.0 (feeder.co; Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36
The exact string can change over time, so we recommend matching on feeder.co. A few sites only answer web browsers, and Feeder visits those with a regular browser User-agent.
Feeder signs its requests with Web Bot Auth (HTTP Message Signatures, RFC 9421). The Signature-Agent header names https://adder.feeder.co, and our public key is published at adder.feeder.co/.well-known/http-message-signatures-directory. Bot protection that checks these signatures can confirm that a request really comes from Feeder, whatever address it comes from.
These are the addresses of Feeder's crawler servers. Each one has a reverse DNS name under feeder.co that points back to the same address, so you can verify a request with a reverse lookup followed by a forward lookup. The list is also available as JSON and plain text.
| Host | IP address |
|---|---|
bhs-crawl1.ovh.feeder.co |
144.217.180.83 |
bhs-crawl2.ovh.feeder.co |
144.217.180.84 |
bhs-spider-app1.ovh.feeder.co |
15.235.82.82 |
bhs-spider-app2.ovh.feeder.co |
144.217.76.102 |
spider-dedicated-crawler1.do.feeder.co |
165.227.116.114 |
spider-dedicated-crawler2.do.feeder.co |
165.227.76.164 |
spider-dedicated-crawler3.do.feeder.co |
209.97.147.246 |
spider-dedicated-crawler4.do.feeder.co |
138.197.71.121 |
spider-dedicated-crawler5.do.feeder.co |
165.227.125.60 |
spider-dedicated-crawler6.do.feeder.co |
142.93.74.129 |
spider-crawler1.do.feeder.co |
68.183.151.227 |
spider-crawler2.do.feeder.co |
174.138.57.235 |
spider-crawler3.do.feeder.co |
68.183.157.237 |
spider-crawler4.do.feeder.co |
159.89.183.157 |
spider-crawler5.do.feeder.co |
174.138.49.150 |
spider-crawler6.do.feeder.co |
159.89.43.145 |
spider-crawler7.do.feeder.co |
104.131.105.190 |
spider-crawler8.do.feeder.co |
157.245.117.2 |
spider-crawler9.do.feeder.co |
157.245.117.25 |
spider-crawler10.do.feeder.co |
167.71.89.133 |
If you believe the Feeder crawler is accessing your site too often and wish to set a rate limit we sincerely urge you to contact us at support@feeder.co We always try to answer and fix issues as fast as possible, so barring holidays or other occasions, expect an answer within 24 hours.