Feeder is an RSS service that crawls feeds periodically to check for new updates. Feeder has custom software and servers to do this, and if you are a site owner you might see our User-agent pop up in your logs.
We only ever crawl feeds that are actively subscribed to by our users. As such the Feeder crawler is acting as an agent on behalf of the user and don't check for presence of robots.txt.
Our User-agent:
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6; +feeder.co/crawler) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Old User-agent:
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2; +feeder.co/crawler) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36
If you believe the Feeder crawler is accessing your site too often and wish to set a rate limit we sincerely urge you to contact us at support@feeder.co We always try to answer and fix issues as fast as possible, so barring holidays or other occasions, expect an answer within 24 hours.