To guarantee the best experience for our mutual clients it might sometimes be required to whitelist our crawlers on your infrastructure. To be able to do that we provide this up-to-date list of DNS entries that will resolve to the server’s IP addresses (IPv4 and IPv6):
- spider-crawler1.do.feeder.co
- spider-crawler2.do.feeder.co
- spider-crawler3.do.feeder.co
- spider-crawler4.do.feeder.co
- spider-crawler5.do.feeder.co
- spider-crawler6.do.feeder.co
- spider-worker1.do.feeder.co
- spider-worker2.do.feeder.co
Below is an automatically generate table of the IP addresses for the above servers. We also provide an unauthenticated JSON API for fetching these IPs:
https://feeder.co/1/service-urls/crawlers.json
https://feeder.co/1/service-urls/crawlers.txt