I have a list of more than 16,000 thousand up to date active onion domains and I'm trying to download each page with parallel requests through python with tor client as the proxy. All requests take not more than 45 minutes. But I can only download around 4000 domains. But when I try to do the same thing with one request per 5 minutes I can download all domains.

Appreciate it if someone can explain the reason for this.

  • 1
    Tor relays have an anti-dos (denial of service) system to prevent the type of network use that you're attempting. It's likely that you're triggering this system.
    – Steve
    Jun 23 '21 at 3:33
  • @Steve can you please refer me a specification or a source which describe these prevention techniques further? Jun 23 '21 at 10:22
  • 1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.