0

I have a list of more than 16,000 thousand up to date active onion domains and I'm trying to download each page with parallel requests through python with tor client as the proxy. All requests take not more than 45 minutes. But I can only download around 4000 domains. But when I try to do the same thing with one request per 5 minutes I can download all domains.

Appreciate it if someone can explain the reason for this.

Improve this question
3
  • 1
    Tor relays have an anti-dos (denial of service) system to prevent the type of network use that you're attempting. It's likely that you're triggering this system.
    – Steve
    Jun 23, 2021 at 3:33
  • @Steve can you please refer me a specification or a source which describe these prevention techniques further?
    – Isuranga Perera
    Jun 23, 2021 at 10:22
  • 1
    github.com/torproject/tor/blob/main/src/core/or/dos.c
    – Steve
    Jun 23, 2021 at 20:33

0

Reset to default

You must log in to answer this question.