0

I configured Tor browser and privoxy using https://jarroba.com/anonymous-scraping-by-tor-network/. When I checked my IP with http://icanhazip.com/, my IP addresses are changed. But, when I tried to scrape desired website, I got:

You are attempting to access "website" using an anonymous private/proxy network. Please disable that and try accessing the site again. 
3
  • That looks like the site is (correctly) detecting that you are connecting via a Tor exit node, and blocking your requests. What's your question?
    – womble
    Oct 4 '19 at 23:51
  • I see, ty, Is there a way to overcome this? Oct 5 '19 at 8:40
  • No there is not a direct way to overcome this. Even if you pay for a VPN service, if the VPN's servers are known the the website, they will block you anyway. There may be another option though. Does the website have any mirrors that allow scraping? For example, project gutenberg does not allow scraping their main site and they may ban your IP if you try. However they will allow you to do so using one of their mirrors. Sometimes you just need to find out of the website offers other options.
    – JSEvans
    Oct 5 '19 at 10:07
3

Sites you visit while using Tor have a relatively easy time to find out that you are using Tor. This is because your connection comes from one of the 1000 or so Exit Relays in the Tor network. You can not "overcome" this while using Tor, all Exit Relays are publicly known. One thing you could try doing is contact the site and ask them nicely not to block access to their service for all Tor users.

1
  • It's probably going to be a little harder to convince a website to be nice to Tor users when you're using Tor to, effectvely, abuse the website by scraping it...
    – womble
    Oct 6 '19 at 4:24

Not the answer you're looking for? Browse other questions tagged or ask your own question.