0

I am doing web scraping, and I am using tor as my proxy service. I am using ubuntu 20.4.x and python 3.9.

As I have installed tor in my local system, I am using:

proxies = {
    'http': 'socks5://127.0.0.1:9050',
    'https': 'socks5://127.0.0.1:9050'
}

My sample code looks like:

import requests
from fake_useragent import UserAgent


url = "https://www.google.com/"

proxies = {
    'http': 'socks5://127.0.0.1:9050',
    'https': 'socks5://127.0.0.1:9050'
}

# Using fake headers to change the request header
headers = {'User-Agent': UserAgent().random}

try:
    resp = requests.get(url, proxies=proxies, headers=headers, timeout=(10, 10))
except Exception as e:
    print(e)

Let's say I have a remote server with IP xx.xx.xx.xx, Can I install tor in this remote server and use the particular server port in my local machine to use proxy, i.e. xx.xx.xx.xx:9050.

If yes, can I use it in my code like:

proxies = {
    'http': 'socks5://xx.xx.xx.xx:9050',
    'https': 'socks5://xx.xx.xx.xx:9050'
}

How to get it installed?

0

Just use SSH port mapping - so you will have your localhost's ports mapped to the Tor remote server's localhost ones and your software won't see any difference. There's no need to open Tor socks proxy ports to the public IP of the server that is hosting your Tor instance. In case of multiple Tor instances - I've used Tor for this task with 10 instances worldwide - on your scraping host you're installing a socks balancing proxy and bind it on the localhost too - and after that you're making SSH tunnels to the remote servers with an actual Tor instances. You can also use STunnel instead of SSH tunnelling - but don't forget to make a certificate verification be set to 3 and use a TLS key if applicaple.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.