I want to make multiple GET requests using Tor to a webpage. I want to use a different ipaddress for each request.
import socks
import socket
socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 9150)
socket.socket = socks.socksocket
import requests
print (requests.get('http://icanhazip.com')).content
Using this, I made one request. How can I change the ipaddress to make another?
There are 2 aspects to your question -
Making requests using Tor
Renewing the connection as per requirement (in your case, after every request)
Part 1
The first one is easy to do with the latest (upwards of v2.10.0) requests library with an additional requirement of requests[socks] for using the socks proxy.
Installation -
pip install requests[socks]
Basic usage -
import requests
def get_tor_session():
session = requests.session()
# Tor uses the 9050 port as the default socks port
session.proxies = {'http': 'socks5://127.0.0.1:9050',
'https': 'socks5://127.0.0.1:9050'}
return session
# Make a request through the Tor connection
# IP visible through Tor
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
# Above should print an IP different than your public IP
# Following prints your normal public IP
print(requests.get("http://httpbin.org/ip").text)
Part 2
To renew the Tor IP, i.e. to have a fresh visible exit IP, you need to be able to connect to the Tor service through it's ControlPort and then send a NEWNYM signal.
Normal Tor installation does not enable the ControlPort by default. You'll have to edit your torrc file and uncomment the corresponding lines.
ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
HashedControlPassword 16:05834BCEDD478D1060F1D7E2CE98E9C13075E8D3061D702F63BCD674DE
Please note that the HashedControlPassword above is for the password "password". If you want to set a different password, replace the HashedControlPassword in the torrc by noting the output from tor --hash-password "<new_password>" where <new_password> is the password that you want to set.
................................................................................
Warning for Windows users: see post here.
There is an issue on windows where the setting for the controlport in the torrc file is ignored if tor was installed using the following command:
tor --service install
To resolve the issue, after editing your torrc file, type the following commands:
tor --service remove
tor --service install -options ControlPort 9051
................................................................................
Okay, so now that we have Tor configured properly, you will have to restart Tor if it is already running.
sudo service tor restart
Tor should now be up & running on the 9051 ControlPort through which we can send commands to it. I prefer to use the official stem library to control Tor.
Installation -
pip install stem
You may now renew the Tor IP by calling the following function.
Renew IP -
from stem import Signal
from stem.control import Controller
# signal TOR for a new connection
def renew_connection():
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password="password")
controller.signal(Signal.NEWNYM)
To verify that Tor has a new exit IP, just rerun the code from Part 1. For some reason unknown to me, you need to create a new session object in order to use the new IP.
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
Here is the code you want to use (download the stem package using pip install stem)
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password='your password set for tor controller port in torrc')
print("Success!")
controller.signal(Signal.NEWNYM)
print("New Tor connection processed")
Good luck and hopefully that works.
You can try pure-python tor protocol implementation Torpy. No need original Tor client or Stem dependency at all.
$ pip3 install torpy[requests]
...
$ python3.7
>>> from torpy.http.requests import TorRequests
>>> with TorRequests() as tor_requests:
... print("build circuit")
... with tor_requests.get_session() as sess:
... print(sess.get("http://httpbin.org/ip").json())
... print(sess.get("http://httpbin.org/ip").json())
... print("renew circuit")
... with tor_requests.get_session() as sess:
... print(sess.get("http://httpbin.org/ip").json())
... print(sess.get("http://httpbin.org/ip").json())
...
build circuit
{'origin': '23.129.64.190, 23.129.64.190'}
{'origin': '23.129.64.190, 23.129.64.190'}
renew circuit
{'origin': '198.98.50.112, 198.98.50.112'}
{'origin': '198.98.50.112, 198.98.50.112'}
So each time when you getting new session you get new identity (basically you get new circuit with new exit node). See more examples at readme file https://github.com/torpyorg/torpy
You can use torrequest library (shameless plug). It's available on PyPI.
from torrequest import TorRequest
with TorRequest() as tr:
response = tr.get('http://ipecho.net/plain')
print(response.text) # not your IP address
tr.reset_identity()
response = tr.get('http://ipecho.net/plain')
print(response.text) # another IP address, not yours
Requests supports proxies using the SOCKS protocol from version 2.10.0.
import requests
proxies = {
'http': 'socks5://localhost:9050',
'https': 'socks5://localhost:9050'
}
url = 'http://httpbin.org/ip'
print(requests.get(url, proxies=proxies).text)
This answer complete the one of Ashish Nitin Patil for windows
(feel free to update this answer)
Part 2
ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
HashedControlPassword 16:05834BCEDD478D1060F1D7E2CE98E9C13075E8D3061D702F63BCD674DE
The HashedControlPassword above is the password. If you want to set a different password in the console navigate to \Tor Browser\Browser\TorBrowser\Tor and type the following commands: tor.exe --hash-password password_XYZ | more). It will give you something like HashedControlPassword 16:54C092A8... This is your password. Now you can add it to the torrc file (Tor Browser\Browser\TorBrowser\Data\Tor\torrc).
You will need then to restart Tor:
tor --service remove
tor --service install -options ControlPort 9051
To check if that works type netstat -an you will now see that port 9051 is open.
Notice that tor --service install -... will create Tor Win32 Service. For some reason, it seems you have to stop the service to use the browser (run services.msc)
EDIT: you will find many pieces of information here (About port number & proxy, Tor, Privoxy, Auto switch user agent...).
This code works fine. Using Tor, it changes the IP address after each request.
import time, socks, socket
from urllib2 import urlopen
from stem import Signal
from stem.control import Controller
nbrOfIpAddresses=3
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password = 'my_pwd')
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050)
socket.socket = socks.socksocket
for i in range(0, nbrOfIpAddresses):
newIP=urlopen("http://icanhazip.com").read()
print("NewIP Address: %s" % newIP)
controller.signal(Signal.NEWNYM)
if controller.is_newnym_available() == False:
print("Waitting time for Tor to change IP: "+ str(controller.get_newnym_wait()) +" seconds")
time.sleep(controller.get_newnym_wait())
controller.close()
The requests in requesocks is super old, it doesn't have response.json() and many other stuff.
I would like to keep my code clean. However, requests currently doesn't have socks5 supported yet (for more detail, read this thread https://github.com/kennethreitz/requests/pull/478)
So I used Privoxy as a http proxy that connects Tor for now.
Install and configure Privoxy on Mac
brew install privoxy
vim /usr/local/etc/privoxy/config
# put this line in the config
forward-socks5 / localhost:9050 .
privoxy /usr/local/etc/privoxy/config
Install and configure Privoxy on Ubuntu
sudo apt-get install privoxy
sudo vim /etc/privoxy/config
# put this line in the config
forward-socks5 / localhost:9050 .
sudo /etc/init.d/privoxy restart
Now I can use Tor like a http proxy. Below is my python script.
import requests
proxies = {
'http': 'http://127.0.0.1:8118',
}
print requests.get('http://httpbin.org/ip', proxies=proxies).text
A good function to renew your IP. Windows example
def renew_tor_ip():
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password="aAjkaI19!!laksjd")
controller.signal(Signal.NEWNYM)
Example of usage
import requests
import time
from stem import Signal
from stem.control import Controller
def get_current_ip():
session = requests.session()
# TO Request URL with SOCKS over TOR
session.proxies = {}
session.proxies['http']='socks5h://localhost:9150'
session.proxies['https']='socks5h://localhost:9150'
try:
r = session.get('http://httpbin.org/ip')
except Exception as e:
print(str(e))
else:
return r.text
#16:8EE7AEE3F32EEEEB605C6AA6C47B47808CA6A81FA0D76546ADC05F0F15 to aAjkaI19!!laksjd
#cmd shell "C:\Users\Arthur\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe" --hash-password aAjkaI19!!laksjd | more
#Torcc config
#ControlPort 9051
#HashedControlPassword 16:8EE7AEE3F32EEEEB605C6AA6C47B47808CA6A81FA0D76546ADC05F0F15
def renew_tor_ip():
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password="aAjkaI19!!laksjd")
controller.signal(Signal.NEWNYM)
for i in range(5):
print(get_current_ip())
renew_tor_ip()
time.sleep(5)
Related
I am posting the question and the answer, I found, as well, incase it would help someone. The following were my minimum requirements:
1. Client machine is Windows 10 and remote server is Linux
2. Connect to remote server via SSH through HTTP Proxy
3. HTTP Proxy uses Basic Authentication
4. Run commands on remote server and display output
The purpose of the script was to login to the remote server, run a bash script (check.sh) present on the server and display the result. The Bash script simply runs a list of commands displaying the overall health of the server.
There have been numerous discussions, here, on how to implement HTTP Proxy or running remote commands using Paramiko. However, I could not find the combination of both.
from urllib.parse import urlparse
from http.client import HTTPConnection
import paramiko
from base64 import b64encode
# host details
host = "remote-server-IP"
port = 22
# proxy connection & socket definition
proxy_url = "http://uname001:passw0rd123#HTTP-proxy-server-IP:proxy-port"
url = urlparse(proxy_url)
http_con = HTTPConnection(url.hostname, url.port)
auth = b64encode(bytes(url.username + ':' + url.password,"utf-8")).decode("ascii")
headers = { 'Proxy-Authorization' : 'Basic %s' % auth }
http_con.set_tunnel(host, port, headers)
http_con.connect()
sock = http_con.sock
# ssh connection
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
ssh.connect(hostname=host, port=port, username='remote-server-uname', password='remote-server-pwd', sock=sock)
except paramiko.SSHException:
print("Connection Failed")
quit()
stdin,stdout,stderr = ssh.exec_command("./check")
for line in stdout.readlines():
print(line.strip())
ssh.close()
I would welcome any suggestions to the code as I am a network analyst and not a coder but keen to learn and improve.
I do not think that your proxy code is correct.
For a working proxy code, see How to ssh over http proxy in Python?, particularly the answer by #tintin.
As it seems that you need to authenticate to the proxy, after the CONNECT command, add a Proxy-Authorization header like:
Proxy-Authorization: Basic <credentials>
where the <credentials> is base-64 encoded string username:password.
cmd_connect = "CONNECT {}:{} HTTP/1.1\r\nProxy-Authorization: Basic <credentials>\r\n\r\n".format(*target)
I want to make an anonymous web request using python 3.
I've tried few suggestions such as: Make requests using Python over Tor
I've managed to get a fake ip using this snippet:
Installation
pip install requests requests[socks]
Basic usage
import requests
def get_tor_session():
session = requests.session()
# Tor uses the 9050 port as the default socks port
session.proxies = {'http': 'socks5://127.0.0.1:9150',
'https': 'socks5://127.0.0.1:9150'}
return session
# Make a request through the Tor connection
# IP visible through Tor
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
# Above should print an IP different than your public IP
# Following prints your normal public IP
print(requests.get("http://httpbin.org/ip").text)
But that works only on port 9150 and when the tor web browser works.
I want to make a request without the tor browser, as i want to Dockerize the whole thing.
I've read about Socks5, and as you can see i've installed it, but when i make a request on port 9050 on the same snippet i get:
requests.exceptions.ConnectionError:
SOCKSHTTPConnectionPool(host='httpbin.org', port=80): Max retries
exceeded with url: /ip (Caused by
NewConnectionError(': Failed to establish a new connection: [WinError
10061] No connection could be made because the target machine actively
refused it',))
How can i solve it?
Thanks!
10061 is 'connection refused'
That means there was nothing listening on that port you tried to connect to, no service is up and running (no open port) or firewall on target IP blocks it
you can test that port with telnet
telnet `IP` `PORT`
And also check this port issue on Windows: here
I was also facing this issue, in my case my tor service was not running, actually I was using kalitorify which is a transparent proxy, and whenever I was using this I was not be able to use normal sites such as google search or similar, so to use these sites I was turning off my kalitorify service which also turns off your tor service
So if you're also using that then also check it once
I'm following the tutorial To Russia With Love, and as part of this I want to change the identity everytime I run the code. I've looked at multiple questions and tutorials and from the looks of it my torrc-defaults file is correct as it has:
ControlPort 9151
CookieAuthentication 1
I wanted to edit the torrc file first but if i touch it Tor won't run anymore, so i read somewhere to look at torrc instead. The torrc file alone only has:
# This file was generated by Tor; if you edit it, comments will not be preserved
# The old torrc file was renamed to torrc.orig.1 or similar, and Tor will ignore it
I don't want to set a password so my understanding from reading the stem documents is that setting the Authentication is enough. I have the following code:
import socks # SocksiPy module
import socket
import requests
SOCKS_PORT = 9150
# Set socks proxy and wrap the urllib module
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', SOCKS_PORT)
socket.socket = socks.socksocket
def getaddrinfo(*args):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, '', (args[0], args[1]))]
socket.getaddrinfo = getaddrinfo
url = "http://google.com"
print requests.get(url).elapsed.total_seconds()
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9151) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)
But when I run it i get the error:
socks.SOCKS5Error: 0x01: General SOCKS server failure
I'm not sure what to do, i've been reading about this for hours and haven't been able to solve the problem. Any hints would be great.
EDIT:
I've also read through This Post, but it doesn't seem like there's a solution.
I also tried putting the Controller statement before setting the sockets, but in that case the IP remains the same and doesn't change.
I ended up solving this problem by moving the controller statement before setting the sockets. Initially it looked like the IP remains the same but turns out if you wait an extra 3 or 4 seconds the IP changes, so I just added a time delay and it runs fine.
Supposing this working code :
import win_inet_pton
import socks
import socket
s = socks.socksocket()
s.set_proxy(socks.SOCKS5, "localhost", 9050)
s.connect(("xmh57jrzrnw6insl.onion",80))
s.send("GET / HTTP/1.1\r\n\r\n")
print 'sended'
data=s.recv(1024)
print data
Tor service is indeed running at port 9050.
In normal condition, python will perform DNS resolution through the SOCKS5 proxy, which is connected to the Tor relay. However, Tor do not handle UDP packet(It handles the resolution directly from the hostname in the TCP packet) so DNS resolution will fail.
How is it possible that this code work? (the equivalent code in java for exemple will fail as the DNS resolution can't be made).
IT is explained in this link: Python requests fails when tryign to connect to .onion site
You simply have to use socks5h instead of socks5
I am trying to figure out how to send data to a server through a proxy. I was hoping this would be possible through tor but being as tor uses SOCKS it apparently isn't possible with httplib (correct me if I am wrong)
This is what I have right now
import httplib
con = httplib.HTTPConnection("google.com")
con.set_tunnel(proxy, port)
con.send("Sent Stuff")
The problem is, it seems to freeze when the tunnel is set. Thanks for your help.
If you want to use http proxy, it should be like this:
import httplib
conn = httplib.HTTPConnection(proxyHost, proxyPort)
conn.request("POST", "http://www.google.com", params)
If you want to use SOCKS proxy, you can use SocksiPy as in this question: How can I use a SOCKS 4/5 proxy with urllib2?
Looks like the correct answer is:
http://bugs.python.org/issue11448#msg130413
import httplib
con = httplib.HTTPConnection(proxyHost, proxyPort)
con.set_tunnel("www.google.com", 80)
con.send("Sent Stuff")
As a follow-up to Khue Vu's answer, here's a complete example, the details of getting this working with a SOCKS proxy were more complex than expected.
First install PySocks with:
pip install PySocks
Then you need to manually set up your SOCKS proxy after instantiating your HTTPConnection and informing it that it's going to be using a proxy:
from http.client import HTTPConnection
from urllib.parse import urlparse, urlencode
import socks
url = urlparse("http://final.destination.example.com:8888/")
conn = HTTPConnection('127.0.0.1', 9000) # use socks proxy address
conn.set_tunnel(url.netloc, url.port) # remote host and port that you actually want to talk to
conn.sock = socks.socksocket() # manually set socket
conn.sock.set_proxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9000) # use socks proxy address
conn.sock.connect((url.netloc, url.port)) # remote host and port that you actually want to talk to
request_path = "%s?%s" % (url.path, url.query)
conn.request("POST", request_path, post_data)
Note that the imports above are python3.x