I'm following the tutorial To Russia With Love, and as part of this I want to change the identity everytime I run the code. I've looked at multiple questions and tutorials and from the looks of it my torrc-defaults file is correct as it has:
ControlPort 9151
CookieAuthentication 1
I wanted to edit the torrc file first but if i touch it Tor won't run anymore, so i read somewhere to look at torrc instead. The torrc file alone only has:
# This file was generated by Tor; if you edit it, comments will not be preserved
# The old torrc file was renamed to torrc.orig.1 or similar, and Tor will ignore it
I don't want to set a password so my understanding from reading the stem documents is that setting the Authentication is enough. I have the following code:
import socks # SocksiPy module
import socket
import requests
SOCKS_PORT = 9150
# Set socks proxy and wrap the urllib module
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', SOCKS_PORT)
socket.socket = socks.socksocket
def getaddrinfo(*args):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, '', (args[0], args[1]))]
socket.getaddrinfo = getaddrinfo
url = "http://google.com"
print requests.get(url).elapsed.total_seconds()
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9151) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)
But when I run it i get the error:
socks.SOCKS5Error: 0x01: General SOCKS server failure
I'm not sure what to do, i've been reading about this for hours and haven't been able to solve the problem. Any hints would be great.
EDIT:
I've also read through This Post, but it doesn't seem like there's a solution.
I also tried putting the Controller statement before setting the sockets, but in that case the IP remains the same and doesn't change.
I ended up solving this problem by moving the controller statement before setting the sockets. Initially it looked like the IP remains the same but turns out if you wait an extra 3 or 4 seconds the IP changes, so I just added a time delay and it runs fine.
Related
Hi I'm trying to figure out how to share my tor connection with another pc in my network.I create Tor using expert bundle and torrc file looks like this.
# Where data will be stored?
DataDirectory C:\Users\someuser\Tor\TorData\data\10002
# Countdown time before exit
ShutdownWaitLength 5
# Where to write PID
PidFile C:\Users\someuser\Tor\TorData\data\10002\pid
# Communication ports
SocksPort xx.xxx.xxx.xx:10002
ControlPort xx.xxx.xxx.xx:10003
# Authentication of Tor
CookieAuthentication 1
# GeoIP file paths?
GeoIPFile C:\Users\someuser\Tor\Data\Tor\geoip
GeoIPv6File C:\Users\someuser\Tor\Data\Tor\geoip6
SocksListenAddress xx.xxx.xxx.xx
SocksPolicy accept xx.xxx.xxx.xx/24
So now I'm trying to use that same tor connection from another pc like this.
import requests
ses = requests.session()
ses.proxies = {'http': '{0}{1}:{2}'.format('socks5://','xx.xxx.xxx.xx', 10002),
'https': '{0}{1}:{2}'.format('socks5://','xx.xxx.xxx.xx', 10002)}
r = ses.get('some http webpage')
print r.content
I did check firewall settings that is ok. Over stem I can connect to control port. But dunno why SockPort doesn't pass.
error that im geting no matter the veriso of interpeter is
File "C:\Python27\lib\site-packages\requests\packages\urllib3\contrib\socks.py", line 73, in _new_conn
conn = socks.create_connection(
AttributeError: 'module' object has no attribute 'create_connection'
Any ideas what am I missing?
I found out that is is because of differance in python interperter after all. It must run in python 3 both sides and it will work like a charm.
I hope someone will help torrc configuration.
I am writing a crawler in Python that will run through Tor. I have Tor working and used code from this YouTube tutorial on how to route my Python requests to go through the Tor SOCKS proxy at 127.0.0.1:9050.
What I can't figure out is how to toggle this on/off within my script. Some requests I want to go through Tor and some I don't. Basically, I can't figure out the correct "close" or "shutdown" method in the socket objects I am using because I don't understand them.
Here's what happens now
import socket
import socks
import requests
def connect_to_socks():
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', 9050, True)
socket.socket = socks.socksocket
r = requests.get('http://wtfismyip.com/text')
print r.text #prints my ordinary IP address
connect_to_socks()
r = requests.get('http://wtfismyip.com/text')
print r.text #prints my Tor IP address
How do I turn off the socket routing to the SOCKS proxy so that it goes through my ordinary internet connection?
I'm hoping to use requests instead of urllib2 as it seems a lot easier but if I have to get into the guts of urllib2 or even httplib I will. But would prefer not to.
Figured it out by listening to this good YouTube tutorial.
Just need to call socket.setdefaultproxy() and it brings me back.
For Python 3 you can set back default socket by using this:
socks.setdefaultproxy(None)
socket.socket = socks.socksocket
I'm writing a proxy software, this proxy software support all standard aready, but now the hardest problem, "socks proxy per domain/url (if '???' in self.host: do socks'" without break the whole script by using monkey patching method, I must use monkey patching method because it is the best without any error so far, but if I use that method, my proxy will use that socks server to download all page, not only pages that I want to use socks proxy only because simply monkey patching method "socket" change the whole socket library and make the whole socket library use that socks proxy, and httplib, urllib based on socket library.
https://github.com/Anorov/PySocks
import urllib2
import socket
import socks
socks.set_default_proxy(socks.SOCKS5, "localhost")
socket.socket = socks.socksocket
urllib2.urlopen("http://www.somesite.com/") # All requests will pass through the SOCKS proxy
I must use monkey patch anyway, opener.open method from that page break a lot page like redirect 30x forever, TLSv1 error.... but the monkey patch is "perfect", no bug, no anything, just work but the whole local proxy will use the socks proxy which is overkill, I want to use socks proxy per page.
After day after day non-stop thinking, I figure something, "If I create a new fresh process using multiprocessing, and then monkey patch socket of that process, urlopen to get content from the domain/url like above without affect my local proxy main process and return the content to my main process and just use that content to display to my web browser, that is just great!"
My method is almost like: "I have a proxy listen at 127.0.0.1:1111 and create another proxy listen at 127.0.0.1:2222, the proxy with port 2222 will be monkey patched socket library to make it download page using my socks proxy, and every time I want to use my socks proxy I will chain my 1111 proxy with 2222 proxy".
Plus if that monkey patching method can be done, probably we can do bandwidth throttle by monkey patching socket and more without break the main process.
My idea may be the born of the greatest monkey patch ever, please help me, I will really appreciate if you can help me write something like demo code.
And here is my answer for my question, thank everybody:
import multiprocessing as mp
import urllib.request
def foo(q):
#q.put("123")
import socks
import socket
socks.set_default_proxy(socks.SOCKS5, "localhost", 10080)
socket.socket = socks.socksocket
r = urllib.request.urlopen("http://httpbin.org/ip")
#print("123")
q.put(r.read())
if __name__ == '__main__':
ctx = mp.get_context('spawn')
q = ctx.Queue()
p = ctx.Process(target=foo, args=(q,))
p.start()
print(q.get())
p.join()
r = urllib.request.urlopen("http://httpbin.org/ip")
print(r.read())
This way will create an isolated process and monkey patch socket library from that process.
Output from ILDE:
b'{\n "origin": "xxx.148.2.18"\n}\n'
b'{\n "origin": "xx.xxx.113.133"\n}\n'
I want to make multiple GET requests using Tor to a webpage. I want to use a different ipaddress for each request.
import socks
import socket
socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 9150)
socket.socket = socks.socksocket
import requests
print (requests.get('http://icanhazip.com')).content
Using this, I made one request. How can I change the ipaddress to make another?
There are 2 aspects to your question -
Making requests using Tor
Renewing the connection as per requirement (in your case, after every request)
Part 1
The first one is easy to do with the latest (upwards of v2.10.0) requests library with an additional requirement of requests[socks] for using the socks proxy.
Installation -
pip install requests[socks]
Basic usage -
import requests
def get_tor_session():
session = requests.session()
# Tor uses the 9050 port as the default socks port
session.proxies = {'http': 'socks5://127.0.0.1:9050',
'https': 'socks5://127.0.0.1:9050'}
return session
# Make a request through the Tor connection
# IP visible through Tor
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
# Above should print an IP different than your public IP
# Following prints your normal public IP
print(requests.get("http://httpbin.org/ip").text)
Part 2
To renew the Tor IP, i.e. to have a fresh visible exit IP, you need to be able to connect to the Tor service through it's ControlPort and then send a NEWNYM signal.
Normal Tor installation does not enable the ControlPort by default. You'll have to edit your torrc file and uncomment the corresponding lines.
ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
HashedControlPassword 16:05834BCEDD478D1060F1D7E2CE98E9C13075E8D3061D702F63BCD674DE
Please note that the HashedControlPassword above is for the password "password". If you want to set a different password, replace the HashedControlPassword in the torrc by noting the output from tor --hash-password "<new_password>" where <new_password> is the password that you want to set.
................................................................................
Warning for Windows users: see post here.
There is an issue on windows where the setting for the controlport in the torrc file is ignored if tor was installed using the following command:
tor --service install
To resolve the issue, after editing your torrc file, type the following commands:
tor --service remove
tor --service install -options ControlPort 9051
................................................................................
Okay, so now that we have Tor configured properly, you will have to restart Tor if it is already running.
sudo service tor restart
Tor should now be up & running on the 9051 ControlPort through which we can send commands to it. I prefer to use the official stem library to control Tor.
Installation -
pip install stem
You may now renew the Tor IP by calling the following function.
Renew IP -
from stem import Signal
from stem.control import Controller
# signal TOR for a new connection
def renew_connection():
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password="password")
controller.signal(Signal.NEWNYM)
To verify that Tor has a new exit IP, just rerun the code from Part 1. For some reason unknown to me, you need to create a new session object in order to use the new IP.
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
Here is the code you want to use (download the stem package using pip install stem)
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password='your password set for tor controller port in torrc')
print("Success!")
controller.signal(Signal.NEWNYM)
print("New Tor connection processed")
Good luck and hopefully that works.
You can try pure-python tor protocol implementation Torpy. No need original Tor client or Stem dependency at all.
$ pip3 install torpy[requests]
...
$ python3.7
>>> from torpy.http.requests import TorRequests
>>> with TorRequests() as tor_requests:
... print("build circuit")
... with tor_requests.get_session() as sess:
... print(sess.get("http://httpbin.org/ip").json())
... print(sess.get("http://httpbin.org/ip").json())
... print("renew circuit")
... with tor_requests.get_session() as sess:
... print(sess.get("http://httpbin.org/ip").json())
... print(sess.get("http://httpbin.org/ip").json())
...
build circuit
{'origin': '23.129.64.190, 23.129.64.190'}
{'origin': '23.129.64.190, 23.129.64.190'}
renew circuit
{'origin': '198.98.50.112, 198.98.50.112'}
{'origin': '198.98.50.112, 198.98.50.112'}
So each time when you getting new session you get new identity (basically you get new circuit with new exit node). See more examples at readme file https://github.com/torpyorg/torpy
You can use torrequest library (shameless plug). It's available on PyPI.
from torrequest import TorRequest
with TorRequest() as tr:
response = tr.get('http://ipecho.net/plain')
print(response.text) # not your IP address
tr.reset_identity()
response = tr.get('http://ipecho.net/plain')
print(response.text) # another IP address, not yours
Requests supports proxies using the SOCKS protocol from version 2.10.0.
import requests
proxies = {
'http': 'socks5://localhost:9050',
'https': 'socks5://localhost:9050'
}
url = 'http://httpbin.org/ip'
print(requests.get(url, proxies=proxies).text)
This answer complete the one of Ashish Nitin Patil for windows
(feel free to update this answer)
Part 2
ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
HashedControlPassword 16:05834BCEDD478D1060F1D7E2CE98E9C13075E8D3061D702F63BCD674DE
The HashedControlPassword above is the password. If you want to set a different password in the console navigate to \Tor Browser\Browser\TorBrowser\Tor and type the following commands: tor.exe --hash-password password_XYZ | more). It will give you something like HashedControlPassword 16:54C092A8... This is your password. Now you can add it to the torrc file (Tor Browser\Browser\TorBrowser\Data\Tor\torrc).
You will need then to restart Tor:
tor --service remove
tor --service install -options ControlPort 9051
To check if that works type netstat -an you will now see that port 9051 is open.
Notice that tor --service install -... will create Tor Win32 Service. For some reason, it seems you have to stop the service to use the browser (run services.msc)
EDIT: you will find many pieces of information here (About port number & proxy, Tor, Privoxy, Auto switch user agent...).
This code works fine. Using Tor, it changes the IP address after each request.
import time, socks, socket
from urllib2 import urlopen
from stem import Signal
from stem.control import Controller
nbrOfIpAddresses=3
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password = 'my_pwd')
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050)
socket.socket = socks.socksocket
for i in range(0, nbrOfIpAddresses):
newIP=urlopen("http://icanhazip.com").read()
print("NewIP Address: %s" % newIP)
controller.signal(Signal.NEWNYM)
if controller.is_newnym_available() == False:
print("Waitting time for Tor to change IP: "+ str(controller.get_newnym_wait()) +" seconds")
time.sleep(controller.get_newnym_wait())
controller.close()
The requests in requesocks is super old, it doesn't have response.json() and many other stuff.
I would like to keep my code clean. However, requests currently doesn't have socks5 supported yet (for more detail, read this thread https://github.com/kennethreitz/requests/pull/478)
So I used Privoxy as a http proxy that connects Tor for now.
Install and configure Privoxy on Mac
brew install privoxy
vim /usr/local/etc/privoxy/config
# put this line in the config
forward-socks5 / localhost:9050 .
privoxy /usr/local/etc/privoxy/config
Install and configure Privoxy on Ubuntu
sudo apt-get install privoxy
sudo vim /etc/privoxy/config
# put this line in the config
forward-socks5 / localhost:9050 .
sudo /etc/init.d/privoxy restart
Now I can use Tor like a http proxy. Below is my python script.
import requests
proxies = {
'http': 'http://127.0.0.1:8118',
}
print requests.get('http://httpbin.org/ip', proxies=proxies).text
A good function to renew your IP. Windows example
def renew_tor_ip():
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password="aAjkaI19!!laksjd")
controller.signal(Signal.NEWNYM)
Example of usage
import requests
import time
from stem import Signal
from stem.control import Controller
def get_current_ip():
session = requests.session()
# TO Request URL with SOCKS over TOR
session.proxies = {}
session.proxies['http']='socks5h://localhost:9150'
session.proxies['https']='socks5h://localhost:9150'
try:
r = session.get('http://httpbin.org/ip')
except Exception as e:
print(str(e))
else:
return r.text
#16:8EE7AEE3F32EEEEB605C6AA6C47B47808CA6A81FA0D76546ADC05F0F15 to aAjkaI19!!laksjd
#cmd shell "C:\Users\Arthur\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe" --hash-password aAjkaI19!!laksjd | more
#Torcc config
#ControlPort 9051
#HashedControlPassword 16:8EE7AEE3F32EEEEB605C6AA6C47B47808CA6A81FA0D76546ADC05F0F15
def renew_tor_ip():
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password="aAjkaI19!!laksjd")
controller.signal(Signal.NEWNYM)
for i in range(5):
print(get_current_ip())
renew_tor_ip()
time.sleep(5)
I'm using python 2.7 and I'd like to get the contents of a webpage that requires sslv3. Currently when I try to access the page I get the error SSL23_GET_SERVER_HELLO, and some searching on the web lead me to the following solution which fixes things in Python 3
urllib.request.install_opener(urllib.request.build_opener(urllib.request.HTTPSHandler(context=ssl.SSLContext(ssl.PROTOCOL_TLSv1))))
How can I get the same effect in python 2.7, as I can't seem to find the equivalent of the context argument for the HTTPSHandler class.
I realize this response is a few years too late, but I also ran into the same problem, and didn't want to depend on libcurl being installed on a machine where I ran this. Hopefully, this will be useful to those who find this post in the future.
The problem is that httplib.HTTPSConnection.connect doesn't have a way to specify SSL context or version. You can overwrite this function before you hit the meat of your script for a quick solution.
An important consideration is that this workaround, as discussed above, will not verify the validity of the server's certificate.
import httplib
import socket
import ssl
import urllib2
def connect(self):
"Connect to a host on a given (SSL) port."
sock = socket.create_connection((self.host, self.port),
self.timeout, self.source_address)
if self._tunnel_host:
self.sock = sock
self._tunnel()
self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, ssl_version=ssl.PROTOCOL_TLSv1)
httplib.HTTPSConnection.connect = connect
opener = urllib2.build_opener()
f = opener.open('https://www.google.com/')
*Note: this alternate connect() function was copy/pasted from httplib.py, and simply modified to specify the ssl_version in the wrap_socket() call
SSL should be handled automatically as long as you have the SSL libraries installed on your server (i.e. you shouldn't have to specificially add it as a handler)
http://docs.python.org/library/urllib2.html#urllib2.build_opener
If the Python installation has SSL support (i.e., if the ssl module can be imported), HTTPSHandler will also be added.
Also, note that urllib and urllib2 have been merged in python 3 so their approach is a little different
Since I was unable to do this using urllib2, I eventually gave in and moved to using the libCurl bindings like #Bruno had suggested in the comments to pastylegs answer.