Im trying to open a simple webpage with python using Urlib2 in a TAILS system but I am unable to make it work, with error 111, connection refused. So that is obviously because TAILS refuses all non-Tor traffic,and the solution should be opening a proxy with:
proxy = urlib2.ProxyHandler({protocol:"127.0.0.1:{}".format(PORT)})
opener = urlib2.build_opener(proxy)
opener.open("https://jojeji")
I tried lots of protocols and ports, asumming that this choice is the only problem I have. Tails has a configuration file in etc/ferm/ferm.conf. In the output part it says:
# White-list access to Tor's SOCKSPort's
daddr 127.0.0.1 proto tcp syn dport 9050 {
mod owner uid-owner _apt ACCEPT;
mod owner uid-owner proxy ACCEPT;
mod owner uid-owner nobody ACCEPT;
}
daddr 127.0.0.1 proto tcp syn mod multiport
destination-ports (9050 9061 9062 9150) {
mod owner uid-owner $amnesia_uid ACCEPT;
}
There are white lists also for:
access to onionshare
access to Monkeysphere
access to CUPS
access to the accesibility daemon
access to system DNS and TOr's DNS PORT
access to Tor's TransPort
access to Tor control port filtrer
access to Tor ControlPort
So wich values should a TAILS user pass to the urlib2 proxy and, is there any better way to acces the internet trough urlib2?
Tails/Tor implements SOCKS5 proxy, not HTTP proxy. AFAIK, this is not directly supported by urlib2. You need to use the SocksiPy module as shown in this answer.
The socks proxy is 127.0.0.1 and the port is 9050.
Your other option is to use pycurl or even command-line curl (with --socks5-hostname=127.0.0.1:9050)
Related
I want to make an anonymous web request using python 3.
I've tried few suggestions such as: Make requests using Python over Tor
I've managed to get a fake ip using this snippet:
Installation
pip install requests requests[socks]
Basic usage
import requests
def get_tor_session():
session = requests.session()
# Tor uses the 9050 port as the default socks port
session.proxies = {'http': 'socks5://127.0.0.1:9150',
'https': 'socks5://127.0.0.1:9150'}
return session
# Make a request through the Tor connection
# IP visible through Tor
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
# Above should print an IP different than your public IP
# Following prints your normal public IP
print(requests.get("http://httpbin.org/ip").text)
But that works only on port 9150 and when the tor web browser works.
I want to make a request without the tor browser, as i want to Dockerize the whole thing.
I've read about Socks5, and as you can see i've installed it, but when i make a request on port 9050 on the same snippet i get:
requests.exceptions.ConnectionError:
SOCKSHTTPConnectionPool(host='httpbin.org', port=80): Max retries
exceeded with url: /ip (Caused by
NewConnectionError(': Failed to establish a new connection: [WinError
10061] No connection could be made because the target machine actively
refused it',))
How can i solve it?
Thanks!
10061 is 'connection refused'
That means there was nothing listening on that port you tried to connect to, no service is up and running (no open port) or firewall on target IP blocks it
you can test that port with telnet
telnet `IP` `PORT`
And also check this port issue on Windows: here
I was also facing this issue, in my case my tor service was not running, actually I was using kalitorify which is a transparent proxy, and whenever I was using this I was not be able to use normal sites such as google search or similar, so to use these sites I was turning off my kalitorify service which also turns off your tor service
So if you're also using that then also check it once
I have setup a django app on a apache server in a VM. The site is accessible when I use the apache server IP in the VM browser. It is also accessible from the host browser through it's local IP. But, I cannot access it through the public IP over the internet. I get a site can't be reached error
I have set up port forwarding such that:
1. All router requests on port 80 are forwarded to local IP
2. All requests to local IP on port 80 are forwarded to the VM
I checked if my port is open on my public ip using http://www.yougetsignal.com/tools/open-ports/
It says that my port is closed. Same results with http://canyouseeme.org/
I am able to ping my public ip successfully.
I have tried disabling all my firewalls but this has not helped. Please tell me if you need any code to be shared. Any help would be appreciated.
Edit:
Extra information: It seems my router's WAN IP is different from my public IP. I can access the site through the WAN IP from the host browser but again, I am not able to access it over the internet.
You just need a public IP address or push your application to the hosting (like this for example https://gpdhost.com/offers/).
ToDo: learn DMZ, learn WAN-LAN packet forwarding process, learn TCP/IP routing, learn public and private IP addressing and learn NAT.
Description: http/https connection conversation (client-outside vs your-server):
1) client: in browser write: sharan-site/;
2) get IP by DNS name from public servers? But public servers don't know your ip:dns-name pair...
=> fail
next example:
1) client: 192.168.1.1/ - where IP is your server
2) so where is it?
=> nowhere, it is private IP address! Fail...
Desc+: Port forwarding it is NAT feature. Your router must have public IP address, and you must setup DMZ like scheme in your local network: https://www.cisco.com/c/en/us/support/docs/ip/network-address-translation-nat/13772-12.html
Desc++: that sites check outside tcp/80 port on your router and PC, and it's open, no questions... But this no help for your task.
Supposing this working code :
import win_inet_pton
import socks
import socket
s = socks.socksocket()
s.set_proxy(socks.SOCKS5, "localhost", 9050)
s.connect(("xmh57jrzrnw6insl.onion",80))
s.send("GET / HTTP/1.1\r\n\r\n")
print 'sended'
data=s.recv(1024)
print data
Tor service is indeed running at port 9050.
In normal condition, python will perform DNS resolution through the SOCKS5 proxy, which is connected to the Tor relay. However, Tor do not handle UDP packet(It handles the resolution directly from the hostname in the TCP packet) so DNS resolution will fail.
How is it possible that this code work? (the equivalent code in java for exemple will fail as the DNS resolution can't be made).
IT is explained in this link: Python requests fails when tryign to connect to .onion site
You simply have to use socks5h instead of socks5
I am creating a proxy server in python, which is based on BaseHTTPServer.
What it does is create a connection to a squid proxy, identifies the browser request(GET, CONNECT, POST etc) and adds a proxy-authorization header to it, and then forwards this request to the squid proxy.
Problem is, as I understand, when I send a connect request, I should relay all the corresponding traffic to the squid proxy. But, as I can see in wireshark, the squid proxy doesn't reply to the 'Client Hello' part of the handshake, which I think is due to squid proxy not understanding binary data of SSL that I am just forwarding to it.
How do I process HTTPS requests in this case?
The code is more or less similar to TinyHTTPProxy : http://www.oki-osk.jp/esc/python/proxy/
RFC 2817 defines the CONNECT method. It is different from other HTTP methods in that the receiving proxy (your Python proxy) is directed to establish a raw TCP tunnel directly to the destination host (called the authority in the RFC).
A proxy can make no assumptions about the data that will be sent over that tunnel; it will not necessarily be HTTP – the client can use the tunnel to speak any protocol it likes. Indeed, SSL ≠ HTTP.
You have two options:
Open a TCP connection directly to the requested destination host.
Make a CONNECT request to your upstream proxy (Squid). This is within spec:
It may be the case that the proxy itself can only reach the
requested origin server through another proxy. In this case, the
first proxy SHOULD make a CONNECT request of that next proxy,
requesting a tunnel to the authority. A proxy MUST NOT respond
with any 2xx status code unless it has either a direct or tunnel
connection established to the authority.
Make sure that your request includes the required Host header.
CONNECT www.google.com:443 HTTP/1.1
Host: www.google.com:443
Proxy-Authorization: ...
I am using SSL tunneling with a proxy server to connect to a target server. I use http to connect to the proxy server and HTTPS to connect to the target server. The SSL tunneling works as it should and I can exchange HTTPS messages with the remote server, but there is a problem. The proxy server returns a header in its reply to urllib2's request to establish the SSL tunnel that I need to see, but I don't see a way to get access to it using urllib2 (Python 2.7.3).
I suppose I could theoretically implement the SSL tunneling handshake myself, but that would get me way deeper into the protocol than I want to be (or with which I feel comfortable).
Is there a way to get access to the reply using urllib2 when establishing the SSL tunnel?
UPDATE:
Here is the code that uses the proxy server to connect to the target server (the proxy server and the target server's URLs are not the actual ones):
proxy_handler = urllib2.ProxyHandler({'https': 'http://proxy.com'})
url_opener = urllib2.build_opener (proxy_handler)
request = urllib2.Request ('https://target_server.com/')
response = url_opener.open (request)
print response.headers.dict
I used WireShark to look at the message traffic. WireShark won't show me the bodies of the messages exchanged with the target server because they are encrypted, but I can see the body of the SSL Tunnel handshake. I can see the header that I'm interested coming back from the proxy server.
How are you calling the https page.
are you using
resp = urllib2.urlopen('https')
resp.info().headers