I have a problem with my ssl server (in Python).
I set the SSL proxy connection in my browser, and try to connect to my ssl server.
This is the server:
import BaseHTTPServer, SimpleHTTPServer
import ssl
httpd = BaseHTTPServer.HTTPServer(('0.0.0.0', 443), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket (httpd.socket, server_side=True, certfile='server.crt', keyfile='server.key', do_handshake_on_connect=False)
httpd.serve_forever()
This is the error:
SSLError: [SSL: HTTPS_PROXY_REQUEST] https proxy request (_ssl.c:1750)
I try to connect to the server in the browser.
its work if I went to address "https://127.0.0.1:443".
But, if I use in the server to proxy, I get the error...
How can I fix this?
I don't think you understand how a proxy server for HTTPS works.
What you are doing is to create a plain HTTPS server. What you should do is to create a HTTP server which handles the CONNECT request and creates a tunnel to the requested target. See http://en.wikipedia.org/wiki/HTTP_tunnel#HTTP_CONNECT_tunneling
Related
I am writing a python socket server with ssl and I am encountering certificate unknown error during ssl handshake.
I have created private key and certificate with openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes command on my own.
This server is intended to run in intranet(under wifi) in my PC, and users will contact my PC IPaddress with their browser. Hence I have not registered this certificate with any CA. and I don't think its mandatory in my case.
Below more details..
echoserver.py
import socket
import ssl
import threading
class echoserver:
def __init__(self,i,p):
self.ip=i
self.port=p
def handler(self,c,a):
msg=c.recv(1024)
c.send(msg)
def serve(self):
echoserver = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
echoserver = ssl.wrap_socket(echoserver, keyfile='keys/key.pem', certfile='keys/cert.pem', server_side=True)
echoserver.bind((self.ip, self.port))
echoserver.listen()
while True:
(c,a)=echoserver.accept()
threading.Thread(target=self.handler, args=(c,a)).start()
es=echoserver('192.168.43.124',443) #My PC's ip assigned under wifi network
es.serve()
#Connecting from mobile phone within same network as https://192.163.43.124
Error in server during ssl handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1108)
What I tried
Adding cert_reqs=ssl.CERT_NONE and ca_certs="/location/to/keys" parameters in wrap_socket function.
Doesn't seems to work. I assume these options are for client side.
Adding do_handshake_on_connect=False in wrap_socket function
In Chrome, When connected server throws same error and thread/connection closed with exception. and chrome seems to send same connection request immediately again, and the second request works flawlessly !!.
In firefox, First connection closed with same error and there is no second request.
Assigning common name in certificate same as IP address
Not working.
Checked certificate_unknown error in ietf specification here. It gives no clue except this explanation certificate_unknown: Some other (unspecified) issue arose in processing the certificate, rendering it unacceptable.
One other thing I noted is, if I use built-in ThreadedHTTPServer in the same way as below, It works beautifully, without any issues I mentioned above.
httpd = self.ThreadedHTTPServer((self.ip, self.port), self.handler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile='keys/key.pem', certfile='keys/cert.pem', server_side=True)
I am not sure why this happens and how should I proceed with this. and not sure how built-in server modules works fine.
Appreciate any leads. Thanks.
Below Python builtin HTTPServer works fine with ssl, not showing any error. Not sure how?
import ssl
from http.server import HTTPServer, BaseHTTPRequestHandler
class requesthandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-type","text/html")
self.end_headers()
self.wfile.write("<html><body>It works</body></html>".encode('utf8'))
httpd = HTTPServer(('192.168.43.124', 443), requesthandler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile='keys/key.pem', certfile='keys/cert.pem', server_side=True)
httpd.serve_forever()
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1108)
This means the client (browser) does not trust your certificate since it is issued by an unknown entity. If you want to use self-signed certificates you have to explicitly import these as trusted for all clients you want to use.
There is no way around this. The certificate in TLS is to make sure that the connection is done with the expected server and not some man in the middle claiming to be the expected server. If a client would trust arbitrary certificates then it would also trust certificates created by a man in the middle attacker.
Below Python builtin HTTPServer works fine with ssl, not showing any error. Not sure how?
The browser will still complain.
The only difference is that the server captures the exception and thus will not crash but continue. You can do the same in your code:
while True:
try:
(c,a)=echoserver.accept()
threading.Thread(target=self.handler, args=(c,a)).start()
except:
pass # ignore error
I want to make an anonymous web request using python 3.
I've tried few suggestions such as: Make requests using Python over Tor
I've managed to get a fake ip using this snippet:
Installation
pip install requests requests[socks]
Basic usage
import requests
def get_tor_session():
session = requests.session()
# Tor uses the 9050 port as the default socks port
session.proxies = {'http': 'socks5://127.0.0.1:9150',
'https': 'socks5://127.0.0.1:9150'}
return session
# Make a request through the Tor connection
# IP visible through Tor
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
# Above should print an IP different than your public IP
# Following prints your normal public IP
print(requests.get("http://httpbin.org/ip").text)
But that works only on port 9150 and when the tor web browser works.
I want to make a request without the tor browser, as i want to Dockerize the whole thing.
I've read about Socks5, and as you can see i've installed it, but when i make a request on port 9050 on the same snippet i get:
requests.exceptions.ConnectionError:
SOCKSHTTPConnectionPool(host='httpbin.org', port=80): Max retries
exceeded with url: /ip (Caused by
NewConnectionError(': Failed to establish a new connection: [WinError
10061] No connection could be made because the target machine actively
refused it',))
How can i solve it?
Thanks!
10061 is 'connection refused'
That means there was nothing listening on that port you tried to connect to, no service is up and running (no open port) or firewall on target IP blocks it
you can test that port with telnet
telnet `IP` `PORT`
And also check this port issue on Windows: here
I was also facing this issue, in my case my tor service was not running, actually I was using kalitorify which is a transparent proxy, and whenever I was using this I was not be able to use normal sites such as google search or similar, so to use these sites I was turning off my kalitorify service which also turns off your tor service
So if you're also using that then also check it once
I have a python file running a server:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import json
class S(BaseHTTPRequestHandler):
def _set_headers(self):
self.send_response(200)
self.send_header('Content-type', 'application/json')
self.end_headers()
def do_GET(self):
self._set_headers()
with open('files.json') as myfile:
self.wfile.write(myfile.read())
....
def run(server_class=HTTPServer, handler_class=S, port=8080):
server_address = ('', port)
httpd = server_class(server_address, handler_class)
print 'Starting httpd...'
httpd.serve_forever()
And another doing a get request:
import requests
SESSION = requests.Session()
SESSION.trust_env = False
url = "http://localhost:8080"
response = SESSION.get(url)
data = response.json()
This works fine when I run the two normally. However when I containerise the app (not the server), requests throws this error at me:
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ec03e5ed0>: Failed to establish a new connection: [Errno 111] Connection refused',))
I assume this is because docker somehow prevents me from connecting to my server on localhost:8080?
Thanks for you help ☺
A container is meant to be an isolated environment. It's a "jail" for running processes protecting the host os. Think of containers as a virtual machine with its own ip address. localhost would be the container trying to connect to itself.
I'm guessing you could contact port 8080 on the host using its public ip address.
It would make more sense to run the server in docker. Then you can map a port in the server container to a port on your host. Then your client would work as expected.
The two most common ways a container can interact with the host os is:
When mapping network ports from the container to to host
When mapping files as volumes from the host into the container
It's fairly restrictive for very good reasons.
I have developed a desktop client using PyQt4, it connect to my web service by requests lib. You know, requests maybe one of the most useful http client, I think it should be no problem. My desktop client works all right until something strange happened.
I use the following code to send request to my server.
response = requests.get(url, headers = self.getHeaders(), timeout=600, proxies = {}, verify = False)
where header only includes auth token.
def getHeaders(self, additional = None):
headers = {
'Auth-Token' : HttpBasicClient.UserAuthToken,
}
if additional is not None:
headers.update(additional)
return headers
I cannot connect to my web service, all the http request pop the same error "'Cannot connect to proxy.', error(10061, '')". For example:
GET Url: http:// api.fangcloud.com/api/v1/user/timestamp
HTTPSConnectionPool(host='api.fangcloud.com', port=443): Max retries exceeded with url: /api/v1/user/timestamp (Caused by ProxyError('Cannot connect to proxy.', error(10061, '')))
this API does nothing but return the timestamp of my server. When I copy the url into Chrome in same machine with same environment, it returns correct response. However, my desktop client can only returns error. Is it anything wrong with requests lib?
I googled this problem of connection error 10061 ("No connection could be made because the target machine actively refused it"). This maybe caused by TCP connect rejection of web server.
The client sends a SYN packet to the server targeting the port (80 for HTTP). A server that is running a service on port 80 will respond with a SYN ACK, but if it is not, it will respond with a RST ACK. Your client reaches the server, but not the intended service. This is one way a server could “actively refuse” a connection attempt.
But why? My client works all right before and Chrome still works. I use no proxy on my machine. Is there anything I miss?
I notice there is a white space in URL, is that correct?
I tested in my ipython with requests.. that the response was:
{
"timestamp": 1472760770,
"success": true
}
For HTTP and HTTPS.
I am using SSL tunneling with a proxy server to connect to a target server. I use http to connect to the proxy server and HTTPS to connect to the target server. The SSL tunneling works as it should and I can exchange HTTPS messages with the remote server, but there is a problem. The proxy server returns a header in its reply to urllib2's request to establish the SSL tunnel that I need to see, but I don't see a way to get access to it using urllib2 (Python 2.7.3).
I suppose I could theoretically implement the SSL tunneling handshake myself, but that would get me way deeper into the protocol than I want to be (or with which I feel comfortable).
Is there a way to get access to the reply using urllib2 when establishing the SSL tunnel?
UPDATE:
Here is the code that uses the proxy server to connect to the target server (the proxy server and the target server's URLs are not the actual ones):
proxy_handler = urllib2.ProxyHandler({'https': 'http://proxy.com'})
url_opener = urllib2.build_opener (proxy_handler)
request = urllib2.Request ('https://target_server.com/')
response = url_opener.open (request)
print response.headers.dict
I used WireShark to look at the message traffic. WireShark won't show me the bodies of the messages exchanged with the target server because they are encrypted, but I can see the body of the SSL Tunnel handshake. I can see the header that I'm interested coming back from the proxy server.
How are you calling the https page.
are you using
resp = urllib2.urlopen('https')
resp.info().headers