I am working on a project using flask-socketio and python-socketio. In order to secure the communication, the system needs to be upgraded with SSL. I am working with Mac and Python 3.7.
I created a certificate using openssl (rootCA.pem) and added it to the Mac's keychain. After that, I issued server.cert and server.key with the rootCA.pem. I built the web server using flask-socketio, added server.cert and server.key on its side. In the end, the website worked well with ssl.
The problem happened when I wanted to secure the communication between flask-socketio server and python-socketio client. When the client tried to connect with server, the connection was refused and raised unknown ca error:
ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:2488)
I did some researches and realized that this problem may be due to that Python 3.7 uses its own private copy of OpenSSL. Because of that, the SocketIO client was unable to verify my self-signed certificate. And this is the information from Python 3.7's installation:
This variant of Python 3.7 includes its own private copy of OpenSSL
1.1.1. The deprecated Apple-supplied OpenSSL libraries are no longer used. This means that the trust certificates in system and user
keychains managed by the Keychain Access application and the security
command line utility are no longer used as defaults by the Python ssl
module. A sample command script is included in /Applications/Python
3.7 to install a curated bundle of default root certificates from the third-party certifi package (https://pypi.org/project/certifi/). If
you choose to use certifi, you should consider subscribing to the
project's email update service to be notified when the certificate
bundle is updated.
So I went to the folder of /Applications/Python 3.7 and executed the Install Certificates.command.
However, it is not the key to the problem. Because python certifi is a carefully curated collection of Root Certificates. It, of course, does not contain my self-signed certificate. So, to solve the problem, I need to let the Python find the rootCA.pem.
In the folder of python certifi, there is a document named cacert.pem. This contains the Root Certificates. So I added the content of rootCA.pem in the end of cacert.pem.
After that, I tried this code in the command line:
openssl verify -CAfile cacert.pem server.crt
And the output:
server.crt: OK
I think, in this case, the cacert.pem can verify the certificate of the server. I know it is not an elegant solution. Please let me know if you have better way to make python find my self-signed certificate.
After that, I tried to connect to flask-socketio server with the python-socketio. This time, I got another error. On the server side, the log information is:
(57183) accepted ('127.0.0.1', 56322)
0fd838477c534dfda802bfb0d130d358: Sending packet OPEN data {'sid': '0fd838477c534dfda802bfb0d130d358', 'upgrades': ['websocket'], 'pingTimeout': 60000, 'pingInterval': 25000}
0fd838477c534dfda802bfb0d130d358: Sending packet MESSAGE data 0
127.0.0.1 - - [10/Oct/2019 08:50:36] "GET /socket.io/?transport=polling&EIO=3&t=1570706436.665149 HTTP/1.1" 200 349 0.000436
(57183) accepted ('127.0.0.1', 56324)
http://localhost:3000 is not an accepted origin.
127.0.0.1 - - [10/Oct/2019 08:50:36] "GET /socket.io/?transport=websocket&EIO=3&sid=0fd838477c534dfda802bfb0d130d358&t=1570706436.685631 HTTP/1.1" 400 122 0.000182
And on the client side, the error was thrown like this:
Traceback (most recent call last):
File "simple_client.py", line 18, in <module>
sio.connect('https://localhost:3000', namespaces="/channel_A")
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/socketio/client.py", line 262, in connect
engineio_path=socketio_path)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/engineio/client.py", line 170, in connect
url, headers, engineio_path)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/engineio/client.py", line 308, in _connect_polling
if self._connect_websocket(url, headers, engineio_path):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/engineio/client.py", line 346, in _connect_websocket
cookie=cookies)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/websocket/_core.py", line 514, in create_connection
websock.connect(url, **options)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/websocket/_core.py", line 226, in connect
self.handshake_response = handshake(self.sock, *addrs, **options)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/websocket/_handshake.py", line 79, in handshake
status, resp = _get_resp_headers(sock)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/websocket/_handshake.py", line 160, in _get_resp_headers
raise WebSocketBadStatusException("Handshake status %d %s", status, status_message, resp_headers)
websocket._exceptions.WebSocketBadStatusException: Handshake status 400 BAD REQUEST
I have no idea why does it happen. The communication between flask-socketio server and python-socketio client works well without SSL. So I think there might be nothing wrong with the code, but I still give the code here. And this is for server:
from flask import Flask, render_template
from flask_socketio import SocketIO
from flask_socketio import Namespace
import eventlet
app = Flask(__name__, template_folder="templates")
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app, engineio_logger=True, logger=True)
# Create a URL route in our application for "/"
#app.route('/')
def home():
"""
This function loads the homepage
"""
return render_template('index.html')
class MyCustomNamespace(Namespace):
def on_connect(self):
print("Client just connected")
def on_disconnect(self):
print("Client just left")
def on_messages(self, data):
print(f"\nReceived data from client: \n {data}\n")
return data
socketio.on_namespace(MyCustomNamespace('/channel_A'))
if __name__ == "__main__":
eventlet.wsgi.server(
eventlet.wrap_ssl(eventlet.listen(("localhost", 3000)),
certfile='server.crt',
keyfile='server.key',
server_side=True), app)
# socketio.run(app, host="localhost", port=3000, debug=True)
This is for client:
import socketio
sio = socketio.Client()
def message_received(data):
print(f"Message {data} received")
#sio.on('connect', namespace="/channel_A")
def on_connect():
print("Connect...")
#sio.on('disconnect', namespace="/channel_A")
def on_disconnect():
print(f"Disconnected from server")
sio.connect('https://localhost:3000', namespaces="/channel_A")
Please help! I know my question is wordy. But I just want to show the way I tried to solve the problem. If there is anything wrong, please let me know. Thank you!
Recent versions of Flask-SocketIO come configured with the most secure settings with regards to cross-origins setups, which is to only allow the same origin. If your frontend app and your Flask app are running on different servers or ports, then you have to configure cross-origin so that they can work together.
For example, if your frontend is hosted at http://localhost:3000, you can allow that as an origin with:
socketio = SocketIO(app, cors_allowed_origins='http://localhost:3000')
Related
I am writing a python socket server with ssl and I am encountering certificate unknown error during ssl handshake.
I have created private key and certificate with openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes command on my own.
This server is intended to run in intranet(under wifi) in my PC, and users will contact my PC IPaddress with their browser. Hence I have not registered this certificate with any CA. and I don't think its mandatory in my case.
Below more details..
echoserver.py
import socket
import ssl
import threading
class echoserver:
def __init__(self,i,p):
self.ip=i
self.port=p
def handler(self,c,a):
msg=c.recv(1024)
c.send(msg)
def serve(self):
echoserver = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
echoserver = ssl.wrap_socket(echoserver, keyfile='keys/key.pem', certfile='keys/cert.pem', server_side=True)
echoserver.bind((self.ip, self.port))
echoserver.listen()
while True:
(c,a)=echoserver.accept()
threading.Thread(target=self.handler, args=(c,a)).start()
es=echoserver('192.168.43.124',443) #My PC's ip assigned under wifi network
es.serve()
#Connecting from mobile phone within same network as https://192.163.43.124
Error in server during ssl handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1108)
What I tried
Adding cert_reqs=ssl.CERT_NONE and ca_certs="/location/to/keys" parameters in wrap_socket function.
Doesn't seems to work. I assume these options are for client side.
Adding do_handshake_on_connect=False in wrap_socket function
In Chrome, When connected server throws same error and thread/connection closed with exception. and chrome seems to send same connection request immediately again, and the second request works flawlessly !!.
In firefox, First connection closed with same error and there is no second request.
Assigning common name in certificate same as IP address
Not working.
Checked certificate_unknown error in ietf specification here. It gives no clue except this explanation certificate_unknown: Some other (unspecified) issue arose in processing the certificate, rendering it unacceptable.
One other thing I noted is, if I use built-in ThreadedHTTPServer in the same way as below, It works beautifully, without any issues I mentioned above.
httpd = self.ThreadedHTTPServer((self.ip, self.port), self.handler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile='keys/key.pem', certfile='keys/cert.pem', server_side=True)
I am not sure why this happens and how should I proceed with this. and not sure how built-in server modules works fine.
Appreciate any leads. Thanks.
Below Python builtin HTTPServer works fine with ssl, not showing any error. Not sure how?
import ssl
from http.server import HTTPServer, BaseHTTPRequestHandler
class requesthandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-type","text/html")
self.end_headers()
self.wfile.write("<html><body>It works</body></html>".encode('utf8'))
httpd = HTTPServer(('192.168.43.124', 443), requesthandler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile='keys/key.pem', certfile='keys/cert.pem', server_side=True)
httpd.serve_forever()
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1108)
This means the client (browser) does not trust your certificate since it is issued by an unknown entity. If you want to use self-signed certificates you have to explicitly import these as trusted for all clients you want to use.
There is no way around this. The certificate in TLS is to make sure that the connection is done with the expected server and not some man in the middle claiming to be the expected server. If a client would trust arbitrary certificates then it would also trust certificates created by a man in the middle attacker.
Below Python builtin HTTPServer works fine with ssl, not showing any error. Not sure how?
The browser will still complain.
The only difference is that the server captures the exception and thus will not crash but continue. You can do the same in your code:
while True:
try:
(c,a)=echoserver.accept()
threading.Thread(target=self.handler, args=(c,a)).start()
except:
pass # ignore error
I am trying to send socket.io message to a nodejs server over HTTPS. After googling around for the socketIO on Python; I have written the below simple code to make a test.
Although I receive no compile error, I receive nothing from the server. When I analyzed via wireshark packetsniffer; I noticed that HTTPS connection is not even established (target server has the valid signed certificate and I can access the server via JS based frontend without any problem)..
from socketIO_client import SocketIO, LoggingNamespace
NODEJS_URL = "https://targetserver.com"
def on_message_response(*args):
print('message', args)
user_value = "test1"
message_value = "hello"
with SocketIO(NODEJS_URL, 443, LoggingNamespace) as socketIO:
socketIO.emit('message', { 'user': user_value, 'message': message_value}, on_message_response)
socketIO.wait_for_callbacks(seconds=1)
Do you have any idea why Python socketIO connection might not be successful?
I'm developing a site with Python and Flask and want to move from localhost in my browser to my team's development server and development site but I'm having problems. I've already looked at the Flask documentation, which recommends using host='0.0.0.0' or specifying SERVER_NAME. Thusfar, neither of these things have helped. Here's a bit of code to show what I'm doing:
app = Flask(__name__)
if __name__ == '__main__':
app.secret_key = 'super secret key'
app.config['SERVER_NAME'] = 'https://example.org/blah'
app.run(host=app.config['SERVER_NAME'], port=4343, debug=True)
As you can see, instead of localhost:500, I want to be able to go into my browser and visit 'https://example.org/blah' and see the same things.
With this piece of code, I get this error:
(py34)user:~/flask$ python app.py
INFO - Connection with server established!
INFO - Server version meets recommendations (v2.9)
Traceback (most recent call last):
File "app.py", line 18, in <module>
app.run(host=app.config['SERVER_NAME'], port=4343, debug=True)
File "/home/me/anaconda3/envs/py34/lib/python3.4/site-packages/flask/app.py", line 772, in run
run_simple(host, port, self, **options)
File "/home/me/anaconda3/envs/py34/lib/python3.4/site-packages/werkzeug/serving.py", line 674, in run_simple
s.bind((hostname, port))
socket.gaierror: [Errno -2] Name or service not known
If instead of using SERVER_NAME I use host='0.0.0.0', I don't get any errors and it will successfully start "Running on http://0.0.0.0:4343/" but I can't follow that url into my browser the same way I can when I connect to my localhost.
What do I need to do to get my flask app running on https://example.org/blah?
Also, if it helps to know, I have to ssh into my server.
If you run with host='0.0.0.0' from the server, you should be able to navigate to example.org:4343 and see your app.
Additionally, SERVER_NAME requires the port, per the documentation:
the name and port number of the server. Required for subdomain support (e.g.: 'myapp.dev:5000')
Finally, if you want to show you app without a port number, you either need to run it at port 80 (or 443) with root permissions, or using Nginx/Apache to route the request from your domain to the app.
I have a Tornado HTTPServer initialized like so:
ssl_options = {
"certfile": "mycert.crt",
"keyfile": "mykey.key"
}
server = tornado.httpserver.HTTPServer(application, xheaders=True,
ssl_options=ssl_options)
It has a WebSocketHandler with an open() method.
When I attempt to open a secure websocket connection via Javascript from the Chrome 30 console, like
var sock = new WebSocket("wss://localhost:9001/mywebsocket");
the connection does not open successfully, there is no log output, and the WebSocketHandler's open() is not called.
How can I open and maintain a secure (SSL) WebSocket connection to a Javascript client in Tornado?
Attempting an insecure connection, like
var sock = new WebSocket("ws://localhost:9001/mywebsocket");
results in the following error output in the Tornado log:
2013-10-08 13:59:55,305 tornado.general 820 : SSL Error on 8 ('192.168.149.27', 62851): [Errno 1] _ssl.c:490: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request
Also in this case, the connection is not opened successfully and open() isn't called.
Additional info: The number after the IP in the error message (62851 in the above example) increases with every request, secure or otherwise. I don't know what that number is but it does indicate that the request is at least getting to the server.
Also, removing ssl_options from the constructor and making insecure (ws://) requests to the server fixes the issue.
The certificate being used is self-signed. To communicate with an HTTP endpoint of the server via curl, I had to use the --insecure flag.
If you're starting the connection from the javascript console, the browser doesn't have a chance to show you the self-signed certificate warning and give you a chance to accept it. If you go to https://localhost:9001 first and accept the certificate there, does it work?
I am writing an rather simple http web server in python3. The web server needs to be simple - only basic reading from config files, etc. I am using only standard libraries and for now it works rather ok.
There is only one requirement for this project, which I can't implement on my own - virtual hosts. I need to have at least two virtual hosts, defined in config files. The problem is, that I can't find a way how can I implement them in python. Does anyone have any guides, articles, maybe some simple implementation how can this be done?
I would be grateful for any help.
Virtual hosts work by obeying the Host: header in the HTTP request.
Just read the headers of the request, and take action based on the value of the Host: header
For a simple HTTP web server, you can start with the WSGI reference implementation:
wsgiref is a reference implementation of the WSGI specification that can be used to add WSGI support to a web server or framework. It provides utilities for manipulating WSGI environment variables and response headers, base classes for implementing WSGI servers, a demo HTTP server that serves WSGI applications,...
Modifying the example server to check the HTTP_HOST header, here is a simple app that responds, depending on the virtual host, with a different text. (Extending the example to use a configuration file is left as an exercise).
import wsgiref
from wsgiref.simple_server import make_server
def my_app(environ,start_response):
from io import StringIO
stdout = StringIO()
host = environ["HTTP_HOST"].split(":")[0]
if host == "127.0.0.1":
print("This is virtual host 1", file=stdout)
elif host == "localhost":
print("This is virtual host 2", file=stdout)
else:
print("Unknown virtual host", file=stdout)
print("Hello world!", file=stdout)
print(file=stdout)
start_response(b"200 OK", [(b'Content-Type',b'text/plain; charset=utf-8')])
return [stdout.getvalue().encode("utf-8")]
def test1():
httpd = make_server('', 8000, my_app)
print("Serving HTTP on port 8000...")
# Respond to requests until process is killed
httpd.serve_forever()