I am a having a very strange issue. The premise is that I am fairly ignorant in both mqtt and python (the latter I don't use it now since at least 5-6 years), but I am making a Unity app for a museum using a 3D tracking system (www.pozyx.io) and I need each of my machines to run a small mqtt-to-OSC client, so that my Unity app can read the position data from the client.
On my development machine, it all works like a charm, using a slightly modified version of the script provided by the sensor producer.
`
API_KEY = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
import paho.mqtt.client as mqtt
import ssl
import json
from pythonosc.udp_client import SimpleUDPClient
host = "mqtt.cloud.pozyxlabs.com"
port = 443
topic = "5c500595601a3f5871a17685"
username = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
password = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
ip = "127.0.0.1" # IP for the OSC UDP
network_port = 8888 # network port for the OSC UDP
osc_udp_client = SimpleUDPClient(ip, network_port)
def on_connect(client, userdata, flags, rc):
print(mqtt.connack_string(rc))
def on_message(client, userdata, msg):
tag_data = json.loads(msg.payload.decode())
for tag in tag_data:
try:
network_id = tag["tagId"]
#print(network_id)
timestamp = tag["timestamp"]
position = tag["data"]["coordinates"]
yaw = tag["data"]["orientation"]["yaw"]
osc_udp_client.send_message("/position", [network_id, timestamp, position["x"], position["y"], position["z"], yaw])
except:
print("Received a bad packet?")
pass
def on_subscribe(client, userdata, mid, granted_qos):
print("Subscribed to topic!")
client = mqtt.Client(transport="websockets")
client.username_pw_set(username, password=password)
client.tls_set_context(context=ssl.create_default_context())
client.on_connect = on_connect
client.on_message = on_message
client.on_subscribe = on_subscribe
client.connect(host, port=port)
client.subscribe(topic)
client.loop_forever()
`
Now that I am in the musem to deploy, of course on the freshly setup windows 10 machines ( I tried both on a NUC and and on a Lenovo Thinkpad), nothing works, and I get each time the following error
C:\Users\Vattenkikare1\Desktop\osc_hans>py osc_hans.py
Traceback (most recent call last):
File "osc_hans.py", line 67, in
client.connect(host, port=port)
File "C:\Users\Vattenkikare1\AppData\Local\Programs\Python\Python37-32\lib\site-packages\paho\mqtt\client.py", line 839, in connect
return self.reconnect()
File "C:\Users\Vattenkikare1\AppData\Local\Programs\Python\Python37-32\lib\site-packages\paho\mqtt\client.py", line 994, in reconnect
sock.do_handshake()
File "C:\Users\Vattenkikare1\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 1117, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)
My laptop is plugged onto the same WiFi and I do have admin rights on all the machines. Would you have any idea on what might be causing the problem? And why might that happen only on the other computers and not on mine?
I did first deploy an exe to those machines, but then on one of them i did a quick python setup with all the modules, but nothing changed.
I did find a few similar issues around, but none that I could relate directly to mine in terms of solution.
I seem to have found the solution in the most random way. I noticed on a post that the Pozyx.io website used the "COMODO RSA Domain Validation Secure Server CA" certificate, and even though the API documentation didn't mention it, I installed it on the test computers and all started working.
Related
I'm trying to create a chat between client and server written in Python, using SSL protocols with mutual authentication (i.e: server authenticates client and client authenticates server using certificates). My host machine is being used as the server, and my laptop is the client.
When attempting to connect to my host ip, I keep getting this error on my laptop:
Traceback (most recent call last):
File "/home/icarus/Codes/RealtimeChat/Chat.py", line 88, in <module>
main()
File "/home/icarus/Codes/RealtimeChat/Chat.py", line 75, in main
connection(ip, port, SSLSock)
File "/home/icarus/Codes/RealtimeChat/Chat.py", line 35, in connection
sock.connect((ip, port))
File "/usr/lib/python3.10/ssl.py", line 1375, in connect
self._real_connect(addr, False)
File "/usr/lib/python3.10/ssl.py", line 1362, in _real_connect
super().connect(addr)
ConnectionRefusedError: [Errno 111] Connection refused
And in the server - which was supposed to print a message saying that a connection was refused - nothing happens, it keeps listening for connections as if nothing happened
Connection function on client side:
def connection(ip, port, sock):
try:
sock.connect((ip, port))
print(f"Connected with {ip}")
except Exception as e:
print("Connection failed: ", e)
sock.close()
Server side:
def acceptConnection(self):
while True:
con, senderIP = self.sock.accept()
# Attempting to wrap connection with SSL socket
try:
SSLSock = self.getSSLSocket(con)
# If exception occurs, close socket and continue listening
except Exception as e:
print("Connection refused: ", e)
con.close()
continue
print(f"{senderIP} connected to the server")
# Adding connection to clients list
self.clients.append(SSLSock)
# Initializing thread to receive and communicate messages
# to all clients
threading.Thread(target=self.clientCommunication, args=(SSLSock, ), daemon=True).start()
This is the main function on my server:
def main():
serverIP = "127.0.0.1"
port = int(input("Port to listen for connections: "))
server = Server()
server.bindSocket(serverIP, port)
server.socketListen(2)
server.acceptConnection()
Everything works fine when I connect from my localhost (e.g I open a server on my host machine on one terminal and use another one on the same machine to connect to it). Both machines have the required certificates to authenticate each other, so I don't think that's the problem. Also, without the SSL implementation, the connection between this two different computers was refused by the server
I've tried using sock.bind('', port) on server side, disabling my firewall, used telnet 127.0.0.1 54321 (on my host machine) to check if the connection was working on the specified port (and it is), and also on the client machine (which showed that the connection was refused). I also tried running both scripts with admin privileges (sudo), but it also didn't work. Any suggestions?
I found what was wrong: I was trying to connect to my public IP address (which I found by searching for "What is my ip" on Google), but instead what should be done is to connect to the private IP address (I guess that's the correct name), and you can see yours using ifconfig on Linux and Mac and ipconfig on Windows using a terminal. By doing this, I could connect two computers that are on my network to my desktop server, I still haven't tested for computers in different networks, but the problem has been solved.
We have a server and client both running python3.
The client connects to the server and authenticates upon initialisation of the client. This completes without issue.
However, if the connection drops, the client catches the error (the socket.recv returning 0) and attempts to re-run the code that connects to the server).
The server recieves the initial request whilst listening on its given recieving socket and then once making a connection the next recv call raises the following error:
File "/usr/lib64/python3.7/ssl.py", line 1056, in recv
return self.read(buflen)
File "/usr/lib64/python3.7/ssl.py", line 931, in read
return self._sslobj.read(len)
ssl.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:2570)
Why would this succeed upon the first connection but then raise this error thereafter? If the client is closed and restarted the error is avoided upon the next connection. However, if the server is closed and the client tries to connect once the server restarts this error is encountered.
At the client end the following error is raised:
File "\Our_Code", line 100, in make_connection
data = self.ssl_sock.recv(1024)
File "C:\Users\Home\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 1226, in recv226, in recv
return self.read(buflen) 101, in read
File "C:\Users\Home\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 1101, in read
return self._sslobj.read(len)
ssl.SSLError: [SSL] internal error (_ssl.c:2633)
This exact formultation suggests to me that the issue is actually with the client and some data that it is saving between reconnects. But it overwrites the sockets for a new connection so I thought all data from the previous connection would be discarded?
The connection function is as:
import socket
import ssl
def make_connection(self, email, password):
try:
self.ssl_sock = []
HOST = "9:9:99:999" # The server's hostname or IP address (not our actual IP)
PORT = 5432 # The port used by the server (not our actual port)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
self.ssl_sock = self.context.wrap_socket(sock)
password = password.encode('utf-8').strip()
password = base64.b64encode(password).decode("utf-8")
tcp_string = (f"^{email}*{password}$")
tcp_string = tcp_string.encode('utf-8')
self.ssl_sock.sendall(tcp_string)
data = self.ssl_sock.recv(1024)
data = data.decode('utf-8')
if data == "^good_connect$":
return True
else:
return False
except Exception:
print(f"make_connection - {traceback.format_exc()}")
I'm still fairly new to python and particularly networking, so I suspect I have made a rookie error. But so far all my searches have returned the obvious, that the TLS version is wrong, but the fact that it authenticates fine on the initial connection suggests to me that that isn't the case.
I'm happy to answer any questions I can about the situation.
I have been struggeling with a strange case of random client MQTT publish failing for certain payloads. It happends randomly when trying to publish some large amount of BASE64 data.
I've finally managed to narrow it down to payloads containing a lot of consequtive forwards slashes (/). I've searched the net to find a good reason why this happends, but havent found anythong. Is it a MQTT feature, a Paho client feature or a broker feature, or just some bug...
Setup:
Python 3.8.8 (Windows 10)
paho-mqtt 1.5.0
mosquitto 1.6.9-1 amd64
On my setup, it fails when I send a payload of 255 '/' to a 1 character topic 'a'. Larger topic length, reduces the possible number of forward slashes.
Code to reproduce error:
import paho.mqtt.client as mqtt_client
import time
address = 'some.server.com'
port = 1883
connected = False
def on_connect(client, userdata, flags, rc):
global connected
connected = True
print("Connected!")
client = mqtt_client.Client()
client.on_connect = on_connect
client.connect(host=address, port=port, keepalive=60)
client.loop_start()
while not connected:
time.sleep(1)
payload = '/'*205
print('Payload: {}'.format(payload))
client.publish(topic='a', payload=payload)
time.sleep(2)
client.loop_stop()
client.disconnect()
print('Done!')
This generates this output:
Connected!
Payload: /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Connected!
Done!
This produces the following error in /var/log/mosquitto/mosquitto.log for the mosquitto broker:
1616605010: New connection from a.b.c.d on port 1883.
1616605010: New client connected from a.b.c.d as auto-CEF15129-E74C-F00A-A6FA-5B5FDA0CEF1D (p2, c1, k60).
1616605011: Socket error on client auto-CEF15129-E74C-F00A-A6FA-5B5FDA0CEF1D, disconnecting.
1616605012: New connection from a.b.c.d on port 1883.
1616605012: New client connected from a.b.c.d as auto-0149B6DB-5997-9E08-366A-304F21FDF2E1 (p2, c1, k60).
1616605013: Client auto-0149B6DB-5997-9E08-366A-304F21FDF2E1 disconnected.
I observe that the client() connects twice, but do not know why, but this is probably caused by a disconnect...
Any Ideas?
Update 1: I've tested this on Linux Ubunit running Python 3.7.3, and same paho-mqtt version, and this does not produce the same error... Seems like some problem in Windows then.
Update 2:
I also tried running mosquitto_pub and experienced the same error, so this has to be Windows-related (or system related) in some way. Possibly firewall? I will close question if I find manage to solve this.
"C:\Program Files\mosquitto\mosquitto_pub.exe" -h some.server.com -t a -m '/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////'
Update 3 The issue is related to OpenVPN. I closed my open VPN connection and MQTT messages were passing! I'm running OpenVPN Client (Windows version 3.2.2 (1455). I have no idea what causes this conlflict...
I am writing a python socket server with ssl and I am encountering certificate unknown error during ssl handshake.
I have created private key and certificate with openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes command on my own.
This server is intended to run in intranet(under wifi) in my PC, and users will contact my PC IPaddress with their browser. Hence I have not registered this certificate with any CA. and I don't think its mandatory in my case.
Below more details..
echoserver.py
import socket
import ssl
import threading
class echoserver:
def __init__(self,i,p):
self.ip=i
self.port=p
def handler(self,c,a):
msg=c.recv(1024)
c.send(msg)
def serve(self):
echoserver = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
echoserver = ssl.wrap_socket(echoserver, keyfile='keys/key.pem', certfile='keys/cert.pem', server_side=True)
echoserver.bind((self.ip, self.port))
echoserver.listen()
while True:
(c,a)=echoserver.accept()
threading.Thread(target=self.handler, args=(c,a)).start()
es=echoserver('192.168.43.124',443) #My PC's ip assigned under wifi network
es.serve()
#Connecting from mobile phone within same network as https://192.163.43.124
Error in server during ssl handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1108)
What I tried
Adding cert_reqs=ssl.CERT_NONE and ca_certs="/location/to/keys" parameters in wrap_socket function.
Doesn't seems to work. I assume these options are for client side.
Adding do_handshake_on_connect=False in wrap_socket function
In Chrome, When connected server throws same error and thread/connection closed with exception. and chrome seems to send same connection request immediately again, and the second request works flawlessly !!.
In firefox, First connection closed with same error and there is no second request.
Assigning common name in certificate same as IP address
Not working.
Checked certificate_unknown error in ietf specification here. It gives no clue except this explanation certificate_unknown: Some other (unspecified) issue arose in processing the certificate, rendering it unacceptable.
One other thing I noted is, if I use built-in ThreadedHTTPServer in the same way as below, It works beautifully, without any issues I mentioned above.
httpd = self.ThreadedHTTPServer((self.ip, self.port), self.handler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile='keys/key.pem', certfile='keys/cert.pem', server_side=True)
I am not sure why this happens and how should I proceed with this. and not sure how built-in server modules works fine.
Appreciate any leads. Thanks.
Below Python builtin HTTPServer works fine with ssl, not showing any error. Not sure how?
import ssl
from http.server import HTTPServer, BaseHTTPRequestHandler
class requesthandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-type","text/html")
self.end_headers()
self.wfile.write("<html><body>It works</body></html>".encode('utf8'))
httpd = HTTPServer(('192.168.43.124', 443), requesthandler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile='keys/key.pem', certfile='keys/cert.pem', server_side=True)
httpd.serve_forever()
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1108)
This means the client (browser) does not trust your certificate since it is issued by an unknown entity. If you want to use self-signed certificates you have to explicitly import these as trusted for all clients you want to use.
There is no way around this. The certificate in TLS is to make sure that the connection is done with the expected server and not some man in the middle claiming to be the expected server. If a client would trust arbitrary certificates then it would also trust certificates created by a man in the middle attacker.
Below Python builtin HTTPServer works fine with ssl, not showing any error. Not sure how?
The browser will still complain.
The only difference is that the server captures the exception and thus will not crash but continue. You can do the same in your code:
while True:
try:
(c,a)=echoserver.accept()
threading.Thread(target=self.handler, args=(c,a)).start()
except:
pass # ignore error
I'm implementing an IMAP proxy which securely communicates with a client.However, I have a problem when handshaking.
The code of my proxy is:
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.bind((host, port))
ssock, addr = self.sock.accept()
context = ssl.SSLContext(ssl.PROTOCOL_TLS)
self.conn_client = context.wrap_socket(ssock)
And I receive the error:
ssl.SSLError: [SSL: UNEXPECTED_MESSAGE] unexpected message (_ssl.c:833)
The code of my tests is:
M = imaplib.IMAP4_SSL(IP_PROXY)
And I receive the error:
ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:777)
However, when the code of the proxy is:
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.bind((host, port))
ssock, addr = self.sock.accept()
self.conn_client = ssl.wrap_socket(ssock, certfile=CERT, server_side= True)
It correctly works but I don't want to use certificate.
Thank you
It correctly works but I don't want to use certificate.
SSL/TLS is almost everywhere used with a certificate to make sure that the client is talking to the expected server and not to some man in the middle. If you don't want to use a certificate you need to either use a different kind of authentication (like PSK) or use no authentication at all ("anonymous authentication" - very bad idea).
In any way you would need to set the relevant ciphers to enable this alternative authentication on both client and server. This can be done with the ciphers attribute to wrap_socket on the server side and in your client code it could probably be done by constructed a SSLContext with the necessary ciphers and using the ssl_context argument to specific the context to be used in IMAP4_SSL.
But this is only for your specific Python based IMAP client. Don't expect that you will be able to configure commonly used IMAP clients like Thunderbird or Outlook to be usable with a server without certificates. And like I said, it is a bad idea in the first place.