I'm currently reading Foundations of Python Network Programming and came across the following example, demonstrating how to use Python's ssl module:
Listing 6-3: Securing a Socket with TLS for Both Client and Server in Python 3.4 or Newer
#!/usr/bin/env python3
# Foundations of Python Network Programming, Third Edition
# https://github.com/brandon-rhodes/fopnp/blob/m/py3/chapter06/safe_tls.py
# Simple TLS client and server using safe configuration defaults
import argparse, socket, ssl
def client(host, port, cafile=None):
purpose = ssl.Purpose.SERVER_AUTH
context = ssl.create_default_context(purpose, cafile=cafile)
raw_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
raw_sock.connect((host, port))
print('Connected to host {!r} and port {}'.format(host, port))
ssl_sock = context.wrap_socket(raw_sock, server_hostname=host)
while True:
data = ssl_sock.recv(1024)
if not data:
break
print(repr(data))
def server(host, port, certfile, cafile=None):
purpose = ssl.Purpose.CLIENT_AUTH
context = ssl.create_default_context(purpose, cafile=cafile)
context.load_cert_chain(certfile)
listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listener.bind((host, port))
listener.listen(1)
print('Listening at interface {!r} and port {}'.format(host, port))
raw_sock, address = listener.accept()
print('Connection from host {!r} and port {}'.format(*address))
ssl_sock = context.wrap_socket(raw_sock, server_side=True)
ssl_sock.sendall('Simple is better than complex.'.encode('ascii'))
ssl_sock.close()
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Safe TLS client and server')
parser.add_argument('host', help='hostname or IP address')
parser.add_argument('port', type=int, help='TCP port number')
parser.add_argument('-a', metavar='cafile', default=None,
help='authority: path to CA certificate PEM file')
parser.add_argument('-s', metavar='certfile', default=None,
help='run as server: path to server PEM file')
args = parser.parse_args()
if args.s:
server(args.host, args.port, args.s, args.a)
else:
client(args.host, args.port, args.a)
The book says to download the following two files, which are a self-signed CA and a second CA (signed by the self-signed CA) for the hostname, "localhost":
ca.crt
localhost.pem
The code is, then, supposed to be run as follows:
Server
python safe_tls.py -s localhost.pem '' 1060
Client
python safe_tls.py -a ca.crt localhost 1060
However, whenever I run the server and client together, I get the following error for each:
Server
Listening at interface '' and port 1060
Connection from host '127.0.0.1' and port 35148
Traceback (most recent call last):
File "safe_tls.py", line 50, in <module>
server(args.host, args.port, args.s, args.a)
File "safe_tls.py", line 35, in server
ssl_sock = context.wrap_socket(raw_sock, server_side=True)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 814, in __init__
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:833)
Client
Connected to host 'localhost' and port 1060
Traceback (most recent call last):
File "safe_tls.py", line 52, in <module>
client(args.host, args.port, args.a)
File "safe_tls.py", line 15, in client
ssl_sock = context.wrap_socket(raw_sock, server_hostname=host)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 814, in __init__
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)
I already tried generating my own self-signed CA and secondary CA (signed by the self-signed CA) for the hostname, "localhost", but I still get the same errors when running the client and server.
Could anyone please tell me what I am doing wrong?
Thank you.
P.S. I am running Python 3.6.5 on Ubuntu 18.04 LTS in case it matters...
UPDATE
I was able to get the code above to work without the secondary certificate, but rather with a single self-signed certificate which I generated as follows:
$ openssl req -x509 -nodes -newkey rsa:4096 -keyout localhost.key -new -out localhost.crt
$ cat localhost.crt localhost.key > localhost.pem
Then, I ran the safe_tls.py program exactly as described above, except with localhost.crt instead of ca.crt.
Still, I would like to be able to sign one CA with another, and then use the secondary CA as I was trying to originally. Any help figuring out how to do this would be greatly appreciated!
Alright, so I ended up finding the answer to my own question, so I'll post it here.
I don't know what is wrong with the ca.crt and localhost.pem files provided at the public repo of Foundations of Python Network Programming; however, I managed to generate and sign my own CAs, getting it to work as I had originally wanted.
Here are the commands I had to type:
$ # I don't know what these files are, but OpenSSL complains if they do not exist...
$ mkdir demoCA
$ touch demoCA/index.txt demoCA/index.txt.attr
$ echo '01' > demoCA/serial
$ # Create a self-signed certificate and private key.
$ openssl req -x509 -nodes -newkey rsa:4096 -keyout ca.key -new -out ca.crt
$ # Create a certificate signing request for server, "localhost", and a private key.
$ openssl req -nodes -newkey rsa:4096 -keyout localhost.key -new -out localhost.csr
$ # Sign the certificate request.
$ openssl ca -keyfile ca.key -cert ca.crt -in localhost.csr -outdir . -out localhost.crt
$ cat localhost.crt localhost.key > localhost.pem
NOTE:
For each of the openssl req ... commands, executed above, I had to fill in some information at the command-line (since I did not create a configuration file). All the information, except for the host name and email must be the same across the two certificates, otherwise the last command, where we sign the certificate request, will fail.
The server and client can now be successfully run as originally shown:
Server
python safe_tls.py -s localhost.pem '' 1060
Client
python safe_tls.py -a ca.crt localhost 1060
Related
Am trying to implement TLS for mqtt and has followed the tutorials from the link below
http://www.steves-internet-guide.com/mosquitto-tls/
I followed exactly how it has been instructed to generate certificates using openssl and pasted in the location of mqtt and changed the conf of mqtt and restarted the service.
But when I try to connect to mqtt using tls it shows the below error message
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1124)
And python code is
client1 = paho.Client("control1")
client1.tls_set(ca_certs="ca.crt")
client1.tls_insecure_set(True)
client1.connect("localhost", 8883)
client1.loop_forever()
where ca.crt is in the project directory.
The message you are receiving indicates that the broker's server certificate is not trusted (because it is self-signed), therefore paho is not being correctly told it is trustworthy.
It is possible your fake certificate authority's root certificate (the ca.crt file you feed to paho) is not properly signed or generated, or the certificates that Mosquitto is using are not signed correctly. Either way, you likely need to start the entire process over to be 100% certain everything was done right.
Generate the fake certificate authority's (CA) signing key
$ openssl genrsa -des3 -out ca.key 2048
Generate a certificate signing request for the fake CA
$ openssl req -new -key ca.key -out ca-cert-request.csr -sha256
Give the organization a name like "Fake Authority" and do not enter a common name (since your fake CA does not actually live on a server with a name)
Create the fake CA's root certificate
$ openssl x509 -req -in ca-cert-request.csr -signkey ca.key -out ca-root-cert.crt -days 365 -sha256
Create the server / mqtt broker's keypair
$ openssl genrsa -out server.key 2048
Create a certificate signing request using the server key to send to the fake CA for identity verification
$ openssl req -new -key server.key -out server-cert-request.csr -sha256
Give the organization a name like "Localhost MQTT Broker Inc." and the common name should be localhost or the exact domain you use to connect to the mqtt broker
Now acting as the fake CA, you receive the server's request for your signature. You have verified the server is who it says it is (an MQTT broker operating on localhost), so create a new certificate & sign it with all the power of your fake authority.
$ openssl x509 -req -in server-cert-request.csr -CA ca-root-cert.crt -CAkey ca.key -CAcreateserial -out server.crt -days 360
Now you have everything you need. Make sure (as in Steve's tutorial) Mosquitto is loading the following in mosquitto.conf:
listener 8883
cafile certs\ca-root-cert.crt
keyfile certs\server.key
certfile certs\server.crt
Make sure paho-mqtt is loading the fake CA's root certificate.
client1.tls_set(ca_certs="ca-root-cert.crt")
This is how it knows that mosquitto's server.crt is legitimately signed by a "real and trusted authority" and is not "self-signed" and thus untrusted. Mosquitto and paho should now be able to securely connect and communicate.
I am new to paho-mqtt. I was trying to publish a topic using my localhost and I encountered the following error :
Traceback (most recent call last):
File "server.py", line 10, in <module>
client1.connect(host,port,keepalive)
File "/usr/local/lib/python2.7/dist-packages/paho_mqtt-1.3.1-py2.7.egg/paho/mqtt/client.py", line 768, in connect
return self.reconnect()
File "/usr/local/lib/python2.7/dist-packages/paho_mqtt-1.3.1-py2.7.egg/paho/mqtt/client.py", line 895, in reconnect
sock = socket.create_connection((self._host, self._port), source_address=(self._bind_address, 0))
File "/usr/lib/python2.7/socket.py", line 575, in create_connection
raise err
socket.error: [Errno 111] Connection refused
My python code is below :
import paho.mqtt.client as paho
port=1883
host = "localhost"
keepalive = 60
def on_publish(client,userdata,result):
print("data published \n")
pass
client1= paho.Client("control1")
client1.on_publish = on_publish
client1.connect(host,port,keepalive)
ret= client1.publish("Robot","Robot 1 move_left")
When I run the same code with iot.eclipse.org as host then it works fine. Any help would be highly appreciated.
I was facing the same issue.
The solution was to install a local MQTT broker.
http://www.steves-internet-guide.com/install-mosquitto-linux/
The exposed docker port for mqtt is usually different than 1883.
I use the official eclipse mosquitto docker and the run example on their page is something like:
sudo docker run -it -p 11883:1883 -p 9001:9001 eclipse-mosquitto
therefore the client should connect to port 11883
client.connect(broker_address, 11883)
I want to transfer a file from a local server (that'll contain the code and file) to a remote server preferably using ssh. Here's the code:
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('Servername', port = 135, username='username', password='password')
print "connected successfully!"
sftp = ssh.open_sftp()
print sftp
sftp.put('G:\TestDocument.txt','G:\TestDocument.txt' )
sftp.close()
print "copied successfully!"
ssh.close()
But I get this error:
No handlers could be found for logger "paramiko.transport"
Traceback (most recent call last): File
"C:/Python27/testtransfer.py", line 5, in
ssh.connect('Servername', port = 135, username='username', password='password') File
"C:\Python27\lib\site-packages\paramiko\client.py", line 392, in
connect
t.start_client(timeout=timeout) File "C:\Python27\lib\site-packages\paramiko\transport.py", line 545, in
start_client
raise e SSHException: Error reading SSH protocol banner
Can you tell me why am I receiving this error? I have purposely used port 135 because port 22 is closed on the target server and 135 (among others) is open.
You can even suggest some other way in which I can transfer files from one server to another using Python.
SSHException: Error reading SSH protocol banner
the error usually means the remote service was not a ssh service. try to make sure the service on port 135 is actually ssh.
And what's the OS on your server? Windows? if so you may need to setup a 3rd-part ssh server on your server. In my opinion if you need to copy files from windows to windows ftp is a much simpler solution. python has a built-in lib ftplib.
I am trying to implement APNS for my iOS push app from this tutorial raywenderlich tutorial.
As per the description I have done everything.
After running this command
openssl s_client -connect gateway.sandbox.push.apple.com:2195 -cert PushChatCert.pem -key PushChatKey.pem
I got a big message showing that everything is connected as expected.
I have two files now PushChatCert.pem and PushChatKey.pem
and token_hex = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
So for sending push notification from server I following PyAPNs.
After doing all these in my python shell it is throwing IOError: [Errno 19] Operation not supported by device
import time
from apns import APNs, Frame, Payload
apns = APNs(use_sandbox=True, cert_file='PushChatCert.pem', key_file='PushChatKey.pem')
# Send a notification
token_hex = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
payload = Payload(alert="Hello World!", sound="default", badge=1)
apns.gateway_server.send_notification(token_hex, payload)
Error
Traceback (most recent call last):
File "<pyshell#10>", line 1, in <module>
apns.gateway_server.send_notification(token_hex, payload)
File "/Library/Python/2.7/site-packages/apns.py", line 544, in send_notification
self.write(self._get_notification(token_hex, payload))
File "/Library/Python/2.7/site-packages/apns.py", line 273, in write
return self._connection().write(string)
File "/Library/Python/2.7/site-packages/apns.py", line 254, in _connection
self._connect()
File "/Library/Python/2.7/site-packages/apns.py", line 230, in _connect
self._ssl = wrap_socket(self._socket, self.key_file, self.cert_file)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 891, in wrap_socket
ciphers=ciphers)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 509, in __init__
self._context.load_cert_chain(certfile, keyfile)
IOError: [Errno 19] Operation not supported by device
Edit :
After running this comment in my terminal
openssl s_client -connect gateway.sandbox.push.apple.com:2195 -cert PushChatCert.pem -key PushChatKey.pem I was getting an entrust issue as follows
depth=1 /C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1C
verify error:num=20:unable to get local issuer certificate
So I downloaded the entrust_2048_ca.cer file and kept in the same folder then from terminal I am running as
openssl s_client -connect gateway.sandbox.push.apple.com:2195 -CAfile entrust_2048_ca.cer -cert PushChatCert.pem -key PushChatKey.pem
Then that problem has been solved but how to execute that in PyAPNs?
Problem has been solved. It was some SSL security issue for that python was not able to access the files from that file dir.
I was following this tutorial apple-push-notification-services.
As described in this tutorial after creating the aps_development.cer file I was doing these to get the .pem files
$ openssl x509 -in aps_development.cer -inform der -out PushChatCert.pem
Convert the private key’s .p12 file into a .pem file:
$ openssl pkcs12 -nocerts -out PushChatKey.pem -in PushChatKey.p12
Enter Import Password:
MAC verified OK
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
After doing all I had PushChatKey.pem and PushChatCert.pem files and using those files I was not able to send push notifications to my device PyAPNs
apns = APNs(use_sandbox=True, cert_file='PushChatKey.pem', key_file='PushChatCert.pem')
How did I solve?
Finally I tried those certificate creation from the beginning but this time following some other tutorial
Create your APNS certificates.
After creating the SSL Certificate that you download as named aps_developer_identity.cer. Double-click on it to install it in the Keychain Access application. The SSL certificate will be used by your provider application so that it can contact the APNs to send push notifications to your applications.
Launch Keychain Assistant from your local Mac and from the 'login' keychain, filter by the 'Certificates' category. You will see an expandable option called “Apple Development iOS Push Services”:
Expand this option then right click on “Apple Development iOS Push Services” -> Export “Apple Development iOS Push Services ...″. Save this as apns-dev-cert.p12 file somewhere you can access it.
Now from apns-dev-cert.p12 you make .pem using these command from your terminal
openssl pkcs12 -in apns-dev-cert.p12 -out apns.crt.pem -clcerts -nokeys
openssl pkcs12 -in apns-dev-cert.p12 -out apns.key.pem -nocerts -nodes
If you want to create single .pem
openssl pkcs12 -in apns-dev-cert.p12 -out apns_development.pem -nodes -clcerts
Now use these apns.crt.pem and apns.key.pem files with PyAPNs and it works like magic.
Create your provision profile carefully.
Thanks.
I have simple client/server SSL code which worked fine on Python 3.2. However, I decided to switch over to 2.7 (due to abundance of third party modules), and now the code is failing. The code is as follows:
Client:
def connect(self):
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.connect(('localhost', 4430))
self.ssl_sock = ssl.wrap_socket(self.sock, cert_reqs = ssl.CERT_NONE, ssl_version = ssl.PROTOCOL_TLSv1)
Server:
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.sock.bind((self.hostname, 4430))
self.sock.listen(5)
while True:
self.newsocket, self.fromaddr = self.sock.accept()
self.ssl_sock = ssl.wrap_socket(
self.newsocket,
server_side = True,
certfile = "cert.pem",
ssl_version=ssl.PROTOCOL_TLSv1
)
self._handle_client(self.ssl_sock)
The cert.pem (located in same directory as server .py file):
-----BEGIN RSA PRIVATE KEY-----
(812 "random" characters here)
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
(1260 "random" characters here)
-----END CERTIFICATE-----
The failure is as follows:
1. server waits on self.sock.accept()
2. client connects with self.sock.connect()
3. server proceeds to wrap_socket, which fails with the following error:
Traceback (most recent call last):
File "C:\Program Files\Python27\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\workspace\projectc\server\server\clientlistener.py", line 49, in run
ssl_version=ssl.PROTOCOL_TLSv1
File "C:\Program Files\Python27\lib\ssl.py", line 381, in wrap_socket
ciphers=ciphers)
File "C:\Program Files\Python27\lib\ssl.py", line 141, in __init__
ciphers)
SSLError: [Errno 336265218] _ssl.c:351: error:140B0002:SSL routines:SSL_CTX_use_PrivateKey_file:system lib
P.S.
1) I have suspicions about the wrap_socket call because even when I use a non-existent file for certfile, the same error results.
2) I tried the alternative wrapping on the python documentation on ssl (i.e. with ssl.SSLContext) and it's odd that 'module' object has no attribute 'SSLContext', when it is part of the ssl module. This wasn't an issue when I tried it with Python 3.2.
Update:
I have found that the problem only occurs when I'm doing "run" from within Eclipse, not so when I run both files from separate command prompts. Still investigating the issue...
Update 2:
I tried a very simple client/server script with the SAME code and it works. Now, with the server code sitting inside a multiprocessing subprocess (launched by the main server process), it seems not to work. Related?
Problem solved. I believe Eclipse was holding some residual information, though I never figured out what information it was holding. It might have to do with the migration from python 3.2 to 2.7, though I previously already changed that in the run config. Resetting the Eclipse environment worked (simply rebooting the computer wasn't enough).
my solution was that when creating my .pem file i set a blank password and assumed it meant no pasword. so the server was still expecting to use a password. i had to manually remove the password.
here is a little how to guide if it helps anyone (also, please note i was coding python for a server side to an iOS app)
NOTE: need to follow directions from apple’s developer website to create certificate first
then export the .p12 file,
by exporting the embedded private key that is created (in ‘keychain access’),
NOT the actual certificate
————————————————————————————————————
————————————————————————————————————
FOR DEVELOPMENT CERT:
After getting the p12 file, it needs to be converted to the PEM format by executing this command from the terminal:
$ openssl pkcs12 -clcerts -nokeys -out apns-dev-cert.pem -in apns_dev.p12
$ openssl pkcs12 -nocerts -out apns-dev-key.pem -in apns_dev.p12
If you wish to remove the passphrase execute the following:
(NOTE: using a ‘blank’ password when exporting/converting, is still indeed setting a password,
hence you should still execute the following if you intend to have no password)
$ openssl rsa -in apns-dev-key.pem -out apns-dev-key-noenc.pem
Finally, you need to combine the key and cert files into a apns-dev.pem file we will use when connecting to APNS:
$ cat apns-dev-cert.pem apns-dev-key-noenc.pem > apns-dev.pem
————————————————————————————————————
FOR PRODUCTION CERT:
After getting the p12 file, it needs to be converted to the PEM format by executing this command from the terminal:
$ openssl pkcs12 -clcerts -nokeys -out apns-prod-cert.pem -in apns_prod.p12
$ openssl pkcs12 -nocerts -out apns-prod-key.pem -in apns_prod.p12
If you wish to remove the passphrase execute the following:
(NOTE: using a ‘blank’ password when exporting/converting, is still indeed setting a password,
hence you should still execute the following if you intend to have no password)
$ openssl rsa -in apns-prod-key.pem -out apns-prod-key-noenc.pem
Finally, you need to combine the key and cert files into a apns-dev.pem file we will use when connecting to APNS:
$ cat apns-prod-cert.pem apns-prod-key-noenc.pem > apns-prod.pem