I'm using Python's requests library to perform client side authentication with certificates. The scenario is the following: CA1 has issued a certificate for an intermediate CA (CA2) and CA2 has issued my client's certificate CLIENT. The server I'm connecting to trusts CA1's cert (but does not have CA2's cert). When I use:
requests.get('https:..', cert=('/path/CLIENT.cert', '/path/CLIENT.key'))
I get an error "certificate verify failed". I assumed that's because the server can not retrieve CA2's cert.
However, I'm unable to find a way to send CA2's cert to the server. If I include it in CLIENT, I get an error about private key and cert mismatch. I have also tried to include the chain of certificates in the verify parameter but there does not seem to be any difference on the result (as far as I understand, certs in the verify parameter are used for server side authentication).
Although I think this must be a quite common scenario, I'm unable to find a solution...
PD: If I verify CLIENT's cert with openssl and the full chain of certificates the validation is successful (so there is no problem with the certificates themselves).
Requests recommends using certifi as a CA bundle. Have you tried installing certifi, adding CA1, and passing the certifi bundle path to requests?
Related
I'm writting a Flask app that connects to external soap service that uses TLS v1.2.
I'm using Python 2.7 and requests library in version 2.18.1.
I've contacted server owner and he told me that I need to include multiple client certificates in TLS connection. It's a chain of 3 certificates which I have in separate .pem files. (root + indermediate + my client certificate).
Server won't let me in if I would have just the last one.
I've tested this with SoapUI and Wireshark and it's true. I receive a response only when I provide the whole chain of 3 certificates.
I get an error from the server when passing just my client certificate.
From requests documentation you can read that as client certificate you can pass just one cert using:
session = requests.session()
session.cert = ('/path/client_cert.pem', '/path/private_key.pem')
response = session.post(SERVICE_URL, data=XML_CONTENT, headers=HEADERS)
I get an error even if my "client_cert.pem" file is a bundle of 3 certificates (just like you do it in session.verify with CA certs). I can see on Wireshark that only the first one is used in TLS connection.
Is there any way to include multiple certificates TLS connection in Python's requests library?
Maybe I should use different library or override some of it's code?
I've got it!
I had some legacy library versions installed.
It seems that this issue was fixed by requests library developers in version 1.23. I also had to update urllib3.
My current requirements.txt is:
requests==2.22.0
urllib3==1.25.2 # compatible with requests 2.22
For following spec everything works perfecly. I've checked TLS connection on Wireshark. All certificates from "client_cert.pem" chain are passed.
If you'll have problems like this in the future remember to check if your requests and urllib3 library versions are compatible.
Thank you guys!
It is given in the documentation of urllib3 that,
In order to enable verification you will need a set of root certificates.
but is there any way to make HTTPS calls by using the SSL certificate of the server itself instead of root certificates.
urllib3 has a property of the PoolManager called assert_fingerprint: this property can be initiated with a string that is the hex-encoded digest of the certificate bytes. When used, this will circumvent the regular chain building logic.
i've written a small OPC-UA-Client in Python which acts as datalogger for PLC's with integrated OPC-UA Server.
The Connection with no security works fine but i want to secure it with a certificate.
I can import trusted certificates to the server and export the server certificate but how can i generate my own certificate ?
Thanks
You can use openssl to generate your own self-Signed certificate.
https://www.openssl.org/source/
Be Carefull with the extensions tho sometimes they demand .der .cer or as .pem and you might get an outform .crt.
How to create them:
https://www.ibm.com/support/knowledgecenter/en/SS8JFY_9.2.0/com.ibm.lmt.doc/Inventory/security/t_ssl_creating_certs.html
https://dzone.com/articles/secure-communication-with-tls-and-the-mosquitto-broker
Types of encryption:
https://blog.storagecraft.com/5-common-encryption-algorithms/
If you show your program maybe I can help you out.
I'm using the following code to interact with a Magento webstore using the XMLRPC api. Magento API Python XMLRPC
Everything was working ok until we made a change on our web server to SSL
Now I'm getting the following error.
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
I'm not sure why the certificate is failing as we have an EV certificate and all browsers are showing this as ok.
My connection string is:
How can I resolve this / over-ride the code
I'm fairly new to Python so please go easy :o)
magento = MagentoAPI("www.website.co.uk", 443, "myUsername", "myPassword", "/api/xmlrpc", True)
Python, or better the OpenSSL library it is using, can not verify the validity of the certificate of the server. There are many possible reasons: bad configuration, missing intermediate or CA certificate, wrong CN...
A first step could be to go to this site and let it test the SSL/TLS capabilities of the server: https://www.ssllabs.com/ssltest/
It will give you hints on how to solve problems as well.
Python verifies certs via its own bundle, check where it is located by
>>> import certifi
>>> certifi.where()
'/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/certifi/cacert.pem'
and add your certificates to the end of that file.
Question
How can I verify that an X.509 certificate is signed by another certificate using PyOpenSSL or Twisted? I want a client to verify that the received server certificate is the one that signed its client certificate.
I've looked through the PyOpenSSL documentation and can't seem to find anything on how to verify a certificate separately from the establishing the SSL connection.
I found a reference to OpenSSL.crypto:X509.verify() in twisted.internet._sslverify:PublicKey.verifyCertificate() , but the twisted method is commented out (in Twisted 13.0) and the X509 method does not exist (in PyOpenSSL 0.13).
pyOpenSSL has no support for verifying a certificate describes a bug for not being able to manually verify a certificate chain, but I'm not entirely sure if that's what I'm trying to do.
Use Case
Certificates:
Generated self-signed CA certificate with openssl.
Generated server certificate signed by CA certificate.
Generated client certificate signed by server certificate.
Setup:
The server is using Twisted's CertificateOptions with its server cert. The CA certs are the CA and server certs to setup a chain where the server cert verifies the received client cert, and the CA cert verifies the server cert (all built-in functionality).
The client is also using CertificateOptions for the client cert. The CA certs only contains the CA cert.
This all works fine (both sides verify each other) but I want to perform an additional step:
In the client set_verify() callback, verify that the client cert is signed by the server cert.
You should be able to do it with something like written here:
http://www.yothenberg.com/validate-x509-certificate-in-python/
which is basically:
load your certificates in PyOpenSSL with load_certificate()
create a X509Store() object
use add_cert() to add your intermediate certificate in the store
create a X509StoreContext() object, initializing it with both your store object and your end certificate
call verify_certificate() on your store context object
In practice, I was unable to make that part, and I think it is for the reasons explained here: https://mail.python.org/pipermail/cryptography-dev/2016-August/000676.html
In short, even in 2016, there still does not seem to be a correct wait to check certificates in PyOpenSSL, which is very sad. Note that the consensus seem to be that if you operate inside a TLS connection, the things are better checked by the connection routine instead of offline through check_certificate()