Am in the process of building a quick python script to periodically check my clients websites are working correctly. One of these checks is to ensure their SSL certificates are current, or to provide an alert if their certificate is about to expire.
The ssl packages provides a way to obtain the peer certificate with the SSLSocket.getpeercert() method but this will only return a certificate if the certificate can be validated. If the CA cert has not been obtained the validation does not work.
What I want to do is obtain the peer certificate even if it can not be validated so I am able to get the information required to both obtain the correct CA certificate and do other checks such as checking the domain name matches, expiry date is in the correct range etc. Does anybody know of a way to obtain this information?
pyCurl and pyOpenSSL look like possible candidates but have not been able to find an example or manage to get them to return the certificate.
Cheers
It may be possible to use a shell script to grab the certificates and then use Python to iterate over certificate output files. Something like:
$ openssl s_client -connect host:port -showcerts > certfile
might work. You might also read the documentation on pyOpenSSL's Connection object, which has a get_peer_certificate() method:
http://packages.python.org/pyOpenSSL/openssl-connection.html#l2h-187
I haven't ever used the pyOpenSSL module, but it's probably your best bet for keeping everything in Python.
Related
i'm having this problem for days now, and can't figure out what exactly is wrong. I'm trying to connect to a server that requires authentication with digital certificates, and have done this before with the requests library in the following manner:
cert = (f'/path/to/cert.crt', f'/path/to/cert.open.key')
response = requests.get(url_server,cert=cert,headers=headers,proxies=proxies)
At first, i received the following error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'. I imagined it had something to do with the server certificate, because i have used the same client certificate to log in other sites before. For testing purposes, i made the call with verify=False in order to ignore any problems with the server certificate, and got this:
SSLError(1, '[SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:852)'),))
I tried to point the verify to a folder with the cert chain files (root and intermediate), and also to the certBundle file when the previous didn´t work, as documented in https://docs.python-requests.org/en/master/user/advanced/#ssl-cert-verification, but to no avail. I'm lacking knowledge about the minutia of the handshake process and therefore can´t think in any new way to debug this issue. Any help would be deeply appreciated.
I figured it out. The client certificate was lacking the certificate chain (for the client). Using Wireshark i intercepted the packages and compared what the browser was sending with the packages requests was sending. This allowed me to see that the browser automatically inserts the client certificate chain to the certificate, while requests doesn't.
I am wanting to use Python to retrieve the remote server certificate (not validate or check it in any way). I have retrieved the server certificate using both methods ``ssl.get_server_certificateandSSLSocket.getpeercert`.
The main reason I had to try SSLSocket.getpeercert over ssl.get_server_certificate was that the timeout value on the TLS handshake was not being honored with the ssl.get_server_certificate. One of the hosts I was trying to the the server certificate had some problem and would hang my python script during the TLS handshake and only the SSLSocket.getpeercert would time this out.
I also notice I cannot retrieve the server certificates from very old systems that use TLS 1.0 or even SSL with SSLSocket.getpeercert and there is no place to specify to the ssl_version like there is in ssl.get_server_certificate.
So I see both methods retrieve the server certificate and each seems to have different issues. But what are the differences with what each does? When would I use one over the other?
I am using Python 3. I am using a SSLContext from the ssl library.
I would like to know how to request and get the remote peer certificate (as with SSLSocket.getpeercert()) but in the CERT_NONE mode. Currently, I get an empty dict (which is expected as per the documentation). So, can I do that without digging into private stuff?
N.B.
I am not trying to reimplement crypto. Actually, I don't need any crypto, however I need the context (and a socket) and the remote certificate (again, for information purpose, not to authenticate or actually encrypt anything).
I found a way without straying away from the default ssl library (which has the advantage of being there):
socket.getpeercert(binary_form=True)
gives the certificate in DER format which is good enough for what I have to do.
Afaik, returning an empty dict is to avoid blindly trusting what people send in clear text and has not been verified.
Python/OpenSSL mix requesting the client certificate certificate with validating it:
CERT_NONE (OpenSSL: SSL_VERIFY_NONE) will not request the certificate from the client at all. This means the client will not send a certificate.
CERT_REQUIRE (OpenSSL: SSL_VERFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT) and CERT_OPTIONAL (OpenSSL: SSL_VERIFY_PEER) will request the certificate but then also fail if the certificate could not be verified.
What you would need is to require the certificate but then have a way to handle the validation yourself and just accept all client certificates. This could be done with a custom verification callback. In OpenSSL this could be done with SSL_CTX_set_verify. But unfortunately the Python ssl library don't seem to expose the necessary interface to have your own validation callback. But it should be possible to do with pyOpenSSL which lets you set your own verification callback.
I have already implemented SSL communication where client application verifies the identity of the SSL server application using flask. Now I want SSL server application to verify the identity of the SSL-client application. Is it possible with flask ? How do I verify client certificate ? During first handshake client is sending CSR and in response I am sending back certificate signed by Self Signed CA certificate.
But I am not yet clear how the client will be verified by server while next communication. Is there any callback for cert verification. Link on Google groups says its not possible to have ssl authentication on Flask. in order to do this one need to use webserver like apache,ngnix. Is this the only way to authenticate client ?
There is one more thing that I want to achieve that I need to identify each client based on their certificate. is that even possible with flask.
my question could be naive as I am not yet much familiar to flask
Disclaimer
Before I start I would note #Emanuel Ey's comment. That you would want to consider if this was being done on a production or development server first. For example; if you are using Apache WebServer the HTTPS component can be done from Apache. The only thing you would do differently is pass through the certificate details as options and your server app would then verify the serial number within the app itself.
It is Possible
But it the way it is possible is not considered good programming practice. Unfortunately, it's not accessible from flask.request and not possible with the Flask package. However, Flask uses Werkzeug and it is possible by patching the werkzeug.serving package where will be writing your main Flask code. It is not recommended because you may want to update Flask or Werkzeug later and your patch might break and need to be re-factored. i.e. from 0.9 to 1.0.
This provides a solution without using a web server. But I would recommend the web server/environment variable combo. It is cleaner and comparatively good practice.
I have done some testing to see if this is easy to implement. I was able to confirm that this method can work using the latest development codebase 'Werkzeug-0.10_devdev_20141223-py2.7'.
You'll probably want to verify of the serial number (seed number) found in each certificate (and maybe even some other variables). As you may know, the serial is unique to each certificate and is determined during the certificate generation process by you on the server side. It helps to store this along with the clients record and certificate information (where appropriate) in order to verify client certificate serial number later on. Note: It may require alterations between hex and base 10 decimal.
Werkzeug dev_2014122
What I did was to add in the following options to the werkzeug.serving.BaseWSGIServer.__init__ call to wrap_socket().
Use these;
server_side=True, ca_certs= '/etc/apache2/ssl/ca.pem', cert_reqs=ssl.CERT_REQUIRED
ca_certs: Use this to verify against, this is the CA cert used to generate the client certificates)
ssl.CERT_REQUIRED: require client certificate verification against ca_certs
Note: If the client certificate is does not pass initial verification you will not be able to fetch the client certificate. It will be None.
Then in my Flask test class I patched verify_request
where
def verify_request(self, request, client_address):
cert = request.getpeercert(True)
raw = decoder.decode(cert)[0]
print "Serial Number of your certificate is: % " % str(raw[0][1])
# todo: do checks & if serial no is ok then return true
return True
werkzeug.serving.BaseWSGIServer.verify_request = verify_request
This proved it is possible but you'll probably want to investigate the request handlers of the HTTPServer class that the BaseWSGIServer inherits to find a better way to do a call back or override.
Werkzeug 0.9.X
If you are using Werkzeug 0.9.X I'm assuming you are using the import from OpenSSL import SSL. see code snippet here. I have not tested this.
Some of the calls you may be interested in for this version would be;
- Context.set_verify(mode, callback)
- Connection.get_peer_certificate()
Clarification
What I do not understand is your reference to sending a CSR during the first handshake. If this is your process of client certificate generation you may want to rethink how you do this in the context of your system and environment. If I could have some more information I could comment further..
Also, 'handshake' in an SSL/TLS context generally refers to the action of creating the secure connection in the first place using an existing certificate. Immediately after handshaking, loosely speaking, a connection is established.
I need to make a HTTPS call to a server with a certificate signed by our corporate authority. How do I configure python to trust this certificate authority?
By default (assuming you're using httplib.HTTPSConnection), you don't have to do anything, this httplib doesn't do any certificate verification (the same applies to urllib).
Of course, this is not a good this and you should verify the server certificate. There are various solutions to this (see this question).
In short, you may have to extend httplib.HTTPConnection to turn the socket into an SSL socket via ssl.wrap_socket manually, so as to be able to insert the verification callbacks (you'll need to verify both the host name and the certificates).
Alternatively, if you're not constrained to httplib, using PycURL would certainly make this cleaner. You can configure CA_INFO (or CA_PATH) to point to your internal CA certificate. In addition, it usually doesn't come with a pre-defined list of CAs, but you can get one from here (converted from the Mozilla list) and add these certificates to the list of CAs you trust if you need it.
If you're looking for sample code to solve this, here it is in PycURL:
import pycurl
curl = pycurl.Curl()
curl.setopt(pycurl.URL, "https://your-secure-website.com/")
curl.setopt(pycurl.SSL_VERIFYPEER, 1)
curl.setopt(pycurl.SSL_VERIFYHOST, 2)
curl.setopt(pycurl.CAINFO, "/path/to/your-corporate-certificate-chain.crt")
curl.perform()
Make sure to place the your-corporate-certificate-chain.crt file in an accessible location and use the pycurl.CAINFO option to point to it.