I want imaplib to display the md5 (or SHA) key of an IMAP Server Certificate to make sure, that there's no MITM (I don't trust the CA, so verifying the chain isn't enough in this case).
Displaying the whole certificate would also be okay.
I'd appreciate any help!!
Chris
You can use the M2Crypto package to parse the full SSL certificate from the IMAP connection's SSL socket. Here is an example:
import imaplib
from M2Crypto import X509
cn = imaplib.IMAP4_SSL('imap.gmail.com', 993)
sock = cn.ssl()
data = sock.getpeercert(1)
cert = X509.load_cert_string(data, X509.FORMAT_DER)
print cert.get_fingerprint()
Prints:
2029AF27C0A55390D670C0BD7AB9747
Use the other attributes on cert to get further information.
I don't know how to do it from imaplib, but you can connect to a secure IMAP server and display the certificate using M2Crypto:
from M2Crypto import SSL
ctx = SSL.Context('sslv3')
c = SSL.Connection(ctx)
c.connect(('localhost', 993)) # automatically checks cert matches host
cert = c.get_peer_cert()
print cert.as_pem()
print cert.as_text()
Note that cert is an X509 object.
Related
I am trying to list the certificates that Python SSL is actually using to verify a remote server, but clearly don't understand how this works.
The following code works but seems to contradict itself.
A self-signed cert is installed in Windows: Local Computer\Trusted Root Certification Authorities\Certificates and works nicely in Chrome etc., but is not present in the certificate list that ssl has loaded.
import ssl
import socket
from pprint import pprint, pformat
hostname = 'myhost.mydom.tld' # self-signed cert
port = 443
context = ssl.create_default_context()
print(context.verify_mode) # VerifyMode.CERT_REQUIRED
# connect to server and fetch cert (verified to fail on invalid cert)
with socket.create_connection((hostname, port)) as sock:
with context.wrap_socket(sock, server_hostname=hostname) as ssock:
print(ssock.version())
cert = ssock.getpeercert()
pprint(cert) # the expected self-signed cert
print('-' * 60)
# print certs we verified against
pprint(context.get_ca_certs()) # self-signed cert not present
print('-' * 60)
# just make extra sure
for cert in context.get_ca_certs():
if 'MYHOST' in pformat(cert).upper():
pprint(cert) # never reached
So why am I not finding the expected self-signed cert in the list when it is in fact verified?
My aim is to achieve SHA1 fingerprint of a third party website's certificate. I am able to get it successfully using openssl command line however, it's not getting same when I tried to achieve it using python code. The SHA1 fingerprint obtained using python code is totally different than the one obtained via openssl.
openssl steps -->
openssl s_client -servername token.actions.githubusercontent.com -showcerts -connect token.actions.githubusercontent.com:443
The above command output contains chain and root certificate;
Certificate chain
0 s:/C=US/ST=California/L=San Francisco/O=GitHub, Inc./CN=*.actions.githubusercontent.com
i:/C=US/O=DigiCert Inc/CN=DigiCert TLS RSA SHA256 2020 CA1
-----BEGIN CERTIFICATE-----
MIIG9jCCBd6gAwIBAgIQCFCR4fqbkQJJbzQZsc87qzANBgkqhkiG9w0BAQsFADBP
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMSkwJwYDVQQDEyBE
aWdpQ2VydCBUTFMgUlNBIFNIQTI1NiAyMDIwIENBMTAeFw0yMjAxMTEwMDAwMDBa
Save the chain certificate with .crt extension as MaingithubOIDC.crt and running below command gives SHA1 fingerprint;
❯ openssl x509 -in MaingithubOIDC.crt -fingerprint -noout
SHA1 Fingerprint=15:E2:91:08:71:81:11:E5:9B:3D:AD:31:95:46:47:E3:C3:44:A2:31
Reference link - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc_verify-thumbprint.html
Python code (version 3.8/3.9) -->
import ssl
import socket
import hashlib
addr = 'token.actions.githubusercontent.com'
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(1)
wrappedSocket = ssl.wrap_socket(sock)
try:
wrappedSocket.connect((addr, 443))
print (wrappedSocket)
except:
response = False
else:
der_cert = wrappedSocket.getpeercert(True)
pem_cert = ssl.DER_cert_to_PEM_cert(wrappedSocket.getpeercert(True))
print(pem_cert)
#Print SHA1 Thumbprint
thumb_sha1 = hashlib.sha1(der_cert).hexdigest()
print("SHA1: " + thumb_sha1)
Python code output;
SHA1: 55a7ef500a3a99f64c99c665daaf3f07403cff3d
So, the SHA1 fingerprint doesn't match with the one obtained using openssl. Am I missing something in python code?
The problem is not the wrong fingerprint calculation from the certificate but that you get the wrong certificate. The server in question is a multi-domain setup which will return different certificates based on the server_name given in the TLS handshake - see Server Name Indication.
The following code will not provide a server_name, which results in a certificate returned for *.azureedge.net, not *.actions.githubusercontent.com as the openssl s_client code gets:
wrappedSocket = ssl.wrap_socket(sock)
try:
wrappedSocket.connect((addr, 443))
To fix this the server_name need to be given:
ctx = ssl.create_default_context()
wrappedSocket = ctx.wrap_socket(sock,
server_hostname='token.actions.githubusercontent.com')
try:
wrappedSocket.connect((addr, 443))
With this change the expected certificate is send by the server and the fingerprint is properly calculated on it.
How do I make a simple request for certificate revocation status to an EJBCA OSCP Responder using the Python requests library?
Example:
# Determine if certificate has been revoked
ocsp_url = req_cert.extensions[2].value[0].access_location.value
ocsp_headers = {"whatGoes: here?"}
ocsp_body = {"What goes here?"}
ocsp_response = requests.get(ocsp_url, ocsp_headers, ocsp_body)
if (ocsp_response == 'revoked'):
return func.HttpResponse(
"Certificate is not valid (Revoked)."
)
Basically it involves the following steps:
retrieve the corresponding cert for a hostname
if a corresponding entry is contained in the certificate, you can query the extensions via AuthorityInformationAccessOID.CA_ISSUERS, which will provide you with a link to the issuer certificate if successful
retrieve the issuer cert with this link
similarly you get via AuthorityInformationAccessOID.OCSP the corresponding OCSP server
with this information about the current cert, the issuer_cert and the ocsp server you can feed OCSPRequestBuilder to create an OCSP request
use requests.get to get the OCSP response
from the OCSP response retrieve the certificate_status
To retrieve a cert for a hostname and port, you can use this fine answer: https://stackoverflow.com/a/49132495. The OCSP handling in Python is documented here: https://cryptography.io/en/latest/x509/ocsp.html.
Code
If you convert the above points into a self-contained example, it looks something like this:
import base64
import ssl
import requests
from urllib.parse import urljoin
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.hashes import SHA256
from cryptography.x509 import ocsp
from cryptography.x509.ocsp import OCSPResponseStatus
from cryptography.x509.oid import ExtensionOID, AuthorityInformationAccessOID
def get_cert_for_hostname(hostname, port):
conn = ssl.create_connection((hostname, port))
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
sock = context.wrap_socket(conn, server_hostname=hostname)
certDER = sock.getpeercert(True)
certPEM = ssl.DER_cert_to_PEM_cert(certDER)
return x509.load_pem_x509_certificate(certPEM.encode('ascii'), default_backend())
def get_issuer(cert):
aia = cert.extensions.get_extension_for_oid(ExtensionOID.AUTHORITY_INFORMATION_ACCESS).value
issuers = [ia for ia in aia if ia.access_method == AuthorityInformationAccessOID.CA_ISSUERS]
if not issuers:
raise Exception(f'no issuers entry in AIA')
return issuers[0].access_location.value
def get_ocsp_server(cert):
aia = cert.extensions.get_extension_for_oid(ExtensionOID.AUTHORITY_INFORMATION_ACCESS).value
ocsps = [ia for ia in aia if ia.access_method == AuthorityInformationAccessOID.OCSP]
if not ocsps:
raise Exception(f'no ocsp server entry in AIA')
return ocsps[0].access_location.value
def get_issuer_cert(ca_issuer):
issuer_response = requests.get(ca_issuer)
if issuer_response.ok:
issuerDER = issuer_response.content
issuerPEM = ssl.DER_cert_to_PEM_cert(issuerDER)
return x509.load_pem_x509_certificate(issuerPEM.encode('ascii'), default_backend())
raise Exception(f'fetching issuer cert failed with response status: {issuer_response.status_code}')
def get_oscp_request(ocsp_server, cert, issuer_cert):
builder = ocsp.OCSPRequestBuilder()
builder = builder.add_certificate(cert, issuer_cert, SHA256())
req = builder.build()
req_path = base64.b64encode(req.public_bytes(serialization.Encoding.DER))
return urljoin(ocsp_server + '/', req_path.decode('ascii'))
def get_ocsp_cert_status(ocsp_server, cert, issuer_cert):
ocsp_resp = requests.get(get_oscp_request(ocsp_server, cert, issuer_cert))
if ocsp_resp.ok:
ocsp_decoded = ocsp.load_der_ocsp_response(ocsp_resp.content)
if ocsp_decoded.response_status == OCSPResponseStatus.SUCCESSFUL:
return ocsp_decoded.certificate_status
else:
raise Exception(f'decoding ocsp response failed: {ocsp_decoded.response_status}')
raise Exception(f'fetching ocsp cert status failed with response status: {ocsp_resp.status_code}')
def get_cert_status_for_host(hostname, port):
print(' hostname:', hostname, "port:", port)
cert = get_cert_for_hostname(hostname, port)
ca_issuer = get_issuer(cert)
print(' issuer ->', ca_issuer)
issuer_cert = get_issuer_cert(ca_issuer)
ocsp_server = get_ocsp_server(cert)
print(' ocsp_server ->', ocsp_server)
return get_ocsp_cert_status(ocsp_server, cert, issuer_cert)
Test 1: Good Certificate
A test call like the following with a good certificate
status = get_cert_status_for_host('software7.com', 443)
print('software7.com:', status, '\n')
results in the following output:
hostname: software7.com port: 443
issuer -> http://cacerts.digicert.com/EncryptionEverywhereDVTLSCA-G1.crt
ocsp_server -> http://ocsp.digicert.com
software7.com: OCSPCertStatus.GOOD
Test 2: Revoked Certificate
Of course you also have to do a counter test with a revoked cert. Here revoked.badssl.com is the first choice:
status = get_cert_status_for_host('revoked.badssl.com', 443)
print('revoked.badssl.com:', status, '\n')
This gives as output:
hostname: revoked.badssl.com port: 443
issuer -> http://cacerts.digicert.com/DigiCertSHA2SecureServerCA.crt
ocsp_server -> http://ocsp.digicert.com
revoked.badssl.com: OCSPCertStatus.REVOKED
AIA Retrieval of the Issuer Certificate
A typical scenario for a certificate relationship looks as follows:
The server provides the server certificate and usually one or more intermediate certificates during the TLS handshake. The word 'usually' is used intentionally: some servers are configured not to deliver intermediate certificates. The browsers then use AIA fetching to build the certification chain.
Up to two entries can be present in the Certificate Authority Information Access extension: The entry for downloading the issuer certificate and the link to the OCSP server.
These entries may also be missing, but a short test script that checks the certs of the 100 most popular servers shows that these entries are usually included in certificates issued by public certification authorities.
The CA Issuers entry may also be missing, but while the information about an OCSP server is available, it can be tested e.g. with OpenSSL using a self-signed certificate:
In this case you would have to determine the issuer certificate from the chain in the TLS handshake, it is the certificate that comes directly after the server certificate in the chain, see also the figure above.
Just for the sake of completeness: There is another case that can sometimes occur especially in conjunction with self-signed certificates: If no intermediate certificates are used, the corresponding root certificate (e.g. available in the local trust store) must be used as issuer certificate.
In Python 3.4, a verify_flags that can be used to check if a certificate was revoked against CRL, by set it to VERIFY_CRL_CHECK_LEAF or VERIFY_CRL_CHECK_CHAIN.
I wrote a simple program for testing. But on my systems, this script failed to verify ANY connections even if it's perfectly valid.
import ssl
import socket
def tlscheck(domain, port):
addr = domain
ctx = ssl.create_default_context()
ctx.options &= ssl.CERT_REQUIRED
ctx.verify_flags = ssl.VERIFY_CRL_CHECK_LEAF
ctx.check_hostname = True
#ctx.load_default_certs()
#ctx.set_default_verify_paths()
#ctx.load_verify_locations(cafile="/etc/ssl/certs/ca-certificates.crt")
sock = ctx.wrap_socket(socket.socket(), server_hostname=addr)
sock.connect((addr, port))
import pprint
print("TLS Ceritificate:")
pprint.pprint(sock.getpeercert())
print("TLS Version:", sock.version())
print("TLS Cipher:", sock.cipher()[0])
exit()
tlscheck("stackoverflow.com", 443)
My code always quits with ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645).
First I suspected that the certificate database was not loaded properly. But after I tried load_default_certs(), set_default_verify_paths(), load_verify_locations(cafile="/etc/ssl/certs/ca-certificates.crt"), and none of them worked.
Also, ctx.options &= ssl.CERT_REQUIRED works as expected, it can tell if a certificate chain is trusted or not. But not for CRLs... It also indicates that my CAs are correct.
I know "/etc/ssl/certs/ca-certificates.crt" contains valid CAs. What is the problem?
To check against CRL you have to manually download the CRL and put them in the right place so that the underlying OpenSSL library will find it. There is no automatic downloading of CRL and specifying the place where to look for the CRL is not intuitive either. What you can do:
get the CRL distribution points from the certificate. For stackoverflow.com one is http://crl3.digicert.com/sha2-ha-server-g5.crl
download the current CRL from there
convert it from the DER format to PEM format, because this is what is expected in the next step:
openssl crl -inform der -in sha2-ha-server-g5.crl > sha2-ha-server-g5.crl.pem
add the location to the verify_locations:
ctx.load_verify_locations(cafile="./sha2-ha-server-g5.crl.pem")
This way you can verify the certificate against the CRL.
What's the easiest way to connect to a SMTP server that supports STARTTLS and get its server SSL certificate? I know it can be done using openssl with something like this
openssl s_client -starttls smtp -crlf -connect 192.168.0.1:25
How can I do it from within Python and I don't want to call openssl and parse its output. I looked at M2Crypto which is an openssl wrapper, but as far as I can tell that doesn't support starttls. An example of how to do it with a Python library would be very much appreciated.
This returns a certificate in binary format (DER-encoded):
import socket, ssl
s = socket.socket()
s.connect(("host", 25))
s.send("STARTTLS\n")
s.recv(1000)
ss = ssl.wrap_socket(s)
certificate_der = ss.getpeercert(True)
This is jus to give you an idea, error handling, etc. is required of course. If you want to decode the information from the certificate you either have to prodivde a certificate authorities bundle/directory for acceptable CAs (getpeercert() will return a meaningfull dict in this case), or use a more capable ssl library, M2Crypto for example.
You could try something like:
import ssl
cert = ssl.get_server_certificate(('imap.gmail.com',993))
to get server's certificate
As I can't comment abbot answer, just remark that depending on the server config you may need to send an EHLO before STARTTLS:
import socket, ssl
hostname = 'test.com'
port = 25
context = ssl.create_default_context()
with socket.create_connection((hostname, port)) as sock:
sock.recv(1000)
sock.send(b'EHLO\nSTARTTLS\n')
sock.recv(1000)
with context.wrap_socket(sock, server_hostname=hostname) as sslsock:
der_cert = sslsock.getpeercert(True)
smtplib provides the starttls() method which should deal with all issues:
http://docs.python.org/library/smtplib.html