I am trying to list the certificates that Python SSL is actually using to verify a remote server, but clearly don't understand how this works.
The following code works but seems to contradict itself.
A self-signed cert is installed in Windows: Local Computer\Trusted Root Certification Authorities\Certificates and works nicely in Chrome etc., but is not present in the certificate list that ssl has loaded.
import ssl
import socket
from pprint import pprint, pformat
hostname = 'myhost.mydom.tld' # self-signed cert
port = 443
context = ssl.create_default_context()
print(context.verify_mode) # VerifyMode.CERT_REQUIRED
# connect to server and fetch cert (verified to fail on invalid cert)
with socket.create_connection((hostname, port)) as sock:
with context.wrap_socket(sock, server_hostname=hostname) as ssock:
print(ssock.version())
cert = ssock.getpeercert()
pprint(cert) # the expected self-signed cert
print('-' * 60)
# print certs we verified against
pprint(context.get_ca_certs()) # self-signed cert not present
print('-' * 60)
# just make extra sure
for cert in context.get_ca_certs():
if 'MYHOST' in pformat(cert).upper():
pprint(cert) # never reached
So why am I not finding the expected self-signed cert in the list when it is in fact verified?
Related
How do I make a simple request for certificate revocation status to an EJBCA OSCP Responder using the Python requests library?
Example:
# Determine if certificate has been revoked
ocsp_url = req_cert.extensions[2].value[0].access_location.value
ocsp_headers = {"whatGoes: here?"}
ocsp_body = {"What goes here?"}
ocsp_response = requests.get(ocsp_url, ocsp_headers, ocsp_body)
if (ocsp_response == 'revoked'):
return func.HttpResponse(
"Certificate is not valid (Revoked)."
)
Basically it involves the following steps:
retrieve the corresponding cert for a hostname
if a corresponding entry is contained in the certificate, you can query the extensions via AuthorityInformationAccessOID.CA_ISSUERS, which will provide you with a link to the issuer certificate if successful
retrieve the issuer cert with this link
similarly you get via AuthorityInformationAccessOID.OCSP the corresponding OCSP server
with this information about the current cert, the issuer_cert and the ocsp server you can feed OCSPRequestBuilder to create an OCSP request
use requests.get to get the OCSP response
from the OCSP response retrieve the certificate_status
To retrieve a cert for a hostname and port, you can use this fine answer: https://stackoverflow.com/a/49132495. The OCSP handling in Python is documented here: https://cryptography.io/en/latest/x509/ocsp.html.
Code
If you convert the above points into a self-contained example, it looks something like this:
import base64
import ssl
import requests
from urllib.parse import urljoin
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.hashes import SHA256
from cryptography.x509 import ocsp
from cryptography.x509.ocsp import OCSPResponseStatus
from cryptography.x509.oid import ExtensionOID, AuthorityInformationAccessOID
def get_cert_for_hostname(hostname, port):
conn = ssl.create_connection((hostname, port))
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
sock = context.wrap_socket(conn, server_hostname=hostname)
certDER = sock.getpeercert(True)
certPEM = ssl.DER_cert_to_PEM_cert(certDER)
return x509.load_pem_x509_certificate(certPEM.encode('ascii'), default_backend())
def get_issuer(cert):
aia = cert.extensions.get_extension_for_oid(ExtensionOID.AUTHORITY_INFORMATION_ACCESS).value
issuers = [ia for ia in aia if ia.access_method == AuthorityInformationAccessOID.CA_ISSUERS]
if not issuers:
raise Exception(f'no issuers entry in AIA')
return issuers[0].access_location.value
def get_ocsp_server(cert):
aia = cert.extensions.get_extension_for_oid(ExtensionOID.AUTHORITY_INFORMATION_ACCESS).value
ocsps = [ia for ia in aia if ia.access_method == AuthorityInformationAccessOID.OCSP]
if not ocsps:
raise Exception(f'no ocsp server entry in AIA')
return ocsps[0].access_location.value
def get_issuer_cert(ca_issuer):
issuer_response = requests.get(ca_issuer)
if issuer_response.ok:
issuerDER = issuer_response.content
issuerPEM = ssl.DER_cert_to_PEM_cert(issuerDER)
return x509.load_pem_x509_certificate(issuerPEM.encode('ascii'), default_backend())
raise Exception(f'fetching issuer cert failed with response status: {issuer_response.status_code}')
def get_oscp_request(ocsp_server, cert, issuer_cert):
builder = ocsp.OCSPRequestBuilder()
builder = builder.add_certificate(cert, issuer_cert, SHA256())
req = builder.build()
req_path = base64.b64encode(req.public_bytes(serialization.Encoding.DER))
return urljoin(ocsp_server + '/', req_path.decode('ascii'))
def get_ocsp_cert_status(ocsp_server, cert, issuer_cert):
ocsp_resp = requests.get(get_oscp_request(ocsp_server, cert, issuer_cert))
if ocsp_resp.ok:
ocsp_decoded = ocsp.load_der_ocsp_response(ocsp_resp.content)
if ocsp_decoded.response_status == OCSPResponseStatus.SUCCESSFUL:
return ocsp_decoded.certificate_status
else:
raise Exception(f'decoding ocsp response failed: {ocsp_decoded.response_status}')
raise Exception(f'fetching ocsp cert status failed with response status: {ocsp_resp.status_code}')
def get_cert_status_for_host(hostname, port):
print(' hostname:', hostname, "port:", port)
cert = get_cert_for_hostname(hostname, port)
ca_issuer = get_issuer(cert)
print(' issuer ->', ca_issuer)
issuer_cert = get_issuer_cert(ca_issuer)
ocsp_server = get_ocsp_server(cert)
print(' ocsp_server ->', ocsp_server)
return get_ocsp_cert_status(ocsp_server, cert, issuer_cert)
Test 1: Good Certificate
A test call like the following with a good certificate
status = get_cert_status_for_host('software7.com', 443)
print('software7.com:', status, '\n')
results in the following output:
hostname: software7.com port: 443
issuer -> http://cacerts.digicert.com/EncryptionEverywhereDVTLSCA-G1.crt
ocsp_server -> http://ocsp.digicert.com
software7.com: OCSPCertStatus.GOOD
Test 2: Revoked Certificate
Of course you also have to do a counter test with a revoked cert. Here revoked.badssl.com is the first choice:
status = get_cert_status_for_host('revoked.badssl.com', 443)
print('revoked.badssl.com:', status, '\n')
This gives as output:
hostname: revoked.badssl.com port: 443
issuer -> http://cacerts.digicert.com/DigiCertSHA2SecureServerCA.crt
ocsp_server -> http://ocsp.digicert.com
revoked.badssl.com: OCSPCertStatus.REVOKED
AIA Retrieval of the Issuer Certificate
A typical scenario for a certificate relationship looks as follows:
The server provides the server certificate and usually one or more intermediate certificates during the TLS handshake. The word 'usually' is used intentionally: some servers are configured not to deliver intermediate certificates. The browsers then use AIA fetching to build the certification chain.
Up to two entries can be present in the Certificate Authority Information Access extension: The entry for downloading the issuer certificate and the link to the OCSP server.
These entries may also be missing, but a short test script that checks the certs of the 100 most popular servers shows that these entries are usually included in certificates issued by public certification authorities.
The CA Issuers entry may also be missing, but while the information about an OCSP server is available, it can be tested e.g. with OpenSSL using a self-signed certificate:
In this case you would have to determine the issuer certificate from the chain in the TLS handshake, it is the certificate that comes directly after the server certificate in the chain, see also the figure above.
Just for the sake of completeness: There is another case that can sometimes occur especially in conjunction with self-signed certificates: If no intermediate certificates are used, the corresponding root certificate (e.g. available in the local trust store) must be used as issuer certificate.
I have a Socket Server running on Ubuntu 18.04 which I am trying to run with SSL/TLS. First I used a self-signed certificate and the code works fine but now I am trying to use a 90-day free SSL certificate signed by ZeroSSL and it doesn't work. Below are the error messages I received on the server and client side upon connection request.
Server side error: ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:852)
Client side error: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)
Server Code:
import ssl
import socket
IP = "0.0.0.0"
PORT = 2021
server_cert = "/home/ubuntu/chandral/ssl_socket_test/Certificates/Ubuntu/certificate.crt"
#server_cert = "/home/ubuntu/chandral/ssl_socket_test/Certificates/Ubuntu/ca_bundle.crt"
server_key = "/home/ubuntu/chandral/SSL/server.key"
context = ssl.create_default_context(purpose=ssl.Purpose.CLIENT_AUTH)
context.load_cert_chain(certfile=server_cert, keyfile=server_key)
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
secure_server_socket = context.wrap_socket(server_socket, server_side=True)
secure_server_socket.bind((IP, PORT))
secure_server_socket.listen(5)
print("listening for connections")
while True:
client_socket, address = secure_server_socket.accept()
print("IP/URL of client:", address)
Client Code:
import ssl
import socket
IP = "192.168.1.1" # IP changed in this post for confidentiality
PORT = 2021
server_sni_hostname = "example.com" # Domain changed in this post for confidentiality
context = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH)
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
secure_client_socket = context.wrap_socket(client_socket, server_side=False, server_hostname=server_sni_hostname)
secure_client_socket.connect((IP, PORT))
secure_client_socket.send(b"Hello World")
server_cert = "/home/ubuntu/chandral/ssl_socket_test/Certificates/Ubuntu/certificate.crt"
#server_cert = "/home/ubuntu/chandral/ssl_socket_test/Certificates/Ubuntu/ca_bundle.crt"
ZeroSSL provides three files: certificate.crt is the leaf certificate of the server, private.key the private key (server.key in your code) and ca_bundle.crt the intermediate certificates and the root certificates.
A server must provide both the leaf certificate and all chain certificates so that the client can build the trust chain to the local root CA certificate. To create such setup you need to concatenate the contents of certificate.crt with ca_bundle.crt and use the result with context.load_cert_chain(...). Note that it might be necessary to add an explicite newline after the certificate.crt in case this is missing at the end of the file. Note also that the self-signed root CA (the last certificate in ca_bundle.crt) does not need to be included since it will be ignored by the client anyway and instead the local root CA will be used. It is actually recommended to leave this out.
I am trying to get Certificate issuer information (Common Name), but the code in the link doesn't work with some URLs.
How can i get Certificate issuer information in python?
import ssl, socket
hostname = 'google.com'
ctx = ssl.create_default_context()
s = ctx.wrap_socket(socket.socket(), server_hostname=hostname)
s.connect((hostname, 443))
cert = s.getpeercert()
subject = dict(x[0] for x in cert['subject'])
issued_to = subject['commonName']
>>> issued_to
u'*.google.com'
For example, I tried hostname "cds.ca", it says
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)
but I could still get the Common name using Internet Explorer (*.cds.ca)
So I think I should use my own certificate(.cer) instead of using getpeercert(), then how should I change that line?
Or, is there any other way to achieve CN with my own certificate file?
If you just want to get the CN or other certificate details no matter if certificate validation succeeds or not, you have to disable the verification. Unfortunately a simple sock.getpeercert() will by design only return an empty dictionary if certificate validation is disabled. That's why one has to use sock.getpeercert(True) to get the binary representation of the certificate and extract the CN using OpenSSL.crypto from it:
import socket
import ssl
import OpenSSL.crypto as crypto
dst = ('cds.ca',443)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(dst)
# upgrade the socket to SSL without checking the certificate
# !!!! don't transfer any sensitive data over this socket !!!!
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
s = ctx.wrap_socket(s, server_hostname=dst[0])
# get certificate
cert_bin = s.getpeercert(True)
x509 = crypto.load_certificate(crypto.FILETYPE_ASN1,cert_bin)
print("CN=" + x509.get_subject().CN)
This is not a duplicate for this post. I tried the solutions there and nothing works in my case.
I am using Windows and Python 3.6.5. I have a python script for a TLS client. The server I need to connect to uses a self-signed certificate. When I try to connect to it using my script, I get this error:
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)
I need to parse the certificate. I tried to add my server's certificate .pem content to file named: cacert.pem which is in: C:\Python36\Lib\site-packages\certifi and run the program again. Nothing change. Here is my scripot. Please help me to make the client make exception for this server as I trust its certificate.
import socket, ssl
import itertools
context = ssl.SSLContext()
context.verify_mode = ssl.CERT_OPTIONAL
context.check_hostname = False
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
domain="192.168.56.3" # my server
ssl_sock = context.wrap_socket(s, server_hostname=domain)
ssl_sock.connect((domain, 443))
print("====== peer's certificate ======")
try:
cert = ssl_sock.getpeercert()
print(cert)
except SSLError as e:
print("Error: ",e)
ssl_sock.close()
This question has been idle for a while, but in case somebody is still struggling with connecting to a server with a self-signed certificate via Python ssl library:
You can use the load_verify_locations method of SSLContext to specify custom self-signed root certs (see Python Docs for load_verify_locations).
The forementioned code could be extended as follows:
...
context = ssl.SSLContext()
context.verify_mode = ssl.CERT_OPTIONAL
context.check_hostname = False
context.load_verify_locations(cafile='/path/to/your/cacert.pem')
...
Be aware that you also need to include public root certs in case you want to connect to other servers with public certificates with the same client/context. (you could for example append your cacert.pem content to the certifi root certificates and reference that folder / path instead).
See also this Python docs paragraph for more information: client-side operations.
I want imaplib to display the md5 (or SHA) key of an IMAP Server Certificate to make sure, that there's no MITM (I don't trust the CA, so verifying the chain isn't enough in this case).
Displaying the whole certificate would also be okay.
I'd appreciate any help!!
Chris
You can use the M2Crypto package to parse the full SSL certificate from the IMAP connection's SSL socket. Here is an example:
import imaplib
from M2Crypto import X509
cn = imaplib.IMAP4_SSL('imap.gmail.com', 993)
sock = cn.ssl()
data = sock.getpeercert(1)
cert = X509.load_cert_string(data, X509.FORMAT_DER)
print cert.get_fingerprint()
Prints:
2029AF27C0A55390D670C0BD7AB9747
Use the other attributes on cert to get further information.
I don't know how to do it from imaplib, but you can connect to a secure IMAP server and display the certificate using M2Crypto:
from M2Crypto import SSL
ctx = SSL.Context('sslv3')
c = SSL.Connection(ctx)
c.connect(('localhost', 993)) # automatically checks cert matches host
cert = c.get_peer_cert()
print cert.as_pem()
print cert.as_text()
Note that cert is an X509 object.