I'm writing a script that connects to a bunch of URLs over HTTPS, downloads their SSL certificate and extracts the CN. Everything works except when I stumble on a site with an invalid SSL certificate. I absolutely do not care if the certificate is valid or not. I just want the CN but Python stubbornly refuses to extract the certificate information if the certificate is not validated. Is there any way to get around this profoundly stupid behavior? Oh, I'm using the built-in socket and ssl libraries only. I don't want to use third-party libraries like M2Crypto or pyOpenSSL because I'm trying to keep the script as portable as possible.
Here's the relevant code:
file = open("list.txt", "r")
for x in file:
server = socket.getaddrinfo(x.rstrip(), "443")[0][4][0]
sslsocket = socket.socket()
sslsocket.connect((server, 443))
sslsocket = ssl.wrap_socket(sslsocket, cert_reqs=ssl.CERT_REQUIRED, ca_certs="cacerts.txt")
certificate = sslsocket.getpeercert()`
The ssl.get_server_certificate can do it:
import ssl
ssl.get_server_certificate(("www.sefaz.ce.gov.br",443))
I think function doc string is more clear than python doc site:
"""Retrieve the certificate from the server at the specified address,
and return it as a PEM-encoded string.
If 'ca_certs' is specified, validate the server cert against it.
If 'ssl_version' is specified, use it in the connection attempt."""
So you can extract common name from binary DER certificate searching for common name object identifier:
def get_commonname(host,port=443):
oid='\x06\x03U\x04\x03' # Object Identifier 2.5.4.3 (COMMON NAME)
pem=ssl.get_server_certificate((host,port))
der=ssl.PEM_cert_to_DER_cert(pem)
i=der.find(oid) # find first common name (certificate authority)
if i!=-1:
i=der.find(oid,i+1) # skip and find second common name
if i!=-1:
begin=i+len(oid)+2
end=begin+ord(der[begin-1])
return der[begin:end]
return None
Ok. I cleaned up olivecoder's code to solve the problem that it assumes there will always be three CNs in the certificate chain (root, intermediate, server) and I condensed it. This is the final code I will be using.
cert = ssl.get_server_certificate(("www.google.com", 443)) #Retrieve SSL server certificate
cert = ssl.PEM_cert_to_DER_cert(cert) #Convert certificate to DER format
begin = cert.rfind('\x06\x03\x55\x04\x03') + 7 #Find the last occurence of this byte string indicating the CN, add 7 bytes to startpoint to account for length of byte string and padding
end = begin + ord(cert[begin - 1]) #Set endpoint to startpoint + the length of the CN
print cert[begin:end] #Retrieve the CN from the DER encoded certificate
Related
It's unclear from the docs what you actually do to verify the jwsRepresentation string from a StoreKit 2 transaction on the server side.
Also "signedPayload" from the Apple App Store Notifications V2 seems to be the same, but there is also no documentation around actually validating that either outside of validating it client side on device.
What gives? What do we do with this JWS/JWT?
(DISCLAIMER: I am a crypto novice so check me on this if I'm using the wrong terms, etc. throughout)
The JWS in jwsRepresentation, and the signedPayload in the Notification V2 JSON body, are JWTs — you can take one and check it out at jwt.io. The job is to validate the JWT signature and extract the payload once you're sufficiently convinced it's really from Apple. Then the payload itself contains information you can use to upgrade the user's account/etc. server side once the data is trusted.
To validate the JWT, you need to find the signature that the JWT is signed with, specified in the JWT header's "x5c" collection, validate the certificate chain, and then validate that the signature is really from Apple.
STEP ONE: Load the well-known root & intermediate certs from Apple.
import requests
from OpenSSL import crypto
ROOT_CER_URL = "https://www.apple.com/certificateauthority/AppleRootCA-G3.cer"
G6_CER_URL = "https://www.apple.com/certificateauthority/AppleWWDRCAG6.cer"
root_cert_bytes: bytes = requests.get(ROOT_CER_URL).content
root_cert = crypto.load_certificate(crypto.FILETYPE_ASN1, root_cert_bytes)
g6_cert_bytes: bytes = requests.get(G6_CER_URL).content
g6_cert = crypto.load_certificate(crypto.FILETYPE_ASN1, g6_cert_bytes)
STEP TWO: Get certificate chain out of the JWT header
import jwt # PyJWT library
# Get the signing keys out of the JWT header. The header will look like:
# {"alg": "ES256", "x5c": ["...base64 cert...", "...base64 cert..."]}
header = jwt.get_unverified_header(apple_jwt_string)
provided_certificates: List[crypto.X509] = []
for cert_base64 in header['x5c']:
cert_bytes = base64url_decode(cert_base64)
cert = crypto.load_certificate(crypto.FILETYPE_ASN1, cert_bytes)
provided_certificates.append(cert)
STEP THREE: Validate the chain is what you think it is -- this ensures the cert chain is signed by the real Apple root & intermediate certs.
# First make sure these are the root & intermediate certs from Apple:
assert provided_certificates[-2].digest('sha256') == g6_cert.digest('sha256')
assert provided_certificates[-1].digest('sha256') == root_cert.digest('sha256')
# Now validate that the cert chain is cryptographically legit:
store = crypto.X509Store()
store.add_cert(root_cert)
store.add_cert(g6_cert)
for cert in provided_certificates[:-2]:
try:
crypto.X509StoreContext(store, cert).verify_certificate()
except crypto.X509StoreContextError:
logging.error("Invalid certificate chain in JWT: %s", apple_jwt)
return None
store.add_cert(cert)
FINALLY: Load & validate the JWT using the now-trusted certificate in the header.
# Now that the cert is validated, we can use it to verify the actual signature
# of the JWT. PyJWT does not understand this certificate if we pass it in, so
# we have to get the cryptography library's version of the same key:
cryptography_version_of_key = provided_certificates[0].get_pubkey().to_cryptography_key()
try:
return jwt.decode(apple_jwt, cryptography_version_of_key, algorithms=["ES256"])
except Exception:
logging.exception("Problem validating Apple JWT")
return None
Voila you now have a validated JWT body from the App Store at your disposal.
Gist of entire solution: https://gist.github.com/taylorhughes/3968575b40dd97f851f35892931ebf3e
So I have a credit card looking like smart card with a chip. This card logins on a website after the card is inserted into the card reader.
Now I have to write a program in python which can read the card and login on that website. After research on internet I found out that I need to extract :
Certificate and
Public key (since private key cannot be extracted)
from the card and then use these 2 things to create a HTTPs connection (example here) . So far I am able to extract certificate in pem format. But i cant find a way to extract key in pem format till now. I used PyKCS11 to read the card. Below is my code:
from asn1crypto import pem, x509
from PyKCS11 import *
import binascii
pkcs11 = PyKCS11Lib()
pkcs11.load(r'C:\Windows\System32\XXXX.dll')
print(pkcs11.getSlotList(tokenPresent=False))
slot = pkcs11.getSlotList(tokenPresent=False)[0]
print(pkcs11.getTokenInfo(slot))
session = pkcs11.openSession(0, CKF_SERIAL_SESSION | CKF_RW_SESSION)
session.login('123456')
result = []
result_pem = []
# find public key and print modulus
pubKey = session.findObjects([(CKA_CLASS, CKO_PUBLIC_KEY)])[0]
modulus = session.getAttributeValue(pubKey, [CKA_MODULUS])[0]
print("\nmodulus: {}".format(binascii.hexlify(bytearray(modulus))))
#find certificates
certs = session.findObjects([(CKA_CLASS, CKO_CERTIFICATE)])
for cert in certs:
cka_value, cka_id = session.getAttributeValue(cert, [CKA_VALUE, CKA_ID])
cert_der = bytes(cka_value)
cert = x509.Certificate.load(cert_der)
# Write out a PEM encoded value
cert_pem = pem.armor('CERTIFICATE', cert_der)
result.append(cert)
result_pem.append(cert_pem)
with open('cert.pem','wb') as f:
f.write(cert_pem)
print(result)
So here are my questions:
1. Is my approach right?
If yes, then how to extract public key in pem format?
How this smart card authentication actually works on client side and server side?
Public key extraxction
If you already have exported the certificate, it is probably easier to extract the public key from there, instead of from the smartcard. You can use openssl for that:
openssl x509 -in cert.pem -pubkey -out pubkey.pem -noout
Authentication
What you are trying to achieve is to open a TLS connection with mutual authentication using a client certificate. If you do this, the private key of your client certificate signs parts of the handshake to authenticate itself towards the server.
Extracting the certificate and the public key from the smartcard won't help you here. You need to find a library, which allows you to use your private key straight from your PKCS#11 token.
I'm using python to load a pfx certificate.
pfx = crypto.load_pkcs12(pfx, self.nfe_a1_password)
cert = pfx.get_certificate()
end = datetime.strptime(cert.get_notAfter(), '%Y%m%d%H%M%SZ')
subj = cert.get_subject()
http://www.pyopenssl.org/en/stable/api/crypto.html#OpenSSL.crypto.PKCS12.get_ca_certificates
I got one interesting certificate that the method get_certificate() returns None and if I call get_ca_certificate it returns the certificate.
What does it mean if the pfx do not return the certificate? The certificate issuer sent me a wrong one?
I want to verify the downloaded file's signature and cert using pyopenssl, but the documentation is not clear and Google is of no help.
I have a root CA cert in user's machine, now when user download the file then I will send a certificate and signature along with it. First I need to verify the certificate with rootCA on machine then I need to verify the signature with file
In openssl I can use following to verify the ca cert
openssl verify -CAfile <root_pem> <cert_pem>
and following to verify the file
openssl dgst <algo> -verify <cert_pub_key> -signature <signature> <file>
I am looking for equivalent way to do it using python, most preferably pyopenssl
I'm still learning about OpenSSL in general, let alone PyOpenSSL. Having said that, I was able to verify a file (your second command) in PyOpenSSL with the following:
from OpenSSL.crypto import load_publickey, FILETYPE_PEM, verify, X509
with open(file_to_verify, 'rb') as f:
file_data = f.read()
with open(signature_filename, 'rb') as f:
signature = f.read()
with open(public_key_filename) as f:
public_key_data = f.read()
# load in the publickey file, in my case, I had a .pem file.
# If the file starts with
# "-----BEGIN PUBLIC KEY-----"
# then it is of the PEM type. The only other FILETYPE is
# "FILETYPE_ASN1".
pkey = load_publickey(FILETYPE_PEM, public_key_data)
# the verify() function expects that the public key is
# wrapped in an X.509 certificate
x509 = X509()
x509.set_pubkey(pkey)
# perform the actual verification. We need the X509 object,
# the signature to verify, the file to verify, and the
# algorithm used when signing.
verify(x509, signature, file_data, 'sha256')
The verify() function will return None in the event that verification is successful (i.e. it does nothing) or it will raise an Exception if something went wrong.
I have generated a private key / CSR from pyOpenSSL - code snippet below:
Key:
key = crypto.PKey()
key.generate_key(type, bits)
if os.path.exists(_keyfile):
print "Certificate file exists, aborting."
print " ", _keyfile
sys.exit(1)
else:
f = open(_keyfile, "w")
f.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, key))
f.close()
return key
CSR:
req = crypto.X509Req()
# Return an X509Name object representing the subject of the certificate.
req.get_subject().countryName = country
req.get_subject().stateOrProvinceName = state
req.get_subject().localityName = location
req.get_subject().organizationName = organisation
req.get_subject().organizationalUnitName = organisational_unit
req.get_subject().CN = nodename
# Add in extensions
#base_constraints = ([
# crypto.X509Extension("keyUsage", False, "Digital Signature, Non Repudiation, Key Encipherment"),
# crypto.X509Extension("basicConstraints", False, "CA:FALSE"),
#])
#x509_extensions = ([])
x509_extensions = []
# If there are SAN entries, append the base_constraints to include them.
if ss:
san_constraint = crypto.X509Extension("subjectAltName", False, ss)
x509_extensions.append(san_constraint)
req.add_extensions(x509_extensions)
# Set the public key of the certificate to pkey.
req.set_pubkey(key)
# Sign the certificate, using the key pkey and the message digest algorithm identified by the string digest.
req.sign(key, "sha1")
# Dump the certificate request req into a buffer string encoded with the type type.
if os.path.exists(_csrfile):
print "Certificate file exists, aborting."
print " ", _csrfile
sys.exit(1)
else:
f = open(_csrfile, "w")
f.write(crypto.dump_certificate_request(crypto.FILETYPE_PEM, req))
f.close()
The error that I get back from the IIS CA is:
ASN1 bad tag value met. 0x8009310b (ASN: 267)
According to Microsoft this is caused by:
This behavior occurs when certificate request is stored in a file in Unicode encoding. Microsoft Certificate Services do not support Unicode-encoded files request files. Only ANSI encoding is supported.
I know that if I generate a CSR from openssl on the command line it is accepted and issued by the IIS CA RESTful webservice without error.
I want to know if there is some way I can generate 'ANSI' encoded files from pyOpenSSL - I am not sure if it is the keyfile or the CSR that is signed with the keyfile that is causing the issues.
I have solved it with the help of this stackoverflow question thanks to #yodatg.
The problem occurs due to a bug in pyOpenSSL that has been fixed.
By issuing:
openssl asn1parse -in certificates/cert.csr
I could see the ASN1 value:
8:d=2 hl=2 l= 1 prim: INTEGER :01
In a working CSR it looks like this:
8:d=2 hl=2 l= 1 prim: INTEGER :00
I then changed my code to include a set_version call on the req object prior to signing:
#set version - IIS CA required this
req.set_version(0)
# Set the public key of the certificate to pkey.
req.set_pubkey(priv_key)
This is now resolved.