I have generated a private key / CSR from pyOpenSSL - code snippet below:
Key:
key = crypto.PKey()
key.generate_key(type, bits)
if os.path.exists(_keyfile):
print "Certificate file exists, aborting."
print " ", _keyfile
sys.exit(1)
else:
f = open(_keyfile, "w")
f.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, key))
f.close()
return key
CSR:
req = crypto.X509Req()
# Return an X509Name object representing the subject of the certificate.
req.get_subject().countryName = country
req.get_subject().stateOrProvinceName = state
req.get_subject().localityName = location
req.get_subject().organizationName = organisation
req.get_subject().organizationalUnitName = organisational_unit
req.get_subject().CN = nodename
# Add in extensions
#base_constraints = ([
# crypto.X509Extension("keyUsage", False, "Digital Signature, Non Repudiation, Key Encipherment"),
# crypto.X509Extension("basicConstraints", False, "CA:FALSE"),
#])
#x509_extensions = ([])
x509_extensions = []
# If there are SAN entries, append the base_constraints to include them.
if ss:
san_constraint = crypto.X509Extension("subjectAltName", False, ss)
x509_extensions.append(san_constraint)
req.add_extensions(x509_extensions)
# Set the public key of the certificate to pkey.
req.set_pubkey(key)
# Sign the certificate, using the key pkey and the message digest algorithm identified by the string digest.
req.sign(key, "sha1")
# Dump the certificate request req into a buffer string encoded with the type type.
if os.path.exists(_csrfile):
print "Certificate file exists, aborting."
print " ", _csrfile
sys.exit(1)
else:
f = open(_csrfile, "w")
f.write(crypto.dump_certificate_request(crypto.FILETYPE_PEM, req))
f.close()
The error that I get back from the IIS CA is:
ASN1 bad tag value met. 0x8009310b (ASN: 267)
According to Microsoft this is caused by:
This behavior occurs when certificate request is stored in a file in Unicode encoding. Microsoft Certificate Services do not support Unicode-encoded files request files. Only ANSI encoding is supported.
I know that if I generate a CSR from openssl on the command line it is accepted and issued by the IIS CA RESTful webservice without error.
I want to know if there is some way I can generate 'ANSI' encoded files from pyOpenSSL - I am not sure if it is the keyfile or the CSR that is signed with the keyfile that is causing the issues.
I have solved it with the help of this stackoverflow question thanks to #yodatg.
The problem occurs due to a bug in pyOpenSSL that has been fixed.
By issuing:
openssl asn1parse -in certificates/cert.csr
I could see the ASN1 value:
8:d=2 hl=2 l= 1 prim: INTEGER :01
In a working CSR it looks like this:
8:d=2 hl=2 l= 1 prim: INTEGER :00
I then changed my code to include a set_version call on the req object prior to signing:
#set version - IIS CA required this
req.set_version(0)
# Set the public key of the certificate to pkey.
req.set_pubkey(priv_key)
This is now resolved.
Related
It's unclear from the docs what you actually do to verify the jwsRepresentation string from a StoreKit 2 transaction on the server side.
Also "signedPayload" from the Apple App Store Notifications V2 seems to be the same, but there is also no documentation around actually validating that either outside of validating it client side on device.
What gives? What do we do with this JWS/JWT?
(DISCLAIMER: I am a crypto novice so check me on this if I'm using the wrong terms, etc. throughout)
The JWS in jwsRepresentation, and the signedPayload in the Notification V2 JSON body, are JWTs — you can take one and check it out at jwt.io. The job is to validate the JWT signature and extract the payload once you're sufficiently convinced it's really from Apple. Then the payload itself contains information you can use to upgrade the user's account/etc. server side once the data is trusted.
To validate the JWT, you need to find the signature that the JWT is signed with, specified in the JWT header's "x5c" collection, validate the certificate chain, and then validate that the signature is really from Apple.
STEP ONE: Load the well-known root & intermediate certs from Apple.
import requests
from OpenSSL import crypto
ROOT_CER_URL = "https://www.apple.com/certificateauthority/AppleRootCA-G3.cer"
G6_CER_URL = "https://www.apple.com/certificateauthority/AppleWWDRCAG6.cer"
root_cert_bytes: bytes = requests.get(ROOT_CER_URL).content
root_cert = crypto.load_certificate(crypto.FILETYPE_ASN1, root_cert_bytes)
g6_cert_bytes: bytes = requests.get(G6_CER_URL).content
g6_cert = crypto.load_certificate(crypto.FILETYPE_ASN1, g6_cert_bytes)
STEP TWO: Get certificate chain out of the JWT header
import jwt # PyJWT library
# Get the signing keys out of the JWT header. The header will look like:
# {"alg": "ES256", "x5c": ["...base64 cert...", "...base64 cert..."]}
header = jwt.get_unverified_header(apple_jwt_string)
provided_certificates: List[crypto.X509] = []
for cert_base64 in header['x5c']:
cert_bytes = base64url_decode(cert_base64)
cert = crypto.load_certificate(crypto.FILETYPE_ASN1, cert_bytes)
provided_certificates.append(cert)
STEP THREE: Validate the chain is what you think it is -- this ensures the cert chain is signed by the real Apple root & intermediate certs.
# First make sure these are the root & intermediate certs from Apple:
assert provided_certificates[-2].digest('sha256') == g6_cert.digest('sha256')
assert provided_certificates[-1].digest('sha256') == root_cert.digest('sha256')
# Now validate that the cert chain is cryptographically legit:
store = crypto.X509Store()
store.add_cert(root_cert)
store.add_cert(g6_cert)
for cert in provided_certificates[:-2]:
try:
crypto.X509StoreContext(store, cert).verify_certificate()
except crypto.X509StoreContextError:
logging.error("Invalid certificate chain in JWT: %s", apple_jwt)
return None
store.add_cert(cert)
FINALLY: Load & validate the JWT using the now-trusted certificate in the header.
# Now that the cert is validated, we can use it to verify the actual signature
# of the JWT. PyJWT does not understand this certificate if we pass it in, so
# we have to get the cryptography library's version of the same key:
cryptography_version_of_key = provided_certificates[0].get_pubkey().to_cryptography_key()
try:
return jwt.decode(apple_jwt, cryptography_version_of_key, algorithms=["ES256"])
except Exception:
logging.exception("Problem validating Apple JWT")
return None
Voila you now have a validated JWT body from the App Store at your disposal.
Gist of entire solution: https://gist.github.com/taylorhughes/3968575b40dd97f851f35892931ebf3e
I'm writting a little program using SSL sockets. A client sends values to a server and when the server gets the value it checks the client's public key to make sure he's expected to send something. So at first the server is getting all the public keys like this :
cert = f.read()
crtObj = crypto.load_certificate(crypto.FILETYPE_PEM, cert)
pubKeyObject = crtObj.get_pubkey()
pubKeyString = crypto.dump_publickey(crypto.FILETYPE_PEM,pubKeyObject)
with this method, the public key is :
b'-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA7GIzek5JgfFzFCGwnx7X\ncE4QULV/9uyoGgd9HbHYyYcItEcSPU39ORXCrNQGxh09k4oFPBYntjD2gIORF8V4\n6EAC10bFaT18OuM1F/37v+K/+BuvCDTqcS9Y0CRalwPFVYB+yttvZ8fnvO2l/TxF\nsLmZh0yY/ajaHxey/ppUQycGy4xA8XD6VlWFM7+I0t/19rrLN9iMFSym/TgYpBbn\nxyZel8rMW/ACS09nSprEu1BuI+myhhej+cuy3wU8byRTwANpqHxsx5cTwp642TVx\nBKbuO8GHAzEKcrFZnrKcsXr9emWV5ouYiVzehOT4Pd3I2W8qSy6x/Ovv/iS3ojT4\ndQIDAQAB\n-----END PUBLIC KEY-----\n'
and when the server wants to get it from the socket connection like this :
test1 = writer.get_extra_info('ssl_object')
der =test1.getpeercert(binary_form=True)
test = test1.getpeercert()
cert = x509.Certificate.load(der)
pubkey= cert.public_key.unwrap()
print(pem.armor("PUBLIC KEY", pubkey.contents).decode("ASCII"))
the public key printed is :
-----BEGIN PUBLIC KEY-----
AoIBAQDsYjN6TkmB8XMUIbCfHtdwThBQtX/27KgaB30dsdjJhwi0RxI9Tf05FcKs
1AbGHT2TigU8Fie2MPaAg5EXxXjoQALXRsVpPXw64zUX/fu/4r/4G68INOpxL1jQ
JFqXA8VVgH7K229nx+e87aX9PEWwuZmHTJj9qNofF7L+mlRDJwbLjEDxcPpWVYUz
v4jS3/X2uss32IwVLKb9OBikFufHJl6Xysxb8AJLT2dKmsS7UG4j6bKGF6P5y7Lf
BTxvJFPAA2mofGzHlxPCnrjZNXEEpu47wYcDMQpysVmespyxev16ZZXmi5iJXN6E
5Pg93cjZbypLLrH86+/+JLeiNPh1AgMBAAE=
-----END PUBLIC KEY-----
So I don't know if it's a matter of format or if it's really not the same public key I'm getting... but it should be.
Sorry for the long post and thank you very much for reading.
Most(if not all) base64 encoding variants use A-Z, a-z, 0-9 characters to represent data the total is 62 character and some base64 encodings varies in the last 2 characters but most of them use + and / which will complete 64. So the above 2 keys seems to be completely different and the second key is also padded because of = sign at end. The standard base64 encoding is RFC 4648 variant which use + and / and make padding as optional not mandatory like RFC 4880 openpgp base64 encoding.
It was in fact the same keys but the way of getting it was wrong : here the right one from the ssl object :
test1 = writer.get_extra_info('ssl_object')
der =test1.getpeercert(binary_form=True)
crtObj = crypto.load_certificate(crypto.FILETYPE_ASN1, der)
pubKeyObject = crtObj.get_pubkey()
pubKeyString = crypto.dump_publickey(crypto.FILETYPE_PEM, pubKeyObject)
I try to use pyOpenSSL for signed a data, I create key pair (private and publique) and certificate.
I'm a beginner with this technology, I use OpenSSl, but if you have suggestions for generate a signed message with private and public key in python, I'm take !
I want to use RSA and DSA algorithm for my tests.
I find m2Crypto, pyCrypto and other. I do not know what is the best for this.
gnupg for python and pyOpenSSl are more recent visibly.
I used function for signed a message with my private key, and I verify the data.
But when I see the function for verify the signature, in parameters I need :
private key, signature, data and digest type.
I do not know where I am wrong in this code, I find some examples, but I do not understand how this can work because the first parameters for the verify function is a X509 object "certificate is a X509 instance corresponding to the private key which generated the signature." and the second is the signature generated with the private key..
This code work perfectly with the private key :
from OpenSSL import crypto
_k = crypto.PKey()
_cert = crypto.X509()
# Create keys
_k.generate_key(crypto.TYPE_RSA, 2048)
# Add argument for create certificate
_cert.gmtime_adj_notBefore(0)
_cert.gmtime_adj_notAfter(0*365*24*60*60) #10 years expiry date
_cert.set_pubkey(_k)
_cert.sign(_k, 'sha256')
# Create key's file
with open("public_key.pem",'w') as f:
f.write(crypto.dump_publickey(crypto.FILETYPE_PEM, _k))
with open("private_key.pem",'w') as f:
f.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, _k))
with open("certificate.pem",'w') as f:
f.write(crypto.dump_certificate(crypto.FILETYPE_PEM, _cert))
#-------------------------------------------------------------------------------
# Open key and load in var
with open("private_key.pem",'r') as f:
priv_key = crypto.load_privatekey(crypto.FILETYPE_PEM, f.read())
with open("public_key.pem",'r') as f:
pub_key = crypto.load_publickey(crypto.FILETYPE_PEM, f.read())
with open("certificate.pem",'r') as f:
cert = crypto.load_certificate(crypto.FILETYPE_PEM, f.read())
# sign message 'hello world' with private key and certificate
sign = crypto.sign(priv_key, "hello world", 'sha256')
print crypto.verify(cert, sign, "hello world", 'sha256')
So, my question is, how use the public key for verify the data ?
If Bob give a public key to alice, How it checks the message with this public key ?
You have a idea ?
Thanks a lot,
Romain
I found a answer in this post.
from OpenSSL.crypto import load_publickey, FILETYPE_PEM, verify, X509
# ... code ...
x509 = X509()
x509.set_pubkey(pub_key)
# ... code ...
print verify(x509, sign, sha, 'sha256')
I'm wondering.. When I get a message, I use the public key for authenticating the transmitter, but why I have to use the signature to do the verification ? To generate the signature, I need the private key ..
Is there something I did not understand in the use of the library ?
ok I understood my mistake : Cryptography Digital signatures
source
References:
https://github.com/bitcoin/bips/blob/master/bip-0070.mediawiki
https://github.com/aantonop/bitcoinbook/blob/develop/selected%20BIPs/bip-0070.mediawiki#paymentdetailspaymentrequest
message PaymentRequest {
##optional uint32 payment_details_version = 1 [default = 1]; # 'x509+sha256' in this case.
##optional string pki_type = 2 [default = "none"];
optional bytes pki_data = 3;
##required bytes serialized_payment_details = 4;
optional bytes signature = 5;
}
The ones with ## at the front are not a problem, I've solved them already.
optional bytes pki_data wants a byte encoded version of 'x509+sha256' so...
x509_bytes = open('/path/to/x509.der', 'rb').read()
pki_data = hashib.sha256(x509_bytes)
Is the above correct?
Next optional bytes signature, 'digital signature over a hash of the protocol buffer serialized variation of the PaymentRequest message'
I'm not sure how to achieve this so any suggestions would be greatly appreciated.
Finally I have...
message X509Certificates {
repeated bytes certificate = 1;
}
repeated bytes certificate 'Each certificate is a DER [ITU.X690.1994] PKIX certificate value. The certificate containing the public key of the entity that digitally signed the PaymentRequest MUST be the first certificate.'
I only have the one cert I got from the comodo root authority so I think I only need to supply the raw byte data of the cert to satisfy this one which already exists in the form of x509_bytes above, so...
repeated bytes certificate = x509_bytes
Am I close??
Also I notice that repeated bytes certificate comes after optional bytes signature but shouldn't I deal with that before message PaymentRequest so that I can serialise it into my http response somehow?
EDIT:
For what it's worth I'm aware that I need to import, instantiate and in some cases serialise these methods before sending them as a request/response but what I'm looking for are the methods on how to manipulate and supply the information required.
Thanks :)
To add PKI data to the PaymentRequest object:
pki_data = X509Certificates()
certificates = [your_cert_der_data, root_cert_der_data]
for cert in certificates:
pki_data.certificate.append(cert)
request.pki_data = pki_data.SerializeToString()
To add a signature (with pycrypto):
from Crypto.Hash import SHA256
from Crypto.Signature import PKCS1_v1_5
from Crypto.PublicKey import RSA
# At this moment request object must contain serialized_payment_details, pki_type and pki_data
request.signature = "" # Add empty signature
request_hash = SHA256.new(request.SerializeToString())
private_key = RSA.importKey(private_key_der_data)
signer = PKCS1_v1_5.new(private_key)
request.signature = signer.sign(request_hash)
result = request.SerializeToString()
I'm writing a script that connects to a bunch of URLs over HTTPS, downloads their SSL certificate and extracts the CN. Everything works except when I stumble on a site with an invalid SSL certificate. I absolutely do not care if the certificate is valid or not. I just want the CN but Python stubbornly refuses to extract the certificate information if the certificate is not validated. Is there any way to get around this profoundly stupid behavior? Oh, I'm using the built-in socket and ssl libraries only. I don't want to use third-party libraries like M2Crypto or pyOpenSSL because I'm trying to keep the script as portable as possible.
Here's the relevant code:
file = open("list.txt", "r")
for x in file:
server = socket.getaddrinfo(x.rstrip(), "443")[0][4][0]
sslsocket = socket.socket()
sslsocket.connect((server, 443))
sslsocket = ssl.wrap_socket(sslsocket, cert_reqs=ssl.CERT_REQUIRED, ca_certs="cacerts.txt")
certificate = sslsocket.getpeercert()`
The ssl.get_server_certificate can do it:
import ssl
ssl.get_server_certificate(("www.sefaz.ce.gov.br",443))
I think function doc string is more clear than python doc site:
"""Retrieve the certificate from the server at the specified address,
and return it as a PEM-encoded string.
If 'ca_certs' is specified, validate the server cert against it.
If 'ssl_version' is specified, use it in the connection attempt."""
So you can extract common name from binary DER certificate searching for common name object identifier:
def get_commonname(host,port=443):
oid='\x06\x03U\x04\x03' # Object Identifier 2.5.4.3 (COMMON NAME)
pem=ssl.get_server_certificate((host,port))
der=ssl.PEM_cert_to_DER_cert(pem)
i=der.find(oid) # find first common name (certificate authority)
if i!=-1:
i=der.find(oid,i+1) # skip and find second common name
if i!=-1:
begin=i+len(oid)+2
end=begin+ord(der[begin-1])
return der[begin:end]
return None
Ok. I cleaned up olivecoder's code to solve the problem that it assumes there will always be three CNs in the certificate chain (root, intermediate, server) and I condensed it. This is the final code I will be using.
cert = ssl.get_server_certificate(("www.google.com", 443)) #Retrieve SSL server certificate
cert = ssl.PEM_cert_to_DER_cert(cert) #Convert certificate to DER format
begin = cert.rfind('\x06\x03\x55\x04\x03') + 7 #Find the last occurence of this byte string indicating the CN, add 7 bytes to startpoint to account for length of byte string and padding
end = begin + ord(cert[begin - 1]) #Set endpoint to startpoint + the length of the CN
print cert[begin:end] #Retrieve the CN from the DER encoded certificate