I need to verify an xml signature contained in the answer to a POST request.
The signature is defined by:
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
<SignedInfo>
<CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
<SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
<Reference URI="">
<Transforms>
<Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
<Transform Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
</Transforms>
<DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
<DigestValue>htti3M3ikfm2RooDTNo3Kv7g0K2ongShUfCDUAWpytc=</DigestValue>
</Reference>
</SignedInfo>
and is against a custom CA.
I have no way to change the answer (it is issued by a State Agency) and all my attempts to verify against the certificate supposedly used to sign did result in errors.
One of my attempts has been using the following short test program:
#!/usr/bin/python3
import signxml
cf = 'certificate.cer'
with open(cf, 'r') as fi:
cer = fi.read()
ver = signxml.XMLVerifier()
f = 'response.xml'
with open(f, 'rb') as fi:
xml = fi.read()
try:
vd = ver.verify(xml, x509_cert=cer)
print('OK')
except signxml.exceptions.InvalidSignature as e:
print(e)
This results in:
Signature verification failed: wrong signature length
Other variations have different errors including:
Signature verification failed: invalid padding
and:
unable to get local issuer certificate
I am a seasoned programmer, but NOT a cryptography expert, so it's quite likely I forgot something trivial (to the knowledgeable).
Please point me in the right direction.
Note:: if required I can provide a full example (of failure) as certificate/answer are not "secret".
Unfortunately things are always a bit more complex than expected.
Answer from #stovfl completely missed the relevant point: I need to verify against a non-standard CA.
I already was struggling to use sigxml package, and I had to overcome the following problems:
sigxml will not work (for me) with plain xml.etree; I had to use lxml.etree with a different syntax.
sigxml installed by plain pip3 install sigxml would bomb with a deprecation error; I had to get latest (non-tagged master) from github with pip3 install git+https://github.com/XML-Security/signxml.git.
Examples on sigxml site completely disregard python3 issues with str vs. bytes.
CA Authority used to sign the Certificate I used to sign the outgoing message is different from the CA Authority used to sign response (I have no way to change this!).
This story has an Happy Ending though.
The following test program works (for me):
from signxml import XMLSigner, XMLVerifier, InvalidCertificate
from lxml import etree
outCAroot = 'outCAroot.pem'
inCAroot = 'inCAroot.pem'
cert = open("example.pem").read().encode()
key = open("example.key").read().encode()
file_name = 'test.tosend'
resp_name = 'test.rsp'
xml = etree.parse(file_name) # (data_to_sign)
signer = XMLSigner(c14n_algorithm='http://www.w3.org/TR/2001/REC-xml-c14n-20010315')
signed_xml = signer.sign(xml, key=key, cert=cert)
try:
result = XMLVerifier().verify(signed_xml, ca_pem_file=outCAroot)
except InvalidCertificate as e:
print(e)
else:
print('outgoing signature Ok.')
# here I send signed_xml to remote server and get the response (NO ERRORS!)
answer_xml = etree.parse(resp_name) # (signed answer)
try:
result = XMLVerifier().verify(answer_xml, ca_pem_file=inCAroot)
except InvalidCertificate as e:
print(e)
else:
print('incoming signature Ok.')
print('===================')
print(result.signed_data.decode())
print('===================')
I hope this will help whoever is (or will be) in my situation.
Question: I need to verify an xml signature contained in the answer to a POST request.
Do it, like the example in the Documentation SignXML: XML Signature in Python:
SignXML uses the ElementTree API (also supported by lxml) to work with XML data.
from signxml import XMLSigner, XMLVerifier
from xml.etree import ElementTree
cert = open("example.pem").read()
key = open("example.key").read()
xml = ElementTree.parse(file_name) #(data_to_sign)
signed_xml = XMLSigner().sign(xml, key=key, cert=cert)
result = XMLVerifier().verify(signed_xml)
XMLVerifier().verify(...)
Verify the XML signature supplied in the data and return the XML node signed by the signature, or raise an exception if the signature is not valid.
class signxml.VerifyResult
The results of a verification return the signed data, the signed xml and the signature xml
Note: Necessarily read about See what is signed and Establish trust!
Relevant: Python elementtree find function reads Signature as empty (None)
Related
I am trying to compare a signature with a certificate for a pdf file in python.
I found this very nice package called endesive.
I followed the example for verifying a pdf signature and I have something like this:
pdf_file_path = "/workspaces/test.pdf"
data = open(pdf_file_path, 'rb').read()
certificates = (
open("/workspaces/certificates/pki.pem", 'rt').read(),
open("/workspaces/certificates/pki-chain.pem", 'rt').read()
)
(hashok, signatureok, certok) = pdf.verify(data, certificates)
print('signature ok?', signatureok)
print('hash ok?', hashok)
print('cert ok?', certok)
This should be pretty straight forward. I read the pdf, I open the certificates and then I 'pdf.verify' to see that everything is in order.
pdf.verify, at one point calls this: signed_data = cms.ContentInfo.load(bcontents)['content'].native which makes ans1crypto raise this error File "/home/vscode/.local/lib/python3.9/site-packages/asn1crypto/core.py", line 4060, in native raise e repeatedly until it gets to
ValueError: Unknown element - context class, constructed method, tag 0
while parsing asn1crypto.core.Sequence
while parsing asn1crypto.cms.SetOfAny
while parsing asn1crypto.cms.CMSAttribute
while parsing asn1crypto.cms.CMSAttributes
while parsing asn1crypto.cms.SignerInfo
What could go wrong here?
Instead of addressing signer data info like this:
signature = signed_data['signer_infos'][0].native['signature']
It should have been addressed like this:
signature = signed_data['signer_infos'][0]['signature'].native
This has been addressed here.
Following advice from Jean-Paul Calderone here on SO, I'm trying to modify the twisted "starttls_server" sample below to support the use of ssl.ClientCertificateOptions, to allow me to specify my private key, certificate, and trusted roots, as per http://twistedmatrix.com/documents/14.0.0/api/twisted.internet.ssl.CertificateOptions.html
from twisted.internet import ssl, protocol, defer, task, endpoints
from twisted.protocols.basic import LineReceiver
from twisted.python.modules import getModule
class TLSServer(LineReceiver):
def lineReceived(self, line):
print("received: " + line)
if line == "STARTTLS":
print("-- Switching to TLS")
self.sendLine('READY')
self.transport.startTLS(self.factory.options)
def main(reactor):
certData = getModule(__name__).filePath.sibling('server.pem').getContent()
cert = ssl.PrivateCertificate.loadPEM(certData)
factory = protocol.Factory.forProtocol(TLSServer)
factory.options = cert.options()
endpoint = endpoints.TCP4ServerEndpoint(reactor, 8000)
endpoint.listen(factory)
return defer.Deferred()
if __name__ == '__main__':
import starttls_server
task.react(starttls_server.main)
My understanding is that I effectively need to replace the cert = ssl.PrivateCertificate... and cert.options = ssl.PrivateCertificate.... lines with something like certopts = ssl.CertificateOptions(privateKey=pKeyData, certificate=certData, trustRoot=caCertsData) (having read the appropriate files in to certData, caCertsData, and pKeyData) and then pass this in to factory.options - but without pasting every variant of code I've tried, I've yet to work this out correctly - my efforts have produced varying results from the classic "OpenSSL.crypto.Error: []" - through to seemingly just dumping the contents of my 3 PEM files to screen and exiting!
Can anyone enlighten me? Thank you :)
cert.options() is already returning a CertificateOptions. The problem is that options takes authorities (as Certificate objects) as positional args, and doesn't let you pass through all the other configuration values through, so you probably want to construct a CertificateOptions directly.
Just change the factory.options = cert.options() line to factory.options = ssl.CertificateOptions(...).
However, CertificateOptions takes a pyOpenSSL PKey object as its privateKey, not the key data. So you'll need to use OpenSSL APIs to load that key, or you can extract it from a PrivateCertificate.
If you read the signature of CertificateOptions very carefully, the required types should be fairly clear. You may also need to consult the pyOpenSSL documentation as well.
I want to verify Windows Mobile InApp Receipt with public key I have certificate (.cer) from that I got .pem file and from that I got public key Here is my code :
import dm.xmlsec.binding as xmlsec
xmlsec.initialize()
from os.path import dirname, basename
from lxml import etree
from lxml.etree import tostring
def verify_file(xml_file, key_file):
"""verify signature in *xml_file* with key in *key_file*.
*xml_file* contains the signed XML document.
It can be a file, a filename string or an HTTP/FTP url.
*key_file* contains the PEM public key. It must be a filename.
An exception is raised when the verification fails.
"""
from lxml.etree import parse
doc = etree.parse(xml_file)
node = doc.find(".//{%s}Signature" % xmlsec.DSigNs)
dsigCtx = xmlsec.DSigCtx()
# Note: we do not provide read access to `dsigCtx.signKey`.
# Therefore, unlike the `xmlsec` example, we must set the key name
# before we assign it to `dsigCtx`
signKey = xmlsec.Key.load(key_file, xmlsec.KeyDataFormatPem, None)
signKey.name = basename(key_file)
# Note: the assignment below effectively copies the key
dsigCtx.signKey = signKey
dsigCtx.verify(node)
from StringIO import StringIO
if __name__ =="__main__" :
verify_file('path/of/ReceiptSHA256.xml', 'path/of/pubkey.pem')
But problem is giving me error each time when i run it dsigCtx.verify(node):
File "_xmlsec.pyx", line 369, in dm.xmlsec.binding._xmlsec.DSigCtx.verify (src/_xmlsec.c:5182)
dm.xmlsec.binding._xmlsec.VerificationError: ('signature verification failed', 2)
and when I debug it then i found actualy dsigCtx.verify(node) gives me error tuple: ('verifying failed with return value', -1) I have valid certificate, .pem file, public key and receipt but still it is giving me error
Please Help me in this Regards,
I need to implement authentication in python from a 3rd party by using SAML2. I have looked into pysaml2 and found that to be quite confusing, and decided to give M2Crypto a chance after I found this question by Ennael.
The SAML token I receive can be found here. I have already extracted all the information I need from the Assertion tag (the user's SSN, IP and the SAML tokens expiration window) but I can't get the verify_signature function from Ennael (and the revised code from Ezra Nugroho) to return True. I have also tried to change verify_EVP.reset_context(md='sha1') to verify_EVP.reset_context(md='sha256') but that didn't work either.
I think my mistake must be in the signed_info part. What do I pass to verify_signature for that part? Do I have to preprocess it in any way? I have been looking into the Transform tag but don't know where too look next.
Any help will be greatly appreciated. If someone needs the XML before the obfuscation to test and help me just PM me.
EDIT This is my code (very similar to the things i linked to. The main function is at the bottom):
def verify_signature(signed_info, cert, signature):
from M2Crypto import EVP, RSA, X509, m2
x509 = X509.load_cert_string(base64.decodestring(cert), X509.FORMAT_DER)
pubkey = x509.get_pubkey().get_rsa()
verify_EVP = EVP.PKey()
verify_EVP.assign_rsa(pubkey)
verify_EVP.reset_context(md='sha1')
verify_EVP.verify_init()
verify_EVP.verify_update(signed_info)
return verify_EVP.verify_final(signature.decode('base64'))
def decode_response(resp):
return base64.b64decode(resp)
def get_xmldoc(xmlstring):
return XML(xmlstring)
def get_signature(doc):
return doc.find('{http://www.w3.org/2000/09/xmldsig#}Signature')
def get_signed_info(signature):
signed_info = signature.find(
'{http://www.w3.org/2000/09/xmldsig#}SignedInfo')
signed_info_str = tostring(signed_info)
# return parse(StringIO(signed_info_str))
return signed_info_str
def get_cert(signature):
ns = '{http://www.w3.org/2000/09/xmldsig#}'
keyinfo = signature.find('{}KeyInfo'.format(ns))
keydata = keyinfo.find('{}X509Data'.format(ns))
certelem = keydata.find('{}X509Certificate'.format(ns))
return certelem.text
def get_signature_value(signature):
return signature.find(
'{http://www.w3.org/2000/09/xmldsig#}SignatureValue').text
def parse_saml(saml):
dec_resp = decode_response(saml)
xml = get_xmldoc(dec_resp)
signature = get_signature(xml)
signed_info = get_signed_info(signature)
cert = get_cert(signature)
signature_value = get_signature_value(signature)
is_valid = verify_signature(signed_info, cert, signature_value)
UPDATE: Is it possible I need some more information from the 3rd party authentication provider? Do I need a private key for any of this?
I faced the same problem, and had to develop a module for it: https://github.com/kislyuk/signxml. I chose to rely only on PyCrypto and pyOpenSSL, since M2Crypto is less popular and not well-maintained, which is a hazard from both compatibility (e.g. PyPy) and security perspectives. I also use lxml for the canonicalization (c14n). From the signxml docs:
from signxml import xmldsig
cert = open("example.pem").read()
key = open("example.key").read()
root = ElementTree.fromstring(data)
xmldsig(root).verify()
You need to canonicalize the signed info before validating the signature. That's what the transformation tag implies. Basically, since the same XML can be formatted differently, one needs to validate an XML signature in a canonical format.
I'm attempting to use M2Crypto to verify a signature contained in an XML response returned from my SSO/SAML provider in my django/python app, but I can't seem to get it to work.
My XML response looks sort of like the second example here.
ETA: And here's a pastebin of my actual XML.
I'm using some code like this to attempt the verification:
def verify_signature(signed_info, cert, signature):
from M2Crypto import EVP, RSA, X509
x509 = X509.load_cert_string(base64.decodestring(cert), X509.FORMAT_DER)
pubkey = x509.get_pubkey().get_rsa()
verify_EVP = EVP.PKey()
verify_EVP.assign_rsa(pubkey)
verify_EVP.reset_context(md='sha1')
verify_EVP.verify_init()
verify_EVP.verify_update(signature.decode('base64'))
result = verify_EVP.verify_final(signed_info)
return result
I can successfully get the NameID from the response, and I know I'm successfully loading the certificate, because I can pull the issuer, etc. out of it.
As for the signature, though, I've tried hashing the passed in XML, encoding/not encoding various pieces, and passing in various bits of XML for the signed_info parameter (the SignedInfo tag, the Response tag, the whole thing), and I've tried using ElementTree/ElementC14N.py to ensure the XML is exclusively canonicalized, as the Transform implies should be done, but I'm not getting a positive result.
What am I missing here? Am I trying to validate against the wrong XML? Something wrong with my verification technique?
You were so close! You should pass to verify_update the signed_info, and then to verify_final pass the signature.
You do need to make sure that your signed_info is correctly canonicalized before verifying the signature.
Here is the correct method:
def verify_signature(signed_info, cert, signature):
from M2Crypto import EVP, RSA, X509
x509 = X509.load_cert_string(base64.decodestring(cert), X509.FORMAT_DER)
pubkey = x509.get_pubkey().get_rsa()
verify_EVP = EVP.PKey()
verify_EVP.assign_rsa(pubkey)
verify_EVP.reset_context(md='sha1')
verify_EVP.verify_init()
verify_EVP.verify_update(signed_info)
result = verify_EVP.verify_final(signature.decode('base64'))
return result
FYI, I was facing the same problem as you, and found no useful software for validating XML signatures in Python, so I wrote a new library: https://github.com/kislyuk/signxml.
from lxml import etree
from signxml import xmldsig
with open("saml2_idp_metadata.xml", "rb") as fh:
cert = etree.parse(fh).find("//ds:X509Certificate").text
root = ElementTree.fromstring(signature_data)
xmldsig(root).verify(x509_cert=cert)