I have the following openssl command which gets the URL from a CRL from a certificate.
openssl x509 -noout -text -in cert.pem | grep -A 4 'X509v3 CRL Distribution Points'"
Instead of an input ofx509 certificate, can I give a string with ----Beginn Certificate,... as input?
I did not get any result of my trys. So pls help me :=)
Related
I am trying to fetch the certificate issuer name from a pem file. But I am getting an error.
ValueError: Unable to load certificate. See https://cryptography.io/en/latest/faq/#why-can-t-i-import-my-pem-file for more details.
Here are the commands which I used to create the keys.
openssl ecparam -name secp256k1 -genkey -noout -out PrivateKey.pem
openssl ec -in PrivateKey.pem -pubout -conv_form compressed -out PublicKey.pem
And when I executing the below command,
openssl x509 -noout -text -in PublicKey.pem
getting the error:
unable to load certificate
140351154677056:error:0909006C:PEM routines:get_name:no start line:../crypto/pem/pem_lib.c:745:Expecting: TRUSTED CERTIFICATE
And here is the my python code to fetch issuer name,
*.py
with open('PublicKey.pem','r') as pub_key:
pubkey = pub_key.read().encode()
cert_info = x509.load_pem_x509_certificate(pubkey, default_backend())
print('cert_info',cert_info)
print('cn',cn)
cn = cert_info.issuer.get_attributes_for_oid(NameOID.COMMON_NAME)[0].value
print('cn------',cn)
Am getting the error Unable to load certificate.
I also need to fetch the serial number.
How can i solve this?
I have 2 certificate files, truststore.jks and keystore.jks. keystore.jks contains a full certificate chain for the kafka endpoint I'm using as well as a private key for my application. According to the group who gave me truststore.jks it contains "a new root CA chain" and is essential for getting my application to connect with the kafka endpoint.
I have tried using this script to generate 3 .pem files that I use in my Python code.
#!/bin/bash
srcFolder=$1
keyStore=$1/$2
password=$3
alias=$4
outputFolder=$5
echo $keyStore
echo "Generating certificate.pem"
keytool -exportcert -alias $alias -keystore $keyStore -rfc -file $outputFolder/certificate.pem -storepass $password
echo "Generating key.pem"
keytool -v -importkeystore -srckeystore $keyStore -srcalias $alias -destkeystore $outputFolder/cert_and_key.p12 -deststoretype PKCS12 -storepass $password -srcstorepass $password
openssl pkcs12 -in $outputFolder/cert_and_key.p12 -nodes -nocerts -out $outputFolder/key.pem -passin pass:$password
echo "Generating CARoot.pem"
keytool -exportcert -alias $alias -keystore $keyStore -rfc -file $outputFolder/CARoot.pem -storepass $password
I use the generated CARoot.pem, certificate.pem, and key.pem files in my code like so.
>>> from kafka import KafkaProducer
>>> producer = KafkaProducer(bootstrap_servers='kafka-dev.kafka.com:port', security_protocol="SSL",
... ssl_check_hostname=True,
... ssl_cafile="CARoot.pem",
... ssl_certfile="certificate.pem",
... ssl_keyfile="key.pem",
... ssl_password="")
It gives this error:
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1131)
I have tried combining truststore.jks and keystore.jks like so:
keytool -importkeystore -srckeystore truststore.jks -destkeystore keystore_copy.jks
I then put keystore_copy.jks through the same .pem conversion process as before, but it produces the same error.
For some context, this is how the Kafka endpoint owners suggest to configure the ssl certs in Java, unfortunately I am the first to connect with Python so they haven't been able to help me with it.
props.setProperty(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SSL");
props.setProperty(SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG, "<kafka-truststore.jks>");
props.setProperty(SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG, "<password>");
props.setProperty(SslConfigs.SSL_KEYSTORE_LOCATION_CONFIG, "<client-keystore.jks>");
props.setProperty(SslConfigs.SSL_KEYSTORE_PASSWORD_CONFIG, "<password>");
props.setProperty(SslConfigs.SSL_KEY_PASSWORD_CONFIG, "<password>");
Is there something I'm missing? I believe I need to utilize both .jks files in my code somehow but the only approach I can think of taking failed.
Additional things I have tried:
pip install --upgrade certifi based on reports of the same error due to that library being out of date.
The following properties which KAFKA endpoint owners are asked you is correct, but looks like the issue with your keystore/truststore having a self signed certificate. It appears ssl.verify or hostName verification is enabled at the server level.
props.setProperty(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SSL");
props.setProperty(SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG, "<kafka-truststore.jks>");
props.setProperty(SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG, "<password>");
props.setProperty(SslConfigs.SSL_KEYSTORE_LOCATION_CONFIG, "<client-keystore.jks>");
props.setProperty(SslConfigs.SSL_KEYSTORE_PASSWORD_CONFIG, "<password>");
props.setProperty(SslConfigs.SSL_KEY_PASSWORD_CONFIG, "<password>");
This is one give you better clarity on this error.
https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/sheng-jie-han/2021/06/03/how-to-fix-certificate-verify-failed-self-signed-c
I managed to figure out the solution on my own. The key here was realizing that I needed to have the entire cert chain from my truststore.jks file present in CARoot.pem for the code to work. Here are the steps that I took:
Generate CARoot.pem from the trust store:
keytool -importkeystore -srckeystore <<truststore-name.jks>> -destkeystore keystore.p12 -srcstoretype jks -deststoretype pkcs12
openssl pkcs12 -in keystore.p12 -out CARoot.pem
Generate certificate.pem from keystore
keytool -exportcert -alias <<alias>> -keystore ./<<keystore—name.jks>> -rfc -file ./certificate.pem -storepass <<password>>
Extract private key and generate RSAkey.pem from keystore:
keytool -importkeystore -srckeystore <<keystore-name.jks>> -destkeystore keystore.p12 -deststoretype PKCS12
openssl pkcs12 -in keystore.p12 -nodes -nocerts -out RSAkey.pem
Example with Kafka-Python Producer:
producer = KafkaProducer(bootstrap_servers='kafka-dev.kafka.com:port', security_protocol="SSL",
ssl_check_hostname=False,
ssl_cafile="CARoot.pem",
ssl_certfile="certificate.pem",
ssl_keyfile="RSAkey.pem",
ssl_password=<<password>>)
I have some Ansible playbooks I want to run against some Windows hosts. I've followed the various Ansible guides for setting up WinRM and they have worked fine, but the default setups are very insecure and I want something more production ready. However, the instructions for how to do this are incredibly sparse. So far I've done the following:
On my Windows box:
Enabled WinRM using the supplied ConfigureRemotingForAnsible.ps1 script
Configured target machine to to use a valid cert on HTTPS/5986 rather than the self-signed one generated by the above script
Enabled both Kerberos and CredSSP auth methods on the target machine in WinRM. Some of my role steps require CredSSP to work reliably.
So far so good, the Windows side seems to work fine. However, getting Ansible to connect is proving a nightmare. I can't figure out how to get Ansible to trust the HTTPS cert on the target despite adding it. On my Centos 7 'push box' I've done the following:
Install Ansible and pip with the pywinrm, requests_kerberos and requests_credssp modules
Added my CA certificate to both /etc/pki/tls/certs and /etc/pki/ca-cert
Set my inventory to the following:
ansible_user=ADMINISTRATOR#DOMAIN.COM
ansible_password=Password1
ansible_port=5986
ansible_connection=winrm
ansible_winrm_scheme=https
/#ansible_winrm_server_cert_validation=ignore
ansible_winrm_transport=credssp
With certification validation turned on the connection fails with the following error:
fatal: [host.domain.com]:
UNREACHABLE! => {
"changed": false,
"msg": "credssp: HTTPSConnectionPool(host='host.domain.com', port=5986): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(\"bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)\",),))",
"unreachable": true}
With cert validation turned off it works fine.
So my question: How do I get Ansible to trust my CA cert?
This isn't simple. The problem is there are 3 different groups that don't really want to play:
Ansible: doesn't want to be responsible for OpenSSL or Windows details
OpenSSL: doesn't want to be used as a certficate authority
Microsoft: doesn't want you to use anything but Windows tools
First, you will need 3 sets of OpenSSL certs/keys: the self-signed Certificate Authority (CA), the server winRM HTTPS, and the target user.
There is nothing magic about the CA cert/key; it can be generated with the following:
openssl req -new -out caroot.req -keyout caroot.key -days 730 ...
openssl x509 -req -in caroot.req -extensions v3_ca -signkey caroot.key -out cacert.pem -days 730
There are a myriad of other options to these OpenSSL commands which may (or not) work in your specific environment. This is one of those instances of "don't want to be responsible for OpenSSL".
Second, you will need the WinRM HTTPS cert/key. First, you have to generate the server cert. The key detail of generating the cert is that the 'common name' attribute on the cert MUST match the target hostname, as with any HTTPS cert.
openssl req -new -out server.req -keyout server.key -days 730 -nodes ...
Now, here comes the tricky part. You can't just sign the key as-is; the signing process must add a couple of additional attributes to the cert or it won't work. Create a text file I'll call 'attributes.txt' with the following two lines:
subjectAltName=DNS:hostname.mycompany.com,DNS:hostname
extendedKeyUsage=serverAuth
The first line, the 'subjectAltName', is required or the Python/OpenSSL client will reject the key. You need to substitute the proper values for the target hostname. The second line is required by Windows, or Windows will not use this as for an HTTPS server.
Now sign the key with the CA generated earlier:
openssl x509 -req -in server.req -out server.pem -days 730 -CA caroot.pem -CAkey caroot.key -extfile attributes.txt
Another little wrinkle: Windows will not import a server key unless you combine the key and cert into a PKCS#12 file. This can be done with the following command, which creates a PKCS#12 file with no password:
openssl pkcs12 -export -password pass: -inkey server.key -in server.pem -out windows.pfx
This will generate a file 'windows.pfx' which we'll need later.
Bored yet?
Next we need another key/cert for the user login. This is assuming that we're going to be using a local user on the system, not a domain user. (You're on your own for that.) Generating the key is almost the same as for the server key, except that the 'commom name' attribute (CN) MUST match the target username. (In later usage, I'll call our user 'ansible').
openssl req -new -out winlogin.req -keyout winlogin.key -days 730
This private key is going to be the login key, analogous to an SSH private key. Next, we must sign the req. As with the server key, we need to add some extended attributes, or it won't work. Our 'attributes.txt' file must contain the following lines:
subjectAltName=otherName:1.3.6.1.4.1.311.20.2.3;UTF8:ansible#localhost
extendedKeyUsage=clientAuth
The oddball identifier in the 'altNames' is some proprietary Microsoft thingy, which must contain the 'username#localhost'. As I mentioned before, I'm using a local user 'ansible'. The extendedKeyUsage attribute is required or Windows will not allow the key to be used for user authentication. Finally, we sign the cert:
openssl x509 -req -in winlogin.req -out winlogin.pem -days 730 -CA caroot.pem -CAkey caroot.key -extfile attributes.txt
Just so you know, I'm starting to get on my own nerves here. We're almost there. Finally, copy the CA cert (caroot.pem), the server PKCS#12 cert/key (windows.pfx), and the user cert (winlogin.pem) to the target windows system somewhere. Run the following PowerShell script from the same directory. It will create the local ansible user and import all of the SSL artifacts into their proper target locations. Sorry if this isn't commercial quality code. This also has a hardcoded password for the ansible user, but that is irrelevant and can be freely changed. If this actually runs successfully, and you survive the shock, the file artifacts (the PFX and PEM files) can be removed from the Windows server.
Also note that this script adds a firewall rule at the end. This, of course, will have to be appropriately modified for your environment.
# master script to enable winRM via HTTPS and allow certificate-based
# authentication
# add the local 'ansible' user
$username = "ansible"
$password = ConvertTo-SecureString -string "Ans!b123" -AsPlainText -Force
New-LocalUser -Name $username -AccountNeverExpires -Description "Ansible Remote Management" -Password $password
Add-LocalGroupMember -Member "ansible" -Group "Administrators"
$credential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $username, $password
# import the Ansible root CA; certlm should show this in the
# 'Trusted Root Certification Authorities' folder
$caroot = Import-Certificate -FilePath ".\caroot.pem" -CertStoreLocation "Cert:\LocalMachine\Root"
# import the user cert; certlm should show this in the 'Trusted People' folder
$userkey = Import-Certificate -FilePath ".\winlogin.pem" -CertStoreLocation "Cert:\LocalMachine\TrustedPeople"
New-Item -Path WSMan:\localhost\ClientCertificate -subject "ansible#localhost" -URI "*" -Issuer $caroot.Thumbprint -Credential $credential -Force
# import the server certs - should appear in 'Personal' folder. The PFX file
# must contain both the cert and private key
$srvcert = Get-ChildItem -Path ".\windows.pfx" | Import-PFXCertificate -CertStoreLocation "Cert:\LocalMachine\MY" -Exportable
# Now create the winRM instance
$selector_set = #{Address= "*" Transport = "HTTPS" }
$value_set = #{CertificateThumbprint = $srvcert.Thumbprint}
New-WSManInstance -ResourceURI "winrm/config/Listener" -SelectorSet $selector_set -ValueSet $value_set
Set-Item -Path WSMan:\localhost\Service\Auth\Certificate -value $true
# add a firewall rule
New-NetFirewallRule -DisplayName "Ansible WinRM" -Direction Inbound -Protocol "TCP" -LocalPort "5986" -Action Allow -RemoteAddress #("192.16.2.3")
Okay, now that Windows is done, we should be ready for Ansible. Within your inventory, you will need the following variables to apply to the Windows systems:
ansible_port: 5986
ansible_connection: winrm
ansible_winrm_ca_trust_path: /path/to/caroot.pem
ansible_winrm_transport: certficate
ansible_winrm_cert_pem: /path/to/winlogin.pem
ansible_winrm_cert_key_pem: /path/to/winlogin.key
At this point, Ansible should connect to the Windows server using HTTPS and certificate authentication. I hope I hit all of the steps, it was a while ago I set this up.
Run this on the ansible control host:
python -c "import ssl; print(ssl.get_default_verify_paths())"
this way you will see where python expects to find the certificate
Ensure that the CA trust certificate is pem-encoded
regards
This worked for me..
https://dodgydudes.se/validate-ca-certificate-in-ansible-connecting-with-winrm/
ansible_winrm_ca_trust_path: /etc/ssl/certs #TLS 1.2
ansible_winrm_transport: ntlm
I was getting SSL error in python while connecting to a secured url. As a quick work around I passed verify=false and it worked. Later I got the .cer file now that file path is given to verify. Now I get
SSLError: unknown error (_ssl.c:2825)
What is the problem?
Is it because I gave .cer instead of .pem?
Can I convert .cer to .pem file ?
How to fix this problem?
Essentially, yes - this "unknown error" is a result of having a .cer file instead of a .pem file. I just ran into this, with a different line number (_ssl.c:4025 or so) but otherwise identical symptoms, and using a .pem certificate file instead fixed it.
Courtesy of HUB and Marcel Friedmann on Server Fault, here's how to convert a certificate between those formats:
Open the certificate file with Notepad++ or similar. If it starts with -----BEGIN CERTIFICATE-----, it's already in the right format - just rename it to .pem.
Otherwise, If you have OpenSSL installed, it's a fairly easy matter:
openssl x509 -inform der -in certificate.cer -out certificate.pem
If you don't have OpenSSL installed, but do have Java's keytool, you can use that, but it's a little convoluted.
First, find a keystore that won't mind some use. If you don't have one, create one:
# create a dummy certificate in the file test.keystore, forcing the keystore to be created
keytool -genkey -alias test -keystore test.keystore
# and now delete the cert
keytool -delete -alias test -keystore test.keystore
# the (empty) keystore will still exist
Then, import the .cer-format cert:
keytool -import -trustcacerts -alias test -file certificate.cer -keystore test.keystore
Finally, export it as .pem (making sure it's actually in the right format):
keytool -exportcert -alias test -file certificate.pem -rfc -keystore test.keystore
If you don't have OpenSSL or Java keytool, you'll need to install one of them, or find another way.
This question pertains to Passbook which is under NDA for the next few days, but this is a generic PKCS7 question.
I have a .p12 file that is exported from my keychain. I am able to separate this into 2 pems using the following commands
openssl pkcs12 -in "mycert.p12" -clcerts -nokeys -out certificate.pem
openssl pkcs12 -in "mycert.p12" -nocerts -out key.pem
The next step is to use this key and certificate to create a signed PKCS7 file. This is easy to do with openssl:
openssl smime -binary -sign \
-signer certificate.pem -inkey key.pem \
-in <datafile> -out signature \
-outform DER
The question is, what is the best way to do this in Google App Engine, assuming I have the certificate and key? Unfortunately I'm a little new to cryptography, but I've been googling around and found PyCrypto and keyczar. Is there an accepted way to do this on App Engine, or will I need to write something? Any recommendations on which package to start with?
I know that openssl is not available on AppEngine, but PyCrypto is if you use python 2.7, right? And I've seen posts of people getting keyczar to work with it. I have not been able to find a simple way of generating PKCS7-encoded data given the key and certificate, though.
Thanks in advance for any guidance.
Here's a way using M2Crypto taken from https://github.com/devartis/passbook
def passwordCallback(*args, **kwds):
return password
smime = SMIME.SMIME()
smime.load_key('key.pem', 'certificate.pem', callback=passwordCallback)
pk7 = smime.sign(SMIME.BIO.MemoryBuffer(manifest), flags=SMIME.PKCS7_DETACHED | SMIME.PKCS7_BINARY)
pem = SMIME.BIO.MemoryBuffer()
pk7.write(pem)
# convert pem to der
der = ''.join(l.strip() for l in pem.read().split('-----')[2].splitlines()).decode('base64')
open('signature', 'w').write(der)