What is the difference between cert and verify?
From Documentation:
verify – (optional) if True, the SSL cert will be verified. A CA_BUNDLE path can also be provided.
cert – (optional) if String, path to ssl client cert file (.pem). If Tuple, (‘cert’, ‘key’) pair.
Does this mean I can do the following:
CA_BUNDLE='path/to/.pem'
requests.get(url=google.com, verify= CA_BUNDLE)
or
Cert='path/to/.pem'
requests.get(url=google.com, cert=Cert)
They both look like they do the same thing. except verify can disable ssl verification.
I am trying to compile my code to an exe using PYinstaller. I am using certifi module that I see already has a cacert.pem file but I guess I still have to bundle it with my code.
In my code do I modify ...verify or cert?...with a path to cacert.pem or just 'cacert.pem'?
I think it is clearly stated in the documentation: SSL Cert Verification
The option cert is to send you own certificate, e.g. authenticate yourself against the server using a client certificate. It needs a certificate file and if the key is not in the same file as the certificate also the key file.
The option verify is used to enable (default) or disable verification of the servers certificate. It can take True or False or a name of a file which contains the trusted CAs. If not given I think (not documented?) it will take the default CA path/file from OpenSSL, which works usually on UNIX (except maybe OS X) and not on windows.
if the *.pem file has this section
-----BEGIN PRIVATE KEY-----
....
-----END PRIVATE KEY-----
then use cert
and if not, then use verify
Related
We have an instance of minio running with a certificate that is signed by our corporate CA. Accessing it with S3 Browser works perfect. Now I try to write a python script to upload files. I try to use the windows cert store to get my CA certs
myssl = ssl.create_default_context()
myhttpclient = urllib3.PoolManager(
cert_reqs='CERT_REQUIRED',
ca_certs=myssl.get_ca_certs()
)
s3dev = Minio("s3dev.mycorp.com:9000,
access_key="myAccessKey",
secret_key="mySecretKey"
secure=True,
http_client=myhttpclient
)
I get an error "TypeError: unhashable type: list"
Getting the CA Certs from Windows cert store with ssl.get_ca_certs() returns a list with all the certs in it which seems logic to me, what am I missing here to get something this simple to work ?
Below is my code:
import hvac
client = hvac.Client(
url='https://vault-abc.net',token='s.d0AGS4FE3o6UxUpVTQ0h0RRd',verify='False'
)
print(client.is_authenticated())
ERROR in output:
in cert_verify
raise IOError("Could not find a suitable TLS CA certificate bundle, " OSError: Could not find a suitable TLS CA certificate
bundle, invalid path: False
I got only token and URL to login on console from client no certificates shared! In other java applications code without using any certificate authentication working but in python code under hvac module or CURL or vault CLI expecting certificates to be passed. Any way I can handle this and fix above error?
Do we have any certificate check skip option?
Agenda is authenticate and do fetch vault secrets using python program, without any certificates need to fetch just with Token & vault URL.
You can disable certificate checks, but for something like Vault that's generally a bad idea (disabling security checks on a security service).
In any case, your problem is simple: You are passing 'False' (a string) where you should be passing False (a boolean) as the verify argument.
Passing a string causes the library to look for a certificate at that path; since there is no certificate at the path 'False', you get the error that you are seeing.
A basic Confluent-Kafka producer and consumer have been created to send plaintext messages.
After successfully sending messages from producer to consumer, additional configs were added to use SSL rather than PLAINTEXT.
The following Configs have been implemented, which result in the following error.
"Message Timed Out"
Producer Configs:
bootstrap.servers: localhost9093
security.protocol: SSL
ssl.keystore.location: ../keystore.p12
ssl.keystore.password: [password]
ssl.ca.location: ../CARoot.pem
ssl.key.location: ../key.pem
ssl.certificate.location: ../cert.pem
ssl.key.password: [password]
Server Configs:
ssl.keystore.type= PKCS12
ssl.keystore.location= ../keystore.p12
ssl.keystore.password= [password]
ssl.ca.location= ..\\CARoot.pem
ssl.certificate.location= ..\\cert.pem
ssl.key.password= [password]
ssl.key.location= ../key.pem
security.inter.broker.protocol= SSL
listeners = PLAINTEXT://localhost:9092,SSL://localhost:9093
advertised.listeners = PLAINTEXT://localhost:9092,SSL://localhost:9093
Are there additional configs required to implement SSL?
Additionally, can anyone summarize the CARoot?
From what I have read, it is a chain of certificates.
Therefore, if there is only one certificate, should CARoot.pem and cert.pem be identical?
This file might be the problem.
The certificate and private key were created in PyOpenSSL.
keystore.p12 was converted from a .jks keystore using keytool.
Is there a way to create a CARoot.pem file using this library?
Thank you.
The producer was timing out due to the formatting of the CA Certificate file.
The solution to the following question was used to resolve the time-out error, which uses OpenSSL rather than PyOpenSSL.
Note: OpenSSL is available in Git Bash.
How to export CA certificate chain from PFX in PEM format without bag attributes
Additionally, there were some changes that were made to the configuration of both the server and producer.
Producer Configurations:
bootstrap.servers: localhost9093
security.protocol: SSL
ssl.ca.location: ../CARoot.pem
ssl.key.location: ../key.pem
ssl.certificate.location: ../cert.pem
Server Configurations:
ssl.protocol= SSL
ssl.keystore.type= JKS
ssl.keystore.location= ../keystore.jks
ssl.keystore.password= [password]
ssl.client.auth= required
security.inter.broker.protocol= SSL
listeners = PLAINTEXT://localhost:9092,SSL://localhost:9093
advertised.listeners = PLAINTEXT://localhost:9092,SSL://localhost:9093
I need to use client certificate file in format p12(PKCS12) to talk to a webserver with scrapy, is there a way to do that ?
I can't offer you a tested and complete solution here, but I know a few places where some adjustments might give you what you need.
The starting point is scrapy's ContextFactory object which defines the SSL/TLS configuration. The standard implementation ScrapyClientContextFactory doesn't use client certificates and also doesn't do any server certificate verification, it just accepts any certificate. (More details)
When looking into the source code however you see the alternative BrowserLikeContextFactory is creating an optionsForClientTLS object.
This object can also take a clientCertificate parameter to authenticate to the server. (Details)
So in theory you need to subclass BrowserLikeContextFactory, write there your own creatorForNetloc method and make it create optionsForClientTLS that also have a clientCertificate
In a gist:
#implementer(IPolicyForHTTPS)
class ClientCertContextFactory(BrowserLikeContextFactory):
def creatorForNetloc(self, hostname, port):
with open('yourcert.pem') as keyAndCert:
myClientCert = twisted.internet.ssl.PrivateCertificate.load(keyAndCert.read())
return optionsForClientTLS(hostname.decode("ascii"),
trustRoot=platformTrust(),
clientCertificate=myClientCert,
extraCertificateOptions={
'method': self._ssl_method,
})
Activating the context factory in settings.py:
DOWNLOADER_CLIENTCONTEXTFACTORY = 'your.package.ClientCertContextFactory'
According to the docs twisted.internet.ssl.PrivateCertificate can only load pem or asn.1 format keys, means you will have to convert your key into pem format:
openssl pkcs12 -in client_ssl.pfx -out client_ssl.pem -clcerts
(Borrowed from Converting pfx to pem using openssl)
Update Conversion for PKCS12 files in p12 format:
openssl pkcs12 -in client_cert.p12 -out client_cert.pem -clcerts
Using python-requests, how can I pin a self-signed .pem certificate for a specific server directly, without using CA root bundles?
Is this currently possible? If yes, can you please provide an example?
I read https://2.python-requests.org/en/v2.8.1/user/advanced/#ssl-cert-verification but am not sure if this applies to what I'm trying to do:
You can also specify a local cert to use as client side certificate, as a single file (containing the private key and the certificate) or as a tuple of both file’s path: requests.get('https://kennethreitz.com', cert=('/path/server.crt', '/path/key')) Response [200]
Because the certificate file is self-signed, this works just as you would do it normally with requests. Below is a step-by-step procedure:
Obtain the self-signed certificate, ideally in some secure, out-of-band manner. For example, I run a webserver that offers HTTPS access via a self-signed certificate, so I downloaded the certificate using scp:
scp <username>#<server>:/path/to/certfile.crt .
Because I use nginx this is already in PEM format, but if it's not you'll need to convert it. That's outside the scope of this answer.
Use the certificate file from inside requests:
r = requests.get('https://yoursite.com/', verify='certfile.crt')
That's all you need to do.
If you can't obtain the certificate in an out-of-band manner you trust, you can obtain the certificate using your browser. All browsers will let you export the certificate via their UIs. This is less-secure: if someone is going to MITM you then they may well have already started, and can offer you their MITM root CA instead of your self-signed cert.
You can also verify certificates against their fingerprints.
For this you need a custom transport adapter for requests.
An example for a simple one can be found here:
https://github.com/untitaker/vdirsyncer/blob/9d3a9611b2db2e92f933df30dd98c341a50c6211/vdirsyncer/utils/init.py#L198
import requests
from requests.packages.urllib3.poolmanager import PoolManager
class _FingerprintAdapter(requests.adapters.HTTPAdapter):
def __init__(self, fingerprint=None, **kwargs):
self.fingerprint = str(fingerprint)
super(_FingerprintAdapter, self).__init__(**kwargs)
def init_poolmanager(self, connections, maxsize, block=False):
self.poolmanager = PoolManager(num_pools=connections,
maxsize=maxsize,
block=block,
assert_fingerprint=self.fingerprint)