We Keep Coding

SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed while generating SMS from python script

I have my python script similar to below, Scripts works fine in my personal laptop.
import plivo
import sys
auth_id = "XXXXXX"
auth_token = "YYYYYYYYYYYY"
test = plivo.RestClient(auth_id, auth_token)
message_created = test.messages.create(
src='ZZZZZZ',
dst='+NNNNN',
text='Testing!!'
)
However while running the script in our organization PC's its throwing error
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.plivo.com', port=443): **Max retries exceeded with url**: /v1/Account/SXXXXXYW/Message/ (Cau
sed by SSLError(SSLError(1, u'[SSL: **CERTIFICATE_VERIFY_FAILED**] certificate verify failed (_ssl.c:590)'),))
I tried to add ssl._create_default_https_context = ssl._create_unverified_context and PYTHONHTTPSVERIFY=0 but unfortunately nothing works for me. Can anyone one help me how to resolve the error?

Try the solution from https://github.com/locustio/locust/issues/417
How to get rid from “SSL: CERTIFICATE_VERIFY_FAILED” Error
On Windows, Python does not look at the system certificate, it uses its own located at ?\lib\site-packages\certifi\cacert.pem.
The solution to your problem:
download the domain validation certificate as *.crt or *pem file
open the file in editor and copy it's content to clipboard
find your cacert.pem location: from requests.utils import
DEFAULT_CA_BUNDLE_PATH;
print(DEFAULT_CA_BUNDLE_PATH)
edit the cacert.pem file and paste your domain validation
certificate at the end of the file.
Save the file and enjoy requests!

Disable Python requests SSL validation for an imported module

I'm running a Python script that uses the requests package for making web requests. However, the web requests go through a proxy with a self-signed cert. As such, requests raise the following Exception:
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",)
I know that SSL validation can be disabled in my own code by passing verify=False, e.g.: requests.get("https://www.google.com", verify=False). I also know that if I had the certificate bundle, I could set the REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE environment variables to point to those files. However, I do not have the certificate bundle available.
How can I disable SSL validation for external modules without editing their code?

Note: This solution is a complete hack.
Short answer: Set the CURL_CA_BUNDLE environment variable to an empty string.
Before:
$ python
import requests
requests.get('http://www.google.com')
<Response [200]>
requests.get('https://www.google.com')
...
File "/usr/local/lib/python2.7/site-packages/requests-2.17.3-py2.7.egg/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",)
After:
$ CURL_CA_BUNDLE="" python
import requests
requests.get('http://www.google.com')
<Response [200]>
requests.get('https://www.google.com')
/usr/local/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning)
<Response [200]>
How it works
This solution works because Python requests overwrites the default value for verify from the environment variables CURL_CA_BUNDLE and REQUESTS_CA_BUNDLE, as can be seen here:
if verify is True or verify is None:
verify = (os.environ.get('REQUESTS_CA_BUNDLE') or
os.environ.get('CURL_CA_BUNDLE'))
The environment variables are meant to specify the path to the certificate file or CA_BUNDLE and are copied into verify. However, by setting CURL_CA_BUNDLE to an empty string, the empty string is copied into verify and in Python, an empty string evaluates to False.
Note that this hack only works with the CURL_CA_BUNDLE environment variable - it does not work with the REQUESTS_CA_BUNDLE. This is because verify is set with the following statement:
verify = (os.environ.get('REQUESTS_CA_BUNDLE') or os.environ.get('CURL_CA_BUNDLE'))
It only works with CURL_CA_BUNDLE because '' or None is not the same as None or '', as can be seen below:
print repr(None or "")
# Prints: ''
print repr("" or None )
# Prints: None

I saw this hack only due to some trouble with my private CA.
The given hack with CURL_CA_BUNDLE='' will not longer work in 2022 with next minor release of requests (that means 2.28 ?).
Please refer to GH issue 6071 which was fixed 6 days before in Feb. 2022

"key values mismatch" when using context.use_certificate_chain_file

When using context.use_certificate_chain_file I get a key error (openssl.Context Python). The error is:
Traceback (most recent call last):
File "/home/user/public_html/application.py", line 363, in <module>
context.use_privatekey_file('/etc/ssl/private/' + HOSTNAME + '.key')
OpenSSL.SSL.Error: [('x509 certificate routines', 'X509_check_private_key', 'key values mismatch')]
It saying a key values mismatch, but I wouldn't think the chain would affect that.
If I comment the context.use_certificate_chain_file line, it works perfectly (but gives an ssl verification error in the browser).
Here is the snippet of my code:
context = openssl.Context(openssl.SSLv23_METHOD)
context.set_options(openssl.OP_NO_SSLv2)
context.set_options(openssl.OP_NO_SSLv3)
context.use_certificate_file('/etc/ssl/certs/' + HOSTNAME + '.crt')
context.use_certificate_chain_file('/etc/ssl/certs/' + HOSTNAME + '.cabundle')
context.use_privatekey_file('/etc/ssl/private/' + HOSTNAME + '.key')
context.set_cipher_list(':'.join(supported_ciphers))
Any ideas why its giving the error?

Any ideas why its giving the error?
The error is propagated up from OpenSSL. Its error 0x0B080074:
$ openssl errstr 0x0B080074
error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch
Based on SSL install problem - “key value mismatch” (but they do match?), you have one of two problems.
First, the private key does not match the public key in the certificate. Second, your certificate_chain_file is missing the intermediate certificates required to build a valid path from the server's certificate to a root. Here, the root would be the CA that signed your certificate.
So your fix is to either (1) ensure the public/private key pair is in fact a pair, or (2) include the necessary intermediate certificates in the chain file.
Without knowing the private key ('/etc/ssl/private/' + HOSTNAME + '.key'), the server certificate ('/etc/ssl/certs/' + HOSTNAME + '.crt') or the contents of the chain file ('/etc/ssl/certs/' + HOSTNAME + '.cabundle'), we really can't give you more details on how to fix it.
You can provide us with the server's certificate with:
cat '/etc/ssl/certs/' + HOSTNAME + '.crt' | openssl x509 -text -noout
You can provide us with the chain file by just cat'ing. It will be 3 or 4 PEM encoded certificates concatenated together:
cat `'/etc/ssl/certs/' + HOSTNAME + '.cabundle'`

Its working now, the chain had to be appended to the crt.

Requests: what is the difference between cert and verify?

What is the difference between cert and verify?
From Documentation:
verify – (optional) if True, the SSL cert will be verified. A CA_BUNDLE path can also be provided.
cert – (optional) if String, path to ssl client cert file (.pem). If Tuple, (‘cert’, ‘key’) pair.
Does this mean I can do the following:
CA_BUNDLE='path/to/.pem'
requests.get(url=google.com, verify= CA_BUNDLE)
or
Cert='path/to/.pem'
requests.get(url=google.com, cert=Cert)
They both look like they do the same thing. except verify can disable ssl verification.
I am trying to compile my code to an exe using PYinstaller. I am using certifi module that I see already has a cacert.pem file but I guess I still have to bundle it with my code.
In my code do I modify ...verify or cert?...with a path to cacert.pem or just 'cacert.pem'?

I think it is clearly stated in the documentation: SSL Cert Verification
The option cert is to send you own certificate, e.g. authenticate yourself against the server using a client certificate. It needs a certificate file and if the key is not in the same file as the certificate also the key file.
The option verify is used to enable (default) or disable verification of the servers certificate. It can take True or False or a name of a file which contains the trusted CAs. If not given I think (not documented?) it will take the default CA path/file from OpenSSL, which works usually on UNIX (except maybe OS X) and not on windows.

if the *.pem file has this section
-----BEGIN PRIVATE KEY-----
....
-----END PRIVATE KEY-----
then use cert
and if not, then use verify

pyOpenSSL creating a pem file

I've created a key pair using the following code in python with pyOpenSSL:
from OpenSSL import crypto
k = crypto.PKey()
k.generate_key(crypto.TYPE_RSA, 2048)
Now how can I create the private and public key .pem files from the key object?
If there is any tutorial available please let me know. I found none. From the manual, it's difficult to know as I'm new to OpenSSL.
What are the chances that the same code will create two same key pairs is there is no specific unique key is being used in RSA?

I know this is an old question - but as I've just found it I thought I'd add an answer.
The easiest way to do this with Python 3.x is to use PyCryptodome.
The in Python (for a 2048-bit key):
from Cryptodome.PublicKey import RSA
key = RSA.generate(2048)
pv_key_string = key.exportKey()
with open ("private.pem", "w") as prv_file:
print("{}".format(pv_key_string.decode()), file=prv_file)
pb_key_string = key.publickey().exportKey()
with open ("public.pem", "w") as pub_file:
print("{}".format(pb_key_string.decode()), file=pub_file)
If you want to check the private key on the (Linux) command-line use:
$ openssl rsa -check -inform pem -noout -in private.pem
RSA key ok
...

I hope this will help people in the future, because I had this same need and couldn't find an answer so I did it myself. Thought I would share it with you.
1. Creating a PEM file
bio_pub = _new_mem_buf() # Memory buffers to write to
bio_priv = _new_mem_buf()
helper = OpenSSL.crypto._PassphraseHelper(OpenSSL.crypto.FILETYPE_PEM, None)
pk = OpenSSL.crypto.PKey()
pk.generate_key(OpenSSL.crypto.TYPE_RSA, n)
# Convert from EVP_PKEY type to RSA type
rsa_pkey = _lib.EVP_PKEY_get1_RSA(pk._pkey)
result_code = _lib.PEM_write_bio_RSAPublicKey(bio_pub, rsa_pkey)
result_code = _lib.PEM_write_bio_RSAPrivateKey(
bio_priv, rsa_pkey, _ffi.NULL, _ffi.NULL, 0,
helper.callback, helper.callback_args)
After this part you will have the public and private keys in your buffers.
To get it as a string you can call the functions:
_bio_to_string(bio_pub), _bio_to_string(bio_priv)
I used these imports for the special "private" functions of OpenSSL.crypto:
import OpenSSL
from OpenSSL._util import lib as _lib, ffi as _ffi
from OpenSSL.crypto import _new_mem_buf, _bio_to_string

You can create a .pem key by follow this tutorial at:
https://help.ubuntu.com/community/OpenSSL
that suppose you want to create a CA(certificate authority) certificate, that
is little complicate because you already have to get a CA from somewhere
because it's not free.
if you only want to create a key juste for your ssl connection test it
better to create
a self-sign certificate.
then make sure first you have install openssl and you have resolve the CN (Common Name) on your serve. without that you will be in trouble to use the created certificate.
for the Self-sign certificate use this command line:
$ openssl genrsa -des3 -passout pass:x -out server.pass.key 2048
$ openssl rsa -passin pass:x -in server.pass.key -out server.key
$ rm server.pass.key
$ openssl req -new -key server.key -out server.csr (list of question to answer)
$ openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
after you got the certificate create you have to activate your
server mod-ssl and add the line where is locate your certificate.
later you have to insert that certificate in your IE certificate
list to get it work with you apache ssl connection daemon.