I have generated following self-signed certificates for my server and client.
I have created ca.crt & ca.key. Using ca.crt & ca.key, I have created server.crt, server.key for server and client.crt, client.key for client respectively.
I am using python requests library as client. Below is the code snippet:
import json
import requests
cert = ("/home/tests/certs/client.crt",
"/home/tests/certs/client.key")
class TestCart():
def test_cart(self, **kwargs):
url = "https://192.168.X.Y/cart"
cart_data = {
'id': kwargs.get('id'),
'items': kwargs.get('items')
}
req_data = json.dumps(cart_data)
resp = requests.post(url,
data=req_data,
verify="/home/certs/ca.cert",
cert=cert)
print resp.text
if __name__ == '__main__':
t_cart = TestCart()
data = {'id': 'ba396e79-0f0f-4952-a931-5a528c9ff72c', 'items': []}
t_cart.test_cart(**data)
This gives exception:
requests.exceptions.SSLError: HTTPSConnectionPool(host='192.168.X.Y',
port=443): Max retries exceeded with url: /cart (Caused by
SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify
failed (_ssl.c:590)'),))
If I use verify=False, code works, but I want to verify. What should be the value of verify in my request ?
It is highly recommended to have a deeper look at the excellent documentation for requests. It has a special chapter about SSL Cert Validation which explains:
You can pass verify the path to a CA_BUNDLE file or directory with certificates of trusted CAs:
>>> requests.get('https://github.com', verify='/path/to/certfile')
Assuming that your server certificate was signed by your ca.crt you should use this for the verify parameter.
EDIT: based on the discussion it looks like that CA and server certificate used the same subject. This means that the certificate validation assumes that this is a self-signed certificate which thus results in an certificate validation error.
Related
on a AWS Lambda function, I have a simple HTTP Request
import requests
def lambda_handler(event, context):
request_headers = {
'X-API-KEY': 'somekey',
'Content-Type':content,
'Host':'somehost',
}
resp = requests.get(some_url_here, headers=request_headers)
api_response = resp.json()
If I set the request requests.get(some_url_here, headers=request_headers, verify=False) I got no issues with the code and it works perfectly, but when I removed it it yields
Error encountered: HTTPSConnectionPool(host='x.xxx.xxx.xx', port=443): Max retries exceeded with url:some_url_here (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)')))
how to fix this issue?
Using curl I can connect to a server that needs specific certificate.
curl -E ./file.crt.pem --key ./file.key.pem -k https://server.url
curl version: 7.29.0
But when using Python's requests library, I get an error:
import requests
cert_file_path = "file.crt.pem"
key_file_path = "file.key.pem"
cert = (cert_file_path, key_file_path)
url = 'https://server.url'
r = requests.post(url, cert=cert, verify=False)
Error:
SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca')])"))
Python version: v3.7
What am I missing?
A comment on this answer helped me figure this out.
Update your code as such:
import requests
cert_file_path = "file.crt.pem"
key_file_path = "file.key.pem"
cert = (cert_file_path, key_file_path)
url = 'https://server.url'
r = requests.post(url, cert=cert, verify="path/to/ca_public_keys.pem") # replace with your file
I'm assuming you're using a self-signed certificate, so you need to specify the .pem file containing the public certificates of the CA that issued your self-signed certificate. Make sure to include the intermediate certificates, otherwise the requests library will throw the tlsv1 alert unknown ca error.
You can check the issuer of your client certificate by typing openssl x509 -noout -in file.crt.pem -issuer in a terminal.
Request module checks environmental variable REQUESTS_CA_BUNDLE for cert file. So just do this
export REQUESTS_CA_BUNDLE=/absolute/path/to/your/file.crt.pem
Your python code will simply be:
import requests
url = 'https://server.url'
r = requests.post(url)
print(r.text)
I'm trying to pull data from an API which is secured by SSL. I wrote a python script to pull the data. Beforehand I have to convert a .p12 file to an openSSL certificate. When I use the following code it works just fine:
# ----- SCRIPT 1 -----
def pfx_to_pem(pfx_path, pfx_password):
''' Decrypts the .pfx file to be used with requests. '''
with tempfile.NamedTemporaryFile(suffix='.pem') as t_pem:
f_pem = open(t_pem.name, 'wb')
pfx = open(pfx_path, 'rb').read()
p12 = OpenSSL.crypto.load_pkcs12(pfx, pfx_password)
f_pem.write(OpenSSL.crypto.dump_privatekey(OpenSSL.crypto.FILETYPE_PEM, p12.get_privatekey()))
f_pem.write(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, p12.get_certificate()))
ca = p12.get_ca_certificates()
if ca is not None:
for cert in ca:
f_pem.write(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert))
f_pem.close()
yield t_pem.name
# read some config
with open('config.json') as config_json:
config = json.load(config_json)
api_url = config['api_url']
cert = config['cert']
cert_pem_path = cert['file']
cert_key_file = cert['pass']
# make the request
with pfx_to_pem(cert_pem_path, cert_key_file) as cert:
r = requests.get(api_url, cert = cert)
Because I'm also using the same functionality to authenticate my Flask web service towards the server I split up the cert file into three files:
# ----- SCRIPT 1 -----
# get certificate
f_pem.write(OpenSSL.crypto.dump_certificate(
OpenSSL.crypto.FILETYPE_PEM, p12.get_certificate())
)
# get keyfile
f_key.write(OpenSSL.crypto.dump_privatekey(
OpenSSL.crypto.FILETYPE_PEM, p12.get_privatekey())
)
# get CA_BUNDLE
ca = p12.get_ca_certificates()
if ca is not None:
for cert in ca:
f_ca.write(
OpenSSL.crypto.dump_certificate(
OpenSSL.crypto.FILETYPE_PEM, cert
))
Then I'm running the web service with the following code:
# ----- SCRIPT 2 -----
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
context.verify_mode = ssl.CERT_REQUIRED
context.load_verify_locations(cert_ca)
context.load_cert_chain(cert_pem, cert_key)
app.run(ssl_context = context, host = '0.0.0.0')
and changed the requests call to
# ----- SCRIPT 1 -----
r = requests.get(api_url, cert = (cert_pem, cert_key), verify = cert_ca)
When trying to pull data from the API I get the error
requests.exceptions.SSLError: HTTPSConnectionPool(host='some.host', port=443): Max retries exceeded with url: /some/path/var?ID=xxxxxx (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:847)'),))
Question 1: What am I doing wrong creating the CA_BUNDLE?
Question 2: Am I handling the creation of the web service correctly? My goal is to verify my server against the server holding the data to eventually be able to receive the data by push request.
EDIT: when connecting to my web service (in a browser) I receive the warning that the connection is not secure, because the certificate is not valid, despite the fact that I imported the .p12 certificate into my browser.
So I'm using the request and json library to call API, in my case I can set-up the request to ignore the certificate and this quickly solved my issue
requests.get(url, headers=headers, verify=False)
the argument verify=False ignore the certificate but when you run your code it will show a warning message as output saying that the certificate is wrong, so you can add this other piece of code to don't get request warning showed:
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
I know that doesn't answer your question but maybe you can try to see if without certificate you are able to get information without problem.
I need help making the following Curl request in Python:
curl --data "username=USERNAME&password=PASSWORD" --header "Authorization: Basic Y2xvdWQtYXBp" -X POST "https://ApiCallUrl/auth/token?grant_type=password" --insecure
I have the following code:
url = 'https://fakepath'
data = "username=velocity_169234&password=velocity_69234"
header = {'Authorization': b'Basic' + b64encode(('cloud-api').encode('utf-8'))}
data = urlencode(dict(username='USERNAME',password='PASSWORD')).encode('ascii')
response = urlopen(Request(url, data, header))
print(response.info())
How do I add the --insecure flag?
I am getting this error
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)>
curl
-k, --insecure
(SSL) This option explicitly allows curl to perform "insecure" SSL connections and transfers.
All SSL connections are attempted to be made secure by using the CA certificate bundle installed by default.
All connections considered "insecure" will fail unless -k/--insecure is used.
requests
Requests can also ignore verifying the SSL certificate if you set verify to False:
requests.get('https://kennethreitz.org', verify=False)
<Response [200]>
By default, verify is set to True. Option verify only applies to host certs.
urlopen don't have such a parameter.
Note: Disable Certificate Verification is a Security Risk!
I need to write a simple test script for rest get using python. What I have is:
import request
url = 'http://myurl......net'
headers ={'content-type':'application/xml'}
r= requests.get(url,headers=headers)
so this give me the following SSL error:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed(_ssl.c:590)
So, i did some research and add " verify = False" to the end of my last line of code, but not I am stuck with: : InsecureRequestsWarning, Unverified request is been made. Addig certificate verification is strongly advised."
What to do to get rid of this message?