I am trying to create a little Python script to send data to a server using the requests module in Python. To make it a bit more secure i want to use self signed certificates made in a program called XCA. When using the certificates in the browser everything works and is secure. When using Postman to send a request with the certificates it works as well. But when i created the Python script it seems to not work or can't get to the certificate.
I have tried to include the CA with the 'Verify' command as well as adding the certificates with the 'cert' command.
When using the 'Verify' command (as seen down below) with the CA i get the error message: Remote end closed connection without response' This message seems to appear everytime i add the CA to this script somehow.
When i use the cert command i get this error message: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))). I have searched for a sollution to this problem and it seemed that the CA should be include in the certifi 'cacert.pem' file. I have done this but then i also get the 'Remote disconect' message.
Code with 'verify':
import requests
import json
url = "https://ip/tapi"
payload = json.dumps ({"command" : "GetUserList"})
headers = {'content-type': 'application/json',
'X-TAPI': '',
'Authorization': 'Basic ',
'connection': 'keep-alive'}
r = requests.request("POST", url, headers=headers, data= payload, verify= 'Tbox_CA.crt')
print(r.text)
Code with 'cert':
import requests
import json
cert_file_path = 'HTTPS_client.crt'
key_file_path = 'HTTPS_client_key.pem'
url = "https://ip/tapi"
payload = json.dumps ({"command" : "GetUserList"})
headers = {'content-type': 'application/json',
'X-TAPI': '',
'Authorization': 'Basic ',
'connection': 'keep-alive'}
cert = cert_file_path, key_file_path
r = requests.request("POST", url, headers=headers, data= payload, cert=cert)
print(r.text)
When using the 'Verify' command (as seen down below) with the CA i get the error message: Remote end closed connection without response' This message seems to appear everytime i add the CA to this script somehow.
Which means that SSL works, the HTTP request was sent, but then the server closes the connection without sending a response. SSL is not the problem here and the certificate validation worked.
When i use the cert command i get this error message: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))).
The cert argument is not useful here since it is about providing a client certificate if the server requests it. Only, your server does not request a client certificate so providing this argument is basically the same as not providing it. Since no useful CA is given the certificate validation works. Since SSL thus already failed it cannot even send the real HTTP request inside the SSL connection, i.e. the error you've received when giving a CA is just masked because this error comes earlier.
Why the request failed from Python but not from the browser or Postman is unknown. There might be some server side bot protection implemented, but this is just a guess since there is nothing known about the actual URL you access.
Related
Whenever am calling an API from python am getting error :
'self._sslobj.do_handshake()
ss.SSLCertVerificationError:[SSL:CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get Local issuer certificate.
During handling of the above exception, another exception occurred:'
Here is the code:
import requests
url = 'some URL'
response = requests.request('GET', url, verify=False)
print(response.text.encode('utf8'))
The same I am doing from postman initially I got error as 'No required SSL Certificates was sent', later on I have added the certificate to Postman then am able to hit the API and successfully getting the response.
Could anyone please what I need to do to overcome the error.
So the program i am developing involves posting documents in bank DMS server. They have provided me server certificate in .cer format which i have inserted in my verify variable in code. They also provided client id and password which i have to embed in the header itself. I generated self signed client certificate and private key and gave them the client certificate in cer format and public key. Also in code i gave path of client certificate and private key in cert tuple.
Upon executing code, i am getting this error:
HTTPSConnectionPool(host='apimuat.xxxbank.com', port=9095): Max retries exceeded with url: /doc-mgmt/v1/uploadDoc (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fb01bd8a160>: Failed to establish a new connection: [Errno 60] Operation timed out'))
File "/Users/fpl_mayank/Documents/FPL/python-virtual-env/uploadDocApi/server.py", line 164, in main
result = requests.post(url,
File "/Users/fpl_mayank/Documents/FPL/python-virtual-env/uploadDocApi/server.py", line 189, in <module>
main()
I have tested it with 'https://postman-echo.com/post' without mentioning cert and verify just to check if my request is going through or not. it is working fine there.
This is my code snippet where i am using request functions.
url='https://apimuat.xxxbank.com:9095/doc-mgmt/v1/uploadDoc'
headers = {"Content-Type": "application/json", "client_id":"af197b22539647fba4db8b971b43e38",
"client_secret":"c1AA406e24074d8887954472C78a924"}
data = req
result = requests.post(url,
data=data,
headers=headers,
cert=('/Users/fpl_mayank/Documents/FPL/python-virtual-
env/uploadDocApi/keystore/dms_csr_certificate_self.cer','/Users/fpl_mayank/Documents/FPL/python-virtual-env/uploadDocApi/keystore/dms_private_key.key'),
verify='/Users/fpl_mayank/Documents/FPL/python-virtual-env/uploadDocApi/truststore/APIM-UAT.cer'
)
res = result.json()
In apidoc it was mentioned, 2-way SSL authentication will be implemented bw client and server. Also i have made virtual-env for this program for that matter. Please help. I am the first one to write an API using python in my company so only way to get my issue resolve is through good ol stackoverflow.
So i solved this. idk exactly what solved it but make sure when working on api's, get the endpoint's ip whitelisted from your network, as per requirement and same goes from their side too. Also i was sending formatted json request having identation and spaces so make sure to keep json in one line.
Using curl I can connect to a server that needs specific certificate.
curl -E ./file.crt.pem --key ./file.key.pem -k https://server.url
curl version: 7.29.0
But when using Python's requests library, I get an error:
import requests
cert_file_path = "file.crt.pem"
key_file_path = "file.key.pem"
cert = (cert_file_path, key_file_path)
url = 'https://server.url'
r = requests.post(url, cert=cert, verify=False)
Error:
SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca')])"))
Python version: v3.7
What am I missing?
A comment on this answer helped me figure this out.
Update your code as such:
import requests
cert_file_path = "file.crt.pem"
key_file_path = "file.key.pem"
cert = (cert_file_path, key_file_path)
url = 'https://server.url'
r = requests.post(url, cert=cert, verify="path/to/ca_public_keys.pem") # replace with your file
I'm assuming you're using a self-signed certificate, so you need to specify the .pem file containing the public certificates of the CA that issued your self-signed certificate. Make sure to include the intermediate certificates, otherwise the requests library will throw the tlsv1 alert unknown ca error.
You can check the issuer of your client certificate by typing openssl x509 -noout -in file.crt.pem -issuer in a terminal.
Request module checks environmental variable REQUESTS_CA_BUNDLE for cert file. So just do this
export REQUESTS_CA_BUNDLE=/absolute/path/to/your/file.crt.pem
Your python code will simply be:
import requests
url = 'https://server.url'
r = requests.post(url)
print(r.text)
I'm getting a
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)>
When I try to validate captchas in my django project.
This is how i do it:
recaptcha_response = request.POST.get('g-recaptcha-response')
print(recaptcha_response)
url = 'https://www.google.com/recaptcha/api/siteverify'
values = {
'secret': settings.CAPTCHA_SECRET_KEY,
'response': recaptcha_response
}
data = urllib.parse.urlencode(values).encode()
req = urllib.request.Request(url, data=data)
response = urllib.request.urlopen(req) # It fails here
result = json.loads(response.read().decode())
print(result)
The site has a valid certificate, and it works on local. In the log i'm getting this:
Request Method: POST
Request URL: http://prod.xxxx.com/evalua
Which is weird because the site works in https. Its on kubernetes, could that be the problem? I really have no idea what the problem IS? I have the captcha keys correctly set up in de recaptcha admin console. And the certificate are not autosign. I use lets encrypt
Check how you build the container image for your app and if it has very old CA certificates in it. You can use something like ADD https://curl.haxx.se/ca/cacert.pem /etc/ssl/certs/cacert.pem to ensure you have the latest standard bundle. You can also switch to Requests and Certifi instead of urllib, as that embeds a copy of the current cert bundle and ensures it is used.
I need to write a simple test script for rest get using python. What I have is:
import request
url = 'http://myurl......net'
headers ={'content-type':'application/xml'}
r= requests.get(url,headers=headers)
so this give me the following SSL error:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed(_ssl.c:590)
So, i did some research and add " verify = False" to the end of my last line of code, but not I am stuck with: : InsecureRequestsWarning, Unverified request is been made. Addig certificate verification is strongly advised."
What to do to get rid of this message?