We Keep Coding

SSL error only in python command window with apify request

I am trying to use endpoint from apify.com. When I run my request in web browser with token everything is fine but if I run my request via requests library from python console I am getting following error:
SSLError: HTTPSConnectionPool(host='', port=443): Max retries exceeded with url: /endpoint?token=token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1131)')))
Moreover if I set verify = False in my request than request is working. Does anyone have an idea what can be wrong? Thanks in advance

I had this issue come up a few weeks ago.
>>> pip install certifi
>>> python -m certifi
I'm not certain that one needs to actually call the module to get it's functionality, but I did and it solved the error. More info on Certifi here. It is also a recommended package extension to requests from their website. I added those lasts bits because I was wary of installing a package that ostensibly was never called after installation.

Solution was to install internal company SSL package for managing SSL connection from python. There was a recent change.

Installing pygame or other python packages on school computers behind firewall

i'm a teacher in a high school and we're using python to code. I'm trying to install packages like pygame and mysql.connector but the firewall is blocking me from using pip...
I get the following error messages:
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1123)'))) - skipping
Any chance there is a way to do this around the firewall? I've already tried contacting our IT tech but they can't or won't help. We need this for our senior pupils and their computing projects.
Thanks!!

You're going to have a bad time if you can't even install pip packages on those PCs.
My recommendation instead would be to create VM images with all the dependencies installed at home and just bring them to the school PCs.
This was the standard at my university's CS degree at least. We were provided with pre-configured environment for pretty much every lecture (operating systems, programming, computer networks etc.) - as a plus, that way the professors didn't have to waste time "troubleshooting" unique issues with environment.
Link for Oracle's Open-source VM

Databricks CLI: SSLError, can't find local issuer certificate

I have installed and configured the Databricks CLI, but when I try using it I get an error indicating that it can't find a local issuer certificate:
$ dbfs ls dbfs:/databricks/cluster_init/
Error: SSLError: HTTPSConnectionPool(host='dbc-12345678-1234.cloud.databricks.com', port=443): Max retries exceeded with url: /api/2.0/dbfs/list?path=dbfs%3A%2Fda
tabricks%2Fcluster_init%2F (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer
certificate (_ssl.c:1123)')))
Does the above error indicate that I need to install a certificate, or somehow configure my environment so that it knows how to find the correct certificate?
My environment is Windows 10 with WSL (Ubuntu 20.04) (the command above is from WSL/Ubuntu command line).
The Databricks CLI was installed into an Anaconda environment including the following certificates and SSL packages:
$ conda list | grep cert
ca-certificates 2020.6.20 hecda079_0 conda-forge
certifi 2020.6.20 py38h32f6830_0 conda-forge
$ conda list | grep ssl
openssl 1.1.1g h516909a_1 conda-forge
pyopenssl 19.1.0 py_1 conda-forge
I get a similar error when I attept to use the REST API with curl:
$ curl -n -X GET https://dbc-12345678-1234.cloud.databricks.com/api/2.0/clusters/list
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

This problem can be solved by disabling the SSL certificate verification. In Databricks CLI you can do so by specifying insecure = True in your Databricks configuration file .databrickscfg.

I established trust to my Databricks instance by setting the environment variable REQUESTS_CA_BUNDLE.
➜ databricks workspace list
Error: SSLError: HTTPSConnectionPool(host='HOSTNAME.azuredatabricks.net', port=443): Max retries exceeded with url: /api/2.0/workspace/list?path=%2F (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))
➜ export REQUESTS_CA_BUNDLE=path/to/ca-bundle
➜ databricks workspace list
Users
Shared
Repos
From GitHub Issue:
Download the root CA certificate used to sign the Databricks certificate. Determine the path to the CA bundle and set the environment variable REQUESTS_CA_BUNDLE. See SSL Cert Verification for more information.

There is a similar issue in GitHub for Azure CLI. The solution is practically the same. Combining that with the Erik's answer:
Download the certificate using your browser and save it to disk
Open you Chrome and go to the Databricks website
Press CTRL + SHIFT + I to open the dev tools
Click Security tab
Click View certificate button
Click Details tab
On the Certification Hierarchy, (the top panel), click the highest node in the tree
Click Export the selected certificate
Choose where you want to save (eg. /home/cert/certificate.crt)
Use the SET command on Windows or the export on Linux to create a env variable called REQUESTS_CA_BUNDLE and point it to the downloaded file in the Step 1. (keep in mind that this need to be done in the same machine as you are trying to use the dbfs not in the cluster) For instance:
Linux
export REQUESTS_CA_BUNDLE=/home/cert/certificate.crt
Windows
set REQUESTS_CA_BUNDLE=c:\temp\cert\certificate.crt
Try to run your command dbfs ls dbfs:/databricks/cluster_init/ again
$ dbfs ls dbfs:/databricks/cluster_init/
It should work!

Why do HTTPS requests produce SSL CERTIFICATE_VERIFY_FAILED error?

Here is my Python code:
import requests
requests.get('https://google.com')
This is the error:
requests.exceptions.SSLError: HTTPSConnectionPool(host='google.com', port=443):
Max retries exceeded with url: / (Caused by SSLError(SSLError(1,
'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)'),))
Using Insomnia gives me an error related with certificates:
My OS is Windows 7 Professional.

requests.get('https://google.com', verify='/path/to/certfile')
or you can skip verifications by doing this:
requests.get('https://google.com', verify=False)
You should specify your CA.

This fixed it: Python referencing old SSL version
The openssl versions used to differ for python and the one offered by homebrew
if
brew install python --with-brewd-openssl
doesn't work
try
brew install openssl
brew install python
after uninstalling python

If you are running under business network, contact the network administrator to apply required configurations at the network level.

You might add header and verify argument to by-pass ssl certificate security.
r = requests.get(URL, headers = {'User-agent': 'your bot 0.1'}, verify=False)
You should specify path your certificate if you have.

In the requests.get you can set the verify flag to False. This way the handshake between the program and the server is going to be ignored.
-- This isn't a guaranteed method because some servers have strict policy to deliver responses to requests.

If you using proxy server,add proxy to your requests.
just like:
proxies = {'http':'http://localhost:port','https':'http://localhost:port'}
requests.get('your_request_website', headers=headers, proxies=proxies)
hope this helps.

I resolved my problem by installing openssl
you can go here and download the Light version or any version suited to your needs:
https://slproweb.com/products/Win32OpenSSL.html

SSL certificate error - Jupyter notebook

I'm using requests in Jupyter notebooks to make a connection to 'https://dynamodb.eu-west-3.amazonaws.com/ (Amazon AWS) & have been getting the error :-
SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))
It works with verify=False but I don't want to implement this due to security.
I've been googling solutions for days, trying to set REQUESTS_CA_BUNDLE from cacert.pem to weak.pem, setting SSL_CERT_FILE similarly, using certifi old_where instead of where, downgrading certifi, reinstalling requests with requests[security] - nothing has worked so far. I can connect fine via my browser, so it must be requests or Jupyter notebooks that has the problem or rather is missing the certificate.. I'm able to use requests to other https locations - I'm not sure if they are using SSL encryption or not. I don't know if I need to enable something in the environment or whether I need to get the AWS certificate into my environment variables (and also I don't know how to do either!)
Any help would be much appreciated! Thanks