Prevent OpenSSL from using system certificates? - python

How can I prevent OpenSSL (specifically, Python's ssl module) from using system certificate authorities?
In other words, I would like it to trust only the certificate authorities which I specify, and nothing else:
ssl_socket = ssl.wrap_socket(newsocket, server_side=True, certfile="my_cert.pem",
ca_certs=MY_TRUSTED_CAs, # <<< Only CAs specified here
cert_reqs=ssl.CERT_REQUIRED, ssl_version=ssl.PROTOCOL_TLSv1)

I've just run a few tests, and listing your selection of CAs in the ca_certs parameters is exactly what you need.
The system I've tried it on is Linux with Python 2.6. If you don't use ca_certs, it doesn't let you use cert_reqs=ssl.CERT_REQUIRED:
Traceback (most recent call last):
File "sockettest.py", line 18, in <module>
cert_reqs=ssl.CERT_REQUIRED, ssl_version=ssl.PROTOCOL_TLSv1)
File "/usr/lib/python2.6/ssl.py", line 350, in wrap_socket
suppress_ragged_eofs=suppress_ragged_eofs)
File "/usr/lib/python2.6/ssl.py", line 113, in __init__
cert_reqs, ssl_version, ca_certs)
ssl.SSLError: _ssl.c:317: No root certificates specified for verification of other-side certificates.
I've also tried to use a client to send a certificate that's not from a CA in the ca_certs parameter, and I get ssl_error_unknown_ca_alert (as expected).
Note that either way, there's no client-certificate CA list send (in the certificate_authorities list in the CertificateRequest TLS message), but that wouldn't be required. It's only useful to help the client choose the certificate.

Related

Python Error SSL: ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:726)

i'm trying to get the SNI from a (IP, destport 443) with python 2.7.15 and i'm using the last version of OpenSSL and the ssl module.
Here there is my code:
import OpenSSL as OSsl #This two modules are imported for the only purpose of getting the SNI using function defined by us down here getSNI
import ssl
ip = "52.85.25.17"
dport = "443"
#With this function we get the Server Name Identification for the trasmissions with Secure Socket Layer identified by the port 443. We only care about the destinationIP and the destinationPort
def getSNI(ip, dport):
if dport != "443":
commonName = "Not SSL"
print commonName
else:
server_certificate = ssl.get_server_certificate((ip, dport))
x509 = OSsl.crypto.load_certificate(OSsl.crypto.FILETYPE_PEM, server_certificate) #x509 is referred to the standard used for PKI (Public Key Infrastructure) used in this case for ciphering our informations about certificate
#FILETYPE_PEM serializes data to a Base64-Encoded
#getting the informations about Certificate
certInfo = x509.get_subject()
commonName = certInfo.commonName
print (commonName)
return commonName
getSNI(ip,dport)
This works, but for the address specified (in the snippet of code i posted here) I get this error:
Traceback (most recent call last):
File "getSNI.py", line 31, in <module>
getSNI(ip,dport)
File "getSNI.py", line 17, in getSNI
server_certificate = ssl.get_server_certificate((ip, dport))
File "/usr/lib/python2.7/ssl.py", line 1023, in get_server_certificate
with closing(context.wrap_socket(sock)) as sslsock:
File "/usr/lib/python2.7/ssl.py", line 369, in wrap_socket
_context=self)
File "/usr/lib/python2.7/ssl.py", line 617, in __init__
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 846, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:726)
I have all modules and packages upgraded, i read a lot of issues about this topic and i don't know how to solve this problem
Edit1: Executing whois I found that this IpAddress is connected to Amazon, so are there any particular issues about Amazon and SNI?
The point of SNI is that there could exists several domains which are resolved to concrete IP address. So, the IP you provided (52.85.25.17) is the one of such addresses. Server couldn't decide which domain's cert you are requesting, so it terminates the connection with error.
Appendix 1. Catching SSLError exceptions
You can catch ssl.SSLError this way:
try:
server_certificate = ssl.get_server_certificate((ip, dport))
...
except ssl.SSLError as e:
common_name = "Handshake Failed"

How i could to make SSl Connection server using username, password by python?

Hello I would ask if i could do connection to extract some data from Tool Server.
so i making this code but i need to modify it to open this tool with username and password and extract data from My tool server.
import socket
import ssl
HOST, PORT = '10.74.159.82', 31039
def handle(conn):
conn.write(b'GET / HTTP/1.1\n')
print(conn.recv() . decode())
def main():
sock = socket.socket(socket.AF_INET)
context = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
conn = context.wrap_socket(sock, server_hostname=HOST)
try:
conn.connect((HOST, PORT))
handle(conn)
finally:
conn.close()
if __name__ == '__main__':
main()
RESULT
!! C:\Users\Admin\.PyCharmCE2018.1\config\venv\Scripts\python.exe!! C:/Users/Admin/.PyCharmCE2018.1/config/codestyles/Under_Building.py
Traceback (most recent call last):
File "C:/Users/Admin/.PyCharmCE2018.1/config/codestyles/Under_Building.py", line 22, in <module>
main()
File "C:/Users/Admin/.PyCharmCE2018.1/config/codestyles/Under_Building.py", line 16, in main
conn.connect((HOST, PORT))
File "C:\Users\Admin\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 1141, in connect
self._real_connect(addr, False)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 1132, in _real_connect
self.do_handshake()
File "C:\Users\Admin\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 1108, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1045)
Process finished with exit code 1
I'm Beginner so i need to learn and supporting
It's hard to see exactly, but it seems the server you're connecting to (on https://10.74.159.82:31039) is using a self-signed certificate (or its CA is, perhaps).
It's perhaps better to create or buy a proper (non-self-signed) certificate, but assuming this is a test server this is fine.
So with what you have, your best bet is to allow self-signed certs in your SSL Context (and also, not to check the hostname - that will probably fail too as you're using an IP address):
context = ssl.create_default_context()
context.check_hostname=False
context.verify_mode=ssl.CERT_NONE
# etc...

Add client certificate and SSL to python app calling Elasticsearch

So I have an Elasticsearch cluster that I have been accessing from .Net web services via a ACL protected http endpoint. I have spun up a new reverse proxy that uses SSL and requires a client certificate to forward the request on to the (now protected) Elasticsearch cluster.
I have this all working fine in .Net, but I have one service that I inherited that running Python. I have not worked with Python much, and am struggling to figure out how to implement this.
I am using ES 2.3.2 and Python 3.2.
For .Net, I have the certificate installed on the hosting machine where the code pulls it directly from the store. I have used OpenSSL and exported this into a .pem (as well as .crt and .key as well). The problems I am encountering are A) how to get the transport to be https instead of http, and B) how to include my cert. I first tried:
client = Elasticsearch(hosts=[self.host], timeout=80, use_ssl=True, verify_certs=True, client_cert=cert_file_path, client_key=key_file_path)
This does not seem to work. So I figured I needed to build up my own connection/transport, so I tried this:
connection = Urllib3HttpConnection(host=self.host['host'], port=self.host['port'], use_ssl=True,ca_certs=cert_file_path, client_cert=cert_file_path, client_key=key_file_path, verify_certs=True )
transport = Transport(hosts=[self.host], connection_class=connection)
client = Elasticsearch(hosts=[self.host], timeout=80, use_ssl=True, verify_certs=True, Transport=transport, client_cert=cert_file_path, client_key=key_file_path)
I need to manually in the debugger change the scheme to 'https', but when I run now, I get:
TypeError("'Urllib3HttpConnection' object is not callable",)
In .Net what I am doing is overriding the HttpConnection in the ConnectionSetings for the ElasticClient. In it, I override CreateHttpWebRequest and add my certificate to the the outbound request:
// add the cert to the collection
webRequest.ClientCertificates.Add(this.clientCertificate);
It seems this should be a common thing to do, but I can't figure it out (though that could be due to my lack of python experience).
Any help out there?
Thanks!
~john
UPDATE
So here is the definition of my host:
{"use_ssl": True, "host": "[my server].cloudapp.net", "port": "443", "scheme": "https", "transport_schema": "https"}
From the documentation:
kwargs – any additional arguments will be passed on to the Transport class and, subsequently, to the Connection instances.
I would assume that the scheme and/or transport_schema would be forwarded on through.
I create the client as:
client = Elasticsearch(hosts=[self.host], timeout=80, use_ssl=True, scheme="https", verify_certs=True, client_cert=cert_file_path, client_key=key_file_path)
When I create the client, I get:
ERROR Exception encountered. Detailed info: ImproperlyConfigured('Scheme specified in connection (https) is not the same as the connection class (Urllib3HttpConnection) specifies (http).',)
This is happening in transport.py in _create_connection():
if 'scheme' in host and host['scheme'] != self.connection_class.transport_schema:
raise ImproperlyConfigured(
'Scheme specified in connection (%s) is not the same as the connection class (%s) specifies (%s).' % (
host['scheme'], self.connection_class.__name__, self.connection_class.transport_schema
))
When I break here, I can see that:
self.kwargs['transport_schema'] = 'https'
but:
self.connection_class.transport_schema = 'http'
I would have thought that the kwarg would have been forwarded on to the connection_class. I manually set the transport_schema to 'https' and this allows me to get past this error.
So that is the first part. After doing this, I get a certificate validation error. This is probably resolveable, I just haven't been through to this part yet. The problems here are:
A) I am prompted for a pass phrase. How can I supply this? This is a service that will be running unattended.
B) Anyway, when I supply it, I get:
GET https://[my server].cloudapp.net:443/ocv_int/_search?request_cache=true [status:N/A request:6.085s]
Traceback (most recent call last):
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connectionpool.py", line 578, in urlopen
chunked=chunked)
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connectionpool.py", line 351, in _make_request
self._validate_conn(conn)
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connectionpool.py", line 814, in _validate_conn
conn.connect()
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connection.py", line 289, in connect
ssl_version=resolved_ssl_version)
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\util\ssl_.py", line 308, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Users\jthoni\AppData\Local\Continuum\Anaconda3-2-3\lib\ssl.py", line 365, in wrap_socket
_context=self)
File "C:\Users\jthoni\AppData\Local\Continuum\Anaconda3-2-3\lib\ssl.py", line 583, in init
self.do_handshake()
File "C:\Users\jthoni\AppData\Local\Continuum\Anaconda3-2-3\lib\ssl.py", line 810, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 94, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connectionpool.py", line 604, in urlopen
raise SSLError(e)
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
P.S. to get my cert files I did the following with my pfx file:
openssl pkcs12 -in E:\[myCert].pfx -out E:\[myCert].pem -nodes
openssl pkcs12 -in E:\[myCert].pfx -nocerts -out E:\[myCert].key
openssl pkcs12 -in E:\[myCert].pfx -clcerts -nokeys -out E:\[myCert].crt

What's wrong with my Python attempt at sending mail via SMTP through Gmail? The SSL wrapper doesn't seem to work

I have the following pretty simple Python code, derived from here:
import ssl
import base64
from socket import *
cc = socket(AF_INET, SOCK_STREAM)
cc.connect(("smtp.gmail.com", 587))
cc.send('helo tester.com\r\n')
cc.send('starttls\r\n')
scc = ssl.wrap_socket(cc, ssl_version=ssl.PROTOCOL_SSLv23)
scc.send('AUTH LOGIN\r\n')
scc.send(base64.b64encode('myBase64EncodedUsernameAndPassword==')+'\r\n')
scc.send
However upon running it I get:
Traceback (most recent call last):
File "Untitled 2.py", line 16, in <module>
scc = ssl.wrap_socket(cc, ssl_version=ssl.PROTOCOL_SSLv23)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 387, in wrap_socket
ciphers=ciphers)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 143, in __init__
self.do_handshake()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 305, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:507: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
What am I doing wrong?
You blindly send data to the server without making any attempts to read and check the answer from the server. But this is required, because only after a successful response to the STARTTLS command you can upgrade the socket to SSL.
Since you don't read the responses from the server and try to upgrade the socket blindly to SSL it will start the SSL handshake by sending the ClientHello but then croak with unknown protocol because instead of reading the ServerHello it will read the plain text data from the server which you were supposed to read earlier.
For details on how to correctly handle SMTP see the standard, that is RFC 5321 and RFC 3207.

Python SSL example from docs gives "Connection reset by peer" error

I'm trying to run the example code provided in the documentation for the ssl module here: http://docs.python.org/2/library/ssl.html#client-side-operation
The server-side code is similar to the example given in the documentation, and it throws this exception:
Traceback (most recent call last):
File "serve.py", line 16, in <module>
ssl_version=ssl.PROTOCOL_TLSv1)
File "/usr/lib/python2.7/ssl.py", line 381, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 143, in __init__
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
self._sslobj.do_handshake()
socket.error: [Errno 104] Connection reset by peer
And the client-side code, also similar to the example in the documentation, throws this exception:
Traceback (most recent call last):
File "client.py", line 8, in <module>
ssl_sock.connect((host, port))
File "/usr/lib/python2.7/ssl.py", line 331, in connect
self._real_connect(addr, False)
File "/usr/lib/python2.7/ssl.py", line 324, in _real_connect
raise e
socket.error: [Errno 104] Connection reset by peer
As far as I can see, I've copied the examples provided in the documentation quite closely, so I don't know what the problem is. All of my TCP, UDP and ICMP ports are open, so I don't think it is a firewall issue.
(I've edited this question to cut out my code for brevity, as it really is quite similar to the example provided in the link. If you want to see my code, look at the history of this question.)
I found the problem. I generated the private key and the certificate using command like this:
$ openssl req -new -x509 -days 365 -nodes -out cert.pem -keyout key.pem
Generating a 1024 bit RSA private key
# ...
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:MyState
Locality Name (eg, city) []:Some City
Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Organization, Inc.
Organizational Unit Name (eg, section) []:My Group
Common Name (eg, YOUR name) []:myserver.mygroup.myorganization.com
Email Address []:ops#myserver.mygroup.myorganization.com
$
The crucial part is that the "common name" entered must match the domain name of the server. I thought that when cacerts is ssl.CERT_NONE, which it is by default for wrap_socket, this wouldn't be checked, but I was wrong. It's always checked. One night's sleep and it's the first thing I thought of to verify!
Hopefully this will be useful to someone else who gets this cryptic error message.
If this doesn't solve it, you might be suffering from deep packet inspection. I got this error again when I was on a university network, but not on any other network, and I'm fairly certain it was because of deep packet inspection.

Categories