I'm trying to run the example code provided in the documentation for the ssl module here: http://docs.python.org/2/library/ssl.html#client-side-operation
The server-side code is similar to the example given in the documentation, and it throws this exception:
Traceback (most recent call last):
File "serve.py", line 16, in <module>
ssl_version=ssl.PROTOCOL_TLSv1)
File "/usr/lib/python2.7/ssl.py", line 381, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 143, in __init__
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
self._sslobj.do_handshake()
socket.error: [Errno 104] Connection reset by peer
And the client-side code, also similar to the example in the documentation, throws this exception:
Traceback (most recent call last):
File "client.py", line 8, in <module>
ssl_sock.connect((host, port))
File "/usr/lib/python2.7/ssl.py", line 331, in connect
self._real_connect(addr, False)
File "/usr/lib/python2.7/ssl.py", line 324, in _real_connect
raise e
socket.error: [Errno 104] Connection reset by peer
As far as I can see, I've copied the examples provided in the documentation quite closely, so I don't know what the problem is. All of my TCP, UDP and ICMP ports are open, so I don't think it is a firewall issue.
(I've edited this question to cut out my code for brevity, as it really is quite similar to the example provided in the link. If you want to see my code, look at the history of this question.)
I found the problem. I generated the private key and the certificate using command like this:
$ openssl req -new -x509 -days 365 -nodes -out cert.pem -keyout key.pem
Generating a 1024 bit RSA private key
# ...
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:MyState
Locality Name (eg, city) []:Some City
Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Organization, Inc.
Organizational Unit Name (eg, section) []:My Group
Common Name (eg, YOUR name) []:myserver.mygroup.myorganization.com
Email Address []:ops#myserver.mygroup.myorganization.com
$
The crucial part is that the "common name" entered must match the domain name of the server. I thought that when cacerts is ssl.CERT_NONE, which it is by default for wrap_socket, this wouldn't be checked, but I was wrong. It's always checked. One night's sleep and it's the first thing I thought of to verify!
Hopefully this will be useful to someone else who gets this cryptic error message.
If this doesn't solve it, you might be suffering from deep packet inspection. I got this error again when I was on a university network, but not on any other network, and I'm fairly certain it was because of deep packet inspection.
Related
i'm trying to get the SNI from a (IP, destport 443) with python 2.7.15 and i'm using the last version of OpenSSL and the ssl module.
Here there is my code:
import OpenSSL as OSsl #This two modules are imported for the only purpose of getting the SNI using function defined by us down here getSNI
import ssl
ip = "52.85.25.17"
dport = "443"
#With this function we get the Server Name Identification for the trasmissions with Secure Socket Layer identified by the port 443. We only care about the destinationIP and the destinationPort
def getSNI(ip, dport):
if dport != "443":
commonName = "Not SSL"
print commonName
else:
server_certificate = ssl.get_server_certificate((ip, dport))
x509 = OSsl.crypto.load_certificate(OSsl.crypto.FILETYPE_PEM, server_certificate) #x509 is referred to the standard used for PKI (Public Key Infrastructure) used in this case for ciphering our informations about certificate
#FILETYPE_PEM serializes data to a Base64-Encoded
#getting the informations about Certificate
certInfo = x509.get_subject()
commonName = certInfo.commonName
print (commonName)
return commonName
getSNI(ip,dport)
This works, but for the address specified (in the snippet of code i posted here) I get this error:
Traceback (most recent call last):
File "getSNI.py", line 31, in <module>
getSNI(ip,dport)
File "getSNI.py", line 17, in getSNI
server_certificate = ssl.get_server_certificate((ip, dport))
File "/usr/lib/python2.7/ssl.py", line 1023, in get_server_certificate
with closing(context.wrap_socket(sock)) as sslsock:
File "/usr/lib/python2.7/ssl.py", line 369, in wrap_socket
_context=self)
File "/usr/lib/python2.7/ssl.py", line 617, in __init__
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 846, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:726)
I have all modules and packages upgraded, i read a lot of issues about this topic and i don't know how to solve this problem
Edit1: Executing whois I found that this IpAddress is connected to Amazon, so are there any particular issues about Amazon and SNI?
The point of SNI is that there could exists several domains which are resolved to concrete IP address. So, the IP you provided (52.85.25.17) is the one of such addresses. Server couldn't decide which domain's cert you are requesting, so it terminates the connection with error.
Appendix 1. Catching SSLError exceptions
You can catch ssl.SSLError this way:
try:
server_certificate = ssl.get_server_certificate((ip, dport))
...
except ssl.SSLError as e:
common_name = "Handshake Failed"
So I have an Elasticsearch cluster that I have been accessing from .Net web services via a ACL protected http endpoint. I have spun up a new reverse proxy that uses SSL and requires a client certificate to forward the request on to the (now protected) Elasticsearch cluster.
I have this all working fine in .Net, but I have one service that I inherited that running Python. I have not worked with Python much, and am struggling to figure out how to implement this.
I am using ES 2.3.2 and Python 3.2.
For .Net, I have the certificate installed on the hosting machine where the code pulls it directly from the store. I have used OpenSSL and exported this into a .pem (as well as .crt and .key as well). The problems I am encountering are A) how to get the transport to be https instead of http, and B) how to include my cert. I first tried:
client = Elasticsearch(hosts=[self.host], timeout=80, use_ssl=True, verify_certs=True, client_cert=cert_file_path, client_key=key_file_path)
This does not seem to work. So I figured I needed to build up my own connection/transport, so I tried this:
connection = Urllib3HttpConnection(host=self.host['host'], port=self.host['port'], use_ssl=True,ca_certs=cert_file_path, client_cert=cert_file_path, client_key=key_file_path, verify_certs=True )
transport = Transport(hosts=[self.host], connection_class=connection)
client = Elasticsearch(hosts=[self.host], timeout=80, use_ssl=True, verify_certs=True, Transport=transport, client_cert=cert_file_path, client_key=key_file_path)
I need to manually in the debugger change the scheme to 'https', but when I run now, I get:
TypeError("'Urllib3HttpConnection' object is not callable",)
In .Net what I am doing is overriding the HttpConnection in the ConnectionSetings for the ElasticClient. In it, I override CreateHttpWebRequest and add my certificate to the the outbound request:
// add the cert to the collection
webRequest.ClientCertificates.Add(this.clientCertificate);
It seems this should be a common thing to do, but I can't figure it out (though that could be due to my lack of python experience).
Any help out there?
Thanks!
~john
UPDATE
So here is the definition of my host:
{"use_ssl": True, "host": "[my server].cloudapp.net", "port": "443", "scheme": "https", "transport_schema": "https"}
From the documentation:
kwargs – any additional arguments will be passed on to the Transport class and, subsequently, to the Connection instances.
I would assume that the scheme and/or transport_schema would be forwarded on through.
I create the client as:
client = Elasticsearch(hosts=[self.host], timeout=80, use_ssl=True, scheme="https", verify_certs=True, client_cert=cert_file_path, client_key=key_file_path)
When I create the client, I get:
ERROR Exception encountered. Detailed info: ImproperlyConfigured('Scheme specified in connection (https) is not the same as the connection class (Urllib3HttpConnection) specifies (http).',)
This is happening in transport.py in _create_connection():
if 'scheme' in host and host['scheme'] != self.connection_class.transport_schema:
raise ImproperlyConfigured(
'Scheme specified in connection (%s) is not the same as the connection class (%s) specifies (%s).' % (
host['scheme'], self.connection_class.__name__, self.connection_class.transport_schema
))
When I break here, I can see that:
self.kwargs['transport_schema'] = 'https'
but:
self.connection_class.transport_schema = 'http'
I would have thought that the kwarg would have been forwarded on to the connection_class. I manually set the transport_schema to 'https' and this allows me to get past this error.
So that is the first part. After doing this, I get a certificate validation error. This is probably resolveable, I just haven't been through to this part yet. The problems here are:
A) I am prompted for a pass phrase. How can I supply this? This is a service that will be running unattended.
B) Anyway, when I supply it, I get:
GET https://[my server].cloudapp.net:443/ocv_int/_search?request_cache=true [status:N/A request:6.085s]
Traceback (most recent call last):
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connectionpool.py", line 578, in urlopen
chunked=chunked)
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connectionpool.py", line 351, in _make_request
self._validate_conn(conn)
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connectionpool.py", line 814, in _validate_conn
conn.connect()
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connection.py", line 289, in connect
ssl_version=resolved_ssl_version)
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\util\ssl_.py", line 308, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Users\jthoni\AppData\Local\Continuum\Anaconda3-2-3\lib\ssl.py", line 365, in wrap_socket
_context=self)
File "C:\Users\jthoni\AppData\Local\Continuum\Anaconda3-2-3\lib\ssl.py", line 583, in init
self.do_handshake()
File "C:\Users\jthoni\AppData\Local\Continuum\Anaconda3-2-3\lib\ssl.py", line 810, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 94, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "E:\Source\Repos\OfficeCustomerVoice\UserExperience\OCVClusteringWebAPI\env3\lib\site-packages\urllib3\connectionpool.py", line 604, in urlopen
raise SSLError(e)
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
P.S. to get my cert files I did the following with my pfx file:
openssl pkcs12 -in E:\[myCert].pfx -out E:\[myCert].pem -nodes
openssl pkcs12 -in E:\[myCert].pfx -nocerts -out E:\[myCert].key
openssl pkcs12 -in E:\[myCert].pfx -clcerts -nokeys -out E:\[myCert].crt
I have the following pretty simple Python code, derived from here:
import ssl
import base64
from socket import *
cc = socket(AF_INET, SOCK_STREAM)
cc.connect(("smtp.gmail.com", 587))
cc.send('helo tester.com\r\n')
cc.send('starttls\r\n')
scc = ssl.wrap_socket(cc, ssl_version=ssl.PROTOCOL_SSLv23)
scc.send('AUTH LOGIN\r\n')
scc.send(base64.b64encode('myBase64EncodedUsernameAndPassword==')+'\r\n')
scc.send
However upon running it I get:
Traceback (most recent call last):
File "Untitled 2.py", line 16, in <module>
scc = ssl.wrap_socket(cc, ssl_version=ssl.PROTOCOL_SSLv23)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 387, in wrap_socket
ciphers=ciphers)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 143, in __init__
self.do_handshake()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 305, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:507: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
What am I doing wrong?
You blindly send data to the server without making any attempts to read and check the answer from the server. But this is required, because only after a successful response to the STARTTLS command you can upgrade the socket to SSL.
Since you don't read the responses from the server and try to upgrade the socket blindly to SSL it will start the SSL handshake by sending the ClientHello but then croak with unknown protocol because instead of reading the ServerHello it will read the plain text data from the server which you were supposed to read earlier.
For details on how to correctly handle SMTP see the standard, that is RFC 5321 and RFC 3207.
How can I prevent OpenSSL (specifically, Python's ssl module) from using system certificate authorities?
In other words, I would like it to trust only the certificate authorities which I specify, and nothing else:
ssl_socket = ssl.wrap_socket(newsocket, server_side=True, certfile="my_cert.pem",
ca_certs=MY_TRUSTED_CAs, # <<< Only CAs specified here
cert_reqs=ssl.CERT_REQUIRED, ssl_version=ssl.PROTOCOL_TLSv1)
I've just run a few tests, and listing your selection of CAs in the ca_certs parameters is exactly what you need.
The system I've tried it on is Linux with Python 2.6. If you don't use ca_certs, it doesn't let you use cert_reqs=ssl.CERT_REQUIRED:
Traceback (most recent call last):
File "sockettest.py", line 18, in <module>
cert_reqs=ssl.CERT_REQUIRED, ssl_version=ssl.PROTOCOL_TLSv1)
File "/usr/lib/python2.6/ssl.py", line 350, in wrap_socket
suppress_ragged_eofs=suppress_ragged_eofs)
File "/usr/lib/python2.6/ssl.py", line 113, in __init__
cert_reqs, ssl_version, ca_certs)
ssl.SSLError: _ssl.c:317: No root certificates specified for verification of other-side certificates.
I've also tried to use a client to send a certificate that's not from a CA in the ca_certs parameter, and I get ssl_error_unknown_ca_alert (as expected).
Note that either way, there's no client-certificate CA list send (in the certificate_authorities list in the CertificateRequest TLS message), but that wouldn't be required. It's only useful to help the client choose the certificate.
I need to set the value of a field in an XML file which exists on a remote Linux box. How do I find out which port I should connect to ?
But even a proper ping is not happening:
import xmlrpclib
server = xmlrpclib.ServerProxy('http://10.77.21.240:9000')
print server.ping()
print "I'm in hurray"
bUT instead I got:
Traceback (most recent call last):
File "ping.py", line 3, in <module>
print server.ping()
File "C:\Python26\lib\xmlrpclib.py", line 1199, in __call__
return self.__send(self.__name, args)
File "C:\Python26\lib\xmlrpclib.py", line 1489, in __request
verbose=self.__verbose
File "C:\Python26\lib\xmlrpclib.py", line 1235, in request
self.send_content(h, request_body)
File "C:\Python26\lib\xmlrpclib.py", line 1349, in send_content
connection.endheaders()
File "C:\Python26\lib\httplib.py", line 892, in endheaders
self._send_output()
File "C:\Python26\lib\httplib.py", line 764, in _send_output
self.send(msg)
File "C:\Python26\lib\httplib.py", line 723, in send
self.connect()
File "C:\Python26\lib\httplib.py", line 704, in connect
self.timeout)
File "C:\Python26\lib\socket.py", line 514, in create_connection
raise error, msg
socket.error: [Errno 10061] No connection could be made because the target machine actively refused it.
What did I do wrong?
A couple of things to try / think about:
Go to a command prompt on the remote host and type "netstat -nap | grep 9000". If you don't get back something interesting it means that nothing is running at port 9000.
You show the remote host at 10.77.21.240. This is an unroutable address on the net (AKA Private Network), so is the server itself (not just your app) pingable? If you are on windows, goto Start -> Run and type "cmd". At the prompt type, "ping 10.77.21.240" and see what you get.
One more thought: the process may be up and running at 9000 on a reachable host, but it may have opened the port as 127.0.0.1:9000 instead of 0.0.0.0:9000. The first address will only be reachable by processes on the same machine, the second one will open the port on all available IP addresses the machine has.
Update in response to comment: The fact that it shouldn't be a problem doesn't eliminate the possibility it is. When you are debugging something that should be working, but isn't, you need to get fairly pedantic about checking each step, allowing yourself no room for 'Oh, I know that couldn't be the problem.' -- this is a verbal 'handwave' (often accompanied by a real handwave). You'd be surprised how often the problem exists in exactly the area you are handwaving! It takes 3 seconds to do the ping test. If it works, you move on, if it doesn't work ...
The first three steps in dealing with any system problem are:
Is it plugged in?
Is it turned on?
Is it configured properly?
And you have to do this for each and every piece of hardware/software in the food chain from your keyboard to the app. I'd guess 80% of 'sudden' failures are items 1 or 2 -- yes, really. Cables are a huge pain in the ass.
When on the phone with novices I normally start by going for the long pass -- if they can get news.google.com in a browser and then click on a random story, then I know that in general the network is OK. Why the news and why a random story? To sidestep browser cache issues. I've lost count of the number of times my older sister has called me up and announced "The Internet is broken!" The first thing we do is the google news test. 99% of the time it works, so I have her fire up reverse-WinVNC (UltraVNC's SingleClick is a God-send), I get on her machine, and then we see what the real problem is.
If the long pass doesn't work, then I see if they can get to their router. Etc. etc.