I am trying to visit gateway.playneverwinter.com with splinter
from splinter import Browser
browser = Browser()
browser.visit('https://gateway.playneverwinter.com')
if browser.is_text_present('Neverwinter'):
print("Yes, we made it to the entrance of the Prime Material Plane!")
else:
print("Fumble")
browser.quit()
It fails with
File "gateway_bot.py", line 10, in <module>
browser.visit('https://gateway.playneverwinter.com')
File "/usr/local/lib/python3.4/dist-packages/splinter/driver/webdriver/__init__.py", line 53, in visit
self.connect(url)
File "/usr/local/lib/python3.4/dist-packages/splinter/request_handler/request_handler.py", line 23, in connect
self._create_connection()
File "/usr/local/lib/python3.4/dist-packages/splinter/request_handler/request_handler.py", line 53, in _create_connection
self.conn.endheaders()
File "/usr/lib/python3.4/http/client.py", line 1061, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.4/http/client.py", line 906, in _send_output
self.send(msg)
File "/usr/lib/python3.4/http/client.py", line 841, in send
self.connect()
File "/usr/lib/python3.4/http/client.py", line 1205, in connect
server_hostname=server_hostname)
File "/usr/lib/python3.4/ssl.py", line 364, in wrap_socket
_context=self)
File "/usr/lib/python3.4/ssl.py", line 578, in __init__
self.do_handshake()
File "/usr/lib/python3.4/ssl.py", line 805, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
Firefox is able to connect and browse this site without issue, tough. After some diagnostic
$ openssl s_client -connect gateway.playneverwinter.com:443
CONNECTED(00000003)
139745006343840:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
I found that it looked like a fixed issue in OpenSSL and that forcing either SSLv3 or TLSv1 allowed me to connect (and that I could then download the target with cURL) e.g. either of
openssl s_client -ssl3 -connect gateway.playneverwinter.com:443
openssl s_client -tls1 -connect gateway.playneverwinter.com:443
According to the comments in the OpenSSL ticket, I expect that the issue is on the server side, but as I do not have access to it, it is quite unhelpful. So, for a quick fix, is there a way to force splinter to use SSLv3 or TLSv1?
After looking into it, the only way I can think of doing that would to be to go into that client.py file and change the initialization of their ssl stuff.
Following #Natecat suggestion, I wrote a monkey patch to force SSLv3 when this error occurs
# Monkey patch splinter to force SSLv3 on `ssl.SSLEOFError`
from splinter import request_handler
import ssl
from http import client as http_client
_old_req = request_handler.request_handler.RequestHandler._create_connection
def _splinter_sslv3_patch(self):
try:
_old_req(self)
except ssl.SSLEOFError:
self.conn = http_client.HTTPSConnection(self.host, self.port,
context=ssl.SSLContext(ssl.PROTOCOL_SSLv3))
self.conn.putrequest('GET', self.path)
self.conn.putheader('User-agent', 'python/splinter')
if self.auth:
self.conn.putheader("Authorization", "Basic %s" % self.auth)
self.conn.endheaders()
request_handler.request_handler.RequestHandler._create_connection = _splinter_sslv3_patch
Related
My setup is Mac OS 10.13.6 (Updated), python 2.7.15 and I'm connected using a VPN (I needed to install the VPN certificate to decrypt HTTPs traffic).
The problem is I'm unable to connect to slack API
>>> import slackclient
>>> client = slackclient.SlackClient(myToken)
>>> client.server.rtm_connect()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/slackclient/server.py", line 131, in rtm_connect
reply = self.api_requester.do(self.token, connect_method, timeout=timeout, post_data=kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/slackclient/slackrequest.py", line 104, in do
proxies=self.proxies
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 112, in post
return request('post', url, data=data, json=json, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/adapters.py", line 511, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='slack.com', port=443): Max retries exceeded with url: /api/rtm.start (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),))
Versions are:
Python 2.7.15
certifi==2018.8.24
chardet==3.0.4
idna==2.7
jenkinsapi==0.3.6
pytz==2018.5
requests==2.19.1
setproctitle==1.1.10
six==1.11.0
slackclient==1.2.1
urllib3==1.23
websocket-client==0.51.0
I've found the problem.
From some reason, python does not use the certificates in my key chain but the ones in the openssl under /Library/Frameworks/Python.framework/Versions/2.7/etc/openssl/cert.pem
I've added the certificate at the bottom of file and the problem went away
Another solution,
In your Keychain access delete slack.com tokens
Another solution, add No cert option when slackclient invoke websocket-client
In file "c:/Python27/Lib/site-packages/slackclient/server.py"
add line 180, sslopt={'cert_reqs':0}
line 176: self.websocket = create_connection(ws_url,
line 177: http_proxy_host=proxy_host,
line 178: http_proxy_port=proxy_port,
line 179: http_proxy_auth=proxy_auth,
line 180: sslopt={'cert_reqs':0})
I'm trying to send apple push notifications with django-ios-notifications app, but i've faced with WantReadError.
Here is a stacktrace:
File ".../local/lib/python2.7/site-packages/ios_notifications/models.py", line 110, in push_notification_to_devices
self._write_message(notification, devices, chunk_size)
File ".../local/lib/python2.7/site-packages/ios_notifications/models.py", line 132, in _write_message
self._connect()
File ".../local/lib/python2.7/site-packages/ios_notifications/models.py", line 100, in _connect
return super(APNService, self)._connect(self.certificate, self.private_key, self.passphrase)
File ".../local/lib/python2.7/site-packages/ios_notifications/models.py", line 64, in _connect
self.connection.do_handshake()
File ".../local/lib/python2.7/site-packages/OpenSSL/SSL.py", line 1076, in do_handshake
self._raise_ssl_error(self._ssl, result)
File ".../local/lib/python2.7/site-packages/OpenSSL/SSL.py", line 847, in _raise_ssl_error
raise WantReadError()
WantReadError
I have no idea how to fix it.
I've created required APNService with correct Certificate and Private key, hostname is gateway.sandbox.push.apple.com.
Command like this
openssl s_client -CApath /etc/ssl/certs/ -connect gateway.sandbox.push.apple.com:2195 -cert my.pem works fine in Ubuntu, it returns
Start Time: 1458724825
Timeout : 300 (sec)
Verify return code: 0 (ok)
UPDATE: problem was solved.
According to https://github.com/stephenmuss/django-ios-notifications/issues/11 it's required to install https://github.com/mjs/gevent_openssl
On Windows Vista SP2 + Python 2.7.10 I can connect to https://www.python.org, but not to https://codereview.appspot.com
The script:
HOST1 = 'https://www.python.org'
HOST2 = 'https://codereview.appspot.com'
import urllib2
print HOST1
urllib2.urlopen(HOST1)
print HOST2
urllib2.urlopen(HOST2)
And the output:
E:\>py test.py
https://www.python.org
https://codereview.appspot.com
Traceback (most recent call last):
File "test.py", line 9, in <module>
urllib2.urlopen(HOST2)
File "C:\Python27\lib\urllib2.py", line 158, in urlopen
return opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 435, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 453, in _open
'_open', req)
File "C:\Python27\lib\urllib2.py", line 413, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 1244, in https_open
context=self._context)
File "C:\Python27\lib\urllib2.py", line 1201, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
How can I troubleshoot, what exactly is wrong with https://codereview.appspot.com/ ?
My guess is that it is related to the alternative chain handling in OpenSSL, as described in detail in Python Urllib2 SSL error. Although Python uses the windows CA store to get the trusted root certificates the validation of the trust chain itself is done within OpenSSL.
According to "Python 2.7.10 Released" Python 2.7.10 on Windows includes OpenSSL 1.0.2a but the fixes regarding alternative chains were done in 1.0.2b only (and had to be fixed fast afterwards because they contained a serious security bug).
If you look at the SSLLabs report for codereview.appspot.com you can see that there are multiple trust chains which probably causes the problem. Contrary to that python.org only has a single trust chain.
To work around the problem it might be necessary to use your own root CA store which must contain the certificate for "/C=US/O=Equifax/OU=Equifax Secure Certificate Authority" to verify codereview.appspot.com correctly. The certificate can be found here and you can give it with the cafile parameter to urllib2.urlopen.
I try to make a local HTTPS connection to a XMLRPC api. Since I upgrade to python 2.7.9 that enable by default certificates verification, I got a CERTIFICATE_VERIFY_FAILED error when I use my API
>>> test=xmlrpclib.ServerProxy('https://admin:bz15h9v9n#localhost:9999/API',verbose=False, use_datetime=True)
>>> test.list_satellites()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/xmlrpclib.py", line 1233, in __call__
return self.__send(self.__name, args)
File "/usr/local/lib/python2.7/xmlrpclib.py", line 1591, in __request
verbose=self.__verbose
File "/usr/local/lib/python2.7/xmlrpclib.py", line 1273, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/local/lib/python2.7/xmlrpclib.py", line 1301, in single_request
self.send_content(h, request_body)
File "/usr/local/lib/python2.7/xmlrpclib.py", line 1448, in send_content
connection.endheaders(request_body)
File "/usr/local/lib/python2.7/httplib.py", line 997, in endheaders
self._send_output(message_body)
File "/usr/local/lib/python2.7/httplib.py", line 850, in _send_output
self.send(msg)
File "/usr/local/lib/python2.7/httplib.py", line 812, in send
self.connect()
File "/usr/local/lib/python2.7/httplib.py", line 1212, in connect
server_hostname=server_hostname)
File "/usr/local/lib/python2.7/ssl.py", line 350, in wrap_socket
_context=self)
File "/usr/local/lib/python2.7/ssl.py", line 566, in __init__
self.do_handshake()
File "/usr/local/lib/python2.7/ssl.py", line 788, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
>>> import ssl
>>> ssl._create_default_https_context = ssl._create_unverified_context
>>> test.list_satellites()
[{'paired': True, 'serial': '...', 'enabled': True, 'id': 1, 'date_paired': datetime.datetime(2015, 5, 26, 16, 17, 6)}]
Does exists a pythonic way to disable default certificate verification in python 2.7.9 ?
I don't realy know if it's good to change "private" global SSL attribute (ssl._create_default_https_context = ssl._create_unverified_context)
You have to provide an unverified SSL context, constructed by hand or using the private function _create_unverified_context() from ssl module:
import xmlrpclib
import ssl
test = xmlrpclib.ServerProxy('https://admin:bz15h9v9n#localhost:9999/API',
verbose=False, use_datetime=True,
context=ssl._create_unverified_context())
test.list_satellites()
Note: this code only works with python >= 2.7.9 (contextparameter was added in Python 2.7.9)
If you want to have a code compatible with previous Python version, you have to use the transport parameter:
import xmlrpclib
import ssl
context = hasattr(ssl, '_create_unverified_context') and ssl._create_unverified_context() \
or None
test = xmlrpclib.ServerProxy('https://admin:bz15h9v9n#localhost:9999/API',
verbose=False, use_datetime=True,
transport=xmlrpclib.SafeTransport(use_datetime=True,
context=context))
test.list_satellites()
It's possible to disable verification using the public ssl APIs existing on Python 2.7.9+:
import xmlrpclib
import ssl
ssl_ctx = ssl.create_default_context()
ssl_ctx.check_hostname = False
ssl_ctx.verify_mode = ssl.CERT_NONE
test = xmlrpclib.ServerProxy('https://admin:bz15h9v9n#localhost:9999/API',
verbose=False, use_datetime=True,
context=ssl_ctx)
test.list_satellites()
I think another way to disable certificate verification could be:
import xmlrpclib
import ssl
s=ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
s.verify_mode=ssl.CERT_NONE
test=xmlrpclib.Server('https://admin:bz15h9v9n#localhost:9999/API',verbose=0,context=s)
With Python 2.6.6 for example:
s = xmlrpclib.ServerProxy('https://admin:bz15h9v9n#localhost:9999/API', transport=None, encoding=None, verbose=0,allow_none=0, use_datetime=0)
It works for me...
I have the following pretty simple Python code, derived from here:
import ssl
import base64
from socket import *
cc = socket(AF_INET, SOCK_STREAM)
cc.connect(("smtp.gmail.com", 587))
cc.send('helo tester.com\r\n')
cc.send('starttls\r\n')
scc = ssl.wrap_socket(cc, ssl_version=ssl.PROTOCOL_SSLv23)
scc.send('AUTH LOGIN\r\n')
scc.send(base64.b64encode('myBase64EncodedUsernameAndPassword==')+'\r\n')
scc.send
However upon running it I get:
Traceback (most recent call last):
File "Untitled 2.py", line 16, in <module>
scc = ssl.wrap_socket(cc, ssl_version=ssl.PROTOCOL_SSLv23)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 387, in wrap_socket
ciphers=ciphers)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 143, in __init__
self.do_handshake()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 305, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:507: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
What am I doing wrong?
You blindly send data to the server without making any attempts to read and check the answer from the server. But this is required, because only after a successful response to the STARTTLS command you can upgrade the socket to SSL.
Since you don't read the responses from the server and try to upgrade the socket blindly to SSL it will start the SSL handshake by sending the ClientHello but then croak with unknown protocol because instead of reading the ServerHello it will read the plain text data from the server which you were supposed to read earlier.
For details on how to correctly handle SMTP see the standard, that is RFC 5321 and RFC 3207.