UPDATE: I managed to do a request with urllib2, but I'm still wondering what is happening here.
I would like to do a HTTPS request with Python.
This works fine with the requests module, but I don't want to use external dependencies, so I'd like to use the standard library.
httplib
When I follow this example I don't get a response. I get a timeout instead. I'm out of ideas as to what would cause this.
Code:
import requests
print requests.get('https://python.org')
from httplib import HTTPSConnection
conn = HTTPSConnection('www.python.org')
conn.request('GET', '/index.html')
print conn.getresponse()
Output:
<Response [200]>
Traceback (most recent call last):
File "test.py", line 6, in <module>
conn.request('GET', '/index.html')
File "C:\Python27\lib\httplib.py", line 1069, in request
self._send_request(method, url, body, headers)
File "C:\Python27\lib\httplib.py", line 1109, in _send_request
self.endheaders(body)
File "C:\Python27\lib\httplib.py", line 1065, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 892, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 854, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 1282, in connect
HTTPConnection.connect(self)
File "C:\Python27\lib\httplib.py", line 831, in connect
self.timeout, self.source_address)
File "C:\Python27\lib\socket.py", line 575, in create_connection
raise err
socket.error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
urllib
This fails for a different (but possibly related) reason. Code:
import urllib
print urllib.urlopen("https://python.org")
Output:
Traceback (most recent call last):
File "test.py", line 10, in <module>
print urllib.urlopen("https://python.org")
File "C:\Python27\lib\urllib.py", line 87, in urlopen
return opener.open(url)
File "C:\Python27\lib\urllib.py", line 215, in open
return getattr(self, name)(url)
File "C:\Python27\lib\urllib.py", line 445, in open_https
h.endheaders(data)
File "C:\Python27\lib\httplib.py", line 1065, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 892, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 854, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 1290, in connect
server_hostname=server_hostname)
File "C:\Python27\lib\ssl.py", line 369, in wrap_socket
_context=self)
File "C:\Python27\lib\ssl.py", line 599, in __init__
self.do_handshake()
File "C:\Python27\lib\ssl.py", line 828, in do_handshake
self._sslobj.do_handshake()
IOError: [Errno socket error] [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:727)
What is requests doing that makes it succeed where both of these libraries fail?
requests.get without timeout parameter mean no timeout at all.
httplib.HTTPSConnection accept parameter timeout in Python 2.6 and newer according to httplib docs. If your problem was caused by timeout, setting high enough timeout should help. Please try replacing:
conn = HTTPSConnection('www.python.org')
with:
conn = HTTPSConnection('www.python.org', timeout=300)
which will give 300 seconds (5 minutes) for processing.
I'm moving my bot from my old, broken laptop to a proper VPS. I'm on an older async version of Discord.py (0.16.0), because I started work on this thing a long time before the rewrite was a thing; and I don't have much experience with Linux, so moving to a Windows Server seemed prudent. I've installed all of the same packages (as far as I know), but I'm consistently getting an error on startup, on the bot.run() line:
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)
The error happens during the handshake, so far as I can tell from the stacktrace, not that I'd know what the implications of that are.
This setup has worked for several years on two different laptops (albeit running Windows 10) without any issues, and the discord.py server doesn't seem to have encountered such issues on a Windows machine (it happens on OS X, but because of some idiosyncrasy of Python 3.6, and this was also happening in Python 3.5. Wrong OS, too.) I tried changing Python installs and installing some of the automatic Windows updates, along with running some commands I found here and there: pip install certifi and pip install incremental.
After several hours of nothing out of curiosity: and frustration and owing to the fact that I have a test bot on one server that I don't mind being trashed, I tried from something I saw in a stackoverflow discussion
bot = commands.Bot(
command_prefix='/',
connector=aiohttp.TCPConnector(verify_ssl=False)
)
(the added line is specifically the whole connector kwarg, which was not there before.)
The odd thing is, this CHANGED the stacktrace, but didn't actually change the error, as shown below:
Before adding verify_ssl = False, I was getting...
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 601, in _create_direct_connection
local_addr=self._local_addr)
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 802, in create_connection
sock, protocol_factory, ssl, server_hostname)
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 828, in _create_connection_transport
yield from waiter
File "C:\Program Files\Python36\lib\asyncio\sslproto.py", line 503, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "C:\Program Files\Python36\lib\asyncio\sslproto.py", line 201, in feed_ssldata
self._sslobj.do_handshake()
File "C:\Program Files\Python36\lib\ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 304, in connect
yield from self._create_connection(req)
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 578, in _create_connection
transport, proto = yield from self._create_direct_connection(req)
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 624, in _create_direct_connection
(req.host, req.port, exc.strerror)) from exc
aiohttp.errors.ClientOSError: [Errno 1] Can not connect to discordapp.com:443 [[SSL: CERTIFICATE_VERIFY_FAILED] certific
ate verify failed (_ssl.c:749)]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "harubotFE98.py", line 43748, in <module> # trust me, I know
bot.run('SNIP')
File "C:\Program Files\Python36\lib\site-packages\discord\client.py", line 519, in run
self.loop.run_until_complete(self.start(*args, **kwargs))
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 466, in run_until_complete
return future.result()
File "C:\Program Files\Python36\lib\site-packages\discord\client.py", line 490, in start
yield from self.login(*args, **kwargs)
File "C:\Program Files\Python36\lib\site-packages\discord\client.py", line 416, in login
yield from getattr(self, '_login_' + str(n))(*args, **kwargs)
File "C:\Program Files\Python36\lib\site-packages\discord\client.py", line 346, in _login_1
data = yield from self.http.static_login(token, bot=is_bot)
File "C:\Program Files\Python36\lib\site-packages\discord\http.py", line 195, in static_login
data = yield from self.get(self.ME)
File "C:\Program Files\Python36\lib\site-packages\discord\http.py", line 105, in request
r = yield from self.session.request(method, url, **kwargs)
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client.py", line 555, in __iter__
resp = yield from self._coro
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client.py", line 198, in _request
conn = yield from self._connector.connect(req)
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 314, in connect
.format(key, exc.strerror)) from exc
aiohttp.errors.ClientOSError: [Errno 1] Cannot connect to host discordapp.com:443 ssl:True [Can not connect to discordap
p.com:443 [[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)]]
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x00000224005401D0>
but after, it was shortened(?) to...
Traceback (most recent call last):
File "harubotFE98.py", line 43748, in <module>
bot.run('NOPE')
File "C:\Program Files\Python36\lib\site-packages\discord\client.py", line 519, in run
self.loop.run_until_complete(self.start(*args, **kwargs))
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 466, in run_until_complete
return future.result()
File "C:\Program Files\Python36\lib\site-packages\discord\client.py", line 491, in start
yield from self.connect()
File "C:\Program Files\Python36\lib\site-packages\discord\client.py", line 444, in connect
self.ws = yield from DiscordWebSocket.from_client(self)
File "C:\Program Files\Python36\lib\site-packages\discord\gateway.py", line 175, in from_client
ws = yield from websockets.connect(gateway, loop=client.loop, klass=cls)
File "C:\Program Files\Python36\lib\site-packages\websockets\py35\client.py", line 19, in __await__
return (yield from self.client)
File "C:\Program Files\Python36\lib\site-packages\websockets\client.py", line 150, in connect
factory, wsuri.host, wsuri.port, **kwds)
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 802, in create_connection
sock, protocol_factory, ssl, server_hostname)
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 828, in _create_connection_transport
yield from waiter
File "C:\Program Files\Python36\lib\asyncio\sslproto.py", line 503, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "C:\Program Files\Python36\lib\asyncio\sslproto.py", line 201, in feed_ssldata
self._sslobj.do_handshake()
File "C:\Program Files\Python36\lib\ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x0000021A50E0E358>
I've also tried messing with something called OpenSSL, but pretty much any discussion I can find on it is dense enough that I can't quite wrap my head around it.
Edit: After some messing around, I've written something of a minimum script to get the error to trigger:
import aiohttp
import asyncio
import ssl
async def main():
async with aiohttp.ClientSession() as cs:
async with cs.get("https://www.discordapp.com") as r:
res = await r.text()
print(res)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
which returns:
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 924, in _wrap_create_connection
await self._loop.create_connection(*args, **kwargs))
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 802, in create_connection
sock, protocol_factory, ssl, server_hostname)
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 828, in _create_connection_transport
yield from waiter
File "C:\Program Files\Python36\lib\asyncio\sslproto.py", line 503, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "C:\Program Files\Python36\lib\asyncio\sslproto.py", line 201, in feed_ssldata
self._sslobj.do_handshake()
File "C:\Program Files\Python36\lib\ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "test2.py", line 12, in <module>
loop.run_until_complete(main())
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 466, in run_until_complete
return future.result()
File "test2.py", line 7, in main
async with cs.get("https://www.discordapp.com") as r:
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client.py", line 1005, in __aenter__
self._resp = await self._coro
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client.py", line 476, in _request
timeout=real_timeout
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 522, in connect
proto = await self._create_connection(req, traces, timeout)
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 854, in _create_connection
req, traces, timeout)
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 992, in _create_direct_connection
raise last_exc
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 974, in _create_direct_connection
req=req, client_error=client_error)
File "C:\Program Files\Python36\lib\site-packages\aiohttp\connector.py", line 929, in _wrap_create_connection
raise ClientConnectorSSLError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorSSLError: Cannot connect to host www.discordapp.com:443 ssl:None [[SSL: CERTIFI
CATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)]
I've also tried a couple of other solutions, such as using a web client to visit discordapp.com:443 (which worked, but did not fix this) and updating python's aiohttp to the most recent build (which did not fix this).
if you are on windows server 2018/2019 (I tried it on a personal Datacenter Server.) you can issue the command below to allow your bot to connect. I had to run it off of an administrative terminal to get it to work but this command fixed all of my problems.
certutil -generateSSTFromWU roots.sst && certutil -addstore -f root roots.sst && del roots.sst
Best of luck to you!
A portable solution to this would be to use ssl_context with certifi's cacert.pem
import asyncio
import ssl
import aiohttp
import certifi
async def main():
ssl_context = ssl.create_default_context(cafile=certifi.where())
async with aiohttp.ClientSession() as cs:
async with cs.get("https://www.discordapp.com", ssl_context=ssl_context) as r:
res = await r.text()
print(res)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Okay. So, this one ended up being a pretty weird one -- I still don't know exactly what went wrong, but the gist of it is that, despite being able to load https through browser fine, the SSL certificate was just not properly loaded for Python (or other things, such as OpenSSL). I figured this out when trying to connect to discordapp.com off of OpenSSL command line (it always returned error 20 -- unable to get local issuer certificate).
I downloaded a different certificate (.pem file) (which was pointed out to me by a friend who works in networking) and, with the -CAfile flag I was able to point OpenSSL to it, which let it work properly.
Now, at this point it was just a matter of making Discord.py see this new certificate and use it for its SSL context, but since I'm on 0.16.12 (an old, deprecated branch), that ended up being easier said than done -- on the bot = commands.Bot() line, I added a kwarg, which looked like this:
sslc = ssl.create_default_context(
cafile = "C:\Program Files\Common Files\SSL\cert.pem" #point this at your pem file
)
con = aiohttp.TCPConnector(ssl_context=sslc)
bot = commands.Bot(
command_prefix='/',
connector=con
)
This... didn't actually fix the issue, but as I would learn from Danny (the lead dev of discord.py) it did seem to change the stacktrace, even if the error remained the same. As it turns out, while this changes the ssl context for the connector, the websocket uses its own, and discord.py -- or at least, 0.16.12 -- doesn't have compatibility for that. So, if you navigate to your \Python36\Lib\site-packages\discord folder, you'll need to find the file with the websocket connection -- that would be gateway.py. Open it up, go to line 66 (at least, in 0.16.12 -- otherwise, look for a line that includes yield from websockets.connect(...). Once you find your websocket connection, all you need to do is add a ssl kwarg with a proper context, which should look something like this:
ws = yield from websockets.connect(gateway, loop=loop, klass=klass,
ssl = ssl.create_default_context(
cafile = "C:\Program Files\Common Files\SSL\cert.pem" #point this at your pem file
)
)
If for some godforsaken reason your setup is cursed such that your python can't see the proper SSL certificates, you're on an old deprecated Async version of Discord.py, and you can't fix it any other way by knowing more about this than I do, this should solve your issue. It worked for me. Good luck!
I have the same problem.
To resolve it , go to you repository application, open "Python" you see "Install Certificats.Command" double click on.
Same double click on "Update Shell Profile.command".
And it will be ok, you can run your code again.
is worked for me.
Python3 exchangelib connect error: _ssl.c:749?
I am trying to connect to my company's Exchange server with exchangelib.
I want to see all the mails I have received on MS Exchange / OWA.
When I run the code below, I get:
File "C:\Program Files\Python36\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "C:\Program Files\Python36\lib\site-packages\urllib3\connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "C:\Program Files\Python36\lib\site-packages\urllib3\connectionpool.py", line 850, in _validate_conn
conn.connect()
File "C:\Program Files\Python36\lib\site-packages\urllib3\connection.py", line 326, in connect
ssl_context=context)
File "C:\Program Files\Python36\lib\site-packages\urllib3\util\ssl_.py", line 329, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Program Files\Python36\lib\ssl.py", line 401, in wrap_socket
_context=self, _session=session)
File "C:\Program Files\Python36\lib\ssl.py", line 808, in __init__
self.do_handshake()
File "C:\Program Files\Python36\lib\ssl.py", line 1061, in do_handshake
self._sslobj.do_handshake()
File "C:\Program Files\Python36\lib\ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)
from exchangelib import DELEGATE, Account, Credentials,NTLM,Configuration,Version,Build
creds = Credentials(username = 'inter.com\\login',password='password')
config = Configuration(service_endpoint='https://alpha.com/EWS/Exchange.asmx',credentials=creds,auth_type=NTLM)
account = Account(primary_smtp_address='login#alpha.com',config=config,autodiscover=False,access_type=DELEGATE,verify_ssl=False)
from exchangelib.protocol import BaseProtocol, NoVerifyHTTPAdapter
BaseProtocol.HTTP_ADAPTER_CLS = NoVerifyHTTPAdapter
Can you please let me know how to use certificates with exchangelib?
How to fix it?
Thanks for your help!
Have you tried disabling ssl verification in your config ?
config = Configuration(service_endpoint ='...', credentials=credentials, auth_type=NTLM, verify_ssl=False)
The sources seem to talk about a parameter like this.
I wrote a few lines of python script to get links from my small site. I tested it, used it and was working fine but suddenly it starts generating connection error. I tried to use it on another site, it still generate same problem. I'm new to python, though. Help please!!! Thanks in advance
Part of the code and error message is as follow:
>>>import urllib,sgmllib
>>> f=urllib.urlopen("http://www.example.com")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\urllib.py", line 87, in urlopen
return opener.open(url)
File "C:\Python27\lib\urllib.py", line 208, in open
return getattr(self, name)(url)
File "C:\Python27\lib\urllib.py", line 345, in open_http
h.endheaders(data)
File "C:\Python27\lib\httplib.py", line 991, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 844, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 806, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 787, in connect
self.timeout, self.source_address)
File "C:\Python27\lib\socket.py", line 571, in create_connection
raise err
IOError: [Errno socket error] [Errno 10061] No connection could be made becausethe target machine actively refused it
>>>
Thank you Johan, Shashank, bmhkim for your contributions.
I just figured out the cause of the problem now and the solution. It was caused by the proxy setting of the host running the code. So, resetting the proxy solve it.
I'm getting an error when ever I try to pull down a web page with urllib.urlopen. I've disabled windows firewall and my AV so its not that. I can access the pages in my browser. I even reinstalled python to rule out it being a broken urllib. Any help would be greatly appreciated.
>>> import urllib
>>> h = urllib.urlopen("http://www.google.com").read()
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
h = urllib.urlopen("http://www.google.com").read()
File "C:\Python26\lib\urllib.py", line 86, in urlopen
return opener.open(url)
File "C:\Python26\lib\urllib.py", line 205, in open
return getattr(self, name)(url)
File "C:\Python26\lib\urllib.py", line 344, in open_http
h.endheaders()
File "C:\Python26\lib\httplib.py", line 904, in endheaders
self._send_output()
File "C:\Python26\lib\httplib.py", line 776, in _send_output
self.send(msg)
File "C:\Python26\lib\httplib.py", line 735, in send
self.connect()
File "C:\Python26\lib\httplib.py", line 716, in connect
self.timeout)
File "C:\Python26\lib\socket.py", line 514, in create_connection
raise error, msg
IOError: [Errno socket error] [Errno 10061] No connection could be made because the target machine actively refused it
>>>
this could be the case:
Just found the problem I had set a
proxy through internet options, that
proxy went offline, and so did my
python shell.
urllib is working just fine.
Try using ethereal (or some similar network sniffer) on your box to determine if the denial coming from your machine or a machine beyond.