I'm getting an error when ever I try to pull down a web page with urllib.urlopen. I've disabled windows firewall and my AV so its not that. I can access the pages in my browser. I even reinstalled python to rule out it being a broken urllib. Any help would be greatly appreciated.
>>> import urllib
>>> h = urllib.urlopen("http://www.google.com").read()
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
h = urllib.urlopen("http://www.google.com").read()
File "C:\Python26\lib\urllib.py", line 86, in urlopen
return opener.open(url)
File "C:\Python26\lib\urllib.py", line 205, in open
return getattr(self, name)(url)
File "C:\Python26\lib\urllib.py", line 344, in open_http
h.endheaders()
File "C:\Python26\lib\httplib.py", line 904, in endheaders
self._send_output()
File "C:\Python26\lib\httplib.py", line 776, in _send_output
self.send(msg)
File "C:\Python26\lib\httplib.py", line 735, in send
self.connect()
File "C:\Python26\lib\httplib.py", line 716, in connect
self.timeout)
File "C:\Python26\lib\socket.py", line 514, in create_connection
raise error, msg
IOError: [Errno socket error] [Errno 10061] No connection could be made because the target machine actively refused it
>>>
this could be the case:
Just found the problem I had set a
proxy through internet options, that
proxy went offline, and so did my
python shell.
urllib is working just fine.
Try using ethereal (or some similar network sniffer) on your box to determine if the denial coming from your machine or a machine beyond.
Related
UPDATE: I managed to do a request with urllib2, but I'm still wondering what is happening here.
I would like to do a HTTPS request with Python.
This works fine with the requests module, but I don't want to use external dependencies, so I'd like to use the standard library.
httplib
When I follow this example I don't get a response. I get a timeout instead. I'm out of ideas as to what would cause this.
Code:
import requests
print requests.get('https://python.org')
from httplib import HTTPSConnection
conn = HTTPSConnection('www.python.org')
conn.request('GET', '/index.html')
print conn.getresponse()
Output:
<Response [200]>
Traceback (most recent call last):
File "test.py", line 6, in <module>
conn.request('GET', '/index.html')
File "C:\Python27\lib\httplib.py", line 1069, in request
self._send_request(method, url, body, headers)
File "C:\Python27\lib\httplib.py", line 1109, in _send_request
self.endheaders(body)
File "C:\Python27\lib\httplib.py", line 1065, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 892, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 854, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 1282, in connect
HTTPConnection.connect(self)
File "C:\Python27\lib\httplib.py", line 831, in connect
self.timeout, self.source_address)
File "C:\Python27\lib\socket.py", line 575, in create_connection
raise err
socket.error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
urllib
This fails for a different (but possibly related) reason. Code:
import urllib
print urllib.urlopen("https://python.org")
Output:
Traceback (most recent call last):
File "test.py", line 10, in <module>
print urllib.urlopen("https://python.org")
File "C:\Python27\lib\urllib.py", line 87, in urlopen
return opener.open(url)
File "C:\Python27\lib\urllib.py", line 215, in open
return getattr(self, name)(url)
File "C:\Python27\lib\urllib.py", line 445, in open_https
h.endheaders(data)
File "C:\Python27\lib\httplib.py", line 1065, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 892, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 854, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 1290, in connect
server_hostname=server_hostname)
File "C:\Python27\lib\ssl.py", line 369, in wrap_socket
_context=self)
File "C:\Python27\lib\ssl.py", line 599, in __init__
self.do_handshake()
File "C:\Python27\lib\ssl.py", line 828, in do_handshake
self._sslobj.do_handshake()
IOError: [Errno socket error] [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:727)
What is requests doing that makes it succeed where both of these libraries fail?
requests.get without timeout parameter mean no timeout at all.
httplib.HTTPSConnection accept parameter timeout in Python 2.6 and newer according to httplib docs. If your problem was caused by timeout, setting high enough timeout should help. Please try replacing:
conn = HTTPSConnection('www.python.org')
with:
conn = HTTPSConnection('www.python.org', timeout=300)
which will give 300 seconds (5 minutes) for processing.
Using that apache libloud docs and valid credentials i get the below error trying to list domains on godaddy. Does libcloud noi longer support godaddy?
>>> from libcloud.dns.types import Provider
>>> from libcloud.dns.providers import get_driver
>>> cls = get_driver(Provider.GODADDY)
>>> driver = cls('twst', 'adfadf', 'dsdfsdf')
>>> zones = driver.list_zones()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/libcloud/dns/drivers/godaddy.py", line 146, in list_zones
'/v1/domains/').object
File "/usr/local/lib/python2.7/dist-packages/libcloud/common/base.py", line 782, in request
headers=headers)
File "/usr/lib/python2.7/httplib.py", line 979, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.7/httplib.py", line 1013, in _send_request
self.endheaders(body)
File "/usr/lib/python2.7/httplib.py", line 975, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 835, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 797, in send
self.connect()
File "/usr/local/lib/python2.7/dist-packages/libcloud/httplib_ssl.py", line 266, in connect
self.timeout)
File "/usr/lib/python2.7/socket.py", line 553, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -2] Name or service not known
>>>
It looks like there was a bug in the driver. The "host" attribute on the connection class was incorrectly set to a URL instead of a hostname.
I pushed a fix for that - https://github.com/apache/libcloud/commit/a3ba6a4751623224f16175df9175ec06b29cdc1a
You can test this change by installing latest in development version from git using pip - pip install git+https://git-wip-us.apache.org/repos/asf/libcloud.git#trunk#egg=apache-libcloud
I have confirmed the change is working locally, but if you encounter any more issues, please let us know.
from libcloud.dns.types import Provider
from libcloud.dns.providers import get_driver
cls = get_driver(Provider.GODADDY)
driver = cls('twst', 'adfadf', 'dsdfsdf')
print driver.list_zones()
...
libcloud.dns.drivers.godaddy.GoDaddyDNSException: <GoDaddyDNSException in MALFORMED_API_KEY: Malformed API key>
In addition to that, I will also go ahead and push a change so a more friendly exception is thrown in case the "host" attribute is set to a value which is not a hostname.
I wrote a few lines of python script to get links from my small site. I tested it, used it and was working fine but suddenly it starts generating connection error. I tried to use it on another site, it still generate same problem. I'm new to python, though. Help please!!! Thanks in advance
Part of the code and error message is as follow:
>>>import urllib,sgmllib
>>> f=urllib.urlopen("http://www.example.com")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\urllib.py", line 87, in urlopen
return opener.open(url)
File "C:\Python27\lib\urllib.py", line 208, in open
return getattr(self, name)(url)
File "C:\Python27\lib\urllib.py", line 345, in open_http
h.endheaders(data)
File "C:\Python27\lib\httplib.py", line 991, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 844, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 806, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 787, in connect
self.timeout, self.source_address)
File "C:\Python27\lib\socket.py", line 571, in create_connection
raise err
IOError: [Errno socket error] [Errno 10061] No connection could be made becausethe target machine actively refused it
>>>
Thank you Johan, Shashank, bmhkim for your contributions.
I just figured out the cause of the problem now and the solution. It was caused by the proxy setting of the host running the code. So, resetting the proxy solve it.
I'm trying to download the HTML of a page (http://www.google.com in this case) but I'm getting back an error. Here is my interactive prompt session:
Python 3.2.2 (default, Sep 4 2011, 09:51:08) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib
>>> import urllib.request
>>> html = urllib.request.urlopen("http://www.google.com")
Traceback (most recent call last):
File "\\****.****.org\myhome\python\lib\urllib\request.py", line 1136, in
do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "\\****.****.org\myhome\python\lib\http\client.py", line 964, in req
uest
self._send_request(method, url, body, headers)
File "\\****.****.org\myhome\python\lib\http\client.py", line 1002, in _s
end_request
self.endheaders(body)
File "\\****.****.org\myhome\python\lib\http\client.py", line 960, in end
headers
self._send_output(message_body)
File "\\****.****.org\myhome\python\lib\http\client.py", line 805, in _se
nd_output
self.send(msg)
File "\\****.****.org\myhome\python\lib\http\client.py", line 743, in sen
d
self.connect()
File "\\****.****.org\myhome\python\lib\http\client.py", line 721, in con
nect
self.timeout, self.source_address)
File "\\****.****.org\myhome\python\lib\socket.py", line 404, in create_c
onnection
raise err
File "\\****.****.org\myhome\python\lib\socket.py", line 395, in create_c
onnection
sock.connect(sa)
socket.error: [Errno 10061] No connection could be made because the target machi
ne actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "\\****.****.org\myhome\python\lib\urllib\request.py", line 138, in
urlopen
return opener.open(url, data, timeout)
File "\\****.****.org\myhome\python\lib\urllib\request.py", line 369, in
open
response = self._open(req, data)
File "\\****.****.org\myhome\python\lib\urllib\request.py", line 387, in
_open
'_open', req)
File "\\****.****.org\myhome\python\lib\urllib\request.py", line 347, in
_call_chain
result = func(*args)
File "\\****.****.org\myhome\python\lib\urllib\request.py", line 1156, in
http_open
return self.do_open(http.client.HTTPConnection, req)
File "\\****.****.org\myhome\python\lib\urllib\request.py", line 1139, in
do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 10061] No connection could be made
because the target machine actively refused it>
>>>
My best guess is that my network's firewall is blocking the connection (most/all ports other than 80 are blocked). I don't see why, though; doesn't Python connect on port 80? Does anyone know what is happening?
urllib takes the proxy settings from Internet Explorer, which is usually under Tools->Internet Options->Connections->Lan Settings
If you have a proxy, make sure it's correct in IE, or set it when using urllib. If you don't use a proxy to browse, make sure that the proxy settings in IE are empty.
I was struggling with this issue few hours until I realized that a 3rd party app I once installed changed the setting in IE, while i was surfing happily in Firefox without a proxy.
Another solution is in the this link which shows how to force urllib to ignore IE's proxy settings
The error comes from operating system level and is not related to Python. It would be same with any programming language.
Contact your network administration to see what's the matter unless you can resolve firewall issues yourself. If you need a HTTP proxy then you can configure one for urllib.
Thanks for your help in advance!
I want to get contents of a website, so I use urllib.urlopen(url).
set url='http://localhost:8080'(tomcat page)
If I use Google App Engine Launcher, run the application, browse http://localhost:8082 , it works well.
But if I specify the address and port for the application:
python `"D:\Program Files\Google\google_appengine\dev_appserver.py" -p 8082 -a 10.96.72.213 D:\pagedemon\videoareademo`
there's something wrong:
Traceback (most recent call last):
File "D:\Program Files\Google\google_appengine\google\appengine\ext\webapp\_webapp25.py", line 701, in __call__
handler.get(*groups)
File "D:\pagedemon\videoareademo\home.py", line 76, in get
wp = urllib.urlopen(url)
File "C:\Python27\lib\urllib.py", line 84, in urlopen
return opener.open(url)
File "C:\Python27\lib\urllib.py", line 205, in open
return getattr(self, name)(url)
File "C:\Python27\lib\urllib.py", line 343, in open_http
errcode, errmsg, headers = h.getreply()
File "D:\Program Files\Google\google_appengine\google\appengine\dist\httplib.py", line 334, in getreply
response = self._conn.getresponse()
File "D:\Program Files\Google\google_appengine\google\appengine\dist\httplib.py", line 222, in getresponse
deadline=self.timeout)
File "D:\Program Files\Google\google_appengine\google\appengine\api\urlfetch.py", line 263, in fetch
return rpc.get_result()
File "D:\Program Files\Google\google_appengine\google\appengine\api\apiproxy_stub_map.py", line 592, in get_result
return self.__get_result_hook(self)
File "D:\Program Files\Google\google_appengine\google\appengine\api\urlfetch.py", line 365, in _get_fetch_result
raise DownloadError(str(err))
DownloadError: ApplicationError: 2 [Errno 11003] getaddrinfo failed
The strangest thing is when I change the url form "http://localhost:8080" to "http://127.0.0.1:8080", it works well!
I googled a lot, but I didn't find any good solutions.Hoping for some help!
Also, I didn't configure any proxy.IE works well.
Your system doesn't necessarily know that localhost should resolve to 127.0.0.1. You might need to put an entry into your hosts file. On Windows, it's located at C:\Windows\System32\drivers\etc\hosts