I need to make a simple header request to every URL in a large set, to check if they are still available. Now I made the following code:
from http import client
for i, triple in enumerate(catalouge):
connection = client.HTTPConnection(triple[2].strip('http://'))
connection.request('HEAD', '/')
print(connection.getresponse().status + ' on entry ' + str(i+1))
Now catalouge is a set of all the links with the 3rd element being the URL which needs to be checked. The .strip('http://') part is needed since I will receive this error otherwise:
http.client.InvalidURL: nonnumeric port:
With this code in place I now receive this error:
Traceback (most recent call last):
[...]
connection.request('HEAD', '/')
File "/usr/lib/python3.4/http/client.py", line 1137, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.4/http/client.py", line 1182, in _send_request
self.endheaders(body)
File "/usr/lib/python3.4/http/client.py", line 1133, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.4/http/client.py", line 963, in _send_output
self.send(msg)
File "/usr/lib/python3.4/http/client.py", line 898, in send
self.connect()
File "/usr/lib/python3.4/http/client.py", line 871, in connect
self.timeout, self.source_address)
File "/usr/lib/python3.4/socket.py", line 494, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.4/socket.py", line 533, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
Did I miss something? Any suggestion would be very appreciated.
Related
I am trying to open a URL using the following code:
import urllib.request
with urllib.request.urlopen("http://www.example.com/") as doc:
html = doc.read()
I've tried with many different websites but I always get this error:
Traceback (most recent call last):
File "/usr/lib/python3.8/urllib/request.py", line 1350, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/lib/python3.8/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 950, in send
self.connect()
File "/usr/lib/python3.8/http/client.py", line 921, in connect
self.sock = self._create_connection(
File "/usr/lib/python3.8/socket.py", line 787, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Try again
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 3, in <module>
with urllib.request.urlopen("http://www.wikipedia.org/") as doc:
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 1379, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.8/urllib/request.py", line 1353, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -3] Try again>
I've searched for [Errno -3] and I also looked into "https://docs.python.org/3/library/urllib.request.html" but I didn't find anything useful.
What am I doing wrong?
I wanted to use you-get to download videos, but when I use it after I installed it, an error ocurred:
my python3.9 is well installed and I have checked the system path. I have also used many ways to install you-get including using pip3, downloading the zip from github and this error always occur. I thought this problem could be caused by my vpn, but when I close vpn, nothing changed.
I opened cmd with administrator as well.
I typed this command: you-get 'https://www.youtube.com/watch?v=nWQwrU1qUrc' --debug
[DEBUG] get_location: http://'https://www.youtube.com/watch?v=nWQwrU1qUrc'
[DEBUG] get_location: http://'https://www.youtube.com/watch?v=nWQwrU1qUrc'
you-get: version 0.4.1500, a tiny downloader that scrapes the web.
you-get: Namespace(version=False, help=False, info=False, url=False, json=False, no_merge=False, no_caption=False, force=False, skip_existing_file_size_check=False, format=None, output_filename=None, output_dir='.', player=None, cookies=None, timeout=600, debug=True, input_file=None, password=None, playlist=False, auto_rename=False, insecure=False, http_proxy=None, extractor_proxy=None, no_proxy=False, socks_proxy=None, stream=None, itag=None, URL=["'https://www.youtube.com/watch?v=nWQwrU1qUrc'"])
Traceback (most recent call last):
File "C:\Python39\lib\urllib\request.py", line 1346, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Python39\lib\http\client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Python39\lib\http\client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Python39\lib\http\client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Python39\lib\http\client.py", line 1010, in _send_output
self.send(msg)
File "C:\Python39\lib\http\client.py", line 950, in send
self.connect()
File "C:\Python39\lib\http\client.py", line 921, in connect
self.sock = self._create_connection(
File "C:\Python39\lib\socket.py", line 822, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "C:\Python39\lib\socket.py", line 953, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\common.py", line 1777, in url_to_module
File "C:\Python39\lib\site-packages\you_get-0.4.1500 py3.9.egg\you_get\common.py", line 393, in get_location
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\common.py", line 408, in urlopen_with_retry
File "C:\Python39\lib\urllib\request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "C:\Python39\lib\urllib\request.py", line 517, in open
response = self._open(req, data)
File "C:\Python39\lib\urllib\request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "C:\Python39\lib\urllib\request.py", line 494, in _call_chain
result = func(*args)
File "C:\Python39\lib\urllib\request.py", line 1375, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Python39\lib\urllib\request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python39\lib\urllib\request.py", line 1346, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Python39\lib\http\client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Python39\lib\http\client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Python39\lib\http\client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Python39\lib\http\client.py", line 1010, in _send_output
self.send(msg)
File "C:\Python39\lib\http\client.py", line 950, in send
self.connect()
File "C:\Python39\lib\http\client.py", line 921, in connect
self.sock = self._create_connection(
File "C:\Python39\lib\socket.py", line 822, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "C:\Python39\lib\socket.py", line 953, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed
Traceback (most recent call last):
File "C:\Python39\Scripts\you-get-script.py", line 33, in <module>
sys.exit(load_entry_point('you-get==0.4.1500', 'console_scripts', 'you-get')())
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\__main__.py", line 92, in main
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\common.py", line 1798, in main
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\common.py", line 1680, in script_main
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\common.py", line 1327, in download_main
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\common.py", line 1788, in any_download
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\common.py", line 1779, in url_to_module
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\common.py", line 393, in get_location
File "C:\Python39\lib\site-packages\you_get-0.4.1500-py3.9.egg\you_get\common.py", line 408, in urlopen_with_retry
File "C:\Python39\lib\urllib\request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "C:\Python39\lib\urllib\request.py", line 517, in open
response = self._open(req, data)
File "C:\Python39\lib\urllib\request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "C:\Python39\lib\urllib\request.py", line 494, in _call_chain
result = func(*args)
File "C:\Python39\lib\urllib\request.py", line 1375, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Python39\lib\urllib\request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
could someone help me pls, thank you so much
If you're intent on using cmd, you should replace ' with " in your command, or since the website contains no spaces anyway, just remove the quotes. So try running:
you-get "https://www.youtube.com/watch?v=nWQwrU1qUrc"
OR
you-get https://www.youtube.com/watch?v=nWQwrU1qUrc
However in the future I'd suggest you just use Windows PowerShell it's much better in my opinion.
I am using python 3.4 and my task is to download link from a website. But while fetching the data I got an error that:
I have tried it from both httplib2 and urllib, but in both cases it is showing the same error.
OUTPUT:
>>> import urllib.request
>>> request = urllib.request.Request(url)
>>> response = urllib.request.urlopen(request)
Traceback (most recent call last):
File "C:\Python34\lib\urllib\request.py", line 1183, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "C:\Python34\lib\http\client.py", line 1137, in request
self._send_request(method, url, body, headers)
File "C:\Python34\lib\http\client.py", line 1182, in _send_request
self.endheaders(body)
File "C:\Python34\lib\http\client.py", line 1133, in endheaders
self._send_output(message_body)
File "C:\Python34\lib\http\client.py", line 963, in _send_output
self.send(msg)
File "C:\Python34\lib\http\client.py", line 898, in send
self.connect()`
File "C:\Python34\lib\http\client.py", line 871, in connect
self.timeout, self.source_address)`
File "C:\Python34\lib\socket.py", line 516, in create_connection
raise err
File "C:\Python34\lib\socket.py", line 507, in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the
connected party did not properly respond after a period of time, or
established connection failed because connected host has failed to respond
This question already has answers here:
"getaddrinfo failed", what does that mean?
(6 answers)
Closed 8 years ago.
i am trying to connect to TOR through python but it doesnt let me the code is:
def tor_connection():
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050, True)
socket.socket = socks.socksocket
def main():
tor_connection()
print('Connected to tor')
con = httplib.HTTPConnection('myip.dnsomatic.com/')
con.request('GET', '/')
response = con.getresponse()
print(response.read())
main()
even though its giving me the next error message:
Traceback (most recent call last):
File "C:/Users/anon/PycharmProjects/Scraper/tor.py", line 198, in <module>
main()
File "C:/Users/anon/PycharmProjects/Scraper/tor.py", line 194, in main
con.request('GET', '/')
File "C:\Python27\lib\httplib.py", line 1001, in request
self._send_request(method, url, body, headers)
File "C:\Python27\lib\httplib.py", line 1035, in _send_request
self.endheaders(body)
File "C:\Python27\lib\httplib.py", line 997, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 850, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 812, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 793, in connect
self.timeout, self.source_address)
File "C:\Python27\lib\socket.py", line 553, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno 11001] getaddrinfo failed
i am just a beginner could someone help me out please? i have tried it in another laptop but its the same error message
It's not a problem of socks. You need to specify the hostname without the trailing /:
>>> # with /
>>> httplib.HTTPConnection('myip.dnsomatic.com/').request('GET', '/')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/httplib.py", line 973, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.7/httplib.py", line 1007, in _send_request
self.endheaders(body)
File "/usr/lib/python2.7/httplib.py", line 969, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 829, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 791, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 772, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 553, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -2] Name or service not known
>>> # without /
>>> httplib.HTTPConnection('myip.dnsomatic.com').request('GET', '/')
>>>
I'm new to python and I'm following a video tutorial.
So here's the code snippet
from urllib.request import urlopen
with urlopen('http://sixty-north/c/t.txt') as story:
story_words = []
for line in story:
line_words = line.decode('utf-8').split()
for word in line_words:
story_words.append(word)
I'm able to access http://sixty-north.com/c/t.txt in my browser.
However when I type this into command prompt: python words.py I get this error:
C:\New folder>python words.py
Traceback (most recent call last):
File "C:\Python33\lib\urllib\request.py", line 1248, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "C:\Python33\lib\http\client.py", line 1065, in request
self._send_request(method, url, body, headers)
File "C:\Python33\lib\http\client.py", line 1103, in _send_request
self.endheaders(body)
File "C:\Python33\lib\http\client.py", line 1061, in endheaders
self._send_output(message_body)
File "C:\Python33\lib\http\client.py", line 906, in _send_output
self.send(msg)
File "C:\Python33\lib\http\client.py", line 844, in send
self.connect()
File "C:\Python33\lib\http\client.py", line 822, in connect
self.timeout, self.source_address)
File "C:\Python33\lib\socket.py", line 417, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno 11001] getaddrinfo failed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "words.py", line 2, in <module>
with urlopen('http://sixty-north/c/t.txt') as story:
File "C:\Python33\lib\urllib\request.py", line 156, in urlopen
return opener.open(url, data, timeout)
File "C:\Python33\lib\urllib\request.py", line 469, in open
response = self._open(req, data)
File "C:\Python33\lib\urllib\request.py", line 487, in _open
'_open', req)
File "C:\Python33\lib\urllib\request.py", line 447, in _call_chain
result = func(*args)
File "C:\Python33\lib\urllib\request.py", line 1274, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Python33\lib\urllib\request.py", line 1251, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
There is no such host: sixty-north. Replace sixty-north with sixty-north.com (notice: .com at the end)
Change host: sixty-north with sixty-north.com
And also, Restart your Internet Connection.
Your Internet is not working properly.