I am using
httplib.HTTPConnection ("http://ipaddr:port")
conn.request("GET", "", params, headers)
I am able to do PUT/GET using ipaddr:port using my firefox client!!.
But I am seeing this error on execution of the script:
File "post_python.py", line 5, in <module>
conn.request("GET", "", params, headers)
File "/usr/lib64/python2.6/httplib.py", line 914, in request
self._send_request(method, url, body, headers)
File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
self.endheaders()
File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
self._send_output()
File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
self.send(msg)
File "/usr/lib64/python2.6/httplib.py", line 739, in send
self.connect()
File "/usr/lib64/python2.6/httplib.py", line 720, in connect
self.timeout)
File "/usr/lib64/python2.6/socket.py", line 553, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -2] Name or service not known"
Please can someone help me ??
Try this instead (without "http://" before the IP address):
conn = httplib.HTTPConnection("x.x.x.x", port)
conn.request("GET", "", params, headers)
You might have a proxy in between that the browser already knows about. If you're under linux try setting http_proxy environment variable.
If it's an IPv6 address, you need to surround it with brackets as per RFC 2732. If I recall correctly, that's the error message you get if you don't use brackets.
httplib.HTTPConnection ("http://[::1]:8080")
conn.request("GET", "", params, headers)
Related
UPDATE: I managed to do a request with urllib2, but I'm still wondering what is happening here.
I would like to do a HTTPS request with Python.
This works fine with the requests module, but I don't want to use external dependencies, so I'd like to use the standard library.
httplib
When I follow this example I don't get a response. I get a timeout instead. I'm out of ideas as to what would cause this.
Code:
import requests
print requests.get('https://python.org')
from httplib import HTTPSConnection
conn = HTTPSConnection('www.python.org')
conn.request('GET', '/index.html')
print conn.getresponse()
Output:
<Response [200]>
Traceback (most recent call last):
File "test.py", line 6, in <module>
conn.request('GET', '/index.html')
File "C:\Python27\lib\httplib.py", line 1069, in request
self._send_request(method, url, body, headers)
File "C:\Python27\lib\httplib.py", line 1109, in _send_request
self.endheaders(body)
File "C:\Python27\lib\httplib.py", line 1065, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 892, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 854, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 1282, in connect
HTTPConnection.connect(self)
File "C:\Python27\lib\httplib.py", line 831, in connect
self.timeout, self.source_address)
File "C:\Python27\lib\socket.py", line 575, in create_connection
raise err
socket.error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
urllib
This fails for a different (but possibly related) reason. Code:
import urllib
print urllib.urlopen("https://python.org")
Output:
Traceback (most recent call last):
File "test.py", line 10, in <module>
print urllib.urlopen("https://python.org")
File "C:\Python27\lib\urllib.py", line 87, in urlopen
return opener.open(url)
File "C:\Python27\lib\urllib.py", line 215, in open
return getattr(self, name)(url)
File "C:\Python27\lib\urllib.py", line 445, in open_https
h.endheaders(data)
File "C:\Python27\lib\httplib.py", line 1065, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 892, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 854, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 1290, in connect
server_hostname=server_hostname)
File "C:\Python27\lib\ssl.py", line 369, in wrap_socket
_context=self)
File "C:\Python27\lib\ssl.py", line 599, in __init__
self.do_handshake()
File "C:\Python27\lib\ssl.py", line 828, in do_handshake
self._sslobj.do_handshake()
IOError: [Errno socket error] [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:727)
What is requests doing that makes it succeed where both of these libraries fail?
requests.get without timeout parameter mean no timeout at all.
httplib.HTTPSConnection accept parameter timeout in Python 2.6 and newer according to httplib docs. If your problem was caused by timeout, setting high enough timeout should help. Please try replacing:
conn = HTTPSConnection('www.python.org')
with:
conn = HTTPSConnection('www.python.org', timeout=300)
which will give 300 seconds (5 minutes) for processing.
I want to get the Url status for websites with the below code. For one website (webscraper.io), I got an error. My script is:
import httplib
url = "http://webscraper.io/"
if 'http' in url:
url = url.replace('http://', '').strip()
conn = httplib.HTTPConnection(url)
conn.request("GET",'')
r1 = conn.getresponse()
print 'r1.Status code=', r1.status
I got the below errors:
Traceback (most recent call last):
File "TestSatusline.py", line 23, in <module>
conn.request("GET",'')
File "/usr/lib/python2.7/httplib.py", line 1017, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.7/httplib.py", line 1051, in _send_request
self.endheaders(body)
File "/usr/lib/python2.7/httplib.py", line 1013, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 864, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 826, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 807, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 553, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -2] Name or service not known
Does anybody has any idea?
thanks
after
if 'http' in url:
url = url.replace('http://', '').strip()
in your code , url is webscraper.io/, it should be webscraper.io
use urlparse
import httplib
import urlparse
url = "http://webscraper.io/"
o = urlparse.urlparse(url)
conn = httplib.HTTPConnection(o.netloc)
conn.request("GET",'')
r1 = conn.getresponse()
print 'r1.Status code=', r1.status
output
r1.Status code= 200
you could take a look at requests. http://docs.python-requests.org/en/master/
I need to make a simple header request to every URL in a large set, to check if they are still available. Now I made the following code:
from http import client
for i, triple in enumerate(catalouge):
connection = client.HTTPConnection(triple[2].strip('http://'))
connection.request('HEAD', '/')
print(connection.getresponse().status + ' on entry ' + str(i+1))
Now catalouge is a set of all the links with the 3rd element being the URL which needs to be checked. The .strip('http://') part is needed since I will receive this error otherwise:
http.client.InvalidURL: nonnumeric port:
With this code in place I now receive this error:
Traceback (most recent call last):
[...]
connection.request('HEAD', '/')
File "/usr/lib/python3.4/http/client.py", line 1137, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.4/http/client.py", line 1182, in _send_request
self.endheaders(body)
File "/usr/lib/python3.4/http/client.py", line 1133, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.4/http/client.py", line 963, in _send_output
self.send(msg)
File "/usr/lib/python3.4/http/client.py", line 898, in send
self.connect()
File "/usr/lib/python3.4/http/client.py", line 871, in connect
self.timeout, self.source_address)
File "/usr/lib/python3.4/socket.py", line 494, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.4/socket.py", line 533, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
Did I miss something? Any suggestion would be very appreciated.
I am trying to run a very simple insertion to Elasticsearch in Python:
es = Elasticsearch({'host': 'localhost', 'port': 9200})
res = es.index(index='data-client_dev', doc_type='test', id=2, body={'author': 'Christophe'}, timeout=60)
print(res['created'])
But I keep having the error pasted at the end.
I am under Ubuntu 14 and am using PyCharm (if this might help). The ES node is up and running locally on my computer.
I tried to change the timeout (or with request_timeout) but it is doing nothing. What is weird is the query is working from the terminal, so maybe it is coming for Pycharm.
I am a beginner so maybe I missed something obvious.
Thanks a lot for your help!
WARNING:elasticsearch:PUT http://port:9200/data-client_dev/test/2?timeout=60 [status:N/A request:20.040s]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 78, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 608, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 224, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 558, in urlopen
body=body, headers=headers)
File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 353, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python2.7/httplib.py", line 979, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.7/httplib.py", line 1013, in _send_request
self.endheaders(body)
File "/usr/lib/python2.7/httplib.py", line 975, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 835, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 797, in send
self.connect()
File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 162, in connect
conn = self._new_conn()
File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 142, in _new_conn
(self.host, self.timeout))
ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7fe723d7e550>, u'Connection to port timed out. (connect timeout=10)')
WARNING:elasticsearch:Connection <Urllib3HttpConnection: http://port:9200> has failed for 1 times in a row, putting on 60 second timeout.
You need to create your Elasticsearch client like this, i.e. by putting the host in a list:
es = Elasticsearch([{'host': 'localhost', 'port': 9200}])
^ ^
| |
add this and this
I have a script which get HTTP Header of a lot of pages on Internet with httplib in Python.
My problem is on a specific domain (and probably others), httplib raise an exception, and I don't understand why.
>>> import httplib
>>> http = httplib.HTTPConnection('iswtc.la')
>>> http.request('GET', '/a')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.6/httplib.py", line 914, in request
self._send_request(method, url, body, headers)
File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
self.endheaders()
File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
self._send_output()
File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
self.send(msg)
File "/usr/lib64/python2.6/httplib.py", line 739, in send
self.connect()
File "/usr/lib64/python2.6/httplib.py", line 720, in connect
self.timeout)
File "/usr/lib64/python2.6/socket.py", line 553, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -2] Name or service not known
What is different on this specific domain, and how can I handle this ?
PS : It's not really my code because this works fine :
>>> http = httplib.HTTPConnection('bit.ly')
>>> http.request('GET', '/a')
bit.ly exists, whereas iswtc.la doesn't:
$ nslookup bit.ly
Non-authoritative answer:
Name: bit.ly
Address: 69.58.188.39
Name: bit.ly
Address: 69.58.188.40
$ nslookup iswtc.la
** server can't find iswtc.la: NXDOMAIN