I have developed a python script which reads streams from multiple cameras and pass it to a server to display information on the frames. But the problem is that the script always fails if even of the ports of the server is not responding. I have tried exception handling but still no luck.
What I want is that if the server port is not responding then the frames must be displayed to the user without any information.
I have used python requests module.
Here is the code snippet:
self.frame = cv2.resize(frame, (self.screen_width, self.screen_height))
try:
payload = {
"frame":frame,
"timestamp": timestamp
}
payload = jsonpickle.encode(payload)
lock.acquire()
response = requests.post(self.frame_target_ip, data=payload, #self.frame_target_ip is the server ip with port number
headers=self.headers)
lock.release()
objects = jsonpickle.decode(response.content)
self.event_data, self.dets = self.parse_objects(objects)
if self.show_bounding_box == True:
draw_frame = self.draw_bounding_boxes(self.frame, self.dets)
self.frame = cv2.cvtColor(draw_frame, cv2.COLOR_BGR2RGB)
except Exception as e:
logging.error("Couldn't reach the frame_target_ip", exc_info=True)
I am using python 3 and windows 10.
Below is error traceback
00:31:52:327-root-INFO--Application Started
00:32:07:682-root-ERROR--Couldn't reach the frame_target_ip
Traceback (most recent call last):
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
raise err
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 677, in urlopen
chunked=chunked,
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 966, in send
self.connect()
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 187, in connect
conn = self._new_conn()
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 172, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x00000295D3350E48>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py", line 449, in send
timeout=timeout
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 725, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=8000): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000295D3350E48>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:/Users/Deepanshu/Desktop/VIDS_1cam/UI_1cam.py", line 221, in set_frame
headers=self.headers)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Deepanshu\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=8000): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000295D3350E48>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
Related
This question already has answers here:
python requests module and connection reuse
(2 answers)
Closed 3 months ago.
im currently working on a python script that executes POST requests a few thousand times to fill dummy data into a database. The POST request sends a string to our backend which fills in this string into the database.
The first 10000 requests are working fine but then a ConnectionError appears.
This is a simplified implementation of my code
sqlfiller.py
import requests
url = "http://localhost:3100/api"
payload = "test"
#40 represents all business-days in two months
for i in range(40):
#600 data entries per day
for i in range(600):
requests.post(url, payload)
As i mentioned, it works fine for a couple thousands POST requests. But sometime this error appears and the program crashes.
Traceback (most recent call last):
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 174, in _new_conn
conn = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\connection.py", line 95, in create_connection
raise err
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
sock.connect(sa)
OSError: [WinError 10048] Normalerweise darf jede Socketadresse (Protokoll, Netzwerkadresse oder Anschluss) nur jeweils einmal verwendet werden
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 239, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\http\client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\http\client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\http\client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\http\client.py", line 1037, in _send_output
self.send(msg)
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\http\client.py", line 975, in send
self.connect()
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 205, in connect
conn = self._new_conn()
^^^^^^^^^^^^^^^^
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x0000020309A81250>: Failed to establish a new connection: [WinError 10048] Normalerweise darf jede Socketadresse (Protokoll, Netzwerkadresse oder Anschluss) nur jeweils einmal verwendet werden
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 489, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=3100): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000020309A81250>: Failed to establish a new connection: [WinError 10048] Normalerweise darf jede Socketadresse (Protokoll, Netzwerkadresse oder Anschluss) nur jeweils einmal verwendet werden'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Workbench\sqlfiller.py", line 55, in <module>
requests.post(url, payload)
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\adriand\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 565, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=3100): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000020309A81250>: Failed to establish a new connection: [WinError 10048] Normalerweise darf jede Socketadresse (Protokoll, Netzwerkadresse oder Anschluss) nur jeweils einmal verwendet werden'))
This german error line WinError 10048 translates to "Only one usage of each socket address (protocol/network address/port) is normally permitted"
Someone knows whats going on?
Thanks :)
Instead of request.post use httplib.HTTPConnection(url) with concurrency .
You need to change your code,That will definitely help to solve this problem.
I am using prometheus with scaphandre (docs) to measure energy consumption in my kubernetes cluster. In the guide the way they use to access prometheus data is to forward the port from the pod to the host, but I've seen it's slower than using a service and I've been having some issues with that too.
I've defined a service to access prometheus data:
# prometheus_service.yaml
apiVersion: v1
kind: Service
metadata:
name: prometheus-service
namespace: default
annotations:
prometheus.io/scrape: 'true'
prometheus.io/port: '9090'
spec:
selector:
app: prometheus
type: NodePort
ports:
- port: 9090
targetPort: 9090
nodePort: 30030
My problem is that I've been trying to get a query result from the URI: 192.168.131.180:30030/api/v1/query?query=scaph_host_power_microwatts, but it sometimes works fine and other times I get a connection refused error.
More precisely I have tried 3 ways to access it:
By using VSCode Thunder Client extension (a REST API extension that works like POSTMAN or Insomnia) and when I send the GET request 50% of the times I get Connection was refused by the server. and the other 50% I get a proper result.
By directly inputting the query in my browser, in this case I always get a result when accessing the URI (and I can tell it is not cached because the query result has a time attribute which is always different).
By making the query through a python program with the requests library (which is my main goal to go on with my app). In this case it also fails 50% of the time.
If the browser had also failed 50% of the times I would've thought that prometheus fails when no new data is available, but since the browser worked fine, there must be another reason.
For additional info: when I change the previous IP for localhost the same thing happens.
Is there a reason for this error?
PD: If you want the python traceback, I leave it here (sorry I wanted to write it in a kind of "See More" or "Expand Block" to avoid pasting it there but didn't know how):
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
conn = connection.create_connection(
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.8/http/client.py", line 1256, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 951, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 187, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 171, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe1d7788520>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
retries = retries.increment(
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='192.168.131.180', port=30030): Max retries exceeded with url: /api/v1/query?query=scaph_host_power_microwatts (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe1d7788520>: Failed to establish a new connection: [Errno 111] Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "pruebametrics.py", line 55, in <module>
foo()
File "pruebametrics.py", line 51, in foo
microwatts =requests.get("http://192.168.131.180:30030/api/v1/query", params={'query': 'scaph_host_power_microwatts'}).json()
File "/usr/lib/python3/dist-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='192.168.131.180', port=30030): Max retries exceeded with url: /api/v1/query?query=scaph_host_power_microwatts (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe1d7788520>: Failed to establish a new connection: [Errno 111] Connection refused'))
We are trying to call An API using Python Shell in AWS glue. We are making an HTTPS call using python 3.6 using "requests" library. The issue is pertaining to AWS glue.
The Job config is
Python Shell
Python 3 ( Glue Version 1.0 )
This is the error that we get
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 309, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f862b3b4160>: Failed to establish a new connection: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 446, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='xyz.com', port=443): Max retries exceeded with url: /api-v2/client/orders/?auth_token=#####&start_date=01-04-2021&end_date=02-04-2021&status_filter=all&page_number=1 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f862b3b4160>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/runscript.py", line 123, in <module>
runpy.run_path(temp_file_path, run_name='__main__')
File "/usr/local/lib/python3.6/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmp/glue-python-scripts-fhh0x612/test.py", line 16, in <module>
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='xyz.com', port=443): Max retries exceeded with url: /api-v2/client/orders/?auth_token=#####0&start_date=01-04-2021&end_date=02-04-2021&status_filter=all&page_number=1 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f862b3b4160>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/runscript.py", line 142, in <module>
raise e_type(e_value).with_traceback(new_stack)
File "/tmp/glue-python-scripts-fhh0x612/test.py", line 16, in <module>
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='xyz.com', port=443): Max retries exceeded with url: /api-v2/client/orders/?auth_token=####&start_date=01-04-2021&end_date=02-04-2021&status_filter=all&page_number=1 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f862b3b4160>: Failed to establish a new connection: [Errno 110] Connection timed out',))
Is there an issue with the security group that AWS Glue refers to or some issue with the API call. We are not able to figure out.
Is there more than one connection from glue? like calling S3 , redshift...etc. if more than one then u may have to add other connections in connection properties. Edit job -> add connection.
One can theoretically download a stanza's model via Python as follows (mirror):
import stanza
stanza.download('en') # This downloads the English models for the neural pipeline
However, the Stanford server is inaccessible from my computer:
(neural-parser2) dernoncourt#ilcompn0:~/temp/stanza$ python
Python 3.6.7 (default, Oct 25 2018, 09:16:13)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import stanza
>>> stanza.download('en')
Downloading https://raw.githubusercontent.com/stanfordnlp/stanza-resources/master/resources_1.0.0.json: 115kB [00:00, 24.5MB/s]
2020-07-07 21:08:34 INFO: Downloading default packages for language: en (English)...
Traceback (most recent call last):
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect
conn = self._new_conn()
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa28f172438>: Failed to establish a new connection: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/urllib3/connectionpool.py", line 725, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='nlp.stanford.edu', port=80): Max retries exceeded with url: /software/stanza/1.0.0/en/default.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa28f172438>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/stanza/utils/resources.py", line 236, in download
request_file(f'{url}/{__resources_version__}/{lang}/default.zip', os.path.join(dir, lang, f'default.zip'), md5=resources[lang]['default_md5'])
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/stanza/utils/resources.py", line 83, in request_file
download_file(url, path)
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/stanza/utils/resources.py", line 66, in download_file
r = requests.get(url, stream=True)
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/mnt/ilcodisk1/user/dernoncourt/pyenv/neural-parser2/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='nlp.stanford.edu', port=80): Max retries exceeded with url: /software/stanza/1.0.0/en/default.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa28f172438>: Failed to establish a new connection: [Errno 110] Connection timed out',))
How can I download a stanza's model manually? I'm hoping the stanza's models are stored in more than one servers.
I use stanza==1.0.1.
I've checked with the Stanford NLP Group on Twitter and they've confirmed that the Standford NLP site (https://nlp.stanford.edu/) is down today (8 July) due to restructuring the Stanford-level data center. They'll be back tomorrow (9 July) and then your script should work again.
They've also now posted this on Twitter:
https://twitter.com/stanfordnlp/status/1280917602718461952?s=20
While using the Python requests module to check for broken links (expected behavior being returned error codes for unavailable URLs such as 404, etc.), I encountered an older link on a page for a domain which no longer exists, and a DNS error was returned.
<snip>
http://listen.grooveshark.com/s/Folk+Song/2Np5i?src=5
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 171, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 56, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
self.send(msg)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
self.connect()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 196, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 180, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x1088ff748>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 445, in send
timeout=timeout
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='listen.grooveshark.com', port=80): Max retries exceeded with url: /s/Folk+Song/2Np5i?src=5 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1088ff748>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./four_finder.py", line 64, in <module>
do_main()
File "./four_finder.py", line 48, in do_main
resp = requests.get(link)
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 513, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='listen.grooveshark.com', port=80): Max retries exceeded with url: /s/Folk+Song/2Np5i?src=5 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1088ff748>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
[user#host:~/dev|22:13:13]
$ nslookup listen.grooveshark.com
(...)
** server can't find listen.grooveshark.com: SERVFAIL
[user#host:~/dev|22:28:05]
What is the best way to preempt this error from occurring? I can't be confident of future input URLs existing (as this example does not exist).
This is solved by using a try/except block and requests.ConnectionError. I hadn't searched SE for the right terms, and a useful answer is here.
try:
resp = requests.get(link)
except requests.ConnectionError as e:
### TODO: Properly handle this DNS error!
print('DNS ERROR: %s' % link)
continue
Also of interest is this answer.