Im using requests to routinely download a webpage and check it for updates, but recently ive been getting these errors :
HTTPConnectionPool(host='somehost', port=someport): Max retries
exceeded with url: someurl (Caused by : [Errno
10060] A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond)
Now this script has been running for weeks with this issue never coming up. Could it be that the site administrator has started blocking my proxy's IP?
I should add that its not against the TOS of the site to scrape it.
Can anyone help me figure out whats the reason for this?
Thanks
The remote connection timed out.
The host you are trying to connect to is not answering; it is not refusing connections, it is just not responding at all to connection attempts.
Perhaps the host is overloaded or down? It could also be caused by the site blocking your IP address by dropping the packets (a firewall DROP rule instead of a REJECT rule).
You can try to connect to the site from a different IP address; if those connections work fine, but not from the original address, there is a higher likelihood that you are deliberately being blocked.
Related
I am trying to run a GET request for a Github url. Unfortunately, I always get an error message.
I tried it for several different websites and it works, just not for github.
I am trying to do it with Jupyter Notebooks in Python, if that is important.
Here is the Error message:
ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /jana-hoh/gdp/main/DP_LIVE_22102021141534889.csv (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7a1c285d60>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
An error message that includes Temporary failure in name resolution indicates that the system's DNS server is unable to convert domain names into their corresponding IP address. Some of the causes are:
Your DNS configuration is correct, but the server is unable to respond to DNS requests at the moment
Firewall rules
No internet connectivity
Most of the times I've encountered this error stemmed from being disconnected to the internet. However, if your internet is working properly, you can try to add another DNS server in /etc/resolv.conf. For example, you can add cloudflare's:
nameserver 1.1.1.1
I registered my own salesforce developer login.
I am able to connect to this from my home computer and my work computer via the salesforce login url.
I am now writing python code to extract from salesforce. The code is below.
The code runs on my work laptop when I am at home and connected to my ISP.
When running the same code on my work laptop at work, (so now using work ISP), the code fails to connect.
The error I get when I run at work is:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='login.salesforce.com', port=443): Max retries exceeded with url: /services/Soap/u/40.0 (Caused by NewConnectionError(': Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))
So expect something going on with firewall and what not.
But am confused. It does not seem right that my work laptop is more "open" to outside when I use my ISP. I would have thought the security / firewall would have been implemented in a layer between work laptop and the ISP. So ISP agnostic. These laptops are meant to be used at home as well. I am not doing anything wrong in that respect.
Python code below.
import unicodecsv
from salesforce_bulk import SalesforceBulk
bulk = SalesforceBulk(password='**', username='**', security_token='**')
job = bulk.create_query_job("Contact", contentType='CSV')
batch = bulk.query(job, "select Id,LastName from Contact")
bulk.close_job(job)
while not bulk.is_batch_done(batch):
sleep(10)
for result in bulk.get_all_results_for_query_batch(batch):
reader = unicodecsv.DictReader(result, encoding='utf-8')
for row in reader:
print(row) # dictionary rows
Oops. Figured it out. Need to add proxies parameter when connecting to salesforce at work. Bit of a revelation. There is a level of security/protection that is missing on a work laptop when it used at home. Did not realise that networks / firewall / security worked in such a way.
I've been searching about this for several days to no avail. I have a python bot ( polling for updates) running OK at home or any public internet. However, when at work behind a firewall, the bot cannot connect to the server. I believe that the application must know the proxy server, user ID and password in order to proceed. But I cannot a find the way or how to include this info in the bot application. Below is the error message:
2017-03-13 07:13:44,233 - telegram.ext.updater - ERROR - Error
while getting Updates: urllib3 HTTPError HTTPSConnectionPool
(host='api.telegram.org', port=443):Max retries exceeded with
url: /botXXXXXXXXX:Token/getUpdates (Caused byNewConnectionError
('<urllib3.connection.VerifiedHTTPSConnection object at 0x031541F0>:
Failed to establish a new connection: [Errno 10061] No connection
could be made because the target machine actively refused it',))
/getUpdates (Caused by NewConnectionError ('
Other thing: The Telegram messenger application runs OK behind this same firewall without any info on the proxy server. So it can connect to the server with no problem. I mention this because another cause could be my company uses websense or something like that to block the telegram server , but it is not the case as the messenger application do work OK.
Thanks a million in advance for any hint.
UPDATE JULY 26th, 2017: The solution was as suggested by Sudheesh. The environmental variables https_proxy needs to be set. At the time of this answer it seems to be I've entered the wrong proxy server or in the wrong way. looking around internet , I've noticed the way to set this is (in windows) is:
set https_proxy=http://proxy_url
Notice the right side of the equal show http ( not https)
Thanks to Sudheesh again!
I am writing an app with wxPython that incorporates pyodbc to access SQL Server. A user must first establish a VPN connection before they can establish a connection with the SQL server. In cases where a user forgets to establish a VPN connection or is simply not authorized to access a particular server, the app will freeze for up to 60+ seconds before it produces an error message. Often, users will get impatient and force-close the app before the error message pops up.
I wonder if there is a way to test whether it's possible to connect to the server without freezing up. I thought about using timeout, but it seems that timeout can be used only after I establish a connection
A sample connection string I use is below:
connection = pyodbc.connect(r'DRIVER={SQL Server};SERVER=ServerName;database=DatabaseName;Trusted_Connection=True;unicode_results=True')
See https://code.google.com/archive/p/pyodbc/wikis/Connection.wiki under timeout
Note: This attribute only affects queries. To set the timeout for the
actual connection process, use the timeout keyword of the
pyodbc.connect function.
So change your connection string to:
connection = pyodbc.connect(r'DRIVER={SQL Server};SERVER=ServerName;database=DatabaseName;Trusted_Connection=True;unicode_results=True', timeout=3)
should work
took a while before it threw an error message about server not existing or access being denied
Your comment conflates two very different kinds of errors:
server not existing is a network error. Either the name has no address, or the address is unreachable. No connection can be made.
access being denied is a response from the server. For the server to respond, a connection must exist. This is not to be confused with connection refused (ECONNREFUSED), which means the remote is not accepting connections on the port.
SQL Server uses TCP/IP. You can use standard network functions to determine if the network hostname of the machine running SQL Server can be found, and if the IP address is reachable. One advantage to using them to "pre-test" the connection is that any error you'll get will be much more specific than the typical there was a problem connecting to the server.
Note that not all delay-inducing errors can be avoided. For example, if the DNS server is not responding, the resolver will typically wait 30 seconds before giving up. If an IP address is valid, but there's no machine with that address, attempting a connection will take a long time to fail. There's no way for the client to know there's no such machine; it could just be taking a long time to get a response.
sunday 10/06 we were running a GAE Datastore throughput benchmark in python and are receiving this error-"[Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time or established connection failed because connected host has failed to respond"
we were generating requests for 6000 users [200 rps, 650 F4_1G instances] everything running fine.
when we added the next 100 users, we began receiving errno 10060 for all users.
we must be exceeding some System[university lab, internet, network devices, network links, GAE, Datastore...] limit. however; no errors are shown on any GAE logs.
thanks tom
From the question I'm understanding that Errno 10060 being generated inside your python test suite.
See this question: Python: URLError: <urlopen error [Errno 10060]
You're probably reaching the maximum number of outbound TCP connections on your test machine.