Python: request.get for github doesn't work - python

I am trying to run a GET request for a Github url. Unfortunately, I always get an error message.
I tried it for several different websites and it works, just not for github.
I am trying to do it with Jupyter Notebooks in Python, if that is important.
Here is the Error message:
ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /jana-hoh/gdp/main/DP_LIVE_22102021141534889.csv (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7a1c285d60>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

An error message that includes Temporary failure in name resolution indicates that the system's DNS server is unable to convert domain names into their corresponding IP address. Some of the causes are:
Your DNS configuration is correct, but the server is unable to respond to DNS requests at the moment
Firewall rules
No internet connectivity
Most of the times I've encountered this error stemmed from being disconnected to the internet. However, if your internet is working properly, you can try to add another DNS server in /etc/resolv.conf. For example, you can add cloudflare's:
nameserver 1.1.1.1

Related

Python-Telegram-Bot not running behind firewall

I've been searching about this for several days to no avail. I have a python bot ( polling for updates) running OK at home or any public internet. However, when at work behind a firewall, the bot cannot connect to the server. I believe that the application must know the proxy server, user ID and password in order to proceed. But I cannot a find the way or how to include this info in the bot application. Below is the error message:
2017-03-13 07:13:44,233 - telegram.ext.updater - ERROR - Error
while getting Updates: urllib3 HTTPError HTTPSConnectionPool
(host='api.telegram.org', port=443):Max retries exceeded with
url: /botXXXXXXXXX:Token/getUpdates (Caused byNewConnectionError
('<urllib3.connection.VerifiedHTTPSConnection object at 0x031541F0>:
Failed to establish a new connection: [Errno 10061] No connection
could be made because the target machine actively refused it',))
/getUpdates (Caused by NewConnectionError ('
Other thing: The Telegram messenger application runs OK behind this same firewall without any info on the proxy server. So it can connect to the server with no problem. I mention this because another cause could be my company uses websense or something like that to block the telegram server , but it is not the case as the messenger application do work OK.
Thanks a million in advance for any hint.
UPDATE JULY 26th, 2017: The solution was as suggested by Sudheesh. The environmental variables https_proxy needs to be set. At the time of this answer it seems to be I've entered the wrong proxy server or in the wrong way. looking around internet , I've noticed the way to set this is (in windows) is:
set https_proxy=http://proxy_url
Notice the right side of the equal show http ( not https)
Thanks to Sudheesh again!

ebaysdk-python Connection Error

I have a django project which is currently being hosted on pythonanywhere that uses the Finding api of the open source project ebaysdk-python. Now on my local machine, the site worked perfectly, however, when I execute the api call I get this error message: HTTPConnectionPool(host='svcs.ebay.com', port=80): Max retries exceeded with url: /services/search/FindingService/v1 (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f6560105150>: Failed to establish a new connection: [Errno 111] Connection refused',)).
Now I have scoured the docs and other related questions, but could not figure out the issue. I have verified that my API keys are correct, and my code to execute the api call is straight from the docs. So that I may be pointed in the correct direction: What is the most likely cause for this error to be raised under these circumstances?
Thank you.

Socrata SODA API is rejecting with Max Retries Exceeded

I am using the REST API Modular Input within Splunk to GET data.SFGov.org data via SODA API. I have an APP TOKEN. I am getting the MAX RETRIES EXCEEDED repeatedly.
Background:
I am building a proto Splunk based stream cursor for SF opendata. I have been testing a GET using the REST API MODULAR INPUT from the configuration screen itself, have not written any python code yet. Here is the ERROR.
11-30-2016 16:24:57.432 -0800 ERROR ExecProcessor - message from "python /Applications/Splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request: HTTPSConnectionPool(host='data.sfgov.org', port=443): Max retries exceeded with url: [REDACTED] (Caused by : [Errno 8] nodename nor servname provided, or not known)
I found out that by mistake, the REST API module's polling interval was set to 60 seconds and it might have caused a problem? (I changed it to ONE DAY to avoid future issues). I then got a new APP TOKEN and tried a GET. I see the get going out in the log, but the same MAX RETRIES EXCEEDED error is coming. I am using the same IP address.
I will be testing for the next few weeks. How do I fix this and gracefully avoid this again?
#chrismetcalf - just flagging you.
Max Retries Exceeded is not an error message that I'd expect to see out of our API, especially if you were only making a call every 60 seconds. I think that may actually be Splunk giving up after trying and failing to make your HTTP call too many times.
The error message Caused by : [Errno 8] nodename nor servname provided, or not known makes me think that there's actually a DNS error on Splunk's side. That's the error message you usually see when a domain name can't be resolved.
Perhaps there's some DNS whitelisting you need to make in your Splunk environment?

sudden 'socket.gaierror': [Errno -2] Name or service not known

I am having the following error:
(Url and api are made up in this example).
(, ConnectionError(MaxRetryError("HTTPConnectionPool(host='urlICantDisplay.com', port=80): Max retries exceeded with url: /some_api/user_id/action_name (Caused by : [Errno -2] Name or service not known)",),), )
I use the same api for many users, but suddenly I start getting this error,
and from then I keep getting the error until I restart the process.
I've read this might be a congestion problem:
Random "[Errno -2] Name or service not known" errors
and pausing between calls might help, but this is a real time application that should not pause.
and also i would have presumed that the api would start working after a while,
I used the api again in the same process after 7 hours and still got the error.
I also read this is a dns error, but as i've said, dns works then suddenly stops working altogether.
Only restarting the process solved it.
I thought about saving the ip of the dns to stop asking the dns server to do it.
But i'm not sure if it will work or even connected.

socket Errno 10060

Im using requests to routinely download a webpage and check it for updates, but recently ive been getting these errors :
HTTPConnectionPool(host='somehost', port=someport): Max retries
exceeded with url: someurl (Caused by : [Errno
10060] A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond)
Now this script has been running for weeks with this issue never coming up. Could it be that the site administrator has started blocking my proxy's IP?
I should add that its not against the TOS of the site to scrape it.
Can anyone help me figure out whats the reason for this?
Thanks
The remote connection timed out.
The host you are trying to connect to is not answering; it is not refusing connections, it is just not responding at all to connection attempts.
Perhaps the host is overloaded or down? It could also be caused by the site blocking your IP address by dropping the packets (a firewall DROP rule instead of a REJECT rule).
You can try to connect to the site from a different IP address; if those connections work fine, but not from the original address, there is a higher likelihood that you are deliberately being blocked.

Categories