from urllib2 import urlopen
from contextlib import closing
import json
import time
import os
while True:
url = 'http://freegeoip.net/json/'
try:
with closing(urlopen(url)) as response:
location = json.loads(response.read())
location_city = location['city']
location_state = location['region_name']
location_country = location['country_name']
#print(location_country)
if location_country == "Germany":
print("You are now surfing from: " + location_country)
os.system(r'firefox /home/user/Documents/alert.html')
except:
print("Could not find location, searching again...")
time.sleep(1)
Its doesn't reply any country can I get help to solve the problem?
Besides of the wrong indentation, your code looks fine.
The problem seems to be that the page itself does not respond. If you try to open it in a browser for example, the connection gets refused.
Probably the api is either overloaded, or does no longer exist.
For one thing, the server appears to be down.
You would probably have noticed this but the bare except hides the fact. In general you should not catch all exceptions, but should catch those that you expect - in this case a urllib2.URLError exception would seem appropriate:
import urllib2
url = 'http://freegeoip.net/json/'
try:
response = urllib2.urlopen(url)
...
except urllib2.URLError as exc:
print('Could not find location due to exception: {}'.format(exc))
If you run the code above you might see this output:
Could not find location due to exception: <urlopen error [Errno 101] Network is unreachable>
The server might have been up earlier, and the problem might actually have a different cause, e.g. json.loads() might be failing. If you change the exception handler as shown above you will be able to see where it's failing.
Related
I'm setting a python unshort url code and I want
I want the long version of the shorted URLs in a file to be written on the screen.
There is no hassle when a normal URL arrives.
But I get an error when a malicious URL arrives.
I use the following code:
import requests
import urllib
dosya=open("urller.txt","r")
satirlar=dosya.readlines()
for satir in satirlar:
resp = urllib.urlopen(satir)
print(resp.url)
dosya.close()
For the script to complete successfully you want to add error handling. Maybe something like the following:
import requests
import urllib
dosya=open("urller.txt","r")
satirlar=dosya.readlines()
for satir in satirlar:
try:
resp = urllib.urlopen(satir)
print(resp.url)
except urllib.error.URLError as e:
print("Failed to open URL {0} Reason: {1}".format(satir, e.reason))
dosya.close()
This code will print all valid response URLs. For any invalid URL it will instead print an error report.
I have to make a series of requests to my localserver and check response. Basically I am trying to hit the right url by brute forcing. This is my code:
for i in range(48,126):
test = chr(i)
urln = '012a4' + test
url = {"tk" : urln}
data = urllib.urlencode(url)
print data
request = urllib2.Request("http://127.0.0.1/brute.php", data)
response = urllib2.urlopen(request)
status_code = response.getcode()
I've to make request like: http://127.0.0.1/brute.php?tk=some_val
I am getting an error because the url is not properly encoding. I am internal server error 500 even when one of the url in series should give 200. manually giving that url confirms it. Also, what is the right way to skip 500/400 errors until I get a 200?
When using urllib2 you should always handle any exceptions that are raised as follows:
import urllib, urllib2
for i in range(0x012a40, 0x12a8e):
url = {"tk" : '{:x}'.format(i)}
data = urllib.urlencode(url)
print data
try:
request = urllib2.Request("http://127.0.0.1/brute.php", data)
response = urllib2.urlopen(request)
status_code = response.getcode()
except urllib2.URLError, e:
print e.reason
This will display the following when the connection fails, and then continue to try the next connection:
[Errno 10061] No connection could be made because the target machine actively refused it
e.reason will give you the textual reason, and e.errno will give you the error code. So you could still stop if the error was something other than 10061 for example.
Lastly, you seem to be cycling through a range of numbers in hex format? You might find it easier to work directly with 0x formatting to build your strings.
It sounds like you will benefit from a try/except block:
for i in range(48,126):
test = 'chr(i)'
new urln = '012a4' + test
url = {"tk" : urln}
data = urllib.urlencode(url)
print data
request = urllib2.Request("http://127.0.0.1/brute.php", data)
try:
response = urllib2.urlopen(request)
except:
status_code = response.getcode()**strong text**
print status_code
You typically would also want to catch the error as well:
except Exception, e:
print e
Or catch specific errors only, for example:
except ValueError:
#do stuff
Though you wouldn't get a ValueError in your code.
I am using the following code to resolve redirects to return a links final url
def resolve_redirects(url):
return urllib2.urlopen(url).geturl()
Unfortunately I sometimes get HTTPError: HTTP Error 429: Too Many Requests. What is a good way to combat this? Is the following good or is there a better way.
def resolve_redirects(url):
try:
return urllib2.urlopen(url).geturl()
except HTTPError:
time.sleep(5)
return urllib2.urlopen(url).geturl()
Also, what would happen if there is an exception in the except block?
It would be better to make sure the HTTP code is actually 429 before re-trying.
That can be done like this:
def resolve_redirects(url):
try:
return urllib2.urlopen(url).geturl()
except HTTPError, e:
if e.code == 429:
time.sleep(5);
return resolve_redirects(url)
raise
This will also allow arbitrary numbers of retries (which may or may not be desired).
https://docs.python.org/2/howto/urllib2.html#httperror
This is a fine way to handle the exception, though you should check to make sure you are always sleeping for the appropriate amount of time between requests for the given website (for example twitter limits the amount of requests per minute and has this amount clearly shown in their api documentation). So just make sure you're always sleeping long enough.
To recover from an exception within an exception, you can simply embed another try/catch block:
def resolve_redirects(url):
try:
return urllib2.urlopen(url).geturl()
except HTTPError:
time.sleep(5)
try:
return urllib2.urlopen(url).geturl()
except HTTPError:
return "Failed twice :S"
Edit: as #jesse-w-at-z points out, you should be returning an URL in the second error case, the code I posted is just a reference example of how to write a nested try/catch.
Adding User-Agent to request header solved my issue:
from urllib import request
from urllib.request import urlopen
url = 'https://www.example.com/abc.json'
req = request.Request(url)
req.add_header('User-Agent', 'abc-bot')
response = request.urlopen(req)
so I want to check if a URL is reachable from python, and I got this code from googling:
def checkUrl(url):
p = urlparse(url)
conn = http.client.HTTPConnection(p.netloc)
conn.request('HEAD', p.path)
resp = conn.getresponse()
return resp.status < 400
Here is my URL: https://eurotableau.nomisonline.com.
It works fine if I just pass that in to the function. The resp.status is 302. However, if I add a port 443 at the end of it, https://eurotableau.nomisonline.com:443, it returns false. The resp.status is 400. I tried both URL in google Chrome, both of them work. So my question is why is this happening? Anyway I can include the port value and still get valid resp.status value (< 400)? Thanks.
Use http.client.HTTPSConnection instead. The plain old HTTPConnection ignores the protocol that is part of the URL.
If you do not require the HEAD method but just wish to check if host is available then why not do:
from urllib2 import urlopen
try:
u = urlopen("https://eurotableau.nomisonline.com")
u.close()
print "Everything fine!"
except Exception, e:
if hasattr(e, "code"):
print "Server is there but something is wrong with rest of URL"
else: print "Server is on vacations or was never there!"
print e
This will establish a connection with server but it won't download any data unless you read it. It'll only read few KB to get the header (like when using HEAD method) and wait for you to request more. But you will close it there.
So, you can catch an exception and see what the problem is, or if there is no exception, just close the connection.
urllib2 will handle HTTPS and protocol://user#URL:PORT for you neatly.
No worries about anything.
In a shorten-er built by web2by i want to validate url's first, if it's not valid goes back to the first page with an error message. this is my code in controller (mvc arch.) but i don't get what's wrong..!!
import urllib
def index():
return dict()
def random_maker():
url = request.vars.url
try:
urllib.urlopen(url)
return dict(rand_url = ''.join(random.choice(string.ascii_uppercase +
string.digits + string.ascii_lowercase) for x in range(6)),
input_url=url)
except IOError:
return index()
Couldn't you check the http response code using httplib. If it was 200 then the page is valid, if it is anything else (like 404) or an error then it is invalid.
See this question: What’s the best way to get an HTTP response code from a URL?
Update:
Based on your comment it looks like your issue is how you are handling the error. You are only handling IOError issues. In your case you can either handle all errors singularly by switching to:
except:
return index()
You could also build your own exception handler by overriding http_default_error. See How to catch 404 error in urllib.urlretrieve for more information.
Or you can switch to urllib2 which has specific errors, You can then handle the specific errors that urllib2 throws like this:
from urllib2 import Request, urlopen, URLError
req = Request('http://jfvbhsjdfvbs.com')
try:
response = urlopen(req)
except URLError, e:
if hasattr(e, 'reason'):
print 'We failed to reach a server.'
print 'Reason: ', e.reason
elif hasattr(e, 'code'):
print 'The server couldn\'t fulfill the request.'
print 'Error code: ', e.code
else:
print 'URL is good!'
The above code with that will return:
We failed to reach a server.
Reason: [Errno 61] Connection refused
The specifics of each exception class is contained in the urllib.error api documentation.
I am not exactly sure how to slot this into your code, because I am not sure exactly what you are trying to do, but IOError is not going to handle the exceptions thrown by urllib.