I use google safe browsing API. So I tested this simple code:
from safebrowsinglookup import SafebrowsinglookupClient
class TestMe:
def testMe(self):
self.key='my_Valid_Key_Here'
self.client=SafebrowsinglookupClient(self.key)
self.response=self.client.lookup('http://www.google.com')
print(self.response)
if __name__=="__main__":
TM=TestMe()
TM.testMe()
I always get this whatever the website I test:
{'website_I_tried','error'}
Note that I had to change some lines in the source code after I installed this API because it was written in Python 2 and I am using Python 3.4.1. How can I resolve this problem?
Update:
To understand why the above problem occured to me, I run this code:
from safebrowsinglookup import SafebrowsinglookupClient
class TestMe:
def testMe(self):
self.key = 'my_key_here'
self.client=SafebrowsinglookupClient(self.key,debug=1)
urls = ['http://www.google.com/','http://addonrock.ru/Debugger.js/']
self.results = self.client.lookup(*urls)
print(self.results['http://www.google.com/'])
if __name__ == "__main__":
TM=TestMe()
TM.testMe()
Now, I got this message:
BODY:
2
http://www.google.com/
http://addonrock.ru/Debugger.js/
URL: https://sb-ssl.google.com/safebrowsing/api/lookup?client=python&apikey=ABQIAAAAAU6Oj8JFgQpt0AXtnVwBYxQYl9AeQCxMD6irIIDtWpux_GHGQQ&appver=0.1&pver=3.0
Unexpected server response
name 'urllib2' is not defined
error
error
The library doesn't support Python3.x.
In this case, you can either make it support Python3 (there is also an opened pull request for Python3 Compatibility), or make the request to "Google Safebrowsing API" manually.
Here's an example using requests:
import requests
key = 'your key here'
URL = "https://sb-ssl.google.com/safebrowsing/api/lookup?client=api&apikey={key}&appver=1.0&pver=3.0&url={url}"
def is_safe(key, url):
response = requests.get(URL.format(key=key, url=url))
return response.text != 'malware'
print(is_safe(key, 'http://addonrock.ru/Debugger.js/')) # prints False
print(is_safe(key, 'http://google.com')) # prints True
Just the same, but without third-party packages (using urllib.request):
from urllib.request import urlopen
key = 'your key here'
URL = "https://sb-ssl.google.com/safebrowsing/api/lookup?client=python&apikey={key}&appver=1.0&pver=3.0&url={url}"
def is_safe(key, url):
response = urlopen(URL.format(key=key, url=url)).read().decode("utf8")
return response != 'malware'
print(is_safe(key, 'http://addonrock.ru/Debugger.js/')) # prints False
print(is_safe(key, 'http://google.com')) # prints True
Related
I would like to get all the GitHub branches of a certain repository. Using the GitHub API as documented here, https://developer.github.com/v3/repos/branches/#list-branches I try to use the GET request documented.
However, when I try unit testing, the response always evaluates to None and I get error: AttributeError: 'NoneType' object has no attribute 'json'. I am unsure if my issue is with the API call, or I am testing the call. See code below:
#staticmethod
def get_branches(git_base, org, repo):
github_repo_url = f"https://{git_base}/api/v3/repos/{org}/{repo}"
collected_all_branches: bool = False
page = 1
github_branches = []
# collect all branches in repo
while not collected_all_branches:
response = (requests.get(f"{github_repo_url}/branches/?per_page=100&page={page}",
headers=HEADERS))
if len(response.json()) == 0:
collected_all_branches = True
else:
github_branches.extend([response["name"] for response in response.json()])
page = page + 1
return github_branches
Unit Test:
#patch.object(requests, "get", return_value=None)
def test_get_branches(self, mock_get):
r = GithubService.get_branches("test", "test", "test")
mock_get.assert_called_with("test", "test", "test")
The test is not set up correctly. Since the unit test verifies the arguments of the requests.get() call you can modify it like this:
#patch("requests.get")
def test_get_branches(self, mock_get):
# Give the actual HEADERS you use in the get_branches() function call, for example something like:
HEADERS = {"Accept": "application/vnd.github+json"}
r = GithubService.get_branches("test", "test", "test")
mock_get.assert_called_with("https://api.github.com/repos/test/test/branches/?per_page=100&page=1", headers=HEADERS)
The test will now pass successfully.
Also, the Github API call is incorrect. The documentation you have linked to says the URL must be of this format:
https://api.github.com/repos/OWNER/REPO/branches
So in get_branches() make the following change:
github_repo_url = f"https://api.github.com/repos/{org}/{repo}"
I am new to bottle. I have searched for the answer but could not get any.
Is there anyway to load a page without redirect?
The code is like this:
from bottle import get, run, template
#get('/list/item')
def listItems():
r = requests.get('my/url/which/works/')
return r.content
if __name__ == '__main__':
run(host='localhost', port=8080 )
and the webpage is empty. I have also tried return r.text, return template(r.content), return str(r.content), return str(r.content.decode('utf-8')), and
nf = urllib.urlopen('my/url/whick/works')
return nf.read()
none of them returns the page I want.
However, if I write this return r.content[0], it works. the page will show the first letter, which is '<'. But if I write return r.content[0:100], it returns a empty page again.
If I run the requests in command line, this is what it returns:
>>> import requests
>>> r = requests.get('my/url/which/works/')
>>>
>>> r.content
'<?xml version="1.0" encoding="utf-8" ?> \n<body copyright="...>\n</body>\n'
Is that possible that anyone can help about this? Thank you very much.
Your question doesn't specify what you expect to see when your code works as expected, but this may help you:
from bottle import get, run, template, response
#get('/list/item')
def listItems():
r = requests.get('my/url/which/works/')
response.content_type = 'text/plain' # added this
return r.content
...
The only change is that your webserver will now be setting the content type of the response to text/plain, which should make your browser render the page.
Longer term, if (as I'm inferring from your question) you intend to return XML responses, then I suggest either installing a browser plugin (here's an example), or, better yet, use curl or wget to see the precise response. This will help you debug your server's responses.
the code can work before but i found it can not work recently. please saw my code as below
def signal_distance(lat1,lng1,lat2,lng2):
import simplejson, urllib
orig_coord =lat1,lng1
dest_coord = lat2,lng2
#API request
url = "https://maps.googleapis.com/maps/api/distancematrix/json?origins={0}&destinations={1}&mode=transit&language=zh-TW&key=AIzaSyBlwZDhGYNTrxXiQblz20v3poJTA7zTVho".format(str(orig_coord),str(dest_coord))
result = simplejson.load(urllib.urlopen(url))
print result
signal_distance(25.082969,121.5549714,24.9988582,121.5788795)
It seems the Distance Matrix API request that generates signal_distance function is not valid since it contains parentheses for origins and destinations parameters.
The specified example returns for me NOT_FOUND Element-level status code:
indicates that the origin and/or destination of this pairing could not
be geocoded
The solution would be to replace request from:
https://maps.googleapis.com/maps/api/distancematrix/json?origins=(<latOrig>,<lngOrig>)&destinations=(<latDest>,<lngDest>)&mode=transit&language=<lang>&key=<Key>
to
https://maps.googleapis.com/maps/api/distancematrix/json?origins=<latOrig>,<lngOrig>&destinations=<latDest>,<lngDest>&mode=transit&language=<lang>&key=<Key>
Modified example
def signal_distance(lat1,lng1,lat2,lng2):
import simplejson, urllib
orig_coord = lat1,lng1
dest_coord = lat2,lng2
orig_coord_str = ' ,'.join(map(str, orig_coord))
dest_coord_str = ' ,'.join(map(str, dest_coord))
#API request
url = "https://maps.googleapis.com/maps/api/distancematrix/json?origins={0}&destinations={1}&mode=transit&language=zh-TW&key=AIzaSyBlwZDhGYNTrxXiQblz20v3poJTA7zTVho".format(orig_coord_str,dest_coord_str)
result = simplejson.load(urllib.urlopen(url))
print result
I am trying to read data sent to python's http.server from a local javascript program posted with AJAX. Everything works in python 2.7 as in this example, but now in python 3+ I can't access the header anymore to get the file length.
# python 2.7 (works!)
class handler_class(SimpleHTTPServer.SimpleHTTPRequestHandler):
def do_POST(self):
if self.path == '/data':
length = int(self.headers.getheader('Content-Length'))
NewData = self.rfile.read(length)
I've discovered I could use urllib.request, as I have mocked up below. However, I am running on a localhost and don't have a full url as I've seen in the examples, and I am starting to second guess if this is even the right way to go? Frustratingly, I can see the content-length printed out in the console, but I can't access it.
# python 3+ (url?)
import urllib.request
class handler_class(http.server.SimpleHTTPRequestHandler):
def do_POST(self):
if self.path == '/data':
print(self.headers) # I can see the content length here but cannot access it!
d = urllib.request.urlopen(url) # what url?
length = int(d.getheader('Content-Length'))
NewData = self.rfile.read(length)
Various url's I have tried are:
self.path
http://localhost:8000/data
/data
and I generally get this error:
ValueError: unknown url type: '/data'
So why is 'urllib.request' failing me and more importantly, how does one access 'self.header' in this Python3 world?
import urllib, urllib2, json
def make_request(method, base, path, params):
if method == 'GET':
return json.loads(urllib2.urlopen(base+path+"?"+urllib.urlencode(params)).read())
elif method == 'POST':
return json.loads(urllib2.urlopen(base+path, urllib.urlencode(params)).read())
api_key = "5f1d5cb35cac44d3b"
print make_request("GET", "https://indit.ca/api/", "v1/version", {"api_key": api_key})
This set of code returns should return back the version and status like {status: 'ok', version: '1.1.0'}
What code do I need to add to print that response ?
It's hard to tell what the problem is without a complete, otherwise-working example (I can't even resolve host indit.ca), but I can explain how you can debug this yourself. Break it down step by step:
import urllib, urllib2, json
def make_request(method, base, path, params):
if method == 'GET':
url = base+path+"?"+urllib.urlencode(params)
print 'url={}'.format(url)
req = urllib2.urlopen(url)
print 'req={}'.format(req)
body = req.read()
print 'body={}'.format(body)
obj = json.loads(body)
print 'obj={}'.format(obj)
return obj
elif method == 'POST':
# You could do the same here, but your test only uses "GET"
return json.loads(urllib2.urlopen(base+path, urllib.urlencode(params)).read())
api_key = "5f1d5cb35cac44d3b"
print make_request("GET", "https://indit.ca/api/", "v1/version", {"api_key": api_key})
Now you can see where it goes wrong. Is it generating the right URL? (What happens if you paste that URL into a browser address bar, or a wget or curl command line?) Does urlopen return the kind of object you expected? Does the body look right? And so on.
Ideally, this will solve the problem for you. If not, at least you'll have a much more specific question to ask, and are much more likely to get a useful answer.