I am trying to write a Python script that will allow me to accomplish what I normally would by using CURL to perform an API "GET". I have browsed through some questions on here but I am still a bit confused on how to do this; from what I have seen, I can use the "requests" library like this:
URL = requests.get("www.example.com/123")
However, the Curl command I normally run uses authentication as well. Here is an example:
curl -X GET -H 'Authorization: hibnn:11111:77788777YT666:CAL1' 'http://api.example.com/v1.11/user?id=123456
Can I still use the requests library when creating a script to pull this data? Specifically, I would need to be able to pass along the authentication info along with the URL itself.
Update:
Here is what my code looks like:
import requests
import json
url = ("api.example.com/v1.11/user?id=123456")
header = {"Authorization": "hibnn:11111:77788777YT666:CAL1"}
response = requests.get(url, headers=header)
print(response.json)
However, I am getting a response [401] error. I think I am probably doing something wrong with my headers, can anyone help?
You may pass headers as keyword argument to the get method.
>>> url = 'https://api.github.com/some/endpoint'
>>> headers = {'user-agent': 'my-app/0.0.1'}
>>> r = requests.get(url, headers=headers)
Reference
Related
I've never really attempted to try and write my own code that calls an API. I have some Python code that I created after discovering Python Requests library.
However, I can't get past this "authentication failed" error message.
The API key I have acquired is from https://fortnitetracker.com/site-api
The code I am using is as follows:
import requests
url = 'https://api.fortnitetracker.com/v1/profile/pc/ninja'
headers = {"TRN-Api-Key": "MY_KEY"}
r = requests.get(url, headers=headers)
After asking for the r.status_code I get a 403 Forbidden. When I ask for r.text I get
u'{"message":"Invalid authentication credentials"}\n'
On the API page,all they say is to pass the API key in the request headers using a GET method.
I even tried passing my credentials I used to register on the site using
r = requests.get(url, headers=headers,auth=('User', 'Pass'))
Still got the same invalid authentcation credentials error.
Is what I am doing correct? What am I missing?
Thanks for your help in advance.
If you are calling an API you need to send post :
r = requests.post(url, headers=headers)
Are you sure you don't have to supply the API key something like this?
{'Auth': 'Token my_key'}
That's how most API's do it, but I can't find any reference to their API docs anywhere.
I am trying to download my data by using Fitbit API. I have figured out how to obtain a certain day's data, which is good. And here is the curl command I used:
curl -i -H "Authorization: Bearer (here goes a very long token)" https://api.fitbit.com/1/user/-/activities/heart/date/2016-6-14/1d/1sec/time/00:00/23:59.json >> heart_rate_20160614.json
However, I would like to collect hundreds of days' data and I don't want to do that manually. So I think I could use a Python loop. I read some other topics like this one and this one but still don't know how to 'translate' these curl commands into python language by using urllib2.
I have tried this:
import urllib2
url = 'https://api.fitbit.com/1/user/-/activities/heart/date/today/1d/1sec/time/00:00/00:01.json'
data = '{Authorization: Bearer (here goes a very long token)}'
req = urllib2.Request(url,data)
f = urllib2.urlopen(req)
but the got an error says "HTTP Error 404: Not Found"
So what is the correct way to 'translate' this curl command to python language? Thanks!
The problem comes from the construction of the Request object : by default, the second parameter is the data that you want to pass along with the request. Instead, you have to specify that you want to pass headers. This is the correct way to do it :
import urllib2
url = 'https://api.fitbit.com/1/user/-/activities/heart/date/2016-6-14/1d/1sec/time/00:00/23:59.json'
hdr = {'Authorization': 'Bearer (token)'}
req = urllib2.Request(url,headers=hdr)
f = urllib2.urlopen(req)
This wields a 401 on my side, but should work with your token.
You can have more informations on urllib2 (and the Request class) here
However, I suggest you take a look at Requests, which is in my opinion easier to use, and very well documented.
Hope it'll be helpful.
You can use the excellent lib requests which is much easier to use than urllib, in my opinion.
First, pip install requests, then in your interpreter:
import requests
response = requests.get(url='https://api.fitbit.com/1/user/-/activities/heart/date/2016-6-14/1d/1sec/time/00:00/23:59.json', headers={'Authorization':'Bearer <TOKEN>'})
if response.ok:
print response.content
else:
print "error", response.content
From here you can easily get the response content via response.content or response.json() if it's a JSON, and write it to a file.
rocksteady's solution worked
He did originally refer to dictionaries. But the following code to send the JSON string also worked wonders using requests:
import requests
headers = {
'Authorization': app_token
}
url = api_url + "/b2api/v1/b2_get_upload_url"
content = json.dumps({'bucketId': bucket_id})
r = requests.post(url, data = content, headers = headers)
I'm working with an API that requires me to send JSON as a POST request to get results. Problem is that Python 3 won't allow me to do this.
The following Python 2 code works fine, in fact it's the official sample:
request = urllib2.Request(
api_url +'/b2api/v1/b2_get_upload_url',
json.dumps({ 'bucketId' : bucket_id }),
headers = { 'Authorization': account_authorization_token }
)
response = urllib2.urlopen(request)
However, using this code in Python 3 only makes it complain about data being invalid:
import json
from urllib.request import Request, urlopen
from urllib.parse import urlencode
# -! Irrelevant code has been cut out !-
headers = {
'Authorization': app_token
}
url = api_url + "/b2api/v1/b2_get_upload_url"
# Tested both with encode and without
content = json.dumps({'bucketId': bucket_id}).encode('utf-8')
request = Request(
url=url,
data=content,
headers=headers
)
response = urlopen(req)
I've tried doing urlencode(), like you're supposed to. But this returns a 400 status code from the web server, because it's expecting pure JSON. Even if the pure JSON data is invalid, I need to somehow force Python into sending it.
EDIT: As requested, here are the errors I get. Since this is a flask application, here's a screenshot of the debugger:
Screenshot
Adding .encode('utf-8') gives me an "Expected string or buffer" error
EDIT 2: Screenshot of the debugger with .encode('utf-8') added
Since I have a similar application running, but the client still was missing, I tried it myself.
The server which is running is from the following exercise:
Miguel Grinberg - designing a restful API using Flask
That's why it uses authentication.
But the interesting part: Using requests you can leave the dictionary as it is.
Look at this:
username = 'miguel'
password = 'python'
import requests
content = {"title":"Read a book"}
request = requests.get("http://127.0.0.1:5000/api/v1.0/projects", auth=(username, password), params=content)
print request.text
It seems to work :)
Update 1:
POST requests are done using requests.post(...)
This here describes it well : python requests
Update 2:
In order to complete the answer:
requests.post("http://127.0.0.1:5000/api/v1.0/projects", json=content)
sends the json-string.
json is a valid parameter of the request and internally uses json.dumps()...
Why next two methods of obtaining authentication token are not equivalent?
First one is using curl in terminal:
curl -X POST "http://myurl.com" -d "grant_type=password&username=me&password=mypass&client_secret=123&client_id=456"
This request successfully returns the token.
If I use requests library for python:
import requests
url = 'http://myurl.com'
query_args = { "grant_type":"password",
"username":'me',
"password":'mypass',
"client_secret":'123',
'client_id':'456'}
r = requests.get(url, data=query_args)
the result I get is r.status_code is 404, so I cannot get the token.
Why the first method works and the second does not?
Also, how to make the second approach work?
Thank you!
So the curl -d flag send data as post data, from the man page:
-d/--data <data>
(HTTP) Sends the specified data in a POST request to
the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded. Compare to -F/--form.
So you should be using request.post. You will also possibly want to use the params attribute not the data one, see the following example:
>>> data = {'test':'12345'}
>>> url = 'http://myurl.com'
>>> r = requests.post(url,params=data)
>>> print r.url
http://myurl.com/?test=12345
>>> r = requests.post(url,data=data)
>>> print r.url
http://myurl.com/
>>>
I have been trying to learn Solr4.0, and I am looking at the JSON document update in their documentation which looks like the following:
cd example/exampledocs
curl 'http://localhost:8983/solr/update/json?commit=true' --data-binary #books.json -H 'Content-type:application/json'
It works fine and I can see the updated documents on my solr index. However, I was wondering how I can use this curl command in python via urllib2. So, something like:
theurl=r"""http://localhost:8983/solr/update/json?commit=true --data-binary #books.json -H 'Content-type:application/json'"""
import urllib2
import httplib
import cookielib
...use urllib2 to post theurl
However, this does not work. It looks like urllib2 does not recognize (such as the -H which obviously seems curl specific) the above formed theurl. How should one be formatting theurl such that I could use it with urllib2?
I would try
import urllib2
with open('books.json', 'rb') as data_file:
my_data = data_file.read()
req = urllib2.Request(url='http://localhost:8983/solr/update/json?commit=true',
data=my_data)
req.add_header('Content-type', 'application/json')
f = urllib2.urlopen(req)
# Begin using data like the following
print f.read()
From this you can see that the --data-binary parameter is just data sent to the server like in a POST request. When that parameter begins with the # sign, it means to read data from a file. In this case, it's the file 'books.json'. You also need to send the header (the -H parameter of curl). So you only need to call the add_header method with the header name and its value.
Hope that gets you started. More info about urllib2 can be found at http://docs.python.org/2/library/urllib2.html
Because urllib2 is not available in Python 3.x I offer this alternative.
This code snippet worked for me using Python 3.3 and the excellent requests library
import requests
def postXml(host, xmlFile):
url = "http://%s:8983/solr/update" % host
headers = {"content-type" : "text/xml" }
params = {"commit" : "false" }
payload = open(xmlFile, "rb").read()
r = requests.post(url, data=payload, params=params, headers=headers)
print("got back: %s" % r.text)