We Keep Coding

Python OAuth Client Credentials Grant Type Returned 400, [error: unsupported grant] type but Curl Works Fine

I am very new to APIs (still learning) and I encountered a very weird issue with Python requests library when trying to initiate an OAuth Authentication flow with Client Credentials Grant Type.
For some reason, whenever I used my Python script (with the help of requests library) to send the HTTP request to the authentication endpoint, I always get
Response Status Code: 400
Response Body/Data returned: {"error":"unsupported_grant_type"}
However, if I tried using curl command line tool to send the request, I will get a successful response with status code 200 with the access token in the response body like this:
{'access_token': 'some access token',
'expires_in': 'num_of_seconds',
'token_type': 'Bearer'}
As a matter of fact, if I tried sending the request using Curl command line tool WITHIN my Python Script (with subprocess.Popen function), I can get the response with status code 200 and the access token with no problem.
Now, with that said, here's the Python script that I used to send the request to initiate the OAuth authentication flow:
import requests
import os
import base64
clientCredentialEndpoint = "https://base_url/path/token"
client_id = os.environ.get('CLIENT_ID')
client_secret = os.environ.get('CLIENT_SECRET')
# -- Encode the <client_id:client_secret> string to base64 --
auth_value = f'{client_id}:{client_secret}'
auth_value_bytes = auth_value.encode('ascii')
auth_value_b64 = base64.b64encode(auth_value_bytes).decode('ascii')
queryParams ={
'grant_type':'client_credentials',
'scope':'get_listings_data'
}
headers = {
'Authorization':f'Basic {auth_value_b64}',
'Content-Type':'application/x-www-form-urlencoded'
}
# send the post request to Authorisation server
response = requests.post(
clientCredentialEndpoint,
params=queryParams,
headers=headers,
)
print(response.status_code)
print(response.text)
whereas the curl command that I used (and worked) to send the request is:
curl -X POST -u '<client_id>:<client_secret>' \
-H "Content-Type: application/x-www-form-urlencoded" \
-d 'grant_type=client_credentials&scope=get_listings_data' \
'https://base_url/path/token'
Again, like I said, if I execute this curl command inside a Python script, it will successfully return the access token with no issue.
Does anyone know what I did wrong in my Python script which caused my request to always fail?
Thanks in advance!

My goodness me, I just realised that the -d in the curl command does not correspond to query params, it stands for 'data'.
Hence, I just need to change my Python script requests.post() a bit so that it looks like this:
response = requests.post(
clientCredentialEndpoint,
data=queryParams,
headers=headers,
)
Hope this helps others.

How to use the bitbucket api to get list of all pull requests for a repo?

How do I construct a complete REST api request (using python) to retrieve the list of all pull requests for a repo based on some filters?
I'm using the url "https://bitbucket.org/api/2.0/repositories/YOUR_NAME/REPO_NAME/pullrequests"
I have the client key and secret from the oAuth settings.
I'm getting the access token using the following POST request.
$ curl -X POST -u "<key>:<secret>" https://bitbucket.org/site/oauth2/access_token -d grant_type=client_credentials
{"access_token": "{access_token}", "scopes": "pullrequest project team account", "expires_in": 7200, "refresh_token": "{refresh_token}", "token_type": "bearer"}
I'm getting a forbidden error.
Is something wrong with the API request?

You don't need to make a POST request to get the list of pull requests.
Just do a GET request with all the filters you want.

You should have read access granted for the user, using whose token you want to retrieve the list of pull requests. Generally 403-forbidden comes when you have no permissions. Once you have at least read access, you can go ahead a get the list using the way suggested by #rajatgoyal715. Also the authorization header value should be Bearer <token>. And add one more header Accept and set it to application/json. Hope this works.

Django REST authentication from desktop app

I have been trying to use the Django-REST authentication to validate the user name /password given in a desktop app.
On the server side, I have installed the following DJANGO-REST-FRAMEWORK-JWT package found here:
https://github.com/GetBlimp/django-rest-framework-jwt
I have gone through the example and when I run the following on the command line get a token as a response:
curl -X POST -d "username=luca&password=letmein123" http://localhost:8000/api-token-auth/
And I get:
{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6InBhbmthaiIsInVzZXJfaWQiOjIsImVtYWlsIjoiIiwiZXhwIjoxNDc5MTE5NzQ2fQ.RA085m-YSnGFheykCCxSVxI_9rW9AC9kEaOkUB5Gm0A"}
I tried something like:
import requests
resp = requests.post('http://127.0.0.1:8000/api-token-auth/', data={}, auth=('luca', 'letmein123'))
However, this always returns response code 400 with Bad request
My question is how can I do that from my desktop python app. I basically want to call the same API with the username and passord and be able to process the response and access protected APIs.

The auth parameter of requests.request is by default meant for Basic/Digest/Custom HTTP Auth, analogous to the -u, --user <user:password> parameter of curl.
You could define your own custom Authentication class to achieve the desired result, but the basic way to achieve the same result as your original curl request is:
resp = requests.post(
'http://localhost:8000/api-token-auth/',
data={"username": "luca", "password": "letmein123"})
The data dictionary can alternatively be supplied as json by using the json parameter if you prefer (the request would be different, but also supported by Django REST framework JWT).
You can then use the token (extracted with token = resp.json()['token']) to access the restricted urls as following:
requests.post(
'http://localhost:8000/some-personal-function/',
json={"foo": 42},
headers={'Authorization': 'JWT ' + token})
By the way looking at response.text might help finding the reason, which would, in the case of your 400 response, contain the following:
'{"username":["This field is required."],"password":["This field is required."]}'

Invalid response (404) with Twitter API in Python (and in general)

tl;dr:
I am trying to set headers in a Python HTTP request for the first time and am getting a 404. I would appreciate any help (see bottom).
I have been experimenting with the Twitter API and have not been having much luck. Eventually I am trying to get all of the media (photos) a user has posted (20 or 50 or whatever per fetch)
In my experience with other APIs, this process would go as follows: Get The userID, Make a get request to some endpoint using that userId, get a JSON feed response.
It seems to be much more complicated in Twitter.
For instance, I do not see any URLs where I can attach an access token or client ID. Instead, in their documentation they show a place where I can retrieve my
Comsumer Key, Consumer Secret, Access Token, and, Access Token Secret
If I enter my request URI and "query", it generates an oAuth Signature, which in this case consists of a
Signature base string, Authorization header and cURL command
This is where things get confusing. It says
Important: This will only be valid for a few minutes. Also remember the cURL command will actually execute the request.
So:
Question 1: right away I am wondering how I can use these credentials to retrieve media over an entire day or a weekend if they become invalid only a few minutes later?
Question 2: Using their "exploring API" console, I can test this query (where I am trying to get the user ID for the Ford" twitter account I use GET https://api.twitter.com/1.1/users/lookup.json?screen_name=hyundai
Typing that into the browser alone returns a 404
{"errors": [{"message": "Bad Authentication data","code": 215}]}
But using their little console APP I can pick "O Auth 1 authentication" (using a twitter app I made) and I get the JSON response I want. Examining the request object in the console shows:
GET /1.1/users/lookup.json?screen_name=hyundai HTTP/1.1
Authorization:
OAuth oauth_consumer_key="555SECRET555",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1421370510",oauth_nonce="1869828628",oauth_version="1.0",oauth_token="333DONTHACKMEPLEASE333",oauth_signature="444SECRET444"
Host:
api.twitter.com
X-Target-URI:
https://api.twitter.com
Connection:
Keep-Alive
tl;dr Version:
So, I thought this was the headers object I would need to send from Python (3) to make the identical request. So here is that code:
import urllib.request
header = {
"Authorization" : "OAuth",
"oauth_consumer_key" :"555SECRET555",
"oauth_signature_method": "HMAC-SHA1",
"oauth_timestamp" : "1421362844",
"oauth_nonce":"1201915763",
"oauth_version": "1.0",
"oauth_token":"333CHANGINGTHIS33333",
"oauth_signature":"222CHANGEDTHIS222",
"Host": "api.twitter.com",
"X-Target-URI": "https://api.twitter.com",
"Connection": "Keep-Alive"
}
endpoint = 'https://api.twitter.com/1.1/users/lookup.json?screen_name=hyundai'
q = urllib.request.Request(endpoint, headers=header)
a = urllib.request.urlopen(q)
print(a.read().decode('utf-8'))
But I get a bad, 404 response.
Have I formatted my headers wrong here or is there another way to do this?

If you capture the network traffic from your request (use http not https), you will see that the headers you send are not the same as the header that are expected. This is why you are getting a 404 response.
What you want is something like
header = {
"Authorization": 'OAuth oauth_consumer_key="555SECRET555",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1421362844",oauth_nonce="1201915763",oauth_version="1.0",oauth_token="333CHANGINGTHIS33333",oauth_signature="222CHANGEDTHIS222"',
"Host": "api.twitter.com",
"X-Target-URI": "https://api.twitter.com",
"Connection": "Keep-Alive"
}
Of course, you could always use an OAuth library, such as RAuth or similar, see
Python: OAuth Library for a discussion

Should a webserver ignore the request body when request method is not POST or PUT?

I was working on a simple API server using tornado and all requests require the parameter access_token. I was playing with curl, and was surprised to find that DELETE and GET requests will not extract this value from the request body--they only allow this param to be passed via the query string.
ie, when I do
curl -i -X DELETE -d access_token=1234 http://localhost:8888/
In the delete method of my web handler, this returns None:
self.get_argument('access_token', None)
However, when I do
curl -i -X DELETE http://localhost:8888/?access_token=1234
This yields "1234" as expected:
self.get_argument('access_token', None)
I examined the tornado source, and found that the body is only parsed for POST and PUT requests: https://github.com/facebook/tornado/blob/4b346bdde80c1e677ca0e235e04654f8d64b365c/tornado/httpserver.py#L258
Is it correct to ignore the request body for GET, HEAD, and DELETE requests, or is this a choice made by the authors of tornado?

This is correct per the HTTP/1.1 protocol specification.
DELETE and GET requests do not accept entity data enclosed in the request.
According to the definition, get requests retrieve their entity data from the request URI.
HEAD requests are defined as identical to GET requests except that the server should not return a message body in the response.
Therefore the authors of tornado were correct to ignore the "post" data for GET, HEAD, and DELETE.
See HTTP/1.1 Method Definitions

It is a good idea to not to accept requests with the payload if they are not POST or PUT. Just because of security reasons. Some servers, e.g. lighttpd, return server error in this case.