We Keep Coding

Python OAuth Client Credentials Grant Type Returned 400, [error: unsupported grant] type but Curl Works Fine

I am very new to APIs (still learning) and I encountered a very weird issue with Python requests library when trying to initiate an OAuth Authentication flow with Client Credentials Grant Type.
For some reason, whenever I used my Python script (with the help of requests library) to send the HTTP request to the authentication endpoint, I always get
Response Status Code: 400
Response Body/Data returned: {"error":"unsupported_grant_type"}
However, if I tried using curl command line tool to send the request, I will get a successful response with status code 200 with the access token in the response body like this:
{'access_token': 'some access token',
'expires_in': 'num_of_seconds',
'token_type': 'Bearer'}
As a matter of fact, if I tried sending the request using Curl command line tool WITHIN my Python Script (with subprocess.Popen function), I can get the response with status code 200 and the access token with no problem.
Now, with that said, here's the Python script that I used to send the request to initiate the OAuth authentication flow:
import requests
import os
import base64
clientCredentialEndpoint = "https://base_url/path/token"
client_id = os.environ.get('CLIENT_ID')
client_secret = os.environ.get('CLIENT_SECRET')
# -- Encode the <client_id:client_secret> string to base64 --
auth_value = f'{client_id}:{client_secret}'
auth_value_bytes = auth_value.encode('ascii')
auth_value_b64 = base64.b64encode(auth_value_bytes).decode('ascii')
queryParams ={
'grant_type':'client_credentials',
'scope':'get_listings_data'
}
headers = {
'Authorization':f'Basic {auth_value_b64}',
'Content-Type':'application/x-www-form-urlencoded'
}
# send the post request to Authorisation server
response = requests.post(
clientCredentialEndpoint,
params=queryParams,
headers=headers,
)
print(response.status_code)
print(response.text)
whereas the curl command that I used (and worked) to send the request is:
curl -X POST -u '<client_id>:<client_secret>' \
-H "Content-Type: application/x-www-form-urlencoded" \
-d 'grant_type=client_credentials&scope=get_listings_data' \
'https://base_url/path/token'
Again, like I said, if I execute this curl command inside a Python script, it will successfully return the access token with no issue.
Does anyone know what I did wrong in my Python script which caused my request to always fail?
Thanks in advance!

My goodness me, I just realised that the -d in the curl command does not correspond to query params, it stands for 'data'.
Hence, I just need to change my Python script requests.post() a bit so that it looks like this:
response = requests.post(
clientCredentialEndpoint,
data=queryParams,
headers=headers,
)
Hope this helps others.

translating simple curl call to python/django request

I'm attempting to translate the following curl request to something that will run in django.
curl -X POST https://api.lemlist.com/api/hooks --data '{"targetUrl":"https://example.com/lemlist-hook"}' --header "Content-Type: application/json" --user ":1234567980abcedf"
I've run this in git bash and it returns the expected response.
What I have in my django project is the following:
apikey = '1234567980abcedf'
hookurl = 'https://example.com/lemlist-hook'
data = '{"targetUrl":hookurl}'
headers = {'Content-Type': 'application/json'}
response = requests.post(f'https://api.lemlist.com/api/hooks/', data=data, headers=headers, auth=('', apikey))
Running this python code returns this as a json response
{}
Any thoughts on where there might be a problem in my code?
Thanks!

Adding to what nikoola said, I think you want that whole data line to be as follows so you aren't passing that whole thing as a string. Requests will handle serializing and converting python objects to json for you [EDIT: if you use the json argument instead of data].
source: https://requests.readthedocs.io/en/master/user/quickstart/#more-complicated-post-requests
Instead of encoding the dict yourself, you can also pass it directly
using the json parameter (added in version 2.4.2) and it will be
encoded automatically:
Note, the json parameter is ignored if either data or files is passed.
Using the json parameter in the request will change the Content-Type
in the header to application/json.
data = {"targetUrl":hookurl}

import requests
headers = {
'Content-Type': 'application/json',
}
data = '{"targetUrl":"https://example.com/lemlist-hook"}'
response = requests.post('https://api.lemlist.com/api/hooks', headers=headers, data=data, auth=('', '1234567980abcedf'))
You can visit this url:-
https://curl.trillworks.com/

Replacing curl with python requests equivalent to POST a file to a server

After writing a file with the snippet below
with open("temp.trig", "wb") as f:
f.write(data)
I use curl to load it into the server
curl -X POST -F file=#"temp.trig" -H "Accept: application/json" http://localhost:8081/demo/upload
which works fine.
I am trying to replace the curl with python requests, as follows:
with open("temp.trig", "rb") as f:
result = requests.post("http://localhost:8081/demo/upload", files={'file': f},
headers = {"Accept": "application/json"})
which attempted to follow the curl as closely as possible. This code results in an error 500 from the server. I suspect it must be something related to the request, because the same server is ok via `curl. Any ideas?

There probably is nothing wrong with your python script.
Differences I've noticed between curl and requests are the following:
obviously, User-Agent headers are different — curl/7.47.0 vs. python-requests/2.22.0
multipart boundary format in Content-Type header is different — ------------------------6debaa3504bbc177 in curl vs. c1e9f4f617de4d0dbdb48fcc5aab67e0 in requests
therefore Content-Length value will almost certainly be different
multipart/form-data format in body is slightly different — curl adds an extra line (Content-Type: text/plain) before file contents
So depending on your file format, server may not be able to parse requests HTTP request format.
I think the best solution for you now is to compare raw HTTP requests from curl and requests and find what differences are significant.
For example:
Open terminal
Launch netcat with nc -l -p 1234 command. This will listen to HTTP requests on localhost on port 1234 and output raw HTTP requests to terminal.
Send your curl request as it is to localhost:1234 in another tab
Execute your python script as it is using URL localhost:1234 in another tab
Compare raw requests from your netcat output

Here's my attempt:
import requests
headers = {
'Accept': 'application/json',
}
files = {
'file': ('temp.trig', open('temp.trig', 'rb')),
}
response = requests.post('http://localhost:8081/demo/upload', headers=headers, files=files)
In case this doesn't work we really need to read more data on the server side, as Ivan Vinogradov explained well.

Unable to upload file to REST API

I have this cURL request that I want to convert into a Python Request code.
The cURL content is
curl -H "X-PrettyPrint: 1"
-F 'json={"title":"PandaTest"};type=application/json'
-F "fileData=#rename.py;type=application/octet-stream"
-X POST https://cs31.salesforce.com/services/data/v39.0/connect/files/users/me
-H 'Authorization: 00Dp000000.....CqqU0.S_5r' --insecure
For more details of the request check the SalesForce docs it contains the HTTP request message - here. Search for the section Upload a file to the Files home.
The Python counter part of it what I've written is
import requests
files = {
"fileData" : open("rename.py", "rb"),
"json" : '{"title":"PandaTest"}'
}
headers = {
'Authorization': 'OAuth 00Dp00000000u....n3ZGuoZK2wYJRCqqU0.S_5r',
"Content-Disposition": "form-data 'fileData'"
}
r = requests.post('https://cs31.salesforce.com/services/data/v39.0/connect/files/users/me/',
data=files, headers=headers)
data = json.loads(r.text)
print data
My request is sent successfully but I get an error Missing expected "fileData" binary parameter.I have a feeling the request which I'm trying to send is not formed correctly. Where exactly did I go wrong?
I have a feeling I'm not handling the 2 -F in the cURL request correctly.

This line:
"fileData" : open("rename.py", "rb"),
is setting "fileName" to the object that is returned when you open a file. If you would like "fileName" to be the actual contents of the file, then change the line to this:
"fileData" : open("rename.py", "rb").read(),
That will read all the bytes in the file and set "fileName" to them.

How to convert cURL in Python requests to change Jenkins config.xml file

I have this cURL:
curl -X POST http://user:pass#blabla.com:8080/job/myproject/config.xml --data-binary "#new_config.xml"
I am basically trying to set a new config for a Jenkins installation by changing the pre-existing config.xml file.
I am trying to convert it to something like this in order to use it more flexibly in my code:
url = "http://host:8080/job/myproject/config.xml"
auth = ('user','pass')
payload = {"--data-binary": "#new_config.xml"}
headers = {"Content-Type" : "application/xml"}
r = requests.post(url, auth=auth, data=payload, headers=headers)
I know that I am using incorrectly the payload and the headers.How should I change them?
I run it and I take a 500 responce code.
I read this post , but I am struggling to apply it in my case.

The --data-binary switch means: post the command line argument as the whole POST body, without wrapping in multipart/form-data or application/x-www-form-encoding containers. # tells curl to load the data from a filename; new_config.xml in this case.
You'll need to open the file object to send the contents as the data argument:
url = "http://host:8080/job/myproject/config.xml"
auth = ('user','pass')
headers = {"Content-Type" : "application/xml"}
with open('new_config.xml', 'rb') as payload:
r = requests.post(url, auth=auth, data=payload, headers=headers)
Note that I pass the file object directly into requests; the data will then be read and pushed to the HTTP socket, streaming the data efficiently.