We Keep Coding

PYTHON: requests.post() how to send request_body encoded as application/x-www-form-urlencoded

I'm doign an app with the Spotify API. My problem is I'm trying to get an access_token but It's not working. In the docs it says i need to send the body encoded as application/x-www-form-urlencoded so I search a little bit and It should work just setting request_body as a dictionary.
This is the code of my function:
def get_access_token(self):
auth_code, code_verifier = self.get_auth_code()
endpoint = "https://accounts.spotify.com/api/token"
# as docs said data should be encoded as application/x-www-form-urlencoded
# as internet says i just need to send it as a dictionary. However it's not working
request_body = {
"client_id": f"{self.client_ID}",
"grant_type": "authorization_code",
"code": f"{auth_code}",
"redirect_uri": f"{self.redirect_uri}",
"code_verifier": f"{code_verifier}"
}
response = requests.post(endpoint, data=request_body)
print(response)
The response I'm getting is always <Response [400]>
Here are the docs, step 4 https://developer.spotify.com/documentation/general/guides/authorization-guide/#authorization-code-flow-with-proof-key-for-code-exchange-pkce
NOTE: I tryed executing this as a curl and it works fine I'm not sure what I'm doing wrong in the python code
here's the command:
curl -d client_id={self.client_ID} -d grant_type=authorization_code -d code={auth_code} -d redirect_uri={self.redirect_uri} -d code_verifier={code_verifier} https://accounts.spotify.com/api/token

You can specify the request type in the request header.
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
response = requests.post(endpoint, data=request_body, headers=headers)
print(response)

Convert curl method into python request

i am trying to convert a curl method into a python POST request.
Here is the curl command:
curl --location --request POST 'https://global.kaleyra.com/api/v4/' --form 'file=#/home/user/Desktop/TWP_MD565041.pdf' --form 'method=wa' --form 'api_key=A3d51xxxxxxxxxxxxxxxxxxxxxxxxxx' --form 'from=971xxxxxxxxxx' --form 'body={
"to":"9198xxxxxxxx",
"type": "document", "document" : {"caption" : ""},
"callback":""
}' --form 'format=json'
And here is the python request ia tried:
payload = {
"method": "wa",
"api_key": "A3dxxxxxxxxxxxxxxxx",
"body": '{"to":'+str(user_number)+',"type": "document"}',
'from': '971xxxxxxxx',
'format':'json',
"file":open("/home/user/Desktop/TWP_MD565041.pdf","rb")}
r = requests.post(url=api_url,headers={}, files=payload)
getting error:
b'{"status":"A401B","message":"Method Not Found"}'
Then i changed request like this:
r = requests.post(url=api_url,headers={}, files=json.dumps(payload))
Now am getting an error:
TypeError: Object of type 'BufferedReader' is not JSON serializable
And i tried this,
with open("/home/user/Desktop/TWP_MD565041.pdf",'rb') as f:
r = session.post(url=api_url,headers=headers, data=json.dumps(payload),files={"file":f})
Then i an getting an error:
ValueError: Data must not be a string.
How can i resolve this?

1) Requests will serialize the JSON for you. Drop the json.dumps method and use the raw payload.
2) I suggest sending the file as such. The with statement is used for larger atomic manipulations, and you really don't need to keep the file open while performing API I/O. Therefore, use the following method, as the file stream is implicitly closed after interpretation.
r = session.post(
url = api_url,
headers = headers,
data = payload,
files = {'file': open('file.pdf','rb')}
)

REST post query to GO server from bash works but fails for Python

I have go server that unmarshals the json it receives it.
It works when I do it using the curl but fails in case of python.
Go server Unmarshal code:
type Data struct {
Namespace string `json:"namespace"`
ContainerId string `json:"containerId"`
}
func notify(w http.ResponseWriter, r *http.Request) {
decoder := json.NewDecoder(r.Body)
var data Data
err := decoder.Decode(&data)
if err != nil {
glog.Errorf("Failed to decode the request json %s \n", err.Error())
return
}
...
}
If I Do curl command it works without complaining:
curl -i -H "Accept: application/json" -H "Content-Type:application/json" -X POST --data '{"namespace": "default", "containerId": "2f7c58d399f2dc35fa1be2abea19301c8e74973ddd72f55a778babf01db5ac26"}' http://mysvc:8080/notify
but if I do the same thing with Python it complains:
jsonPrep['containerId'] = "2f7c58d399f2dc35fa1be2abea19301c8e74973ddd72f55a778babf01db5ac26"
jsonPrep['namespace'] = "default"
headers = {'Content-type': 'application/json', 'Accept': 'application/json'}
r = requests.post('http://mysvc:8080/notify', json=json.dumps(jsonPrep), headers=headers)
the go server complains :
E1026 15:49:48.974117 1 main.go:59] Failed to decode the request json json: cannot unmarshal string into Go value of type main.Data
I don't see what is different when I do curl vs rest query in python.
Can anyone help me identify the issue?

The json argument to requests.post() is for passing a value that has not had json.dumps() called on it yet. requests calls json.dumps() on the json argument itself, so because you're passing json=json.dumps(jsonPrep), jsonPrep will end up being JSONified twice, which is not what you want.
Either use data:
requests.post(..., data=json.dumps(jsonPrep), ...)
or get rid of the json.dumps():
requests.post(..., json=jsonPrep, ...)

python google geolocation api using wifi mac

I'm trying to use Google's API for geolocation giving wifi data to determine location. This is their intro. And this is my code
#author: Keith
"""
import requests
payload = {
"considerIp": "false",
"wifiAccessPoints": [
{
"macAddress": "00:25:9c:cf:1c:ac",
"signalStrength": -43,
"signalToNoiseRatio": 0
},
{
"macAddress": "00:25:9c:cf:1c:ad",
"signalStrength": -55,
"signalToNoiseRatio": 0
}
],
'key':'<MyAPIKey>'
}
r = requests.post('https://www.googleapis.com/geolocation/v1/geolocate',
params=payload)
print(r.text)
This is the output
{
"location": {
"lat": 32.3643098,
"lng": -88.703656
},
"accuracy": 6061.0
}
The request ignored all of the payload except the key portion and just found the geolocation using my IP address. So I'm sending the json payload incorrectly. I know this is probably really simple, but I'm stuck and couldn't find an example of python being used with requests to do this type of API query. Thanks
Edit:
Picked up the cURL library and executed this command with success:
curl -d #your_filename.json -H "Content-Type: application/json" -i "https://www.googleapis.com/geolocation/v1/geolocate?key=<myapikey>"
and got the output I expected. I just want to be able to do the same thing in requests, but the data I'm trying to send is in "your_filename.json".

Please try the following:
r = requests.post('https://www.googleapis.com/geolocation/v1/geolocate?key='+ '<MyAPIKey>', json=payload)
Note the key was moved to query params (URL) and json argument was used in place of params.

Okay I figured out the error of my ways. Taking a better look at requests.post function, I see that I'm not using the parameters argument correctly. After this, it worked perfectly,
r = requests.post('https://www.googleapis.com/geolocation/v1/geolocate', params=parameters, json=mydata, headers=headers)

Translate this request from Bash to Python?

I was given a request in Bash and I have to translate it to Python 2.7. I did this kind of translations several times, but now I am not able to make it work and I do not understand why.
First of all, I was given this Bash request:
curl -X POST -v -u user#domain:password --data "#file.json" -H "Content-Type:application/json" http://destination_url_a
With the file file.json, whose content is the following one:
{
"username":"user#domain",
"password":"password",
"shortName":"a-short-name",
"visibility":"PRIVATE",
"sitePreset":"site-dashboard",
"title":"A Title",
"description":"A description."
}
If I execute the Bash line in my computer, the result is succesful.
As always, I tried to use requests library in Python to make it work. What I did is:
import requests
from requests.auth import HTTPBasicAuth
import json
data = {
"username": "user#domain",
"password": "password",
"shortName": "a-short-name",
"visibility": "PRIVATE",
"sitePreset": "site-dashboard",
"title": "A Title",
"description": "A description.",
}
headers = {'Content-Type': 'application/json'}
data_json = json.dumps(data)
r = requests.post(
url='http://destination_url_a',
data=data_json,
headers=headers,
auth=HTTPBasicAuth('user#domain', 'password'),
verify=False,
)
Unfortunately, the response, stored in r variable, is an error, despite the status code is 200.
What could be happening? Does anyone find a problem in my code or has any idea?
EDIT
However, this is another example very similar which worked perfectly:
Bash:
curl -v -H "Content-Type:application/json" -X POST --data "#file.json" -u user#domain:password http://destination_url_b
My Python code
import requests
from requests.auth import HTTPBasicAuth
import json
data = {
"userName": "user#domain",
"password": "password",
"firstName": "Firstname",
"lastName": "Lastname",
"email": "email#domain.com",
"disableAccount": "False",
"quota": -1,
"groups": ["a_group",],
}
headers = {'Content-Type': 'application/json'}
data_json = json.dumps(data)
r = requests.post(
url='http://destination_url_b',
data=data_json,
headers=headers,
auth=HTTPBasicAuth('user#domain', 'password'),
verify=False,
)
It seems to be almost the same to the other request, but this works. Different data is sent, and to a different subdomain (both are sent to the same domain). Will these modifications be important if we are talking about the User-Agent you mentioned?

Sometimes, webservices filter on user-agent. User agent of curl and python are different. That may explain.
Try to forge a "legitimate" user-agent by modifying the request header.

Finally, there was no error in the code neither in the User-Agent.
The problem was that the destination application did not accept sites with the same short-name value.
What I was doing is creating the site from Bash, which worked, then removing it from the interface of the app and trying to create it from Python with the same data. I was getting error when doing that because in spite of having removed the site, I had to remove some residual data of it from the trashcan of the app too. If not, app returned an error since it considered that the site I was trying to create already existed.
So if I had introduced different short-name in each attempt, I would not have had any error.