We Keep Coding

Python requests PUT request with json parameter fails and data parameter succeeds

Problem
I've looked at some of the documentation about the json and data parameters and the differences between them. I think I understand the difference, best explained here, in my opinion.
However, I have a specific request that fails on PUT using json, but fails using data, and I'm not sure why. Can someone clarify why this is the case? Could it be that there is a list in the payload?
Context
I have requests==2.28.0 installed. Below is the code that submits the PUT requests to an API for PagerDuty, the incident management software, one using data (successful) and one using json (failing). Otherwise they are identical.
The weird thing is that their examples use the json parameter.
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', data=payload, headers=headers)
print("response: ", response)
Result: response: <Response [200]>
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', json=payload, headers=headers)
print("response: ", response)
Result: response: <Response [400]>

Your payload is a string while json parameter takes a dictionary. That's the whole point of the json argument (you don't have to encode it yourself):
If you need that header set and you don’t want to encode the dict yourself, you can also pass it directly using the json parameter (added in version 2.4.2) and it will be encoded automatically:
You should pass a dictionary if you want to use the json parameter:
payload = {
"source_incidents": [
{
"id": child_incident_id,
"type": "incident_reference"
}
]
}
which is more readable anyway.

Alternatively you could use json.loads to parse your string:
import json
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', data=json.loads(payload), headers=headers)
print("response: ", response)

That is what the requests library does with json data.
Converts your Python object to JSON using the json encoder.
Sets the "content-type" header to "application/json".
It is possible to implement it as shown here.
def put(uri, data=None, json=None):
if json and data:
raise Exception()
if json:
payload = json.dumps(json)
else:
payload = data
...
So the first request returns 200 because you passed valid JSON OBJECT through the "data" parameter.
And the second request fails because you passed STRING through the JSON that will be dumped with "json.dumps(obj)" for example.
As a result, it would be nothing more than a string, which is also a valid JSON object but not a javascript object.
As shown here, if you pass a string through "json.dumps" and a dictionary, it returns two different values: a string and an object.
>>> json.dumps("{}")
'"{}"'
>>> json.dumps({})
'{}'
>>>

How to convert a python post request to a Postman POST request

I have a python post request to a server where my flask app is hosted. It works fine and I am able to get the desired data.
But I want to test the API using POSTMAN. I am unable to do that because I am unfamiliar with POSTMAN to some extent.
Below is the python code that I have.
import requests
import ast
import json
resp1 = {}
url = 'http://myflaskapiurl:port/route'
files = {'file': open(r'file that should be uploaded to the server', 'rb')}
r = requests.post(url, files=files, data={"flag":2})
headers = {
'content-type': "multipart/form-data",
'Content-Type': "application/x-www-form-urlencoded",
'cache-control': "no-cache",
}
resp1 = ast.literal_eval(r.text)
print(resp1)
I am struggling with the question whether the data and file that I am trying to post to the server should be in raw json or form-data or x-www-form-urlencoded section of body. Also what should be the actual structure.
Because every time I POST this data using form-data or x-www-form-urlencoded section of body I get the error saying
werkzeug.exceptions.BadRequestKeyError
werkzeug.exceptions.HTTPException.wrap..newcls: 400 Bad Request: KeyError: 'file'

This is how it should look like:
The "params" tab should be empty, maybe you're adding a second file parameter there?

Tomtom webfleet order api gives 40 general error

I am trying to insert a destination order using action=insertDestinationOrder
I am using POST method with all required parameters, but keep getting
{
"errorCode": 40,
"errorMsg": "general error"
}
I have checked using postman too. But still same.
Below is the request using python requests package.
import requests
url = "https://csv.telematics.tomtom.com/extern"
payload = "orderid=TO0049&country=DE&city=Cologne&latitude=50974519&ordertype=delivery%20order&zip=50735&longitude=6977319&street=Am%20Niehler%20Hafen%20%26%20Stapelkai%2C%2050735%20Cologne%20(Niehl)&account=XXXX&username=XXXX&password=XXXX&apikey=XXXX&lang=en&action=insertDestinationOrder&ordertext=Am%20Niehler%20Hafen%20%26%20Stapelkai%2C%2050735%20Cologne%20(Niehl)&useUTF8=true&outputformat=json"
headers = {
'content-type': "application/x-www-form-urlencoded"
}
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
Need some help.

Try adding a ? to the end of the url, or to the start of the payload:
url = "https://csv.telematics.tomtom.com/extern?"
or
payload = "?orderid=TO0049&country=DE&city=Cologne&latitude=50974519&ordertype=delivery%20order&zip=50735&longitude=6977319&street=Am%20Niehler%20Hafen%20%26%20Stapelkai%2C%2050735%20Cologne%20(Niehl)&account=XXXX&username=XXXX&password=XXXX&apikey=XXXX&lang=en&action=insertDestinationOrder&ordertext=Am%20Niehler%20Hafen%20%26%20Stapelkai%2C%2050735%20Cologne%20(Niehl)&useUTF8=true&outputformat=json"

Python Post request with REQUESTS/ Shiftplanning API

I am trying to call the shiftplanning API with the following code:
import requests
url= "https://humanity.com/api/"
payload = {"key": "keyvalue", "request": { "module": "staff.login", "method": "GET","username": "myusername", "password": "mypassword"}}"
headers = {
'content-type': "application/x-www-form-urlencoded",
'cache-control': "no-cache",
}
r = requests.post(url, data = payload
, headers= headers)
r.text
I used Postman to test the api beforehand and it worked fine. The payload in the Postman code creator, however, looks like this:
payload = "data={\r\n \"key\": \"keyvalue\",\r\n \"request\": {\r\n \"module\": \"staff.login\",\r\n \"method\": \"GET\",\r\n \"username\": \"myusername\",\r\n \"password\": \mypassword\"\r\n }\r\n}"
I assume, that the payload variable actually needs to be formatted as a string, as the doc at shiftplanning api says:
Remember that any data passed to the API is to be formatted as JSON, and its string value needs to be sent via the POST HTTP request method all as the post variable 'data' (Checkout the 9th line from an example below). Also Content-Type needs to be set to application/x-www-form-urlencoded.
However, no matter what I do, I always get the html of the api as a return, not the json response to my http request.
Using
r.json()
doesn't do anything, except raising an error, due to the returned html.
I also tried using
json = payload
in the post request, that didn't change anything either.`
Any help and explanation is appreciated. I'd really like to understand what I am doing wrong, as it seems to be pretty straightforward to execute.

Change content-type in headers to application/json if you want json response. Postman automatically sets that header for you but you need to set it in your Python code.

Converting cURL to Python Requests error

I'm trying to convert the cURL to Python request but doesn't work.
cURL: curl -kv -H 'Content-Type: application/json' 'https://IP-address/api/v1/login' -d '{"username":"api", "password":"APIPassword"}'
My Python requests code:
import requests
url = "https://IP-address/api/v1/login"
payload = "'{\"username\":\"api\", \"password\":\"APIPassword\"}'"
headers = {
'Content-Type': "application/json",
'cache-control': "no-cache",
}
response = requests.request("GET", url, headers=headers, data=payload, verify=False)
print(response.text)
Which doesn't work and gives me 400 bad requests error.
I tried converting using the https://curl.trillworks.com/
which gives me the following code which doesn't work either.
import requests
url = 'https://IP-address/api/v1/login'
headers = {
'Content-Type': 'application/json',
}
data = '{"username":"api", "password":"APIPassword"}'
output = requests.get(url, data=data, verify=False)
print (output)
Can anyone please help me identify the issue here.
Edit: I have edited 2nd script to produce output: Which gives 500 Error

Use the json parameter in requests.post for json data. It also takes care of the headers.
data = {"username":"api", "password":"APIPassword"}
response = requests.post(url, json=data, verify=False)

Another way to make sure you're sending valid JSON in your payload would be to use the json python library to format your payload via json.dumps(), which returns a string representing a json object from an object. This was especially useful to me when I needed to send a nested json object in my payload.
import json
import requests
url = 'https://sample-url.com'
headers = { 'Content-Type': 'application/json', 'Authorization': f'{auth_key}'}
payload = { "key": "value",
"key": ["v1", "v2"],
"key": {
"k": "v"
}
...
}
r = requests.post(url, headers=headers, data=json.dumps(payload))