We Keep Coding

Python requests PUT request with json parameter fails and data parameter succeeds

Problem
I've looked at some of the documentation about the json and data parameters and the differences between them. I think I understand the difference, best explained here, in my opinion.
However, I have a specific request that fails on PUT using json, but fails using data, and I'm not sure why. Can someone clarify why this is the case? Could it be that there is a list in the payload?
Context
I have requests==2.28.0 installed. Below is the code that submits the PUT requests to an API for PagerDuty, the incident management software, one using data (successful) and one using json (failing). Otherwise they are identical.
The weird thing is that their examples use the json parameter.
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', data=payload, headers=headers)
print("response: ", response)
Result: response: <Response [200]>
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', json=payload, headers=headers)
print("response: ", response)
Result: response: <Response [400]>

Your payload is a string while json parameter takes a dictionary. That's the whole point of the json argument (you don't have to encode it yourself):
If you need that header set and you don’t want to encode the dict yourself, you can also pass it directly using the json parameter (added in version 2.4.2) and it will be encoded automatically:
You should pass a dictionary if you want to use the json parameter:
payload = {
"source_incidents": [
{
"id": child_incident_id,
"type": "incident_reference"
}
]
}
which is more readable anyway.

Alternatively you could use json.loads to parse your string:
import json
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', data=json.loads(payload), headers=headers)
print("response: ", response)

That is what the requests library does with json data.
Converts your Python object to JSON using the json encoder.
Sets the "content-type" header to "application/json".
It is possible to implement it as shown here.
def put(uri, data=None, json=None):
if json and data:
raise Exception()
if json:
payload = json.dumps(json)
else:
payload = data
...
So the first request returns 200 because you passed valid JSON OBJECT through the "data" parameter.
And the second request fails because you passed STRING through the JSON that will be dumped with "json.dumps(obj)" for example.
As a result, it would be nothing more than a string, which is also a valid JSON object but not a javascript object.
As shown here, if you pass a string through "json.dumps" and a dictionary, it returns two different values: a string and an object.
>>> json.dumps("{}")
'"{}"'
>>> json.dumps({})
'{}'
>>>

Python post request throwing 400 'Bad Request' error with requests library but works with cURL

I have a script that calls a POST endpoint but getting a 400 error. Meanwhile, the corresponding cURL request is successful.
First, here is the cURL:
curl -X 'POST' \
'http://localhost:8080/api/predict?Key=123testkey' \
-H 'accept: application/json' \
-H 'Content-Type: multipart/form-data' \
-F 'file=#156ac81cde4b3f22faa4055b53867f38.jpg;type=image/jpeg'
And translated to requests:
import requests
url = 'http://localhost:8080/api/predict?Key=123testkey'
headers = {
'accept': 'application/json',
'Content-Type': 'multipart/form-data',
}
params = {'Key' : '123testkey'}
files = {'image': open('156ac81cde4b3f22faa4055b53867f38.jpg', 'rb')}
response = requests.post(url, files=files, params=params, headers=headers)
Have also tried using a URL that does not include the key, since the key is already specified in params:
import requests
url = 'http://localhost:8080/api/predict'
headers = {
'accept': 'application/json',
'Content-Type': 'multipart/form-data',
}
params = {'Key' : '123testkey'}
files = {'image': open('156ac81cde4b3f22faa4055b53867f38.jpg', 'rb')}
response = requests.post(url, files=files, params=params, headers=headers)
I thought this should be simple but I consistently get the 400 error with requests no matter what I try. Any suggestions?
Edit: have also tried 'image/jpeg' instead of 'image' to no avail.
Edit: replacing the "image" key with "file" unfortunately didn't work either
Edit: It works in postman desktop just fine, and generates the following code. However, this code also throws an error.
The generated code from postman:
import requests
url = "http://localhost:8080/api/predict?Key=123test"
payload={}
files=[
('file',('images19.jpg',open('156ac81cde4b3f22faa4055b53867f38.jpg','rb'),'image/jpeg'))
]
headers = {
'Accept': 'application/json',
'Content-Type': 'multipart/form-data'
}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.text)
And the error from the previously generated code from postman:
{"detail":"There was an error parsing the body"}
Any help figuring out what is going on would be much appreciated!

Your issue is in the variable files you need to add with the key 'file' instead of 'image' that's the difference between your curl and your python code, also remove the header because when you pass the file parameter the request set the proper header for send files. for example:
import requests
url = 'http://localhost:8080/api/predict?Key=123testkey'
params = {'Key' : '123testkey'}
files = {'file': open('156ac81cde4b3f22faa4055b53867f38.jpg', 'rb')}
response = requests.post(url, files=files, params=params)

Python API Error - 'faultstring': 'JSONThreatProtection

First time trying to extract data via an API. I'm getting stuck with how to pass the raw data. I have been able to get it to work in postman but I can't quite figure it out with Python.
I've been getting this error:
{'fault': {'faultstring': 'JSONThreatProtection[JSON-Threat-Protection-1]: Execution failed. reason: Expecting { or [ at line 1', 'detail': {'errorcode': 'steps.jsonthreatprotection.ExecutionFailed'}}}
I'm pretty sure its the data portion that is wrong but I haven't been able to figure out if it's just a syntax error or something bigger that I'm missing.
My code as follows:
import requests
import json
url = "https://url/customers/shipmentstatus"
headers = {
"Authorization": "Bearer TOKEN",
"Content-Type": "application/json"
}
data = {
"Id": [
"AZ1234"
]
}
response = requests.post(url, headers=headers, data=data)
print(response.json())

Your code posts the data as form encoded data, not as JSON. That's because you use the data parameter to supply the payload. To fix, encode the data as JSON using json.dumps():
response = requests.post(url, headers=headers, data=json.dumps(data))
But a better way is to pass the data to requests using the json parameter:
response = requests.post(url, headers=headers, json=data)
Using the json parameter is easier and it will also add the Content-Type: application/json header for you.

Python requests Json body includes some thai values, needs to be encoded as it is

I have some json containing some thai values. It looks something like
{
"TitleName": "คุณ",
"FirstName": "Simar"
}
I need to make a Http POST request with this json body with the exact thai value.I am using Python 3 requests library to make the request.
I tried this
headers = {
'Content-Type': "application/json",
'Authorization': "xxx",
'cache-control': "no-cache",
'Postman-Token': "xxx"
}
response = requests.request("POST", url, json=request, headers=headers)
It generates json values as
"TitleName": "\\u0e04\\u0e38\\u0e13",
"FirstName": "Simar"
I also tried this
json_request = json.dumps(self.request_data,ensure_ascii=False).encode('utf8')
response = requests.request("POST", url, json=json_request, headers=headers)
It generates json values as
"TitleName": "\xe0\xb8\x84\xe0\xb8\xb8\xe0\xb8\x93",
"FirstName": "Simar"
But I want json values to be generated as
"TitleName": "คุณ",
"FirstName": "Simar"
Help will be appreciated. Thanks in advance.

To preserve non-ascii characters in POST requests you need to serialise to json manually, and explicitly set the content-type header.
data = json.dumps(my_dict, ensure_ascii=False)
r = requests.post(url, headers={'content-type': 'application/json'},
data=data.encode('utf-8'))

Calling Curl API on Python

I want to call CURL API on python.
curl -X POST -H "Authorization:Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5" \
-H "Content-Type: application/json" \
-d '{"module_id":"[MODULE_ID]", "text": "example text"}' \
-D - \
https://api.tocall.com/
I used requests module for making request and json module for converting object to string. But I'm getting 404.
Where am I wrong?
import requests
import json
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
'Content-Type': 'application/json',
}
url = "https://api.tocall.com/"
data = '{"module_id":"[MODULE_ID]", "text": "example text"}'
response= requests.post(url, data=json.dumps(data), headers=headers)
print(response.status_code)

You are encoding your data as JSON twice. json.dumps() takes an object and converts to JSON. In this case, you are converting a string to JSON. This should work better:
import requests
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
}
url = "https://api.tocall.com/"
data = {"module_id":"[MODULE_ID]", "text": "example text"}
response= requests.post(url, json=data, headers=headers)
print(response.status_code)
If it still doesn't work and you need more help, you should include real details about your API so we can reproduce the issue.

json.dumps turns a Python dict to a string, but your data is already a string. The easiest thing to do is write data as a dict then use json.dumps on that.

Add the Host header, so that the final server knows on which virtual host to route the request,
Change:
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
'Content-Type': 'application/json',
}
For:
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
'Content-Type': 'application/json',
'Host' : 'api.tocall.com'
}
I think this will fix your issue. Eventually you might want to update the default headers, not craft your own ones. Try to use the session features of requests to perform consistent queries.
Note: as stated by other answers, you have other JSON encoding issues, but that's not the reason why you are getting 404.