Problem
I've looked at some of the documentation about the json and data parameters and the differences between them. I think I understand the difference, best explained here, in my opinion.
However, I have a specific request that fails on PUT using json, but fails using data, and I'm not sure why. Can someone clarify why this is the case? Could it be that there is a list in the payload?
Context
I have requests==2.28.0 installed. Below is the code that submits the PUT requests to an API for PagerDuty, the incident management software, one using data (successful) and one using json (failing). Otherwise they are identical.
The weird thing is that their examples use the json parameter.
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', data=payload, headers=headers)
print("response: ", response)
Result: response: <Response [200]>
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', json=payload, headers=headers)
print("response: ", response)
Result: response: <Response [400]>
Your payload is a string while json parameter takes a dictionary. That's the whole point of the json argument (you don't have to encode it yourself):
If you need that header set and you don’t want to encode the dict yourself, you can also pass it directly using the json parameter (added in version 2.4.2) and it will be encoded automatically:
You should pass a dictionary if you want to use the json parameter:
payload = {
"source_incidents": [
{
"id": child_incident_id,
"type": "incident_reference"
}
]
}
which is more readable anyway.
Alternatively you could use json.loads to parse your string:
import json
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', data=json.loads(payload), headers=headers)
print("response: ", response)
That is what the requests library does with json data.
Converts your Python object to JSON using the json encoder.
Sets the "content-type" header to "application/json".
It is possible to implement it as shown here.
def put(uri, data=None, json=None):
if json and data:
raise Exception()
if json:
payload = json.dumps(json)
else:
payload = data
...
So the first request returns 200 because you passed valid JSON OBJECT through the "data" parameter.
And the second request fails because you passed STRING through the JSON that will be dumped with "json.dumps(obj)" for example.
As a result, it would be nothing more than a string, which is also a valid JSON object but not a javascript object.
As shown here, if you pass a string through "json.dumps" and a dictionary, it returns two different values: a string and an object.
>>> json.dumps("{}")
'"{}"'
>>> json.dumps({})
'{}'
>>>
Related
This is my first project using API/python. Basically, I want to get information from trackhive.com to see where is the package. But first, to understand how it works, I'm just trying to create a track.
On the website, they give me this example:
from urllib2 import Request, urlopen
values = """
{
"tracking_number": "9361289676090919095393",
"slug": "usps"
}
"""
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer Token'
}
request = Request('https://private-anon-6175dd5596-trackhive.apiary-proxy.com/trackings
', data=values, headers=headers)
response_body = urlopen(request).read()
As I'm using Python 3, my code is
import json
import urllib.request
values = """
{
"tracking_number": "9361289676090919095393",
"slug": "usps"
}
"""
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer mytokenAPI'
}
url = 'https://private-anon-6175dd5596-trackhive.apiary-proxy.com/trackings'
request = urllib.request.Request(url,data=values, headers=headers)
response_body = urllib.request.urlopen(request, data=bytes(json.dumps(headers), encoding="utf-8")).read()
But when it calls the "urllib.request.urlopen", it returns "HTTPError: Bad Request". What am I doing wrong?
On the website, they said that the endpoint is located at: https://api.trackinghive.com. In their code, they are accessing: 'https://private-anon-6175dd5596-trackhive.apiary-proxy.com/trackings'. If I haven't access to their example, how could I know that I suppose to access this URL, and not something like "https://api.trackinghive.com/trackings"? Isn't it suppose to use the endpoint somewhere?
(I'm sorry about all these questions, I'm a little confused about how these codes with API and URL works)
I appreciate any help ^^
Edit:
Link to documentation:
https://trackhive.docs.apiary.io/#introduction/api-requirements/end-point
If I register on https://my.trackinghive.com/login and generate API key then code works for me.
I use requests instead of urllib because it is easier to create POST request. And documentation shows that it has to be POST, not GET.
And it is easier to send JSON data. It doesn't need header Content-Type': 'application/json' because requests will add it automatically when is use json=.... And it doesn't need to convert values to one string because requests will do it automatically.
BTW: I don't need module json to work with data. I use it only to convert it to string with indentation to display it.
import requests
values = {
"tracking_number": "9361289676090919095393",
"slug": "usps"
}
token = 'eyJh....'
headers = {
'Authorization': f'Bearer {token}' # both works for me
#'Authorization': f'{token}'
}
url = 'https://api.trackinghive.com/trackings'
r = requests.post(url, headers=headers, json=values)
print('\n--- as text ---\n')
print(r.text)
print()
print('\n--- as dict/list ---\n')
data = r.json()
print('keys:', list(data.keys()))
print('message:', data["meta"]["message"][0])
# - show it more readable -
print('\n--- more readable ---\n')
import json
text = json.dumps(data, indent=2)
print(text)
Result:
--- as text ---
{"meta":{"code":409,"message":["Tracking already exists."]},"data":{"_id":"60a5b5a0aa4e400011a0c657","current_status":"Pending","return_to_sender":false}}
--- as dict/list ---
keys: ['meta', 'data']
messge: Tracking already exists.
--- more readable ---
{
"meta": {
"code": 409,
"message": [
"Tracking already exists."
]
},
"data": {
"_id": "60a5b5a0aa4e400011a0c657",
"current_status": "Pending",
"return_to_sender": false
}
}
First time trying to extract data via an API. I'm getting stuck with how to pass the raw data. I have been able to get it to work in postman but I can't quite figure it out with Python.
I've been getting this error:
{'fault': {'faultstring': 'JSONThreatProtection[JSON-Threat-Protection-1]: Execution failed. reason: Expecting { or [ at line 1', 'detail': {'errorcode': 'steps.jsonthreatprotection.ExecutionFailed'}}}
I'm pretty sure its the data portion that is wrong but I haven't been able to figure out if it's just a syntax error or something bigger that I'm missing.
My code as follows:
import requests
import json
url = "https://url/customers/shipmentstatus"
headers = {
"Authorization": "Bearer TOKEN",
"Content-Type": "application/json"
}
data = {
"Id": [
"AZ1234"
]
}
response = requests.post(url, headers=headers, data=data)
print(response.json())
Your code posts the data as form encoded data, not as JSON. That's because you use the data parameter to supply the payload. To fix, encode the data as JSON using json.dumps():
response = requests.post(url, headers=headers, data=json.dumps(data))
But a better way is to pass the data to requests using the json parameter:
response = requests.post(url, headers=headers, json=data)
Using the json parameter is easier and it will also add the Content-Type: application/json header for you.
I have some json containing some thai values. It looks something like
{
"TitleName": "คุณ",
"FirstName": "Simar"
}
I need to make a Http POST request with this json body with the exact thai value.I am using Python 3 requests library to make the request.
I tried this
headers = {
'Content-Type': "application/json",
'Authorization': "xxx",
'cache-control': "no-cache",
'Postman-Token': "xxx"
}
response = requests.request("POST", url, json=request, headers=headers)
It generates json values as
"TitleName": "\\u0e04\\u0e38\\u0e13",
"FirstName": "Simar"
I also tried this
json_request = json.dumps(self.request_data,ensure_ascii=False).encode('utf8')
response = requests.request("POST", url, json=json_request, headers=headers)
It generates json values as
"TitleName": "\xe0\xb8\x84\xe0\xb8\xb8\xe0\xb8\x93",
"FirstName": "Simar"
But I want json values to be generated as
"TitleName": "คุณ",
"FirstName": "Simar"
Help will be appreciated. Thanks in advance.
To preserve non-ascii characters in POST requests you need to serialise to json manually, and explicitly set the content-type header.
data = json.dumps(my_dict, ensure_ascii=False)
r = requests.post(url, headers={'content-type': 'application/json'},
data=data.encode('utf-8'))
I'm trying to convert the cURL to Python request but doesn't work.
cURL: curl -kv -H 'Content-Type: application/json' 'https://IP-address/api/v1/login' -d '{"username":"api", "password":"APIPassword"}'
My Python requests code:
import requests
url = "https://IP-address/api/v1/login"
payload = "'{\"username\":\"api\", \"password\":\"APIPassword\"}'"
headers = {
'Content-Type': "application/json",
'cache-control': "no-cache",
}
response = requests.request("GET", url, headers=headers, data=payload, verify=False)
print(response.text)
Which doesn't work and gives me 400 bad requests error.
I tried converting using the https://curl.trillworks.com/
which gives me the following code which doesn't work either.
import requests
url = 'https://IP-address/api/v1/login'
headers = {
'Content-Type': 'application/json',
}
data = '{"username":"api", "password":"APIPassword"}'
output = requests.get(url, data=data, verify=False)
print (output)
Can anyone please help me identify the issue here.
Edit: I have edited 2nd script to produce output: Which gives 500 Error
Use the json parameter in requests.post for json data. It also takes care of the headers.
data = {"username":"api", "password":"APIPassword"}
response = requests.post(url, json=data, verify=False)
Another way to make sure you're sending valid JSON in your payload would be to use the json python library to format your payload via json.dumps(), which returns a string representing a json object from an object. This was especially useful to me when I needed to send a nested json object in my payload.
import json
import requests
url = 'https://sample-url.com'
headers = { 'Content-Type': 'application/json', 'Authorization': f'{auth_key}'}
payload = { "key": "value",
"key": ["v1", "v2"],
"key": {
"k": "v"
}
...
}
r = requests.post(url, headers=headers, data=json.dumps(payload))
I want to call CURL API on python.
curl -X POST -H "Authorization:Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5" \
-H "Content-Type: application/json" \
-d '{"module_id":"[MODULE_ID]", "text": "example text"}' \
-D - \
https://api.tocall.com/
I used requests module for making request and json module for converting object to string. But I'm getting 404.
Where am I wrong?
import requests
import json
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
'Content-Type': 'application/json',
}
url = "https://api.tocall.com/"
data = '{"module_id":"[MODULE_ID]", "text": "example text"}'
response= requests.post(url, data=json.dumps(data), headers=headers)
print(response.status_code)
You are encoding your data as JSON twice. json.dumps() takes an object and converts to JSON. In this case, you are converting a string to JSON. This should work better:
import requests
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
}
url = "https://api.tocall.com/"
data = {"module_id":"[MODULE_ID]", "text": "example text"}
response= requests.post(url, json=data, headers=headers)
print(response.status_code)
If it still doesn't work and you need more help, you should include real details about your API so we can reproduce the issue.
json.dumps turns a Python dict to a string, but your data is already a string. The easiest thing to do is write data as a dict then use json.dumps on that.
Add the Host header, so that the final server knows on which virtual host to route the request,
Change:
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
'Content-Type': 'application/json',
}
For:
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
'Content-Type': 'application/json',
'Host' : 'api.tocall.com'
}
I think this will fix your issue. Eventually you might want to update the default headers, not craft your own ones. Try to use the session features of requests to perform consistent queries.
Note: as stated by other answers, you have other JSON encoding issues, but that's not the reason why you are getting 404.