I have a cURL command that I would like to port to Python.
curl -XGET "http://localhost:9200/nuix-7674bc4a60b74ea7bac8996a98b0cb94;item;schema-version=1/_search" -H 'Content-Type: application/json' -d'
{
"query": {
"regexp": {
"content": "(p)hotos"
}
}
}'
It successfully returns a non-error response.
Here is what I tried in Python 3.6 using the requests package.
import requests
import json
# api-endpoint
url = "http://localhost:9200/nuix-7674bc4a60b74ea7bac8996a98b0cb94;item;schema-version=1/_search"
# headers
headers = {'Content-type': 'application/json'}
# Define JSON String
params = """
{
"query": {
"regexp":{
"content": "(p)hotos"
}
}
}
"""
params = json.loads(params)
print(params)
# sending get request and saving the response as response object
response = requests.get(url=url, params=params, headers=headers)
# extracting data in json format
data = response.json()
print(data['hits']['total'])
print('DONE')
The response response._content states this error:
b'{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/nuix-7674bc4a60b74ea7bac8996a98b0cb94;item;schema-version=1/_search] contains unrecognized parameter: [query]"}],"type":"illegal_argument_exception","reason":"request [/nuix-7674bc4a60b74ea7bac8996a98b0cb94;item;schema-version=1/_search] contains unrecognized parameter: [query]"},"status":400}'
What is the correct way to set params?
requests expects a dict as params not a string. I'm not sure this is your problem but you could try rewriting as:
params = {
"query": {
"regexp":{
"content": "(p)hotos"
}
}
}
Check out this section from the docs for a walkthrough: http://docs.python-requests.org/en/master/user/quickstart/#passing-parameters-in-urls
The solution is to use the json parameter, rather than params.
response = requests.get(url=url, json=params, headers=headers)
I probably could have rewritten the JSON string to a dictionary, but I am using a tool -- Kibana that automatically generates a cURL payload. This solution allows me to copy and paste the JSON payload into my Python script.
Related
Problem
I've looked at some of the documentation about the json and data parameters and the differences between them. I think I understand the difference, best explained here, in my opinion.
However, I have a specific request that fails on PUT using json, but fails using data, and I'm not sure why. Can someone clarify why this is the case? Could it be that there is a list in the payload?
Context
I have requests==2.28.0 installed. Below is the code that submits the PUT requests to an API for PagerDuty, the incident management software, one using data (successful) and one using json (failing). Otherwise they are identical.
The weird thing is that their examples use the json parameter.
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', data=payload, headers=headers)
print("response: ", response)
Result: response: <Response [200]>
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', json=payload, headers=headers)
print("response: ", response)
Result: response: <Response [400]>
Your payload is a string while json parameter takes a dictionary. That's the whole point of the json argument (you don't have to encode it yourself):
If you need that header set and you don’t want to encode the dict yourself, you can also pass it directly using the json parameter (added in version 2.4.2) and it will be encoded automatically:
You should pass a dictionary if you want to use the json parameter:
payload = {
"source_incidents": [
{
"id": child_incident_id,
"type": "incident_reference"
}
]
}
which is more readable anyway.
Alternatively you could use json.loads to parse your string:
import json
payload = f'{{"source_incidents": [{{"id": "{child_incident_id}", "type": "incident_reference"}}]}}'
headers = {
'Content-Type': "application/json",
'Accept': "application/vnd.pagerduty+json;version=2",
'From': email,
'Authorization': f"Token token={read_write_api_token}"
}
response = requests.put(f'https://api.pagerduty.com/incidents/{parent_incident_id}/merge', data=json.loads(payload), headers=headers)
print("response: ", response)
That is what the requests library does with json data.
Converts your Python object to JSON using the json encoder.
Sets the "content-type" header to "application/json".
It is possible to implement it as shown here.
def put(uri, data=None, json=None):
if json and data:
raise Exception()
if json:
payload = json.dumps(json)
else:
payload = data
...
So the first request returns 200 because you passed valid JSON OBJECT through the "data" parameter.
And the second request fails because you passed STRING through the JSON that will be dumped with "json.dumps(obj)" for example.
As a result, it would be nothing more than a string, which is also a valid JSON object but not a javascript object.
As shown here, if you pass a string through "json.dumps" and a dictionary, it returns two different values: a string and an object.
>>> json.dumps("{}")
'"{}"'
>>> json.dumps({})
'{}'
>>>
I have to send commands to an API using Python.
The API's documentation was written to send receive CURL commands.
I converted the curl string
curl -H "Authorization: Bearer accesstoken" https://www.website.com/feed.php?command=COMMAND
Into this:
import requests
headers = {
'Authorization': 'Bearer accesstoken',
}
params = {
'command': "COMMAND",
}
response = requests.get('https://www.website.com/feed.php', params=params, headers=headers)
The problem is if the command I want to send is a complex command like:
curl -H "Authorization: Bearer accesstoken" https://www.website.com/feed.php?command=COMMAND&wake=10
the conversion to Python won't work as this "&wake=10" isn't accepted.
Any ideas on how to circumvent this?
You can pass multiple query parameters in the params dictionary - requests will turn them into the required URL format.
import requests
headers = {
'Authorization': 'Bearer accesstoken',
}
params = {
'command': "COMMAND",
'wake': 10
}
response = requests.get('https://www.website.com/feed.php', params=params, headers=headers)
You need to split on the &, resulting in multiple key + value pairs in params.
params = {
"command": "COMMAND",
"wake": "10"
}
This is my first project using API/python. Basically, I want to get information from trackhive.com to see where is the package. But first, to understand how it works, I'm just trying to create a track.
On the website, they give me this example:
from urllib2 import Request, urlopen
values = """
{
"tracking_number": "9361289676090919095393",
"slug": "usps"
}
"""
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer Token'
}
request = Request('https://private-anon-6175dd5596-trackhive.apiary-proxy.com/trackings
', data=values, headers=headers)
response_body = urlopen(request).read()
As I'm using Python 3, my code is
import json
import urllib.request
values = """
{
"tracking_number": "9361289676090919095393",
"slug": "usps"
}
"""
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer mytokenAPI'
}
url = 'https://private-anon-6175dd5596-trackhive.apiary-proxy.com/trackings'
request = urllib.request.Request(url,data=values, headers=headers)
response_body = urllib.request.urlopen(request, data=bytes(json.dumps(headers), encoding="utf-8")).read()
But when it calls the "urllib.request.urlopen", it returns "HTTPError: Bad Request". What am I doing wrong?
On the website, they said that the endpoint is located at: https://api.trackinghive.com. In their code, they are accessing: 'https://private-anon-6175dd5596-trackhive.apiary-proxy.com/trackings'. If I haven't access to their example, how could I know that I suppose to access this URL, and not something like "https://api.trackinghive.com/trackings"? Isn't it suppose to use the endpoint somewhere?
(I'm sorry about all these questions, I'm a little confused about how these codes with API and URL works)
I appreciate any help ^^
Edit:
Link to documentation:
https://trackhive.docs.apiary.io/#introduction/api-requirements/end-point
If I register on https://my.trackinghive.com/login and generate API key then code works for me.
I use requests instead of urllib because it is easier to create POST request. And documentation shows that it has to be POST, not GET.
And it is easier to send JSON data. It doesn't need header Content-Type': 'application/json' because requests will add it automatically when is use json=.... And it doesn't need to convert values to one string because requests will do it automatically.
BTW: I don't need module json to work with data. I use it only to convert it to string with indentation to display it.
import requests
values = {
"tracking_number": "9361289676090919095393",
"slug": "usps"
}
token = 'eyJh....'
headers = {
'Authorization': f'Bearer {token}' # both works for me
#'Authorization': f'{token}'
}
url = 'https://api.trackinghive.com/trackings'
r = requests.post(url, headers=headers, json=values)
print('\n--- as text ---\n')
print(r.text)
print()
print('\n--- as dict/list ---\n')
data = r.json()
print('keys:', list(data.keys()))
print('message:', data["meta"]["message"][0])
# - show it more readable -
print('\n--- more readable ---\n')
import json
text = json.dumps(data, indent=2)
print(text)
Result:
--- as text ---
{"meta":{"code":409,"message":["Tracking already exists."]},"data":{"_id":"60a5b5a0aa4e400011a0c657","current_status":"Pending","return_to_sender":false}}
--- as dict/list ---
keys: ['meta', 'data']
messge: Tracking already exists.
--- more readable ---
{
"meta": {
"code": 409,
"message": [
"Tracking already exists."
]
},
"data": {
"_id": "60a5b5a0aa4e400011a0c657",
"current_status": "Pending",
"return_to_sender": false
}
}
I'm trying to convert the cURL to Python request but doesn't work.
cURL: curl -kv -H 'Content-Type: application/json' 'https://IP-address/api/v1/login' -d '{"username":"api", "password":"APIPassword"}'
My Python requests code:
import requests
url = "https://IP-address/api/v1/login"
payload = "'{\"username\":\"api\", \"password\":\"APIPassword\"}'"
headers = {
'Content-Type': "application/json",
'cache-control': "no-cache",
}
response = requests.request("GET", url, headers=headers, data=payload, verify=False)
print(response.text)
Which doesn't work and gives me 400 bad requests error.
I tried converting using the https://curl.trillworks.com/
which gives me the following code which doesn't work either.
import requests
url = 'https://IP-address/api/v1/login'
headers = {
'Content-Type': 'application/json',
}
data = '{"username":"api", "password":"APIPassword"}'
output = requests.get(url, data=data, verify=False)
print (output)
Can anyone please help me identify the issue here.
Edit: I have edited 2nd script to produce output: Which gives 500 Error
Use the json parameter in requests.post for json data. It also takes care of the headers.
data = {"username":"api", "password":"APIPassword"}
response = requests.post(url, json=data, verify=False)
Another way to make sure you're sending valid JSON in your payload would be to use the json python library to format your payload via json.dumps(), which returns a string representing a json object from an object. This was especially useful to me when I needed to send a nested json object in my payload.
import json
import requests
url = 'https://sample-url.com'
headers = { 'Content-Type': 'application/json', 'Authorization': f'{auth_key}'}
payload = { "key": "value",
"key": ["v1", "v2"],
"key": {
"k": "v"
}
...
}
r = requests.post(url, headers=headers, data=json.dumps(payload))
I want to call CURL API on python.
curl -X POST -H "Authorization:Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5" \
-H "Content-Type: application/json" \
-d '{"module_id":"[MODULE_ID]", "text": "example text"}' \
-D - \
https://api.tocall.com/
I used requests module for making request and json module for converting object to string. But I'm getting 404.
Where am I wrong?
import requests
import json
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
'Content-Type': 'application/json',
}
url = "https://api.tocall.com/"
data = '{"module_id":"[MODULE_ID]", "text": "example text"}'
response= requests.post(url, data=json.dumps(data), headers=headers)
print(response.status_code)
You are encoding your data as JSON twice. json.dumps() takes an object and converts to JSON. In this case, you are converting a string to JSON. This should work better:
import requests
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
}
url = "https://api.tocall.com/"
data = {"module_id":"[MODULE_ID]", "text": "example text"}
response= requests.post(url, json=data, headers=headers)
print(response.status_code)
If it still doesn't work and you need more help, you should include real details about your API so we can reproduce the issue.
json.dumps turns a Python dict to a string, but your data is already a string. The easiest thing to do is write data as a dict then use json.dumps on that.
Add the Host header, so that the final server knows on which virtual host to route the request,
Change:
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
'Content-Type': 'application/json',
}
For:
headers = {
'Authorization': 'Token 00d2e3a10c82420414b2d36d28fb5afc2cd8e8a5',
'Content-Type': 'application/json',
'Host' : 'api.tocall.com'
}
I think this will fix your issue. Eventually you might want to update the default headers, not craft your own ones. Try to use the session features of requests to perform consistent queries.
Note: as stated by other answers, you have other JSON encoding issues, but that's not the reason why you are getting 404.