I need to create an endpoint that can receive the following JSON and recognize the objects contained in it:
{
"data": [
{
"start": "A", "end": "B", "distance": 6
},
{
"start": "A", "end": "E", "distance": 4
}
]
}
I created a model to handle a single object:
class GraphBase(BaseModel):
start: str
end: str
distance: int
And with it, I could save it in a database. But now I need to receive a list of objects and save them all.
I tried to do something like this:
class GraphList(BaseModel):
data: Dict[str, List[GraphBase]]
#app.post("/dummypath")
async def get_body(data: schemas.GraphList):
return data
But I keep getting this error on FastApi: Error getting request body: Expecting property name enclosed in double quotes: line 1 column 2 (char 1) and this message on the response:
{
"detail": "There was an error parsing the body"
}
I'm new to python and even newer to FastApi, how can I transform that JSON to a list of GraphBaseto save them in my db?
This is a working example.
from typing import List
from pydantic import BaseModel
from fastapi import FastAPI
app = FastAPI()
class GraphBase(BaseModel):
start: str
end: str
distance: int
class GraphList(BaseModel):
data: List[GraphBase]
#app.post("/dummypath")
async def get_body(data: GraphList):
return data
I could try this API on the autogenerated docs.
Or, on the console (you may need to adjust the URL depending on your setting):
curl -X 'POST' \
'http://localhost:8000/dummypath' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"data": [
{
"start": "string",
"end": "string",
"distance": 0
}
]
}'
The error looks like the data problem. And I found that you have extra spaces at several places. Try the following:
{
"data": [
{
"start": "A", "end": "B", "distance": 6
},
{
"start": "A", "end": "E", "distance": 4
}
]
}
The positions of extra spaces (which I removed) are below:
Related
resp = {
"Name": "test",
"os": "windows",
"Agent": {
"id": "2",
"status": [
{
"code": "123",
"level": "Info",
"displayStatus": "Ready",
"message": "running",
"time": "2022-01-18T09:51:08+00:00"
}
]
}
I am trying to get the time value from the JSON.
I tried the below code but faced error with dict
resp1 = json.loads(resp)
resp2 = resp1.values()
creation_time = resp2.get("Agent").get("status")
val= creation_time["time"]
print(val) ## Thrwoing error as dict_values has no 'get'
Any suggestion on python how to take this time values
Few problems I noticed
You are trying to load a Dict type using the json's loads function which is supposed to get a string in json format (ex: '{ "name":"John", "age":30, "city":"New York"}')
You tried to access resp2 before declaration (I guessed you meant "resp1?")
You're using resp3 without declaration.
You are missing }
You don't need the .value() function because it will return a list.
Also creation time is a list with one object, so you need to access it too.
Considering all this, you can change it as follows:
import json
resp = '{ "Name": "test", "os": "windows","Agent": {"id": "2","status": [{"code": "123","level": "Info","displayStatus": "Ready","message": "running","time": "2022-01-18T09:51:08+00:00"}]}}'
resp1 = json.loads(resp)
creation_time = resp1.get("Agent").get("status")
val= creation_time[0]["time"]
print(val)
You just need to access the dicts using [] like so:
resp = {"Name": "test", "os": "windows", "Agent": {"id": "2","status": [{"code": "123","level": "Info","displayStatus": "Ready","message": "running","time": "2022-01-18T09:51:08+00:00"}]}}
creation_time = resp["Agent"]["status"]
val= creation_time[0]["time"]
print(val)
Output:
2022-01-18T09:51:08+00:00
I am working on a new project in HubSpot that returns nested JSON like the sample below. I am trying to access the associated contacts id, but am struggling to reference it correctly (the id I am looking for is the value '201' in the example below). I've put together this script, but this script only returns the entire associations portion of the JSON and I only want the id. How do I reference the id correctly?
Here is the output from the script:
{'contacts': {'paging': None, 'results': [{'id': '201', 'type': 'ticket_to_contact'}]}}
And here is the script I put together:
import hubspot
from pprint import pprint
client = hubspot.Client.create(api_key="API_KEY")
try:
api_response = client.crm.tickets.basic_api.get_page(limit=2, associations=["contacts"], archived=False)
for x in range(2):
pprint(api_response.results[x].associations)
except ApiException as e:
print("Exception when calling basic_api->get_page: %s\n" % e)
Here is what the full JSON looks like ('contacts' property shortened for readability):
{
"results": [
{
"id": "34018123",
"properties": {
"content": "Hi xxxxx,\r\n\r\nCan you clarify on how the blocking of script happens? Is it because of any CSP (or) the script will decide run time for every URL’s getting triggered from browser?\r\n\r\nRegards,\r\nLogan",
"createdate": "2019-07-03T04:20:12.366Z",
"hs_lastmodifieddate": "2020-12-09T01:16:12.974Z",
"hs_object_id": "34018123",
"hs_pipeline": "0",
"hs_pipeline_stage": "4",
"hs_ticket_category": null,
"hs_ticket_priority": null,
"subject": "RE: call followup"
},
"createdAt": "2019-07-03T04:20:12.366Z",
"updatedAt": "2020-12-09T01:16:12.974Z",
"archived": false
},
{
"id": "34018892",
"properties": {
"content": "Hi Guys,\r\n\r\nI see that we were placed back on the staging and then removed again.",
"createdate": "2019-07-03T07:59:10.606Z",
"hs_lastmodifieddate": "2021-12-17T09:04:46.316Z",
"hs_object_id": "34018892",
"hs_pipeline": "0",
"hs_pipeline_stage": "3",
"hs_ticket_category": null,
"hs_ticket_priority": null,
"subject": "Re: Issue due to server"
},
"createdAt": "2019-07-03T07:59:10.606Z",
"updatedAt": "2021-12-17T09:04:46.316Z",
"archived": false,
"associations": {
"contacts": {
"results": [
{
"id": "201",
"type": "ticket_to_contact"
}
]
}
}
}
],
"paging": {
"next": {
"after": "35406270",
"link": "https://api.hubapi.com/crm/v3/objects/tickets?associations=contacts&archived=false&hs_static_app=developer-docs-ui&limit=2&after=35406270&hs_static_app_version=1.3488"
}
}
}
You can do api_response.results[x].associations["contacts"]["results"][0]["id"].
Sorted this out, posting in case anyone else is struggling with the response from the HubSpot v3 Api. The response schema for this call is:
Response schema type: Object
String results[].id
Object results[].properties
String results[].createdAt
String results[].updatedAt
Boolean results[].archived
String results[].archivedAt
Object results[].associations
Object paging
Object paging.next
String paging.next.after
String paging.next.linkResponse schema type: Object
String results[].id
Object results[].properties
String results[].createdAt
String results[].updatedAt
Boolean results[].archived
String results[].archivedAt
Object results[].associations
Object paging
Object paging.next
String paging.next.after
String paging.next.link
So to access the id of the contact associated with the ticket, you need to reference it using this notation:
api_response.results[1].associations["contacts"].results[0].id
notes:
results[x] - reference the result in the index
associations["contacts"] -
associations is a dictionary object, you can access the contacts item
by it's name
associations["contacts"].results is a list - reference
by the index []
id - is a string
In my case type was ModelProperty or CollectionResponseProperty couldn't reach dict anyhow.
For the record this got me to go through the results.
for result in list(api_response.results):
ID = result.id
I am trying to dynamically add a map element using the nftables JSON API from python. In my firewall I have the following map in the router table in the ip family:
map port_forwards {
type inet_service: ipv4_addr . inet_service;
}
Here is a minimal example of what I am trying to do:
import nftables
nft_cmd = {"nftables": [
{ "add": { "element":{
"family": "ip",
"table": "router",
"name": "port_forwards",
"elem": { "map": {
"key": "80",
"data": "172.16.0.1 . 80"
}}
}}}
]}
nft = nftables.Nftables()
nft.json_validate(nft_cmd)
rc, _output, error = nft.json_cmd(nft_cmd)
if rc != 0:
raise RuntimeError(f"Error running nftables command: {error}")
This results in the following error:
RuntimeError: Error running nftables command: internal:0:0-0: Error: Unexpected JSON type object for immediate value.
internal:0:0-0: Error: Invalid set.
internal:0:0-0: Error: Parsing command array at index 0 failed.
I assume I am mis-understanding the spec somehow (https://manpages.debian.org/unstable/libnftables1/libnftables-json.5.en.html), but I can't figure out the correct usage.
UPDATE: I have discovered nft can echo your command in json format. This is the command:
sudo nft -e -j add element ip router port_forwards '{80 : 172.16.0.1 . 8080 }'
and the response pretty-printed:
{"nftables": [
{"add": {"element": {
"family": "ip",
"table": "router",
"name": "port_forwards",
"elem": {"set": [[
80,
{"concat": ["172.16.0.1", 8080]}
]]}
}}}
]}
Unfortunately copying this into the above python code still results in the same error
It turns out that the "elem" property takes the array directly instead of being wrapped in a "set" object. This was hinted at by the error:
Unexpected JSON type object for immediate value.
The working code is shown below:
import nftables
nft_cmd = {"nftables": [
{ "add": { "element":{
"family": "ip",
"table": "router",
"name": "port_forwards",
"elem": [[
80,
{"concat": ["172.16.0.1", 8080]}
]]
}}}
]}
nft = nftables.Nftables()
nft.json_validate(nft_cmd)
rc, _output, error = nft.json_cmd(nft_cmd)
if rc != 0:
raise RuntimeError(f"Error running nftables command: {error}")
I want to convert this json :
{
"rate_limit_by":
[{ "type": "IP",
"extract_from_header": "X-Forwarded-For"
}]
}
to this:
"{\"rate_limit_by\": [{\"type\": \"IP\", \"extract_from_header\": \"X-Forwarded-For\"}]}".
So that i can send it as part of payload in request in Python.
And i have tried multiple methods for the same. json.dumps doesnt work cause it doesnt escape characters in this case & .replace(""",r"\"") doesnt work cause it creates the string like this :
{\\"rate_limit_by\\": [{\\"type\\": \\"IP\\", \\"extract_from_header\\": \\"X-Forwarded-For\\"}]}
(Below is the example of curl but i want to send the data in specific format using python request.)
My upstream expects data in certain format, as of now am sending data to upstream as below:
curl -i --request POST --data "rule_name=only_ip" \
--data-binary "#data.txt" \
--url http://localhost:8001/plugin/rules
Where data.txt looks like this:
rule={
"rate_limit_by": [
{ "type":"IP", "extract_from_header": "X-Forwarded-For" }
]
}
Am trying to convert it to :
curl -i --request POST -H 'Content-Type: application/json' --data-binary #data.json http://localhost:8001/plugin/rules
Where data.json should like this
{
"rule_name" : "test_ip",
"rule":"{\"rate_limit_by\": [{\"type\": \"IP\", \"extract_from_header\": \"X-Forwarded-For\"}]}"
}
Now the value of "rule" is string with character escape.
This am trying to achieve & am doing post using python.
And below is the code for same:-
import requests
import json
import re
url = 'http://localhost:8001/plugin/rules'
rule = {
"rate_limit_by":
[{ "type": "IP",
"extract_from_header": "X-Forwarded-For"
}]
}
rule = json.dumps(json.dumps(rule))
print(rule) #this output the data in correct format
obj = {
"rule_name" : "test_ip",
"rule": rule #but when used it here its get wrapped in two \\
}
headers = {'Content-Type': 'application/json', 'Accept': 'text/plain'}
print(obj)
r = requests.post(url, data=obj, headers=headers)
print(r.text)
desired is what you say you need in your something.json file. The following prints True. See https://repl.it/repls/DistantTeemingProtocol.
import json
desired = r'''{
"rule_name" : "test_ip",
"rule":"{\"rate_limit_by\": [{\"type\": \"IP\", \"extract_from_header\": \"X-Forwarded-For\"}]}"
}'''
d = {
"rate_limit_by": [{
"type": "IP",
"extract_from_header": "X-Forwarded-For"
}]
}
s = json.dumps(d)
xxx = json.dumps({"rule_name": "test_ip", "rule": s}, indent=4)
o = json.loads(desired)
yyy = json.dumps(o, indent=4)
print(xxx == yyy)
If you're going to POST using requests, then you should not be posting the string but should instead be posting the dictionary.
I.e.,
r = requests.post(url, json={"rule_name": "test_ip", "rule": s})
Do you mean you want to access the items inside somehow?
You should drop the "[]" because that part doesn't really make sense.
import json
x = str({
"rate_limit_by":
[{ "type": "IP",
"extract_from_header": "X-Forwarded-For"
}]
})
x = x.replace("[","")
x = x.replace("]","")
x = eval(x)
d = json.dumps(x)
l = json.loads(d)
l['rate_limit_by']['type']
This outputs "IP". Now you have a dictionary of all you need called l.
I am using the Fiware Orion Context Broker and I want to POST some data using a python script. The command line (which works fine) looks like this:
curl -X POST -H "Accept: application/json" -H "Fiware-ServicePath: /orion" -H "Fiware-Service: orion" -H "Content-Type: application/json" -d '{"id": "JetsonTX1", "type": "sensor", "title": {"type": "Text","value": "Init"}, "percentage": { "type": "Text", "value": "0%"}}' "http://141.39.159.63:1026/v2/entities/"
My Python script:
import requests
import json
url = 'http://141.39.159.63:1026/v2/entities/'
data = '''{
"title": {
"value": "demo",
"type": "Text"
},
"percentage": {
"type": "Text",
"value": "0%"
}'''
data_json = json.dumps(data)
headers = {"Accept": "application/json", "Fiware-ServicePath": "/bonseyes", "Fiware-Service": "bonseyes", "Content-Type": "application/json"}
response = requests.post(url, data=data_json, headers=headers)
print(response.json())
This is what response.json() returns:
{u'description': u'Errors found in incoming JSON buffer', u'error': u'ParseError'}
Any ideas how to fix this?
Thank you!
You probably should not pass data as string like so:
data = '''{
"title": {
"value": "demo",
"type": "Text"
},
"percentage": {
"type": "Text",
"value": "0%"
}'''
Pass it as normal dict:
data = {
"title": {
"value": "demo",
"type": "Text"
},
"percentage": {
"type": "Text",
"value": "0%"
}}
The request library will automatically convert this dictionary for you. Also make sure you want to use the data parameter and not the json. Below expert from the documentation should clear out why.
def post(url, data=None, json=None, **kwargs):
r"""Sends a POST request.
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary (will be form-encoded), bytes, or file-like object to send in the body of the :class:`Request`.
:param json: (optional) json data to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""
From your comment it seems you should pass your data like so:
response = requests.post(url, json=data_json, headers=headers)
Because your endpoint requires json and not form-encoded bytes
And there is also missing curly brace at the end.
In my opinion you should try using keyValues option for OCB. It will make your payload way shorter. I use similar python program for updating values, therefore PATCH request in my approach:
#Sorting out url and payload for request
data = '{"' + attribute + '":' + value + '}'
headers = {'Content-type': 'application/json'}
url = urlOfOCB + '/v2/entities/' + entityId + '/attrs?options=keyValues'
r = requests.patch(url, (data), headers=headers)
You can read about this option here. As I can see you are not defining any new type for your attributes, therefore it will be "Text" by default, while using keyValues.
Attribute/metadata type may be omitted in requests. When omitted in attribute/metadata creation or in update operations, a default is used for the type depending on the value:
If value is a string, then type Text is used
If value is a number, then type Number is used.
If value is a boolean, then type Boolean is used.
If value is an object or array, then StructuredValue is used.
If value is null, then None is used.
More on those things you can find here.