We Keep Coding

Python jsonpath-ng : How to build a json document from jsonpath and values?

Developing in python, I would like to create a json document (tree) from a list of jsonpath and corresponding values.
For example, from the jsonpath
"$.orders[0].client"
and a value "acme", it would create:
{'orders':[{'client':'acme'}]}
And then, I should be able to add a new node in the json document, at the location specified by another jsonpath with a given value.
For example adding jsonpath $.orders[0].orderid and value "123", this would result in an updated json document:
{'orders':[{'client':'acme', 'orderid' : 123}]}
I have tried to understand jsonpath-ng for that purpose, but I don't understand how I could use it for that purpose (and not even if this is possible).
Anyone who could help?
Thanks in advance,
Best regards,
Laurent

If I understand your question correctly you are asking for a JSON to JSON transformation. JSONPath is not the right tool then. There are specialized libraries for this, e.g. JSONBender. You can transform some JSON like this:
import json
from jsonbender import bend, K, S
MAPPING = {
'client': S('orders', 0, 'client'),
'orderid': S('orders', 0, 'orderid')
}
source = {
"orders" : [
{
"orderid": 123,
"client" : "acme"
},
{
"orderid": 321,
"client" : "two"
}
]
}
result = bend(MAPPING, source)
print(json.dumps(result))
{"client": "acme", "orderid": 123}
You can certainly bend the result even further but simpler output might work even better.

How to Create your own JSON parser library in python

Question asked to me in an Interview.
To write a Library in python which can parse the JSON data.
It can be a JSONObject or JSON String or anything else.
It should be able to handle all the types of data types
example JSON Data
{
"name": "JaneDoe",
"age": 42,
"smoking": false,
"education": {
"school": "abcSchool",
"University": "xyz"
},
"certificates": ["ccna", "python", "aws"],
"salary": 4200.0,
"profile_img": "https://ii.abc.com/jpeg/profiles/jane.jpg"
}

https://www.json.org/json-ru.html gives all you need to start - there are simple block diagrams, showing how to parse different supported types, like this:
So just rewrite them in Python

Best practice for collections in jsons: array vs dict/map

I need to pass data in a python back-end to a front end through an api call, using a json format. In the python back end, the data is in a dictionary structure, which I can easily and directly convert to a json. But should I?
My front-end developer believes the answer is no, for reasons related to best practice.
But I challenge that:
Is the best to structure a json as it is in python, or should it rather be converted to some other form, such as several arrays (as would be necessary in my example case below)?
Or, differently put:
What should be the governing principles related to collections/dicts/maps/arrays for interfacing information through jsons?
I've done some googling for an answer, but I've not come across much that addresses this directly. Links would be appreciated.
(Note about the example below: of course if the data is written to a database, it would probably make most sense for the front-end to access the database directly, but let's assume this is not the case)
Example:
In the back end there is a collection of objects called pets:
each item in the collection has a unique pet_id, some non-optional properties, e.g. name and date_of_birth, some optional properties registration_certificate_nr, adopted_from_kennel, some lists like siblings and children and some objects like medication.
Assuming that the front end needs all of this info at some point, it could be
{
"pets": {
"17-01-24-01": {
"name": "Buster",
"date_of_birth": "04/01/2017",
"registration_certificate_nr": "AAD-1123-1432"
},
"17-03-04-01": {
"name": "Hooch",
"date_of_birth": "05/02/2015",
"adopted_from_kennel": "Pretoria Shire",
"children": [
"17-05-01-01",
"17-05-01-02",
"17-05-01-03"
]
},
"17-05-01-01": {
"name": "Snappy",
"date_of_birth": "17-05-01",
"siblings": [
"17-05-01-02",
"17-05-01-03"
]
},
"17-05-01-02": {
"name": "Gizmo",
"date_of_birth": "17-05-01",
"siblings": [
"17-05-01-01",
"17-05-01-03"
]
},
"17-05-01-03": {
"name": "Toothless",
"date_of_birth": "17-05-01",
"siblings": [
"17-05-01-01",
"17-05-01-03"
],
"medication": [
{
"name": "anti-worm",
"code": "aw445",
"dosage": "1 pill per day"
},
{
"name": "disinfectant",
"code": "pdi-2",
"dosage": "as required"
}
]
}
}
}

JSON formatting is a somewhat subjective matter, and related disagreements are usually best settled between colleagues.
That being said, there are some potentially valid criticisms to be made against the JSON format in the question, especially if we are trying to create a consistent, RESTful API.
The 2 pain points that stand out:
A map collection is represented in JSON, which isn't really JSON standard compliant, or particularly RESTful.
None of the pet objects have an id defined. There is a pet_id mentioned in the question, but it seems to be maintained separately from the pet object itself. If a value is accessed in the pets map in the question, a user of the API would have to manually add the pet_id to the provided pet object in order to have the id available further down the line, when the full JSON may no longer be available.
The closest things we have to guiding standards in this situation is the REST architectural style and the JSON standard.
We can start by looking at the JSON standard. Here is a quote from the JSON wiki:
JavaScript syntax defines several native data types that are not included in the JSON standard: Map, Set, Date, Error, Regular Expression, Function, Promise, and undefined.
The key takeaway here is that JSON is not meant to represent the map data type. Python dictionaries are a map implementation, so directly serializing a dictionary to JSON with the intent to represent a map-like collection goes against the intended use of JSON.
For an individual object like a pet, the JSON object is appropriate, but for collections there is one option: the JSON array. There is a usage example with the JSON array further down in this answer.
There may be edge cases where deviating from the standard makes sense, but I don't see a reason in this scenario.
There are also some shortcomings in the JSON format from a RESTful design perspective. RESTful API design is nice because it encourages one to keep things simple and consistent. It also happens to be a de facto industry standard.
In a RESTful HTTP API, this is how fetching a single pet resource should look:
Request: GET /api/pets/17-01-24-01
Response: 200 {
"id": "17-01-24-01",
"name": "Buster",
"date_of_birth": "04/01/2017",
"registration_certificate_nr": "AAD-1123-1432"
}
The response is a completely defined resource with an explicitly defined id. It is also the simplest complete JSON representation of a pet.
Next, we define what fetching multiple pet resources looks like, assuming only 2 pets are defined:
Request: GET /api/pets
Response: 200 [
{
"id": "17-01-24-01",
"name": "Buster",
"date_of_birth": "04/01/2017",
"registration_certificate_nr": "AAD-1123-1432"
},
{
"id": "17-03-04-01",
"name": "Hooch",
"date_of_birth": "05/02/2015",
"adopted_from_kennel": "Pretoria Shire",
"children": [
"17-05-01-01",
"17-05-01-02",
"17-05-01-03"
]
}
]
The above response format is the most straight forward way to pluralize the single resource response format, thus keeping the API as simple and consistent as possible. (for the sake of brevity, I only used 2 of the sample resources from the question). Once again, the ids are explicitly defined, and belong to their respective pet objects.
Nothing is gained from adding map keys to the above format.
Proponents of the JSON format in the question may suggest to just add the id field into each pet object in order to work around pain point 2, but that would raise the question of repeating data within the response. Why does the id need to be both inside and outside the object? Surely it should only be on the inside? After eliminating the redundant data, the result will look like the response above.
That is the REST argument. There are use cases where REST doesn't really work, but this is far from that.
PS. Front ends should never access databases directly. The API is responsible for writing to and reading from whatever data persistence infrastructure is used. In a lot of bigger real world systems, there is even an additional BFF layer between the front end and the API(s), separating the front end and the DB even further.

Variable changes its format on printing

I am trying to store a variable with Key Value data in a file using Python, but when I try printing it, it comes up in a different format.
I want the result to be printed like this-
data={"name":'name',"description": "This is my offering","icon":"/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library","version": "1.0.0",
"design": {
"#self": "#self"
}
}
This is the output I get while printing the data-
{'icon': '/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library', 'design': {'#self': '#self'}, 'name': 'name', 'version': '1.0.0', 'description': 'This is my offering'}

You haven't stated what is important to you when printing, nor how you are currently attempting to print.
There is no formatting within a dictionary. Any formatting in your code is merely to make the code look human readable and is not actually stored within your data dictionary (only formatting within each string element is retained, ie, between a pair of quotes).
If it is merely the format (multiple lines and indents) that you are concerned about, the easiest way to resolve that is to use either the Pretty Print module or the JSON module - either should do the job, depending on your preferences for how you want the data to look and how much control you want to have over the printed output format. In particular, the JSON output occupies more vertical screen space, but some people may think that it is marginally more human readable.
PrettyPrint pprint:
import pprint
data={"name":'name',"description": "This is my offering","icon":"/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library","version": "1.0.0","design": {"#self": "#self"}}
pp = pprint.PrettyPrinter(indent=4)
pp.pprint(data)
>>>
{ 'description': 'This is my offering',
'design': { '#self': '#self'},
'icon': '/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library',
'name': 'name',
'version': '1.0.0'}
>>>
JSON dumps:
import json
data={"name":'name',"description": "This is my offering","icon":"/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library","version": "1.0.0","design": {"#self": "#self"}}
print(json.dumps(data, indent=4))
>>>
{
"icon": "/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library",
"design": {
"#self": "#self"
},
"name": "name",
"version": "1.0.0",
"description": "This is my offering"
}
>>>
If you are concerned about the the order in which the items are printed, then you'll need to have an array that stores the keys in their preferred order (dictionaries don't have any inherant ordering), and then iterate through your keys and print the dictionary items out manually one by one (perhaps using a list comprehension on your keys array).

Python doesn't respect the indention or newlines you use to define your data structure, and ignores any format you had when print()ing it. You don't have very many options here, but perhaps you can use the json.dumps() function to format your code. The format does not match your expected output exactly, but it comes fairly close:
>>> data = {"name":'name',"description": "This is my offering","icon":"/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library","version": "1.0.0",
"design": {
"#self": "#self"
}
}
>>> import json
>>> print(json.dumps(data, indent=2))
{
"description": "This is my offering",
"version": "1.0.0",
"icon": "/csa/api/blobstore/Magic_RGB_blue_NT.png?tag=library",
"name": "name",
"design": {
"#self": "#self"
}
}
>>>

Note that a variable doesn't contain any formatting to be changed. The python interpreter reads text from the REPL or a file and turns them into instructions for the CPU of your machine. None of this involves formatting until you call print(). By default, print() calls str() which then provides some very basic formatting. For dictionaries, this includes the curly braces, colons, and commas. If you want anything more than this, you will need to do it yourself. Alternatively, you can find a Python module that helps reduce some of the tedium.

Which API response format to choose if there are json and text?

I'm coding an API library in Python, I always chose json before as the response format but this API provides text and json formats, I'm not asking which one is easier or better, I want to know the advantages and the disadvantages of using both as I only worked with json before.
I thought about using text format, it's very easy to parse but the only thing I thought about was the nested elements, but after checking the example they're separated with underscores _ for example:
name=VALUE
lastname=VALUE
age=VALUE
contact_email=VALUE
contact_phone=VALUE
contact_mobile=VALUE
same json response:
{
"name": "VALUE",
"lastname": "VALUE",
"age": "VALUE",
"contact": {
"email": "VALUE",
"phone": "VALUE",
"mobile": "VALUE"
}
}
So is there any advantages or disadvantages of using text over json or the other way around ?

IMHO the main advantages of json over text would be:
DRY - there are third party libs for json processing which are quite performant
The parsed json ends up being a dict, which you cam refer by keys easily
For the text variant, there is the .ini format, which could offer something aling the lines of structure, although the format you described above is not really designed for nested fields and structures.
I'd also look at who's consuming your API. E.g. if it's a web app, then json is the accepted format. if your consumer is something else which is more comfortable with the text format...

JSON can be easier as you can use the json library.
The return you've shown there can be put into a dictionary with the line:
import json
json.loads(return_string)
Much easier to parse than the text!