We Keep Coding

Assign variable value in JSON payload fomat

I am using python 3.6.3a. I would like to generate payload for each of the json records. I am using each variable to access the record. How to assign variable value (each in this case) in payload? I tried {each} and other methods but didn't work.
code snippet below.
json_records = [{"description":"<p>This is scenario1<\/p>","owner":"deb",
"priority":"high"},
{"description":"<p>This is scenario2<\/p>","owner":"deb",
"priority":"medium"}]
json_object = json.loads(json_records)
for each in json_object:
payload = """
{
"subject": "test",
"fieldValues": [
{each}
]
}
"""

There are two ways to approach this problem.
One way could be creating a dict() object and inserting keys as you wish, then json.dumps(object) to convert into string payload as in:
import json
json_records = [{"description":"This is scenario1</p>","owner":"deb","priority":"high"}
,{"description":"This is scenario2</p>","owner":"deb","priority":"medium"}]
for obj in json_records:
payload = dict()
payload['subject'] = 'test'
for key,value in obj.items():
payload['fieldName'] = {
key:value
}
print(json.dumps(payload))
#{"subject": "test", "fieldName": {"priority": "high"}}
#{"subject": "test", "fieldName": {"priority": "medium"}}
Second way is to create a textual payload from string as in, however if you need a valid JSON at the end, this would require a post-step of validation (something like try json.loads(payload) - So I'd just use the first method. I would use this method only if I have a specific requirements to generate the payload in a certain way.
import json
json_records = [{"description":"This is scenario1</p>","owner":"deb","priority":"high"}
,{"description":"This is scenario2</p>","owner":"deb","priority":"medium"}]
# json_object = json.loads(json_records) # json.loads works only on byte-like strings. your object is already in python in this case.
for obj in json_records:
payload = """
{
"subject": "test",
"fieldValues": [
%s
]
}
""" % (obj["priority"])
print(payload)
#{
# "subject": "test",
# "fieldValues": [
# high
# ]
# }
#
#
# {
# "subject": "test",
# "fieldValues": [
# medium
# ]
# }

You could make payload a Template string and use it to put the data in each JSON record into the format you want. Bracket {} characters have not special meaning in Templates, which is what makes using them easy.
Doing that will create a valid string representation of a dictionary containing everything. You can turn this into an actual Python dictionary data-structure using the ast.literal_eval() function, and then convert that into JSON string format — which I think is the final format you're after.
rom ast import literal_eval
import json
from string import Template
from textwrap import dedent
json_records = '''[{"description":"<p>This is scenario1<\/p>","owner":"deb",
"priority":"high"},
{"description":"<p>This is scenario2<\/p>","owner":"deb",
"priority":"medium"}]'''
json_object = json.loads(json_records)
payload = Template(dedent("""
{
"subject": "test",
"fieldValues": [
$each
]
}""")
)
for each in json_object:
obj = literal_eval(payload.substitute(dict(each=each)))
print(json.dumps(obj, indent=2))
Output:
{
"subject": "test",
"fieldValues": [
{
"description": "<p>This is scenario1</p>",
"owner": "deb",
"priority": "high"
}
]
}
{
"subject": "test",
"fieldValues": [
{
"description": "<p>This is scenario2</p>",
"owner": "deb",
"priority": "medium"
}
]
}

Append to an array inside a JSON object based on key in python

I have some JSON I'm looping through in the following format. I need to create an object for each unique primary key found in the source data and append to an array. I'm not sure how I would create the object on first encounter of the key and append to it on the next encounter. My initial attempt just creates a new object for each object in the source. Wasn't able to find an example in python only js.
Source data format:
[
...
{
"Id": "NOT NEEDED DATA",
"Client": {
"Id": "KEY",
"Name": "NOT NEEDED DATA"
},
"Name": "DESIRED DATAPOINT"
},
...
]
Desired format:
[
...
{
"client_id": "KEY",
"locations": ["DATA", "DATA"]
}
...
]
pseudocode
for i in sourcedata:
client_id = i['Client']['Id']
location_name = i['Name']
obj = {
"client_id": client_id,
"locations": [location_name]
}
new_array.append(obj)

You can first iterate and build a dictionary for then creating a list of dictionaries as specified in your output format.
from collections import defaultdict
# create and populate the dictionary
d = defaultdict(list)
for i in sourcedata:
client_id = i['Client']['Id']
location_name = i['Name']
d[client_id].append(location_name)
# build the result
res = [{"client_id": k, "locations": v} for k,v in d.items()]

How to change values in a json file using JsonPath in python

I have the following Json file: car_models.json
{
"name":"John",
"age":30,
"cars":
[
{
"car_model": "Mustang",
"car_brand": "Ford"
},
{
"car_model": "cx-5",
"car_brand": "Mazda"
}
]
}
I have another json file data_change.json, which contains details about the jsonpath and their values:
{
"testcase_ID": "test_1A",
"description": "Some description",
"request_change_data": [
{
"element_path": "$.cars.[0].car_model",
"element_value": "focus"
}
]
}
I want to read the data_change.json content, use the element_path from here, parse through car_models.json and update its value to the value from data_change.json.
As in, I want to use the jsonPath - $cars[0].car_model, parse through car_models.json, and change the value of car_model from Mustang to focus. So my updated car_models.json should be the following:
{
"name":"John",
"age":30,
"cars":
[
{
"car_model": "focus",
"car_brand": "Ford"
},
{
"car_model": "cx-5",
"car_brand": "Mazda"
}
]
}
How can I do this in python?

guessing that the Expected answer needs to have "focus" and not "ford"
The following should give you this:
import json
import re
with open('cars_model.json') as f:
cars_model = json.load(f)
with open('data_change.json') as f:
data_change = json.load(f)
for elements in data_change['request_change_data']:
element_path = elements['element_path']
#Reg ex to get you the number (as a string) between the square brackets
position_match = re.match(r"^.*\[(.*)\].*$", element_path)
position = int(position_match.group(1))
print position
# Split on "period" to get the thing to match
thing_to_change = element_path.split(".")[1]
print thing_to_change
value = elements['element_value']
print value
cars_model['cars'][0][thing_to_change] = value
print cars_model

Parsing a nested JSON keys and getting the values in a CSV format

I have a nested JSON data like this of about 5000 records.
{
"data": {
"attributes": [
{
"alert_type": "download",
"severity_level": "med",
"user": "10.1.1.16"
},
{
"alert_type": "download",
"severity_level": "low",
"user": "10.2.1.18"
}
]
}
}
Now , I need to parse this JSON and get only certain fields in a CSV format. Let's we would need alert_type & user in a CSV format.
I tried to parse this JSON dictionary:
>>> import json
>>> resp = '{"data":{"attributes":[{"alert_type":"download","severity_level":"med","user":"10.1.1.16"},{"alert_type":"download","severity_level":"low","user":"10.2.1.18"}]}}'
>>> user_dict = json.loads(resp)
>>> event_cnt = user_dict['data']['attributes']
>>> print event_cnt[0]['alert_type']
download
>>> print event_cnt[0]['user']
10.1.1.16
>>> print event_cnt[0]['alert_type'] + "," + event_cnt[0]['user']
download,10.1.1.16
>>>
How to get all the elements/values of a particular keys in a CSV format and in a single iteration ?
Output:
download,10.1.1.16
download,10.2.1.18

Simple list comprehension:
>>> jdict=json.loads(resp)
>>> ["{},{}".format(d["alert_type"],d["user"]) for d in jdict["data"]["attributes"]]
['download,10.1.1.16', 'download,10.2.1.18']
Which you can join for your desired output:
>>> li=["{},{}".format(d["alert_type"],d["user"]) for d in jdict["data"]["attributes"]]
>>> print '\n'.join(li)
download,10.1.1.16
download,10.2.1.18

Since {"data":{"attributes": is a list, you can loop over it and print the values for desired keys (d is the user dict):
for item in d['data']['attributes']:
print(item['alert_type'],',',item['user'], sep='')

You could make it somewhat data-driven like this:
import json
DESIRED_KEYS = 'alert_type', 'user'
resp = '''{ "data": {
"attributes": [
{
"alert_type": "download",
"severity_level": "med",
"user": "10.1.1.16"
},
{
"alert_type": "download",
"severity_level": "low",
"user": "10.2.1.18"
}
]
}
}
'''
user_dict = json.loads(resp)
for attribute in user_dict['data']['attributes']:
print(','.join(attribute[key] for key in DESIRED_KEYS))
To handle attributes that don't have all the keys, you could instead use this as the last line which will assign missing values a default value (such as a blank string as shown) instead of it causing an exception.
print(','.join(attribute.get(key, '') for key in DESIRED_KEYS))

Using jq, a one-line solution is straightforward:
$ jq -r '.data.attributes[] | [.alert_type, .user] | #csv' input.json
"download","10.1.1.16"
"download","10.2.1.18"
If you don't want the strings to be quoted, use join(",") instead of #csv

Behaviour of dictionary.append in python

I am writing some code where my highest aim is, to deliver some information in json format. I want to display servernames and serverurls like this: ( My aim is not the pretty print but I just displayed it like this, so it is more obvious )
{
"server": [
{ "name" : "some_server_name", "url" : "some_server_url" }
{ "name" : "another_server_name", "url" : "another_server_url" }
]
}
I got now two lists, one with the servernames and the other with serverulrs. I iterate over the names and then access to the matching url. After that is done I want to append it to a dictionary. When I got this dictionary. I want to append this to my json_dict. I created that dict because I have two other lists which I want to display too in my json information. ( You can ignore rdp and cert because they are already displayed correctly )
def build_dict(server_name, server_url, rdp, cert):
json_dict = defaultdict(list)
server_dict = {}
if server_name and server_url:
for i, iter_name in enumerate(server_name):
server_dict.update( name = iter_name, url = server_url[i] )
json_dict['server'].append(server_dict)
if rdp:
for iter_rdp in rdp:
json_dict['config'].append(iter_rdp)
if cert:
for iter_cert in cert:
json_dict['cert'].append(iter_cert)
return json_dict
The entries of my servernames list:
[u'123name', u'qwer123']
And the entries of my serverurls list:
[u'123url', u'qwer123']
Unfortunately the final output looks like this:
{"cert": ["123url", "123123", "123123123"],
"config": ["123rdp", "123123", "123123123"],
"server": [
{"url": "qwer123", "name": "qwer123"},
{"url": "qwer123", "name": "qwer123"}
]
}
You can ignore cert and config because this already works as expected. I just don't understand why I have:
{"url": "qwer123", "name": "qwer123"},
{"url": "qwer123", "name": "qwer123"}
And not:
{"url": "123name", "name": "123url"},
{"url": "qwer123", "name": "qwer123"}
as output.

As jonrsharpe stated, you have your server_dict at the wrong place. You should consider putting it in your for loop, so you just update the two entries which you recently received through your for loop.
if server_name and server_url:
for i, iter_name in enumerate(server_name):
server_dict = dict( name = iter_name, url = server_url[i] )
json_dict['server'].append(server_dict)