Trying to append to a nested json file
My goal is to append some values to a JSON file.
Here is my original JSON file
{
"web": {
"all": {
"side": {
"tags": [
"admin"
],
"summary": "Generates",
"operationId": "Key",
"consumes": [],
"produces": [
"application/json"
],
"responses": {
"200": {
"description": "YES",
"schema": {
"type": "string"
}
}
},
"Honor": [
{
"presidential": []
}
]
}
}
}
}
It is my intention to add two additional lines inside the key "Honor", with the values "Required" : "YES" and "Prepay" : "NO". As a result of appending the two values, I will have the following JSON file.
{
"web": {
"all": {
"side": {
"tags": [
"admin"
],
"summary": "Generates",
"operationId": "Key",
"consumes": [],
"produces": [
"application/json"
],
"responses": {
"200": {
"description": "YES",
"schema": {
"type": "string"
}
}
},
"Honor": [
{
"presidential": [],
"Required" : "YES",
"Prepay" : "NO"
}
]
}
}
}
}
Below is the Python code that I have written
import json
def write_json(data,filename ="exmpleData.json"):
with open(filename,"w") as f:
json.dump(data,f,indent=2)
with open ("exmpleData.json") as json_files:
data= json.load(json_files)
temp = data["Honor"]
y = {"required": "YES","type": "String"}
temp.append(y)
write_json(data)
I am receiving the following error message:
** temp = data["Honor"] KeyError: 'Honor'
**
I would appreciate any guidance that you can provide to help me achieve my goal. I am running Python 3.7
'Honor' is deeply nested in other dictionaries, and its value is a 1-element list containing a dictionary. Here's how to access:
import json
def write_json(data, filename='exmpleData.json'):
with open(filename, 'w') as f:
json.dump(data, f, indent=2)
with open('exmpleData.json') as json_files:
data = json.load(json_files)
# 'Honor' is deeply nested in other dictionaries
honor = data['web']['all']['side']['Honor']
# Its value is a 1-element list containing another dictionary.
honor[0]['Required'] = 'YES'
honor[0]['Prepay'] = 'NO'
write_json(data)
I'd recommend that you practice your fundamentals a bit more since you're making many mistakes in your data structure handling. The good news is, your JSON load/dump is fine.
The cause of your error message is that data doesn't have an "Honor" property. Data only has a "web" property, which contains "all" which contains "side" which contains "Honor", which contains an array with a dictionary that holds the properties you are trying to add to. So you want to set temp with temp = data['web']['all']['side']['Honor'][0]
You also cannot use append on python dictionaries. Instead, check out dict.update().
Related
I am trying to populate a pandas DataFrame with select information from JSON output fetched from an API.
candidate_list = []
for candidate in candidate_response['data']:
if 'error' not in candidate_response:
candidate_list.append([candidate['id'], candidate['attributes']['first_name'], candidate['attributes']
['last_name'], candidate['relationships']['educations']['data']['id']])
The DataFrame populates fine until I add candidate['relationships']['educations']['data']['id'], which throws TypeError: list indices must be integers or slices, not str.
When trying to get the values of the indexes for ['id'] by using candidate['relationships']['educations']['data'][0]['id'] instead, I get IndexError: list index out of range.
The JSON output looks something like:
"data": [
{
"attributes": {
"first_name": "Tester",
"last_name": "Testman",
"other stuff": "stuff",
},
"id": "732887",
"relationships": {
"educations": {
"data": [
{
"id": "605372",
"type": "educations"
},
{
"id": "605371",
"type": "educations"
},
{
"id": "605370",
"type": "educations"
}
]
}
},
How would I go about successfully filling a column in the DataFrame with the 'id's under 'relationships'>'educations'>'data'?
Please note then when using candidate['relationships']['educations']['data']['id'] you get that error because at data there is a list, and not a dictionary. And you cannot access dictionary by name.
Assuming, what you are trying to achieve is one entry per data.attributes.relationships.educations.data entry. Complete code that works and does what you are trying is:
import json
json_string = """{
"data": [
{
"attributes": {
"first_name": "Tester",
"last_name": "Testman",
"other stuff": "stuff"
},
"id": "732887",
"relationships": {
"educations": {
"data": [
{
"id": "605372",
"type": "educations"
},
{
"id": "605371",
"type": "educations"
},
{
"id": "605370",
"type": "educations"
}
]
}
}
}
]
}"""
candidate_response = json.loads(json_string)
candidate_list = []
for candidate in candidate_response['data']:
if 'error' not in candidate_response:
for data in candidate['relationships']['educations']['data']:
candidate_list.append(
[
candidate['id'],
candidate['attributes']['first_name'],
candidate['attributes']['last_name'],
data['id']
]
)
print(candidate_list)
Code run available at ideone.
I have analyzed your code and also ran it on Jupyter notebook all looks good, I am getting the output,
The error you got list indices must be integers or slices, not str, that is because you were not using the index, this required because the value which you are looking for that is in the list.
and about this error: IndexError: list index out of range. Maybe some code typo mistake is done from your side otherwise the code is fine.
here is the output of your following code:
candidate_list = []
for candidate in candidate_response['data']:
if 'error' not in candidate_response:
candidate_list.append([candidate['id'], candidate['attributes']['first_name'], candidate['attributes']['last_name'],candidate['relationships']['educations']['data'][0]['id']])
Output
probably for any candidate candidate['relationships']['educations']['data'] is an empty list
I want to create a list of keys from JSON (actually JSON Schema). JSON is complex and nested so I am using the jsonpath-ng library.
I can't manage to get the list of keys. On the other hand, extracting the values is successful with this syntax: print([match.value for match in tagNames]).
Attempt to extract the keys using similar syntax, like this...
from jsonpath_ng import jsonpath, parse
import json
filePath = "./sample.json"
with open(filePath, "r") as jsonFile:
jsonData = json.load(jsonFile)
tagNames = parse("tags[*].*").find(jsonData)
print([match.key for match in tagNames])
returns this error:
AttributeError: 'DatumInContext' object has no attribute 'key'
although I am getting the 'key' suggestion after I type 'match.'
Example of JSON I work with:
{
"tags": [
{
"name": "auth",
"description": "Authentication"
},
{
"name": "b2b",
"description": "B 2b"
}
],
"paths": {
"/rest/auth": {
"post": {
"tags": [
"auth"
],
"operationId": "generateToken",
"consumes": [
"application/json"
],
"produces": [
"*/*"
],
"responses": {
"200": {
"description": "Generated"
},
"201": {
"description": "Created"
}
}
},
"get": {
"tags": [
"status"
],
"operationId": "check Token",
"consumes": [
"application/json"
],
"produces": [
"*/*"
],
"responses": {
"200": {
"description": "Generated"
},
"201": {
"description": "Created"
}
}
}
}
}
}
Task 1 - a simpler task of getting the keys under tags (expected result: name, description).
Task 2 - more complicated case is getting the keys from paths (expected result: post, get).
If jsonpath doesn't support extracting the keys, does anyone have any other suggestion for nested JSONs?
I also tried to convert the match result to plain python dict in order to get the keys with .keys(), but without success.
If I understand you correctly, you may be looking for something like this:
from jsonpath_ng import parse
import json
my_str = """[your json string above]"""
data = json.loads(my_str)
First expression:
sonpath_expr = parse('$.tags')
for d in jsonpath_expr.find(data):
for k in d.value[0].keys():
print(k)
Output:
name
description
Second expression:
jsonpath_expr = parse('$.paths..*..*')
for k in jsonpath_expr.find(data)[0].context.value.keys():
print(k)
Output:
post
get
I have a csv file and trying to compose JSON from it. There are mulitple records in a file but I am just giving one set of sample records here.This structure is driven on the claimID. There is nesting on the claimLineDetail and claimSpecDiag.I guess I have to create some sort of list to handle this then the problem is how am I going to append it in the required structure. I really need some guidance here to achieve the desired result. Is it possible to break out different sections and append it later, I am not sure just assuming, as there are multiple columns.
Code :
import csv,json
data = []
with open('JsonRequestPricingMedical.csv','r') as f:
reader = csv.DictReader(f)
for row in reader:
print row
csv file :
claimId,subscriberId,claimType,claimSubType,providerId,totalChargeAmt,claimLineNo,pos_code,procedureCode,subdiagnosisCode,svcLineFromDt,svcLineToDt,chargedAmt,clmLineUnits,presentOnAdmit,diagnosisCode
18A000730400,101924200,M,M,002664514003,585,1,11,92014,H43393,2017-06-19,2017-06-19,160,1,U,H43393
18A000730400,101924200,M,M,002664514003,585,2,12,92015,H43395,2017-06-19,2017-06-19,160,2,U,H43394
Desired JSON
[
{
"claimsHeader":" {
"claimId": "18A000730400",
"subscriberId": "101924200",
"claimType":{
"code": "M"
},
"claimSubType": {
"code": "M"
},
"providerId" :"002664514003",
"totalChargeAmt": "585",
"claimLineDetail" :[
{
"claimLineNo": "1",
"placeOfService": {
"code": "11"
},
"procedureCode": {
"code": "92014"
},
"subDiagnosisCd": {
"code": "H43393"
},
"svcLineFromDt": "2017-06-19",
"svcLineToDt": "2017-06-19",
"chargedAmt": "160",
"clmLineUnits": "1",
},
{
"claimLineNo": "2",
"placeOfService": {
"code": "12"
},
"procedureCode": {
"code": "92015"
},
"subDiagnosisCd": {
"code": "H433945
},
"svcLineFromDt": "2017-06-19",
"svcLineToDt": "2017-06-19",
"chargedAmt": "160",
"clmLineUnits": "2",
}
],
{
"claimSpecDiag": [
"presentOnAdmit": "",
"diagnosisCode": "H43393",
},
{
"presentOnAdmit": "",
"diagnosisCode": "H43394",
}
]
}
]
When you read a csv, each line represents variables separated by a special char, in your case, comas: ",".
You can get each variable separated by doing line_variables = row.split(',')
Just pass the first line, and for all the other, do something like:
result = {
"claimsHeader":" {
"claimId": line_variables[0],
"subscriberId": line_variables[1],
"claimType":{
"code": line_variables[2]
}
...
Finaly, just add the result to a list (created just before your for loop) with your_list.append(result).
I want to generate below json file using python.
{
"type": "SubInterface",
"MTU": 1500,
"enabled": "True",
"vlanId": vlanid,
"subIntfId": subinterid,
"name": "abc",
"id": "intid",
"managementOnly": "False",
"activeMACAddress":"active-mac",
"standbyMACAddress":"standby-mac",
"securityZone": {
"name": "Zonename",
"id": "securityid",
"type": "SecurityZone"
},
"ifname": "interface-name",
"ipv4": {
"static": {
"address": ipv4address,
"netmask": "ipv4subnet"
}
},
"ipv6": {
"enableIPV6": "True",
"addresses": [
{
"address": ipv6address,
"prefix": "ipv6prefix",
"enforceEUI64": "False"
}
]
}
}
here is my code-
import json
data={}
data["type"]="Subinterface"
data["MTU"]="1500"
data["enabled"]= "True"
data["vlanId"]="vlanid"
data["subIntfId"]="subinterid"
data["name"]= "Port-channel24"
data["id"]= "intid"
data["managementOnly"]= "False"
data["activeMACAddress"]="active-mac"
data["standbyMACAddress"]="standby-mac"
data['securityZone']=[]
data['securityZone'].append({
"name": "Zonename",
"id": "securityid",
"type": "SecurityZone"
})
data["ifname"]="interface-name"
data['ipv4']=[]
data['ipv4'].append({
data["static"]=[]
data["static"].append({
"address": "ipv4address",
"netmask": "ipv4subnet"
})
})
with open('data1.txt', 'w') as outfile:
json.dump(data, outfile)
while executing it gives me below error -
automation) -bash-4.1$ python json23.py
File "json23.py", line 23
data["static"]=[]
^
SyntaxError: invalid syntax
How to generate json with nested values
It is because you have unclosed brackets on the previous line: data['ipv4'].append({. If you close up those brackets you should be all good.
The problem is with this block:
data['ipv4']=[]
data['ipv4'].append({
data["static"]=[]
data["static"].append({
"address": "ipv4address",
"netmask": "ipv4subnet"
})
})
As a commented pointed out, your desired output does not have lists for ipv4 and static. I suspect you want it to read:
data['ipv4'] = {
"static": {
"address": "ipv4address",
"netmask": "ipv4subnet"
}
}
Or if you insist on square bracket notation (though I'm not sure why you would):
data['ipv4'] = {}
data['ipv4']['static'] = {}
data['ipv4']['static']['address'] = 'ipv4address'
data['ipv4']['static']['netmask'] = 'ipv4subnet'
Some other notes:
You don't need to initialise an empty list and then append to it - you can just initialise the list with the values that you want.
I'd suggest that initialising this dict directly, instead of dynamically building it like here, is preferable where possible.
I've a nested json structure, I'm using objectpath (python API version), but I don't understand how to select and filter some information (more precisely the nested information in the structure).
EG.
I want to select the "description" of the action "reading" for the user "John".
JSON:
{
"user":
{
"actions":
[
{
"name": "reading",
"description": "blablabla"
}
]
"name": "John"
}
}
CODE:
$.user[#.name is 'John' and #.actions.name is 'reading'].actions.description
but it doesn't work (empty set but in my JSON it isn't so).
Any suggestion?
Is this what you are trying to do?
import objectpath
data = {
"user": {
"actions": {
"name": "reading",
"description": "blablabla"
},
"name": "John"
}
}
tree = objectpath.Tree(data)
result = tree.execute("$.user[#.name is 'John'].actions[#.name is 'reading'].description")
for entry in result:
print entry
Output
blablabla
I had to fix your JSON. Also, tree.execute returns a generator. You could replace the for loop with print result.next(), but the for loop seemed more clear.
import objectpath import *
your_json = {"name": "felix", "last_name": "diaz"}
# This json path will bring all the key-values of your json
your_json_path='$.*'
my_key_values = Tree(your_json).execute(your_json_path)
# If you want to retrieve the name node...then specify it.
my_name= Tree(your_json).execute('$.name')
# If you want to retrieve a the last_name node...then specify it.
last_name= Tree(your_json).execute('$.last_name')
I believe you're just missing a comma in JSON:
{
"user":
{
"actions": [
{
"name": "reading",
"description": "blablabla"
}
],
"name": "John"
}
}
Assuming there is only one "John", with only one "reading" activity, the following query works:
$.user[#.name is 'John'].actions[0][#.name is 'reading'][0].description
If there could be multiple "John"s, with multiple "reading" activities, the following query will almost work:
$.user.*[#.name is 'John'].actions..*[#.name is 'reading'].description
I say almost because the use of .. will be problematic if there are other nested dictionaries with "name" and "description" entries, such as
{
"user": {
"actions": [
{
"name": "reading",
"description": "blablabla",
"nested": {
"name": "reading",
"description": "broken"
}
}
],
"name": "John"
}
}
To get a correct query, there is an open issue to correctly implement queries into arrays: https://github.com/adriank/ObjectPath/issues/60