I want to create a json file that can be used as a configuration file. I have different files from multiple companies that report the same information with different column names.
I want to use the information provided in the json file to run a python script to consolidate all the information from all files and companies in one master file.
The structure looks like follows:
{"companies":
{"company1": [
{"path": "C:/USER/Path/Company1/",
"files": [
{
{"_CO": {"ID": "ID", "Report Number": "Report_Number"}},
{"_TR": {"ID": "Trade_Ident", "Report Number": "Number of Report"}},
},
],
},
],
},
{"company2": [
{"path": "C:/USER/Path/Company2/",
"files": [
{
{"_CO": {"ID": "Identification", "Report Number": "Report-Number"}},
{"_TR": {"ID": "Ident", "Report Number": "NumberReport"}},
},
],
},
],
},
},
However, I receive the following error when trying to read the .json in python.
json.decoder.JSONDecodeError: Expecting property name enclosed in
double quotes: line 6 column 5 (char 90)
To read the file I use:
import json
path = "/user_folder/USER/Desktop/Data/"
file = "ConfigFile.json"
with open(path+file) as f:
my_test = json.load(f)
Any help appreciated, as I can't figure out my mistake in the file structure.
You're getting error because your json file is incorrectly formatted and thus calling json.load() will raise an JSONDecodeError.
Your json structure should look like,
{
"companies": {
"company1": [
{
"path": "C:/USER/Path/Company1/",
"files": [
{
"_CO": {
"ID": "ID",
"Report Number": "Report_Number"
}
},
{
"_TR": {
"ID": "Trade_Ident",
"Report Number": "Number of Report"
}
}
]
}
],
"company2": [
{
"path": "C:/USER/Path/Company2/",
"files": [
{
"_CO": {
"ID": "Identification",
"Report Number": "Report-Number"
}
},
{
"_TR": {
"ID": "Ident",
"Report Number": "NumberReport"
}
}
]
}
]
}
}
Hope it helps you!
You have some object (the ones with curly braces) without keys, for example in
{
{"_CO": {"ID": "ID", "Report Number": "Report_Number"}}, ...
Objects in JSON are key-value pairs. Just remove the external set of braces and it should be ok.
You can use some online JSON formatter/validator just like this one, and it will easily point out the problem. Otherwise, you can use some JSON linter for your editor. It just does the work for you and also improves indentation :)
Related
Trying to append to a nested json file
My goal is to append some values to a JSON file.
Here is my original JSON file
{
"web": {
"all": {
"side": {
"tags": [
"admin"
],
"summary": "Generates",
"operationId": "Key",
"consumes": [],
"produces": [
"application/json"
],
"responses": {
"200": {
"description": "YES",
"schema": {
"type": "string"
}
}
},
"Honor": [
{
"presidential": []
}
]
}
}
}
}
It is my intention to add two additional lines inside the key "Honor", with the values "Required" : "YES" and "Prepay" : "NO". As a result of appending the two values, I will have the following JSON file.
{
"web": {
"all": {
"side": {
"tags": [
"admin"
],
"summary": "Generates",
"operationId": "Key",
"consumes": [],
"produces": [
"application/json"
],
"responses": {
"200": {
"description": "YES",
"schema": {
"type": "string"
}
}
},
"Honor": [
{
"presidential": [],
"Required" : "YES",
"Prepay" : "NO"
}
]
}
}
}
}
Below is the Python code that I have written
import json
def write_json(data,filename ="exmpleData.json"):
with open(filename,"w") as f:
json.dump(data,f,indent=2)
with open ("exmpleData.json") as json_files:
data= json.load(json_files)
temp = data["Honor"]
y = {"required": "YES","type": "String"}
temp.append(y)
write_json(data)
I am receiving the following error message:
** temp = data["Honor"] KeyError: 'Honor'
**
I would appreciate any guidance that you can provide to help me achieve my goal. I am running Python 3.7
'Honor' is deeply nested in other dictionaries, and its value is a 1-element list containing a dictionary. Here's how to access:
import json
def write_json(data, filename='exmpleData.json'):
with open(filename, 'w') as f:
json.dump(data, f, indent=2)
with open('exmpleData.json') as json_files:
data = json.load(json_files)
# 'Honor' is deeply nested in other dictionaries
honor = data['web']['all']['side']['Honor']
# Its value is a 1-element list containing another dictionary.
honor[0]['Required'] = 'YES'
honor[0]['Prepay'] = 'NO'
write_json(data)
I'd recommend that you practice your fundamentals a bit more since you're making many mistakes in your data structure handling. The good news is, your JSON load/dump is fine.
The cause of your error message is that data doesn't have an "Honor" property. Data only has a "web" property, which contains "all" which contains "side" which contains "Honor", which contains an array with a dictionary that holds the properties you are trying to add to. So you want to set temp with temp = data['web']['all']['side']['Honor'][0]
You also cannot use append on python dictionaries. Instead, check out dict.update().
I want to create the JSON file from CSV file using the generic python script.
Found hone package from GitHub but some of the functionalities missing in that code.
csv to json
I want to code like generic template CSV to JSON.
[
{
"birth": {
"day": "7",
"month": "May",
"year": "1985"
},
"name": "Bob",
"reference": "TRUE",
"reference name": "Smith"
}
]
Only handled above type of JSON only.
[
{
"Type": "AwsEc2Instance",
"Id": "i-cafebabe",
"Partition": "aws",
"Region": "us-west-2",
"Tags": {
"billingCode": "Lotus-1-2-3",
"needsPatching": "true"
},
"Details": {
"AwsEc2Instance": {
"Type": "i3.xlarge",
"ImageId": "ami-abcd1234",
"IpV4Addresses": [ "54.194.252.215", "192.168.1.88" ],
"IpV6Addresses": [ "2001:db812341a2b::123" ],
"KeyName": "my_keypair",
"VpcId": "vpc-11112222",
"SubnetId": "subnet-56f5f633",
"LaunchedAt": "2018-05-08T16:46:19.000Z"
}
}
}
]
I want to handle nested array[] ,{}
I have done something like this before and below code can be modified as I have not seen your dataset.
dataframe = pd.read_excel('dataframefilepath', encoding='utf-8', header=0)
'''Adding to list to finally save it as JSON'''
df = []
for (columnName, columnData) in dataframe.iteritems():
if dataframe.columns.get_loc(columnName) > 0:
for indata, rwdata in dataframe.iterrows():
for insav, rwsave in df_to_Save.iterrows():
if rwdata.Selected_Prediction == rwsave.Selected_Prediction:
#print()
df_to_Save.loc[insav, 'Value_to_Save'] = rwdata[dataframe.columns.get_loc(columnName)]
#print(rwdata[dataframe.columns.get_loc(columnName)])
df.append(df_to_Save.set_index('Selected_Prediction').T.to_dict('record'))
df = eval(df)
'''Saving in JSON format'''
path_to_save = '\\your path'
with open(path_to_save, 'w') as json_file:
json.dump(df, json_file)
i need parse terraform file, write in JSON format. I have to extract two data, resource and id, this is example file:
{
"version": 1,
"serial": 1,
"modules": [
{
"path": [
"root"
],
"outputs": {
},
"resources": {
"aws_security_group.vpc-xxxxxxx-test-1": {
"type": "aws_security_group",
"primary": {
"id": "sg-xxxxxxxxxxxxxx",
"attributes": {
"description": "test-1",
"name": "test-1"
}
}
},
"aws_security_group.vpc-xxxxxxx-test-2": {
"type": "aws_security_group",
"primary": {
"id": "sg-yyyyyyyyyyyy",
"attributes": {
"description": "test-2",
"name": "test-2"
}
}
}
}
}
]
}
I need export for any resources, the first key and value of id, in this case, aws_security_group.vpc-xxxxxxx-test-1 sg-xxxxxxxxxxxxxx and aws_security_group.vpc-xxxxxxx-test-2 sg-yyyyyyyyyyyy
I have tried to write this in python:
#!/usr/bin/python3.6
import json
import objectpath
with open('file.json') as json_file:
data = json.load(json_file)
json_tree = objectpath.Tree(data['modules'])
result = tuple(json_tree.execute('$..resources[0]'))
result is
('aws_security_group.vpc-xxxxxxx-test-1', 'aws_security_group.vpc-xxxxxxx-test-2')
It's'ok but I can't extract the id, any help is appreciated, also use other methods
Thanks
I don't know objectpath, but I think you need:
tree.execute('$..resources[0]..primary.id')
or even just
tree.execute('$..resources[0]..id')
I want to generate below json file using python.
{
"type": "SubInterface",
"MTU": 1500,
"enabled": "True",
"vlanId": vlanid,
"subIntfId": subinterid,
"name": "abc",
"id": "intid",
"managementOnly": "False",
"activeMACAddress":"active-mac",
"standbyMACAddress":"standby-mac",
"securityZone": {
"name": "Zonename",
"id": "securityid",
"type": "SecurityZone"
},
"ifname": "interface-name",
"ipv4": {
"static": {
"address": ipv4address,
"netmask": "ipv4subnet"
}
},
"ipv6": {
"enableIPV6": "True",
"addresses": [
{
"address": ipv6address,
"prefix": "ipv6prefix",
"enforceEUI64": "False"
}
]
}
}
here is my code-
import json
data={}
data["type"]="Subinterface"
data["MTU"]="1500"
data["enabled"]= "True"
data["vlanId"]="vlanid"
data["subIntfId"]="subinterid"
data["name"]= "Port-channel24"
data["id"]= "intid"
data["managementOnly"]= "False"
data["activeMACAddress"]="active-mac"
data["standbyMACAddress"]="standby-mac"
data['securityZone']=[]
data['securityZone'].append({
"name": "Zonename",
"id": "securityid",
"type": "SecurityZone"
})
data["ifname"]="interface-name"
data['ipv4']=[]
data['ipv4'].append({
data["static"]=[]
data["static"].append({
"address": "ipv4address",
"netmask": "ipv4subnet"
})
})
with open('data1.txt', 'w') as outfile:
json.dump(data, outfile)
while executing it gives me below error -
automation) -bash-4.1$ python json23.py
File "json23.py", line 23
data["static"]=[]
^
SyntaxError: invalid syntax
How to generate json with nested values
It is because you have unclosed brackets on the previous line: data['ipv4'].append({. If you close up those brackets you should be all good.
The problem is with this block:
data['ipv4']=[]
data['ipv4'].append({
data["static"]=[]
data["static"].append({
"address": "ipv4address",
"netmask": "ipv4subnet"
})
})
As a commented pointed out, your desired output does not have lists for ipv4 and static. I suspect you want it to read:
data['ipv4'] = {
"static": {
"address": "ipv4address",
"netmask": "ipv4subnet"
}
}
Or if you insist on square bracket notation (though I'm not sure why you would):
data['ipv4'] = {}
data['ipv4']['static'] = {}
data['ipv4']['static']['address'] = 'ipv4address'
data['ipv4']['static']['netmask'] = 'ipv4subnet'
Some other notes:
You don't need to initialise an empty list and then append to it - you can just initialise the list with the values that you want.
I'd suggest that initialising this dict directly, instead of dynamically building it like here, is preferable where possible.
I've a nested json structure, I'm using objectpath (python API version), but I don't understand how to select and filter some information (more precisely the nested information in the structure).
EG.
I want to select the "description" of the action "reading" for the user "John".
JSON:
{
"user":
{
"actions":
[
{
"name": "reading",
"description": "blablabla"
}
]
"name": "John"
}
}
CODE:
$.user[#.name is 'John' and #.actions.name is 'reading'].actions.description
but it doesn't work (empty set but in my JSON it isn't so).
Any suggestion?
Is this what you are trying to do?
import objectpath
data = {
"user": {
"actions": {
"name": "reading",
"description": "blablabla"
},
"name": "John"
}
}
tree = objectpath.Tree(data)
result = tree.execute("$.user[#.name is 'John'].actions[#.name is 'reading'].description")
for entry in result:
print entry
Output
blablabla
I had to fix your JSON. Also, tree.execute returns a generator. You could replace the for loop with print result.next(), but the for loop seemed more clear.
import objectpath import *
your_json = {"name": "felix", "last_name": "diaz"}
# This json path will bring all the key-values of your json
your_json_path='$.*'
my_key_values = Tree(your_json).execute(your_json_path)
# If you want to retrieve the name node...then specify it.
my_name= Tree(your_json).execute('$.name')
# If you want to retrieve a the last_name node...then specify it.
last_name= Tree(your_json).execute('$.last_name')
I believe you're just missing a comma in JSON:
{
"user":
{
"actions": [
{
"name": "reading",
"description": "blablabla"
}
],
"name": "John"
}
}
Assuming there is only one "John", with only one "reading" activity, the following query works:
$.user[#.name is 'John'].actions[0][#.name is 'reading'][0].description
If there could be multiple "John"s, with multiple "reading" activities, the following query will almost work:
$.user.*[#.name is 'John'].actions..*[#.name is 'reading'].description
I say almost because the use of .. will be problematic if there are other nested dictionaries with "name" and "description" entries, such as
{
"user": {
"actions": [
{
"name": "reading",
"description": "blablabla",
"nested": {
"name": "reading",
"description": "broken"
}
}
],
"name": "John"
}
}
To get a correct query, there is an open issue to correctly implement queries into arrays: https://github.com/adriank/ObjectPath/issues/60