I have a JSON file which I want to take and put into python objects. It has two parts, staff and assets and I want to load them into two separate ones. Here is a sample of the JSON file:
{
"staff": [
{
"id": "DA7153",
"name": [
"Fran\u00c3\u00a7ois",
"Ullman"
],
"department": {
"name": "Admin"
},
"server_admin": "true"
},
{
"id": "DA7356",
"name": [
"Bob",
"Johnson"
],
"department": {
"name": "Admin"
},
"server_admin": "false"
},
],
"assets": [
{
"asset_name": "ENGAGED SLOTH",
"asset_type": "File",
"owner": "DA8333",
"details": {
"security": {
"cia": [
"HIGH",
"INTERMEDIATE",
"LOW"
],
"data_categories": {
"Personal": "true",
"Personal Sensitive": "true",
"Customer Sensitive": "true"
}
},
"retention": 2
},
"file_type": "Document",
"server": {
"server_name": "ISOLATED UGUISU",
"ip": [
10,
234,
148,
52
]
}
},
{
"asset_name": "ISOLATED VIPER",
"asset_type": "File",
"owner": "DA8262",
"details": {
"security": {
"cia": [
"LOW",
"HIGH",
"LOW"
],
"data_categories": {
"Personal": "false",
"Personal Sensitive": "false",
"Customer Sensitive": "true"
}
},
"retention": 2
},
},
]
I have tried to create a class for staff but whenever I do I get the error "TypeError: dict expected at most 1 argument, got 3"
The code I am using looks like this:
import json
with open('Admin_sample.json') as f:
admin_json = json.load(f)
class staffmem(admin_json):
def __init__(self, id, name, department, server_admin):
self.id = id
self.name = name
self.deparment = department[name]
self.server_admin = server_admin
def staffid(self):
return self.id
print(staffmem.staffid)
I just can't work it out. Any help would be appreciated.
Thanks.
The following should be a good starting point but you have to fix few things. Note that I am using get() everywhere to provide a "safe" default if the keys do not exist:
import json
class StaffMember:
def __init__(self, json_entry):
self.name = ",".join(json_entry.get("name"))
self.id = json_entry.get("id")
self.dept = json_entry.get("department", {}).get("name")
self.server_admin = (
True
if json_entry.get("server_admin", "false").lower() == "true"
else False
)
# Get the data
with open("/tmp/test.data") as f:
data = json.load(f)
# For every entry in the data["staff"] create object and index them by ID
all_staff = {}
for json_entry in data.get("staff", []):
tmp = StaffMember(json_entry)
all_staff[tmp.id] = tmp
print(all_staff)
print(all_staff['DA7153'].name)
Output:
$ python3 /tmp/test.py
{'DA7153': <__main__.StaffMember object at 0x1097b2d50>, 'DA7356': <__main__.StaffMember object at 0x1097b2d90>}
François,Ullman
Potential Improvements:
Unicode handling
Add getters/setters
Instead of passing json dict in ctor, consider adding a from_json() static method to create your object
Error handling on missing values
Consider using a dataclass in py3 if this object is used to only/mainly store data
Consider the namedtuple approach from the comments if you do not intend to modify the object (read-only)
Notes:
The json you provided is not correct - you will need to fix it
Your syntax is wrong in your example and the naming convention is not much pythonic (read more here
Related
Trying to append to a nested json file
My goal is to append some values to a JSON file.
Here is my original JSON file
{
"web": {
"all": {
"side": {
"tags": [
"admin"
],
"summary": "Generates",
"operationId": "Key",
"consumes": [],
"produces": [
"application/json"
],
"responses": {
"200": {
"description": "YES",
"schema": {
"type": "string"
}
}
},
"Honor": [
{
"presidential": []
}
]
}
}
}
}
It is my intention to add two additional lines inside the key "Honor", with the values "Required" : "YES" and "Prepay" : "NO". As a result of appending the two values, I will have the following JSON file.
{
"web": {
"all": {
"side": {
"tags": [
"admin"
],
"summary": "Generates",
"operationId": "Key",
"consumes": [],
"produces": [
"application/json"
],
"responses": {
"200": {
"description": "YES",
"schema": {
"type": "string"
}
}
},
"Honor": [
{
"presidential": [],
"Required" : "YES",
"Prepay" : "NO"
}
]
}
}
}
}
Below is the Python code that I have written
import json
def write_json(data,filename ="exmpleData.json"):
with open(filename,"w") as f:
json.dump(data,f,indent=2)
with open ("exmpleData.json") as json_files:
data= json.load(json_files)
temp = data["Honor"]
y = {"required": "YES","type": "String"}
temp.append(y)
write_json(data)
I am receiving the following error message:
** temp = data["Honor"] KeyError: 'Honor'
**
I would appreciate any guidance that you can provide to help me achieve my goal. I am running Python 3.7
'Honor' is deeply nested in other dictionaries, and its value is a 1-element list containing a dictionary. Here's how to access:
import json
def write_json(data, filename='exmpleData.json'):
with open(filename, 'w') as f:
json.dump(data, f, indent=2)
with open('exmpleData.json') as json_files:
data = json.load(json_files)
# 'Honor' is deeply nested in other dictionaries
honor = data['web']['all']['side']['Honor']
# Its value is a 1-element list containing another dictionary.
honor[0]['Required'] = 'YES'
honor[0]['Prepay'] = 'NO'
write_json(data)
I'd recommend that you practice your fundamentals a bit more since you're making many mistakes in your data structure handling. The good news is, your JSON load/dump is fine.
The cause of your error message is that data doesn't have an "Honor" property. Data only has a "web" property, which contains "all" which contains "side" which contains "Honor", which contains an array with a dictionary that holds the properties you are trying to add to. So you want to set temp with temp = data['web']['all']['side']['Honor'][0]
You also cannot use append on python dictionaries. Instead, check out dict.update().
I want to print a user from a JSON list into Python that I select however I can only print all the users. How do you print a specific user? At the moment I have this which prints all the users out in a ugly format
import json
with open('Admin_sample.json') as f:
admin_json = json.load(f)
print(admin_json['staff'])
The JSON file looks like this
{
"staff": [
{
"id": "DA7153",
"name": [
"Fran\u00c3\u00a7ois",
"Ullman"
],
"department": {
"name": "Admin"
},
"server_admin": "true"
},
{
"id": "DA7356",
"name": [
"Bob",
"Johnson"
],
"department": {
"name": "Admin"
},
"server_admin": "false"
},
],
"assets": [
{
"asset_name": "ENGAGED SLOTH",
"asset_type": "File",
"owner": "DA8333",
"details": {
"security": {
"cia": [
"HIGH",
"INTERMEDIATE",
"LOW"
],
"data_categories": {
"Personal": "true",
"Personal Sensitive": "true",
"Customer Sensitive": "true"
}
},
"retention": 2
},
"file_type": "Document",
"server": {
"server_name": "ISOLATED UGUISU",
"ip": [
10,
234,
148,
52
]
}
},
{
"asset_name": "ISOLATED VIPER",
"asset_type": "File",
"owner": "DA8262",
"details": {
"security": {
"cia": [
"LOW",
"HIGH",
"LOW"
],
"data_categories": {
"Personal": "false",
"Personal Sensitive": "false",
"Customer Sensitive": "true"
}
},
"retention": 2
},
},
]
I just can't work it out. Any help would be appreciated.
Thanks.
You need to index into the staff list, e.g.:
print(admin_json['staff'][0])
I suggest reading up a bit on dictionaries in Python. Dictionary values can be set to any object: in this case, the value of the staff key is set to a list of dicts. Here's an example that will loop through all the staff members and print their names:
staff_list = admin_json['staff']
for person in staff_list:
name_parts = person['name']
full_name = ' '.join(name_parts) # combine name parts into a string
print(full_name)
Try something like this:
import json
def findStaffWithId(allStaff, id):
for staff in allStaff:
if staff["id"] == id:
return staff
return {} # no staff found
with open('Admin_sample.json') as f:
admin_json = json.load(f)
print(findStaffWithId(admin_json['staff'], "DA7356"))
You can list all the users name with
users = [user["name"] for user in admin_json['staff']]
You have two lists in this JSON file. When you try to parse it, you'll be reach a list. For example getting the first staff id:
print(admin_json['staff'][0]['id'])
This will print:
DA7153
When you use "json.loads" this will simply converts JSON file to the Python dictionary. For further info:
https://docs.python.org/3/tutorial/datastructures.html#dictionaries
I want to create a json file that can be used as a configuration file. I have different files from multiple companies that report the same information with different column names.
I want to use the information provided in the json file to run a python script to consolidate all the information from all files and companies in one master file.
The structure looks like follows:
{"companies":
{"company1": [
{"path": "C:/USER/Path/Company1/",
"files": [
{
{"_CO": {"ID": "ID", "Report Number": "Report_Number"}},
{"_TR": {"ID": "Trade_Ident", "Report Number": "Number of Report"}},
},
],
},
],
},
{"company2": [
{"path": "C:/USER/Path/Company2/",
"files": [
{
{"_CO": {"ID": "Identification", "Report Number": "Report-Number"}},
{"_TR": {"ID": "Ident", "Report Number": "NumberReport"}},
},
],
},
],
},
},
However, I receive the following error when trying to read the .json in python.
json.decoder.JSONDecodeError: Expecting property name enclosed in
double quotes: line 6 column 5 (char 90)
To read the file I use:
import json
path = "/user_folder/USER/Desktop/Data/"
file = "ConfigFile.json"
with open(path+file) as f:
my_test = json.load(f)
Any help appreciated, as I can't figure out my mistake in the file structure.
You're getting error because your json file is incorrectly formatted and thus calling json.load() will raise an JSONDecodeError.
Your json structure should look like,
{
"companies": {
"company1": [
{
"path": "C:/USER/Path/Company1/",
"files": [
{
"_CO": {
"ID": "ID",
"Report Number": "Report_Number"
}
},
{
"_TR": {
"ID": "Trade_Ident",
"Report Number": "Number of Report"
}
}
]
}
],
"company2": [
{
"path": "C:/USER/Path/Company2/",
"files": [
{
"_CO": {
"ID": "Identification",
"Report Number": "Report-Number"
}
},
{
"_TR": {
"ID": "Ident",
"Report Number": "NumberReport"
}
}
]
}
]
}
}
Hope it helps you!
You have some object (the ones with curly braces) without keys, for example in
{
{"_CO": {"ID": "ID", "Report Number": "Report_Number"}}, ...
Objects in JSON are key-value pairs. Just remove the external set of braces and it should be ok.
You can use some online JSON formatter/validator just like this one, and it will easily point out the problem. Otherwise, you can use some JSON linter for your editor. It just does the work for you and also improves indentation :)
Hello I am completely new to flask and python. I am using an API to geocode
and i get a json which is
"info": {
"copyright": {
"imageAltText": "\u00a9 2015 MapQuest, Inc.",
"imageUrl": "http://api.mqcdn.com/res/mqlogo.gif",
"text": "\u00a9 2015 MapQuest, Inc."
},
"messages": [],
"statuscode": 0
},
"options": {
"ignoreLatLngInput": false,
"maxResults": -1,
"thumbMaps": true
},
"results": [
{
"locations": [
{
"adminArea1": "US",
"adminArea1Type": "Country",
"adminArea3": "",
"adminArea3Type": "",
"adminArea4": "",
"adminArea4Type": "County",
"adminArea5": "",
"adminArea5Type": "City",
"adminArea6": "",
"adminArea6Type": "Neighborhood",
"displayLatLng": {
"lat": 33.663512,
"lng": -111.958849
},
"dragPoint": false,
"geocodeQuality": "ADDRESS",
"geocodeQualityCode": "L1AAA",
"latLng": {
"lat": 33.663512,
"lng": -111.958849
},
"linkId": "25438895i35930428r65831359",
"mapUrl": "http://www.mapquestapi.com/staticmap/v4/getmap?key=&rand=1009123942",
"postalCode": "",
"sideOfStreet": "R",
"street": "",
"type": "s",
"unknownInput": ""
}
],
"providedLocation": {
"city": " ",
"postalCode": "",
"state": "",
"street": "E Blvd"
}
}
]
}
RIght now i am doing this
data=json.loads(r)
return jsonify(data)
and this prints all the data as shown above. I need to get the latlng array from locations which is in results. I have tried
data.get("results").get("locations") and hundreds of combinations like that but i still cant get it to work. I basically need to store the lat and long in a session variable. Any help is appreciated
Assuming you just have one location as in your example:
from __future__ import print_function
import json
r = ...
data = json.loads(r)
latlng = data['results'][0]['locations'][0]['latLng']
latitude = latlng['lat']
longitude = latlng['lng']
print(latitude, longitude) # 33.663512 -111.958849
data.get("results") will return a list type object. As list object does not have get attribute, you can not do data.get("results").get("locations")
According to the json you provided, you can do like this:
data.get('results')[0].get('locations') # also a list
This will give you the array. Now you can get the lat and lng like this:
data.get('results')[0].get('locations')[0].get('latLng').get('lat') # lat
data.get('results')[0].get('locations')[0].get('latLng').get('lng') # lng
I summarize my comments as follows:
You can use data as a dict of dict and list.
A quick ref to dict and list:
A dictionary’s keys are almost arbitrary values.
get(key[, default])
Return the value for key if key is in the dictionary, else default. If default is not given, it defaults to None, so that this method never raises a KeyError.
official docs about stdtypes
I've a nested json structure, I'm using objectpath (python API version), but I don't understand how to select and filter some information (more precisely the nested information in the structure).
EG.
I want to select the "description" of the action "reading" for the user "John".
JSON:
{
"user":
{
"actions":
[
{
"name": "reading",
"description": "blablabla"
}
]
"name": "John"
}
}
CODE:
$.user[#.name is 'John' and #.actions.name is 'reading'].actions.description
but it doesn't work (empty set but in my JSON it isn't so).
Any suggestion?
Is this what you are trying to do?
import objectpath
data = {
"user": {
"actions": {
"name": "reading",
"description": "blablabla"
},
"name": "John"
}
}
tree = objectpath.Tree(data)
result = tree.execute("$.user[#.name is 'John'].actions[#.name is 'reading'].description")
for entry in result:
print entry
Output
blablabla
I had to fix your JSON. Also, tree.execute returns a generator. You could replace the for loop with print result.next(), but the for loop seemed more clear.
import objectpath import *
your_json = {"name": "felix", "last_name": "diaz"}
# This json path will bring all the key-values of your json
your_json_path='$.*'
my_key_values = Tree(your_json).execute(your_json_path)
# If you want to retrieve the name node...then specify it.
my_name= Tree(your_json).execute('$.name')
# If you want to retrieve a the last_name node...then specify it.
last_name= Tree(your_json).execute('$.last_name')
I believe you're just missing a comma in JSON:
{
"user":
{
"actions": [
{
"name": "reading",
"description": "blablabla"
}
],
"name": "John"
}
}
Assuming there is only one "John", with only one "reading" activity, the following query works:
$.user[#.name is 'John'].actions[0][#.name is 'reading'][0].description
If there could be multiple "John"s, with multiple "reading" activities, the following query will almost work:
$.user.*[#.name is 'John'].actions..*[#.name is 'reading'].description
I say almost because the use of .. will be problematic if there are other nested dictionaries with "name" and "description" entries, such as
{
"user": {
"actions": [
{
"name": "reading",
"description": "blablabla",
"nested": {
"name": "reading",
"description": "broken"
}
}
],
"name": "John"
}
}
To get a correct query, there is an open issue to correctly implement queries into arrays: https://github.com/adriank/ObjectPath/issues/60