Is there an easy way to format a dictionary in python for nice output?
I am learning how to interact with an API/XMLRPC in python at the moment. After making a request, I get a dictionary back formatted like the following:
{'category_id': '9', 'parent_id': '3', 'name': 'Headboard', 'is_active': '1', 'position': '6', 'level': '3', 'children': []}, {'category_id': '10', 'parent_id': '3', 'name': 'Mattress', 'is_active': '1', 'position': '7', 'level': '3', 'children': []},
This is a wall of text, easily a few pages. Is there an easy way to display this data nicely, or perhaps just to output the name of each category on one line?
edit:
Here is an attempt to print it via pprint, which ended up omitting a lot of the data:
import xmlrpc.client
import pprint
svc = xmlrpc.client.ServerProxy('https://example.com/api/xmlrpc/')
session = svc.login('apiuser', 'apikey')
temp = svc.call(session, 'catalog_category.tree')
pp = pprint.PrettyPrinter(indent=4)
pp.pprint(temp)
You can use pprint.pprint:
>>> pprint([{'category_id': '9', 'parent_id': '3', 'name': 'Headboard', 'is_active': '1', 'position': '6', 'level': '3', 'children': []}, {'category_id': '10', 'parent_id': '3', 'name': 'Mattress', 'is_active': '1', 'position': '7', 'level': '3', 'children': []}])
[{'category_id': '9',
'children': [],
'is_active': '1',
'level': '3',
'name': 'Headboard',
'parent_id': '3',
'position': '6'},
{'category_id': '10',
'children': [],
'is_active': '1',
'level': '3',
'name': 'Mattress',
'parent_id': '3',
'position': '7'}]
To display just the category names you can do:
>>> [x['name'] for x in ...]
Alternatively you can use json.dump(s) + the JSON viewer of your choice (plenty of online choices available, or just your local browser).
Edit
Processing in a recursive manner:
import copy
t2 = copy.deepcopy(temp) # Modify for printing.
items = [t2]
while items:
item = items.pop(-1)
del item['category_id']
del item['is_active']
del item['level']
del item['position']
... # Whatever other keys you want to delete.
items += item.get('children', [])
pprint(t2)
This will give you a list of category names:
list_of_dicts = [{'category_id': '9', 'parent_id': '3', 'name': 'Headboard', 'is_active': '1', 'position': '6', 'level': '3', 'children': []}, {'category_id': '10', 'parent_id': '3', 'name': 'Mattress', 'is_active': '1', 'position': '7', 'level': '3', 'children': []}]
category_names = [dict['name'] for dict in list_of_dicts]
print(category_names)
OUTPUT:
['Headboard', 'Mattress']
If the data is actually a dictionary of dictionaries, such that it is in the form: { "key_1": {}, "key_2": {} ... "key_n": {} }
then the following code will create a list of the names of categories:
dict_of_dicts = {"key_a": {'category_id': '9', 'parent_id': '3', 'name': 'Headboard', 'is_active': '1', 'position': '6', 'level': '3', 'children': []}, "key_b": {'category_id': '10', 'parent_id': '3', 'name': 'Mattress', 'is_active': '1', 'position': '7', 'level': '3', 'children': []}}
category_names = [dict["name"] for dict in dict_of_dicts.values()]
print(category_names)
OUTPUT:
['Headboard', 'Mattress']
Related
Image of Code
Im trying to pull the key values pair for the dictionary associated to the "awayBattingTotals". However, im encountering the below error that i do not know how to fix.
Snippet of the JSON response is below
{
'namefield': '9 Lopez, N SS',
'ab': '3',
'r': '0',
'h': '1',
'doubles': '0',
'triples': '0',
'hr': '0',
'rbi': '0',
'sb': '0',
'bb': '0',
'k': '0',
'lob': '2',
'avg': '.248',
'ops': '.599',
'personId': 670032,
'battingOrder': '900',
'substitution': False,
'note': '',
'name': 'Lopez, N',
'position': 'SS',
'obp': '.305',
'slg': '.294'
}],
'awayBattingTotals': {
'namefield': 'Totals',
'ab': '33',
'r': '2',
'h': '7',
'hr': '1',
'rbi': '2',
'bb': '0',
'k': '8',
'lob': '13',
'avg': '',
'ops': '',
'obp': '',
'slg': '',
'name': 'Totals',
'position': '',
'note': '',
'substitution': False,
'battingOrder': '',
'personId': 0
},
'homeBattingTotals': {
'namefield': 'Totals',
'ab': '34',
'r': '4',
'h': '9',
'hr': '2',
'rbi': '4',
'bb': '1',
'k': '7',
'lob': '13',
'avg': '',
'ops': '',
'obp': '',
'slg': '',
'name': 'Totals',
'position': '',
'note': '',
'substitution': False,
'battingOrder': '',
'personId': 0
},
The below is obtained via
statsapi.boxscore_data(662647)
summary = statsapi.boxscore(662647)
From the above im trying to run
summary["awayBattingTotals"]["Totals"]
to pull the below values:
`awayBattingTotals': {'namefield': 'Totals', 'ab': '33', 'r': '2', 'h': '7', 'hr': '1', 'rbi': '2', 'bb': '0', 'k': '8', 'lob': '13',`
but i keep getting the below error:
TypeError: string indices must be integers`
As Barmar mentioned, it seemed like the data wasn't behaving as json...
Switching the single to double quotes in the json-like text of the response allows me to reach into it with json.loads() like so:
mysecond = '''{"awayBattingTotals": {"namefield": "Totals", "ab": "33", "r": "2"}}'''
myload = json.loads(mysecond)
print myload
Result:
{u'awayBattingTotals': {u'r': u'2', u'ab': u'33', u'namefield': u'Totals'}}
This failed in the same way you described when I cut and pasted the json response you included in your question:
import json
myjson = """{'awayBattingTotals': { 'namefield': 'Totals',
'ab': '33',
'r': '2'}}"""
print json.loads(myjson)
result:
TypeError: string indices must be integers, not str
I'm new to coding and got myself stuck trying to recursively sort a list of tuples. Here is my code:
# table consists on [(code, parent), (code, parent), ...]
table = [('1', ''),
('1.1', '1'),
('2', ''),
('2.1','2'),
('2.1.1','2.1'),
('3',''),
('3.1','3'),
('4',''),
('4.1','4'),
('4.1.1','4.1'),
('4.1.2','4.1')]
content = {}
def rec(table, parent=None):
while True:
try:
_code = table[0][0]
_parent = table [0][1]
if _parent == '':
content[_code] = _parent
return rec(table[1:])
else:
if _parent in content:
if content[_parent] == '':
content[_parent] = table[0]
else:
content[_parent] = content[_parent], table[0]
return rec(table[1:], parent=_parent)
else:
content[_parent] = table[0]
return rec(table[1:], parent=_parent)
except IndexError:
break
return content
print(rec(table))
The output I'm getting:
{'1': ('1.1', '1'), '2': ('2.1', '2'), '2.1': ('2.1.1', '2.1'), '3':('3.1', '3'), '4': ('4.1', '4'), '4.1': (('4.1.1', '4.1'), ('4.1.2','4.1'))}
But the desired output would be:
{'1': ('1.1', '1'), '2': ('2.1', '2'), {'2.1': ('2.1.1', '2.1')}, '3': ('3.1','3'), '4': ('4.1', '4'), {'4.1': ('4.1.1', '4.1'), ('4.1.2', '4.1')}
I need something like:
{'node_id': '1', 'name':'somename', 'children': [{'node_id': '1.1' ,'name':'somename', 'children': [{'node_id': '1.1.1', 'name':'somename', 'children': [{'node_id': '1.1.1.1', 'name':'somename', 'children': []}]}, {'node_id': '1.1.2', 'name':'somename', 'children': []}, {'node_id': '1.1.3', 'name':'somename', 'children': []}]}, {'node_id': '1.2', 'name':'somename', 'children': []}]}
Any thoughts on how to achieve what I'm aiming for?
For your output to be a tree of nested dictionaries, it will need to have a regular structure where every node is a dictionary with values representing a dictionary of children down to the leaf nodes which would have an empty dictionary for their children.
Here's a simple loop that will build the tree:
table = [('1', ''),
('1.1', '1'),
('2', ''),
('2.1','2'),
('2.1.1','2.1'),
('3',''),
('3.1','3'),
('4',''),
('4.1','4'),
('4.1.1','4.1'),
('4.1.2','4.1')]
tree = { node:dict() for link in table for node in link }
for child,parent in table:
tree[parent].update({child:tree[child]})
output:
print(tree[""]) # "" is te root
{
'1': { '1.1': {}},
'2': {
'2.1': { '2.1.1': {}}
},
'3': { '3.1': {}},
'4': {
'4.1': {
'4.1.1': {},
'4.1.2': {}
}
}
}
as a side benefit, this structure gives you an index for all the nodes in the tree
With dictionaries of attributes (one of which is the list of children), the same approach can be used:
tree = { node:{"id":node,"children":[]} for link in table for node in link }
for child,parent in table:
tree[parent]["children"].append(tree[child])
output:
print(tree[""]["children"]) # children of root
[ { 'id': '1',
'children': [ { 'id': '1.1', 'children': []} ]
},
{ 'id': '2',
'children': [
{ 'id': '2.1',
'children': [ {'id': '2.1.1', 'children': []} ]
}
]
},
{ 'id': '3',
'children': [ { 'id': '3.1','children': []} ]
},
{ 'id': '4',
'children': [
{ 'id': '4.1',
'children': [
{ 'id': '4.1.1', 'children': []},
{ 'id': '4.1.2', 'children': []}
]
}
]
}
]
A recursive approach is nice but would preform much slower and will not produce an index to access the nodes by their Id's:
def tree(links,node=""):
return {"id":node, "children":[tree(links,child) for child,parent in links if parent==node] }
root = tree(table)
You can use recursion:
table = [('1', ''), ('1.1', '1'), ('2', ''), ('2.1', '2'), ('2.1.1', '2.1'), ('3', ''), ('3.1', '3'), ('4', ''), ('4.1', '4'), ('4.1.1', '4.1'), ('4.1.2', '4.1')]
def to_dict(d):
return {'node_id':d, 'children':[*map(to_dict, [a for a, b in table if b == d])]}
result = [to_dict(a) for a, b in table if not b]
Output:
[{'node_id': '1', 'children': [{'node_id': '1.1', 'children': []}]}, {'node_id': '2', 'children': [{'node_id': '2.1', 'children': [{'node_id': '2.1.1', 'children': []}]}]}, {'node_id': '3', 'children': [{'node_id': '3.1', 'children': []}]}, {'node_id': '4', 'children': [{'node_id': '4.1', 'children': [{'node_id': '4.1.1', 'children': []}, {'node_id': '4.1.2', 'children': []}]}]}]
Edit: supposing your tuples in table have additional information:
table = [('1', '', 'someval0'), ('1.1', '1', 'someval1'), ('2', '', 'someval2'), ('2.1', '2', 'someval3'), ('2.1.1', '2.1', 'someval4'), ('3', '', 'someval5'), ('3.1', '3', 'someval6'), ('4', '', 'someval7'), ('4.1', '4', 'someval8'), ('4.1.1', '4.1', 'someval9'), ('4.1.2', '4.1', 'someval10')]
def to_dict(d):
return {**(dict(zip(['node_id', 'name'], d))), 'children':[*map(to_dict, [(a, *c) for a, b, *c in table if b == d[0]])]}
result = [to_dict((a, *c)) for a, b, *c in table if not b]
Output:
[{'node_id': '1', 'name': 'someval0', 'children': [{'node_id': '1.1', 'name': 'someval1', 'children': []}]}, {'node_id': '2', 'name': 'someval2', 'children': [{'node_id': '2.1', 'name': 'someval3', 'children': [{'node_id': '2.1.1', 'name': 'someval4', 'children': []}]}]}, {'node_id': '3', 'name': 'someval5', 'children': [{'node_id': '3.1', 'name': 'someval6', 'children': []}]}, {'node_id': '4', 'name': 'someval7', 'children': [{'node_id': '4.1', 'name': 'someval8', 'children': [{'node_id': '4.1.1', 'name': 'someval9', 'children': []}, {'node_id': '4.1.2', 'name': 'someval10', 'children': []}]}]}]
I am facing this problem, I want to separate the dataset that has completed and not complete.
So, I want to put flag like 'complete' in the JSON. Example as in output.
This is the data that i have
data=[{'id': 'abc001',
'demo':{'gender':'1',
'job':'6',
'area':'3',
'study':'3'},
'ex_data':{'fam':'small',
'scholar':'2'}},
{'id': 'abc002',
'demo':{'gender':'1',
'edu':'6',
'qual':'3',
'living':'3'},
'ex_data':{'fam':'',
'scholar':''}},
{'id': 'abc003',
'demo':{'gender':'1',
'edu':'6',
'area':'3',
'sal':'3'}
'ex_data':{'fam':'big',
'scholar':NaN}}]
Output
How can I put the flag and also detect NaN and NULL in JSON?
Output=[{'id': 'abc001',
'completed':'yes',
'demo':{'gender':'1',
'job':'6',
'area':'3',
'study':'3'},
'ex_data':{'fam':'small',
'scholar':'2'}},
{'id': 'abc002',
'completed':'no',
'demo':{'gender':'1',
'edu':'6',
'qual':'3',
'living':'3'},
'ex_data':{'fam':'',
'scholar':''}},
{'id': 'abc003',
'completed':'no',
'demo':{'gender':'1',
'edu':'6',
'area':'3',
'sal':'3'}
'ex_data':{'fam':'big',
'scholar':NaN}}]
Something like this should work for you:
data = [
{
'id': 'abc001',
'demo': {
'gender': '1',
'job': '6',
'area': '3',
'study': '3'},
'ex_data': {'fam': 'small',
'scholar': '2'}
},
{
'id': 'abc002',
'demo': {
'gender': '1',
'edu': '6',
'qual': '3',
'living': '3'},
'ex_data': {'fam': '',
'scholar': ''}},
{
'id': 'abc003',
'demo': {
'gender': '1',
'edu': '6',
'area': '3',
'sal': '3'},
'ex_data': {'fam': 'big',
'scholar': None}
}
]
def browse_dict(dico):
empty_values = 0
for key in dico:
if dico[key] is None or dico[key] == "":
empty_values += 1
if isinstance(dico[key], dict):
for k in dico[key]:
if dico[key][k] is None or dico[key][k] == "":
empty_values += 1
if empty_values == 0:
dico["completed"] = "yes"
else:
dico["completed"] = "no"
for d in data:
browse_dict(d)
print(d)
Output :
{'id': 'abc001', 'demo': {'gender': '1', 'job': '6', 'area': '3', 'study': '3'}, 'ex_data': {'fam': 'small', 'scholar': '2'}, 'completed': 'yes'}
{'id': 'abc002', 'demo': {'gender': '1', 'edu': '6', 'qual': '3', 'living': '3'}, 'ex_data': {'fam': '', 'scholar': ''}, 'completed': 'no'}
{'id': 'abc003', 'demo': {'gender': '1', 'edu': '6', 'area': '3', 'sal': '3'}, 'ex_data': {'fam': 'big', 'scholar': None}, 'completed': 'no'}
Note that I changed NaN to None, because here you are most likely showing a python dictionary, not a JSON file since you are using data =
In a dictionary, the NaN value would be changed for None.
If you have to convert your JSON to a dictionary, refer to the JSON module documentation.
Also please check your dictionary syntax. You missed several commas to separate data.
You should try
The Input is
data = [{'demo': {'gender': '1', 'job': '6', 'study': '3', 'area': '3'}, 'id': 'abc001', 'ex_data': {'scholar': '2', 'fam': 'small'}}, {'demo': {'living': '3', 'gender': '1', 'qual': '3', 'edu': '6'}, 'id': 'abc002', 'ex_data': {'scholar': '', 'fam': ''}}, {'demo': {'gender': '1', 'area': '3', 'sal': '3', 'edu': '6'}, 'id': 'abc003', 'ex_data': {'scholar': None, 'fam': 'big'}}]
Also, Nan will not work in Python. So, instead of Nan we have used None.
for item in data:
item["completed"] = 'yes'
for key in item.keys():
if isinstance(item[key],dict):
for inner_key in item[key].keys():
if (not item[key][inner_key]):
item["completed"] = "no"
break
else:
if (not item[key]):
item["completed"] = "no"
break
The Output will be
data = [{'demo': {'gender': '1', 'job': '6', 'study': '3', 'area': '3'}, 'completed': 'yes', 'id': 'abc001', 'ex_data': {'scholar': '2', 'fam': 'small'}}, {'demo': {'living': '3', 'edu': '6', 'qual': '3', 'gender': '1'}, 'completed': 'no', 'id': 'abc002', 'ex_data': {'scholar': '', 'fam': ''}}, {'demo': {'edu': '6', 'gender': '1', 'sal': '3', 'area': '3'}, 'completed': 'no', 'id': 'abc003', 'ex_data': {'scholar': None, 'fam': 'big'}}]
I have a list of campuses:
campus = [{'id': '1', 'dlin': '1'}, {'id': '2', 'dlin': '1'},{'id': '3', 'dlin': '1'},{'id': '4', 'dlin': '2'},{'id': '5', 'dlin': '2'},{'id': '6', 'dlin': '1'}, ]
each campus belongs to a school with a unique dlin. I want to have a list in which I have some other lists, each having a few dictionaries.
I run the below code:
schools = []
for i in campus:
ls = []
for j in campus:
if i['dlin'] == j['dlin']:
ls.append(j)
# campus_copy.remove(j)
schools.append(ls)
[print(item) for item in schools]
the result is:
[{'id': '1', 'dlin': '1'}, {'id': '2', 'dlin': '1'}, {'id': '3', 'dlin': '1'}, {'id': '6', 'dlin': '1'}]
[{'id': '1', 'dlin': '1'}, {'id': '2', 'dlin': '1'}, {'id': '3', 'dlin': '1'}, {'id': '6', 'dlin': '1'}]
[{'id': '1', 'dlin': '1'}, {'id': '2', 'dlin': '1'}, {'id': '3', 'dlin': '1'}, {'id': '6', 'dlin': '1'}]
[{'id': '4', 'dlin': '2'}, {'id': '5', 'dlin': '2'}]
[{'id': '4', 'dlin': '2'}, {'id': '5', 'dlin': '2'}]
[{'id': '1', 'dlin': '1'}, {'id': '2', 'dlin': '1'}, {'id': '3', 'dlin': '1'}, {'id': '6', 'dlin': '1'}]
I have to either remove the duplicate members from schools or modify the code such that I do not get duplicates.
When I try to remove duplicates from schools, I see that dic item is not hashable so I can not do it.
To solutions are available that are somewhat similar to my problem.
Remove duplicates from list of dictionaries within list of dictionaries
Remove duplicate dict in list in Python
However, I cannot figure out what to do?
does anybody know how to solve the problem?
what I expect to get is:
[{'id': '1', 'dlin': '1'}, {'id': '2', 'dlin': '1'}, {'id': '3', 'dlin': '1'}, {'id': '6', 'dlin': '1'}]
[{'id': '4', 'dlin': '2'}, {'id': '5', 'dlin': '2'}]
One possible solution is storing the dlin as key in dictionary (and dictionaries cannot have multiple equal keys) rather than removing duplicates explicitly afterwards:
campus = [{'id': '1', 'dlin': '1'}, {'id': '2', 'dlin': '1'},{'id': '3', 'dlin': '1'},{'id': '4', 'dlin': '2'},{'id': '5', 'dlin': '2'},{'id': '6', 'dlin': '1'}, ]
schools = {}
for c in campus:
schools.setdefault(c['dlin'], []).append(c)
for s in schools.values():
print(s)
Prints:
[{'id': '1', 'dlin': '1'}, {'id': '2', 'dlin': '1'}, {'id': '3', 'dlin': '1'}, {'id': '6', 'dlin': '1'}]
[{'id': '4', 'dlin': '2'}, {'id': '5', 'dlin': '2'}]
Based on the answer of Andrej, I solved another part of the question I had and I wanted just to share it here:
My question:
I am now involved in another issue related to the previous one:
I have this list of dictionaries, each informaton of a campus. multiple campuses might belong to a school. I have to distinguish and cluster them based on the similarity of their names.
campus = [
{'id': '1', 'name': 'seneca - york'},
{'id': '2', 'name': 'seneca college - north gate campus'},
{'id': '3', 'name': 'humber college - toronto campus'},
{'id': '4', 'name': 'humber college'},
{'id': '5', 'name': 'humber collge - waterloo campus'},
{'id': '6', 'name': 'university of waterloo toronto campus'},
]
my expected result can be reached by this small and neat code:
schools = {}
for c in campus:
schools.setdefault(c['name'][:4], []).append(c)
print(schools)
For two days I try to traverse a list of dicts into a tree.
`list_of_dicts = [
{'name':Category1, 'id': '7', 'parent_id': '7', 'level': '1'}
{'name':Category3, 'id': '33', 'parent_id': '7', 'level': '2'}
{'name':Category5, 'id': '334', 'parent_id': '33', 'level': '3'}
{'name':Category10, 'id': '23', 'parent_id': '7', 'level': '2'}
{'name':Category2, 'id': '8', 'parent_id': '8', 'level': '1'}
{'name':Category6, 'id': '24', 'parent_id': '8', 'level': '2'}
]`
As informations, we know a category on top level (1), has its own id as its parent_id, children have the id of its parent as parent_id and the level.
In a first step the list need to turn in something like a tree:
`traversed_list = [
{'name':Category1, 'id': '7', 'parent_id': '7', 'level': '1', 'children':
[
{'name':Category3, 'id': '33', 'parent_id': '7', 'level': '2', 'children': [
{'name':Category5, 'id': '334', 'parent_id': '33', 'level': '3', 'children':[]}]}
{'name':Category10, 'id': '23', 'parent_id': '7', 'level': '2', 'children':[]}
]}
{'name':Category2, 'id': '8', 'parent_id': '8', 'level': '1', 'children':
[{'name':Category6, 'id': '24', 'parent_id': '8', 'level': '2', 'children':[]}]
}]`
The following code:
import copy
def treeify(lst):
tree = [copy.deepcopy(cat) for cat in lst if cat['level'] == '1']
for el in tree:
el["children"] = []
for i in xrange(len(lst)):
for j in xrange(len(tree)):
if lst[i]["parent_id"] == tree[j]["id"]:
tree[j]["children"].append(copy.deepcopy(lst[i]))
return tree
list_of_dicts = [
{'name':"Category1", 'id': '7', 'parent_id': '7', 'level': '1'},
{'name':"Category3", 'id': '33', 'parent_id': '7', 'level': '2'},
{'name':"Category5", 'id': '334', 'parent_id': '33', 'level': '3'},
{'name':"Category10", 'id': '23', 'parent_id': '7', 'level': '2'},
{'name':"Category2", 'id': '8', 'parent_id': '8', 'level': '1'},
{'name':"Category6", 'id': '24', 'parent_id': '8', 'level': '2'}
]
tree = treeify(list_of_dicts)
for d in tree:
print d
prints
{'id': '7', 'parent_id': '7', 'children': [{'id': '7', 'parent_id': '7', 'name': 'Category1', 'level': '1'}, {'id': '33', 'parent_id': '7', 'name': 'Category3', 'level': '2'}, {'id': '23', 'parent_id': '7', 'name': 'Category10', 'level': '2'}], 'name': 'Category1', 'level': '1'}
{'id': '8', 'parent_id': '8', 'children': [{'id': '8', 'parent_id': '8', 'name': 'Category2', 'level': '1'}, {'id': '24', 'parent_id': '8', 'name': 'Category6', 'level': '2'}], 'name': 'Category2', 'level': '1'}