I came across this recursive function that's supposed flatten a dictionary:
def flatten(data, prefix='', separator='.'):
"""Flattens a nested dict structure. """
if not isinstance(data, dict):
return {prefix: data} if prefix else data
result = {}
for (key, value) in data.items():
result.update(flatten(value,_get_new_prefix(prefix, key, separator),
separator=separator))
return result
def _get_new_prefix(prefix, key, separator):
return (separator.join((prefix, str(key))) if prefix else str(key))
it's also supposed to be fed with this data:
nested = {
'fullname': 'Alessandra',
'age': 41,
'phone-numbers': ['+447421234567', '+447423456789'],
'residence': {'address': {'first-line': 'Alexandra Rd','second-line': '',},
'zip': 'N8 0PP',
'city': 'London',
'country': 'UK',
},
}
I'm trying to figure out how it works and particularly how the "prefix" parameter works and in what case it will not be empty.
The flatten() function builds the path down the tree, where the names in the path are separated by the separator. The prefix is added to the beginning of that path.
prefix is only "empty" if you set it to None. That will have the effect of suppressing the prefix.
For example, compare the output of this:
[print(k, ' = ', v) for k,v in flatten(nested, prefix='xxx', separator='/').items()]
...with this:
[print(k, ' = ', v) for k,v in flatten(nested, prefix=None).items()]
Related
im given the following input:
'family': 'man: name, woman: name, child: name , grandma: name, grandpa: name'
where 'family' is a key, and its value is a bunch of other key-value pairs. as you can tell, you cant parse it using json() because this string is not structured in json format. ive been trying for hours to parse this string into a dictionary/json valid string so i could work with it properly and change name values accordingly. would appreciate the help.
Assuming that no key and no value ever contains a comma or a colon, you could do this:
def split_to_dict(string: str) -> dict[str, str]:
output = {}
for pair in string.split(','):
key, value = pair.split(':')
output[key.strip()] = value.strip()
return output
Calling it like this:
s = 'man: name, woman: name, child: name , grandma: name, grandpa: name'
d = split_to_dict(s)
print(d)
gives the following:
{'man': 'name', 'woman': 'name', 'child': 'name', 'grandma': 'name', 'grandpa': 'name'}
For readability purposes, I would refrain from doing this in a dictionary comprehension.
You first need to convert it to a correct json.
d = {'family': 'man: name, woman: name, child: name , grandma: name, grandpa: name'}
d['family'] = re.sub("(\w+)", r'"\1"', d['family'])
# now parse it
json.loads("{" + d['family'] + "}")
Assuming the input is loaded in as a dictionary and we want to turn the value of family into a corresponding dictionary of key-value pairs. We can do that by doing some string processing as such:
>>> d = {'family': 'man: name, woman: name, child: name , grandma: name, grandpa: name'}
>>> d['family'] = { x.split(':')[0] : x.split(':')[1].strip() for x in d['family'].split(', ')}
>>>
>>> d['family']
>>> {'man': 'name','woman': 'name', 'child': 'name', 'grandma': 'name', 'grandpa': 'name'}
We first split the value of family into key-value strings, and then we further split each key-value string to turn it into the corresponding key-value pair in the list comprehension of the new dictionary.
I have a dictionary of values. This is for a company name.
It has 3 keys:
{'html_attributions': [],
'result' : {'Address': '123 Street', 'website' :'123street.com'
'status': 'Ok' }
I have a dataframe of many dictionaries. I want to loop through each row's dictionary and get the necessary information I want.
Currently I am writing for loops to retrieve these information. Is there a more efficient way to retrieve these information?
addresses = []
for i in range(len(testing)):
try:
addresses.append(testing['Results_dict'][i]['result']['Address'])
except:
addresses.append('No info')
What I have works perfectly fine. However I would like something that would be more efficient. Perhaps using the get() method? but I don't know how I can call to get the inside of 'result'.
Try this:
def get_address(r):
try:
return r['result']['Address']
except Exception:
return 'No info'
addresses = df['Results_dict'].map(get_address)
This guards against cases where Result_dict is None, not a dict, or any key along the path way does not exist.
This is a way faster solution if the data is big:
addresses = list(map(lambda x: x.get('result').get('Address', 'No info'), testing['Results_dict']))
Here is how I deal with nested dict keys:
Example:
def keys_exists(element, *keys):
if not isinstance(element, dict):
raise AttributeError('keys_exists() expects dict as first argument.')
if len(keys) == 0:
raise AttributeError('keys_exists() expects at least two arguments,
one given.')
_element = element
for key in keys:
try:
_element = _element[key]
except KeyError:
return False
return True
For data :
{'html_attributions': [],
'result' : {'Address': '123 Street', 'website' :'123street.com'
'status': 'Ok' }
if you want to check result exists or not use above function like this
`print 'result (exists/Not): {}'.format(keys_exists(data,"result"))`
To check address exist inside result Try this
`print 'result > Address (exists/not): {}'.format(keys_exists(data, "result", "Address"))`
It will return output in True/False
Below is an example of a complicated dictionary that I am looping through. I want to check if "AccountRoot" appears in the dictionary. Be aware that I am looping through many of these dictionaries and the format changes. Thus I was wondering if there was a function like .find(). I could not find any and it seems like .find() does not work.
Example Dict;
{'hash': '752F3B5CEE85F3C2DC60041DCAC4777BECE9CC11585225383F8178EBC2ACFB16',
'ledger_index': 108843,
'date': '2013-01-18T22:27:20+00:00',
'tx': {'TransactionType': 'OfferCreate',
'Flags': 0,
'Sequence': 3,
'TakerPays': '499950000',
'TakerGets': {'value': '0.05',
'currency': 'BTC',
'issuer': 'r4aZ4aqXHfrcYfuFrTqDmSopfgPHnRS9MZ'},
'Fee': '10',
'SigningPubKey': '027008A4A7AED7B5426EAC46691CFCAC8CA3CF2773D1CAC4074F0BC58EC24BE883',
'TxnSignature': '3046022100C38236B533936B4A328346D5246570976B8A1390655EC1B6F4090C42AE73FD8D022100D49E5498C40D90AF7BD02F2818EE04F1D0F6B0C76F0325997190D56BF4B9D82D',
'Account': 'r4aZ4aqXHfrcYfuFrTqDmSopfgPHnRS9MZ'},
'meta': {'TransactionIndex': 0,
'AffectedNodes': [{'CreatedNode': {'LedgerEntryType': 'DirectoryNode',
'LedgerIndex': '0A624575D3C02D544B92F23F6A8BDF3B10745427B731613820A1695AFF11993B',
'NewFields': {'ExchangeRate': '5E2386099B1BF000',
'RootIndex': '0A624575D3C02D544B92F23F6A8BDF3B10745427B731613820A1695AFF11993B',
'TakerGetsCurrency': '0000000000000000000000004254430000000000',
'TakerGetsIssuer': 'E767BCB9E1A31C46C16F42DA9DDE55792767F565'}}},
{'CreatedNode': {'LedgerEntryType': 'DirectoryNode',
'LedgerIndex': '165845E192D2217A6518C313F3F4B2FD676EE1619FF50CB85E2386099B1BF000',
'NewFields': {'ExchangeRate': '5E2386099B1BF000',
'RootIndex': '165845E192D2217A6518C313F3F4B2FD676EE1619FF50CB85E2386099B1BF000',
'TakerGetsCurrency': '0000000000000000000000004254430000000000',
'TakerGetsIssuer': 'E767BCB9E1A31C46C16F42DA9DDE55792767F565'}}},
{'ModifiedNode': {'LedgerEntryType': 'AccountRoot',
'PreviousTxnLgrSeq': 108839,
'PreviousTxnID': '8B2921C5222A6814BCF7602A18FEACE94797A644AF893A43FB642C172CC14ED0',
'LedgerIndex': '481DA662E465CC7888FD3750A0952F2003D78DCAA8CB2E91088E862BB7D30B98',
'PreviousFields': {'Sequence': 3,
'OwnerCount': 0,
'Balance': '9999999980'},
'FinalFields': {'Flags': 0,
'Sequence': 4,
'OwnerCount': 1,
'Balance': '9999999970',
'Account': 'r4aZ4aqXHfrcYfuFrTqDmSopfgPHnRS9MZ'}}},
{'CreatedNode': {'LedgerEntryType': 'Offer',
'LedgerIndex': '9AABB5DCD201AE7FB0F9B7F90083F48B7451977B2419339ADFEBD8876B54EB66',
'NewFields': {'Sequence': 3,
'BookDirectory': '165845E192D2217A6518C313F3F4B2FD676EE1619FF50CB85E2386099B1BF000',
'TakerPays': '499950000',
'TakerGets': {'value': '0.05',
'currency': 'BTC',
'issuer': 'r4aZ4aqXHfrcYfuFrTqDmSopfgPHnRS9MZ'},
'Account': 'r4aZ4aqXHfrcYfuFrTqDmSopfgPHnRS9MZ'}}}],
'TransactionResult': 'tesSUCCESS'}}
Answer here:
Finding a key recursively in a dictionary
Posting this so people can encounter the answer if they search using different terms.
I would use alecxe's answer using the stack of iterators pattern defined here by Gareth Rees: http://garethrees.org/2016/09/28/pattern/
Code in case other links are destroyed:
def search(d, key, default=None):
"""
Return a value corresponding to the specified key in the (possibly
nested) dictionary d. If there is no item with that key, return
default.
"""
stack = [iter(d.items())]
while stack:
for k, v in stack[-1]:
if isinstance(v, dict):
stack.append(iter(v.items()))
break
elif k == key:
return v
else:
stack.pop()
return default
This code allows you to avoid the problem of exceeding the maximum recursion depth present in some other solutions.
Edit: realized you're simply trying to find out if a value exists in the dictionary.
You can simply modify the for loop to something like this, and it should work for a simple true/false search.
def search(d, key, default=False):
"""
Return a value corresponding to the specified key in the (possibly
nested) dictionary d. If there is no item with that key, return
default.
"""
stack = [iter(d.items())]
while stack:
for k, v in stack[-1]:
if isinstance(v, dict):
stack.append(iter(v.items()))
break
elif k == key:
return True
elif v == key:
return True
else:
stack.pop()
return default
There's potentially an easier way, but here's how I've done something similar. I've modified it for your use case.
import copy
def traverse_dict(_obj):
_obj_2 = copy.deepcopy(_obj)
if isinstance(_obj_2, dict):
for key, value in _obj_2.items():
if key == 'your_value':
do_your_stuff()
_obj_2[key] = traverse_dict(value)
elif isinstance(_obj, list):
for offset in range(len(_obj_2)):
_obj_2[offset] = traverse_dict(_obj_2[offset])
return _obj_2
I found a method to iterate a python dictionary object recursively on this forum. However, I wish to extend that function so that I get a string similar to the structure of a file path. With my function below, I expect an output in the form of
/key1/value1
/key2/value2
/key3/key3a/value3a
/key4/key4a/key4a1/value4a1
/key4/key4a/key4a2/value4a2
/key4/key4a/key4a3/value4a3
/key4/key4b/key4b1/key4b1a/value4b1a
/key4/key4b/key4b1/key4b1b/value4b1b
/key4/key4b/key4b1/key4b1c/value4b1c
/key4/key4c/key4c1/key4c1a/value4c1a
/key4/key4c/key4c1/key4c1b/value4c1b
/key4/key4c/key4c1/key4c1c/value4c1c
Unfortunately, I hit a block. I cannot figure out how to achieve that. Below is the code that I came up with. Any help is greatly appreciated.
import sys
import collections
dict_object = {
'key1': 'value1',
'key2': 'value2',
'key3': {'key3a': 'value3a'},
'key4': {
'key4a': {
'key4a1': 'value4a1',
'key4a2': 'value4a2',
'key4a3': 'value4a3'
},
'key4b': {
'key4b1': {
'key4b1a': 'value4b1a',
'key4b1b': 'value4b1b',
'key4b1c': 'value4b1c'
},
'key4c': {
'key4c1': {
'key4c1a': 'value4c1a',
'key4c1b': 'value4c1b',
'key4c1c': 'value4c1c'
}
}
}
}
}
def print_dict(dictionary, path='', parent=''):
""" This finction recursively prints nested dictionaries."""
#Sort the dictionary object by its keys
if isinstance(dictionary, dict):
dictionary = collections.OrderedDict(sorted(dictionary.items()))
else:
dictionary = sorted(dictionary.items(), key=operator.itemgetter(1))
#iterate each sorted dictionary key
for key, value in dictionary.iteritems():
if isinstance(value, dict):
path = ''
path = '%s/%s/%s' % (path, parent, key)
#Repeat this funtion for nested {} instances
print_dict(value, path, key)
else:
#Print the last node i.e PATH + KEY + VALUE
print '%s/%s/%s' % (path, key, value)
if __name__ == '__main__':
print_dict(dict_object)
Your function appears overly complicated. Only actually print when you have an object that's not a dictionary, otherwise recurse for all values in a dictionary. I simplified path handling to just one string:
def print_dict(ob, path=''):
if not isinstance(ob, dict):
print '{}/{}'.format(path, ob)
else:
for key, value in sorted(ob.items()):
print_dict(value, '{}/{}'.format(path, key))
I didn't bother with creating OrderedDict objects; all you need is iteration in sorted order.
This produces the expected output:
>>> print_dict(dict_object)
/key1/value1
/key2/value2
/key3/key3a/value3a
/key4/key4a/key4a1/value4a1
/key4/key4a/key4a2/value4a2
/key4/key4a/key4a3/value4a3
/key4/key4b/key4b1/key4b1a/value4b1a
/key4/key4b/key4b1/key4b1b/value4b1b
/key4/key4b/key4b1/key4b1c/value4b1c
/key4/key4b/key4c/key4c1/key4c1a/value4c1a
/key4/key4b/key4c/key4c1/key4c1b/value4c1b
/key4/key4b/key4c/key4c1/key4c1c/value4c1c
using app engine - yes i know all about django templates and other template engines.
Lets say i have a dictionary or a simple object, i dont know its structure and i want to serialize it into html.
so if i had
{'data':{'id':1,'title':'home','address':{'street':'some road','city':'anycity','postal':'somepostal'}}}
want i want is that rendered in some form of readable html using lists or tables;
data:
id:1
title:home
address:
street: some road
city: anycity
postal:somepostal
now i know i can do
for key in dict.items
print dict[key]
but that wont dive into the child values and list each key, value pair when the key/value is a dictionary - ie the address dict.
Is their a module for python that is lightweight/fast that will do this nicely. or does anyone have any simple code they can paste that might do this.
Solution
All the solutions here were useful. pprint is no doubt the more stable means of printing the dictionary, though it falls short of returning anything near html. Though still printable.
I ended up with this for now:
def printitems(dictObj, indent=0):
p=[]
p.append('<ul>\n')
for k,v in dictObj.iteritems():
if isinstance(v, dict):
p.append('<li>'+ k+ ':')
p.append(printitems(v))
p.append('</li>')
else:
p.append('<li>'+ k+ ':'+ v+ '</li>')
p.append('</ul>\n')
return '\n'.join(p)
It converts the dict into unordered lists which is ok for now. some css and perhaps a little tweaking should make it readable.
Im going to reward the answer to the person that wrote the above code, i made a couple of small changes as the unordered lists were not nesting. I hope all agree that many of the solutions offered proved useful, But the above code renders a true html representation of a dictionary, even if crude.
The example made by pyfunc could easily be modified to generate simple nested html lists.
z = {'data':{'id':1,'title':'home','address':{'street':'some road','city':'anycity','postal':'somepostal'}}}
def printItems(dictObj, indent):
print ' '*indent + '<ul>\n'
for k,v in dictObj.iteritems():
if isinstance(v, dict):
print ' '*indent , '<li>', k, ':', '</li>'
printItems(v, indent+1)
else:
print ' '*indent , '<li>', k, ':', v, '</li>'
print ' '*indent + '</ul>\n'
printItems(z,0)
Not terribly pretty of course, but somewhere to start maybe. If all you want to do is visualize data, the pprint module really is good enough. You could just use the "pre" tag on the result from pprint and put that on your web page.
the pprint version would look something like this:
import pprint
z = {'data':{'id':1,'title':'home','address':{'street':'some road','city':'anycity','postal':'somepostal'}}}
print '<pre>', pprint.pformat(z), '</pre>'
And the html output look something like this:
{'data': {'address': {'city': 'anycity',
'postal': 'somepostal',
'street': 'some road'},
'id': 1,
'title': 'home'}}
Which isn't that pretty, but it at least shows the data in a more structured way.
import pprint
pprint.pprint(yourDict)
Well, no HTML, but similar to your for/print approach.
EDIT: or use:
niceText = pprint.pformat(yourDict)
this will give you the same nice output with all indents, etc. Now you can iterate over lines and format it into HTML:
htmlLines = []
for textLine in pprint.pformat(yourDict).splitlines():
htmlLines.append('<br/>%s' % textLine) # or something even nicer
htmlText = '\n'.join(htmlLines)
Here's my simple solution, It can handle any level of nested dictionary.
import json
temp_text = {'decision': {'date_time': None, 'decision_type': None},
'not_received': {'date_time': '2019-04-15T19:18:43.825766'},
'received': {'date_time': None},
'rfi': {'date_time': None},
'under_review': {'date_time': None}}
dict_text_for_html = json.dumps(
temp_text, indent=4
).replace(' ', ' ').replace(',\n', ',<br>').replace('\n', '<br>')
html view of python dict
I needed something similar, but also wanted to pretty print lists, and lists inside the dict. Here's what I came up:
def format(self, obj, indent = 1):
if isinstance(obj, list):
htmls = []
for k in obj:
htmls.append(self.format(k,indent+1))
return '[<div style="margin-left: %dem">%s</div>]' % (indent, ',<br>'.join(htmls))
if isinstance(obj, dict):
htmls = []
for k,v in obj.iteritems():
htmls.append("<span style='font-style: italic; color: #888'>%s</span>: %s" % (k,self.format(v,indent+1)))
return '{<div style="margin-left: %dem">%s</div>}' % (indent, ',<br>'.join(htmls))
return str(obj)
Then, if you're using webapp on appengine, you can just do the following:
self.response.out.write(self.format(obj))
This is an example of the output:
Look at my implementation:
def pretty_items(r, d, nametag="<strong>%s: </strong>", itemtag='<li>%s</li>',
valuetag="%s", blocktag=('<ul>', '</ul>')):
if isinstance(d, dict):
r.append(blocktag[0])
for k, v in d.iteritems():
name = nametag % k
if isinstance(v, dict) or isinstance(v, list):
r.append(itemtag % name)
pretty_items(r, v)
else:
value = valuetag % v
r.append(itemtag % (name + value))
r.append(blocktag[1])
elif isinstance(d, list):
r.append(blocktag[0])
for i in d:
if isinstance(i, dict) or isinstance(i, list):
r.append(itemtag % " - ")
pretty_items(r, i)
else:
r.append(itemtag % i)
r.append(blocktag[1])
Will output all items in HTML format using <ul> and <li> tags. And is also optional to change the tags. And then, just use CSS to handle with the indentation.
None of the above examples give good results, so I wrote two of my own functions that create beautiful looking html output for dictionaries.
def dict_to_html(dd, level=0):
"""
Convert dict to html using basic html tags
"""
import simplejson
text = ''
for k, v in dd.iteritems():
text += '<br>' + ' '*(4*level) + '<b>%s</b>: %s' % (k, dict_to_html(v, level+1) if isinstance(v, dict) else (simplejson.dumps(v) if isinstance(v, list) else v))
return text
def dict_to_html_ul(dd, level=0):
"""
Convert dict to html using ul/li tags
"""
import simplejson
text = '<ul>'
for k, v in dd.iteritems():
text += '<li><b>%s</b>: %s</li>' % (k, dict_to_html_ul(v, level+1) if isinstance(v, dict) else (simplejson.dumps(v) if isinstance(v, list) else v))
text += '</ul>'
return text
You could use pretty print (pprint)
or if you want to do some further processing of display then you have to run through the dict yourself.
Be warned that the code is crude and will require numerous refinements. Solution uses recursion too, which is bad, if the recursion depth is higher.
z = {'data':{'id':1,'title':'home','address':{'street':'some road','city':'anycity','postal':'somepostal', 'telephone':{'home':'xxx','offie':'yyy'}}}}
def printItems(dictObj, indent):
it = dictObj.iteritems()
for k,v in it:
if isinstance(v, dict):
print ' '*indent , k, ':'
printItems(v, indent+1)
else:
print ' '*indent , k, ':', v
printItems(z,0)
Output:
data :
address :
city : anycity
postal : somepostal
street : some road
telephone :
home : xxx
offie : yyy
id : 1
title : home
Here is my version with support of lists (labels are verbose names of keys in dictionary):
def render_value(value, labels):
if isinstance(value, (list, tuple)):
return render_list(value, labels)
elif isinstance(value, dict):
return render_dict(value, labels)
else:
return value
def render_list(lst, labels):
items = [
'<li>%s</li>' % render_value(value, labels)
for value in lst
]
return '\n'.join(['\n<ul>'] + items + ['</ul>\n'])
def render_dict(dct, labels):
items = []
for key, value in dct.items():
if not value: continue
key = labels.get(key, key)
value = render_value(value, labels)
items.append('<li><b>%s</b>: %s</li>' % (key, value))
return '\n'.join(['\n<ul>'] + items + ['</ul>\n'])
imagine we have this :{name: "a", children:[{name: "b", children: [] },{..},{..}]
def ConvertDictToUlLi():
jsonResult = GetSomeRecursiveDict()
def CreateHtml(DictItem, output):
output = "<li>"+DictItem["name"] if jsonResult.has_key("name") else " "
if len(DictItem["children"]) > 0:
output = output + "<ul>"
for item in DictItem["children"]:
output = output + " "+CreateHtml(item, output)+" "
output = output + "</ul>"
return output+"</li>"
result = "<ul class='tree'>"+CreateHtml(jsonResult, "")+"</ul>"
return result