I have a nested dictionary that looks like below,
{
'product_list.show_date': "May '21",
'product_list.new_users':
{
'product_list.product':
{
'A': None,
'B': 377,
'C': None,
'D': 67,
'E': None,
'F': 1,
'G': None
}
}
}
And I want to clear it out in a way that parent keys are not there. So basically, I want a dictionary that is not nested. Like below,
{
'product_list.show_date': "May '21",
'A': None,
'B': 377,
'C': None,
'D': 67,
'E': None,
'F': 1,
'G': None
}
I am using the recursive function to do this, but it's not 100% correct.
Here's my code,
def clear_nd(d, nested_dict):
for key in nested_dict:
if type(nested_dict[key]) != dict:
d[key] = nested_dict[key]
elif type(nested_dict[key]) == dict:
nested_dict = nested_dict[key]
clear_nd(d, nested_dict)
return d
d = {}
clear_nd(d, nested_dict)
For below example,
nested_dict = {
'product_list.show_date': "May '21",
'product_list.new_users': {
'product_list.product': {
'A': None,
'B': 377,
'C': None,
'D': 67,
'E': None,
'F': 1,
'G': None
},
'prod.product': {
'Alk': None,
'Bay': 377,
'Lent': None,
'R': 67,
'Ter': None,
'Wi': 1,
'e': None
}
},
'duct_list.new_users': {
'pdust.product': {
'H': None,
'y': 377,
'nt': None,
'C': 67,
'sfer': None,
's': 1,
'le': None
}
}
}
Does Pandas or any other library has a way to do this. Structure of the nested dictionary is dynamic so we won't know how deep it is. And Keys will also change, so we won't able to know beforehand what are the keys in the dictionary. Any help will be appreciated. Thanks!!
If you allow the lower level tag labels to take prefixes of higher level tag labels, you can use the Pandas function pandas.json_normalize, which handles nested dict and turn it into a flat table Pandas dataframe.
Then, use pandas.DataFrame.to_dict to turn the Pandas dataframe to a dict. For example,
import pandas as pd
d = {
'product_list.show_date': "May '21",
'product_list.new_users':
{
'product_list.product':
{
'A': None,
'B': 377,
'C': None,
'D': 67,
'E': None,
'F': 1,
'G': None
}
}
}
pd.json_normalize(d).to_dict('records')[0]
Result:
{'product_list.show_date': "May '21",
'product_list.new_users.product_list.product.A': None,
'product_list.new_users.product_list.product.B': 377,
'product_list.new_users.product_list.product.C': None,
'product_list.new_users.product_list.product.D': 67,
'product_list.new_users.product_list.product.E': None,
'product_list.new_users.product_list.product.F': 1,
'product_list.new_users.product_list.product.G': None}
Related
Lets say I have this dict:
dictos = {
"a": {
"b": {
"c": 3,
"d": 4,
"e": [],
"f": [{"g": 5}, 'test', {"h": 6, "i": 7}]
}
}
}
And lets say I want to delete "c": 3 pair. What I am doing:
import dpath.util
path = "a/b/c"
dpath.util.delete(dictos, path)
It is working great. The output is:
{
'a': {
'b': {
'd': 4,
'e': [],
'f': [{'g': 5}, 'test', {'h': 6, 'i': 7}]
}
}
}
The problem is when I am trying to delete key:value pair inside the list.
Lets say I want to delete "h":6. So when doing:
path = "a/b/f[2]/h"
dpath.util.delete(dictos, path)
I am getting:
dpath.exceptions.PathNotFound: Could not find a/b/f[2]/h to delete
it.
So the question basically is how to delete items from nested dicts that are in a list?
It seems the library expects the same separator to be used for all segments i.e. use a/b/f/2/h
path = "a/b/f/2/h"
dpath.util.delete(dictos, path)
print(dictos)
Result:
{'a': {'b': {'d': 4, 'e': [], 'f': [{'g': 5}, 'test', {'i': 7}]}}}
Consider this input dictionary:
my_dict = {
'group1':{
'type1': {'val1' : 45, 'val2' : 12, 'val3' : 65},
'type2': {'val5' : 65, 'val6' : 132, 'val7' : 656},
},
'group2':{
'type3': {'val11' : 45, 'val12' : 123, 'val13' : 3},
'type4': {'val51' : 1, 'val61' : 2, 'val71' : 3, },
},
}
I would like to remove the last 'level' (the one that only has numbers), and get something like:
new_dict = {
'group1':{
'type1': ['val1', 'val2', 'val3'],
'type2': ['val5', 'val6', 'val7'],
},
'group2':{
'type3': ['val11', 'val12', 'val13'],
'type4': ['val51', 'val61', 'val71'],
},
}
I am currently doing it by manually looping and so on, but I wonder if there is a way to do it more efficiently.
If your dictionary doesn't have a fixed number of nesting levels, you could write a recursive function to do it:
def stripBottom(d):
if not isinstance(d,dict):
return d
elif not any(isinstance(v,dict) for v in d.values()):
return list(d)
else:
return {k:stripBottom(v) for k,v in d.items()}
Output:
my_dict = {
'group1':{
'type1': {'val1' : 45, 'val2' : 12, 'val3' : 65},
'type2': {'val5' : 65, 'val6' : 132, 'val7' : 656},
},
'group2':{
'type3': {'val11' : 45, 'val12' : 123, 'val13' : 3},
'type4': {'val51' : 1, 'val61' : 2, 'val71' : 3, },
},
}
print(stripBottom(my_dict))
{'group1': {'type1': ['val1', 'val2', 'val3'],
'type2': ['val5', 'val6', 'val7']},
'group2': {'type3': ['val11', 'val12', 'val13'],
'type4': ['val51', 'val61', 'val71']}}
If your dictionary has variable levels of nesting or if you want to drop a specific level, you would need to add a parameter to the function:
def stripBottom(d,level=0):
if not isinstance(d,dict):
return d
elif level == 0:
return list(d)
else:
return {k:stripBottom(v,level-1) for k,v in d.items()}
print(stripBottom(my_dict,1))
{'group1': ['type1', 'type2'], 'group2': ['type3', 'type4']}
You don't give an example of the actual code you are using to change the dict, but likely you will always need 2 levels of looping to iterate over groups, and then types. Something like:
newdict = {}
for k, grp in my_dict.items():
newdict[k] = {}
for typ, val in grp.items():
newdict[k][typ] = list(val)
One option to be more efficient is to not pre-compute these updted values, but instead when you come to use the 'group' data, wrap the output in list at that point. (Though obviously you will still be iterating when the dict is being used, to get these nested dicts).
I have a input request which consists of multiple dictionaries with similar keys and values as dictionaries.
Here's the input I have.
req = {"main1":
{"x": {"a":220},"y": {"b":66}},
"main2":
{"x": {"c":"1000","d":"copper"},
"y": {"c":"1200","d":"Copper"}}}
Output I need:
{'cable1': {'a': 220, 'c': '1000', 'd': 'copper'}, 'cable2': {'b': 66, 'c': '1200', 'd': 'Copper'}}
Here's the try i made:
actual_req = []
for attr1, attr2 in req.items():
for j, k in attr2.items():
actual_req.append(k)
actual_req[0].update(actual_req[2])
actual_req[1].update(actual_req[3])
data = {'cable1': actual_req[0], 'cable2': actual_req[1]}
print(data)
I just need the more generic way of writing in short. It should also handle input with
req = {"main1":
{"x": {"a":220},"y": {"b":66}},
"main2":
{"x": {"c":"1000","d":"cooper"},
"y": {"c":"1200","d":"Copper"}},
"main3":
{"x": {"e":20},"y": {"f":6}}}
Just update final dictionary as you go through values of original dictionary:
req = {"main1":
{"x": {"a":220},"y": {"b":66}},
"main2":
{"x": {"c":"1000","d":"copper"},
"y": {"c":"1200","d":"Copper"}}}
final_dict = {'cable1': {}, 'cable2': {}}
for v in req.values():
final_dict['cable1'].update(v['x'])
final_dict['cable2'].update(v['y'])
Final dict would look like this:
{'cable1': {'a': 220, 'c': '1000', 'd': 'copper'}, 'cable2': {'b': 66, 'c': '1200', 'd': 'Copper'}}
Using Python's requests package, I have a body for a POST that has a bunch of None values, ex (dictionary, not JSON):
{
'name': 'John',
'surname': None,
'somelist': [
{
'a': 1,
'b': None
},
{
'a': None,
'b': 2
}
],
'otherdict': {
'c': False,
'd': None
}
}
and I would like the actual body that gets sent out has all the entries with values None removed, as opposed to converted to JSON null:
{
'name': 'John',
'somelist': [
{
'a': 1
},
{
'b': 2
}
],
'otherdict': {
'c': False
}
}
Does the requests package have an option that does this or do I need to do it on my end?
You can do this with recursion (remove keys with None values or recursively call the clean method if the value is not None):
def clean(d):
if type(d) == list:
return [clean(e) for e in d]
elif type(d) == dict:
for k, v in list(d.items()):
if v is None:
del d[k]
else:
d[k] = clean(v)
return d
print(clean(data))
Output
{'somelist': [{'a': 1}, {'b': 2}], 'otherdict': {'c': False}, 'name': 'John'}
heres my dict:
{
1: { 'A': u'Eggs',
'B': 1400,
'C': u'Jibber',
'D': u'355'},
2: { 'A': u'Avocados',
'B': 1000,
'C': u'Jabber',
'D': u'356'},
..}
Template.mustache
{{#each}}
<li><strong>{{A}}</strong></li>
<li><strong>{{B}}</strong></li>
...
{{/each}}
{{#empty}}
<p>The list is empty.</p>
{{/empty}}
If the variable is called data, how do I iterate over its items if they are only identifiable by index?