List of dictionaries:
data = [{
'a':{'l':'Apple',
'b':'Milk',
'd':'Meatball'},
'b':{'favourite':'coke',
'dislike':'juice'}
},
{
'a':{'l':'Apple1',
'b':'Milk1',
'd':'Meatball2'},
'b':{'favourite':'coke2',
'dislike':'juice3'}
}, ...
]
I need to join all nested dictionaries to reach at the expected output:
[{'d': 'Meatball', 'b': 'Milk', 'l': 'Apple', 'dislike': 'juice', 'favourite': 'coke'},
{'d': 'Meatball2', 'b': 'Milk1', 'l': 'Apple1', 'dislike': 'juice3', 'favourite': 'coke2'}]
I try nested list comprehension, but cannot join dict together:
L = [y for x in data for y in x.values()]
print (L)
[{'d': 'Meatball', 'b': 'Milk', 'l': 'Apple'},
{'dislike': 'juice', 'favourite': 'coke'},
{'d': 'Meatball2', 'b': 'Milk1', 'l': 'Apple1'},
{'dislike': 'juice3', 'favourite': 'coke2'}]
I am looking for the fastest solution.
You can do the following, using itertools.chain:
>>> from itertools import chain
# timeit: ~3.40
>>> [dict(chain(*map(dict.items, d.values()))) for d in data]
[{'l': 'Apple',
'b': 'Milk',
'd': 'Meatball',
'favourite': 'coke',
'dislike': 'juice'},
{'l': 'Apple1',
'b': 'Milk1',
'dislike': 'juice3',
'favourite': 'coke2',
'd': 'Meatball2'}]
The usage of chain, map, * make this expression a shorthand for the following doubly nested comprehension which actually performs better on my system (Python 3.5.2) and isn't that much longer:
# timeit: ~2.04
[{k: v for x in d.values() for k, v in x.items()} for d in data]
# Or, not using items, but lookup by key
# timeit: ~1.67
[{k: x[k] for x in d.values() for k in x} for d in data]
Note:
RoadRunner's loop-and-update approach outperforms both these one-liners at timeit: ~1.37
You can do this with 2 nested loops, and dict.update() to add inner dictionaries to a temporary dictionary and add it at the end:
L = []
for d in data:
temp = {}
for key in d:
temp.update(d[key])
L.append(temp)
# timeit ~1.4
print(L)
Which Outputs:
[{'l': 'Apple', 'b': 'Milk', 'd': 'Meatball', 'favourite': 'coke', 'dislike': 'juice'}, {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2', 'favourite': 'coke2', 'dislike': 'juice3'}]
You can use functools.reduce along with a simple list comprehension to flatten out the list the of dicts
>>> from functools import reduce
>>> data = [{'b': {'dislike': 'juice', 'favourite': 'coke'}, 'a': {'l': 'Apple', 'b': 'Milk', 'd': 'Meatball'}}, {'b': {'dislike': 'juice3', 'favourite': 'coke2'}, 'a': {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2'}}]
>>> [reduce(lambda x,y: {**x,**y},d.values()) for d in data]
>>> [{'dislike': 'juice', 'l': 'Apple', 'd': 'Meatball', 'b': 'Milk', 'favourite': 'coke'}, {'dislike': 'juice3', 'l': 'Apple1', 'd': 'Meatball2', 'b': 'Milk1', 'favourite': 'coke2'}]
Time benchmark is as follows:
>>> import timeit
>>> setup = """
from functools import reduce
data = [{'b': {'dislike': 'juice', 'favourite': 'coke'}, 'a': {'l': 'Apple', 'b': 'Milk', 'd': 'Meatball'}}, {'b': {'dislike': 'juice3', 'favourite': 'coke2'}, 'a': {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2'}}]
"""
>>> min(timeit.Timer("[reduce(lambda x,y: {**x,**y},d.values()) for d in data]",setup=setup).repeat(3,1000000))
>>> 1.525032774952706
Time benchmark of other answers on my machine
>>> setup = """
data = [{'b': {'dislike': 'juice', 'favourite': 'coke'}, 'a': {'l': 'Apple', 'b': 'Milk', 'd': 'Meatball'}}, {'b': {'dislike': 'juice3', 'favourite': 'coke2'}, 'a': {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2'}}]
"""
>>> min(timeit.Timer("[{k: v for x in d.values() for k, v in x.items()} for d in data]",setup=setup).repeat(3,1000000))
>>> 2.2488374650129117
>>> min(timeit.Timer("[{k: x[k] for x in d.values() for k in x} for d in data]",setup=setup).repeat(3,1000000))
>>> 1.8990078769857064
>>> code = """
L = []
for d in data:
temp = {}
for key in d:
temp.update(d[key])
L.append(temp)
"""
>>> min(timeit.Timer(code,setup=setup).repeat(3,1000000))
>>> 1.4258553800173104
>>> setup = """
from itertools import chain
data = [{'b': {'dislike': 'juice', 'favourite': 'coke'}, 'a': {'l': 'Apple', 'b': 'Milk', 'd': 'Meatball'}}, {'b': {'dislike': 'juice3', 'favourite': 'coke2'}, 'a': {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2'}}]
"""
>>> min(timeit.Timer("[dict(chain(*map(dict.items, d.values()))) for d in data]",setup=setup).repeat(3,1000000))
>>> 3.774383604992181
If you have nested dictionaries with only 'a' and 'b' keys, then I suggest the following solution I find fast and very easy to understand (for readability purpose):
L = [x['a'] for x in data]
b = [x['b'] for x in data]
for i in range(len(L)):
L[i].update(b[i])
# timeit ~1.4
print(L)
Related
I have a nested dictionary ( i.e. sample_dict), where for each day, we need to find the smallest three values (in ascending manner), after which the result has to be stored in a new dictionary.
The sample_dict is as follows:
sample_dict ={ '2020-12-22': {'A': 0.0650,'B': 0.2920, 'C': 0.0780, 'D': 1.28008, 'G': 3.122},
'2020-12-23': {'B': 0.3670, 'C': 0.4890, 'G':1.34235, 'H': 0.227731},
'2020-12-24': {'A': 0.3630, 'B': 0.3960, 'C': 0.0950, 'Z':0.3735},
'2020-12-25': {'C': 0.8366, 'B': 0.4840},
'2020-12-26': {'Y': 5.366}}
The final dictionary (i.e. result) after selecting the smallest three for each date would look like:
Can someone suggest a solution using for loops.
Let's use heapq.nsmallest inside a dictionary comprehension to select the smallest 3 items per subdict:
from operator import itemgetter
import heapq
for k, v in sample_dict.items():
# Look ma, no `sorted`
sample_dict[k] = dict(heapq.nsmallest(3, v.items(), key=itemgetter(1)))
print (sample_dict)
# {'2020-12-22': {'A': 0.065, 'C': 0.078, 'B': 0.292},
# '2020-12-23': {'H': 0.227731, 'B': 0.367, 'C': 0.489},
# '2020-12-24': {'C': 0.095, 'A': 0.363, 'Z': 0.3735},
# '2020-12-25': {'B': 0.484, 'C': 0.8366},
# '2020-12-26': {'Y': 5.366}}
This is pretty fast because it does not need to sort the array, and updates sample_dict in-place.
Try using this dictionary comprehension:
print({k: dict(sorted(sorted(v.items(), key=lambda x: x[1]), key=lambda x: x[0])[:3]) for k, v in sample_dict.items()})
Output:
{'2020-12-22': {'A': 0.065, 'B': 0.292, 'C': 0.078}, '2020-12-23': {'B': 0.367, 'C': 0.489, 'G': 1.34235}, '2020-12-24': {'A': 0.363, 'B': 0.396, 'C': 0.095}, '2020-12-25': {'B': 0.484, 'C': 0.8366}, '2020-12-26': {'Y': 5.366}}
This should work for your purposes.
sample_dict = {'2020-12-22': {'A': 0.0650, 'B': 0.2920, 'C': 0.0780, 'D': 1.28008, 'G': 3.122},
'2020-12-23': {'B': 0.3670, 'C': 0.4890, 'G':1.34235, 'H': 0.227731},
'2020-12-24': {'A': 0.3630, 'B': 0.3960, 'C': 0.0950, 'Z':0.3735},
'2020-12-25': {'C': 0.8366, 'B': 0.4840},
'2020-12-26': {'Y': 5.366}}
results_dict = {day[0]:{sample[0]:sample[1] for sample in sorted(day[1].items(), key=lambda e: e[1])[:3]} for day in sample_dict.items()}
# Output
{'2020-12-22': {'A': 0.065, 'B': 0.292, 'C': 0.078},
'2020-12-23': {'B': 0.367, 'C': 0.489, 'H': 0.227731},
'2020-12-24': {'A': 0.363, 'C': 0.095, 'Z': 0.3735},
'2020-12-25': {'B': 0.484, 'C': 0.8366},
'2020-12-26': {'Y': 5.366}}
I have a dictionary as follows:
Each key has a dictionary associated with it.
dict_sample = {'a': {'d0': '1', 'd1': '2', 'd2': '3'}, 'b': {'d0': '1'}, 'c': {'d1': '1'}}
I need the output as follows:
output_dict = {'d0': {'a': 1, 'b': 1}, 'd1': {'a': 2, 'c': 1}, 'd2': {'a': 3}}
I'd appreciate any help on the pythonic way to achieve this. Thank You !
I believe this produces the desired output
>>> from collections import defaultdict
>>> d = defaultdict(dict)
>>>
>>> dict_sample = {'a': {'d0': '1', 'd1': '2', 'd2': '3'}, 'b': {'d0': '1'}, 'c': {'d1': '1'}}
>>>
>>> for key, value in dict_sample.items():
... for k, v in value.items():
... d[k][key] = v
...
>>> d
defaultdict(<class 'dict'>, {'d0': {'a': '1', 'b': '1'}, 'd1': {'a': '2', 'c': '1'}, 'd2': {'a': '3'}})
You can use dict.setdefault on a new dict with a nested loop:
d = {}
# for each key and sub-dict in the main dict
for k1, s in dict_sample.items():
# for each key and value in the sub-dict
for k2, v in s.items():
# this is equivalent to d[k2][k1] = int(v), except that when k2 is not yet in d,
# setdefault will initialize d[k2] with {} (a new dict)
d.setdefault(k2, {})[k1] = int(v)
d would become:
{'d0': {'a': 1, 'b': 1}, 'd1': {'a': 2, 'c': 1}, 'd2': {'a': 3}}
I have a dictionary:
oldDict = {'a': 'apple', 'b': 'boy', 'c': 'cat'}
I want a new dictionary with one of the values in the old dictionary as new key and all the elements as values:
newDict = {'apple': {'a': 'apple', 'b': 'boy', 'c': 'cat'}}
I tried doing this:
newDict['apple'] = oldDict
This does not seem to be working. Please note that I don't have a variable newDict in my script. I just have oldDict and I need to modify the same to make this change in every loop. In the end I will have one dictionary, which will look like this.
oldDict = {'apple': {'a': 'apple', 'b': 'boy', 'c': 'cat'}, 'dog': {'d': 'dog', 'e': 'egg'}}
You need to duplicate your dictionary so you won't create a circular reference.
>>> newDict = {'apple': {'a': 'apple', 'b': 'boy', 'c': 'cat'}}
>>> newDict['potato'] = dict(newDict)
>>> newDict
{'apple': {'a': 'apple', 'c': 'cat', 'b': 'boy'}, 'potato': {'apple': {'a': 'apple', 'c': 'cat', 'b': 'boy'}}}
you need to first create/declare the dictionary before you can added items into it
oldDict = {'a': 'apple', 'b': 'boy', 'c': 'cat'}
newDict = {}
newDict['apple'] = oldDict
# {'apple': {'a': 'apple', 'b': 'boy', 'c': 'cat'}}
# you can 1st create the newDict outside the loop then update it inside the loop
>>> newDict = {}
>>> dict1 = {'a': 'apple', 'b': 'boy', 'c': 'cat'}
>>> dict2 = {'d': 'dog', 'e': 'egg'}
>>> newDict['apple'] = dict1
>>> newDict['dog'] = dict2
>>> newDict
>>> {'apple': {'a': 'apple', 'b': 'boy', 'c': 'cat'}, 'dog': {'d': 'dog', 'e': 'egg'}}
or you could do as below
newDict = {'apple':oldDict}
I've got the below dictionary and list - how do I loop over the dictionary checking if b == '1' while passing '1' as variable from a list?
dic = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
lst = ['1']
I want to return {'a':0, 'b':'1'}, {'a':0, 'b':'1'}.
This is a general solution using filter; the built-in method, you will have to adopt it to your needs:
>>> list(filter(lambda d: d['b'] in lst, dic['info']))
[{'b': '1', 'a': 0}, {'b': '1', 'a': 0}]
Converting the filter object into a list using list constructor is necessary only in Python3, whereas in Python2, it is not required:
>>> filter(lambda d: d['b'] in lst, dic['info'])
[{'b': '1', 'a': 0}, {'b': '1', 'a': 0}]
EDIT: To make the solution more general in case multiple items in lst, then consider the following:
>>> dic
{'info': [{'b': '1', 'a': 0}, {'b': '3', 'a': 0}, {'b': '3', 'a': 0}, {'b': '1', 'a': 0}, {'b': '2', 'a': '1'}]}
>>>
>>> lst
['1', '2']
>>> def filter_dict(dic_lst, lst):
lst_out = []
for sub_d in dic_lst:
if any(x == sub_d['b'] for x in lst):
lst_out.append(sub_d)
return lst_out
>>> filter_dict(dic['info'], lst)
[{'b': '1', 'a': 0}, {'b': '1', 'a': 0}, {'b': '2', 'a': '1'}]
OR:
>>> list(map(lambda x: list(filter(lambda d: d['b'] in x, dic['info'])),lst))
[[{'b': '1', 'a': 0}, {'b': '1', 'a': 0}], [{'b': '2', 'a': '1'}]]
Just a simple list comprehension:
In [22]: dic = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
In [23]: lst = ['1']
In [25]: [sub_dict for sub_dict in dic['info'] if sub_dict['b'] == lst[0]]
Out[25]: [{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}]
You could use a filter approach:
filter(lambda x:x['b'] in list, dic['info'])
It will create a generator which you can materialize in a list:
result = list(filter(lambda x:x['b'] in list, dic['info']))
Mind I would however rename your list variable since you here override a reference to the list type.
from collections import defaultdict
dic = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
d = defaultdict(list)
for each in dic['info']:
d[each['b']].append(each)
out:
defaultdict(list,
{'1': [{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}],
'3': [{'a': 0, 'b': '3'}, {'a': 0, 'b': '3'}]})
in:
d['1']
out:
[{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}]
Build an index dict to avoid iterate again.
First go my simple loop and iteration way
Input:
>>> dic
{'info': [{'a': 0, 'b': '1'}, {'a': 0, 'b': '3'}, {'a': 0, 'b': '3'}, {'a': 0, 'b': '1'}]}
>>> l
['1']
New List variable for result.
>>> result = []
Algo
Iterate diction by iteritems method of dictionary.
Value of main dictionary is list data type. so again iterate list by for loop.
Check b key is present in sub dictionary and check its value is present in given list l.
If yes, then append to result list.
code:
>>> for k,v in dic.iteritems():
... for i in v:
... if "b" in i and i["b"] in l:
... result.append(i)
...
Output:
>>> result
[{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}]
>>>
Notes:
Do not use list as variable name because list is reversed keyword for Python
Read basic things of dictionary and list which has properties.
Try to write code first.
You can make use of a list comprehension, or just do it using filter.
list comprehension
dict = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
lst = ['1']
result = [i for i in dict['info'] if i['b'] == lst[0]]
print result # [{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}]
filter
dict = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
list(filter(lambda i: i['b'] in lst, dic['info']))
# [{'b': '1', 'a': 0}, {'b': '1', 'a': 0}]
I want to filter out last value in tuple in list of dictionaries. For example
X = [{'a': (123, 12445), 'b': (234, 34)}, {'a': (45666, 4), 'b': (3, 12)}]
to
Y = [{'a': 123, 'b': 234}, {'a': 45666, 'b': 3} ]
My attempt:
[{ j: i[j][0] } for j in i.keys() for i in result ]
But I get this:
[{'a': 123}, {'a': 45666}, {'b': 234}, {'b': 3}]
Any help?
Edit:
Dictionary comprehensions only work in Python 2.7+. Since you are on Python 2.6, you can use dict with a generator expression:
>>> X = [{'a': (123, 12445), 'b': (234, 34)}, {'a': (45666, 4), 'b': (3, 12)}]
>>> [dict((k, v[0]) for k,v in x.iteritems()) for x in X]
[{'b': 234, 'a': 123}, {'b': 3, 'a': 45666}]
>>>