Related
I have a nested dictionary ( i.e. sample_dict), where for each day, we need to find the smallest three values (in ascending manner), after which the result has to be stored in a new dictionary.
The sample_dict is as follows:
sample_dict ={ '2020-12-22': {'A': 0.0650,'B': 0.2920, 'C': 0.0780, 'D': 1.28008, 'G': 3.122},
'2020-12-23': {'B': 0.3670, 'C': 0.4890, 'G':1.34235, 'H': 0.227731},
'2020-12-24': {'A': 0.3630, 'B': 0.3960, 'C': 0.0950, 'Z':0.3735},
'2020-12-25': {'C': 0.8366, 'B': 0.4840},
'2020-12-26': {'Y': 5.366}}
The final dictionary (i.e. result) after selecting the smallest three for each date would look like:
Can someone suggest a solution using for loops.
Let's use heapq.nsmallest inside a dictionary comprehension to select the smallest 3 items per subdict:
from operator import itemgetter
import heapq
for k, v in sample_dict.items():
# Look ma, no `sorted`
sample_dict[k] = dict(heapq.nsmallest(3, v.items(), key=itemgetter(1)))
print (sample_dict)
# {'2020-12-22': {'A': 0.065, 'C': 0.078, 'B': 0.292},
# '2020-12-23': {'H': 0.227731, 'B': 0.367, 'C': 0.489},
# '2020-12-24': {'C': 0.095, 'A': 0.363, 'Z': 0.3735},
# '2020-12-25': {'B': 0.484, 'C': 0.8366},
# '2020-12-26': {'Y': 5.366}}
This is pretty fast because it does not need to sort the array, and updates sample_dict in-place.
Try using this dictionary comprehension:
print({k: dict(sorted(sorted(v.items(), key=lambda x: x[1]), key=lambda x: x[0])[:3]) for k, v in sample_dict.items()})
Output:
{'2020-12-22': {'A': 0.065, 'B': 0.292, 'C': 0.078}, '2020-12-23': {'B': 0.367, 'C': 0.489, 'G': 1.34235}, '2020-12-24': {'A': 0.363, 'B': 0.396, 'C': 0.095}, '2020-12-25': {'B': 0.484, 'C': 0.8366}, '2020-12-26': {'Y': 5.366}}
This should work for your purposes.
sample_dict = {'2020-12-22': {'A': 0.0650, 'B': 0.2920, 'C': 0.0780, 'D': 1.28008, 'G': 3.122},
'2020-12-23': {'B': 0.3670, 'C': 0.4890, 'G':1.34235, 'H': 0.227731},
'2020-12-24': {'A': 0.3630, 'B': 0.3960, 'C': 0.0950, 'Z':0.3735},
'2020-12-25': {'C': 0.8366, 'B': 0.4840},
'2020-12-26': {'Y': 5.366}}
results_dict = {day[0]:{sample[0]:sample[1] for sample in sorted(day[1].items(), key=lambda e: e[1])[:3]} for day in sample_dict.items()}
# Output
{'2020-12-22': {'A': 0.065, 'B': 0.292, 'C': 0.078},
'2020-12-23': {'B': 0.367, 'C': 0.489, 'H': 0.227731},
'2020-12-24': {'A': 0.363, 'C': 0.095, 'Z': 0.3735},
'2020-12-25': {'B': 0.484, 'C': 0.8366},
'2020-12-26': {'Y': 5.366}}
List of dictionaries:
data = [{
'a':{'l':'Apple',
'b':'Milk',
'd':'Meatball'},
'b':{'favourite':'coke',
'dislike':'juice'}
},
{
'a':{'l':'Apple1',
'b':'Milk1',
'd':'Meatball2'},
'b':{'favourite':'coke2',
'dislike':'juice3'}
}, ...
]
I need to join all nested dictionaries to reach at the expected output:
[{'d': 'Meatball', 'b': 'Milk', 'l': 'Apple', 'dislike': 'juice', 'favourite': 'coke'},
{'d': 'Meatball2', 'b': 'Milk1', 'l': 'Apple1', 'dislike': 'juice3', 'favourite': 'coke2'}]
I try nested list comprehension, but cannot join dict together:
L = [y for x in data for y in x.values()]
print (L)
[{'d': 'Meatball', 'b': 'Milk', 'l': 'Apple'},
{'dislike': 'juice', 'favourite': 'coke'},
{'d': 'Meatball2', 'b': 'Milk1', 'l': 'Apple1'},
{'dislike': 'juice3', 'favourite': 'coke2'}]
I am looking for the fastest solution.
You can do the following, using itertools.chain:
>>> from itertools import chain
# timeit: ~3.40
>>> [dict(chain(*map(dict.items, d.values()))) for d in data]
[{'l': 'Apple',
'b': 'Milk',
'd': 'Meatball',
'favourite': 'coke',
'dislike': 'juice'},
{'l': 'Apple1',
'b': 'Milk1',
'dislike': 'juice3',
'favourite': 'coke2',
'd': 'Meatball2'}]
The usage of chain, map, * make this expression a shorthand for the following doubly nested comprehension which actually performs better on my system (Python 3.5.2) and isn't that much longer:
# timeit: ~2.04
[{k: v for x in d.values() for k, v in x.items()} for d in data]
# Or, not using items, but lookup by key
# timeit: ~1.67
[{k: x[k] for x in d.values() for k in x} for d in data]
Note:
RoadRunner's loop-and-update approach outperforms both these one-liners at timeit: ~1.37
You can do this with 2 nested loops, and dict.update() to add inner dictionaries to a temporary dictionary and add it at the end:
L = []
for d in data:
temp = {}
for key in d:
temp.update(d[key])
L.append(temp)
# timeit ~1.4
print(L)
Which Outputs:
[{'l': 'Apple', 'b': 'Milk', 'd': 'Meatball', 'favourite': 'coke', 'dislike': 'juice'}, {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2', 'favourite': 'coke2', 'dislike': 'juice3'}]
You can use functools.reduce along with a simple list comprehension to flatten out the list the of dicts
>>> from functools import reduce
>>> data = [{'b': {'dislike': 'juice', 'favourite': 'coke'}, 'a': {'l': 'Apple', 'b': 'Milk', 'd': 'Meatball'}}, {'b': {'dislike': 'juice3', 'favourite': 'coke2'}, 'a': {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2'}}]
>>> [reduce(lambda x,y: {**x,**y},d.values()) for d in data]
>>> [{'dislike': 'juice', 'l': 'Apple', 'd': 'Meatball', 'b': 'Milk', 'favourite': 'coke'}, {'dislike': 'juice3', 'l': 'Apple1', 'd': 'Meatball2', 'b': 'Milk1', 'favourite': 'coke2'}]
Time benchmark is as follows:
>>> import timeit
>>> setup = """
from functools import reduce
data = [{'b': {'dislike': 'juice', 'favourite': 'coke'}, 'a': {'l': 'Apple', 'b': 'Milk', 'd': 'Meatball'}}, {'b': {'dislike': 'juice3', 'favourite': 'coke2'}, 'a': {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2'}}]
"""
>>> min(timeit.Timer("[reduce(lambda x,y: {**x,**y},d.values()) for d in data]",setup=setup).repeat(3,1000000))
>>> 1.525032774952706
Time benchmark of other answers on my machine
>>> setup = """
data = [{'b': {'dislike': 'juice', 'favourite': 'coke'}, 'a': {'l': 'Apple', 'b': 'Milk', 'd': 'Meatball'}}, {'b': {'dislike': 'juice3', 'favourite': 'coke2'}, 'a': {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2'}}]
"""
>>> min(timeit.Timer("[{k: v for x in d.values() for k, v in x.items()} for d in data]",setup=setup).repeat(3,1000000))
>>> 2.2488374650129117
>>> min(timeit.Timer("[{k: x[k] for x in d.values() for k in x} for d in data]",setup=setup).repeat(3,1000000))
>>> 1.8990078769857064
>>> code = """
L = []
for d in data:
temp = {}
for key in d:
temp.update(d[key])
L.append(temp)
"""
>>> min(timeit.Timer(code,setup=setup).repeat(3,1000000))
>>> 1.4258553800173104
>>> setup = """
from itertools import chain
data = [{'b': {'dislike': 'juice', 'favourite': 'coke'}, 'a': {'l': 'Apple', 'b': 'Milk', 'd': 'Meatball'}}, {'b': {'dislike': 'juice3', 'favourite': 'coke2'}, 'a': {'l': 'Apple1', 'b': 'Milk1', 'd': 'Meatball2'}}]
"""
>>> min(timeit.Timer("[dict(chain(*map(dict.items, d.values()))) for d in data]",setup=setup).repeat(3,1000000))
>>> 3.774383604992181
If you have nested dictionaries with only 'a' and 'b' keys, then I suggest the following solution I find fast and very easy to understand (for readability purpose):
L = [x['a'] for x in data]
b = [x['b'] for x in data]
for i in range(len(L)):
L[i].update(b[i])
# timeit ~1.4
print(L)
I've got the below dictionary and list - how do I loop over the dictionary checking if b == '1' while passing '1' as variable from a list?
dic = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
lst = ['1']
I want to return {'a':0, 'b':'1'}, {'a':0, 'b':'1'}.
This is a general solution using filter; the built-in method, you will have to adopt it to your needs:
>>> list(filter(lambda d: d['b'] in lst, dic['info']))
[{'b': '1', 'a': 0}, {'b': '1', 'a': 0}]
Converting the filter object into a list using list constructor is necessary only in Python3, whereas in Python2, it is not required:
>>> filter(lambda d: d['b'] in lst, dic['info'])
[{'b': '1', 'a': 0}, {'b': '1', 'a': 0}]
EDIT: To make the solution more general in case multiple items in lst, then consider the following:
>>> dic
{'info': [{'b': '1', 'a': 0}, {'b': '3', 'a': 0}, {'b': '3', 'a': 0}, {'b': '1', 'a': 0}, {'b': '2', 'a': '1'}]}
>>>
>>> lst
['1', '2']
>>> def filter_dict(dic_lst, lst):
lst_out = []
for sub_d in dic_lst:
if any(x == sub_d['b'] for x in lst):
lst_out.append(sub_d)
return lst_out
>>> filter_dict(dic['info'], lst)
[{'b': '1', 'a': 0}, {'b': '1', 'a': 0}, {'b': '2', 'a': '1'}]
OR:
>>> list(map(lambda x: list(filter(lambda d: d['b'] in x, dic['info'])),lst))
[[{'b': '1', 'a': 0}, {'b': '1', 'a': 0}], [{'b': '2', 'a': '1'}]]
Just a simple list comprehension:
In [22]: dic = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
In [23]: lst = ['1']
In [25]: [sub_dict for sub_dict in dic['info'] if sub_dict['b'] == lst[0]]
Out[25]: [{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}]
You could use a filter approach:
filter(lambda x:x['b'] in list, dic['info'])
It will create a generator which you can materialize in a list:
result = list(filter(lambda x:x['b'] in list, dic['info']))
Mind I would however rename your list variable since you here override a reference to the list type.
from collections import defaultdict
dic = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
d = defaultdict(list)
for each in dic['info']:
d[each['b']].append(each)
out:
defaultdict(list,
{'1': [{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}],
'3': [{'a': 0, 'b': '3'}, {'a': 0, 'b': '3'}]})
in:
d['1']
out:
[{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}]
Build an index dict to avoid iterate again.
First go my simple loop and iteration way
Input:
>>> dic
{'info': [{'a': 0, 'b': '1'}, {'a': 0, 'b': '3'}, {'a': 0, 'b': '3'}, {'a': 0, 'b': '1'}]}
>>> l
['1']
New List variable for result.
>>> result = []
Algo
Iterate diction by iteritems method of dictionary.
Value of main dictionary is list data type. so again iterate list by for loop.
Check b key is present in sub dictionary and check its value is present in given list l.
If yes, then append to result list.
code:
>>> for k,v in dic.iteritems():
... for i in v:
... if "b" in i and i["b"] in l:
... result.append(i)
...
Output:
>>> result
[{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}]
>>>
Notes:
Do not use list as variable name because list is reversed keyword for Python
Read basic things of dictionary and list which has properties.
Try to write code first.
You can make use of a list comprehension, or just do it using filter.
list comprehension
dict = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
lst = ['1']
result = [i for i in dict['info'] if i['b'] == lst[0]]
print result # [{'a': 0, 'b': '1'}, {'a': 0, 'b': '1'}]
filter
dict = {'info': [{'a':0, 'b':'1'},{'a':0, 'b':'3'},{'a':0, 'b':'3'},{'a':0, 'b':'1'}]}
list(filter(lambda i: i['b'] in lst, dic['info']))
# [{'b': '1', 'a': 0}, {'b': '1', 'a': 0}]
I'd like to find a way to print a list of dictionnaries line by line, so that the result be clear and easy to read
the list is like this.
myList = {'1':{'name':'x',age:'18'},'2':{'name':'y',age:'19'},'3':{'name':'z',age:'20'}...}
and the result should be like this:
>>> '1':{'name':'x',age:'18'}
'2':{'name':'y',age:'19'}
'3':{'name':'z',age:'20'} ...
Using your example:
>>> myList = {'1':{'name':'x','age':'18'},'2':{'name':'y','age':'19'},'3':{'name':'z','age':'20'}}
>>> for k, d in myList.items():
print k, d
1 {'age': '18', 'name': 'x'}
3 {'age': '20', 'name': 'z'}
2 {'age': '19', 'name': 'y'}
More examples:
A list of dictionaries:
>>> l = [{'a':'1'},{'b':'2'},{'c':'3'}]
>>> for d in l:
print d
{'a': '1'}
{'b': '2'}
{'c': '3'}
A dictionary of dictionaries:
>>> D = {'d1': {'a':'1'}, 'd2': {'b':'2'}, 'd3': {'c':'3'}}
>>> for k, d in D.items():
print d
{'b': '2'}
{'c': '3'}
{'a': '1'}
If you want the key of the dicts:
>>> D = {'d1': {'a':'1'}, 'd2': {'b':'2'}, 'd3': {'c':'3'}}
>>> for k, d in D.items():
print k, d
d2 {'b': '2'}
d3 {'c': '3'}
d1 {'a': '1'}
>>> import json
>>> dicts = {1: {'a': 1, 'b': 2}, 2: {'c': 3}, 3: {'d': 4, 'e': 5, 'f':6}}
>>> print(json.dumps(dicts, indent=4))
{
"1": {
"a": 1,
"b": 2
},
"2": {
"c": 3
},
"3": {
"d": 4,
"e": 5,
"f": 6
}
}
One more option - pprint, made for pretty-printing.
The pprint module provides a capability to “pretty-print” arbitrary Python data structures in a form which can be used as input to the interpreter.
List of dictionaries:
from pprint import pprint
l = [{'a':'1'},{'b':'2'},{'c':'3'}]
pprint(l, width=1)
Output:
[{'a': '1'},
{'b': '2'},
{'c': '3'}]
Dictionary with dictionaries in values:
from pprint import pprint
d = {'a':{'b':'c'}},{'d':{'e':'f'}}
pprint(d, width=1)
Output:
({'a': {'b': 'c'}},
{'d': {'e': 'f'}})
myList = {'1':{'name':'x','age':'18'},
'2':{'name':'y','age':'19'},
'3':{'name':'z','age':'20'}}
for item in myList:
print(item,':',myList[item])
Output:
3 : {'age': '20', 'name': 'z'}
2 : {'age': '19', 'name': 'y'}
1 : {'age': '18', 'name': 'x'}
item is used to iterate keys in the dict, and myList[item] is the value corresponding to the current key.
I was writing a Python Code that creates dictionaries dynamically,initializes it to a reference dictionary, and modifying a particular value in the dictionary. But,I found that not only I am getting unexpected results,but the reference dictionary is also getting modified.
My Code:
tdict={'a':'1','b':'2','c':'3'}
newdict={}
for i in range(5):
newdict['name'+str(i)]=tdict
newdict['name'+str(i)]['a']='value'+str(i)
print 'tdict: ',tdict
print 'newdict: ',newdict
And the result:
tdict: {'a': 'value0', 'c': '3', 'b': '2'}
tdict: {'a': 'value1', 'c': '3', 'b': '2'}
tdict: {'a': 'value2', 'c': '3', 'b': '2'}
tdict: {'a': 'value3', 'c': '3', 'b': '2'}
tdict: {'a': 'value4', 'c': '3', 'b': '2'}
newdict: {'name4': {'a': 'value4', 'c': '3', 'b': '2'}, 'name2': {'a': 'value4', 'c': '3', 'b': '2'}, 'name3': {'a': 'value4', 'c': '3', 'b': '2'}, 'name0': {'a': 'value4', 'c': '3', 'b': '2'}, 'name1': {'a': 'value4', 'c': '3', 'b': '2'}}
whereas I expected my 'newdict' to be like:
newdict: {'name4': {'a': 'value4', 'c': '3', 'b': '2'}, 'name2': {'a': 'value2', 'c': '3', 'b': '2'}, 'name3': {'a': 'value3', 'c': '3', 'b': '2'}, 'name0': {'a': 'value0', 'c': '3', 'b': '2'}, 'name1': {'a': 'value1', 'c': '3', 'b': '2'}}
Can anyone please help me figuring out why this is happening? Also, why is the reference dictionary 'tdict' getting changed when I am not assigning any any value to it?
Thanks in advance
You are storing a reference to tdict in every value of your newdict dictionary:
newdict['name'+str(i)]=tdict
You are then modifying the key 'a' of tdict by doing
# newdict['name'+str(i)] is a reference to tdict
newdict['name'+str(i)]['a']='value'+str(i)
# this is equivalent to doing
tdict['a']='value'+str(i)
What you maybe want is storing a copy of tdict in your newdict dictionary:
newdict['name'+str(i)]=dict(tdict)
Creating a new dictionary by using an existing dictionary as constructor argument creates a shallow copy where you can assign new values to existing keys. What you cannot (or what you don't want) is modifying mutable values in this dictionary. Example:
>>> a={'a': 1, 'b': 2, 'c': [1,2,3]}
>>> b=dict(a)
>>> b['a']=9
>>> a
{'a': 1, 'c': [1, 2, 3], 'b': 2}
>>> b
{'a': 9, 'c': [1, 2, 3], 'b': 2}
>>> b['c'].append(99)
>>> a
{'a': 1, 'c': [1, 2, 3, 99], 'b': 2}
>>> b
{'a': 9, 'c': [1, 2, 3, 99], 'b': 2}
If you want to modify mutable values in a dictionary you need to create a deep copy:
>>> import copy
>>> a={'a': 1, 'b': 2, 'c': [1,2,3]}
>>> b=copy.deepcopy(a)
>>> b['a']=9
>>> b['c'].append(99)
>>> a
{'a': 1, 'c': [1, 2, 3], 'b': 2}
>>> b
{'a': 9, 'c': [1, 2, 3, 99], 'b': 2}
Is just cause you are making a reference to tdict and not a copy. In order to copy you can either use
newdict['name'+str(i)] = tdict.copy()
or
newdict['name'+str(i)] = dict(tdict)
Hope it helps