I got something like this:
> d1 = {'System tests': {'failed': 5, 'passed': 0, 'total': 5},
'Func tests': {'failed': 5, 'passed': 0, 'total': 5}}
> d2 = {'System tests': {'failed': 1, 'passed': 1, 'total': 2},
'Func tests': {'failed': 3, 'passed': 2, 'total': 5}}
> d3 = {'System tests': {'failed': 0, 'passed': 0, 'total': 0},
'Func tests': {'failed': 1, 'passed': 0, 'total': 1}}
I would like to sum values 'failed', passed and total into one dictionary
so the output should be like this:
d4 = {'System tests': {'failed': 6, 'passed': 1, 'total': 7},
'Func tests': {'failed': 9, 'passed': 2, 'total': 11}
What is the easiest solution to do such thing?
I can use basic libraries, except collections.
The solution must be generic, for example if some other dictionaries would appear in the future
input:
d1 = {'a': 100, 'b': 200, 'c':300}
d2 = {'a': 300, 'b': 200, 'd':400}
d = {k : d1.get(k, 0) + d2.get(k,0) for k in set(d1.keys()) | set(d2.keys())}
Output:
{'a': 400, 'b': 400, 'c': 300, 'd': 400}
Using Counter and defaultdict that is fairly straight forward like:
Code:
d4 = defaultdict(Counter)
for d in d1, d2, d3:
for k, subd in d.items():
d4[k].update(subd)
Test Code:
d1 = {'System tests': {'failed': 5, 'passed': 0, 'total': 5},
'Func tests': {'failed': 5, 'passed': 0, 'total': 5}}
d2 = {'System tests': {'failed': 1, 'passed': 1, 'total': 2},
'Func tests': {'failed': 3, 'passed': 2, 'total': 5}}
d3 = {'System tests': {'failed': 0, 'passed': 0, 'total': 0},
'Func tests': {'failed': 1, 'passed': 0, 'total': 1}}
from collections import Counter, defaultdict
d4 = defaultdict(Counter)
for d in d1, d2, d3:
for k, subd in d.items():
d4[k].update(subd)
print(d4)
Results:
defaultdict(<class 'collections.Counter'>, {
'System tests': Counter({'total': 7, 'failed': 6, 'passed': 1}),
'Func tests': Counter({'total': 11, 'failed': 9, 'passed': 2})
})
This will give you the output you are looking for, with no libraries.
d4 = {}
for d in d1, d2, d3:
for test, results in d.items():
if test not in d4:
d4[test] = {}
for key, value in results.items():
if key in d4[test]:
d4[test][key] += value
else:
d4[test][key] = value
Result:
{'System tests': {'failed': 6, 'passed': 1, 'total': 7}, 'Func tests': {'failed': 9, 'passed': 2, 'total': 11}}
If you cannot use collections.Counter, another solution is to use reduce
from functools import reduce # reduce was moved here in Python3
def add_dicts(dict_list):
def add(fst, snd):
return {k: fst[k] + snd[k] for k in fst}
return reduce(add, dict_list[1:], dict_list[0])
Using this in your code would look like this.
dict_list = [d1, d2, d3]
d4 = {}
for k in dict_list[0]:
d4[k] = add_dicts([d[k] for d in dict_list])
Although, this assumes all your dict are correctly formated.
You can use zip along with dict.items:
d1 = {'System tests': {'failed': 5, 'passed': 0, 'total': 5},
'Func tests': {'failed': 5, 'passed': 0, 'total': 5}}
d2 = {'System tests': {'failed': 1, 'passed': 1, 'total': 2},
'Func tests': {'failed': 3, 'passed': 2, 'total': 5}}
d3 = {'System tests': {'failed': 0, 'passed': 0, 'total': 0},
'Func tests': {'failed': 1, 'passed': 0, 'total': 1}}
def combine_dicts(*d):
return {a:{i:sum(h[i] for h in [b, c, d]) for i in ['failed', 'passed', 'total']} for [a, b], [_, c], [_, d] in zip(*d)}
print(combine_dicts(d1.items(), d2.items(), d3.items()))
Output:
{'System tests': {'failed': 6, 'total': 7, 'passed': 1}, 'Func tests': {'failed': 9, 'total': 11, 'passed': 2}}
Related
I want to build a function GridGen, so that it does the following.
GridGen([a], -1, 1) --> [{a: -1}, {a: 0}, {a: -1}]
GridGen([a, b], -1, 1) --> [{a: -1, b: -1},{a: 0, b: -1}, {a: 1, b: -1},
{a: -1, b: 0}, {a: 0, b: 0}, {a: 1, b: 0},
{a: -1, b: 1}, {a: 0, b: 1}, {a: 1, b: 1}]
GridGen([a,b,c], -1, 1) --> [{a: -1, b: -1, c: -1},{a: 0, b: -1, c: -1}, {a: 1, b: -1, c:-1}, ... ]
At the moment I achieve this with two functions with a simple recursion.
from sage.all import *
def TensorMergeDict(dicts):
if len(dicts) == 2:
return flatten([[dicts[0][i] | dicts[1][j] for i in range(len(dicts[0]))] for j in range(len(dicts[1]))])
else:
print(TensorMergeDict([dicts[0], TensorMergeDict(dicts[1:])]))
return
def GridGen(vars, minV, maxV, step = 1):
dicts = [[{e: i} for i in range(minV, maxV + 1, step)] for e in vars]
return TensorMergeDict(dicts)
where sage provides the convenient flatten function to flatten a list.
I wonder if there is better/efficient way to do it? It feels like there should be some existing function in Python or SageMath that facilitates such operation.
How about itertools.product? -
from itertools import product
def grid(keys, lo, hi):
for p in product(range(lo, hi + 1), repeat=len(keys)):
yield {k:v for k,v in zip(keys, p)}
for x in grid("abc", -1, 1):
print(x)
{'a': -1, 'b': -1, 'c': -1}
{'a': -1, 'b': -1, 'c': 0}
{'a': -1, 'b': -1, 'c': 1}
{'a': -1, 'b': 0, 'c': -1}
{'a': -1, 'b': 0, 'c': 0}
{'a': -1, 'b': 0, 'c': 1}
{'a': -1, 'b': 1, 'c': -1}
{'a': -1, 'b': 1, 'c': 0}
{'a': -1, 'b': 1, 'c': 1}
{'a': 0, 'b': -1, 'c': -1}
{'a': 0, 'b': -1, 'c': 0}
{'a': 0, 'b': -1, 'c': 1}
{'a': 0, 'b': 0, 'c': -1}
{'a': 0, 'b': 0, 'c': 0}
{'a': 0, 'b': 0, 'c': 1}
{'a': 0, 'b': 1, 'c': -1}
{'a': 0, 'b': 1, 'c': 0}
{'a': 0, 'b': 1, 'c': 1}
{'a': 1, 'b': -1, 'c': -1}
{'a': 1, 'b': -1, 'c': 0}
{'a': 1, 'b': -1, 'c': 1}
{'a': 1, 'b': 0, 'c': -1}
{'a': 1, 'b': 0, 'c': 0}
{'a': 1, 'b': 0, 'c': 1}
{'a': 1, 'b': 1, 'c': -1}
{'a': 1, 'b': 1, 'c': 0}
{'a': 1, 'b': 1, 'c': 1}
keys can be string or array -
for x in grid(["foo", "bar"], -1, 1):
print(x)
{'foo': -1, 'bar': -1}
{'foo': -1, 'bar': 0}
{'foo': -1, 'bar': 1}
{'foo': 0, 'bar': -1}
{'foo': 0, 'bar': 0}
{'foo': 0, 'bar': 1}
{'foo': 1, 'bar': -1}
{'foo': 1, 'bar': 0}
{'foo': 1, 'bar': 1}
This question already has answers here:
Elegant way to remove fields from nested dictionaries
(11 answers)
Closed 7 months ago.
I have a small example of a nested dictionary (in my case a collections defaultdict):
all_dict = {'d1': {'a': 2, 'b': 4, 'c': 10},
'd2': {'a': 1, 'b': 23,'c': 0},
'd3': {'a': 4, 'b': 12,'c': 4},
'd4': {'a': 0, 'b': 4, 'c': 3},
'd5': {'a': 4, 'b': 0, 'c': 1}}
And I would like to filter all the zero values in all_dict. In my real data, the nested dictionaries are very big. I would like any general "pythonic" solution.
I tried some comprehension but I failed.
I would like some like:
all_dict_filtered =
{'d1': {'a': 2, 'b': 4, 'c': 10},
'd2': {'a': 1, 'b': 23},
'd3': {'a': 4, 'b': 12,'c': 4},
'd4': {'b': 4, 'c': 3},
'd5': {'a': 4, 'c': 1}}
Any tip would be great.
Thank you for your time and attention.
Paulo
I have this but is ugly:
filtered = defaultdict(dict)
for k1, v1 in all_dict.items():
for k2, v2 in v1.items():
if v2 > 0:
filtered[k1] = filtered.get(k1, {})
filtered[k1][k2] = v2
defaultdict(dict,
{'d1': {'a': 2, 'b': 4, 'c': 10},
'd2': {'a': 1, 'b': 23},
'd3': {'a': 4, 'b': 12, 'c': 4},
'd4': {'b': 4, 'c': 3},
'd5': {'a': 4, 'c': 1}})
Simple one-liner using dict comprehensions:
import pprint
all_dict = {
'd1': {'a': 2, 'b': 4, 'c': 10},
'd2': {'a': 1, 'b': 23, 'c': 0},
'd3': {'a': 4, 'b': 12, 'c': 4},
'd4': {'a': 0, 'b': 4, 'c': 3},
'd5': {'a': 4, 'b': 0, 'c': 1}
}
pprint.pprint({k: {key: value for key, value in v.items() if value != 0} for k, v in all_dict.items()})
Output:
{'d1': {'a': 2, 'b': 4, 'c': 10},
'd2': {'a': 1, 'b': 23},
'd3': {'a': 4, 'b': 12, 'c': 4},
'd4': {'b': 4, 'c': 3},
'd5': {'a': 4, 'c': 1}}
Try:
all_dict = {
"d1": {"a": 2, "b": 4, "c": 10},
"d2": {"a": 1, "b": 23, "c": 0},
"d3": {"a": 4, "b": 12, "c": 4},
"d4": {"a": 0, "b": 4, "c": 3},
"d5": {"a": 4, "b": 0, "c": 1},
}
for k, v in all_dict.items():
all_dict[k] = {kk: vv for kk, vv in v.items() if vv != 0}
print(all_dict)
Prints:
{
"d1": {"a": 2, "b": 4, "c": 10},
"d2": {"a": 1, "b": 23},
"d3": {"a": 4, "b": 12, "c": 4},
"d4": {"b": 4, "c": 3},
"d5": {"a": 4, "c": 1},
}
took me some time, but here you go:
dicty = {'d1': {'a': 2, 'b': 4, 'c': 10},
'd2': {'a': 1, 'b': 23,'c': 0},
'd3': {'a': 4, 'b': 12,'c': 4},
'd4': {'a': 0, 'b': 4, 'c': 3},
'd5': {'a': 4, 'b': 0, 'c': 1}}
listy = []
for value in dicty:
for value2 in dicty[value]:
if dicty[value][value2] == 0:
listy.append((value, value2))
for inpurity in listy:
dicty[inpurity[0]].pop(inpurity[1])
print(listy)
print(dicty)
Try this:
import copy
all_dict = {'d1': {'a': 2, 'b': 4, 'c': 10},
'd2': {'a': 1, 'b': 23,'c': 0},
'd3': {'a': 4, 'b': 12,'c': 4},
'd4': {'a': 0, 'b': 4, 'c': 3},
'd5': {'a': 4, 'b': 0, 'c': 1}}
all_dict_filtered = copy.deepcopy(all_dict)
for i in all_dict:
for j in all_dict[i] :
if all_dict[i][j] == 0 :
all_dict_filtered[str(i)].pop(str(j))
print(all_dict_filtered)
Output : {'d1': {'a': 2, 'b': 4, 'c': 10}, 'd2': {'a': 1, 'b': 23}, 'd3': {'a': 4, 'b': 12, 'c': 4}, 'd4': {'b': 4, 'c': 3}, 'd5': {'a': 4, 'c': 1}}
I have multiple (~40) lists that contain dictionaries, that I would like to find
which are those list items (in this case dictionaries) that are common in all lists
how many times each unique item appears across all lists.
Some examples of the lists are:
a = [{'A': 0, 'B': 0},
{'A': 0, 'C': 1},
{'D': 1, 'C': 0},
{'D': 1, 'E': 0}]
b = [{'A': 0},
{'B': 0, 'C': 1},
{'D': 1, 'C': 0},
{'D': 1, 'E': 0}]
c = [{'C': 0},
{'B': 1},
{'D': 1, 'C': 0, 'E': 0},
{'D': 1, 'E': 0}]
What I tried so far, it is the following code, but it returned values that were not common in all lists...
def flatten(map_groups):
items = []
for group in map_groups:
items.extend(group)
return items
def intersection(map_groups):
unique = []
items = flatten(map_groups)
for item in items:
if item not in unique and items.count(item) > 1:
unique.append(item)
return unique
all_lists = [a,b,c]
intersection(all_lists)
What I would expect to get as a result would be:
1. {'D': 1, 'E': 0} as a common item in all lists
2. {'D': 1, 'E': 0}, 3
{'D': 1, 'C': 0}, 2
{'A': 0, 'B': 0}, 1
{'A': 0, 'C': 1}, 1
{'A': 0}, 1
{'B': 0, 'C': 1}, 1
{'C': 0},
{'B': 1},
{'D': 1, 'C': 0, 'E': 0}
To count things, python comes with a nice class: collections.Counter. Now the question is: What do you want to count?
For example, if you want to count the dictionaries that have the same keys and values, you can do something like this:
>>> count = Counter(tuple(sorted(x.items())) for x in a+b+c)
>>> count.most_common(3)
[((('C', 0), ('D', 1)), 2), ((('D', 1), ('E', 0)), 2), ((('A', 0), ('B', 0)), 1)]
The dictionaries here are converted to tuples with sorted items to make them comparable and hashable. Getting for example the 3 most common back as a list of dictionaries is also not too hard:
>>> [dict(x[0]) for x in count.most_common(3)]
[{'C': 0, 'D': 1}, {'D': 1, 'E': 0}, {'A': 0, 'B': 0}]
You can use a nested for loop:
a = [{'A': 0, 'B': 0},
{'A': 0, 'C': 1},
{'D': 1, 'C': 0},
{'D': 1, 'E': 1}]
b = [{'A': 0},
{'B': 0, 'C': 1},
{'D': 1, 'C': 0},
{'D': 1, 'E': 0}]
c = [{'C': 0},
{'B': 1},
{'D': 1, 'C': 0, 'E': 0},
{'D': 1, 'E': 0}]
abc_list = [*a, *b, *c]
abc = list()
for d in abc_list:
for i in abc:
if d == i[0]:
abc[abc.index(i)] = (d, i[1] + 1)
continue
abc.append((d, 1))
print(abc)
Output:
[({'A': 0, 'B': 0}, 1),
({'A': 0, 'C': 1}, 1),
({'D': 1, 'C': 0}, 2),
({'D': 1, 'E': 1}, 1),
({'A': 0}, 1),
({'B': 0, 'C': 1}, 1),
({'D': 1, 'E': 0}, 2),
({'C': 0}, 1),
({'B': 1}, 1),
({'D': 1, 'C': 0, 'E': 0}, 1)]
Explanation:
The line
[*a, *b, *c]
unpacks all the values in lists a, b and c into a single list, which \i named abc_list.
The continue statement where I put it means to directly continue to the next iteration of the inner for loop, without reaching abc.append((d, 1)).
The above output answers question 2. For question 1, we can use the built-in max() method on the abc list, with a custom key:
print(max(ABC, key=lambda x:x[1])[0])
Of course, it will only return one dictionary, {'D': 1, 'C': 0}. If you want to print out multiple dictionaries that appear the most frequently:
m = max(abc, key=lambda x:x[1])[1]
for d in abc:
if d[1] == m:
print(d[0])
Output:
{'D': 1, 'C': 0}
{'D': 1, 'E': 0}
How can I flatten a dictionary of dictonaries in Python, and put them into a list? For example, say I have the following dict:
data = { id1 : {x: 1, y: 2, z: 3}, id2 : {x: 4, y: 5, z: 6}}
How do I get:
[{id: id1, x: 1, y: 2, z: 3}, {id: id2, x: 4, y: 5, z: 6}]
With python 3.5 and higher
>>> data = { 'id1' : {'x': 1, 'y': 2, 'z': 3}, 'id2' : {'x': 4, 'y': 5, 'z': 6}}
>>> [{**v, 'id':k} for k,v in data.items()]
[{'x': 1, 'y': 2, 'z': 3, 'id': 'id1'}, {'x': 4, 'y': 5, 'z': 6, 'id': 'id2'}]
On older python versions
>>> [dict(v, id=k) for k,v in data.iteritems()]
[{'x': 1, 'y': 2, 'z': 3, 'id': 'id1'}, {'x': 4, 'y': 5, 'z': 6, 'id': 'id2'}]
>>>
I have the following dictionary
stocklist = {'a': 0, 'b': 0, 'c': 0}
And I want to setup a grid of HEIGHT by WIDTH where each cell in the grid has it's own unique version of a stocklist with different values. Will this work?
stockmap = [[stocklist for w in range(WIDTH)] for h in range(HEIGHT)]
I have other lists of width by height where each cell contains only one value, and they work fine.
But previous to this I tried to solve my issue by using Classes and it was a nightmare as my instances contained a list that kept being identical.
I'm worried that if I start coding the above I'll end up with the same problem.
In your example each 'cell' of your grid will point to the same dictionary - stocklist. So if you modify one 'cell' actually all of them will change.
If you need to store different dict in each cell you should create deep copies of the stocklist.
try:
import copy
stocklist = {'a': 0, 'b': 0, 'c': 0}
stockmap = [[copy.deepcopy(stocklist) for w in range(WIDTH)] for h in range(HEIGHT)]
In the simple example, where your stocklist does not contain any nested dict also
`stockmap = [[dict(stocklist) for w in range(WIDTH)] for h in range(HEIGHT)]`
will work. However remember that if your stocklist would be something like {'a': 0, 'b': {'c': 0}}, the internal - nested dict {'c': 0} will not be deep copied and each 'cell' will share that dict.
As I suggested above, you need to instantiate a new object, for example using dict
stockmap = [[dict(stocklist) for w in range(WIDTH)] for h in range(HEIGHT)]
otherwise the very same dictionary instance would be used.
Let's check it out:
with your example
>>> HEIGHT = 3
>>> WIDTH = 3
>>> stocklist = {'a': 0, 'b': 0, 'c': 0}
>>> stockmap = [[stocklist for w in range(WIDTH)] for h in range(HEIGHT)]
>>> stockmap
[[{'a': 0, 'b': 0, 'c': 0}, {'a': 0, 'b': 0, 'c': 0}, {'a': 0, 'b': 0, 'c': 0}], [{'a': 0, 'b': 0, 'c': 0}, {'a': 0, 'b': 0, 'c': 0}, {'a': 0, 'b': 0, 'c': 0}], [{'a': 0, 'b': 0, 'c': 0}, {'a': 0, 'b': 0, 'c': 0}, {'a': 0, 'b': 0, 'c': 0}]]
>>> stocklist['a']=9
>>> stockmap
[[{'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}], [{'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}], [{'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}]]
as you can clearly see, modifying one item in the original dictionary affects the newly created list (grid)
Whereas doing
>>> stockmap = [[dict(stocklist) for w in range(WIDTH)] for h in range(HEIGHT)]
>>> stockmap
[[{'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}], [{'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}], [{'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}]]
>>> stocklist['a']=5
>>> stockmap
[[{'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}], [{'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}], [{'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}, {'a': 9, 'b': 0, 'c': 0}]]
leaves the grid unaltered
Note: as #damgad correctly points out, dict would not work for nested dictionaries. In such cases you need to use copy.deepcopy