I know this is simple, but I've been searching for about an hour and was unable to find a good answer. I know there has to be something more elegant than iterating through the keys to look for matches and then adding the values.
What I have is:
test_dict = [{'Bob':2}, {'Jane':1}, {'Marco':1}, {'Suzy':2}, {'Bob':1},{'Mark':3}, {'Ellen':1}, {'Suzy':1}]
What I want to do is add the values together when the keys match (in this case Bob and Suzy). This would eliminate the duplicate keys by adding their values together. It would look like:
test_dict = [{'Bob':3}, {'Jane':1}, {'Marco':1}, {'Suzy':3},{'Mark':3}, {'Ellen':1}]
What I have tried is:
from collections import Counter
final = Counter(test_dict)
As well as other collections module items. I'd really prefer not to loop through the dictionary to compare each key for a match and then add the values together. This seems like a really inefficient idea but I can't think of (or find) anything else.
You were on the right track with Counter:
>>> sum((Counter(d) for d in test_dict), Counter())
Counter({'Bob': 3, 'Ellen': 1, 'Jane': 1, 'Marco': 1, 'Mark': 3, 'Suzy': 3})
To do the same with an explicit loop would be like:
>>> counter = Counter()
>>> for dict_ in test_dict:
... counter.update(**dict_)
...
>>> counter
Counter({'Bob': 3, 'Ellen': 1, 'Jane': 1, 'Marco': 1, 'Mark': 3, 'Suzy': 3})
You can also use defaultdict, where you extract each key and value form test_dict and add it to the defaultdict which takes care of instantiating keys that do not yet exist.
from collections import defaultdict
dd = defaultdict(int)
for d in test_dict:
dd[d.keys()[0]] += d.values()[0]
>>> dd
defaultdict(int,
{'Bob': 3,
'Ellen': 1,
'Jane': 1,
'Marco': 1,
'Mark': 3,
'Suzy': 3})
Related
I have a question regarding merging two dictionaries. It is not a simple merge, but merging in a way that I shall take first key value pair from the first element, then the first element from the second dictionary and so on.
For example:
dict1 = {"zero":0,"two":2, "four":4, "six": 6, "eight":8,"ten":10}
dict2 = {"one":1,"three":3,"five":5,"seven":7, "nine":9}
I need to have the following:
dict3 = {"zero":0,"one":1,"two":2,"three":3,"four":4, ... "ten":10 }
Would appreaciate any advise
The answer from #Andrei Vintila is along the right lines but dict_keys are not subscriptable and using the smallest size misses some dictionary items. A looping approach which does work (and for any number of elements in either dict) is:
dict1_keys = list(dict1.keys())
dict2_keys = list(dict2.keys())
s1 = len(dict1_keys)
s2 = len(dict2_keys)
max_size = max(s1, s2)
dict3 = {}
for index in range(max_size):
if(index < s1):
key1 = dict1_keys[index]
dict3[key1] = dict1[key1]
if(index < s2):
key2 = dict2_keys[index]
dict3[key2] = dict2[key2]
print(dict3)
which produces:
{'zero': 0, 'one': 1, 'two': 2, 'three': 3, 'four': 4, 'five': 5, 'six': 6, 'seven': 7, 'eight': 8, 'nine': 9, 'ten': 10}
You need to create one more dictionary which will merge the other two dictionaries, after that we will only need to sort the new dictionary items. Here is the code:
dict1 = {"zero":0,"two":2, "four":4, "six": 6, "eight":8,"ten":10}
dict2 = {"one":1,"three":3,"five":5,"seven":7, "nine":9}
dict3 = {**dict1, **dict2}
dict3 = dict(sorted(dict3.items(), key=lambda x:x[1]))
print(dict3)
If you’re on python 3.7+ then the dictionaries maintain the insertion order.
dict1_keys = dict1.keys()
dict2_keys = dict2.keys()
The way you do it is you have to get the keys of the first and second dictionary and loop through them using an index and once you exhaust the keys of one of your dictionaries and still have keys left in the other then you just copy over the remaining keys and values.
smallest_size = min(len(dict1_keys), len(dict2_keys))
for index in range(smallest_size):
key1 = dict1_keys[index]
key2 = dict2_keys[index]
dict3[key1] = dict1[key1]
dict3[key2] = dict2[key2]
I’m aware this might not be the Python way to tackle this however if you’re brushing up this is one way to do it…
If you want a simple loop approach the key is to use enumerate to get an index you can use to access the second dict (and check you havent run off the end of the second dict - I'm assuming here the second dict is same length or one shorter):
dict1 = {"zero":0,"two":2, "four":4, "six": 6, "eight":8,"ten":10}
dict2 = {"one":1,"three":3,"five":5,"seven":7, "nine":9}
result = {}
keys2 = list(dict2.keys())
for i, k in enumerate(dict1):
result[k] = dict1[k]
if i < len(dict2):
result[keys2[i]] = dict2[keys2[i]]
If you wanted a more functional approach you could do this:
from functools import reduce
result = reduce(dict.__or__, ({k1: v1, k2: v2} for (k1, v1), (k2, v2) in zip(dict1.items(), dict2.items())))
which works fine if both dicts are the same length. If the second is shorter, you'll need to manually append the remaining key-value from the first dict afterwards
Simple iteration with zip(). However zip() only return the shortest argument of iterable. In this case, the last key:value of the longest dict will not be included. dict.update() will update the dict to ensure all keys and values are added:
dict1 = {"zero":0,"two":2, "four":4, "six": 6, "eight":8,"ten":10}
dict2 = {"one":1,"three":3,"five":5,"seven":7, "nine":9, "eleven":11, "twelve":12}
dict3 = {}
for k1, k2 in zip(dict1, dict2):
dict3[k1], dict3[k2] = k1, k2
dict3.update(dict1); dict3.update(dict2)
print(dict3)
# {'zero': 0, 'one': 1, 'two': 2, 'three': 3, 'four': 4, 'five': 5, 'six': 6, 'seven': 7, 'eight': 8, 'nine': 9, 'ten': 10, 'eleven': 11, 'twelve': 12}
So dictionaries are not sequences like a list (as despite being "ordered" since 3.7, in most cases the order doesn't matter ). There is such a thing as an ordered dictionary from the collections module
There are several ways to combine dictionaries
The easiest way in your case is .update() so:
dict1.update(dict2)
To merge all the keys/val pairs into one dictionary
You can also use kwargs:
dict3 = {**dict1, **dict2}
If you still want them sorted you can use the sorted with different function
sorted(dict3,key=[insert]) # note can also use reverse=True
Different key functions:
str: if an increasing name like a,b,c etc
lamda x: dict3[x]: if increase value such as 0,1,2,3 etc
could also make a key list and search from that (though a bit more hacky)
key_list =[]
#Note only works if len(dict1) = len(dict2)
for item1, item2 in zip(dict1,dict2):
key_list.extend([item1,item2])
And then do key=lambda x: key_list.index(x)
Hope this helps!
What is the most Pythonic way to take a list of dicts and sum up all the values for matching keys from every row in the list?
I did this but I suspect a comprehension is more Pythonic:
from collections import defaultdict
demandresult = defaultdict(int) # new blank dict to store results
for d in demandlist:
for k,v in d.iteritems():
demandresult[k] = demandresult[k] + v
In Python - sum values in dictionary the question involved the same key all the time, but in my case, the key in each row might be a new key never encountered before.
I think that your method is quite pythonic. Comprehensions are nice but they shouldn't really be overdone, and they can lead to really messy one-liners, like the one below :).
If you insist on a dict comp:
demand_list = [{u'2018-04-29': 1, u'2018-04-30': 1, u'2018-05-01': 1},
{u'2018-04-21': 1},
{u'2018-04-18': 1, u'2018-04-19': 1, u'2018-04-17' : 1}]
d = {key:sum(i[key] for i in demand_list if key in i)
for key in set(a for l in demand_list for a in l.keys())}
print(d)
>>>{'2018-04-21': 1, '2018-04-17': 1, '2018-04-29': 1, '2018-04-30': 1, '2018-04-19': 1, '2018-04-18': 1, '2018-05-01': 1}
Here is another one-liner (ab-)using collections.ChainMap to get the combined keys:
>>> from collections import ChainMap
>>> {k: sum(d.get(k, 0) for d in demand_list) for k in ChainMap(*demand_list)}
{'2018-04-17': 1, '2018-04-21': 1, '2018-05-01': 1, '2018-04-30': 1, '2018-04-19': 1, '2018-04-29': 1, '2018-04-18': 1}
This is easily the slowest of the methods proposed here.
The only thing that seemed unclear in your code was the double-for-loop. It may be clearer to collapse the demandlist into a flat iterable—then the loopant presents the logic as simply as possible. Consider:
demandlist = [{
u'2018-04-29': 1,
u'2018-04-30': 1,
u'2018-05-01': 1
}, {
u'2018-04-21': 1
}, {
u'2018-04-18': 1,
u'2018-04-19': 1,
u'2018-04-17': 1
}]
import itertools as it
from collections import defaultdict
demandresult = defaultdict(int)
for k, v in it.chain.from_iterable(map(lambda d: d.items(), demandlist)):
demandresult[k] = demandresult[k] + v
(With this, print(demandresult) prints defaultdict(<class 'int'>, {'2018-04-29': 1, '2018-04-30': 1, '2018-05-01': 1, '2018-04-21': 1, '2018-04-18': 1, '2018-04-19': 1, '2018-04-17': 1}).)
Imagining myself reading this for the first time (or a few months later), I can see myself thinking, "Ok, I'm collapsing demandlist into a key-val iterable, I don't particularly care how, and then summing values of matching keys."
It's unfortunate that I need that map there to ensure the final iterable has key-val pairs… it.chain.from_iterable(demandlist) is a key-only iterable, so I need to call items on each dict.
Note that unlike many of the answers proposed, this implementation (like yours!) minimizes the number of scans over the data to just one—performance win (and I try to pick up as many easy performance wins as I can).
I suppose you want to return a list of summed values of each dictionary.
list_of_dict = [
{'a':1, 'b':2, 'c':3},
{'d':4, 'e':5, 'f':6}
]
sum_of_each_row = [sum(v for v in d.values()) for d in list_of_dict] # [6,15]
If you want to return the total sum, just simply wrap sum() to "sum_of_each_row".
EDIT:
The main problem is that you don't have a default value for each of the keys, so you can make use of the method dict.setdefault() to set the default value when there's a new key.
list_of_dict = [
{'a':1, 'b':1},
{'b':1, 'c':1},
{'a':2}
]
d = {}
d = {k:d[k]+v if k in d.keys() else d.setdefault(k,v)
for row in list_of_dict for k,v in row.items()} # {'a':3, 'b':2, 'c':1}
I have a dictionary such as below:
grocery={
'James': {'Brocolli': 3, 'Carrot': 3, 'Cherry': 5},
'Jill': {'Apples': 2, 'Carrot': 4, 'Tomatoes': 8},
'Sunny': {'Apples': 5, 'Carrot': 2, 'Cherry': 2, 'Chicken': 3, 'Tomatoes': 6}
}
food={}
for a,b in grocery.items():
for i,j in b.items():
food[i]+=(b.get(i,0))
I am trying to calculate total of each food item and it is not working as expected.
For eg: I would like to count total of Carrot, total of Apples and so on.
The above code is giving me following error:
File "dictionary1.py", line 6, in <module>
food[i]+=(b.get(i,0))
KeyError: 'Cherry
How to sum up total of each item?
Simply do
from collections import defaultdict
food = defaultdict(int) <-- default value of 0 to every non existent key
..and your code should work :)
PS. You get the error because you are trying to add values to uninitialized keys... Don't assume that non existent keys start from 0...
Your food dictionary is empty and has no keys at the start; you can't just sum up a value to something that isn't there yet.
Instead of +=, get the current value or a default, using dict.get() again:
food[i] = food.get(i, 0) + b.get(i,0)
You don't really need to use b.get() here, as you already have the values of b in the variable j:
food[i] = food.get(i, 0) + j
You could also use a collections.defaultdict() object to make keys 'automatically' exist when you try to access them, with a default value:
from collections import defaultdict
food = defaultdict(int) # insert int() == 0 when a key is not there yet
and in the inner loop then use food[i] += j.
I strongly recommend you use better names for your variables. If you iterate over dict.values() rather than dict.items(), you can look at the values only when you don't need the keys (like for the outer for loop):
food = {}
for shopping in grocery.values():
for name, quantity in shopping.items():
food[name] = food.get(name, 0) + quantity
Another option is to use a dedicated counting and summing dictionary subclass, called collections.Counter(). This class directly supports summing your groceries in a single line:
from collections import Counter
food = sum(map(Counter, grocery.values()), Counter())
map(Counter, ...) creates Counter objects for each of your input dictionaries, and sum() adds up all those objects (the extra Counter() argument 'primes' the function to use an empty Counter() as a starting value rather than an integer 0).
Demo of the latter:
>>> from collections import Counter
>>> sum(map(Counter, grocery.values()), Counter())
Counter({'Tomatoes': 14, 'Carrot': 9, 'Cherry': 7, 'Apples': 7, 'Brocolli': 3, 'Chicken': 3})
A Counter is still a dictionary, just one with extra functionality. You can always go back to a dictionary by passing the Counter to dict():
>>> food = sum(map(Counter, grocery.values()), Counter())
>>> dict(food)
{'Brocolli': 3, 'Carrot': 9, 'Cherry': 7, 'Apples': 7, 'Tomatoes': 14, 'Chicken': 3}
You get the error, because in the beginning the keys, i.e. 'Apples', 'Tomatoes', ..., do not exist in food. You can correct this with a try-except block:
grocery={
"Jill":{"Apples":2, "Tomatoes":8,"Carrot":4},
"James":{"Carrot":3,"Brocolli":3,"Cherry":5},
"Sunny":{"Chicken":3,"Apples":5,"Carrot":2,"Tomatoes":6,"Cherry":2}
}
food={}
for a,b in grocery.items():
for i,j in b.items():
try:
food[i] += j
except KeyError:
food[i] = j
Also, you can get rid of the b.get(i,0) statement, because you already iterate through b and only get values (j) that actually exist in b.
I have been trying to count the values in a dictionary with respect to the key. However, I could not achieved desired result. I will demonstrate with more details below:
from collections import Counter
d = {'a': ['Adam','Adam','John'], 'b': ['John','John','Joel'], 'c': ['Adam','Adam','John}
# create a list of only the values you want to count,
# and pass to Counter()
c = Counter([values[1] for values in d.itervalues()])
print c
My output:
Counter({'Adam': 2, 'John': 1})
I want it to count everything in the list, not just first value of the list. Also, I want my result to be with respect to the key. I will show you my desired output below:
{'a': [{'Adam': 1, 'John': 2}, 'b':{'John': 2, 'Joel': 1}, 'c':{'Adam': 2, 'John': 1 }]}
Is it possible to get this desired output? Or anything close to it? I would like to welcome any suggestions or ideas that you have. Thank you.
Try this using dict comprehension
from collections import Counter
d = {'a': ['Adam','Adam','John'], 'b': ['John','John','Joel'], 'c': ['Adam','Adam','John'}
c = {i:Counter(j) for i,j in d.items()}
print c
You're picking only the first elements in the each list with values[1], instead, you want to iterate through each values using a for that follows the first:
>>> from collections import Counter
>>> d = {'a': ['Adam','Adam','John'], 'b': ['John','John','Joel'], 'c': ['Adam','Adam','John']}
>>> Counter([v for values in d.itervalues() for v in values]) # iterate through each value
Counter({'John': 4, 'Adam': 4, 'Joel': 1})
I have a list of data, which has 2 values:
a 12
a 11
a 5
a 12
a 11
I would like to use a dictionary, so I can end up with a list of values for each of the key. Column 1 may have a different entry, like 'b', so I can arrange data based on column 1 as key, while column 2 is the data for each key
[a:12,11,5]
How do I achieve this? From what I read, if 2 values has the same key, the last one override the previous one, so only one key is in the dictionary.
d={}
for line in results:
templist=line.split(' ')
thekey=templist[0]
thevalue=templist[1]
if thevalue in d:
d[thekey].append(thevalue)
else:
d[thekey]=[thevalue]
Am I approaching the problem using the wrong way?
Python dicts can have only one value for a key, so you cannot assign multiple values in the fashion you are trying to.
Instead, store the mutiple values in a list corresponding to the key so that the list becomes the one value corresponding to the key:
d = {}
d["a"] = []
d["a"].append(1)
d["a"].append(2)
>>> print d
{'a': [1, 2]}
You can use a defaultdict to simplify this, which will initialise the key if it doesn't exist with an empty list as below:
from collections import defaultdict
d = defaultdict(list)
d["a"].append(1)
d["a"].append(2)
>>> print d
defaultdict(<type 'list'>, {'a': [1, 2]})
If you don't want repeat values for a key, you can use a set instead of list. But do note that a set is unordered.
from collections import defaultdict
d = defaultdict(set)
d["a"].add(1)
d["a"].add(2)
d["a"].add(1)
>>> print d
defaultdict(<type 'set'>, {'a': set([1, 2])})
If you need to maintain order, either use sorted at runtime, or use a list
with an if clause to check for values
from collections import defaultdict
d = defaultdict(list)
for item in (1, 2, 1, 2, 3, 4, 1, 2):
if item not in d["a"]:
d["a"].append(item)
>>> print d
defaultdict(<type 'list'>, {'a': [1, 2, 3, 4]})