How to combine two list containing dictionary with similar keys? - python

Assuming that there are two python list with the same structure like this:
var1 = [{'a':1,'b':2},{'c':2,'d':5,'h':4},{'c':2,'d':5,'e':4}]
var2 = [{'a':3,'b':2},{'c':1,'d':5,'h':4},{'c':5,'d':5,'e':4}]
In my case, i need to combine both of those list, so i'll get this value :
result = [{'a':4,'b':4},{'c':3,'d':10,'h':8},{'c':7,'d':10,'e':8}]
How can i do that?

zip-based one-liner comprehension:
result = [{k: d1[k]+d2[k] for k in d1} for d1, d2 in zip(var1, var2)]
This assumes that two dicts at the same index always have identical key sets.

Use list comprehensions to put the code in one line,
result = [{key : d1.get(key, 0)+d2.get(key, 0)
for key in set(d1.keys()) | set(d2.keys())} # union two sets
for d1, d2 in zip(var1, var2)]
print(result)
[{'a': 4, 'b': 4}, {'h': 8, 'c': 3, 'd': 10}, {'c': 7, 'e': 8, 'd': 10}]
This code takes into consideration the case that two dictionaries may not have the same keys.

var1 = [{'a':1,'b':2},{'c':2,'d':5,'h':4},{'c':2,'d':5,'e':4}]
var2 = [{'a':3,'b':2},{'c':1,'d':5,'h':4},{'c':5,'d':5,'e':4}]
res = []
for i in range(len(var1)):
dic = {}
dic1, dic2 = var1[i], var2[i]
for key, val in dic1.items(): // dic1.iteritems() in python 2.
dic[key] = dic1[key] + dic2[key]
res.append(dic)
>>>print(res)
[{'a': 4, 'b': 4}, {'c': 3, 'd': 10, 'h': 8}, {'c': 7, 'd': 10, 'e': 8}]

var1 = [{'a': 1, 'b': 2}, {'c': 2, 'd': 5, 'h': 4}, {'c': 2, 'd': 5, 'e': 4}]
var2 = [{'a': 3, 'b': 2}, {'c': 1, 'd': 5, 'h': 4}, {'c': 5, 'd': 5, 'e': 4}]
ret = []
for i, ele in enumerate(var1):
d = {}
for k, v in ele.items():
value = v
value += var2[i][k]
d[k] = value
ret.append(d)
print(ret)

For the sake of completeness, another zip-based one-liner that will work even if the dicts are uneven in the both lists:
result = [{k: d1.get(k, 0) + d2.get(k, 0) for k in set(d1) | set(d2)} for d1, d2 in zip(var1, var2)]

Would something like this help?
ar1 = [{'a':1,'b':2},{'c':2,'d':5,'h':4},{'c':2,'d':5,'e':4}]
var2 = [{'a':3,'b':2},{'c':1,'d':5,'h':4},{'c':5,'d':5,'e':4}]
combined_var = zip(var1, var2)
new_d = {}
list_new_ds = []
for i, j in combined_var:
new_d = {}
for key in i and j:
new_d[key] = i[key] + j[key]
list_new_ds.append(new_d)
list_new_ds = [{'a': 4, 'b': 4}, {'h': 8, 'c': 3, 'd': 10}, {'c': 7, 'e': 8, 'd': 10}]
To explain, the zip function merges the lists as a list of tuples. I then unpack the tuples and iterate through the keys in each dictionary and add the values for the same keys together using a new dictionary to store them. I then append the value to a list, and then re-initialise the temporary dictionary to empty before looking at the next tuple in the zipped list.
The order is different due to dictionary behaviour I believe.
I am a novice, so would appreciate any critiques of my answer!

Related

Group all keys with the same value in a dictionary of sets

I am trying to transform a dictionary of sets as the values with duplication to a dictionary with the unique sets as the value and at the same time join the keys together.
dic = {'a': {1, 2, 3}, 'b': {1, 2}, 'c': {1, 3, 2}, 'd': {1, 2, 3}}
Should be changed to
{'a-c-d': {1, 2, 3}, 'b': {1, 2}}
My try is as below, but I think there has to be a better way.
def transform_dictionary(dic: dict) -> dict:
dic = {k: frozenset(v) for k, v in dic.items()}
key_list = list(dic.keys())
value_list = list(dic.values())
dict_transformed = {}
for v_uinque in set(value_list):
sub_key_list = []
for i, v in enumerate(value_list):
if v == v_uinque:
sub_key_list.append(str(key_list[i]))
dict_transformed['-'.join(sub_key_list)] = set(v_uinque)
return dict_transformed
print(transform_dictionary(dic))
You can "invert" the input dictionary into a dictionary mapping frozensets into a set of keys.
import collections
dic = {'a': {1, 2, 3}, 'b': {1, 2}, 'c': {1, 3, 2}, 'd': {1, 2, 3}}
keys_per_set = collections.defaultdict(list)
for key, value in dic.items():
keys_per_set[frozenset(value)].append(key)
Then invert that dictionary mapping back into the desired form:
{'-'.join(keys): value for (value, keys) in keys_per_set.items()}
Output:
{'a-c-d': frozenset({1, 2, 3}), 'b': frozenset({1, 2})}
This will turn the values into a frozenset, but you could "thaw" them with a set(value) in the last list comprehension.
from itertools import groupby
dic_output = {'-'.join(v):g for g,v in groupby(sorted(dic_input,
key=dic_input.get),
key=lambda x: dic_input[x])}
Output
{'b': {1, 2}, 'a-c-d': {1, 2, 3}}

Given key value pairs, sort output alphabetically and increment values

dict1 = {'a': 10, 'b': 8, 'c':5}
dict2 = {'d': 6, 'c': 4, 'a':20}
Given two dictionaries, I'd like an output of.
output = {'a':30, 'b':8, 'c':9, 'd':6}
This is what I've so far, not quite sure what I'd do next.
I'm looking for a solution that is efficient in time/space complexity.
def merge_dict(dict1, dict2):
merged_dictionaries = {**dict1, **dict2}
return merged_dictionaries
dict1 = {'a': 10, 'b': 8, 'c':5}
dict2 = {'d': 6, 'c': 4, 'a':20}
merge_dictionaries = merge_dict (dict1, dict2)
sorted_dictionary = sorted(merge_dictionaries)
If the values are numeric, you can use counters:
from collections import Counter
def merge_dicts(*dicts):
return dict(sum(map(Counter, dicts), Counter()))
dict1 = merge_dicts(dict1, dict2)
dict1
# {'a': 30, 'b': 8, 'c': 9, 'd': 6}
This might be a bit excessive for only two dictionaries, so another option is:
for k, v in dict2.items():
dict1[k] = dict1.setdefault(k, 0) + v
dict1
# {'a': 30, 'b': 8, 'c': 9, 'd': 6}
Which updates dict1 in-place.
Finally, if you really need the result sorted (python3.7+), use
result = {k : dict1[k] for k in sorted(dict1)}
You can use a dict comprehension that iterates over a sorted union of the keys of the two dicts, and outputs values that are sums of the respective values of two dicts by the given keys, defaulting to 0:
{k: dict1.get(k, 0) + dict2.get(k, 0) for k in sorted(dict1.keys() | dict2.keys())}
This returns:
{'a': 30, 'b': 8, 'c': 9, 'd': 6}
result = dict(Counter(dict1) + Counter(dict2))
result = {k: result[k] for k in sorted(result)}
First merge the dicts together by turning them into Counters and convert the result it back into a dict, then sort the dict by keys.
You can Try Collections for add two dictionary..
from collections import Counter
def merged_dic():
dict1 = {'a': 10, 'b': 8, 'c':5}
dict2 = {'d': 6, 'c': 4, 'a':20}
a = Counter(dict1)
b = Counter(dict2)
c = a+b
print(dict(c))
merged_dic()
Output:- {'a': 30, 'b': 8, 'c': 9, 'd': 6}

How to convert list of dictionaries to dictionaries

mylist = [{'a': 1, 'b': 2}, {'c': 3, 'd': 4}, {'e': 5, 'f': 6}]
i want it as
myDict ={'a': 1, 'b': 2,'c': 3, 'd': 4,'e': 5, 'f': 6}
You can make use of ChainMap.
from collections import ChainMap
myDict = dict(ChainMap(*mylist ))
This will take each dictionary and iterate through its key value pairs in for (k,v) in elem.items() part and assign them to a new dictionary.
mylist = [{'a': 1, 'b': 2}, {'c': 3, 'd': 4}, {'e': 5, 'f': 6}]
new_dict = {k:v for elem in mylist for (k,v) in elem.items()}
print new_dict
This will replace the duplicated keys.
I would create a new dictionary, iterate over the dictionaries in mylist, then iterate over the key/value pairs in that dictionary. From there, you can add each key/value pair to myDict.
mylist = [{'a': 1, 'b': 2}, {'c': 3, 'd': 4}, {'e': 5, 'f': 6}]
myDict = {}
for Dict in mylist:
for key in Dict:
myDict[key] = Dict[key]
print(myDict)

Store variables in dictionary for large data

I can print variables in python.
for h in jl1["results"]["attributes-list"]["volume-attributes"]:
state = str(h["volume-state-attributes"]["state"])
if aggr in h["volume-id-attributes"]["containing-aggregate-name"]:
if state == "online":
print(h["volume-id-attributes"]["owning-vserver-name"]),
print(' '),
print(h["volume-id-attributes"]["name"]),
print(' '),
print(h["volume-id-attributes"]["containing-aggregate-name"]),
print(' '),
print(h["volume-space-attributes"]["size-used"]
These print function returns for example 100 lines. Now I want to print only top 5 values based on filter of "size-used".
I am trying to take these values in dictionary and filter out top five values for "size-used" but not sure how to take them in dictionary.
Some thing like this
{'vserver': (u'rcdn9-c01-sm-prod',), 'usize': u'389120', 'vname': (u'nprd_root_m01',), 'aggr': (u'aggr1_n01',)}
Any other options like namedtuples is also appreciated.
Thanks
To get a list of dictionaries sorted by a certain key, use sorted. Say I have a list of dictionaries with a and b keys and want to sort them by the value of the b element:
my_dict_list = [{'a': 3, 'b': 1}, {'a': 1, 'b': 4}, {'a': 4, 'b': 4},
{'a': 2, 'b': 7}, {'a': 2, 'b': 4.3}, {'a': 2, 'b': 9}, ]
my_sorted_dict_list = sorted(my_dict_list, key=lambda element: element['b'], reverse=True)
# Reverse is set to True because by default it sorts from smallest to biggest; we want to reverse that
# Limit to five results
biggest_five_dicts = my_sorted_dict_list[:5]
print(biggest_five_dicts) # [{'a': 2, 'b': 9}, {'a': 2, 'b': 7}, {'a': 2, 'b': 4.3}, {'a': 1, 'b': 4}, {'a': 4, 'b': 4}]
heapq.nlargest is the obvious way to go here:
import heapq
interesting_dicts = ... filter to keep only the dicts you care about (e.g. online dicts) ...
for large in heapq.nlargest(5, interesting_dicts,
key=lambda d: d["volume-space-attributes"]["size-used"]):
print(...)

Find common members that are in two lists of dictionaries

This may be a duplicate but the closest I could find was Comparing 2 lists consisting of dictionaries with unique keys in python which did not work for me.
So I have two lists of dictionaries.
y = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
y = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
How do I compare these two lists so my compare results in the intersection of the two lists. I can't convert it to set since it says unhashable type (dict)
Your question and it's title seem at odds with each other.
The intersection of the 2 lists would be the common elements of both list. The question title requests the elements that are not in both lists. Which is it that you want?
For the intersection, it is not very efficient (being O(n^2) in time), but this list comprehension will do it:
>>> a = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
>>> b = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
>>> [d for d in a if d in b]
[{'a': 1, 'b': 2, 'c': 3}]
y1 = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
y2 = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
print [x for x in y1 if x in y2] # prints [{'a': 1, 'c': 3, 'b': 2}]
A dict (or list) is not hashable, however, a tuple is. You can convert the list of dicts to a set of tuples. Perform the intersection and then convert back
the code to convert to a set-of-tuples
y_tupleset = set(tuple(sorted(d.items())) for d in y)
the code to convert back the intersected set-of-tuples to a list-of-dicts
y_dictlist = [dict(it) for it in list(y_tupleset)]
Thus, the full code would be:
y0 = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
y1 = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
y0_tupleset = set(tuple(sorted(d.items())) for d in y0)
y1_tupleset = set(tuple(sorted(d.items())) for d in y1)
y_inter = y0_tupleset.intersection(y1_tupleset)
y_inter_dictlist = [dict(it) for it in list(y_inter)]
print(y_inter_dictlist)
# prints the following line
[{'a': 1, 'c': 3, 'b': 2}]
edit: d.items() is valid on python3, for python2, it should be replaced with d.iteritems()
Pick your poison:
y1 = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
y2 = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
y3 = [{'a': 1, 'b': 2, 'c': 3}, {'a': 4, 'b': 2, 'c': 6}]
# Returns a list of keys that are in both dictionaries
def intersect_keys(d1, d2):
return [k for k in d1 if k in d2]
# Returns a list of values that are in both dictionaries
def intersect_vals(d1, d2):
return [v for v in d1.itervalues() if v in d2.itervalues()]
# Returns a list of (key,value) pairs that are in both dictionaries
def intersect_pairs(d1, d2):
return [(k,v) for (k,v) in d1.iteritems() if k in d2 and d2[k] == v]
print(intersect_keys(*y1)) # ['a', 'c', 'b']
print(intersect_vals(*y1)) # [3]
print(intersect_pairs(*y1)) # []
print(intersect_keys(*y2)) # ['a', 'c', 'b']
print(intersect_vals(*y2)) # []
print(intersect_pairs(*y2)) # []
print(intersect_keys(*y3)) # ['a', 'c', 'b']
print(intersect_vals(*y3)) # [2]
print(intersect_pairs(*y3)) # [('b', 2)]
Note: the examples compare the two elements of the y* list, which was how I interpreted your question. You could of course use something like:
print(intersect_pairs(y1[0], y2[0]))
To compute the intersection the first dictionary in the y1 and y2 lists.

Categories