I want to get a possible structure with all possible keys of a single field in the database, which is stored in JSON(dict) format. The structure in my task is very important, so storing in list is not suitable.There can be many levels of nesting much more than two.
This example:
dicts = {'a':1, 'b':2, 'c':{'in_c1': 2}}, \
{'a':1, 'd':2, 'c':{'dict_in_c2': {'v': 2}}},\
{'e':57}
Should return:
{'a': 1, 'b':2, 'c': {'in_c1': 2, 'dict_in_c2': {'v': 2}}, 'd': 2, 'e': 57}
The text(values of keys) is not important to me, it is better to replace it with something similar like none or an empty string.
How I can do this?
This might not be able to handle all cases, but it covers some basic cases.
I am assuming that
keys that have an integer value ( like a and b ) will NOT have any dictionary values.
keys that have a dictionary as value ( like c ) will NOT have any non-dictionary values. Also, the flattening has only been performed on the keys of the dictionary at the first level, but not on the nested levels.
Here is the code:
dicts = {'a':1, 'b':2, 'c':{'in_c1': 2}}, \
{'a':1, 'd':2, 'c':{'dict_in_c2': {'v': 2}}},\
{'e':57}
result = dict()
for dictionary in dicts:
for key, value in dictionary.items():
if not isinstance(value,dict):
result[key] = value
continue
if not result.get(key, None):
result[key] = dict()
for k, v in value.items():
result[key][k] = v
print(result)
Result -
{'b': 2, 'a': 1, 'c': {'dict_in_c2': {'v': 2}, 'in_c1': 2}, 'e': 57, 'd': 2}
If I understand your question correctly, you have a list of dicts, which you want to put it into a single dict.
If the list of dicts is:
dicts = [{'a':1, 'b':2, 'c':{'dict_in_c': 2}},
{'a':1, 'd':2, 'c':{'dict_in_c2': 2}},
{'e':57}]
Then you can do:
newdict={}
for eachdict in dicts:
for eachkey in eachdict.keys():
newdict[eachkey]=eachdict[eachkey]
newdict will be:
{'a': 1, 'b': 2, 'c': {'dict_in_c2': 2}, 'd': 2, 'e': 57}
Related
I have a data set as below
tmp_dict = {
'a': ?,
'b': ?,
'c': ?,
}
and I have a data is a list of dictionaries like
tmp_list = [tmp_dict1, tmp_dict2, tmp_dict3....]
and I found some of dictionaries are not perfectly have keys about 'a','b','c'.
How do I check and fill the key is not existing
You could try something like this:
# List of keys to look for in each dictionary
dict_keys = ['a','b','c']
# Generate the dictionaries for demonstration purposes only
tmp_dict1 = {'a':[1,2,3], 'b':[4,5,6]}
tmp_dict2 = {'a':[7,8,9], 'b':[10,11,12], 'c':[13,14,15]}
tmp_dict3 = {'a':[16,17,18], 'c':[19,20,21]}
# Add the dictionaries to a list as per OP instructions
tmp_list = [tmp_dict1, tmp_dict2, tmp_dict3]
#--------------------------------------------------------
# Check for missing keys in each dict.
# Print the dict name and keys missing.
# -------------------------------------------------------
for i, dct in enumerate(tmp_list, start=1):
for k in dict_keys:
if dct.get(k) == None:
print(f"tmp_dict{i} is missing key:", k)
OUTPUT:
tmp_dict1 is missing key: c
tmp_dict3 is missing key: b
I think you want this.
tmp_dict = {'a':1, 'b': 2, 'c':3}
default_keys = tmp_dict.keys()
tmp_list = [{'a': 1}, {'b': 2,}, {'c': 3}]
for t in tmp_list:
current_dict = t.keys()
if default_keys - current_dict:
t.update({diff: None for diff in list(default_keys-current_dict)})
print(tmp_list)
Output:
[{'a': 1, 'c': None, 'b': None}, {'b': 2, 'a': None, 'c': None}, {'c': 3, 'a': None, 'b': None}]
You can compare the keys in the dictionary with a set containing all the expected keys.
for d in tmp_list:
if set(d) != {'a', 'b', 'c'}:
print(d)
say we have two nested dictionaries:
dict1={'s1':{'A':{'C':'3','D':'4'},'B':{'E':'5','F':'6'}}}
dict2 = {'s1':[{'C':'3a','D':'4a'},{'C':'3b','D':'4b'}], 'B': {'E':'5a','F':'6a'}}
I can replace the value in dict1 based on a key with value dict2
dict1['E']=dict2['E']
which will result in ..
dict1={'s1':{'A':{'C':'3','D':'4'},'B':{'E':'5a','F':'6'}}}
Now I want to find, C and D in dict1 and replace with list of C and D from dict2
Output should be like:
dict1={'s1':{'A':[{'C':'3a','D':'4a'},{'C':'3b','D':'4b'}],'B':{'E':'5a','F':'6'}}}
without affecting original keys A and B in dict 1
we can also create a new dictionary copying dict1 and make modifications..
but the structure of dict1 should remain intact
Possibly, you haven't written dict2 properly, it neither a dictionary nor a set. Maybe this (dict2={'s1':[{'C':'3a','D':'4a'},{'C':'3b','D':'4b'}], 'B': {'E':'5a','F':'6a'}}) is what you meant.
If this assumption is correct, the you can change the value of key A in dict1 this way:
dict1['s1']['A'] = dict2['s1']
print(dict1)
Result
{'s1': {'A': [{'C': '3a', 'D': '4a'}, {'C': '3b', 'D': '4b'}], 'B': {'E': '5', 'F': '6'}}}
I have two dictionaries:
dict1 = {'a': 1,
'b': 2,
'c': 3,
'd': 4,
'x': 5}
and
dict2 = {'a': 'start',
'b': 'start',
'c': 'end',
'd': 'end'}
I am trying to create a new dictionary that maps the values start and end as keys to a dictionary that would contain the info of dict1, while keeping those that are not present in dict2 as keys, e.g.:
dict3 = {'start': {'a': 1, 'b': 2},
'end': {'c': 3, 'd': 4},
'x': {'x': 5}
}
Use dict.setdefault() to create the nested dictionaries in dict3 if not yet there, and dict.get() to determine the key in the top-level output dictionary:
dict3 = {}
for k, v in dict1.items():
nested = dict3.setdefault(dict2.get(k, k), {})
nested[k] = v
So dict2.get(k, k) will produce the value from dict2 for a given key from dict1, using the key itself as a default. So for the 'x' key, that'll produce 'x' as there is no mapping in dict2 for that key.
Demo:
>>> dict3 = {}
>>> for k, v in dict1.items():
... nested = dict3.setdefault(dict2.get(k, k), {})
... nested[k] = v
...
>>> dict3
{'start': {'a': 1, 'b': 2}, 'end': {'c': 3, 'd': 4}, 'x': {'x': 5}}
I actually figured it out while abstracting the example and typing up my question here (should have maybe done this earlier...). Anyways: Yay!
So here is my solution, in case it may help someone. If someone knows a swifter or more elegant way to do it, I would be glad to learn!
dict3 = dict()
for k, v in dict1.items():
# if the key of dict1 exists also in dict2
if k in dict2.keys():
# get its value (the keys-to-be for the new dict3)
new_key = dict2[k]
# if the new key is already in the new dict
if new_key in dict3.keys():
# appends new dict entry to dict3
dict3[new_key].update({k: v})
# otherwise create a new entry
else:
dict3[new_key] = {k: v}
# if there is no corresponding mapping present
else:
# treat the original key as the new key and add to dict3
no_map = k
dict3[no_map] = {k: v}
I want to write a code which takes the following inputs:
list (list of maps)
request_keys (list of strings)
operation (add,substract,multiply,concat)
The code would look at the list for the maps having the same value for all keys except the keys given in request_keys. Upon finding two maps for which the value in the search keys match, the code would do the operation (add,multiple,substract,concat) on the two maps and combine them into one map. This combination map would basically replace the other two maps.
i have written the following peice of code to do this. The code only does add operation. It can be extended to make the other operations
In [83]: list
Out[83]:
[{'a': 2, 'b': 3, 'c': 10},
{'a': 2, 'b': 3, 'c': 3},
{'a': 2, 'b': 4, 'c': 4},
{'a': 2, 'b': 3, 'c': 2},
{'a': 2, 'b': 3, 'c': 3}]
In [84]: %cpaste
Pasting code; enter '--' alone on the line to stop or use Ctrl-D.
:def func(list,request_keys):
: new_list = []
: found_indexes = []
: for i in range(0,len(list)):
: new_item = list[i]
: if i in found_indexes:
: continue
: for j in range(0,len(list)):
: if i != j and {k: v for k,v in list[i].iteritems() if k not in request_keys} == {k: v for k,v in list[j].iteritems() if k not in request_keys}:
: found_indexes.append(j)
: for request_key in request_keys:
: new_item[request_key] += list[j][request_key]
: new_list.append(new_item)
: return new_list
:--
In [85]: func(list,['c'])
Out[85]: [{'a': 2, 'b': 3, 'c': 18}, {'a': 2, 'b': 4, 'c': 4}]
In [86]:
What i want to know is, is there a faster, more memory efficient, cleaner and a more pythonic way of doing the same?
Thank you
You manually generate all the combinations and then compare each of those combinations. This is pretty wasteful. Instead, I suggest grouping the dictionaries in another dictionary by their matching keys, then adding the "same" dictionaries. Also, you forgot the operator parameter.
import collections, operator, functools
def func(lst, request_keys, op=operator.add):
matching_dicts = collections.defaultdict(list)
for d in lst:
key = tuple(sorted(((k, d[k]) for k in d if k not in request_keys)))
matching_dicts[key].append(d)
for group in matching_dicts.values():
merged = dict(group[0])
merged.update({key: functools.reduce(op, (g[key] for g in group))
for key in request_keys})
yield merged
What this does: First, it creates a dictionary, mapping the key-value pairs that have to be equal for two dictionaries to match to all those dictionaries that have those key-value pairs. Then it iterates the dicts from those groups, using one of that group as a prototype and updating it with the sum (or product, or whatever, depending on the operator) of the all the dicts in that group for the required_keys.
Note that this returns a generator. If you want a list, just call it like list(func(...)), or accumulate the merged dicts in a list and return that list.
from itertools import groupby
from operator import itemgetter
def mergeDic(inputData, request_keys):
keys = inputData[0].keys()
comparedKeys = [item for item in keys if item not in request_keys]
grouper = itemgetter(*comparedKeys)
result = []
for key, grp in groupby(sorted(inputData, key = grouper), grouper):
temp_dict = dict(zip(comparedKeys, key))
for request_key in request_keys:
temp_dict[request_key] = sum(item[request_key] for item in grp)
result.append(temp_dict)
return result
inputData = [{'a': 2, 'b': 3, 'c': 10},
{'a': 2, 'b': 3, 'c': 3},
{'a': 2, 'b': 4, 'c': 4},
{'a': 2, 'b': 3, 'c': 2},
{'a': 2, 'b': 3, 'c': 3}]
from pprint import pprint
pprint(mergeDic(inputData,['c']))
i just picked up python not too long ago.
An example below
i have a dictionary within a list
myword = [{'a': 2},{'b':3},{'c':4},{'a':1}]
I need to change it to the output below
[{'a':3} , {'b':3} , {'c':4}]
is there a way where i can add the value together? I tried using counter, but it prints out the each dict out.
what i did using Counter:
for i in range(1,4,1):
text = myword[i]
Print Counter(text)
The output
Counter({'a': 2})
Counter({'b': 3})
Counter({'c': 4})
Counter({'a': 1})
i have read the link below but what they compared was between 2 dict.
Is there a better way to compare dictionary values
Thanks!
Merge dictionaries into one dictionary (Counter), and split them.
>>> from collections import Counter
>>> myword = [{'a': 2}, {'b':3}, {'c':4}, {'a':1}]
>>> c = Counter()
>>> for d in myword:
... c.update(d)
...
>>> [{key: value} for key, value in c.items()]
[{'a': 3}, {'c': 4}, {'b': 3}]
>>> [{key: value} for key, value in sorted(c.items())]
[{'a': 3}, {'b': 3}, {'c': 4}]