We Keep Coding

Get a dictionary with saved structure of all keys in nested dictionaries

I want to get a possible structure with all possible keys of a single field in the database, which is stored in JSON(dict) format. The structure in my task is very important, so storing in list is not suitable.There can be many levels of nesting much more than two.
This example:
dicts = {'a':1, 'b':2, 'c':{'in_c1': 2}}, \
{'a':1, 'd':2, 'c':{'dict_in_c2': {'v': 2}}},\
{'e':57}
Should return:
{'a': 1, 'b':2, 'c': {'in_c1': 2, 'dict_in_c2': {'v': 2}}, 'd': 2, 'e': 57}
The text(values of keys) is not important to me, it is better to replace it with something similar like none or an empty string.
How I can do this?

This might not be able to handle all cases, but it covers some basic cases.
I am assuming that
keys that have an integer value ( like a and b ) will NOT have any dictionary values.
keys that have a dictionary as value ( like c ) will NOT have any non-dictionary values. Also, the flattening has only been performed on the keys of the dictionary at the first level, but not on the nested levels.
Here is the code:
dicts = {'a':1, 'b':2, 'c':{'in_c1': 2}}, \
{'a':1, 'd':2, 'c':{'dict_in_c2': {'v': 2}}},\
{'e':57}
result = dict()
for dictionary in dicts:
for key, value in dictionary.items():
if not isinstance(value,dict):
result[key] = value
continue
if not result.get(key, None):
result[key] = dict()
for k, v in value.items():
result[key][k] = v
print(result)
Result -
{'b': 2, 'a': 1, 'c': {'dict_in_c2': {'v': 2}, 'in_c1': 2}, 'e': 57, 'd': 2}

If I understand your question correctly, you have a list of dicts, which you want to put it into a single dict.
If the list of dicts is:
dicts = [{'a':1, 'b':2, 'c':{'dict_in_c': 2}},
{'a':1, 'd':2, 'c':{'dict_in_c2': 2}},
{'e':57}]
Then you can do:
newdict={}
for eachdict in dicts:
for eachkey in eachdict.keys():
newdict[eachkey]=eachdict[eachkey]
newdict will be:
{'a': 1, 'b': 2, 'c': {'dict_in_c2': 2}, 'd': 2, 'e': 57}

Remove the duplicate from list of dict in python

I have dict like this
d=[{'a':1,'b':2},{'a':2,'b':2},{'a':3},{'a':1}]
So i need like
d=[{'a':1,'b':2},{'a':2,'b':2},{'a':3}]
Remove duplicate

Learn more about Python data types and theirs functions. Here is one tutorial for dictionaries
As what I know using list of dictionaries is a bad practice
Here is my solution it could be not so elegant, but it is working. Also remove duplicates means remove ALL of them, so I remove all list elements where they were repeated.
d = [{'a': 1, 'b': 2}, {'a': 2, 'b': 2}, {'a': 3}, {'a': 1}]
temp = []
for i in d:
for x, y in i.items():
temp_var = [x, y]
if temp_var in temp:
d.pop(d.index(i))
else:
temp.append(temp_var)
print(d)
# result [{'a': 1, 'b': 2}, {'a': 3}]
P.S.: Keep learning and have a nice day :)

how to add same dictionary to a list with modified value

csv={“a”:0}
list_=[]
for i in range(2):
csv["a"]+=1
list_.append(csv)
print(list_,csv)
I am getting output like this:
[{'a': 2}, {'a': 2}]
I need to get an output like this
[{'a': 1}, {'a': 2}]

Because python pass variables to functions as reference and not as value, you need to pass a copy of the original dictionnary to list_.append():
csv={"a":0}
list_=[]
for i in range(2):
csv["a"]+=1
list_.append(dict(csv))
print(list_)

You can do a list-comprehension:
list_ = [{'a': x} for x in range(1, 3)]
# [{'a': 1}, {'a': 2}]

Pythonic way of finding duplicate maps in a list while ignoring certain keys, and combining the duplicate maps to make a new list

I want to write a code which takes the following inputs:
list (list of maps)
request_keys (list of strings)
operation (add,substract,multiply,concat)
The code would look at the list for the maps having the same value for all keys except the keys given in request_keys. Upon finding two maps for which the value in the search keys match, the code would do the operation (add,multiple,substract,concat) on the two maps and combine them into one map. This combination map would basically replace the other two maps.
i have written the following peice of code to do this. The code only does add operation. It can be extended to make the other operations
In [83]: list
Out[83]:
[{'a': 2, 'b': 3, 'c': 10},
{'a': 2, 'b': 3, 'c': 3},
{'a': 2, 'b': 4, 'c': 4},
{'a': 2, 'b': 3, 'c': 2},
{'a': 2, 'b': 3, 'c': 3}]
In [84]: %cpaste
Pasting code; enter '--' alone on the line to stop or use Ctrl-D.
:def func(list,request_keys):
: new_list = []
: found_indexes = []
: for i in range(0,len(list)):
: new_item = list[i]
: if i in found_indexes:
: continue
: for j in range(0,len(list)):
: if i != j and {k: v for k,v in list[i].iteritems() if k not in request_keys} == {k: v for k,v in list[j].iteritems() if k not in request_keys}:
: found_indexes.append(j)
: for request_key in request_keys:
: new_item[request_key] += list[j][request_key]
: new_list.append(new_item)
: return new_list
:--
In [85]: func(list,['c'])
Out[85]: [{'a': 2, 'b': 3, 'c': 18}, {'a': 2, 'b': 4, 'c': 4}]
In [86]:
What i want to know is, is there a faster, more memory efficient, cleaner and a more pythonic way of doing the same?
Thank you

You manually generate all the combinations and then compare each of those combinations. This is pretty wasteful. Instead, I suggest grouping the dictionaries in another dictionary by their matching keys, then adding the "same" dictionaries. Also, you forgot the operator parameter.
import collections, operator, functools
def func(lst, request_keys, op=operator.add):
matching_dicts = collections.defaultdict(list)
for d in lst:
key = tuple(sorted(((k, d[k]) for k in d if k not in request_keys)))
matching_dicts[key].append(d)
for group in matching_dicts.values():
merged = dict(group[0])
merged.update({key: functools.reduce(op, (g[key] for g in group))
for key in request_keys})
yield merged
What this does: First, it creates a dictionary, mapping the key-value pairs that have to be equal for two dictionaries to match to all those dictionaries that have those key-value pairs. Then it iterates the dicts from those groups, using one of that group as a prototype and updating it with the sum (or product, or whatever, depending on the operator) of the all the dicts in that group for the required_keys.
Note that this returns a generator. If you want a list, just call it like list(func(...)), or accumulate the merged dicts in a list and return that list.

from itertools import groupby
from operator import itemgetter
def mergeDic(inputData, request_keys):
keys = inputData[0].keys()
comparedKeys = [item for item in keys if item not in request_keys]
grouper = itemgetter(*comparedKeys)
result = []
for key, grp in groupby(sorted(inputData, key = grouper), grouper):
temp_dict = dict(zip(comparedKeys, key))
for request_key in request_keys:
temp_dict[request_key] = sum(item[request_key] for item in grp)
result.append(temp_dict)
return result
inputData = [{'a': 2, 'b': 3, 'c': 10},
{'a': 2, 'b': 3, 'c': 3},
{'a': 2, 'b': 4, 'c': 4},
{'a': 2, 'b': 3, 'c': 2},
{'a': 2, 'b': 3, 'c': 3}]
from pprint import pprint
pprint(mergeDic(inputData,['c']))

How to access key in Python dictionary

I know it's possible to traverse a dictionary by using iteritems() or keys(). But what if I got a list of dics such as: l = [{'a': 1}, {'b': 2}, {'c': 3}] and I want to compose a string using its keys, e.g., s = 'a, b, c'?
One solution is to copy all the keys into a list in advance, and compose the string I want. Just wondering if there is a better solution.

You can use itertools.chain() for it and make use of the fact that iterating over a dict will yield its keys.
import itertools
lst = [{'a': 1}, {'b': 2}, {'c': 3}]
print ', '.join(itertools.chain(*lst))
This also works it there are dicts with more than one element.
If you do not want duplicate elements, use a set:
print ', '.join(set(itertools.chain(*lst)))

Assuming each dictionary will only have a single key:
', '.join(a.keys()[0] for a in l)
if not, maybe something like:
>>> l = [{'a': 1, 'd': 4}, {'b': 2}, {'c': 3}]
>>> ', '.join(', '.join(a.keys()) for a in l)
'a, d, b, c'

Iterate over the dictionaries and then over the keys of each dictionary:
>>> lst = [{'a': 1}, {'b': 2}, {'c': 3}]
>>> ', '.join(key for dct in lst for key in dct.keys())
'a, b, c'

Try this:
''.join([i.keys()[0] for i in l])
You can put any delimiter you want inside of the quotes. Just bear in mind that the dictionary isn't ordered, so you may get weird looking strings as a result of this.