I have the following dictionary:
my_dict = {'key1': {'key2': {'foo': 'bar'} } }
and I would like to append an entry to key1->key2->key3 with value 'blah' yielding:
my_dict = {'key1': {'key2': {'foo': 'bar', 'key3': 'blah'} } }
I am looking for a generic solution that is independent of the number of keys, i.e. key1->key2->key3->key4->key5 should work as well, even though keys from key3 on downwards do not exist. So that I get:
my_dict = {'key1': {'key2': {'foo': 'bar', 'key3': {'key4': {'key5': 'blah'} } } } }
Thanks in advance.
You can use the reduce() function to traverse a series of nested dictionaries:
def get_nested(d, path):
return reduce(dict.__getitem__, path, d)
Demo:
>>> def get_nested(d, path):
... return reduce(dict.__getitem__, path, d)
...
>>> my_dict = {'key1': {'key2': {'foo': 'bar', 'key3': {'key4': {'key5': 'blah'}}}}}
>>> get_nested(my_dict, ('key1', 'key2', 'key3', 'key4', 'key5'))
'blah'
This version throws an exception when a key doesn't exist:
>>> get_nested(my_dict, ('key1', 'nonesuch'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in get_nested
KeyError: 'nonesuch'
but you could replace dict.__getitem__ with lambda d, k: d.setdefault(k, {}) to have it create empty dictionaries instead:
def get_nested_default(d, path):
return reduce(lambda d, k: d.setdefault(k, {}), path, d)
Demo:
>>> def get_nested_default(d, path):
... return reduce(lambda d, k: d.setdefault(k, {}), path, d)
...
>>> get_nested_default(my_dict, ('key1', 'nonesuch'))
{}
>>> my_dict
{'key1': {'key2': {'key3': {'key4': {'key5': 'blah'}}, 'foo': 'bar'}, 'nonesuch': {}}}
To set a value at a given path, traverse over all keys but the last one, then use the final key in a regular dictionary assignment:
def set_nested(d, path, value):
get_nested_default(d, path[:-1])[path[-1]] = value
This uses the get_nested_default() function to add empty dictionaries as needed:
>>> def set_nested(d, path, value):
... get_nested_default(d, path[:-1])[path[-1]] = value
...
>>> my_dict = {'key1': {'key2': {'foo': 'bar'}}}
>>> set_nested(my_dict, ('key1', 'key2', 'key3', 'key4', 'key5'), 'blah')
>>> my_dict
{'key1': {'key2': {'key3': {'key4': {'key5': 'blah'}}, 'foo': 'bar'}}}
An alternative to Martijn Pieters's excellent answer would be to use a nested defaultdict, rather than a regular dictionary:
from collections import defaultdict
nested = lambda: defaultdict(nested) # nested dictionary factory
my_dict = nested()
You can set values by using regular nested dictionary access semantics, and empty dictionaries will be created to fill the middle levels as necessary:
my_dict["key1"]["key2"]["key3"] = "blah"
This of course requires that the number of keys be known in advance when you write the code to set the value. If you want to be able to handle a variable-length list of keys, rather than a fixed number, you'll need functions to do the getting and setting for you, like in Martijn's answer.
Related
Very similar to this question but with an added caveat.
I have two lists identical in length. One contains my keys, and the other contains a list of dictionaries belonging to said keys. Example below (yes the Nones are intentional)
keys = ['key1', 'key2, 'key3']
vals = [[{'subkey1': 'val1', 'subkey2': 'val2'}], [{'subkey1: 'val2'},None],[{'subkey1':'val3'}, {'subkey3':'val1}, None, None]]
What I'd like to do is match each dictionary in each list to a key to create a nested dict.
So something like below:
final = {'key1':{'subkey1': 'val1', 'subkey2': 'val2'}, 'key2':{'subkey1': 'val2'}, 'key3':{'subkey1':'val3'}, {'subkey3':'val1'}}
I know we can use dictionary comprehension to do this. I created a dictionary of empty dicts.
s = {}
for i in keys:
for v in vals:
s[i] = {}
break
Then I wrote the following to iterate through the list of dicts I'd like to pair up accounting for the non dictionary values in my list.
x = 0
while x < len(keys):
for i in keys:
for element in vals[x]:
if isinstance(element, dict) == True:
s[str(i)].append(element)
else:
print('no')
x=x+1
However when I run this, I get AttributeError: 'NoneType' object has no attribute 'append'
How can I append to a dictionary using this for loop?
For the less readable dictionary comprehension solution
keys = ['key1', 'key2', 'key3']
vals = [
[{'subkey1': 'val1', 'subkey2': 'val2'}],
[{'subkey1': 'val2'}, None],
[{'subkey1': 'val3'}, {'subkey3':'val1'}, None, None]
]
s = {
k: {
sk: sv for d in (x for x in v if x is not None) for sk, sv in d.items()
} for k, v in zip(keys, vals)
}
Gives
{'key1': {'subkey1': 'val1', 'subkey2': 'val2'},
'key2': {'subkey1': 'val2'},
'key3': {'subkey1': 'val3', 'subkey3': 'val1'}}
This can be done fairly easily using a defaultdict:
from collections import defaultdict
keys = ['key1', 'key2', 'key3']
vals = [
[{'subkey1': 'val1', 'subkey2': 'val2'}],
[{'subkey1': 'val2'},None],
[{'subkey1':'val3'}, {'subkey3':'val1'}, None, None]
]
final = defaultdict(dict)
for idx, key in enumerate(keys):
for d in vals[idx]:
if d is not None:
final[key].update(d)
Output:
defaultdict(<class 'dict'>, {
'key1': {'subkey1': 'val1', 'subkey2': 'val2'},
'key2': {'subkey1': 'val2'},
'key3': {'subkey1': 'val3', 'subkey3': 'val1'}
})
can't use append on a dict
try this maybe?
keys = ['key1', 'key2', 'key3']
vals = [[{'subkey1': 'val1', 'subkey2': 'val2'}], [{'subkey1': 'val2'},None],[{'subkey1':'val3'}, {'subkey3':'val1'}, None, None]]
s = {}
for i in keys:
s[i] = {}
for i, key in enumerate(keys):
for element in vals[i]:
if isinstance(element, dict) == True:
s[str(key)].update(element)
print(s)
Output:
{'key1': {'subkey1': 'val1', 'subkey2': 'val2'}, 'key2': {'subkey1': 'val2'}, 'key3': {'subkey1': 'val3', 'subkey3': 'val1'}}
I have some records like that in collection:
{
'_id': 1,
'test_field': [{'key1': 'value1'}, {'key2': 'value2'}]
}
test_field is a list of dicts. I need to push new dict in that list if any key does not exist and if it does I need to update that key’s value.
Examples:
{'key1': 'test_value'} → 'test_field': [{'key1': 'test_value'}, {'key2': 'value2'}]
{'test_key': 'test_value2'} → 'test_field': [{'key1': 'value1'}, {'key2': 'value2'}, {'test_key': 'test_value_2'}]
Help please
if you need a function in python to do it, this might work for you.
def modify_test_field(my_dict, test_field, new_key, new_val):
my_dict[test_field] = [obj for obj in my_dict[test_field] if new_key not in obj]
my_dict[test_field].append({new_key: new_val})
and call it like modify_test_field(orig_dict, 'test_field', new_key, new_val)
How can I turn a list of dictionaries into a single dictionary?
For example, let's say my initial list is as:
Dictionary_list = [{key:value}, {key2:value2}, {key3:value3}]
I need the resultant dictionary as:
New_dictionary = {key:value, key2:value2, key3:value3}
Another solution would be to create an empty dictionary and update it:
>>> my_list = [{'key':'value'}, {'key2':'value2'}, {'key3':'value3'}]
>>> my_dict = {}
>>> for d in my_list: my_dict.update(d)
...
>>> my_dict
{'key': 'value', 'key2': 'value2', 'key3': 'value3'}
In general, the update() method is mighty useful, typically when you want to create "environments" containing variables from successive dictionaries.
You may use dictionary comprehension to achieve this as:
>>> my_list = [{'key':'value'}, {'key2':'value2'}, {'key3':'value3'}]
>>> my_dict = {k: v for item in my_list for k, v in item.items()}
>>> my_dict
{'key3': 'value3', 'key2': 'value2', 'key': 'value'}
Note: If your initial list of dictionaries will have any "key" present in more than one dictionary, above solution will end up keeping the last "value" of "key" from the initial list.
Functional programming answer:
from functools import reduce # depending on version of python you might need this.
my_list = [{'key':'value'}, {'key2':'value2'}, {'key3':'value3'}]
def func(x,y):
x.update(y)
return x
new_dict = reduce(func, my_list)
>>> new_dict
{'key': 'value', 'key2': 'value2', 'key3': 'value3'}
One liner:
new_dict = reduce(lambda x, y: x.update(y) or x, my_list) # use side effect in lambda
I'm trying to create a defaultdict within a defaultdict based on a key value. My thinking may be completely wrong here, but here's the code for a basic defaultdict;
def record():
return {
'count': 0,
'key1': Counter(),
}
1_record = defaultdict(record)
But what if I want to add a key as a defaultdict like this;
def record():
return {
'count': 0,
'key1': Counter(),
'key2': {
'count': 0,
'nested_key1': Counter()
}
}
In the above how could I make 'key2' a defaultdict? Is this even possible or am I approaching the problem the wrong way?
You definitely can have a "recursive" defaultdict:
>>> from collections import defaultdict
>>> def record():
... return {
... 'key': defaultdict(record)
... }
...
>>> d = defaultdict(record)
>>>
>>> d['foo']
{'key': defaultdict(<function record at 0x10b396f50>, {})}
>>> d['foo']['key']['bar']
{'key': defaultdict(<function record at 0x10b396f50>, {})}
>>> d
defaultdict(<function record at 0x10b396f50>, {'foo': {'key': defaultdict(<function record at 0x10b396f50>, {'bar': {'key': defaultdict(<function record at 0x10b396f50>, {})}})}})
However, swapping out the names of the keys at different levels would probably require some special-casing and would make the code a bit more messy...
I would like to create a python directory whose values are to be evaluated separately.
So, for example, in the following non-working example I define
a = {'key1': 'value1', 'key2': 42, 'key3': foo(20)}
for which e.g.
def foo(max):
"""Returns random float between 0 and max."""
return max*random.random()
and after processing of the dict I want to have e.g.
a_processes = {'key1': 'value1', 'key2': 42, 'key3': 12.238746374}
The example does not work since the value of key key3 is immediately evaluated, and foo(20) is no callable. The way it could work would be to use something like
a = {'key1': 'value1', 'key2': 42, 'key3': foo}
but here foo will miss its arguments. One way to handle this would be to define the dict in the following way
a = {'key1': 'value1', 'key2': 42, 'key3': [foo, 20]}
with the following processing scheme
a_processed = dict([k,process(v)] for k,v in a.items())
in which process is a meaningful function checking if its argument is a list, or if the first element is a callable which gets called with the remaining arguments.
My question is about a better way/idea to implement my basic idea?
Use the functools.partial() to create a callable that'll apply a set of arguments to another callable:
from functools import partial
a = {'key1': 'value1', 'key2': 42, 'key3': partial(foo, 20)}
Now the key3 value is a callable object; a['key3']() will in turn call foo(20) and return the result.
You'd still need to 'process' this dictionary (testing with the callable() function:
{k: v() if callable(v) else v for k, v in a.iteritems()}
of course. If you are using Python 3, use a.items() instead.
A functools.partial setup let's you add additional arguments as needed (provided foo() supports them):
a['key3']('additional', arguments='accepted')
You could even create a dictionary subclass that does this automatically when accessing a key:
def CallablesDict(dict):
def __getitem__(self, key):
value = super(CallablesDict, self).__getitem__(key)
return value() if callable(value) else value
then define a as:
a = CallablesDict({'key1': 'value1', 'key2': 42, 'key3': partial(foo, 20)})
print a['key3'] # calls foo(20)
I often use an argument-free lambda here, just because I like the syntax:
>>> def foo(x):
... return x*100
...
>>> a = {'key1': 'value1', 'key2': 42, 'key3': lambda: foo(20)}
>>> a
{'key3': <function <lambda> at 0x96ae80c>, 'key2': 42, 'key1': 'value1'}
>>> {k: v() if callable(v) else v for k,v in a.iteritems()}
{'key3': 2000, 'key2': 42, 'key1': 'value1'}