Related
I need to compare only the keys of two nested dictionaries. The primary usage is for the live tests of external API responses to prevent response change.
For example, this two dictionary matched, however their values differ:
EDIT: this is a sample and the actual dictionaries have dynamic keys, probably larger, and consists of integers, strings, and boolean
dict1 = {"guid": {"id": {"addr": "foo", "creation_num": "4"}}}
dict2 = {"guid": {"id": {"addr": "bar", "creation_num": "2"}}}
I try to do this by resetting the values of dictionaries with this method:
def reset_values(dictionary, reset_value=0):
for key, value in dictionary.items():
if type(value) is dict:
dictionary[key] = reset_values(dictionary[key], reset_value)
else:
dictionary[key] = reset_value
return dictionary
This method works, but is there a more Pythonic and straightforward way?
EDIT
#Bheid is correct that by flattening the key list my solution would get tripped up when two dictionaries have the same keys but at different nesting levels. Any easy fix is to change this line:
if isinstance(v, dict):
klist.extend(get_keys(v))
to:
if isinstance(v, dict):
klist.append(get_keys(v))
Same idea, but the edited version preserves nested key levels.
If I understand the problem you are trying to solve it is to the compare keys of the two dictionaries (as well as subkeys of nested dictionaries) irrespective of the associated values. If two dictionaries have the same keys (and subkeys) then they are the "same" for your purposes. If that problem statement is correct then generating an in-order list of keys/sub-keys for one dictionary and comparing that list to the same for a second dictionary should be sufficient for your purposes:
dict1 = {"guid": {"id": {"addr": "foo", "creation_num": "4"}}}
dict2 = {"guid": {"id": {"addr": "bar", "creation_num": "2"}}}
def get_keys(d):
klist = []
for k, v in d.items():
klist.append(k)
if isinstance(v, dict):
klist.extend(get_keys(v))
return klist
print(get_keys(dict1) == get_keys(dict2))
Output:
True
You can solve this by creating a dictionary hash of keys. This solution also takes into account different key ordering.
import hashlib
def has_exact_keys(d: dict, z: dict) -> bool:
return dict_keys_hash(d) == dict_keys_hash(z)
def dict_keys_hash(d: dict) -> str:
hash = hashlib.md5() # switch to different hashing algorithm if you want
for k in sorted(d.keys()):
if isinstance(d[k], dict):
hash.update(dict_hash(d[k]).encode('utf-8'))
hash.update(k.encode('utf-8'))
return hash.hexdigest()
Disclaimer: I am the author of the ndicts package recommended in this answer.
Answer
You can use my package ndicts to do that. To install it:
pip install ndicts
Then simply convert your dictionaries into NestedDict:
from ndicts import NestedDict
dict1 = {"guid": {"id": {"addr": "foo", "creation_num": "4"}}}
dict2 = {"guid": {"id": {"addr": "bar", "creation_num": "2"}}}
nd1 = NestedDict(dict1)
nd2 = NestedDict(dict2)
You can then check if the keys are equal:
>>> nd1.keys() == nd2.keys()
True
Here is what the keys of a NestedDict are:
>>> for k in nd1.keys():
... print(k)
('guid', 'id', 'addr')
('guid', 'id', 'creation_num')
Comment to your solution
Your method works but as a potentially unwanted side effect it will modify the input dictionary:
>>> dict1
{"guid": {"id": {"addr": "foo", "creation_num": "4"}}}
>>> reset_values(dict1)
{'guid': {'id': {'addr': 0, 'creation_num': 0}}}
>>> dict1
{'guid': {'id': {'addr': 0, 'creation_num': 0}}}
This can be easily fixed by deepcopying the input dictionary at the beginning of your function.
You are using recursion, which is fine. However, you can also use a for loop or reduce which have some advantages. See answer to "Best way to get nested dictionary items".
One last thing, I would aim at creating a function that compares keys directly or that returns all the keys in an iterable, rather than resetting the values of your dictionaries and then comparing them.
Let's say I have a dict that looks like this:
d['a']['1'] = 'foo'
d['a']['2'] = 'bar'
d['b']['1'] = 'baz'
d['b']['2'] = 'boo'
If I want to get every item where the first key is 'a', I can just do d['a'] and I will get all of them. However, what if I want to get all items where the second key is '1'? The only way I can think of is to make a second dictionary with a reverse order of the keys, which requires duplicating the contents. Is there a way to do this within a single structure?
Edit: forgot to mention: I want to do this without iterating over everything. I'm going to be dealing with dicts with hundreds of thousands of keys, so I need something scalable.
You're dealing with three dictionaries in this example: One with the values "foo" and "bar", one with the values "baz" and "boo", and an outer dictionary that maps the keys "a" and "b" to those first two inner dictionaries. You can iterate over the keys of both the outer and inner dictionaries with a nested for loop:
items = []
for outer_key in d:
for inner_key in d[outer_key]:
if inner_key == "1":
items.append(d[outer_key][inner_key])
break # No need to keep checking keys once you've found a match
If you don't care about the keys of the outer dictionary, you can also use d.values() to ignore the keys and just see the inner dictionaries, then do a direct membership check on those:
items = []
for inner_dict in d.values():
if "1" in inner_dict:
items.append(inner_dict["1"])
This can also be written as a list comprehension:
items = [inner_dict["1"] for inner_dict in d.values() if "1" in inner_dict]
What you want sounds very similar to a tree-structure which can be implemented as a dictionary-of-dictionaries. Here's a simple implement taken from one of the answers to the question What is the best way to implement nested dictionaries?:
class Tree(dict):
def __missing__(self, key):
value = self[key] = type(self)()
return value
def get_second_level(self, second_key):
found = []
for level2 in self.values():
if second_key in level2:
found.append(level2[second_key])
return found
d = Tree()
d['a']['1'] = 'foo'
d['a']['2'] = 'bar'
d['b']['1'] = 'baz'
d['b']['2'] = 'boo'
d['c']['2'] = 'mox'
d['c']['3'] = 'nix'
print(d) # -> {'a': {'1': 'foo', '2': 'bar'}, 'b': {'1': 'baz', '2': 'boo'},
# 'c': {'2': 'mox', '3': 'nix'}}
print(d['a']) # -> {'1': 'foo', '2': 'bar'}
print(d['a']['1']) # -> foo
print(d['a']['2']) # -> bar
print()
second_key = '1'
found = d.get_second_level(second_key)
print(f'Those with a second key of {second_key!r}') # -> Those with a second key of '1'
print(f' {found}') # -> ['foo', 'baz']
So after sleeping on it the solution I came up with was to make three dicts, the main one where the data is actually stored and identified by a tuple (d['a', '1'] = 'foo') and the other two are indexes that store all possible values of key B under key A where (A,B) is a valid combination (so a['a'] = ['1', '2'], b['1'] = ['a', 'b']. I don't entirely like this, since it still requires a hefty storage overhead and doesn't scale efficiently to higher numbers of keys, but it gets the job done without iterating and without duplicating the data. If anyone has a better idea, I'll be happy to hear it.
How do I add a key to an existing dictionary? It doesn't have an .add() method.
You create a new key/value pair on a dictionary by assigning a value to that key
d = {'key': 'value'}
print(d) # {'key': 'value'}
d['mynewkey'] = 'mynewvalue'
print(d) # {'key': 'value', 'mynewkey': 'mynewvalue'}
If the key doesn't exist, it's added and points to that value. If it exists, the current value it points to is overwritten.
I feel like consolidating info about Python dictionaries:
Creating an empty dictionary
data = {}
# OR
data = dict()
Creating a dictionary with initial values
data = {'a': 1, 'b': 2, 'c': 3}
# OR
data = dict(a=1, b=2, c=3)
# OR
data = {k: v for k, v in (('a', 1), ('b',2), ('c',3))}
Inserting/Updating a single value
data['a'] = 1 # Updates if 'a' exists, else adds 'a'
# OR
data.update({'a': 1})
# OR
data.update(dict(a=1))
# OR
data.update(a=1)
Inserting/Updating multiple values
data.update({'c':3,'d':4}) # Updates 'c' and adds 'd'
Python 3.9+:
The update operator |= now works for dictionaries:
data |= {'c':3,'d':4}
Creating a merged dictionary without modifying originals
data3 = {}
data3.update(data) # Modifies data3, not data
data3.update(data2) # Modifies data3, not data2
Python 3.5+:
This uses a new feature called dictionary unpacking.
data = {**data1, **data2, **data3}
Python 3.9+:
The merge operator | now works for dictionaries:
data = data1 | {'c':3,'d':4}
Deleting items in dictionary
del data[key] # Removes specific element in a dictionary
data.pop(key) # Removes the key & returns the value
data.clear() # Clears entire dictionary
Check if a key is already in dictionary
key in data
Iterate through pairs in a dictionary
for key in data: # Iterates just through the keys, ignoring the values
for key, value in d.items(): # Iterates through the pairs
for key in d.keys(): # Iterates just through key, ignoring the values
for value in d.values(): # Iterates just through value, ignoring the keys
Create a dictionary from two lists
data = dict(zip(list_with_keys, list_with_values))
To add multiple keys simultaneously, use dict.update():
>>> x = {1:2}
>>> print(x)
{1: 2}
>>> d = {3:4, 5:6, 7:8}
>>> x.update(d)
>>> print(x)
{1: 2, 3: 4, 5: 6, 7: 8}
For adding a single key, the accepted answer has less computational overhead.
"Is it possible to add a key to a Python dictionary after it has been created? It doesn't seem to have an .add() method."
Yes it is possible, and it does have a method that implements this, but you don't want to use it directly.
To demonstrate how and how not to use it, let's create an empty dict with the dict literal, {}:
my_dict = {}
Best Practice 1: Subscript notation
To update this dict with a single new key and value, you can use the subscript notation (see Mappings here) that provides for item assignment:
my_dict['new key'] = 'new value'
my_dict is now:
{'new key': 'new value'}
Best Practice 2: The update method - 2 ways
We can also update the dict with multiple values efficiently as well using the update method. We may be unnecessarily creating an extra dict here, so we hope our dict has already been created and came from or was used for another purpose:
my_dict.update({'key 2': 'value 2', 'key 3': 'value 3'})
my_dict is now:
{'key 2': 'value 2', 'key 3': 'value 3', 'new key': 'new value'}
Another efficient way of doing this with the update method is with keyword arguments, but since they have to be legitimate python words, you can't have spaces or special symbols or start the name with a number, but many consider this a more readable way to create keys for a dict, and here we certainly avoid creating an extra unnecessary dict:
my_dict.update(foo='bar', foo2='baz')
and my_dict is now:
{'key 2': 'value 2', 'key 3': 'value 3', 'new key': 'new value',
'foo': 'bar', 'foo2': 'baz'}
So now we have covered three Pythonic ways of updating a dict.
Magic method, __setitem__, and why it should be avoided
There's another way of updating a dict that you shouldn't use, which uses the __setitem__ method. Here's an example of how one might use the __setitem__ method to add a key-value pair to a dict, and a demonstration of the poor performance of using it:
>>> d = {}
>>> d.__setitem__('foo', 'bar')
>>> d
{'foo': 'bar'}
>>> def f():
... d = {}
... for i in xrange(100):
... d['foo'] = i
...
>>> def g():
... d = {}
... for i in xrange(100):
... d.__setitem__('foo', i)
...
>>> import timeit
>>> number = 100
>>> min(timeit.repeat(f, number=number))
0.0020880699157714844
>>> min(timeit.repeat(g, number=number))
0.005071878433227539
So we see that using the subscript notation is actually much faster than using __setitem__. Doing the Pythonic thing, that is, using the language in the way it was intended to be used, usually is both more readable and computationally efficient.
dictionary[key] = value
If you want to add a dictionary within a dictionary you can do it this way.
Example: Add a new entry to your dictionary & sub dictionary
dictionary = {}
dictionary["new key"] = "some new entry" # add new dictionary entry
dictionary["dictionary_within_a_dictionary"] = {} # this is required by python
dictionary["dictionary_within_a_dictionary"]["sub_dict"] = {"other" : "dictionary"}
print (dictionary)
Output:
{'new key': 'some new entry', 'dictionary_within_a_dictionary': {'sub_dict': {'other': 'dictionarly'}}}
NOTE: Python requires that you first add a sub
dictionary["dictionary_within_a_dictionary"] = {}
before adding entries.
The conventional syntax is d[key] = value, but if your keyboard is missing the square bracket keys you could also do:
d.__setitem__(key, value)
In fact, defining __getitem__ and __setitem__ methods is how you can make your own class support the square bracket syntax. See Dive Into Python, Classes That Act Like Dictionaries.
You can create one:
class myDict(dict):
def __init__(self):
self = dict()
def add(self, key, value):
self[key] = value
## example
myd = myDict()
myd.add('apples',6)
myd.add('bananas',3)
print(myd)
Gives:
>>>
{'apples': 6, 'bananas': 3}
This popular question addresses functional methods of merging dictionaries a and b.
Here are some of the more straightforward methods (tested in Python 3)...
c = dict( a, **b ) ## see also https://stackoverflow.com/q/2255878
c = dict( list(a.items()) + list(b.items()) )
c = dict( i for d in [a,b] for i in d.items() )
Note: The first method above only works if the keys in b are strings.
To add or modify a single element, the b dictionary would contain only that one element...
c = dict( a, **{'d':'dog'} ) ## returns a dictionary based on 'a'
This is equivalent to...
def functional_dict_add( dictionary, key, value ):
temp = dictionary.copy()
temp[key] = value
return temp
c = functional_dict_add( a, 'd', 'dog' )
Let's pretend you want to live in the immutable world and do not want to modify the original but want to create a new dict that is the result of adding a new key to the original.
In Python 3.5+ you can do:
params = {'a': 1, 'b': 2}
new_params = {**params, **{'c': 3}}
The Python 2 equivalent is:
params = {'a': 1, 'b': 2}
new_params = dict(params, **{'c': 3})
After either of these:
params is still equal to {'a': 1, 'b': 2}
and
new_params is equal to {'a': 1, 'b': 2, 'c': 3}
There will be times when you don't want to modify the original (you only want the result of adding to the original). I find this a refreshing alternative to the following:
params = {'a': 1, 'b': 2}
new_params = params.copy()
new_params['c'] = 3
or
params = {'a': 1, 'b': 2}
new_params = params.copy()
new_params.update({'c': 3})
Reference: What does `**` mean in the expression `dict(d1, **d2)`?
There is also the strangely named, oddly behaved, and yet still handy dict.setdefault().
This
value = my_dict.setdefault(key, default)
basically just does this:
try:
value = my_dict[key]
except KeyError: # key not found
value = my_dict[key] = default
E.g.,
>>> mydict = {'a':1, 'b':2, 'c':3}
>>> mydict.setdefault('d', 4)
4 # returns new value at mydict['d']
>>> print(mydict)
{'a':1, 'b':2, 'c':3, 'd':4} # a new key/value pair was indeed added
# but see what happens when trying it on an existing key...
>>> mydict.setdefault('a', 111)
1 # old value was returned
>>> print(mydict)
{'a':1, 'b':2, 'c':3, 'd':4} # existing key was ignored
This question has already been answered ad nauseam, but since my
comment
gained a lot of traction, here it is as an answer:
Adding new keys without updating the existing dict
If you are here trying to figure out how to add a key and return a new dictionary (without modifying the existing one), you can do this using the techniques below
Python >= 3.5
new_dict = {**mydict, 'new_key': new_val}
Python < 3.5
new_dict = dict(mydict, new_key=new_val)
Note that with this approach, your key will need to follow the rules of valid identifier names in Python.
If you're not joining two dictionaries, but adding new key-value pairs to a dictionary, then using the subscript notation seems like the best way.
import timeit
timeit.timeit('dictionary = {"karga": 1, "darga": 2}; dictionary.update({"aaa": 123123, "asd": 233})')
>> 0.49582505226135254
timeit.timeit('dictionary = {"karga": 1, "darga": 2}; dictionary["aaa"] = 123123; dictionary["asd"] = 233;')
>> 0.20782899856567383
However, if you'd like to add, for example, thousands of new key-value pairs, you should consider using the update() method.
Here's another way that I didn't see here:
>>> foo = dict(a=1,b=2)
>>> foo
{'a': 1, 'b': 2}
>>> goo = dict(c=3,**foo)
>>> goo
{'c': 3, 'a': 1, 'b': 2}
You can use the dictionary constructor and implicit expansion to reconstruct a dictionary. Moreover, interestingly, this method can be used to control the positional order during dictionary construction (post Python 3.6). In fact, insertion order is guaranteed for Python 3.7 and above!
>>> foo = dict(a=1,b=2,c=3,d=4)
>>> new_dict = {k: v for k, v in list(foo.items())[:2]}
>>> new_dict
{'a': 1, 'b': 2}
>>> new_dict.update(newvalue=99)
>>> new_dict
{'a': 1, 'b': 2, 'newvalue': 99}
>>> new_dict.update({k: v for k, v in list(foo.items())[2:]})
>>> new_dict
{'a': 1, 'b': 2, 'newvalue': 99, 'c': 3, 'd': 4}
>>>
The above is using dictionary comprehension.
First to check whether the key already exists:
a={1:2,3:4}
a.get(1)
2
a.get(5)
None
Then you can add the new key and value.
Add a dictionary (key,value) class.
class myDict(dict):
def __init__(self):
self = dict()
def add(self, key, value):
#self[key] = value # add new key and value overwriting any exiting same key
if self.get(key)!=None:
print('key', key, 'already used') # report if key already used
self.setdefault(key, value) # if key exit do nothing
## example
myd = myDict()
name = "fred"
myd.add('apples',6)
print('\n', myd)
myd.add('bananas',3)
print('\n', myd)
myd.add('jack', 7)
print('\n', myd)
myd.add(name, myd)
print('\n', myd)
myd.add('apples', 23)
print('\n', myd)
myd.add(name, 2)
print(myd)
I think it would also be useful to point out Python's collections module that consists of many useful dictionary subclasses and wrappers that simplify the addition and modification of data types in a dictionary, specifically defaultdict:
dict subclass that calls a factory function to supply missing values
This is particularly useful if you are working with dictionaries that always consist of the same data types or structures, for example a dictionary of lists.
>>> from collections import defaultdict
>>> example = defaultdict(int)
>>> example['key'] += 1
>>> example['key']
defaultdict(<class 'int'>, {'key': 1})
If the key does not yet exist, defaultdict assigns the value given (in our case 10) as the initial value to the dictionary (often used inside loops). This operation therefore does two things: it adds a new key to a dictionary (as per question), and assigns the value if the key doesn't yet exist. With the standard dictionary, this would have raised an error as the += operation is trying to access a value that doesn't yet exist:
>>> example = dict()
>>> example['key'] += 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'key'
Without the use of defaultdict, the amount of code to add a new element would be much greater and perhaps looks something like:
# This type of code would often be inside a loop
if 'key' not in example:
example['key'] = 0 # add key and initial value to dict; could also be a list
example['key'] += 1 # this is implementing a counter
defaultdict can also be used with complex data types such as list and set:
>>> example = defaultdict(list)
>>> example['key'].append(1)
>>> example
defaultdict(<class 'list'>, {'key': [1]})
Adding an element automatically initialises the list.
Adding keys to dictionary without using add
# Inserting/Updating single value
# subscript notation method
d['mynewkey'] = 'mynewvalue' # Updates if 'a' exists, else adds 'a'
# OR
d.update({'mynewkey': 'mynewvalue'})
# OR
d.update(dict('mynewkey'='mynewvalue'))
# OR
d.update('mynewkey'='mynewvalue')
print(d) # {'key': 'value', 'mynewkey': 'mynewvalue'}
# To add/update multiple keys simultaneously, use d.update():
x = {3:4, 5:6, 7:8}
d.update(x)
print(d) # {'key': 'value', 'mynewkey': 'mynewvalue', 3: 4, 5: 6, 7: 8}
# update operator |= now works for dictionaries:
d |= {'c':3,'d':4}
# Assigning new key value pair using dictionary unpacking.
data1 = {4:6, 9:10, 17:20}
data2 = {20:30, 32:48, 90:100}
data3 = { 38:"value", 99:"notvalid"}
d = {**data1, **data2, **data3}
# The merge operator | now works for dictionaries:
data = data1 | {'c':3,'d':4}
# Create a dictionary from two lists
data = dict(zip(list_with_keys, list_with_values))
dico["new key"] = "value"
So I have an array of tuples something like this
query_results = [("foo", "bar"), ("foo", "qux"), ("baz", "foo")]
I would like to achieve something like:
{
"foo": ["bar", "qux"],
"baz": ["foo"]
}
So I have tried using this
from itertools import groupby
grouped_results = {}
for key, y in groupby(query_results, lambda x: x[0]):
grouped_results[key] = [y[1] for u in list(y)]
The issue I have is although the number of keys are correct, the number of values in each array is dramatically lower than it should be. Can anyone explain why this happens and what I should be doing?
You better use a defaultdict for this:
from collections import defaultdict
result = defaultdict(list)
for k,v in query_results:
result[k].append(v)
Which yields:
>>> result
defaultdict(<class 'list'>, {'baz': ['foo'], 'foo': ['bar', 'qux']})
If you wish to turn it into a vanilla dictionary again, you can - after the for loop - use:
result = dict(result)
this then results in:
>>> dict(result)
{'baz': ['foo'], 'foo': ['bar', 'qux']}
A defaultdict is constructed with a factory, here list. In case the key cannot be found in the dictionary, the factory is called (list() constructs a new empty list). The result is then associated with the key.
So for each key k that is not yet in the dictionary, we will construct a new list first. We then call .append(v) on that list to append values to it.
Well why not use a simple for loop?
grouped_results = {}
for key, value in query_results:
grouped_results.setdefault(key, []).append(value)
Output:
{'foo': ['bar', 'qux'], 'baz': ['foo']}
How about using a defaultdict?
d = defaultdict(list)
for pair in query_results:
d[pair[0]].append(pair[1])
How do I add a key to an existing dictionary? It doesn't have an .add() method.
You create a new key/value pair on a dictionary by assigning a value to that key
d = {'key': 'value'}
print(d) # {'key': 'value'}
d['mynewkey'] = 'mynewvalue'
print(d) # {'key': 'value', 'mynewkey': 'mynewvalue'}
If the key doesn't exist, it's added and points to that value. If it exists, the current value it points to is overwritten.
I feel like consolidating info about Python dictionaries:
Creating an empty dictionary
data = {}
# OR
data = dict()
Creating a dictionary with initial values
data = {'a': 1, 'b': 2, 'c': 3}
# OR
data = dict(a=1, b=2, c=3)
# OR
data = {k: v for k, v in (('a', 1), ('b',2), ('c',3))}
Inserting/Updating a single value
data['a'] = 1 # Updates if 'a' exists, else adds 'a'
# OR
data.update({'a': 1})
# OR
data.update(dict(a=1))
# OR
data.update(a=1)
Inserting/Updating multiple values
data.update({'c':3,'d':4}) # Updates 'c' and adds 'd'
Python 3.9+:
The update operator |= now works for dictionaries:
data |= {'c':3,'d':4}
Creating a merged dictionary without modifying originals
data3 = {}
data3.update(data) # Modifies data3, not data
data3.update(data2) # Modifies data3, not data2
Python 3.5+:
This uses a new feature called dictionary unpacking.
data = {**data1, **data2, **data3}
Python 3.9+:
The merge operator | now works for dictionaries:
data = data1 | {'c':3,'d':4}
Deleting items in dictionary
del data[key] # Removes specific element in a dictionary
data.pop(key) # Removes the key & returns the value
data.clear() # Clears entire dictionary
Check if a key is already in dictionary
key in data
Iterate through pairs in a dictionary
for key in data: # Iterates just through the keys, ignoring the values
for key, value in d.items(): # Iterates through the pairs
for key in d.keys(): # Iterates just through key, ignoring the values
for value in d.values(): # Iterates just through value, ignoring the keys
Create a dictionary from two lists
data = dict(zip(list_with_keys, list_with_values))
To add multiple keys simultaneously, use dict.update():
>>> x = {1:2}
>>> print(x)
{1: 2}
>>> d = {3:4, 5:6, 7:8}
>>> x.update(d)
>>> print(x)
{1: 2, 3: 4, 5: 6, 7: 8}
For adding a single key, the accepted answer has less computational overhead.
"Is it possible to add a key to a Python dictionary after it has been created? It doesn't seem to have an .add() method."
Yes it is possible, and it does have a method that implements this, but you don't want to use it directly.
To demonstrate how and how not to use it, let's create an empty dict with the dict literal, {}:
my_dict = {}
Best Practice 1: Subscript notation
To update this dict with a single new key and value, you can use the subscript notation (see Mappings here) that provides for item assignment:
my_dict['new key'] = 'new value'
my_dict is now:
{'new key': 'new value'}
Best Practice 2: The update method - 2 ways
We can also update the dict with multiple values efficiently as well using the update method. We may be unnecessarily creating an extra dict here, so we hope our dict has already been created and came from or was used for another purpose:
my_dict.update({'key 2': 'value 2', 'key 3': 'value 3'})
my_dict is now:
{'key 2': 'value 2', 'key 3': 'value 3', 'new key': 'new value'}
Another efficient way of doing this with the update method is with keyword arguments, but since they have to be legitimate python words, you can't have spaces or special symbols or start the name with a number, but many consider this a more readable way to create keys for a dict, and here we certainly avoid creating an extra unnecessary dict:
my_dict.update(foo='bar', foo2='baz')
and my_dict is now:
{'key 2': 'value 2', 'key 3': 'value 3', 'new key': 'new value',
'foo': 'bar', 'foo2': 'baz'}
So now we have covered three Pythonic ways of updating a dict.
Magic method, __setitem__, and why it should be avoided
There's another way of updating a dict that you shouldn't use, which uses the __setitem__ method. Here's an example of how one might use the __setitem__ method to add a key-value pair to a dict, and a demonstration of the poor performance of using it:
>>> d = {}
>>> d.__setitem__('foo', 'bar')
>>> d
{'foo': 'bar'}
>>> def f():
... d = {}
... for i in xrange(100):
... d['foo'] = i
...
>>> def g():
... d = {}
... for i in xrange(100):
... d.__setitem__('foo', i)
...
>>> import timeit
>>> number = 100
>>> min(timeit.repeat(f, number=number))
0.0020880699157714844
>>> min(timeit.repeat(g, number=number))
0.005071878433227539
So we see that using the subscript notation is actually much faster than using __setitem__. Doing the Pythonic thing, that is, using the language in the way it was intended to be used, usually is both more readable and computationally efficient.
dictionary[key] = value
If you want to add a dictionary within a dictionary you can do it this way.
Example: Add a new entry to your dictionary & sub dictionary
dictionary = {}
dictionary["new key"] = "some new entry" # add new dictionary entry
dictionary["dictionary_within_a_dictionary"] = {} # this is required by python
dictionary["dictionary_within_a_dictionary"]["sub_dict"] = {"other" : "dictionary"}
print (dictionary)
Output:
{'new key': 'some new entry', 'dictionary_within_a_dictionary': {'sub_dict': {'other': 'dictionarly'}}}
NOTE: Python requires that you first add a sub
dictionary["dictionary_within_a_dictionary"] = {}
before adding entries.
The conventional syntax is d[key] = value, but if your keyboard is missing the square bracket keys you could also do:
d.__setitem__(key, value)
In fact, defining __getitem__ and __setitem__ methods is how you can make your own class support the square bracket syntax. See Dive Into Python, Classes That Act Like Dictionaries.
You can create one:
class myDict(dict):
def __init__(self):
self = dict()
def add(self, key, value):
self[key] = value
## example
myd = myDict()
myd.add('apples',6)
myd.add('bananas',3)
print(myd)
Gives:
>>>
{'apples': 6, 'bananas': 3}
This popular question addresses functional methods of merging dictionaries a and b.
Here are some of the more straightforward methods (tested in Python 3)...
c = dict( a, **b ) ## see also https://stackoverflow.com/q/2255878
c = dict( list(a.items()) + list(b.items()) )
c = dict( i for d in [a,b] for i in d.items() )
Note: The first method above only works if the keys in b are strings.
To add or modify a single element, the b dictionary would contain only that one element...
c = dict( a, **{'d':'dog'} ) ## returns a dictionary based on 'a'
This is equivalent to...
def functional_dict_add( dictionary, key, value ):
temp = dictionary.copy()
temp[key] = value
return temp
c = functional_dict_add( a, 'd', 'dog' )
Let's pretend you want to live in the immutable world and do not want to modify the original but want to create a new dict that is the result of adding a new key to the original.
In Python 3.5+ you can do:
params = {'a': 1, 'b': 2}
new_params = {**params, **{'c': 3}}
The Python 2 equivalent is:
params = {'a': 1, 'b': 2}
new_params = dict(params, **{'c': 3})
After either of these:
params is still equal to {'a': 1, 'b': 2}
and
new_params is equal to {'a': 1, 'b': 2, 'c': 3}
There will be times when you don't want to modify the original (you only want the result of adding to the original). I find this a refreshing alternative to the following:
params = {'a': 1, 'b': 2}
new_params = params.copy()
new_params['c'] = 3
or
params = {'a': 1, 'b': 2}
new_params = params.copy()
new_params.update({'c': 3})
Reference: What does `**` mean in the expression `dict(d1, **d2)`?
There is also the strangely named, oddly behaved, and yet still handy dict.setdefault().
This
value = my_dict.setdefault(key, default)
basically just does this:
try:
value = my_dict[key]
except KeyError: # key not found
value = my_dict[key] = default
E.g.,
>>> mydict = {'a':1, 'b':2, 'c':3}
>>> mydict.setdefault('d', 4)
4 # returns new value at mydict['d']
>>> print(mydict)
{'a':1, 'b':2, 'c':3, 'd':4} # a new key/value pair was indeed added
# but see what happens when trying it on an existing key...
>>> mydict.setdefault('a', 111)
1 # old value was returned
>>> print(mydict)
{'a':1, 'b':2, 'c':3, 'd':4} # existing key was ignored
This question has already been answered ad nauseam, but since my
comment
gained a lot of traction, here it is as an answer:
Adding new keys without updating the existing dict
If you are here trying to figure out how to add a key and return a new dictionary (without modifying the existing one), you can do this using the techniques below
Python >= 3.5
new_dict = {**mydict, 'new_key': new_val}
Python < 3.5
new_dict = dict(mydict, new_key=new_val)
Note that with this approach, your key will need to follow the rules of valid identifier names in Python.
If you're not joining two dictionaries, but adding new key-value pairs to a dictionary, then using the subscript notation seems like the best way.
import timeit
timeit.timeit('dictionary = {"karga": 1, "darga": 2}; dictionary.update({"aaa": 123123, "asd": 233})')
>> 0.49582505226135254
timeit.timeit('dictionary = {"karga": 1, "darga": 2}; dictionary["aaa"] = 123123; dictionary["asd"] = 233;')
>> 0.20782899856567383
However, if you'd like to add, for example, thousands of new key-value pairs, you should consider using the update() method.
Here's another way that I didn't see here:
>>> foo = dict(a=1,b=2)
>>> foo
{'a': 1, 'b': 2}
>>> goo = dict(c=3,**foo)
>>> goo
{'c': 3, 'a': 1, 'b': 2}
You can use the dictionary constructor and implicit expansion to reconstruct a dictionary. Moreover, interestingly, this method can be used to control the positional order during dictionary construction (post Python 3.6). In fact, insertion order is guaranteed for Python 3.7 and above!
>>> foo = dict(a=1,b=2,c=3,d=4)
>>> new_dict = {k: v for k, v in list(foo.items())[:2]}
>>> new_dict
{'a': 1, 'b': 2}
>>> new_dict.update(newvalue=99)
>>> new_dict
{'a': 1, 'b': 2, 'newvalue': 99}
>>> new_dict.update({k: v for k, v in list(foo.items())[2:]})
>>> new_dict
{'a': 1, 'b': 2, 'newvalue': 99, 'c': 3, 'd': 4}
>>>
The above is using dictionary comprehension.
First to check whether the key already exists:
a={1:2,3:4}
a.get(1)
2
a.get(5)
None
Then you can add the new key and value.
Add a dictionary (key,value) class.
class myDict(dict):
def __init__(self):
self = dict()
def add(self, key, value):
#self[key] = value # add new key and value overwriting any exiting same key
if self.get(key)!=None:
print('key', key, 'already used') # report if key already used
self.setdefault(key, value) # if key exit do nothing
## example
myd = myDict()
name = "fred"
myd.add('apples',6)
print('\n', myd)
myd.add('bananas',3)
print('\n', myd)
myd.add('jack', 7)
print('\n', myd)
myd.add(name, myd)
print('\n', myd)
myd.add('apples', 23)
print('\n', myd)
myd.add(name, 2)
print(myd)
I think it would also be useful to point out Python's collections module that consists of many useful dictionary subclasses and wrappers that simplify the addition and modification of data types in a dictionary, specifically defaultdict:
dict subclass that calls a factory function to supply missing values
This is particularly useful if you are working with dictionaries that always consist of the same data types or structures, for example a dictionary of lists.
>>> from collections import defaultdict
>>> example = defaultdict(int)
>>> example['key'] += 1
>>> example['key']
defaultdict(<class 'int'>, {'key': 1})
If the key does not yet exist, defaultdict assigns the value given (in our case 10) as the initial value to the dictionary (often used inside loops). This operation therefore does two things: it adds a new key to a dictionary (as per question), and assigns the value if the key doesn't yet exist. With the standard dictionary, this would have raised an error as the += operation is trying to access a value that doesn't yet exist:
>>> example = dict()
>>> example['key'] += 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'key'
Without the use of defaultdict, the amount of code to add a new element would be much greater and perhaps looks something like:
# This type of code would often be inside a loop
if 'key' not in example:
example['key'] = 0 # add key and initial value to dict; could also be a list
example['key'] += 1 # this is implementing a counter
defaultdict can also be used with complex data types such as list and set:
>>> example = defaultdict(list)
>>> example['key'].append(1)
>>> example
defaultdict(<class 'list'>, {'key': [1]})
Adding an element automatically initialises the list.
Adding keys to dictionary without using add
# Inserting/Updating single value
# subscript notation method
d['mynewkey'] = 'mynewvalue' # Updates if 'a' exists, else adds 'a'
# OR
d.update({'mynewkey': 'mynewvalue'})
# OR
d.update(dict('mynewkey'='mynewvalue'))
# OR
d.update('mynewkey'='mynewvalue')
print(d) # {'key': 'value', 'mynewkey': 'mynewvalue'}
# To add/update multiple keys simultaneously, use d.update():
x = {3:4, 5:6, 7:8}
d.update(x)
print(d) # {'key': 'value', 'mynewkey': 'mynewvalue', 3: 4, 5: 6, 7: 8}
# update operator |= now works for dictionaries:
d |= {'c':3,'d':4}
# Assigning new key value pair using dictionary unpacking.
data1 = {4:6, 9:10, 17:20}
data2 = {20:30, 32:48, 90:100}
data3 = { 38:"value", 99:"notvalid"}
d = {**data1, **data2, **data3}
# The merge operator | now works for dictionaries:
data = data1 | {'c':3,'d':4}
# Create a dictionary from two lists
data = dict(zip(list_with_keys, list_with_values))
dico["new key"] = "value"