I'd like to be able to setup a mock that allows me to return something when I apply the builtin dict method.
I've tried using __iter__ to no avail. I can't seem to get anything but an empty dictionary:
import mock
mocked_object = mock.MagicMock()
mocked_object.__iter__.return_value = [1, 2, 3]
dict(mocked_object)
# {}
From dict documentation
If a positional argument is given and it is a mapping object, a dictionary is created with the same key-value pairs as the mapping object. Otherwise, the positional argument must be an iterable object. Each item in the iterable must itself be an iterable with exactly two objects. The first object of each item becomes a key in the new dictionary, and the second object the corresponding value.
MagicMock objects expose a keys method just because are mock objects and so dict() will consider they mapping objects. Unfortunately that way is little bit complicated to use if we want that on dict call mock become a dictionary with predefined key-values. The follow examples show how to implement a dict conversion to a predefined dictionary by use mapping object protocol:
>>> m = MagicMock()
>>> d = {"a":"A", "b":"B", "c":"C"}
>>> m.keys.return_value.__iter__.return_value = ["a", "b", "c"]
>>> m.__getitem__.side_effect = ["A","B","C"]
>>> dict(m)
{'a': 'A', 'c': 'C', 'b': 'B'}
>>> #Little bit generic
>>> m.keys.return_value.__iter__.return_value = d.keys()
>>> m.__getitem__.side_effect = lambda k:d[k]
>>> dict(m)
{'a': 'A', 'c': 'C', 'b': 'B'}
Both are little bit hard to read and in our test we would like something of simpler to read.
To lead dict to use iterator instead mapping we can just remove keys method from our mock and set __iter__.return_value:
>>> del m.keys
>>> m.__iter__.return_value = [("a","A"),("b","B"),("c","C")]
>>> dict(m)
{'a': 'A', 'c': 'C', 'b': 'B'}
>>> #Little bit generic
>>> m.__iter__.return_value = d.items()
>>> dict(m)
{'a': 'A', 'c': 'C', 'b': 'B'}
IMHO that is a simple and neat way to set up your mock and get a predefined dictionary from dict call.
Actually I think you need to do something like this:
mocked_object.keys.return_value.__iter__.return_value = [1, 2, 3]
With this, the dict method will give you an object with those keys, and the result of getattr(mocked_object, '1') (so, another mocked method) as values. I think you might be able to do what you want by mocking the keys() method as well if you want more control on the result.
Related
I have a dictionary where key is string and value is list.
Now while adding a value associated with given key, I always have to check if there is any list yet, otherwise I have to initialize as empty list somewhat like following snippet:
if not k in myDict:
myDict[k] = []
myDict[k].append(v)
I am wondering if there is any way to combine these steps into single one in python 3.7.
There are at least three ways:
Use dict.setdefault
>>> data = {}
>>> data.setdefault('foo', []).append(42)
>>> data
{'foo': [42]}
Use defaultdict, which unlike .setdefault, takes a callable:
>>> from collections import defaultdict
>>> data = defaultdict(list)
>>> data
defaultdict(<class 'list'>, {})
>>> data['foo'].append(42)
>>> data
defaultdict(<class 'list'>, {'foo': [42]})
Finally, subclass dict and implement __missing__:
>>> class MyDict(dict):
... def __missing__(self, key):
... self[key] = value = []
... return value
...
>>> data = MyDict()
>>> data['foo'].append(42)
>>> data
{'foo': [42]}
Note, you can think of the last one as the most flexible, you have access to the actual key that's missing when you deal with it. defaultdict is a class factory, and it generates a subclass of dict as well. But, the callable is not passed any arguments, nevertheless, it is sufficient for most needs.
Further, note that the defaultdict and __missing__ approaches will keep the default behavior, this may be undesirable after you create your data structure, you probably want a KeyError usually, or at least, you don't want mydict[key] to add a key anymore.
In both cases, you can just create a regular dict from the dict subclasses, e.g. dict(data). This should generally be very fast, even for large dict objects, especially if it is a one-time cost. For defaultdict, you can also set the default_factory to None and the old behavior returns:
>>> data = defaultdict(list)
>>> data
defaultdict(<class 'list'>, {})
>>> data['foo']
[]
>>> data['bar']
[]
>>> data
defaultdict(<class 'list'>, {'foo': [], 'bar': []})
>>> data.default_factory = None
>>> data['baz']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'baz'
>>>
You can use dict.setdefault:
>>> d = {}
>>> d.setdefault('k', []).append(1)
>>> d
{'k': [1]}
>>> d.setdefault('k', []).append(2)
>>> d
{'k': [1, 2]}
Help on method_descriptor in dict:
dict.setdefault = setdefault(...)
D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D
My attempt to programmatically create a dictionary of lists is failing to allow me to individually address dictionary keys. Whenever I create the dictionary of lists and try to append to one key, all of them are updated. Here's a very simple test case:
data = {}
data = data.fromkeys(range(2),[])
data[1].append('hello')
print data
Actual result: {0: ['hello'], 1: ['hello']}
Expected result: {0: [], 1: ['hello']}
Here's what works
data = {0:[],1:[]}
data[1].append('hello')
print data
Actual and Expected Result: {0: [], 1: ['hello']}
Why is the fromkeys method not working as expected?
When [] is passed as the second argument to dict.fromkeys(), all values in the resulting dict will be the same list object.
In Python 2.7 or above, use a dict comprehension instead:
data = {k: [] for k in range(2)}
In earlier versions of Python, there is no dict comprehension, but a list comprehension can be passed to the dict constructor instead:
data = dict([(k, []) for k in range(2)])
In 2.4-2.6, it is also possible to pass a generator expression to dict, and the surrounding parentheses can be dropped:
data = dict((k, []) for k in range(2))
Try using a defaultdict instead:
from collections import defaultdict
data = defaultdict(list)
data[1].append('hello')
This way, the keys don't need to be initialized with empty lists ahead of time. The defaultdict() object instead calls the factory function given to it, every time a key is accessed that doesn't exist yet. So, in this example, attempting to access data[1] triggers data[1] = list() internally, giving that key a new empty list as its value.
The original code with .fromkeys shares one (mutable) list. Similarly,
alist = [1]
data = dict.fromkeys(range(2), alist)
alist.append(2)
print(data)
would output {0: [1, 2], 1: [1, 2]}. This is called out in the dict.fromkeys() documentation:
All of the values refer to just a single instance, so it generally doesn’t make sense for value to be a mutable object such as an empty list.
Another option is to use the dict.setdefault() method, which retrieves the value for a key after first checking it exists and setting a default if it doesn't. .append can then be called on the result:
data = {}
data.setdefault(1, []).append('hello')
Finally, to create a dictionary from a list of known keys and a given "template" list (where each value should start with the same elements, but be a distinct list), use a dictionary comprehension and copy the initial list:
alist = [1]
data = {key: alist[:] for key in range(2)}
Here, alist[:] creates a shallow copy of alist, and this is done separately for each value. See How do I clone a list so that it doesn't change unexpectedly after assignment? for more techniques for copying the list.
You could use a dict comprehension:
>>> keys = ['a','b','c']
>>> value = [0, 0]
>>> {key: list(value) for key in keys}
{'a': [0, 0], 'b': [0, 0], 'c': [0, 0]}
This answer is here to explain this behavior to anyone flummoxed by the results they get of trying to instantiate a dict with fromkeys() with a mutable default value in that dict.
Consider:
#Python 3.4.3 (default, Nov 17 2016, 01:08:31)
# start by validating that different variables pointing to an
# empty mutable are indeed different references.
>>> l1 = []
>>> l2 = []
>>> id(l1)
140150323815176
>>> id(l2)
140150324024968
so any change to l1 will not affect l2 and vice versa.
this would be true for any mutable so far, including a dict.
# create a new dict from an iterable of keys
>>> dict1 = dict.fromkeys(['a', 'b', 'c'], [])
>>> dict1
{'c': [], 'b': [], 'a': []}
this can be a handy function.
here we are assigning to each key a default value which also happens to be an empty list.
# the dict has its own id.
>>> id(dict1)
140150327601160
# but look at the ids of the values.
>>> id(dict1['a'])
140150323816328
>>> id(dict1['b'])
140150323816328
>>> id(dict1['c'])
140150323816328
Indeed they are all using the same ref!
A change to one is a change to all, since they are in fact the same object!
>>> dict1['a'].append('apples')
>>> dict1
{'c': ['apples'], 'b': ['apples'], 'a': ['apples']}
>>> id(dict1['a'])
>>> 140150323816328
>>> id(dict1['b'])
140150323816328
>>> id(dict1['c'])
140150323816328
for many, this was not what was intended!
Now let's try it with making an explicit copy of the list being used as a the default value.
>>> empty_list = []
>>> id(empty_list)
140150324169864
and now create a dict with a copy of empty_list.
>>> dict2 = dict.fromkeys(['a', 'b', 'c'], empty_list[:])
>>> id(dict2)
140150323831432
>>> id(dict2['a'])
140150327184328
>>> id(dict2['b'])
140150327184328
>>> id(dict2['c'])
140150327184328
>>> dict2['a'].append('apples')
>>> dict2
{'c': ['apples'], 'b': ['apples'], 'a': ['apples']}
Still no joy!
I hear someone shout, it's because I used an empty list!
>>> not_empty_list = [0]
>>> dict3 = dict.fromkeys(['a', 'b', 'c'], not_empty_list[:])
>>> dict3
{'c': [0], 'b': [0], 'a': [0]}
>>> dict3['a'].append('apples')
>>> dict3
{'c': [0, 'apples'], 'b': [0, 'apples'], 'a': [0, 'apples']}
The default behavior of fromkeys() is to assign None to the value.
>>> dict4 = dict.fromkeys(['a', 'b', 'c'])
>>> dict4
{'c': None, 'b': None, 'a': None}
>>> id(dict4['a'])
9901984
>>> id(dict4['b'])
9901984
>>> id(dict4['c'])
9901984
Indeed, all of the values are the same (and the only!) None.
Now, let's iterate, in one of a myriad number of ways, through the dict and change the value.
>>> for k, _ in dict4.items():
... dict4[k] = []
>>> dict4
{'c': [], 'b': [], 'a': []}
Hmm. Looks the same as before!
>>> id(dict4['a'])
140150318876488
>>> id(dict4['b'])
140150324122824
>>> id(dict4['c'])
140150294277576
>>> dict4['a'].append('apples')
>>> dict4
>>> {'c': [], 'b': [], 'a': ['apples']}
But they are indeed different []s, which was in this case the intended result.
You can use this:
l = ['a', 'b', 'c']
d = dict((k, [0, 0]) for k in l)
You are populating your dictionaries with references to a single list so when you update it, the update is reflected across all the references. Try a dictionary comprehension instead. See
Create a dictionary with list comprehension in Python
d = {k : v for k in blah blah blah}
You could use this:
data[:1] = ['hello']
Simple question.
What hashing algorithm is used in python's default dict?
>>> g = { 'a' : 1, 'b' : 2, 'c' : 3 }
>>> g
{'a': 1, 'c': 3, 'b': 2}
>>> g.keys()
['a', 'c', 'b']
>>>
I expected ['a','b','c'] on g.keys()
Linear probe (guess not)? double hash?
There is no guarantee that Python will use any particular method - different implementations could use any one they wish. dicts are unordered, so it doesn't matter how it's implemented (provided it fulfills certain obligations).
As to how CPython does it...
My attempt to programmatically create a dictionary of lists is failing to allow me to individually address dictionary keys. Whenever I create the dictionary of lists and try to append to one key, all of them are updated. Here's a very simple test case:
data = {}
data = data.fromkeys(range(2),[])
data[1].append('hello')
print data
Actual result: {0: ['hello'], 1: ['hello']}
Expected result: {0: [], 1: ['hello']}
Here's what works
data = {0:[],1:[]}
data[1].append('hello')
print data
Actual and Expected Result: {0: [], 1: ['hello']}
Why is the fromkeys method not working as expected?
When [] is passed as the second argument to dict.fromkeys(), all values in the resulting dict will be the same list object.
In Python 2.7 or above, use a dict comprehension instead:
data = {k: [] for k in range(2)}
In earlier versions of Python, there is no dict comprehension, but a list comprehension can be passed to the dict constructor instead:
data = dict([(k, []) for k in range(2)])
In 2.4-2.6, it is also possible to pass a generator expression to dict, and the surrounding parentheses can be dropped:
data = dict((k, []) for k in range(2))
Try using a defaultdict instead:
from collections import defaultdict
data = defaultdict(list)
data[1].append('hello')
This way, the keys don't need to be initialized with empty lists ahead of time. The defaultdict() object instead calls the factory function given to it, every time a key is accessed that doesn't exist yet. So, in this example, attempting to access data[1] triggers data[1] = list() internally, giving that key a new empty list as its value.
The original code with .fromkeys shares one (mutable) list. Similarly,
alist = [1]
data = dict.fromkeys(range(2), alist)
alist.append(2)
print(data)
would output {0: [1, 2], 1: [1, 2]}. This is called out in the dict.fromkeys() documentation:
All of the values refer to just a single instance, so it generally doesn’t make sense for value to be a mutable object such as an empty list.
Another option is to use the dict.setdefault() method, which retrieves the value for a key after first checking it exists and setting a default if it doesn't. .append can then be called on the result:
data = {}
data.setdefault(1, []).append('hello')
Finally, to create a dictionary from a list of known keys and a given "template" list (where each value should start with the same elements, but be a distinct list), use a dictionary comprehension and copy the initial list:
alist = [1]
data = {key: alist[:] for key in range(2)}
Here, alist[:] creates a shallow copy of alist, and this is done separately for each value. See How do I clone a list so that it doesn't change unexpectedly after assignment? for more techniques for copying the list.
You could use a dict comprehension:
>>> keys = ['a','b','c']
>>> value = [0, 0]
>>> {key: list(value) for key in keys}
{'a': [0, 0], 'b': [0, 0], 'c': [0, 0]}
This answer is here to explain this behavior to anyone flummoxed by the results they get of trying to instantiate a dict with fromkeys() with a mutable default value in that dict.
Consider:
#Python 3.4.3 (default, Nov 17 2016, 01:08:31)
# start by validating that different variables pointing to an
# empty mutable are indeed different references.
>>> l1 = []
>>> l2 = []
>>> id(l1)
140150323815176
>>> id(l2)
140150324024968
so any change to l1 will not affect l2 and vice versa.
this would be true for any mutable so far, including a dict.
# create a new dict from an iterable of keys
>>> dict1 = dict.fromkeys(['a', 'b', 'c'], [])
>>> dict1
{'c': [], 'b': [], 'a': []}
this can be a handy function.
here we are assigning to each key a default value which also happens to be an empty list.
# the dict has its own id.
>>> id(dict1)
140150327601160
# but look at the ids of the values.
>>> id(dict1['a'])
140150323816328
>>> id(dict1['b'])
140150323816328
>>> id(dict1['c'])
140150323816328
Indeed they are all using the same ref!
A change to one is a change to all, since they are in fact the same object!
>>> dict1['a'].append('apples')
>>> dict1
{'c': ['apples'], 'b': ['apples'], 'a': ['apples']}
>>> id(dict1['a'])
>>> 140150323816328
>>> id(dict1['b'])
140150323816328
>>> id(dict1['c'])
140150323816328
for many, this was not what was intended!
Now let's try it with making an explicit copy of the list being used as a the default value.
>>> empty_list = []
>>> id(empty_list)
140150324169864
and now create a dict with a copy of empty_list.
>>> dict2 = dict.fromkeys(['a', 'b', 'c'], empty_list[:])
>>> id(dict2)
140150323831432
>>> id(dict2['a'])
140150327184328
>>> id(dict2['b'])
140150327184328
>>> id(dict2['c'])
140150327184328
>>> dict2['a'].append('apples')
>>> dict2
{'c': ['apples'], 'b': ['apples'], 'a': ['apples']}
Still no joy!
I hear someone shout, it's because I used an empty list!
>>> not_empty_list = [0]
>>> dict3 = dict.fromkeys(['a', 'b', 'c'], not_empty_list[:])
>>> dict3
{'c': [0], 'b': [0], 'a': [0]}
>>> dict3['a'].append('apples')
>>> dict3
{'c': [0, 'apples'], 'b': [0, 'apples'], 'a': [0, 'apples']}
The default behavior of fromkeys() is to assign None to the value.
>>> dict4 = dict.fromkeys(['a', 'b', 'c'])
>>> dict4
{'c': None, 'b': None, 'a': None}
>>> id(dict4['a'])
9901984
>>> id(dict4['b'])
9901984
>>> id(dict4['c'])
9901984
Indeed, all of the values are the same (and the only!) None.
Now, let's iterate, in one of a myriad number of ways, through the dict and change the value.
>>> for k, _ in dict4.items():
... dict4[k] = []
>>> dict4
{'c': [], 'b': [], 'a': []}
Hmm. Looks the same as before!
>>> id(dict4['a'])
140150318876488
>>> id(dict4['b'])
140150324122824
>>> id(dict4['c'])
140150294277576
>>> dict4['a'].append('apples')
>>> dict4
>>> {'c': [], 'b': [], 'a': ['apples']}
But they are indeed different []s, which was in this case the intended result.
You can use this:
l = ['a', 'b', 'c']
d = dict((k, [0, 0]) for k in l)
You are populating your dictionaries with references to a single list so when you update it, the update is reflected across all the references. Try a dictionary comprehension instead. See
Create a dictionary with list comprehension in Python
d = {k : v for k in blah blah blah}
You could use this:
data[:1] = ['hello']
I mean why cant we put key of dict as dict?
that means we can't have dictionary having key as another dictionary...
Short answer: because they are mutable containers.
If a dict was hashed, its hash would change as you changed its contents.
This is easy to deal with. Wrap a dict in a frozenset before you hash it. Then, when you need to use it, convert it back to a dict.
>>> unhashable = {'b': 'a', 'a': 'b'}
>>> hashable = frozenset(unhashable.items())
>>> unhashable = dict(hashable)
>>> unhashable
{'a': 'b', 'b': 'a'}
Note that dictionary key order is undefined anyway, so the change in key order doesn't matter.
As others have said, the hash value of a dict changes as the contents change.
However if you really need to use dicts as keys, you can subclass dict to make a hashable version.
>>> class hashabledict(dict):
... def __hash__(self):
... return id(self)
...
>>> hd = hashabledict()
>>> d = dict()
>>> d[hd] = "foo"
>>> d
{{}: 'foo'}
>>> hd["hello"] = "world"
>>> d
{{'hello': 'world'}: 'foo'}
This replaces the hash value used for the dict with the object's address in memory.
None of the mutable container types in Python are hashable, because they are mutable and thus their hash value can change over their lifetime.
For maybe the wrong reasons I've ran into this problem a bunch of times; where I want to reference a full dict as a key to something. I don't need it to be mutable, but I do want to preserve and easily access the dict's members.
The easiest way I've found to make the dict immutable and quickly usable as a key value is to make it a JSON (or serialize in your favorite alternative).
For example:
>>> import json
>>> d = {'hey':1, 'there':2}
>>> d_key = json.dumps(d)
>>> d_key
'{"there": 2, "hey": 1}'
>>> d2 = {d_key: 'crazytown'}
>>> d2
{'{"there": 2, "hey": 1}': 'crazytown'}
It's easy to manipulate, since it's just a string. And, it can be de-serialized into an object if you want to reference its members.