The order of objects stored in dictionaries in python3.5 changes over different executions of the interpreter, but it seems to stay the same for the same interpreter instance.
$ python3 <(printf 'print({"a": 1, "b": 2})\nprint({"a": 1, "b": 2})\nprint({"a": 1, "b": 2})\nprint({"a": 1, "b": 2})')
{'b': 2, 'a': 1}
{'b': 2, 'a': 1}
{'b': 2, 'a': 1}
{'b': 2, 'a': 1}
$ python3 <(printf 'print({"a": 1, "b": 2})\nprint({"a": 1, "b": 2})\nprint({"a": 1, "b": 2})\nprint({"a": 1, "b": 2})')
{'a': 1, 'b': 2}
{'a': 1, 'b': 2}
{'a': 1, 'b': 2}
{'a': 1, 'b': 2}
I always thought the order was based off of the hash of the key. Why is the order different between different executions of python?
Dictionaries use hash function, and the order is based on the hash of the key all right.
But, as stated somewhere in this Q&A, starting from python 3.3, the seed of the hash is randomly chosen at execution time (not to mention that it depends on the python versions) .
Note that as of Python 3.3, a random hash seed is used as well, making hash collisions unpredictable to prevent certain types of denial of service (where an attacker renders a Python server unresponsive by causing mass hash collisions). This means that the order of a given dictionary is then also dependent on the random hash seed for the current Python invocation.
So each time you execute your program, you may get a different order.
Since order of dictionaries are not guaranteed (not before python 3.6 anyway), this is an implementation detail that you shouldn't consider.
dictionaries are inherently unordered. expecting any standardized behavior of the "order" is not realistic.
to keep the ordering, make an ordered list of .keys()
Related
I'm trying to implement unit tests to test for the approximate equality of dictionaries. I'm aware unittest has AssertDictEqual and AssertAlmostEqual, but I need AssertDictEqual to test for approximate rather than precise values, and AssertAlmostEqual does not work for dictionaries. The dictionaries that will be tested may be nested several times so I want to avoid using a loop.
Could someone please suggest how I could overwrite the AssertDictEqual method to test for approximate equality of each value? (Assume values are floats that need to be accurate to the closest tenth)
Example: The following should pass a test where it's being tested against the same dictionary but with all integers.
{
'A': {'a': 1.01, 'b': 2, 'c': 3},
'B': {'d': 4, 'e': 5, 'f': 6},
'C': {'g': 7, 'h': 8, 'i': 9}
}
Due to changes in dict implementation in Python 3.6 it is now ordered by default. Do sets preserve order as well now?
I could not find any information about it but as both of those data structures are very similar in the way they work under the hood I thought it might be the case.
I know there is no promise for dicts to be ordered in all cases but they are most of the time. As stated in Python docs:
The order-preserving aspect of this new implementation is considered an implementation detail and should not be relied upon
No, sets are still unordered.
You can verify this just by displaying a set that should have a "well-defined hash order"1 to make sure we don't accidentally get a set that looks ordered but actually isn't:
>>> a_set = {3,2,1}
>>> a_set
{1, 2, 3}
>>> list(a_set)
[1, 2, 3]
If it were ordered you would expect {3, 2, 1} and [3, 2, 1] as result of the examples.
While dicts are actually ordered (same example just a bit modified):
>>> a_dict = {3: 3, 2: 2, 1:1}
>>> a_dict
{3: 3, 2: 2, 1: 1}
>>> list(a_dict)
[3, 2, 1]
1 "well-defined hash order":
For integers that satisfy 0 <= integer < sys.hash_info.modulus the hash is just the number itself. That means if the set is ordered "based" on the hash (and not ordered based on the insertion "time") and the hash values don't collide (that's why I used small numbers and numbers that only differ by one) the order should be deterministic because they occupy slots inside the set that are next to each other:
Either from smallest to highest
or a from a specific value to the highest and then from the smallest to the specific value. This case happens if the next (in the sense of neighboring) free slot in the set is the first one.
As an example for the latter:
>>> a_set = {6,7,8,9}
>>> a_set
{8, 9, 6, 7}
sets are not ordered in Python 3.6, not even as a CPython implementation detail. A simple example illustrates this:
>>> import string
>>> string.digits
'0123456789'
>>> set(string.digits)
{'7', '0', '2', '8', '6', '9', '1', '5', '4', '3'}
The Python 3 docs are clear on this:
A set is an unordered collection with no duplicate elements.
Example:
>>> x = {'a' : 3, 'b' : 5, 'c' : 6}
>>> x['d'] = x
>>> x
{'a': 3, 'b': 5, 'c': 6, 'd': {...}}
>>> x['d']
{'a': 3, 'b': 5, 'c': 6, 'd': {...}}
>>> x['d']['d']
{'a': 3, 'b': 5, 'c': 6, 'd': {...}}
>>> x['d']['d']['d']
{'a': 3, 'b': 5, 'c': 6, 'd': {...}}
I guess it is infinitely looped since it is referencing itself. I just wanted to know if there is any use case for such dictionaries in real world? If yes, any examples?
No actually I don't think that's a very useful thing. The author's opinion even leans the other way that it's hard enough to have objects copied in all the right spots in python.
One obvious possibility would be to subclass dict and create (or interface) an RPG like Nethack inside the python CLI that way. You could even add some sort of UI to it. Basically, dictionaries and other mapping types are "useful enough" in python without recursion, though.
This question already has answers here:
Order of keys in dictionary
(3 answers)
Closed 7 years ago.
I have two lists which I'm mapping into a dictionary.
The two lists are-
a = ['a','b','c','d'] and b = [1,2,3,4] .
When I run the command
>>> d = dict(zip(a,b))
>>> d
I get
{'a': 1, 'c': 3, 'b': 2, 'd': 4}
whereas the expected value is {'a': 1, 'b': 2, 'c': 3, 'd': 4}
Why this change in the order of the keys?
There is no inherent "obvious" order in the keys of a dict. Admittedly, the docs only spell it out for CPython, but also note
If items(), keys(), values(), iteritems(), iterkeys(), and
itervalues() are called with no intervening modifications to the
dictionary, the lists will directly correspond.
which says by omission that otherwise they might change.
(Note that there is an order, but it involves the hashes of the keys, so it's not as easy as "a before b", and in particular, since a few years back, it is liable to change with each new call of the executable.)
There is no order in a dictionary.
{'a': 1, 'b': 2, 'c': 3, 'd': 4} == {'a': 1, 'c': 3, 'b': 2, 'd': 4}
I have a list
In [4]: a = [1, 2, 3, 3, 2, 4]
from which I would like to remove duplicates via a comprehension using a sentinel list (see below why):
In [8]: [x if x not in seen else seen.append(x) for x in a]
Out[8]: [1, 2, 3, 3, 2, 4]
It seems that seen is not taken into account (neither updated, not checked). Why is it so?
As for the reason why using a convoluted method: The list I have is of the form
[{'a': 3, 'b': 4}, {'a': 10, 'b': 4}, {'a': 5, 'b': 5}]
and I want to remove duplicates based on the value of a specific key (b in the case above, to leave [{'a': 3, 'b': 4}, {'a': 5, 'b': 5}] (I do not care which dict is removed). The idea would be to build a sentinel list with the values of b and keep only the dicts without b equal to any element in that sentinel list.
Since x is not in seen, you are never adding it to seen either; the else branch is not executed when x not in seen is true.
However, you are using a conditional expression; it always produces a value; either x or the result of seen.append() (which is None), so you are not filtering, you are mapping here.
If you wanted to filter, move the test to an if section after the for loop:
seen = set()
[x for x in a if not (x in seen or seen.add(x))]
Since you were using seen.append() I presume you were using a list; I switched you to a set() instead, as membership tests are way faster using a set.
So x is excluded only if a) x in seen is true (so we have already seen it), or seen.append(x) returned a true value (None is not true). Yes, this works, if only a little convoluted.
Demo:
>>> a = [1, 2, 3, 3, 2, 4]
>>> seen = set()
>>> [x for x in a if not (x in seen or seen.add(x))]
[1, 2, 3, 4]
>>> seen
set([1, 2, 3, 4])
Applying this to your specific problem:
>>> a = [{'a': 3, 'b': 4}, {'a': 10, 'b': 4}, {'a': 5, 'b': 5}]
>>> seen = set()
>>> [entry for entry in a if not (entry['b'] in seen or seen.add(entry['b']))]
[{'a': 3, 'b': 4}, {'a': 5, 'b': 5}]
You never execute the else part of the if, because you do not update when you match the first time. You could do this:
[seen.append(x) or x for x in lst if x not in seen]
This way the or returns the last value (and executes the update using append (which always returns None, to let the or continue looking for truth-y value).
Maybe you can use the fact that dict keys are a set for this. If you want to prioritize the last items use reversed (last item is prioritized here):
>>> lst = [{'a': 3, 'b': 4}, {'a': 10, 'b': 4}, {'a': 5, 'b': 5}]
>>> filtered = {item['b']: item for item in reversed(lst)}
>>> filtered.values()
[{'a': 3, 'b': 4}, {'a': 5, 'b': 5}]
This uses 'b' as the key to map a value to, so only a single elemnt can be mapped to a value of 'b', which effectively creates a set over 'b'.
note: this will return the values in random order. To fix it nicely, for big datasets, I'd create another mapping, of each object to it's index in the original list (O(n)), and use that mapping as a sorting function of the final result (O(n*log(n))). That's beyond the scope of this answer.
I'm always queasy making use of operator precedence as execution flow control. I feel that the below is marginally more explicit and palatable, although it does carry the additional cost of tuple creation.
b_values = set()
[(item, b_values.add(item['b']))[0] for item in original_list
if item['b'] not in b_values]
But really when you're maintaining/updating some sort of state, I think the best format is the simple for-loop:
output_list = []
b_values = set()
for item in original_list:
if item['b'] not in b_values:
output_list.append(item)
b_values.add(item['b'])