We Keep Coding

why Python list() does not always have the same order?

I am using Python 3.5 and the documentation for it at
https://docs.python.org/3.5/library/stdtypes.html#sequence-types-list-tuple-range
says:
list([iterable])
(...)
The constructor builds a list whose items are the same and in the same order as iterable’s items.
OK, for the following script:
#!/usr/bin/python3
import random
def rand6():
return random.randrange(63)
random.seed(0)
check_dict = {}
check_dict[rand6()] = 1
check_dict[rand6()] = 1
check_dict[rand6()] = 1
print(list(check_dict))
I always get
[24, 48, 54]
But, if I change the function to:
def rand6():
return bytes([random.randrange(63)])
then the order returned is not always the same:
>./foobar.py
[b'\x18', b'6', b'0']
>./foobar.py
[b'6', b'0', b'\x18']
Why?

Python dictionaries are implemented as hash tables. In most Python versions (more on this later), the order you get the keys when you iterate over a dictionary is the arbitrary order of the values in the table, which has only very little to do with the order in which they were added (when hash collisions occur, the order of insertions can matter a little bit). This order is implementation dependent. The Python language does not offer any guarantee about the order other than that it will remain the same for several iterations over a dictionary if no keys are added or removed in between.
For your dictionary with integer keys, the hash table doesn't do anything fancy. Integers hash to themselves (except -1), so with the same numbers getting put in the dict, you get a consistent order in the hash table.
For the dictionary with bytes keys however, you're seeing different behavior due to Hash Randomization. To prevent a kind of dictionary collision attack (where a webapp implemented in Python could be DoSed by sending it data with thousands of keys that hash to the same value leading to lots of collisions and very bad (O(N**2)) performance), Python picks a random seed every time it starts up and uses it to randomize the hash function for Unicode and byte strings as well as datetime types.
You can disable the hash randomization by setting the environment variable PYTHONHASHSEED to 0 (or you can pick your own seed by setting it to any positive integer up to 2**32-1).
It's worth noting that this behavior has changed in Python 3.6. Hash randomization still happens, but a dictionary's iteration order is no longer based on the hash values of the keys. While the official language policy is still that the order is arbitrary, the implementation of dict in CPython now preserves the order that its values were added. You shouldn't rely upon this behavior when using regular dicts yet, as it's possible (though it appears unlikely at this point) that the developers will decide it was a mistake and change the implementation again. If you want to guarantee that iteration occurs in a specific order, use the collections.OrderedDict class instead of a normal dict.

Am I using `all` correctly?

A user asked (Keyerror while using pandas in PYTHON 2.7) why he was having a KeyError while looking in a dictionary and how he could avoid this exception.
As an answer, I suggested him to check for the keys in the dictionary before. So, if he needed all the keys ['key_a', 'key_b', 'key_c'] in the dictionary, he could test it with:
if not all([x in dictionary for x in ['key_a', 'key_b', 'key_c']]):
continue
This way he could ignore dictionaries that didn't have the expected keys (the list of dictionaries is created out of JSON formatted lines loaded from a file). *Refer to the original question for more details, if relevant to this question.
A user more experienced in Python and SO, which I would consider an authority on the matter for its career and gold badges told me I was using all incorrectly. I was wondering if this is really the case (for what I can tell, that works as expected) and why, or if there is a better way to check if a couple of keys are all in a dictionary.

Yes that will work fine, but you don't even need the list comprehension
if not all(x in dictionary for x in ['key_a', 'key_b', 'key_c']):
continue
If you have the surrounding [], it will evaluate all the elements before calling all. If you remove them, the inner expression is a generator, and will short-circuit upon the first False.

Sum across all list positions in a dictionary

This is a question and answer I wanted to share, since I found it very useful.
Suppose I have a dictionary accessible with different keys. And at each position of the dictionary I have a list of a fixed length:
a={}
a["hello"]=[2,3,4]
a["bye"]=[0,10,100]
a["goodbye"]=[2,5,50]
I was interested to compute the sum across all entries in a using only position 1 of their respective lists.
In the example, I wanted to sum:
finalsum=sum([3,10,5]) #-----> 18

Just skip the keys entirely, since they don't really matter.
sum(i[1] for i in a.itervalues())
Also as a side note, you don't need to do a.keys() when iterating over a dict, you can just say for key in a and it will use the keys.
You can use a.values() to get a list of all the values in a dict. As far as I can tell, the keys are irrelevant. a.itervalues() works by iterating rather than constructing a new list. By using this, and a generator expression as the argument to sum, there are no extraneous lists created.

I used list-comprehensions for my one line solution(here separated in two lines):
elements=[a[pos][1] for pos in a.keys()] #----> [3,5,10]
finalsum=sum(elements)
I'm happy with this solution :) , but, any other suggestions?

python : how to disable auto sort when creating dictionary

i need help for this case :
m={}
m[1]=1
m[333]=333
m[2]=2
# Result:
{1: 1, 2: 2, 333: 333}
so even when i didn't enter '333' the last, i got this '333' listed in the end of the dictionary when print it out. why is this 'dictionary' doing auto sort? and how disable it? i can creata a function to re-sort to fix the order. but that's not what i want, i just simply want to print and get output order just like the order when i input the data. Is there any good explanation and is there any solution ?

It is not sorting. dict is not ordered at all, so you cannot influence the key order in any way. There is collections.OrderedDict in 2.7 and 3.1+, there is also standalone module for 2.4-2.6.

Items stored in a dictionary do not have any inherent order. The order they are printed out is entirely down to the hash values for each of the keys and the other items in the dictionary.

Long time after this questions was posted, but just for those who land on this page, since 3.6? dictionaries preserve the order that items are added in.

Python equiv. of PHP foreach []?

I am fetching rows from the database and wish to populate a multi-dimensional dictionary.
The php version would be roughly this:
foreach($query as $rows):
$values[$rows->id][] = $rows->name;
endforeach;
return $values;
I can't seem to find out the following issues:
What is the python way to add keys to a dictionary using an automatically numbering e.g. $values[]
How do I populate a Python dictionary using variables; using, for example, values[id] = name, will not add keys, but override existing.
I totally have no idea how to achieve this, as I am a Python beginner (programming in general, actually).

values = collections.defaultdict(list)
for rows in query:
values[rows.id].append(rows.name)
return values

Just a general note:
Python's dictionaries are mappings without order, while adding numerical keys would allow "sequential" access, in case of iteration there's no guarantee that order will coincide with the natural order of keys.
It's better not to translate from PHP to Python (or any other language), but rather right code idiomatic to that particular language. Have a look at the many open-source code that does the same/similar things, you might even find a useful module (library).

all_rows=[]
for row in query:
all_rows.append(row['col'])
print(all_rows)

You can do:
from collections import defaultdict
values = defaultdict(list)
for row in query:
values[row.id].append(row.name)
return values
Edit: forgot to return the values.