I'm looking at an output from 2to3 that includes this change:
- for file_prefix in output.keys():
+ for file_prefix in list(output.keys()):
where output is a dictionary.
What is the significance of this change? Why does 2to3 do this?
How does this change make the code Python 3 compatible?
In Python 3, the .keys() method returns a view object rather than a list, for efficiency's sake.
In the iteration case, this doesn't actually matter, but where it would matter is if you were doing something like foo.keys()[0] - you can't index a view. Thus, 2to3 always adds an explicit list conversion to make sure that any potential indexing doesn't break.
You can manually remove the list() call anywhere that a view would work fine; 2to3 just isn't smart enough to tell which case is which.
(Note that the 2.x version could call iterkeys() instead, since it's not indexing.)
In Python 2.x, dict.keys() returns a list.
In Python 3.x, dict.keys() returns a view and must be passed to list() in order to make it a list.
Since the Python 2.x code doesn't need a list it should call dict.iterkeys() instead.
In Python 2, .keys() returns a list of the keys, but in Python 3 it returns a non-list iterator. Since 2to3 can't know whether you really needed the keys to be a list, it has to err on the side of caution and wrap the call in list so you really get a list.
In Python2, keys returned a list while in 3 the return of keys is a dict_keys object. So if you were dependent on the list-result behavior, the explicit conversion is necessary.
Related
I want to change a Python 2 code to Python 3. In particular, I would like to change a.itervalues to a.values.
I would to know if it is possible to define a new function so that when the program comes across .itervalues, it will run my new function instead of its original one?
Thank you very much.
as #aws_apprentice said : go to https://docs.python.org/3/library/2to3.html
specifically read this section :
dict Fixes dictionary iteration methods. dict.iteritems() is converted
to dict.items(), dict.iterkeys() to dict.keys(), and dict.itervalues()
to dict.values(). Similarly, dict.viewitems(), dict.viewkeys() and
dict.viewvalues() are converted respectively to dict.items(),
dict.keys() and dict.values(). It also wraps existing usages of
dict.items(), dict.keys(), and dict.values() in a call to list.
so all you need to do is run 2to3 script on your files and you customize your code to suit Python 3 convention.
In python 2, I used map to apply a function to several items, for instance, to remove all items matching a pattern:
map(os.remove,glob.glob("*.pyc"))
Of course I ignore the return code of os.remove, I just want all files to be deleted. It created a temp instance of a list for nothing, but it worked.
With Python 3, as map returns an iterator and not a list, the above code does nothing.
I found a workaround, since os.remove returns None, I use any to force iteration on the full list, without creating a list (better performance)
any(map(os.remove,glob.glob("*.pyc")))
But it seems a bit hazardous, specially when applying it to methods that return something. Another way to do that with a one-liner and not create an unnecessary list?
The change from map() (and many other functions from 2.7 to 3.x) returning a generator instead of a list is a memory saving technique. For most cases, there is no performance penalty to writing out the loop more formally (it may even be preferred for readability).
I would provide an example, but #vaultah nailed it in the comments: still a one-liner:
for x in glob.glob("*.pyc"): os.remove(x)
I am getting an error saying "dict_key object does not support indexing" at:
return len(G[G.keys()[0]])
I realised it used to work in python 2.7.x but not in python 3.How should i change this statement to make it work in python 3.
In Python 2.x, G.keys() returns a list, but Python 3.x returns a dict_keys object instead. The solution is to wrap G.keys() with call to list(), to convert it into the correct type:
return len(G[list(G.keys())[0]])
In Python 3, the objects returned by keys, values, and items are dictionary view objects, which don't support indexing.
Try, instead:
len(next(iter(G.values())))
This gets the dictionary view object for the dictionary's values, gets its iterator, grabs the first item from the iterator (the first value in the dictionary), and returns its length.
Unlike other methods that create a new list of the keys or values, it should take approximately the same amount of time no matter the size of the dictionary.
It works in both Python 2 and Python 3 (though to be efficient you'd need to use itervalues or viewvalues on Python 2).
I was wondering what is the correct way to check a key:value pair of a dict. Lets say I have this dict
dict_ = {
'key1':'val1',
'key2':'val2'
}
I can check a condition like this
if dict_['key1'] == 'val1'
but I feel like there is a more elegant way that takes advantage of the dict data structure.
What you're doing already does take advantage of the data structure, which is why it's "the one obvious way" to do what you want to do. (You can find examples like this all over the tutorial, the reference docs, and the stdlib implementation.)
However, I can see what you're thinking: the dict is in some sense a container of key-value pairs (even if it's only a collections.Container of keys…), so… shouldn't there be some way to just check whether a key-value pair exists?
Up to Python 2.6, there really isn't.* But in 3.0, the items() method returns a special set-like view of the key-value pairs. And 2.7 backported that functionality, under the name viewitems. So:
('key1', 'val1') in d.viewitems()
But I don't think that's really clearer or cleaner; "items" feels like a lower-level way to think of dictionaries than "mappings", which is what both your original code and smci's answer rely on.
It's also less concise, it doesn't work in 2.6 or earlier, and many dict-like mapping objects don't support it,** and it's and slightly slower on 2.7 to boot, but these are probably less important, and not what you asked about.
* Well, there is, but only by iterating over all of the items with iteritems, or using items to effectively do the same exhaustive search behind your back, neither of which is what you want.
** In fact, in 2.7, it's not actually possible to support it with a pure-Python class…
If you want to avoid throwing KeyError if dict doesn't even contain 'key1':
if dict_.get('key1')=='val1':
(However, throwing an exception for missing key is perfectly fine Python idiom.)
Otherwise, #Cyber is correct that it's already fine! (What exactly is the problem?)
There is a has_key function
dict_.has_key('key1')
This returns a boolean true or false.
Alternatively, you can have you get function return a default value when the key is not present.
dict_.get('key3','Default Value')
Modified typo*
I recently wrote some code that looked something like this:
# dct is a dictionary
if "key" in dct.keys():
However, I later found that I could achieve the same results with:
if "key" in dct:
This discovery got me thinking and I began to run some tests to see if there could be a scenario when I must use the keys method of a dictionary. My conclusion however is no, there is not.
If I want the keys in a list, I can do:
keys_list = list(dct)
If I want to iterate over the keys, I can do:
for key in dct:
...
Lastly, if I want to test if a key is in dct, I can use in as I did above.
Summed up, my question is: am I missing something? Could there ever be a scenario where I must use the keys method?...or is it simply a leftover method from an earlier installation of Python that should be ignored?
On Python 3, use dct.keys() to get a dictionary view object, which lets you do set operations on just the keys:
>>> for sharedkey in dct1.keys() & dct2.keys(): # intersection of two dictionaries
... print(dct1[sharedkey], dct2[sharedkey])
In Python 2.7, you'd use dct.viewkeys() for that.
In Python 2, dct.keys() returns a list, a copy of the keys in the dictionary. This can be passed around an a separate object that can be manipulated in its own right, including removing elements without affecting the dictionary itself; however, you can create the same list with list(dct), which works in both Python 2 and 3.
You indeed don't want any of these for iteration or membership testing; always use for key in dct and key in dct for those, respectively.
Source: PEP 234, PEP 3106
Python 2's relatively useless dict.keys method exists for historical reasons. Originally, dicts weren't iterable. In fact, there was no such thing as an iterator; iterating over sequences worked by calling __getitem__, the element access method, with increasing integer indices until an IndexError was raised. To iterate over the keys of a dict, you had to call the keys method to get an explicit list of keys and iterate over that.
When iterators went in, dicts became iterable, because it was more convenient, faster, and all around better to say
for key in d:
than
for key in d.keys()
This had the side-effect of making d.keys() utterly superfluous; list(d) and iter(d) now did everything d.keys() did in a cleaner, more general way. They couldn't get rid of keys, though, since so much code already called it.
(At this time, dicts also got a __contains__ method, so you could say key in d instead of d.has_key(key). This was shorter and nicely symmetrical with for key in d; the symmetry is also why iterating over a dict gives the keys instead of (key, value) pairs.)
In Python 3, taking inspiration from the Java Collections Framework, the keys, values, and items methods of dicts were changed. Instead of returning lists, they would return views of the original dict. The key and item views would support set-like operations, and all views would be wrappers around the underlying dict, reflecting any changes to the dict. This made keys useful again.
Assuming you're not using Python 3, list(dct) is equivalent to dct.keys(). Which one you use is a matter of personal preference. I personally think dct.keys() is slightly clearer, but to each their own.
In any case, there isn't a scenario where you "need" to use dct.keys() per se.
In Python 3, dct.keys() returns a "dictionary view object", so if you need to get a hold of an unmaterialized view to the keys (which could be useful for huge dictionaries) outside of a for loop context, you'd need to use dct.keys().
key in dict
is much faster than checking
key in dict.keys()