In python 2, I used map to apply a function to several items, for instance, to remove all items matching a pattern:
map(os.remove,glob.glob("*.pyc"))
Of course I ignore the return code of os.remove, I just want all files to be deleted. It created a temp instance of a list for nothing, but it worked.
With Python 3, as map returns an iterator and not a list, the above code does nothing.
I found a workaround, since os.remove returns None, I use any to force iteration on the full list, without creating a list (better performance)
any(map(os.remove,glob.glob("*.pyc")))
But it seems a bit hazardous, specially when applying it to methods that return something. Another way to do that with a one-liner and not create an unnecessary list?
The change from map() (and many other functions from 2.7 to 3.x) returning a generator instead of a list is a memory saving technique. For most cases, there is no performance penalty to writing out the loop more formally (it may even be preferred for readability).
I would provide an example, but #vaultah nailed it in the comments: still a one-liner:
for x in glob.glob("*.pyc"): os.remove(x)
Related
I have a "best practices" question here. I am using map in a way that it may not be intended to be used - using the elements of a list to change the state of a different object. the final list output is not actually changed. Is this appropriate?
For example:
class ToBeChanged(object):
def __init__(self):
self.foo_lst = [1,2,3,4]
def mapfunc(self, arg):
if arg in ['foo', 'bar']:
self.foo_lst.append(arg)
else:
pass
test = ToBeChanged()
list_to_map = [1,2,37,'foo']
map(lambda x: test.mapfunc(x), list_to_map)
It is not appropriate. In Python 2, you'll be creating a new list of the same length as list_to_map, and immediately discarding it; waste! And the lambda even makes it more complicated.
Better to use a for loop:
for x in list_to_map:
test.mapfunc(x)
More concise and readable.
And if you're still thinking of using this in Python 3 (by forcing the lazy object to be evaluated in some way), consider those who will maintain your code; map gives the impression you want to create a new iterable from the list.
map is the worst.
Because if you tried to run the code in python3 it wouldn't even perform the calls since in python3 map is lazy.
In any case both calling map or a list-comprehension are expressions and expressions should be as side-effect free as possible, their purpose is to return a value.
So if you don't have a value to return you should just use the plain statements: i.e. explicit for
Is there a way I can implement the code block below using map or list comprehension or any other faster way, keeping it functionally the same?
def name_check(names, n_val):
lower_names = names.lower()
for item in set(n_val):
if item in lower_names:
return True
return False
Any help here is appreciated
A simple implementation would be
return any(character in names_lower for character in n_val)
A naive guess at the complexity would be O(K*2*N) where K is the number of characters in names and N is the number of characters in n_val. We need one "loop" for the call to lower*, one for the inner comprehension, and one for any. Since any is a built-in function and we're using a generator expression, I would expect it to be faster than your manual loop, but as always, profile to be sure.
To be clear, any short-circuits, so that behaviour is preserved
Notes on Your Implementation
On using a set: Your intuition to use a set to reduce the number of checks is a good one (you could add it to my form above, also), but it's a trade-off. In the case that the first element short circuits, the extra call to set is an additional N steps to produce the set expression. In the case where you wind up checking each item, it will save you some time. It depends on your expected inputs. If n_val was originally an iterable, you've lost that benefit and allocated all the memory up front. If you control the input to the function, why not just recommend it's called using lists that don't have duplicates (i.e., call set() on its input), and leave the function general?
* #Neopolitan pointed out that names_lower = names.lower() should be called out of the loop, as your original implementation called it, else it may (will?) be called repeatedly in the generator expression
I have a dictionary self.what_to_build
I am iterating on each element, and apply another method to each element using the following way:
[self.typeBuild(obj_type,dest) for obj_type,dest in self.what_to_build.items()]
It is my understanding this builds a list in memory, while there is no real impact on the program, I would like to refrain from this, I really do not need the list, just applying the method.
How would I do this same map, in the most Pythonic way, without doing a list comprehension
Just use a regular loop:
for obj_type,dest in self.what_to_build.items():
self.typeBuild(obj_type, dest)
A list comprehension indeed creates a list object with the return values of the self.typeBuild() calls, which is a waste of CPU and memory if you don't need those return values.
Don't get too hung up trying to write 'compact' code; readability is found in just the right level of verbosity.
This question already has answers here:
Are list-comprehensions and functional functions faster than "for loops"?
(8 answers)
Closed 8 years ago.
Doing some test. I came with a doubt, which option is better, using a map or a for (based on performance, space, time,etc).
map(myfunction,xrange(n))
or
for element in xrange(n):
myfunction(element)
Thanks.
As you've written it, definitely go with the for loop. map will create an unnecessary list (of all None since that is presumably the return value of myfunction) on python2.x -- and on python3.x, it won't call your function at all until you actually iterate over the results!
In other words, only use map when you actually want results that you are going to iterate over. Never use it for the side-effects of a function. For forward compatibility, never assume that map's return value is a list -- If you need a list, use a list comprehension:
[myfunction(e) for e in lst]
If you do want an iterable and you are trying to decide between the analogous generator expression and map -- it sort of boils down to a matter of preference. The amount of difference in speed is almost always negligible. I think most prefer the generator expression in terms of aesthetics, but some projects (depending on the developers) may still prefer map. Be consistent with the surrounding code...
One thing I think about is
for element in list:
# you can do something to element here
myfunction(element)
And there is no way to do the same thing in map(myfunction,list)
And also some side effect consideration.
i wanted to do something like this but this code return list of None (i think it's because list.reverse() is reversing the list in place):
map(lambda row: row.reverse(), figure)
i tried this one, but the reversed return an iterator :
map(reversed, figure)
finally i did something like this , which work for me , but i don't know if it's the right solution:
def reverse(row):
"""func that reverse a list not in place"""
row.reverse()
return row
map(reverse, figure)
if someone has a better solution that i'm not aware of please let me know
kind regards,
The mutator methods of Python's mutable containers (such as the .reverse method of lists) almost invariably return None -- a few return one useful value, e.g. the .pop method returns the popped element, but the key concept to retain is that none of those mutators returns the mutated container: rather, the container mutates in-place and the return value of the mutator method is not that container. (This is an application of the CQS principle of design -- not quite as fanatical as, say, in Eiffel, the language devised by Bertrand Meyer, who also invented CQS, but that's just because in Python "practicality beats purity, cfr import this;-).
Building a list is often costlier than just building an iterator, for the overwhelmingly common case where all you want to do is loop on the result; therefore, built-ins such as reversed (and all the wonderful building blocks in the itertools module) return iterators, not lists.
But what if you therefore have an iterator x but really truly need the equivalent list y? Piece of cake -- just do y = list(x). To make a new instance of type list, you call type list -- this is such a general Python idea that it's even more crucial to retain than the pretty-important stuff I pointed out in the first two paragraphs!-)
So, the code for your specific problem is really very easy to put together based on the crucial notions in the previous paragraphs:
[list(reversed(row)) for row in figure]
Note that I'm using a list comprehension, not map: as a rule of thumb, map should only be used as a last-ditch optimization when there is no need for a lambda to build it (if a lambda is involved then a listcomp, as well as being clearer as usual, also tends to be faster anyway!-).
Once you're a "past master of Python", if your profiling tells you that this code is a bottleneck, you can then know to try alternatives such as
[row[::-1] for row in figure]
applying a negative-step slicing (aka "Martian Smiley") to make reversed copies of the rows, knowing it's usually faster than the list(reversed(row)) approach. But -- unless your code is meant to be maintained only by yourself or somebody at least as skilled at Python -- it's a defensible position to use the simplest "code from first principles" approach except where profiling tells you to push down on the pedal. (Personally I think the "Martian Smiley" is important enough to avoid applying this good general philosophy to this specific use case, but, hey, reasonable people could differ on this very specific point!-).
You can also use a slice to get the reversal of a single list (not in place):
>>> a = [1,2,3,4]
>>> a[::-1]
[4, 3, 2, 1]
So something like:
all_reversed = [lst[::-1] for lst in figure]
...or...
all_reversed = map(lambda x: x[::-1], figure)
...will do what you want.
reversed_lists = [list(reversed(x)) for x in figure]
map(lambda row: list(reversed(row)), figure)
You can also simply do
for row in figure:
row.reverse()
to change each row in place.