I'm trying to improve the computing time of my code so I want to replace for loops with map functions.
For each key in the dictionary I check if it is bigger than a specific value and inserting it to a new dictionary under the same key:
My original code is:
dict1={'a':-1,'b':0,'c':1,'d':2,'e':3}
dict_filt = {}
for key in dict1.keys():
if dict1[key]>1:
dict_filt[key] = dict1[key]*10
print (dict_filt)
output is: {'d': 20, 'e': 30}
and this works
but when I try with map:
dict1={'a':-1,'b':0,'c':1,'d':2,'e':3}
dict_filt = {}
def for_filter (key):
if dict1[key]>1:
dict_filt[key] = dict1[key]*10
map (for_filter ,dict1.keys())
print (dict_filt)
I get an empty dictionary
I tried to make it work with lambda:
map (lambda x: for_filter(x) ,dict1.keys())
or define the dictionarys as global but it still doesnt work.
I'll be glad to get some help
I don't need the original dictionary so if it's simpler to work on one dictionary it's still ok
Use a dictionary-comprehension instead of map:
{k: v * 10 for k, v in dict1.items() if v > 1}
Code:
dict1 = {'a':-1,'b':0,'c':1,'d':2,'e':3}
print({k: v * 10 for k, v in dict1.items() if v > 1})
# {'d': 20, 'e': 30}
map is lazy: if you do not consume the values, the function for_filter is not applied. Since you are using a side effect to populate dict_filt, nothing will happen unless you force the evaluation of the map:
Replace:
map(for_filter, dict1.keys())
By:
list(map(for_filter, dict1)) # you don't need keys here
And you will get the expected result.
But note that this is a misuse of map. You should use a dict comprehension (see #Austin's answer).
EDIT: More on map and lazyness.
TLDR;
Look at the doc:
map(function, iterable, ...)
Return an iterator that applies function to every item of iterable, yielding the results.
Explanation
Consider the following function:
>>> def f(x):
... print("x =", x)
... return x
...
This function returns its parameter and performs a side effect (print the value). Let's try to apply this function to a simple range with the map function:
>>> m = map(f, range(5))
Nothing is printed! Let's look at the value of m:
>>> m
<map object at 0x7f91d35cccc0>
We were expecting [0, 1, 2, 3, 4] but we got a strange <map object at 0x7f91d35cccc0>. That's lazyness: map does not really apply the function but creates an iterator. This iterator returns, on each next call, a value:
>>> next(m)
x = 0
0
That value is the result of the application of the function f to the next element of the mapped iterable (range). Here, 0 is the returned value and x = 0 the result of the print side effect. What is important here is that this value does not exist before you pull it out of the iterator. Hence the side effect is not performed before you pull the vlaue out of the iterator.
If we continue to call next, we'll exhaust the iterator:
...
>>> next(m)
x = 4
4
>>> next(m)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
Another way to get all the values of the iterator is to create a list. That's not a cast, but rather the constrution of a new object and the consumption of the old one:
>>> m = map(f, range(5))
>>> list(m)
x = 0
x = 1
x = 2
x = 3
x = 4
[0, 1, 2, 3, 4]
We see that the side effect print is performed for every element of the range, and then the list [0, 1, 2, 3, 4] is returned.
In your case, the function doesn't print anything, but makes an assignement to an external variable dict_filt. The function is not applied unless you consume the map iterator.
I repeat: do not use map (or any list/dict comprehension) to perform a side effect (map comes from the functional world where side effect do not exist).
Related
I was playing with the map object and noticed that it didn't print if I do list() beforehand. When I viewed only the map beforehand, the printing worked. Why?
map returns an iterator and you can consume an iterator only once.
Example:
>>> a=map(int,[1,2,3])
>>> a
<map object at 0x1022ceeb8>
>>> list(a)
[1, 2, 3]
>>> next(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>> list(a)
[]
Another example where I consume the first element and create a list with the rest
>>> a=map(int,[1,2,3])
>>> next(a)
1
>>> list(a)
[2, 3]
As per the answer from #newbie, this is happening because you are consuming the map iterator before you use it. (Here is another great answer on this topic from #LukaszRogalski)
Example 1:
w = [[1,5,7],[2,2,2,9],[1,2],[0]]
m = map(sum,w) # map iterator is generated
list(m) # map iterator is consumed here (output: [13,15,3,0])
for v in m:
print(v) # there is nothing left in m, so there's nothing to print
Example 2:
w = [[1,5,7],[2,2,2,9],[1,2],[0]]
m = map(sum,w) #map iterator is generated
for v in m:
print(v) #map iterator is consumed here
# if you try and print again, you won't get a result
for v in m:
print(v) # there is nothing left in m, so there's nothing to print
So you have two options here, if you only want to iterate the list once, Example 2 will work fine. However, if you want to be able to continue using m as a list in your code, you need to amend Example 1 like so:
Example 1 (amended):
w = [[1,5,7],[2,2,2,9],[1,2],[0]]
m = map(sum,w) # map iterator is generated
m = list(m) # map iterator is consumed here, but it is converted to a reusable list.
for v in m:
print(v) # now you are iterating a list, so you should have no issue iterating
# and reiterating to your heart's content!
It's because it return an generator so clearer example:
>>> gen=(i for i in (1,2,3))
>>> list(gen)
[1, 2, 3]
>>> for i in gen:
print(i)
>>>
Explanation:
it's because to convert it into the list it basically loops trough than after you want to loop again it will think that still continuing but there are no more elements
so best thing to do is:
>>> M=list(map(sum,W))
>>> M
[13, 15, 3, 0]
>>> for i in M:
print(i)
13
15
3
0
You can either use this:
list(map(sum,W))
or this:
{*map(sum,W)}
I am using the locals function to make a iterable object populated with the values of a,b,c provided when I call this function. When I iterate through the dictionairy that locals returns, I iterate through the variable names, rather than there values. But l.values is not iterable, I guess it points to a function, not a list like I was expecting.
Traceback (most recent call last):
file.py on line ?, in getUserOutputs
userOutput = _runaedlz(testInputs[i])
file.py on line ?, in _runaedlz
return extraNumber(*_fArgs_dfasruobxant)
file.py on line 5, in extraNumber
for i in l:
TypeError: 'builtin_function_or_method' object is not iterable
Is there a better way to make a list of these values? I could do it in a more brute force way, but I'm trying to learn all these built in functions and what they are good for.
Or is there a way to iterate through the dictionairies values that I'm missing?
Two different ways to accomplish the same thing, yes?
def extraNumber(a, b, c):
l=locals()
l=l.values
print(l)
for i in l:
if(l.count(i)==1):
return i
A potential problem with locals() is that it returns a dictionary, and the order of values in a dictionary isn't defined, hence listing the values might permute them relative to the order in which they were passed to the function. If you really want to do something like this, you can get the order defined by using the function's __code__ object's __co_varnames attribute. Something like:
def extraNumber(a, b, c):
loc = locals() #should be the first line, so that only parameters have values
vals = list(loc.values()) #order might have been changed
lvals = extraNumber.__code__.co_varnames
orderedVals = [loc[x] for x in lvals if x in loc]
return vals, orderedVals #both returned for comparison
When I ran it I got:
>>> extraNumber(1,2,3)
([3, 2, 1], [1, 2, 3])
Different runs (especially in fresh Python shells) will have different orders in the first returned list, but the second will always be [1,2,3]
>>> def extraNumber(a,b,c):
... for i in locals().values():
... yield i
...
>>>
>>> for n in extraNumber(5,10,7):
... n
...
10
5
7
I've used a yield statement because the extraNumber function will produce one value for each argument value.
As an alternative, if you want the entire list at once you could use something like this.
>>> def extraNumber(a,b,c):
... parameterValues=list(locals().values())
... return parameterValues
...
>>> extraNumber(5,10,7)
[10, 5, 7]
I'm from a C++ background so this problem seems a little absurd to me:
Let's suppose I have a function:
def scale(data, factor):
for val in data:
val *= factor
This doesn't work as intended, if I pass a list, it changes nothing, but
def scale(data, factor):
for index, val in enumerate(data):
data[index] *= factor
and lst = [val * factor for val in lst] works properly.
How does Python handle argument passing? How do I know if the actual reference, or alias is passed?
if you want to mutate the list, you need to reference the elements. This version uses map (it could be written using list comprehensions)
def scale(data, factor):
return map(lambda x : x*factor, data)
a lambda function is an anonymous function.
>>> (lambda x : x + 1) (5)
6
The x takes the place of the variable in this case 5 + 1
So in this case, we traverse the list applying the function f(x) -> x * factor to every element of the list. The original list is not mutated, but instead we return a new version.
In python basic data types are passed by value - for example int, str, bool etc are passed by value
Derived data types like classes, enum, list, dict are passed by reference.
In your example, the problem is how you use the for loop - not the function argument. If you do:
for val in lst:
val += 1
The values inside lst won't get updated because the val is not the same as lst[0], lst[1] and so on IF val is of the basic data types. So, even here, the val is copied by value.
Second, In your example with enumerate:
But when you loop over the enumerated list, you are using data[index] - which modifies the element in the actual list.
And finally, In your example with the generator:
lst = [val * factor for val in lst] - here the generator loops over every element and creates a new list which is again stored in lst. This is something like a = a + 2 but extended to lists.
This behaviour is so because the basic data types are passed by value and the derived data types like lists are passed by reference consider this
>>> x = 24
>>> x + 1
25
>>> x
24
but on the otherhand with a list
>>> y = [1, 2, 3, 4]
>>> y.remove(2)
>>> y
[1,3,4]
so you should always be careful to reassign values back when performing operations on them in the case of the basic data ypes and also be careful with datatypes that are passed by reference because you could accidentally modify a variable without knowing
So I'm trying to do this.
a = []
map(lambda x: a.append(x),(i for i in range(1,5)))
I know map takes a function but so why doesn't it append to the list? Or is append not a function?
However printing a results to a still being empty
now an interesting thing is this works
a = []
[a.append(i) for i in range(5)]
print(a)
aren't they basically "saying" the same thing?
It's almost as if that list comprehension became some sort of hybrid list-comprehension function thing
So why doesn't the lambda and map approach work?
I am assuming you are using Python 3.x , the actual reason why your code with map() does not work is because in Python 3.x , map() returns a generator object , unless you iterate over the generator object returned by map() , the lambda function is not called . Try doing list(map(...)) , and you should see a getting filled.
That being said , what you are doing does not make much sense , you can just use -
a = list(range(5))
append() returns None so it doesn't make sense using that in conjunction with map function. A simple for loop would suffice:
a = []
for i in range(5):
a.append(i)
print a
Alternatively if you want to use list comprehensions / map function;
a = range(5) # Python 2.x
a = list(range(5)) # Python 3.x
a = [i for i in range(5)]
a = map(lambda i: i, range(5)) # Python 2.x
a = list(map(lambda i: i, range(5))) # Python 3.x
[a.append(i) for i in range(5)]
The above code does the appending too, however it also creates a list of None values as the size of range(5) which is totally a waste of memory.
>>> a = []
>>> b = [a.append(i) for i in range(5)]
>>> print a
[0, 1, 2, 3, 4]
>>> print b
[None, None, None, None, None]
The functions map and filter have as first argument a function reference that is called for each element in the sequence (list, tuple, etc.) provided as second argument AND the result of this call is used to create the resulting list
The function reduce has as first argument a function reference that is called for first 2 elems in the sequence provided as second argument AND the result is used together with the third elem in another call, then the result is used with the fourth elem, and so on. A single value results in the end.
>>> map(lambda e: e+10, [i for i in range(5)])
[10, 11, 12, 13, 14]
>>> filter(lambda e: e%2, [i for i in range(5)])
[1, 3]
>>> reduce(lambda e1, e2: e1+e2, [i for i in range(5)])
10
Explanations:
map example: adds 10 to each elem of list [0,1,2,3,4]
filter example: keeps only elems that are odd of list [0,1,2,3,4]
reduce example: add first 2 elems of list [0,1,2,3,4], then the result and the third elem of list, then the result and fourth elem, and so on.
This map doesn't work because the append() method returns None and not a list:
>>> a = []
>>> type(a.append(1))
<class 'NoneType'>
To keep it functional why not use reduce instead?
>>> from functools import reduce
>>> reduce(lambda p, x: p+[x], (i for i in range(5)), [])
[0, 1, 2, 3, 4]
Lambda function will not get triggered unless you wrap the call to map function in list() like below
list(map(lambda x: a.append(x),(i for i in range(1,5))))
map only returns a generator object which needs to be iterated in order to create a list. Above code will get the lambda called.
However this code does not make much sense considering what you are trying to achieve
I am wondering if there is there is a simple Pythonic way (maybe using generators) to run a function over each item in a list and result in a list of returns?
Example:
def square_it(x):
return x*x
x_set = [0,1,2,3,4]
squared_set = square_it(x for x in x_set)
I notice that when I do a line by line debug on this, the object that gets passed into the function is a generator.
Because of this, I get an error:
TypeError: unsupported operand type(s) for *: 'generator' and 'generator'
I understand that this generator expression created a generator to be passed into the function, but I am wondering if there is a cool way to accomplish running the function multiple times only by specifying an iterable as the argument? (without modifying the function to expect an iterable).
It seems to me that this ability would be really useful to cut down on lines of code because you would not need to create a loop to fun the function and a variable to save the output in a list.
Thanks!
You want a list comprehension:
squared_set = [square_it(x) for x in x_set]
There's a builtin function, map(), for this common problem.
>>> map(square_it, x_set)
[0,1,4,9,16] # On Python 3, a generator is returned.
Alternatively, one can use a generator expression, which is memory-efficient but lazy (meaning the values will not be computed now, only when needed):
>>> (square_it(x) for x in x_set)
<generator object <genexpr> at ...>
Similarly, one can also use a list comprehension, which computes all the values upon creation, returning a list.
Additionally, here's a comparison of generator expressions and list comprehensions.
You want to call the square_it function inside the generator, not on the generator.
squared_set = (square_it(x) for x in x_set)
As the other answers have suggested, I think it is best (most "pythonic") to call your function explicitly on each element, using a list or generator comprehension.
To actually answer the question though, you can wrap your function that operates over scalers with a function that sniffs the input, and has different behavior depending on what it sees. For example:
>>> import types
>>> def scaler_over_generator(f):
... def wrapper(x):
... if isinstance(x, types.GeneratorType):
... return [f(i) for i in x]
... return f(x)
... return wrapper
>>> def square_it(x):
... return x * x
>>> square_it_maybe_over = scaler_over_generator(square_it)
>>> square_it_maybe_over(10)
100
>>> square_it_maybe_over(x for x in range(5))
[0, 1, 4, 9, 16]
I wouldn't use this idiom in my code, but it is possible to do.
You could also code it up with a decorator, like so:
>>> #scaler_over_generator
... def square_it(x):
... return x * x
>>> square_it(x for x in range(5))
[0, 1, 4, 9, 16]
If you didn't want/need a handle to the original function.
Note that there is a difference between list comprehension returning a list
squared_set = [square_it(x) for x in x_set]
and returning a generator that you can iterate over it:
squared_set = (square_it(x) for x in x_set)