I have a question about how Python(3) internally loops when computing multiple maps. Here's a nonsense example:
from random import randint
A = [randint(0,20) for _ in range(100)]
map1 = map(lambda a: a+1, A)
map2 = map(lambda a: a-1, map1)
B = list(map2)
Because map() produces a lazy expression, nothing is actually computed until list(map2) is called, correct?
When it does finally do this computation, which of these methods is it more akin to?
Loop method 1:
A = [randint(0,20) for _ in range(100)]
temp1 = []
for a in A:
temp1.append(a+1)
B = []
for t in temp1:
B.append(t-1)
Loop method 2:
A = [randint(0,20) for _ in range(100)]
B = []
for a in A:
temp = a+1
B.append(temp-1)
Or does it compute in some entirely different manner?
In general, the map() function produces a generator, which in turn doesn't produce any output or calculate anything until it's explicitly asked to. Converting a generator to a list is essentially akin to asking it for the next element until there is no next element.
We can do some experiments on the command line in order to find out more:
>>> B = [i for i in range(5)]
>>> map2 = map(lambda b:2*b, B)
>>> B[2] = 50
>>> list(map2)
[0, 2, 100, 6, 8]
We can see that, even though we modify B after creating the generator, our change is still reflected in the generator's output. Thus, it seems that map holds onto a reference to the original iterable from which it was created, and calculates one value at a time only when it's asked to.
In your example, that means the process goes something like this:
A = [2, 4, 6, 8, 10]
b = list(map2)
b[0] --> next(map2) = (lambda a: a-1)(next(map1))
--> next(map1) = (lambda a: a+1)(next(A))
--> next(A) = A[0] = 2
--> next(map1) = 2+1 = 3
--> next(map2) = 3-1 = 2
...
In human terms, the next value of map2 is calculated by asking for the next value of map1. That, in turn, is calculated from A that you originally set.
This can be investigated by using map on functions with side-effects. Generally speaking, you shouldn't do this for real code, but it's fine for investigating the behaviour.
def f1(x):
print('f1 called on', x)
return x
def f2(x):
print('f2 called on', x)
return x
nums = [1, 2, 3]
map1 = map(f1, nums)
map2 = map(f2, map1)
for x in map2:
print('printing', x)
Output:
f1 called on 1
f2 called on 1
printing 1
f1 called on 2
f2 called on 2
printing 2
f1 called on 3
f2 called on 3
printing 3
So, each function is called at the latest time it could possibly be called; f1(2) isn't called until the loop is finished with the number 1. Nothing needs to be done with the number 2 until the loop needs the second value from the map.
Related
From this answer: How do chained assignments work?, I understand that chained assignement in Python :
x = y = z # (1)
is equivalent to:
temp = z
x = temp
y = temp
But is (1) also equivalent to:
x = z
y = x
?
Or is there a slight difference (for example when z = some_function())? If so, which difference?
In the very example you give, yes, the effects of the two approaches are practically identical because both involve simply assigning the same reference to a number of names.
Be aware, however, that if the expressions in the assignment targets involve more complex evaluations, the two approaches could be different.
For example, consider the following chain expression, where x is initialized as a dict and expensive_func is a time-consuming function that returns a key:
x[expensive_func()] = y = some_function()
While it would be indeed equivalent to the following:
temp = some_function()
x[expensive_func()] = temp
y = temp
it would not be be equivalent to the second approach:
x[expensive_func()] = some_function()
y = x[expensive_func()]
since expensive_func would then have to be called twice, doubling the time taken, and triggering the side effect of the function twice, if it has any.
Also, consider the following code:
obj = []
x = []
x[:] = y = obj
print(id(obj), id(x), id(y))
where the output would show that y gets assigned the same reference as obj, while x is different.
That code is then indeed equivalent to:
obj = []
x = []
temp = obj
x[:] = temp
y = temp
print(id(obj), id(x), id(y))
But not equivalent to:
obj = []
x = []
x[:] = obj
y = x[:]
print(id(obj), id(x), id(y))
The latter of which would show y getting a different reference from both obj and x.
I always find using examples to be the best way to understand things (in general).
Let's say we have a func:
def function_sample ():
print(20)
If you print it:
print(function_sample)
<function function_sample at 0x7f8f840a01f0>
returns the function object.
When assigning to a variable a function without parentheses (without calling/running it).
x = function_sample
print(x)
you will get the same message: <function function_sample at 0x7f8f840a01f0>
However, if you run it (with parentheses).
print(x())
You will see :
20
None
Why None? It's because Python functions have a default return value, which is None if no return expression is given, or return is given on its own.
Another sample:
def another_sample(some):
print(some)
y = another_sample
print(y)
As you probably have guessed it : <function another_sample at 0x7f8f7e747700>
If you try to print y() you will get an error because the some argument is missing.
But if we add one:
print(y(5))
5
None
One last example:
def third_sample ():
return 20
aa = third_sample # without running the func
bb = third_sample() # calling/running the func
print(aa) # function object
print(bb) # 20
The 2 approaches you have shown are both functional and legit in terms of going about chaining and using previous variables. No difference at all
When assigning variables to the same number of variables, instead of doing the typical:
x = 0
y = 0
OR using tuple unpacking approach:
(x,y) = 0,0
You could just do like what you have (chained assignment):
x = y = 0
This could be used with any object (being called on) for the RHS, and that:
x = y = some_object()
is the same as:
tmp = some_object()
x = tmp
y = tmp
and when you del tmp, the xand y become useless or nothing.
I am trying to write a function that will create a list of lambda functions and then passing an array to this list.
I figured out how to hard code this function list, however, I cannot seem to figure out how to use a for loop to create the list.
For example, let's take a very simple function where we multiply every element of A by 1, then 2, then 3, ... and so on so that each row corresponds to the element of A and each column corresponds to the number at which A is multiplied by.
import numpy as np
A = np.array([1,2,3,4])
def f():
F3 = lambda x: 3*x
F2 = lambda x: 2*x
F1 = lambda x: 1*x
F0 = lambda x: 0*x
return lambda x: np.stack((F3(x),F2(x),F1(x),F0(x)),axis=1)
F = f()
F(A)
My output is then.
array([[ 3, 2, 1, 0],
[ 6, 4, 2, 0],
[ 9, 6, 3, 0],
[12, 8, 4, 0]])
The code above only goes to 3*x. What would I do if I want to follow the pattern to n*x? My basic idea would be as follows (however, this does not work):
import numpy as np
A = np.array([1,2,3,4])
def _f():
return lambda x: n*x
def f(N):
F = []
for n in range(N):
F.append(lambda x: _f(n))
return np.array(F)
F = f(5)
F(A)
In real life, my function _f() is far more complicated. The motivation behind this is that I would rather have my program iterate through each _f only once and then perform the calculation F(A) in one shot.
The desired output of can be achieved by the following code, however, this will iterate through the loop each time F is called.
import numpy as np
A = np.array([1,2,3,4])
def _f(n,x):
return n*x
def f(N,x):
F = []
for n in range(N):
F.append(_f(n,x))
return np.array(F)
F = f(5,A)
print(F.T)
This would return:
[[ 0 1 2 3 4]
[ 0 2 4 6 8]
[ 0 3 6 9 12]
[ 0 4 8 12 16]]
This loop is broken:
for n in range(N):
F.append(lambda x: _f(n))
because n is lazily read in the body of the function (so all the functions stored read the final value of n from the loop).
The easiest fix is to bind n as a default argument; default arguments are bound eagerly at definition time:
for n in range(N):
F.append(lambda x, n=n: _f(n))
^^^^^ Changed part
If you want to avoid the default argument, make your factory function do the eager binding for you:
def _f(n): # Now receives n as an argument to outer function
return lambda x: n*x # While inner function uses n from closure and x passed when called
then use it with:
for n in range(N):
F.append(_f(n))
_f (properly defined; it should take n as an argument rather than treating n as a global variable) is already the function which, when called, returns your desired function.
# Or _f = lambda n: lambda x: n * x
def _f(n):
return lambda x: n * x
F = [_f(n) for n in range(N)]
That said, you can avoid lambda expressions and their scoping issues altogether by using functools.partial:
from functools import partial
from operator import mul
F = [partial(mul, n) for n in range(N)]
I'm aware yield generates a value on the fly, by my understanding this means it doesn't keep the value in the memory, and therefore the current value shouldn't be able to interact with the last values.
But I just want to be sure that's the case, could someone confirm if it's possible or not?
I'm going to use 5 as the value in number.
Example without generator:
def factorial(number):
result = number
if number <= 1:
return 1
else:
for x in reversed(range(1, number)): # (4,1) reversed
result *= x # 5*4*3*2*1
return result # returns 120
Is it possible to do the same thing by using the yield function? how?
Thank you
Generators can be stateful:
def fibs():
a, b = 1, 1
while True:
yield b
a, b = b, a + b
g = fibs()
for i in range(10):
print next(g)
Here the state is in the local variables. They are kept alive while the iterator generated by the generator is alive.
EDIT. I'm blind it was a factorial
def factorials():
i = 1
a = 1
while True:
yield a
i+=1
a*=i
or if you need a function not a stream of them then here's a one liner
print reduce(lambda a, b: a*b, (range(1, 10+1)))
I'm trying to write an infinite generator that will repeat every positive integer n times. So for example, if I create f = inf_repeat(3), printing the output of f 10 times would result in:
1 1 1 2 2 2 3 3 3 4
I am close but not quite there. Here's what I've got:
# courtesy of http://stackoverflow.com/questions/279561/what-is-the-python-equivalent-of-static-variables-inside-a-function
# a generator that yields items instead of returning a list
def static_var(varname, value):
def decorate(func):
setattr(func, varname, value)
return func
return decorate
def inf_repeat(k):
count_limit = k
#static_var("counter", 0)
#static_var("number", 0)
def func():
while True:
if func.counter == count_limit:
func.counter = 0
func.number += 1
func.counter += 1
yield func.number
return func
My problem is that this doesn't behave entirely like an iterator. The following commands work:
f3 = inf_repeat(3)
print next(f3())
But it's irritating to have to call f3 with parens. I'd like to be able to use the standard iterator syntax I've seen, such as:
print(f3.next())
and
new_list = [iter(f3)]*5
What do I need to modify in my function to get to that point? Looking at a variety of generator tutorials, it seemed that yield was sufficient to create a generator, but clearly that's not the case.
Also I have no objective to using a module. I checked itertools but maybe I missed something that could do what I want without all this code?
You just need to call the generator object (what you called f3) at some point. You can call it when you create it:
f3 = inf_repeat(3)()
or even inside inf_repeat
# change last line to this
return func()
Putting yield in your function makes it a generator function --- that is, a function that, when called, returns a generator. If you want to get the generator, you need to call your generator function at some point.
Incidentally, your implementation is needlessly complex. You can get your desired behavior much more simply without all those decorators and nested functions:
def inf_repeat(k):
number = k
while True:
yield number // k
number += 1
Then:
>>> list(itertools.islice(inf_repeat(3), 10))
[1, 1, 1, 2, 2, 2, 3, 3, 3, 4]
>>> list(itertools.islice(inf_repeat(4), 13))
[1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4]
Here's a simple solution:
def inf_repeat(N):
i = 1
while True:
for n in range(N):
yield i
i += 1
# Testing:
f = inf_repeat(3)
for j in range(10):
print f.next()
Here's a solution using itertools:
def inf_repeat(N):
return chain.from_iterable(repeat(i, N) for i in count(1))
Another solution using itertools
import itertools as it
def inf_repeat(k):
for i in it.count(1):
for j in [i]*k:
yield j
for n in inf_repeat(3): print n
produces
1
1
1
2
2
2
...
def f(n):
i = 0
while True:
yield i // n
i += 1
OK, our professor explained (kinda) this problem, but it still doesn't make much sense.
Question: Implement the function knice(f,a,b,k) that will return 1 if for some integer a <= x <= b and some integer n <= k, n applications of f on x will be x, (e.g. f(f(f...(f(x)))) = x) and 0 if not.
What the professor provided was:
def knice(f,a,b,k):
f(f(f(...(f(x)))) = x
for i = a to b:
y = f(i)
if y = i break
for j = z to k:
y = f(y)
if y = i break
Personally, that example makes no sense to me, so looking to see if I can get clarification.
OP EDIT 1/19/2012 3:03pm CST
This is the final function that was figured out with the help of the GTA:
def f(x):
return 2*x-3
def knice(f,a,b,k):
x = a
while x <= b:
n = 1
y = f(x)
if y == x:
return 1
while n <= k:
y = f(y)
n=n+1
if y == x:
return 1
x=x+1
return 0
Ignore his code; you should write whatever you feel comfortable with and work out the kinks later.
You want to work out whether
f(a) = a, or f(f(a)) = a, or ..., or f^n(a) = a, or,
f(a+1) = a+1, or f(f(a+1)) = a+1, or ..., or f^n(a+1) = a+1, or,
...
f(b) = b, or f(f(b)) = b, or ..., or f^n(b) = b.
An obvious algorithm should come to mind immediately: try all these values one-by-one! You will need two (nested) loops, because you are iterating over a rectangle of values. Can you now see what to do?
Yeah, I can see why that might be confusing.
Was f(f(f(...(f(x)))) = x wrapped in triple-double-quotes? That's a function documentation string, sort of like commenting your code. It shouldn't have been stand-alone without something protecting it.
Imagine f was called increment_by_one.
Calling increment_by_one 10 times like that on an x of 2 would give 12. No matter how many times you increment, you never seem to get back 2.
Now imagine f was called multiply_by_one.
Calling multiply_by_one 5 times like that on an x of 3 would give 3. Sweet.
So, some example outputs you can test against (you have to write the functions)
knice(increment_by_one, 1, 3, 5) would return 0.
knice(multiply_by_one, 1, 3, 5) would return 1.
As another hint, indentation is important in python.
Here's a concrete example. Start small, and suppose you called knice(f, a=1, b=2, k=1). For k==1, we don't have to worry about iterating the function. The only values of x to consider are 1 and 2, so knice can return 1 (i.e., True) if f(1)==1 or f(2)==2.
Now suppose you called knice(f, a=1, b=2, k=2). You'll have to check f(f(1)) and f(f(2)) as well.
As k gets bigger, you'll have to call f more. And as the range between a and b gets bigger, you'll have to try more values of x as an argument to f.