We Keep Coding

What happens when you invoke a function that contains yield?

I read here the following example:
>>> def double_inputs():
... while True: # Line 1
... x = yield # Line 2
... yield x * 2 # Line 3
...
>>> gen = double_inputs()
>>> next(gen) # Run up to the first yield
>>> gen.send(10) # goes into 'x' variable
If I understand the above correctly, it seems to imply that Python actually waits until next(gen) to "run up to" to Line 2 in the body of the function. Put another way, the interpreter would not start executing the body of the function until we call next.
Is that actually correct?
To my knowledge, Python does not do AOT compilation, and it doesn't "look ahead" much except for parsing the code and making sure it's valid Python. Is this correct?
If the above are true, how would Python know when I invoke double_inputs() that it needs to wait until I call next(gen) before it even enters the loop while True?

Correct. Calling double_inputs never executes any of the code; it simply returns a generator object. The presence of the yield expression in the body, discovered when the def statement is parsed, changes the semantics of the def statement to create a generator object rather than a function object.

The function contains yield is a generator.
When you call gen = double_inputs(), you get a generator instance as the result. You need to consume this generator by calling next.
So for your first question, it is true. It runs lines 1, 2, 3 when you first call next.
For your second question, I don't exactly get your point. When you define the function, Python knows what you are defining, it doesn't need to look ahead when running it.
For your third question, the key is yield key word.

Generator-function is de iure a function, but de facto it is an iterator, i.e. a class (with implemented __next__(), __iter()__, and some other methods.)
          In other words, it is a class disguised as a function.
It means, that “calling” this function is in reality making an instance of this class, and explains, why the “called function” does initially nothing. This is the answer to your 3rd question.
The answer to your 1st question is surprisingly no.
Instances always wait for calling its methods, and the __next__() method (indirectly launched by calling the next() build-in function) is not the only method of generators. Other method is the .send(), and you may use gen.send(None) instead of your next(gen).
The answer to your 2nd question is no. Python interpreter by no mean "look ahead" and there are no exceptions, including your
... except for parsing the code and making sure it's valid Python.
Or the answer to this question is yes, if you mean “parsing only up to the next command”. ;-)

returning values of function in python

Why do I have to return function in the else case? Can't I just apply the defined function because I have to only return the value of b and store it?
def gcd_a(a,b):
if a==0:
return b
else:
gcd_a(b%a,a)

I think the main concept you are missing is that in order to get the result of a recursive function (or any function for that matter), the function that you access the result of must return the value*
Right now, when you call gcd_a, a recursive call to gcd_a will eventually return a value, however it will be lost, since the function that you are accessing the result of does not return a value.
To show you this is true, let's add in a log statement that prints when the function is returning:
def gcd_a(a,b):
if a==0:
print('finally returning!', b)
return b
else:
gcd_a(b%a,a)
Now if we call:
print(gcd_a(10, 99))
We get the following output:
finally returning! 1
None
Somewhere, a recursive call to gcd_a found your condition to be True, and returned 1, however, this result is not printed, because it is not returned by your call to gcd_a
* Unless you do something strange like updating a global variable, etc...

If your function finishes without returning anything, it will implicitly return None.
def double(val):
val * 2
value = double(10)
print(value)
Which is fine, except it doesn't have a return statement, and so all you get is None
The same applies to recursive functions: you can do all the recursive function calls you want, but if you don't actually return the result of the function, then it'll return None.

IMHO, the problem is in the way you are thinking about function which has to be cleared. The fact that you are having this doubt in a recursive function call is incidental because you are calling the same function again and again with different arguments causing different branches to be executed.
How will a function that doesn't return anything in any of its branches be helpful to its caller? In your case, what would happen if your function's caller calls it with an argument that hits your else block? It will return nothing and this won't help the caller!
Now in your recursive case, if your caller calls the function with an argument that hits the if block, then it would work as expected. However, if it hits the else block, then it would become a caller and call the same function again. For simplicity, let us assume that this time it hits the if condition and returns something to its caller. However, will that reach the original caller that initiated everything? The answer is no because you are not returning it to him!
Most of the times you would need a return for every branch in a function
unless you are doing it on purpose for a side effect.

In the else block, if you don't return the function call, the outer function returns None because python interpreter just runs whatever is returned by the gcd function.
Let's assume the following code:
def func():
10
In that function, it just runs 10, but it doesn't mean that there is some return from the function.

How can I have my function called again after storing its result in a variable, each time the variable is accessed?

I have a concept where I store values returned by functions in variables which makes it easier for me.
But I am having the problem that the value in the variable isn't dynamically calling the function each time. So it returns the same value all the time.
I have made a code snip to illustrate it in a easy way:
def value():
resp = requests.get('http://www.google.com').elapsed.total_seconds()
return resp
test = value()
while True:
print test
time.sleep(10)
Output:
0.00649
0.00649
In this case, in the while true when I print test, it returns the same value, even though I am calling the function value(). How can I solve this issue? I know I can put the function in the while loop, but I want to have it as a variable.

The previous answers are correct, but please let me elaborate.
In the world of Python, things have very precise meanings but it's not always clear what is what if you are just getting started.
Expressions are things that have a value, and they include things like 123, some_variable and 10 / 2. The name some_variable is called an identifier, simply because it identifies a value. Just like the name Scruffy might identify your dog.
Statements are things that affect the program flow or the state of your program, but lack a value. The following are examples of statements:
if x > 10:
x -= 1
And
def foo(x):
print("The value of x is", x)
Unlike in JavaScript, for example, where almost everything is an expression (and has a value), you can not do this:
my_function = def foo(x): print("The value of x is", x)
The previous def statement will create a function by the name foo, but the statement itself doesn't have a value. The name foo will refer to a value, however. It means that when you write foo, the value will be a thing, and this thing is a function!
The expression x() on the other hand will do two things. First it will look up the value of the identifier x. Hopefully this value is a function. After that, the parentheses means that the value will be called. What happens if x is not a function, then?
>>> x = 10
>>> x()
Traceback (most recent call last):
File "<ipython-input-3-7354d77c61ac>", line 1, in <module>
x()
TypeError: 'int' object is not callable
Big Fat Error: 10 is not a "callable". Callable means something that can be called like a function.
The parentheses are, as you probably know, a list of arguments. In this case the list is simply empty so when you write x() you are saying "call whatever the name 'x' is referring to but don't send any arguments".
A function call always has a value, since it's an expression. If you don't explicitly return anything, the value will simply be None.
To answer your question, finally, let's play the substitution game. The rules are simple, any expression can be replaced by its value:
def value():
resp = requests.get('http://www.google.com').elapsed.total_seconds()
return resp
This is a statement, so it doesn't have a value. A function is created with the name value, however.
The function consists of two statements (no value, again), a variable assignment and a return statement.
The thing to the right of the = is an expression however. Short story:
requests is referring to the requests module
get is referring to a module-global function in the above module
get('...') is calling this function, and something is returned.
The "something" has a property called elapsed, which has a property called total_seconds.
total_seconds is an identifier that refers to a callable. The callable is called without any arguments (total_seconds()) and something is returned. Probably a number, based on the name. Let's say its value is always 10, for simplicity.
The next statement is another assignment:
test = value()
This can be thought of as "let the name 'test' refer to the value that is returned by the callable identified by the name 'value' when it is called with an empty argument list". In our case, the function object called value will be called, resp will be assigned the value 10, then the return statement will let the caller know that this call is sending the value 10 back. The name test will refer to the value 10, from now on.
Let's go over the loop, quickly:
while True:
print test
time.sleep(10)
Do the following until the end of Time:
print (a statement in Python 2, an expression in Python 3!) has the side-effect of printing stuff to the screen. Otherwise it doesn't do much.
The stuff, in this case, is whatever the value of the expression test is. We already know that the identifier test is referring to the value 10. It will simply print "10" to the screen.
Sleep for ten seconds.
Repeat.
You probably want to invoke some function at each iteration of the loop ("invoke" is basically latin for "call", I like fancy words). Otherwise the program will just print "10", "10", "10", over and over again. To fix this this, you first have to change the expression evaluated as part of the print statement from just an identifier (test) to a function call:
print test()
But this will, as we saw before, raise a Big Fat Error since 10 is not a callable function. To fix it (that's what programmers do, right?) you also need to change the value of test from 10, since it's not a callable, to the function. A function can be referred to simply by its name, so just change this line:
test = value() # equals ten
To this:
test = value # equals the function called "value"
The function now has two names, the old name "value", and the new name "test". Each step in the loop will request the page again and return the new time it took for the request to complete. If the request times out you will have a different kind of crash, but that's another story.
Further information can be found in the Python Language Reference.

test = value() isn't storing the function it's storing the results. You should just call value() in your loop or if you want to assign the function it would be test = value and then test() in your loop.

test = value() calls the function and stores the return value. It does not store the function. test = value would store the function, but the you need to print test() in order to call it.
def value():
resp = requests.get('http://www.google.com').elapsed.total_seconds()
return resp
test = value
while True:
print test()
time.sleep(10)

You can change your code like this..
def value():
resp = requests.get('http://www.google.com').elapsed.total_seconds()
return resp
test = value()
while True:
print test
time.sleep(10)
change this line
test = value
to
test = value()
Now means you are storing your function in a variable.

Calling gen.send() with a new generator in Python 3.3+?

From PEP342:
Because generator-iterators begin execution at the top of the generator's function body, there is no yield expression to receive a value when the generator has just been created. Therefore, calling send() with a non-None argument is prohibited when the generator iterator has just started, ...
For example,
>>> def a():
... for i in range(5):
... print((yield i))
...
>>> g = a()
>>> g.send("Illegal")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't send non-None value to a just-started generator
Why is this illegal? The way I understood the use of yield here, it pauses execution of the function, and returns to that spot the next time that next() (or send()) is called. But it seems like it should be legal to print the first result of (yield i)?
Asked a different way, in what state is the generator 'g' directly after g = a(). I assumed that it had run a() up until the first yield, and since there was a yield it returned a generator, instead of a standard synchronous object return.
So why exactly is calling send with non-None argument on a new generator illegal?
Note: I've read the answer to this question, but it doesn't really get to the heart of why it's illegal to call send (with non-None) on a new generator.

Asked a different way, in what state is the generator 'g' directly after g = a(). I assumed that it had run a() up until the first yield, and since there was a yield it returned a generator, instead of a standard synchronous object return.
No. Right after g = a() it is right at the beginning of the function. It does not run up to the first yield until after you advance the generator once (by calling next(g)).
This is what it says in the quote you included in your question: "Because generator-iterators begin execution at the top of the generator's function body..." It also says it in PEP 255, which introduced generators:
When a generator function is called, the actual arguments are bound to function-local formal argument names in the usual way, but no code in the body of the function is executed.
Note that it does not matter whether the yield statement is actually executed. The mere occurrence of yield inside the function body makes the function a generator, as documented:
Using a yield expression in a function definition is sufficient to cause that definition to create a generator function instead of a normal function.

What happened when I change the variable 'x' to 'fab'?Something about decorator and pass the function variable

I define a Factorial function named fab. I use generator to avoid stack overflow.But something I can't understand came up when I try to write a decorator version, which is more intuitive :
import types
def TCO(f):
def inner(*nkw,**kw):
gen=f(*nkw,**kw)
while isinstance(gen,types.GeneratorType):
gen=gen.next()
return gen
return inner
def fab(n,s=1):
if n<2:
yield s
else:
yield fab(n-1,s*n)
x=TCO(fab)
print x(2500) #this works fine, overcoming the limitation of tail-recursion.
fab=TCO(fab) #now just change the variable name.
print fab(5) #this woks fine.
print fab(2500) #this will raise an error :maximum recursion limit exceeded
Why? I know it has something to do with the same name fab, but why fab(5) works fine? I think when I define fab=TCO(fab), I actually change the object refered by f in inner to object TCO(fab). So when fab(5) runs, the gen will never be a generator! Because the inner never returns generator!
I am mad...Why ?

fab=TCO(fab)
statment now makes the variable fab points to the function inner. After that, when
yield fab(n-1,s*n)
is encountered, it is not actually calling the original fab function but the inner function which in turn calls the original fab function. Basically, you are coming out of the generator and entering another function every time. Thats why you are getting maximum recursion limit exceeded error.

I think this would be a good example to explain what happens:
def f(x):
print 'original'
if x > 0:
return f(x-1)
return 0
g = f
def f(x):
print 'new'
return x
print g(5)
result:
original
new
4
this proves:
1.when g(5) runs, origninal `f` is called, not new `f`.
2.as 5>0, 'return f(x-1)' is executed
3.new `f` is called when `f(x-1)` runs. so the result is 4.

Your original version yields a generator which yields a generator which yields a generator, etc. To evaluate that linked list you then use the while loop in you TCO decorator. It all is based on the fact that you are linking generators to avoid a large stack.
This concept is broken when you assign something new to the variable fab. Then yielding fab(n-1, s*n) in fab is accessing the new fab variable and thus not returning a generator anymore but a value. To compute that value, the stack is used, hence you can get the stack overflow problem.