We Keep Coding

Python 3X evaluation behavior of dictionary get() method

I recently had following code in mind and wondered what was wrong with it. Previously I used the .get method of dictionaries with success, but now i wanted to pass arguments too and this is where i noticed a somewhat weird behavior:
def string_encoder(nmstr):
return nmstr.encode('UTF-8')
def int_adder(nr_int):
return int(nr_int) + int(nr_int)
def selector(fun, val):
return {'str_en': string_encoder(val),
'nr_add': int_adder(val)}.get(fun, string_encoder(val))
selector('str_en', 'Test') -> ValueError
selector('str_en', 1) -> AttributeError
The above code will never run.
To inspect the issue i supplied a small piece of code:
def p1(pstr):
print('p1: ', pstr)
return pstr
def p2(pstr):
print('p2: ', pstr)
return pstr
def selector_2(fun, val):
return {'p1': p1(val),
'p2': p2(val)}.get(fun, p2(val))
selector_2('p1', 'Test')
Out[]: p1: Test
p2: Test
p2: Test
'Test'
I would expect the following .get('p1', 'test') to output 'p1: test' test.
But as it appears to me, every argument is evaluated, even if it is not selected. So my question is: Why is every argument evaluated with the .get method, or how can this behavior be explained?

dict creation is eager, as is argument evaluation. So before get even runs, you've called string_encoder twice, and int_adder once (and since the behaviors are largely orthogonal, you'll get an error for anything but a numeric str like "123").
You need to avoid calling the function until you know which one to call (and ideally, only call that function once).
The simplest solution is to have the dict and get call contain the functions themselves, rather than the result of calling them; you'll end up with whichever function wins, and you can then call that function. For example:
def selector(fun, val):
# Removed (val) from all mentions of functions
return {'str_en': string_encoder,
'nr_add': int_adder}.get(fun, string_encoder)(val) # <- But used it to call resulting function
Given string_encoder is your default, you could remove 'str_en' handling entirely to simplify to:
return {'nr_add': int_adder}.get(fun, string_encoder)(val)
which leads to the realization that you're not really getting anything out of the dict. dicts have cheap lookup, but you're rebuilding the dict every call, so you didn't save a thing. Given that you really only have two behaviors:
Call int_adder if fun is 'nr_add'
Otherwise, call string_encoder
the correct solution is just an if check which is more efficient, and easier to read:
def selector(fun, val):
if fun == 'nr_add':
return int_adder(val)
return string_encoder(val)
# Or if you love one-liners:
return int_adder(val) if fun == 'nr_add' else string_encoder(val)
If your real code has a lot of entries in the dict, not just two, one of which is unnecessary, then you can use a dict for performance, but build it once at global scope and reference it in the function so you're not rebuilding it every call (which loses all performance benefits of dict), e.g.:
# Built only once at global scope
_selector_lookup_table = {
'str_en': string_encoder,
'nr_add': int_adder,
'foo': some_other_func,
...
'baz': yet_another_func,
}
def selector(fun, val):
# Reused in function for each call
return _selector_lookup_table.get(fun, default_func)(val)

If you want to avoid evaluation of functions and only chooses the function, do this instead for your second block (the syntax will also work for your first block):
def selector_2(fun, val):
return {'p1': p1,
'p2': p2}.get(fun)(val)

Getting inputs to a function from a python object

I have a class. This class has a list of functions that are to be evaluated by a different program.
class SomeClass(object):
def __init__(self, context):
self.functions_to_evaluate = []
There is a function that adds functions to an instance of SomeClass, via something like:
new_function = check_number(5)
SomeClassInstance.functions_to_evaluate.append(new_function)
Where check_number is a function that will check if number is greater than 10, let's say.
If I take SomeClassInstance.functions_to_evaluate and print it, I get a bunch of python objects, like so:
<some_library.check_number object at 0x07B35B90>
I am wondering if it is possible for me to extract the input given to check_number, so something like:
SomeClassInstance.functions_to_evaluate[0].python_feature() that will return "5" or whatever the input to check_number was to me.

You can use the standard library functools.partial, which creates a new partially applied function *.
>>> from functools import partial
>>> def check_number(input):
... return input > 10
>>> fn = partial(check_number, 5)
>>> fn.args # this attribute gives you back the bound arguments, as a tuple.
(5,)
>>> fn() # calls the function with the bound arguments.
False
*: actually the partial object is not a function instance, but it is a callable, and from a duck-type perspective it's a function.

If new_function = check_number(5) is a closure, then you can extract this value using __closure__[0].cell_contents:
Example:
def foo(x):
def inn(y):
return x
return inn
s = foo(5)
print(s.__closure__[0].cell_contents)
Output:
5

I understand your confusion, but:
new_function = check_number(5)
Is calling the function, and the new_function variable gets assigned the return value of the function.
If you have this check_number function:
def check_number(input):
return input > 10
Then it will return False, and new_function will be False. Never <some_library.check_number object at 0x07B35B90>.
If you're getting <some_library.check_number object at 0x07B35B90> then your check_number() function is returning something else.

There are probably several ways to skin this cat. But I'd observe first and foremost that you're not adding python function objects to the functions_to_evaluate list, you're adding the evaluations of functions.
You could simply add a tuple of function, args to the list:
SomeClassInstace.functions_to_evaluate.append((check_number, 5))
And then you can:
for f, args in SomeClassInstance.functions_to_evaluate:
print(args)

Can anyone explain how this functional program work?

def apply_twice(func,arg):
return func(func(arg))
def add_five(x):
return x+5
print (apply_twice(add_five,10))
The output I get is 20.
This one is actually confusing me like how is it working.Can anybody explain me how this is working by breaking it down

The function apply_twice(func,arg) takes two arguments, a function object func and an argument to pass to the function func called arg.
In Python, functions can easily be passed around to other functions as arguments, they are not treated differently than any other argument type (i.e first class citizens).
Inside apply_twice, func is called twice in the line:
func(func(arg))
Which, alternatively, can be viewed in a more friendly way as:
res = func(arg)
func(res)
If you replace func with the name of the function passed in add_five you get the following:
res = add_five(arg) # equals: 15
add_five(res) # result: 20
which, of course, returns your expected result.
The key point to remember from this is that you shouldn't think of functions in Python as some special construct, functions are objects just like ints, listss and everything else is.

Expanding the code it executes as follows, starting with the print call:
apply_twice(add_five,10))
add_five(add_five(10)) # add_five(10) = 15
add_five(15) # add_five(15) = 20
Which gives you the result: 20.
When apply_twice is called, you are passing in a function object and a value. As you can see in the apply_twice definition, where you see func that is substituted with the function object passed to it (in this case, add_five). Then, starting with the inner func(arg) call, evaluate the result, which is then passed to add_five again, in the outer return func( ... ) call.

What you need to understand here is that
apply_twice(func,arg)
is a higher function which accepts two arguments (another function named func and an argument arg). The way it works is that it first evaluate the value of the other function, then use the value as an argument inside the higher function.
remember we have a function add_five(x) which add 5 to the argument supply in it...
then this function add_five(x) is then passed as an argument to another function called
apply_twice_(func,arg) which return func(func(arg)).
now splitting func(func(arg)) we have
func(arg) #lets called it a
then func(func(arg))==func(a) since a = func(agr)
and (a) is our add_five(x) function, after it add 5, then the value we got is re-used as another fresh argument to add another 5 to it, that is why we have 20 as our result.
Another example is:
def test(func, arg):
return func(func(arg))
def mult(x):
return x * x
print(test(mult, 2))
which give 16 as result.

Is there a way to check if function is recursive in python?

I want to write a testing function for an exercise, to make sure a function is implemented correctly.
So I got to wonder, is there a way, given a function "foo", to check if it is implemented recursively?
If it encapsulates a recursive function and uses it it also counts. For example:
def foo(n):
def inner(n):
#more code
inner(n-1)
return inner(n)
This should also be considered recursive.
Note that I want to use an external test function to perform this check. Without altering the original code of the function.

Solution:
from bdb import Bdb
import sys
class RecursionDetected(Exception):
pass
class RecursionDetector(Bdb):
def do_clear(self, arg):
pass
def __init__(self, *args):
Bdb.__init__(self, *args)
self.stack = set()
def user_call(self, frame, argument_list):
code = frame.f_code
if code in self.stack:
raise RecursionDetected
self.stack.add(code)
def user_return(self, frame, return_value):
self.stack.remove(frame.f_code)
def test_recursion(func):
detector = RecursionDetector()
detector.set_trace()
try:
func()
except RecursionDetected:
return True
else:
return False
finally:
sys.settrace(None)
Example usage/tests:
def factorial_recursive(x):
def inner(n):
if n == 0:
return 1
return n * factorial_recursive(n - 1)
return inner(x)
def factorial_iterative(n):
product = 1
for i in xrange(1, n+1):
product *= i
return product
assert test_recursion(lambda: factorial_recursive(5))
assert not test_recursion(lambda: factorial_iterative(5))
assert not test_recursion(lambda: map(factorial_iterative, range(5)))
assert factorial_iterative(5) == factorial_recursive(5) == 120
Essentially test_recursion takes a callable with no arguments, calls it, and returns True if at any point during the execution of that callable the same code appeared twice in the stack, False otherwise. I think it's possible that it'll turn out this isn't exactly what OP wants. It could be modified easily to test if, say, the same code appears in the stack 10 times at a particular moment.

from inspect import stack
already_called_recursively = False
def test():
global already_called_recursively
function_name = stack()[1].function
if not already_called_recursively:
already_called_recursively = True
print(test()) # One recursive call, leads to Recursion Detected!
if function_name == test.__name__:
return "Recursion detected!"
else:
return "Called from {}".format(function_name)
print(test()) # Not Recursion, "father" name: "<module>"
def xyz():
print(test()) # Not Recursion, "father" name: "xyz"
xyz()
The output is
Recursion detected!
Called from <module>
Called from xyz
I use the global variable already_called_recursively to make sure I only call it once, and as you can see, at the recursion it says "Recursion Detected", since the "father" name is the same as the current function, which means I called it from the same function aka recursion.
The other prints are the module-level call, and the call inside xyz.
Hope it helps :D

I have not yet verified for myself if Alex's answer works (though I assume it does, and far better than what I'm about to propose), but if you want something a little simpler (and smaller) than that, you can simply use sys.getrecursionlimit() to error it out manually, then check for that within a function. For example, this is what I wrote for a recursion verification of my own:
import sys
def is_recursive(function, *args):
try:
# Calls the function with arguments
function(sys.getrecursionlimit()+1, *args)
# Catches RecursionError instances (means function is recursive)
except RecursionError:
return True
# Catches everything else (may not mean function isn't recursive,
# but it means we probably have a bug somewhere else in the code)
except:
return False
# Return False if it didn't error out (means function isn't recursive)
return False
While it may be less elegant (and more faulty in some instances), this is far smaller than Alex's code and works reasonably well for most instances. The main drawback here is that with this approach, you're making your computer process through every recursion the function goes through until reaching the recursion limit. I suggest temporarily changing the recursion limit with sys.setrecursionlimit() while using this code to minimize the time taken to process through the recursions, like so:
sys.setrecursionlimit(10)
if is_recursive(my_func, ...):
# do stuff
else:
# do other stuff
sys.setrecursionlimit(1000) # 1000 is the default recursion limit

How to use iter(v,w) with a function?

I was study iter(), in its official document, it says i can do iter(v,w) , so that iter() will call v until it return the value w, then it stops.
But I tried for half hour, and still can't work out a function that can return multiple result.
Here is my code, I expect it to return 1,2,3,4,5:
def x():
for i in range(10):
return i
a = iter(x, 5)
a.next()
I know that when I return i, that I was actually quit the function.
Maybe it's impossible to return result for multiple times for a function.
But how should I use a function to make that iter(x,5) work properly?

iter() calls the function each time. Your function returns the same value on each call (the first number in the range(10) list).
You could change the function to use a global to illustrate how iter() with two arguments works:
i = 0
def f():
global i
i += 1
return i
for x in iter(f, 5):
print x
Now each time f() is called, a new number is returned. You could use a default argument, or an instance with state and a method on that instance, too. As long as the function returns something different when called more than once it'll fit the iter(a, b) usecase.
iter() with two arguments is most often called with a method, where the state of an instance changes with each call. The .readline() method on a file object, for example:
for line in iter(fileobject.readline, ''):
which would work exactly like iterating over the fileobject iterable directly, except it wouldn't use the internal file iteration buffer. That could sometimes be a requirement (see the file.next() method for more information on the file iteration buffer).
You can of course pass in a lambda function too:
for chunk in iter(lambda: fileobject.read(2048), ''):
Now we are reading the file object is chunks of up to 2048 bytes instead of line by line.

After #Martjin Pieters's answer, I've got the idea.
And this is the piece of code I wrote which can use iter(v,w) correctly:
import random
def x():
return random.randrange(1,10)
a = iter(x,5)
while True:
print a.next()
In this code, a.next() will return the value a get from x(), until x() returns 5, then it will stop.

You could also use a generator function, via the yield keyword. Example:
def x():
for i in range(10):
yield i