Why does Python throw a RecursionError on line 10 but not on line 13?
def fib(n, a = [0,1]):
if len(a) > n:
return a[n]
a.append(fib(n - 1, a) + fib(n - 2, a))
return a[n]
def main():
x = 998
print(fib(x)) # RecursionError
for i in range(1000):
print(fib(i)) # No Error
main()
Looking further into this, it looks like the previously mentioned mutable default argument treatment by python is actually the reason behind this. Instead of creating a new list every time the function is called, the list is created once when the function is defined.
Here is a function exploiting this feature, so that calling fib(998) doesn't throw the RecursionError:
def fib(n, a = [0,1]):
if len(a) > n:
return a[n]
while len(a) <= n:
a.append(fib(len(a) - 1) + fib(len(a) - 2))
return a[n]
Related
So was playing around with memory_profiler module and noticed something weird. I was working on a function used to find nCr combinations using recursion. I made 2 version of of my function :
ver 1:
#profile
def c(n, r, memo={}):
pair = (n, r)
if pair in memo:
return memo[pair]
if r == 1:
return n
if n - r < r:
return c(n, n - r)
if r == 0 or r == n:
return 1
return c(n - 1, r - 1) + c(n - 1, r)
print(c(20, 10))
ver 2:
#profile
def c(n, r, memo={}):
pair = (n, r)
if pair in memo:
return memo[pair]
if r == 1:
return n
if n - r < r:
return c(n, n - r)
if r == 0 or r == n:
return 1
memo[pair] = c(n - 1, r - 1) + c(n - 1, r)
return memo[pair]
print(c(20, 10))
memory_profiler :
ver 1 :
ver 2 :
As you can see ver 1 and ver 2 are the same except for the last 2 lines of the function. But running memory_profiler on them they have great difference in performance. The thing I'm most puzzled by is although I forgot to pass memo in the function call it still results in such great performance improvement.
Each function call results in a new empty memo dict for that function. So why does line 8 in version 1 evaluates as False and fails to return whereas the same line in ver 2 evaluates True and returns 36 times. Shouldn't they both realistically be false as I'm trying to find key in an empty dictionary ?
Shouldn't there be virtually no observable difference in performance ?
Your assumption that
Each function call results in a new empty memo dict for that function.
is incorrect. Unlike in JavaScript, in Python default parameter values are evaluated at definition time, once. If the c function is called without a value for the memo parameter, the instance of the dict constructed at definition time will be used as its value each time. At construction time it is empty, but subsequent invocations may mutate it. Observe the following:
def accumulate(a, b=[]):
b.append(a)
print(b)
accumulate(1)
accumulate('foo')
accumulate(None)
The above will print [1], then [1, 'foo'], then [1, 'foo', None].
Using default parameter values for memoisation is a pretty bad idea. Not only is it rather obscure and confusing, it also allows the caller to break the function by actually providing a value for the memo parameter. A better idea would be to use a global/non-local binding, or a decorator such as functools.cache or functools.lru_cache.
Here is the code I currently have.
def fibonacci(n):
if n == 1:
return 1
elif n == 2:
return 1
else:
value = fibonacci(n - 1) + fibonacci(n - 2)
return value
This currently takes quite some time to calculate values greater than n = 30. Is there a more computationally efficient method to accomplish this?
Adding a value cache to trade some memory for a reduced processing time can be a useful method. A purely recursive program will attempt to calculate values over and over again, however this takes time for larger values. If the values do not change, then storing them can be helpful. It is important to note, however, that should values be volatile you might need a different approach.
fibonacci_value_cache = {}
def fibonacci(n):
if n == 1:
return 1
elif n == 2:
return 1
elif n in fibonacci_value_cache:
return fibonacci_value_cache[n]
else:
fibonacci_value_cache[n] = fibonacci(n - 1) + fibonacci(n - 2)
return fibonacci_value_cache[n]
n = 100
print("Fib " + str(n) + ": " + str(fibonacci(n)))
Here, we check if the value is in the dictionary and return it if it is, otherwise we calculate it and add it to the dictionary. This means that we are make better use of the processor by not calculating the same value multiple times.
There's a recipe for a decorator that uses as an example exactly what you want. It's named Memoize in the PythonDecoratorLibrary.
It may seem like overkill, but having the memoized decorator around could be useful for other future tasks. That said, here it is in its entirety (although I changed the print at the end):
import collections
import functools
class memoized(object):
'''Decorator. Caches a function's return value each time it is called.
If called later with the same arguments, the cached value is returned
(not reevaluated).
'''
def __init__(self, func):
self.func = func
self.cache = {}
def __call__(self, *args):
if not isinstance(args, collections.Hashable):
# uncacheable. a list, for instance.
# better to not cache than blow up.
return self.func(*args)
if args in self.cache:
return self.cache[args]
else:
value = self.func(*args)
self.cache[args] = value
return value
def __repr__(self):
'''Return the function's docstring.'''
return self.func.__doc__
def __get__(self, obj, objtype):
'''Support instance methods.'''
return functools.partial(self.__call__, obj)
#memoized
def fibonacci(n):
"Return the nth fibonacci number."
if n in (0, 1):
return n
return fibonacci(n-1) + fibonacci(n-2)
print(fibonacci(12))
Using idea of Dynamic Programming, and store the intermediate results to save computational cost, it could be very efficient. The code below cost less than 0.02s for n=10000 on my laptop.
def fib(n): # return Fibonacci series up to n
result = []
a, b = 0, 1
for i in range(n):
result.append(b)
a, b = b, a + b
return result
No need for caching/memoization. Here's a Python 3 implementation that expresses the Fibonacci sequence as powers of a matrix, then does efficient exponentiation via halving and squaring. The result is O(log n) in both time and storage.
def matrix_fib(n):
if n == 1:
return [0,1]
else:
f = matrix_fib(n // 2)
c = f[0] * f[0] + f[1] * f[1]
d = f[1] * (f[1] + 2 * f[0])
return [c,d] if (n & 1) == 0 else [d,c+d]
def fib(n):
return n if n == 0 else matrix_fib(n)[1]
print(fib(1000000))
On my laptop this coughs up the value of the millionth Fibonacci number in a little over half a second, and the bulk of that is probably in the big integer arithmetic and formatting of the output—the result is ridiculously large. You don't need to worry about stack overflow, though. The call stack depth for this is only log2(1000000) = 20.
If Python does not support method overloading (besides *args and **kwargs or PEP 3124), then why does this overload work?
# the sum from 1 to n
def sum(n):
if n > 0:
return n + sum(n - 1)
else:
return 0
print(sum(3))
# the sum from n to m, inclusive
def sum(n, m):
if n <= m:
return n + sum(n + 1, m)
else:
return 0
print(sum(3,5))
... while more baffling, this one does not:
# the sum of elements in an array
def sumArray(A):
return sumArray(A, len(A)-1)
# a helper for the above
def sumArray(A, i):
if i < 0:
return 0
else:
return A[i] + sumArray(A, i-1)
print(sumArray([1,2,3]))
You aren't overloading. You're hiding one thing behind another by using the same name for different objects. Try
sum = 42
and see how print(sum(3, 5)) ceases to work.
Function definitions are variable assignments. They create a function and assign it to the variable matching the name you used. You're seeing the ordinary effects of reassigning a variable.
def sum(n):
...
This assigns a function of 1 argument to the variable sum.
print(sum(3))
This uses the function assigned to that variable.
def sum(n, m):
...
This assigns a new function to the variable sum, replacing the first function.
print(sum(3,5))
This uses the new function. If you had tried to use the old function, you wouldn't find it.
# the sum of elements in an array
def sumArray(A):
return sumArray(A, len(A)-1)
# a helper for the above
def sumArray(A, i):
if i < 0:
return 0
else:
return A[i] + sumArray(A, i-1)
print(sumArray([1,2,3]))
This assigns a function to sumArray, then assigns a different function to sumArray, then tries to use the value from the first assignment. It finds the second function, and fails.
In your first example, you define function and use it, then overwrite it with another, and use the new one, just like with regular variables:
a = 1
print(a)
a = 2
print(a)
I am currently new to Python and I'm not sure why i'm getting the error:
a<r raised exception RuntimeError: maximum recursion depth exceeded while calling a Python object
when I do this:
a = Rational(1,3)
r = Rational(0,5)
print(a<r)
My current code is:
class Rational:
def _gcd(x,y):
while y != 0:
x, y = y, x % y
return x
def __init__(self, num = 0, denom = 1):
gcd = Rational._gcd(num, denom)
self.num = int(num / gcd)
self.denom = int(denom / gcd)
def __lt__(self, right):
return Rational(self.num, self.denom) < Rational(right.num, right.denom)
It also happens for all the other relational operators when I do the same thing.
Can someone enlighten me on this particular matter? How do I approach or fix this?
Thanks!
This line:
Rational(self.num, self.denom) < Rational(right.num, right.denom)
… is calling the __lt__ method again, leading to an infinite recursion. Try a different approach, assuming that we're using Python 3.x (or in Python 2.x, that from __future__ import division was executed beforehand), this should work:
self.num/self.denom < right.num/right.denom
I am trying to write a short function in Python 3.3.2. Here's my module:
from math import sqrt
phi = (1 + sqrt(5))/2
phinverse = (1-sqrt(5))/2
def fib(n): # Write Fibonacci numbers up to n using the generating function
list = []
for i in range(0,n):
list = list.append(int(round((phi**i - phinverse**i)/sqrt(5), 0)))
return(list)
def flib(n): # Gives only the nth Fibonacci number
return(int(round((phi**n - phinverse**n)/sqrt(5), 0)))
if __name__ == "__main__":
import sys
fib(int(sys.argv[1]))
When I run fibo.fib(6), I get the following error:
list = list.append(int(round((phi**i - phinverse**i)/sqrt(5), 0)))
AttributeError: 'NoneType' object has no attribute 'append'
How do I rectify this error?
The return type of
list.append
is None
When you do list = list.append(int(round((phi**i - phinverse**i)/sqrt(5), 0)))
it is assigning list=None
Just do
for i in range(0,n):
list.append(int(round((phi**i - phinverse**i)/sqrt(5), 0)))
Also, list is an builtin type. So use a different variable name.
The append call doesn't return a list, it will update in place.
list = list.append(int(round((phi**i - phinverse**i)/sqrt(5), 0)))
should become
list.append(int(round((phi**i - phinverse**i)/sqrt(5), 0)))
You should probably also call the argument something other than list because that word it used to identify the list class as well.
You could also use a list comprehension:
def fib(n):
'''Write Fibonacci numbers up to n using the generating function'''
return [int(round((phi**i - phinverse**i)/sqrt(5), 0))) for i in range(0, n)]