There's a huge difference in speed between these two recursive functions and I don't understand why. These are solutions for "The Grandest Staircase of Them All" from Google's Foobar. Essentially, you need to determine how many possible staircases can be made with n blocks (a staircase is a sequence of natural numbers in which each number is higher than the previous, with at least two "steps"—for instance, [1, 2, 3] is a staircase consisting of 6 blocks).
Here's the code I wrote:
def solution(n):
return stairs(n, n)
def stairs(lastStair, n):
if (n < 1):
return (1 if n == 0 else 0)
if (lastStair == 1):
return 0
#recursively call stairs() for each possible next step height, and add the results together
return sum([stairs(i, n - i) for i in range(1, lastStair)])
print(solution(200))
And here's what someone else wrote (taken from here):
from functools import lru_cache
#lru_cache(maxsize=None)
def count(height, left):
if left == 0:
return 1
if left < height:
return 0
return count(height + 1, left - height) + count(height + 1, left)
print(count(1, 200) - 1)
Both give correct results. However, my version is much, much slower. On my computer and with n = 200, solution(n) takes about two hours to run, whereas count(1, n) takes less than a second. I added the caching code to mine and the runtime remained the same. What causes solution(200) to take 10,000x as long? Or, perhaps the right question is, why doesn't caching benefit solution(200)?
The #lru_cache is the difference.
Short answer: The lru_cache stores the results of the previous computations while your solution computes everything over and over again. Take some small stairs(e.g. stairs 20, 20) and look how many times do you compute stairs(4,4) or stairs(1,1). You will be surprised.
The more in depth explanation can be found for example here. There they explain the #lru_cache magic with a similar problem. Also stairs, also recursion, however they are not building a new staircase but counting the ways of traversing it.
Related
I have this function:
def rec(lst):
n = len(lst)
if n <= 1:
return 1
return rec(lst[n // 2:]) + rec(lst[:n // 2])
How can I find the time complexity of this function?
Usually in such problems drawing the recursion tree helps.
Look at this photo I added, note how each level sums up to N (since slicing is the thing here doing the work),
and the depth of the tree is logN (this is easy to show, since we divide by 2 each time, you can find an explanation here). So what we have is the function doing O(n) n*logn times which means in total we have O(n*logn).
Now another way of understanding this is using the "Master Theorem" (I encourage you to look it up and learn about it)
We have here T(n) = 2T(n/2) + O(n), so according to the theorem a=2, b=2 so log_b(a) is equal to 1, and therefore
we have (according to the 2nd case of the theorem):
T(n)=O(logn*(n**(log_b(a)))=O(nlogn)
I've made two functions for computing the Fibonacci Sequence, one using recursion with memory and one using a loop;
def fib_rec(n, dic = {0 : 0, 1 : 1}):
if n in dic:
return dic[n]
else:
fib = fib_rec(n - 2, dic) + fib_rec(n - 1, dic)
dic[n] = fib
return fib
def fib_loop(n):
if n == 0 or n == 1:
return n
else:
smaller = 0
larger = 1
for i in range(1, n):
smaller, larger = larger, smaller + larger
return larger
I've heard that the Fibonacci Sequence often is solved using recursion, but I'm wondering why. Both my algorithms are of linear time complexity, but the one using a loop will not have to carry a dictionary of all past Fibonacci numbers, it also won't exceed Python's recursion depth.
Is this problem solved using recursion only to teach recursion or am I missing something?
The usual recursive O(N) Fibonacci implementation is more like this:
def fib(n, a=0, b=1):
if n == 0: return a
if n == 1: return b
return fib(n - 1, b, a + b)
The advantage with this approach (aside from the fact that it uses O(1) memory) is that it is tail-recursive: some compilers and/or runtimes can take advantage of that to secretly convert it to a simple JUMP instruction. This is called tail-call optimization.
Python, sadly, doesn't use this strategy, so it will use extra memory for the call stack, which as you noted quickly runs into Python's recursion depth limit.
The Fibonacci sequence is mostly a toy problem, used for teaching people how to write algorithms and about big Oh notation. It has elegant functional solutions as well as showing the strengths of dynamic programming (basically your dictionary-based solution), but it's also practically a solved problem.
We can also go a lot faster. The page https://www.nayuki.io/page/fast-fibonacci-algorithms describes how. It includes a fast doubling algorithm written in Python:
#
# Fast doubling Fibonacci algorithm (Python)
# by Project Nayuki, 2015. Public domain.
# https://www.nayuki.io/page/fast-fibonacci-algorithms
#
# (Public) Returns F(n).
def fibonacci(n):
if n < 0:
raise ValueError("Negative arguments not implemented")
return _fib(n)[0]
# (Private) Returns the tuple (F(n), F(n+1)).
def _fib(n):
if n == 0:
return (0, 1)
else:
a, b = _fib(n // 2)
c = a * (b * 2 - a)
d = a * a + b * b
if n % 2 == 0:
return (c, d)
else:
return (d, c + d)
I am studying Python by the book "a beginner guide to python 3" written by Mr.John Hunt. In chapter 8, which is about recursion, there is an exercise, that demands a code in which a prime number is found by recursion. I wrote first code below independently, but the answer key is written in different structure. Because I am very doubtful about recursion, What is your analysis about these two? Which is more recursive?
My code:
def is_prime(n, holder = 1):
if n == 2:
return True
else:
if (n-1 + holder)%(n-1) == 0:
return False
else:
return is_prime(n-1, holder+1)
print('is_prime(9):', is_prime(9))
print('is_prime(31):', is_prime(31))
Answer key:
def is_prime(n, i=2):
# Base cases
if n <= 2:
return True if (n == 2) else False
if n % i == 0:
return False
if i * i > n:
return True
# Check for next divisor
return is_prime(n, i + 1)
print('is_prime(9):', is_prime(9))
print('is_prime(31):', is_prime(31))
My suggestion in this case would be not to use recursion at all. Whilst I understand that you want to use this as a learning example of how to use recursion, it is also important to learn when to use recursion.
Recursion has a maximum allowed depth, because the deeper the recursion, the more items need to be put on the call stack. As such, this is not a good example to use recursion for, because it is easy to reach the maximum in this case. Even the "model" example code suffers from this. The exact maximum recursion depth may be implementation-dependent, but for example, if I try to use it to compute is_prime(1046527) then I get an error:
RecursionError: maximum recursion depth exceeded while calling a Python object
and inserting a print(i) statement shows that it is encountered when i=998.
A simple non-recursive equivalent of the "model" example will not have this problem. (There are more efficient solutions, but this one is trying to stay close to the model solution apart from not using recursion.)
def is_prime(n):
if n == 2:
return True
i = 2
while i * i <= n:
if n % i == 0:
return False
i += 1
return True
(In practice you would probably also want to handle n<2 cases.)
If you want a better example of a problem to practise recursive programming, check out the Tower of Hanoi problem. In this case, you will find that using recursion allows you to make a simpler and cleaner solution than is possible without it, while being unlikely to involve exceeding the maximum recursion depth (you are unlikely to need to consider a tower 1000 disks high, because the solution would require a vast number of moves, 2^1000-1 or about 10^301).
As another good example of where recursion can be usefully employed, try using turtle graphics to draw a Koch snowflake.
I'd say the Answer Key needs improvement. We can make it faster and handle the base cases more cleanly:
def is_prime(n, i=3):
# Base cases
if n < 2:
return False
if n % 2 == 0:
return n == 2
if i * i > n:
return True
if n % i == 0:
return False
# Check for next divisor
return is_prime(n, i + 2)
The original answer key starts at 2 and counts up by 1 -- here we start at 3 and count up by 2.
As far as your answer goes, there's a different flaw to consider. Python's default stack depth is 1,000 frames, and your function fails shortly above input of 1,000. The solution above uses recursion more sparingly and can handle input of up to nearly 4,000,000 before hitting up against Python's default stack limit.
Yes your example seems to work correctly. Note However, that by the nature of the implementation, the answer key is more efficient. To verify that a number n is a prime number, your algorithm uses a maximum of n-1 function calls, while the provided answer stops after reaching the iteration count of sqrt(n). Checking higher numbers makes generally no sense since if n is dividable without remainder by a value a > sqrt(n) it has to also be dividable by b = n % a.
Furthermore, your code raises an exception for evaluating at n = 1 since the modulo of 0 is not defined.
I have coded a greedy recursive algorithm to Find minimum number of coins that make a given change. Now I need to estimate its time complexity. As the algorithm has nested "ifs" depending on the same i (n * n), with the inner block halving the recursive call (log(2)n), I believe the correct answer could be O(n*log(n)), resulting from the following calculation:
n * log2(n) * O(1)
Please, give me your thoughts on whether my analysis is correct and feel free to also suggest improvements on my greedy recursive algorithm.
This is my recursive algorithm:
coins = [1, 5, 10, 21, 25]
coinsArraySize = len(coins)
change = 63
pickedCoins = []
def findMin(change, i, pickedCoins):
if (i>=0):
if (change >= coins[i]):
pickedCoins.append(coins[i])
findMin(change - coins[i], i, pickedCoins)
else:
findMin(change, i-1, pickedCoins)
findMin(change, coinsArraySize-1, pickedCoins)
Each recursive call decreases change by at least 1, and there is no branching (that is, your recursion tree is actually a straight line, so no recursion is actually necessary). Your running time is O(n).
What is n? The runtime depends on both the amount and the specific coins. For example, suppose you have a million coins, 1 through 1,000,000, and try to make change for 1. The code will go a million recursive levels deep before it finally finds the largest coin it can use (1). Same thing in the end if you have only one coin (1) and try to make change for 1,000,000 - then you find the coin at once, but go a million levels deep picking that coin a million times.
Here's a non-recursive version that improves on both of those: use binary search to find the next usable coin, and once a suitable coin is found use it as often as possible.
def makechange(amount, coins):
from bisect import bisect_right
# assumes `coins` is sorted. and that coins[0] > 0
right_bound = len(coins)
result = []
while amount > 0:
# Find largest coin <= amount.
i = bisect_right(coins, amount, 0, right_bound)
if not i:
raise ValueError("don't have a coin <=", amount)
coin = coins[i-1]
# How many of those can we use?
n, amount = divmod(amount, coin)
assert n >= 1
result.extend([coin] * n)
right_bound = i - 1
return result
It still takes O(amount) time if asked to make change for a million with the only coin being 1, but because it has to build a result list with a million copies of 1. If there are a million coins and you ask for change for 1, though, it's O(log2(len(coins))) time. The first could be slashed by changing the output format to a dict, mapping a coin to the number of times that coin is used. Then the first case would be cut to O(1) time.
As is, the time it takes is proportional to the length of the result list, plus some (usually trivial) time for a number of binary searches equal to the number of distinct coins used. So "a bad case" is one where every coin needs to be used; e.g.,
>>> coins = [2**i for i in range(10)]
>>> makechange(sum(coins), coins)
[512, 256, 128, 64, 32, 16, 8, 4, 2, 1]
That's essentially O(n + n log n) where n is len(coins).
Finding an optimal solution
As #Stef noted in a comment, the greedy algorithm doesn't always find a minimal number of coins. That's substantially harder. The usual approach is via dynamic programming, with worst case O(amt * len(coins)) time. But that's also best case: it works "bottom up", finding the smallest number of coins to reach 1, then 2, then 3, then 4, ..., and finally amt.
So I'm going to suggest a different approach, using breadth-first tree search, working down from the initial amount until reaching 0. Worst-case O() behavior is the same, but best-case time is far better. For the comment's:
mincoins(10000, [1, 2000, 3000])
case it looks at less than 20 nodes before finding the optimal 4-coin solution. Because it's a breadth-first search, it knows there's no possible shorter path to the root, so can stop right away then.
For a worst-case example, try
mincoins(1000001, range(2, 200, 2))
All the coins are even numbers, so it's impossible for any collection of them to sum to the odd target. The tree has to be expanded half a million levels deep before it realizes 0 is unreachable. But while the branching factor at high levels is O(len(coins)), the total number of nodes in the entire expanded tree is bounded above by amt + 1 (pigeonhole principle: the dict can't have more than amt + 1 keys, so any number of nodes beyond that are necessarily duplicate targets and so are discarded as soon as they're generated). So, in reality, the tree in this case grows wide very quickly, but then quickly becomes very narrow and very deep.
Also note that this approach makes it very easy to reconstruct the minimal collection of coins that sum to amt.
def mincoins(amt, coins):
from collections import deque
coins = sorted(set(coins)) # increasing, no duplicates
# Map amount to coin that was subtracted to reach it.
a2c = {amt : None}
d = deque([amt])
while d:
x = d.popleft()
for c in coins:
y = x - c
if y < 0:
break # all remaining coins too large
if y in a2c:
continue # already found cheapest way to get y
a2c[y] = c
d.append(y)
if not y: # done!
d.clear()
break
if 0 not in a2c:
raise ValueError("not possible", amt, coins)
picks = []
a = 0
while True:
c = a2c[a]
if c is None:
break
picks.append(c)
a += c
assert a == amt
return sorted(picks)
I've got, what I think is a valid solution to problem 2 of Project Euler (finding all even numbers in the Fibonacci sequence up to 4,000,000). This works for lower numbers, but crashes when I run it with 4,000,000. I understand that this is computationally difficult, but shouldn't it just take a long time to compute rather than crash? Or is there an issue in my code?
import functools
def fib(limit):
sequence = []
for i in range(limit):
if(i < 3):
sequence.append(i)
else:
sequence.append(sequence[i-1] + sequence[i-2])
return sequence
def add_even(x, y):
if(y % 2 == 0):
return x + y
return x + 0
print(functools.reduce(add_even,fib(4000000)))
The problem is about getting the Fibonacci numbers that are smaller than 4000000. Your code tries to find the first 4000000 Fibonacci values instead. Since Fibonacci numbers grow exponentially, this will reach numbers too large to fit in memory.
You need to change your function to stop when the last calculated value is more than 4000000.
Another possible improvement is to add the numbers as you are calculating them instead of storing them in a list, but this won't be necessary if you stop at the appropriate time.