I am stumbling at anlayzing the time and space complexities of recursive functions:
consider:
def power(a, n):
if n==0:
return 1
else:
return a*power(a, n-1)
When finding time complexity of this: I think T(n) = c + T(n-1) where c is the constant cost of the multiplication.
This will probably lead to: c*n cost, i.e. linear cost O(n). But recursions are usually exponential in cost.
Also, consider this function:
def power(a,n):
if n==0:
return 1
if n%2 ==0:
return power(a*a, n//2)
else:
return a*power(a*a, n//2)
The above function will go on till: T(n) = c + T(n/2) which means the cost will be c*log(n) means log(n) complexity.
If the analysis is correct then recursion looks to be as fast as iterative algorithms, so where does the overhead come from and are there any exponential recursive algorithms?
It is not true that recursions are exponential in complexity. In fact there is a theorem that each recursive algorithm has an iterative analogue and vice versa(possibly using additional memory). For explanation on how to do this see here for instance. Also have a look at the section in wikipedia that compares recursion and iteration.
When a recursive function calls itself more than once in some of its flows, you may end up with exponential complexity as is the famous example with fibonacci numbers:
def fib(n):
if n < 2:
return 1
return fib(n - 1) + fib(n - 2)
But this does not mean there is no faster recursive implementation. For instance using memoization you can get that down to linear complexity.
Still recursive implementations really are a bit slower, because the stack frame should be stored when doing a recursive call and it should be restored when returning the value.
Related
I have the following recursive Python function:
def f(n):
if n <=2:
result = n
else:
result = f(n-2) + f(n-3)
return result
What would you say is the algorithmic complexity of it?
I was thinking this function is really similar to the Fibonacci recursive function which has a complexity of O(2^N) but I am not sure if that would be correct for this specific case.
Just write the complexity function.
Assuming all atomic operations here worth 1 (they are O(1), so, even if it is false to say that they are all equal, as long as we just need the big-O of our complexity, at the end, it is equivalent to the reality), the complexity function is
def complexity(n):
if n<=2:
return 1
else:
return 1 + complexity(n-2) + complexity(n-3)
So, complexity of f computation is almost the same thing as f!
Which is not surprising: the only possible direct return value for f is 2. All other values are sum of other f. And in absence of any mechanism to avoid to recompute redundant value, you can say that if f(1000) = 2354964531714, then it took 2354964531714/2 addition of f(?)=2 to get to that result.
So, number of additions to compute f(n) is O(f(n))
And since f(n) is O(1.32ⁿ) (f(n)/1.32ⁿ → ∞, while f(n)/1.33ⁿ → 0. So it is exponential, with a base somewhere in between 1.32 and 1.33), it is an exponential complexity.
Let's suppose, guessing wildly, that T(n) ~ rT(n-1) for large values of n. Then our recurrence relation...
T(n) = c + T(n-2) + T(n-3)
Can be rewritten...
r^3T(n-3) ~ c + rT(n-3) + T(n-3)
Letting x = T(n-3), we have
(r^3)x ~ c + rx + x
Rearranging, we get
x(r^3 - r - 1) ~ c
And finally:
r^3 - r - 1 ~ c/x
Assuming again that x is very large, c/x ~ 0, so
r^3 - r - 1 ~ 0
I didn't have much look solving this analytically, however, as chrslg finds and Wolfram Alpha confirms, this has a root around ~1.32-1.33, and so there is real value of r that works; and so the time complexity is bounded by an exponential function with that base. If I am able to find an analytical solution, I will post with details (or if anybody else can do it, please leave it here.)
def f1(seq,thelast):
for i in range(0,thelast):
print(seq[i])
def f2(seq,thefirst,thelast):
if thefirst==thelast:
f1(seq,thelast)
else:
for i in range(thefirst,thelast):
temp= seq[thefirst]
seq[thefirst]=seq[i]
seq[i]=temp
f2(seq, thefirst+1, thelast)
temp=seq[thefirst]
seq[thefirst]=seq[i]
seq[i]=temp
I have thought it has a for loop and a recursion and found it has complexity of O(n) but apperently it does not have this complexity. Am I missing something?
You should solve the corresponding recurrence relation:
T(n) = n T(n-1) + c for n>1 (with T(1) = n)
Indeed you have n recursive calls with decreased input n. By iteration, you find it is O(n*n!).
If you try simulating the code, you'll see it prints all permutations of seq that are n!. Printing the sequence has a cost of n. Hence you get O(n*n!). You can verify it also by drawing the recursion tree.
Suppose we have the following functions:
def modulo(a, b):
"""Take numbers a and b and compute a % b."""
if b <= 0:
return None
div = int(a / b)
return a - div*b
def power(a, b):
"""Take number a and non-negative integer b.
Compute a**b."""
if b == 0:
return 1
else:
return a * power(a, b - 1)
For modulo, is the big-O notation O(a / b)? Because the time it takes to execute depends on a and b inputs. However, I've seen modulus computations online where the big-O notation is O(logn), so not sure if that's applicable here. Can anyone clarify?
For power, would the big-O notation be O(a^b) because it involves powers?
Following the previous answered question
Time complexity is calculated for repetitive tasks. Therefore, if you don't have loops in general, you don't talk about time complexity. You are just applying one thing.
So for modulo you can 'ignore it' (O(c)) c is a constant.
For the recursive function, since we are re-entering in the function each time, until b==0, then the complexity is O(b) similar to O(n) from the previous question. (linear)
If you google Recursion it will show you did you mean recursion and if you keep clicking it, it keeps taking you to the same page, therefore, recursion is a repetitive task. That is why it you consider it similar to a loop.
modulo has no loop, it just does constant time operations (math), so it's O(1). power will decrease b by 1 until it's 0, and the rest of the function is O(1), so it's O(1*b) or O(b).
I have this function:
def rec(lst):
n = len(lst)
if n <= 1:
return 1
return rec(lst[n // 2:]) + rec(lst[:n // 2])
How can I find the time complexity of this function?
Usually in such problems drawing the recursion tree helps.
Look at this photo I added, note how each level sums up to N (since slicing is the thing here doing the work),
and the depth of the tree is logN (this is easy to show, since we divide by 2 each time, you can find an explanation here). So what we have is the function doing O(n) n*logn times which means in total we have O(n*logn).
Now another way of understanding this is using the "Master Theorem" (I encourage you to look it up and learn about it)
We have here T(n) = 2T(n/2) + O(n), so according to the theorem a=2, b=2 so log_b(a) is equal to 1, and therefore
we have (according to the 2nd case of the theorem):
T(n)=O(logn*(n**(log_b(a)))=O(nlogn)
I have two functions fib1 and fib2 to calculate Fibonacci.
def fib1(n):
if n < 2:
return 1
else:
return fib1(n-1) + fib1(n-2)
def fib2(n):
def fib2h(s, c, n):
if n < 1:
return s
else:
return fib2h(c, s + c, n-1)
return fib2h(1, 1, n)
fib2 works fine until it blows up the recursion limit. If understand correctly, Python doesn't optimize for tail recursion. That is fine by me.
What gets me is fib1 starts to slow down to a halt even with very small values of n. Why is that happening? How come it doesn't hit the recursion limit before it gets sluggish?
Basically, you are wasting lots of time by computing the fib1 for the same values of n over and over. You can easily memoize the function like this
def fib1(n, memo={}):
if n in memo:
return memo[n]
if n < 2:
memo[n] = 1
else:
memo[n] = fib1(n-1) + fib1(n-2)
return memo[n]
You'll notice that I am using an empty dict as a default argument. This is usually a bad idea because the same dict is used as the default for every function call.
Here I am taking advantage of that by using it to memoize each result I calculate
You can also prime the memo with 0 and 1 to avoid needing the n < 2 test
def fib1(n, memo={0: 1, 1: 1}):
if n in memo:
return memo[n]
else:
memo[n] = fib1(n-1) + fib1(n-2)
return memo[n]
Which becomes
def fib1(n, memo={0: 1, 1: 1}):
return memo.setdefault(n, memo.get(n) or fib1(n-1) + fib1(n-2))
Your problem isn't python, it's your algorithm. fib1 is a good example of tree recursion. You can find a proof here on stackoverflow that this particular algorithm is (~θ(1.6n)).
n=30 (apparently from the comments) takes about a third of a second. If computational time scales up as 1.6^n, we'd expect n=100 to take about 2.1 million years.
Think of the recursion trees in each. The second version is a single branch of recursion with the addition taking place in the parameter calculations for the function calls, and then it returns the values back up. As you have noted, Python doesn't require tail recursion optimization, but if tail call optimization were a part of your interpreter, the tail recursion optimization could be triggered as well.
The first version, on the other hand, requires 2 recursion branches at EACH level! So the number of potential executions of the function skyrockets considerably. Not only that, but most of the work is repeated twice! Consider: fib1(n-1) eventually calls fib1(n-1) again, which is the same as calling fib1(n-2) from the point of reference of the first call frame. But after that value is calculated, it must be added to the value of fib1(n-2) again! So the work is needlessly duplicated many times.