Need help proving the time complexity of a recursive function.
Supposedly it's 2^n. I need to prove that this is the case.
def F(n):
if n == 0:
return 0
else:
result = 0
for i in range(n):
result += F(i)
return n*result+n`
Here's another version that does the same thing. The assignment said to use an array to store values in an attempt to reduce the time complexity so what I did was this
def F2(n,array):
if n < len(array):
answer = array[n]
elif n == 0:
answer = 0
array.append(answer)
else:
result = 0
for i in range(n):
result += F2(i,array)
answer = n*result+n
array.append(answer)
return answer
Again what I am looking for is the explanation of how to find the complexities of the two snippets of code, not interested in just knowing the answer.
All and any help greatly appreciated.
PS: for some reason, I can't get "def F2" to stay in the code block...sorry about that
Okay, the first function you wrote is an example of Exhaustive Search where you are exploring every possible branch that can be formed from a set of whole numbers up to n (which you have passed in the argument and you are using for loop for that). To explain you the time complexity I am going to consider the recursion stack as a Tree (to represent a recursive function call stack you can either use a stack or use an n-ary Tree)
Let's call you first function F1:
F1(3), now three branches will be formed for each number in the set S (set is the whole numbers up to n). I have taken n = 3, coz it will be easy for me to make the diagram for it. You can try will other larger numbers and observe the recursion call stack.
3
/| \
0 1 2 ----> the leftmost node is returns 0 coz (n==0) it's the base case
| /\
0 0 1
|
0 ----> returns 0
So here you have explored every possibility branches. If you try to write the recursive equation for the above problem then:
T(n) = 1; n is 0
= T(n-1) + T(n-2) + T(n-3) + ... + T(1); otherwise
Here,
T(n-1) = T(n-2) + T(n-3) + ... T(1).
So, T(n-1) + T(n-2) + T(n-3) + ... + T(1) = T(n-1) + T(n-1)
So, the Recursive equation becomes:
T(n) = 1; n is 0
= 2*T(n-1); otherwise
Now you can easily solve this recurrence relation (or use can use Masters theorem for the fast solution). You will get the time complexity as O(2^n).
Solving the recurrence relation:
T(n) = 2T(n-1)
= 2(2T(n-1-1) = 4T(n-2)
= 4(2T(n-3)
= 8T(n-3)
= 2^k T(n-k), for some integer `k` ----> equation 1
Now we are given the base case where n is 0, so let,
n-k = 0 , i.e. k = n;
Put k = n in equation 1,
T(n) = 2^n * T(n-n)
= 2^n * T(0)
= 2^n * 1; // as T(0) is 1
= 2^n
So, T.C = O(2^n)
So this is how you can get the time complexity for your first function. Next, if you observe the recursion Tree formed above (each node in the tree is a subproblem of the main problem), you will see that the nodes are repeating (i.e. the subproblems are repeating). So you have used a memory in your second function F2 to store the already computed value and whenever the sub-problems are occurring again (i.e. repeating subproblems) you are using the pre-computed value (this saves time for computing the sub-problems again and again). The approach is also known as Dynamic Programming.
Let's now see the second function, here you are returning answer. But, if you see your function you are building an array named as array in your program. The main time complexity goes there. Calculating its time complexity is simple because there is always just one level of recursion involved (or casually you can say no recursion involved) as every number i which is in range of number n is always going to be less than the number n, So the first if condition gets executed and control returns from there in F2. So each call can't go deeper than 2 level in the call stack.
So,
Time complexity of second function = time taken to build the array;
= 1 comparisions + 1 comparisions + 2 comparisions + ... + (n-1) comparisions
= 1 + 2 + 3 + ... + n-1
= O(n^2).
Let me give you a simple way to observe such recursions more deeply. You can print the recursion stack on the console and observe how the function calls are being made. Below I have written your code where I am printing the function calls.
Code:
def indent(n):
for i in xrange(n):
print ' '*i,
# second argument rec_cnt is just taken to print the indented function properly
def F(n, rec_cnt):
indent(rec_cnt)
print 'F(' + str(n) + ')'
if n == 0:
return 0
else:
result = 0
for i in range(n):
result += F(i, rec_cnt+1)
return n*result+n
# third argument is just taken to print the indented function properly
def F2(n, array, rec_cnt):
indent(rec_cnt)
print 'F2(' + str(n) + ')'
if n < len(array):
answer = array[n]
elif n == 0:
answer = 0
array.append(answer)
else:
result = 0
for i in range(n):
result += F2(i, array, rec_cnt+1)
answer = n*result+n
array.append(answer)
return answer
print F(4, 1)
lis = []
print F2(4, lis, 1)
Now observe the output:
F(4)
F(0)
F(1)
F(0)
F(2)
F(0)
F(1)
F(0)
F(3)
F(0)
F(1)
F(0)
F(2)
F(0)
F(1)
F(0)
96
F2(4)
F2(0)
F2(1)
F2(0)
F2(2)
F2(0)
F2(1)
F2(3)
F2(0)
F2(1)
F2(2)
96
In the first function call stack i.e. F1, you see that each call is explored up to 0, i.e. we are exploring each possible branch up to 0 (the base case), so, we call it Exhaustive Search.
In the second function call stack, you can see that the function calls are getting only two levels deep, i.e. they are using the pre-computed value to solve the repeated subproblems. Thus, it's time complexity is lesser than F1.
Related
Here is a code that finds the value of the function T given by the recurrence relation for a given value of n:
def T(n):
if n <= 0:
return 1
else:
return T(n-1) + (n-1) * T(n-2)
print T(3)
#output
2
However, the code invokes the function T twice. I have added the print statement before the second return to show this:
def T(n):
if n <= 0:
return 1
else:
print (T(n-1) + (n-1) * T(n-2))
return T(n-1) + (n-1) * T(n-2)
print T(3)
#output
1
2
1
2
Is there an efficient way of invoking the function T only once? I'm using Python 2. I'd like something short and simple, nothing too complicated.
You're going to call T 2^(n-1) times, for n >1, and there is no way to reduce that to only once without removing the recursion.
If you meant to say that you want to reduce the repetitive calls, such as T(3) calls T(2) and T(1) while T(2) also calls T(1), then you need to use a process called memoization
For example, using a dictionary of inputs to results.
seen = dict()
def T(n):
if n <= 0:
return 1
else:
if n in seen:
return seen[n]
else:
result = T(n-1) + (n-1) * T(n-2)
seen[n] = result
return result
print(T(3))
You can redefine the function from T(n) = T(n-1) + (n-1)*T(n-2) to T(n+1) = T(n) + n*T(n-1) which will allow the recursion to move forward rather than backward provided you give it the current n and previous values. By only needing the previous value instead of the two previous values, the recursive progression is easier to grasp.
def T(N,n=1,Tm=1,Tn=1): # n,Tm,Tn represent n,T(n-1),T(n)
if n>N: return Tm # stop at n=N+1 and return T(n-1)
return T(N,n+1,Tn,Tn+n*Tm)
note: the function returns a when n>N in order to cover the case where N is zero.
Actually, this could also have been written without transposing the function by moving the stop condition to N+2 instead of N+1:
def T(N,n=2,Tn_2=1,Tn_1=1): # n,Tn_2,Tn_1 represent n,T(n-2),T(n-1)
if n==N+2: return Tn_2 # stop at n=N+2 and return T(n-2)
return T(N,n+1,Tn_1,Tn_1+(n-1)*Tn_2)
which can be further simplified to:
def T(N,n=1,a=1,b=1): return T(N-1,n+1,b,a*n+b) if N else a
The first function:
def f1(n):
if n == 1:
return 1
return f(f(n-1))
The second function:
def f2(n):
if n == 1:
return 1
return 1 + f(f(n-1))
Now I can see why both of the function's space complexity is O(n) since the recursion depth is equal to n.
But about time complexity, I'm not being able to calculate it like I used to do with the equation for normal recursion, lets say instead of f(f(n-1)) we had f(n-1) in the first function. then it would be T(n) = 1 + T(n-1) = 2 + T(n-2)=... = n so O(n), I can intuitively understand that it might stay the same for f1, since all the returns are 1 so I might have 2n iterations which is O(n) but I have no idea how to deal it formally.
For f2 I couldn't know how to reach the time complexity, intuition failed me here, and I would really appreciate any help in how to analyze these recursive calls.
The Final answers are: f1: Time Complexity: O(n), Space Complexity: O(n).
f2: Time complexity: O(2^n), Space complexity: O(n).
As I think you realize, in f1 you're performing two operations:
Call f1(n - 1)
Call f1 again with the result from the first operation.
You can see that calling f1 with anything... or at least anything greater than zero... returns 1.
In step 2, you're just calling f1 again with 1, when returns immediately. O(N).
On to f2. Let's examine the behavior of this function.
When n = 1, we return 1.
When n = 2, we return 1 + f2(1). We know that f2(1) returns 1, so f2(2) returns 2.
When n = 3, we return 1 + f2(2), so 3.
Etc.
So it looks like f2(n) just returns n.
Every time we call f2 we are doing the same things we were doing in f1: call self with n - 1, then call self again with whatever that number is.
In other words, we're calling f2 twice with n - 1. For every n we double the number of calls, so O(2^N).
def fibonacci(n):
for i in range(n,1):
fab=0
if(i>1):
fab=fab+i
i=i-1
return fab
elif i==0:
return 0
else:
return 1
n1 = int(input("enter the nth term: "))
n2=fibonacci(n1)
print(n2)
The only way your code can return none is if you enter an invalid range, where the start value is greater than or equal to the stop value (1)
you probably just need range(n) instead of range(n, 1)
You can do this too:
def fibonacci(n):
return 0 if n == 1 else (1 if n == 2 else (fibonacci(n - 1) + fibonacci(n - 2) if n > 0 else None))
print(fibonacci(12))
You may need to use recursion for for nth Fibonacci number:
ex:
def Fibonacci(n):
if n==1:
return 0
elif n==2:
return 1
else:
return Fibonacci(n-1)+Fibonacci(n-2)
print(Fibonacci(9))
# output:21
If you do not plan to use large numbers, you can use the easy and simple typical recursive way of programming this function, although it may be slow for big numbers (>25 is noticeable), so take it into account.
def fibonacci(n):
if n<=0:
return 0
if n==1:
return 1
return fibonacci(n-1)+fibonacci(n-2)
You can also add a cache for the numbers you already stepped in, in order to make it run much faster. It will consume a very small amount of extra memory but it allows you to calculate larger numbers instantaneously (you can test it with fibonacci(1000), which is almost the last number you can calculate due to recursion limitation)
cache_fib = {}
def fibonacci(n):
if n<=0:
return 0
if n==1:
return 1
if n in cache_fib.keys():
return cache_fib[n]
result = fibonacci(n-1)+fibonacci(n-2)
cache_fib[n] = result
return result
In case you really need big numbers, you can do this trick to allow more recursion levels:
cache_fib = {1:1}
def fibonacci(n):
if n<=0:
return 0
if n in cache_fib.keys():
return cache_fib[n]
max_cached = max(cache_fib.keys())
if n-max_cached>500:
print("max_cached:", max_cached)
fibonacci(max_cached+500)
result = fibonacci(n-1)+fibonacci(n-2)
cache_fib[n] = result
return result
range(n,1) creates a range starting with n, incrementing in steps of 1 and stopping when n is larger or equal to 1. So, in case n is negative or zero, your loop will be executed. But in case n is 1 or larger, the loop will never be executed and the function just returns None.
If you would like a range going from n downwards to 1, you can use range(n,1,-1) where -1 is the step value. Note that when stepping down, the last number is included range(5,1,-1) is [5,4,3,2,1] while when stepping upwards range(1,5) is [1,2,3,4] the last element is not included. range(n) with only one parameter also exists. It is equivalent to range(0, n) and goes from 0 till n-1, which means the loop would be executed exactly n times.
Also note that you write return in every clause of the if statement. That makes that your function returns its value and interrupts the for loop.
Further, note that you set fab=0 at the start of your for loop. This makes that it is set again and again to 0 each time you do a pass of the loop. Therefore, it is better to put fab=0 just before the start of the loop.
As others have mentioned, even with these changes, your function will not calculate the Fibonacci numbers. A recursive function is a simple though inefficient solution. Some fancy playing with two variables can calculate Fibonacci in a for loop. Another interesting approach is memorization or caching as demonstrated by #Ganathor.
Here is a solution that without recursion and without caching. Note that Fibonacci is a very special case where this works. Recursion and caching are very useful tools for more real world problems.
def fibonacci(n):
a = 0
b = 1
for i in range(n):
a, b = a + b, a # note that a and b get new values simultaneously
return a
print (fibonacci(100000))
And if you want a really, really fast and fancy code:
def fibonacci_fast(n):
a = 1
b = 0
p = 0
q = 1
while n > 0 :
if n % 2 == 0 :
p, q = p*p + q*q, 2*p*q + q*q
n = n // 2
else:
a, b = b*q + a*q + a*p, b*p + a*q
n = n - 1
return b
print (fibonacci_fast(1000000))
Note that this relies on some special properties of the Fibonacci sequence. It also gets slow for Python to do calculations with really large numbers. The millionth Fibonacci number has more than 200,000 digits.
I am stuck on this Big O notation on how it is supposed to be O(n^3). Where did my thought process go wrong?
I know that a nested for loop is O(n^2) and that the while loop is probably a O(nlogn) function because the for loop is a O(n) function and the value for the while loop is being multiplied by two which makes it O(logn). That being said, the answer is stated to be O(n^3) and I'm confused how this came to be unless the recursive part of the function has something to do with it?
def do_stuff2(n, x=1.23):
if n <= 0:
return 0
val = 1
for i in range(n//2):
for j in range(n//4):
x += 2*x + j/2 + i*1.2
while val <= n:
for i in range(n):
x += val**2 + i//2
val *= 2
x += do_stuff2(n - 1, x/2)
return x
I believe that the x does not affect the big o notation because it is a constant because it is not used in deciding how many times any of the loops loop.
So again, I expected the output of the function to be O(n^2), but the actual output is O(n^3)
Your function has two nested for loops, that's O(n^2):
for i in range(n//2):
for j in range(n//4):
x += 2*x + j/2 + i*1.2
But on top of that, your do_stuff2() function takes an argument n and calls itself until n <= 0, meaning that's one more O(n).
I'm working on solving the Project Euler problem 25:
What is the first term in the Fibonacci sequence to contain 1000
digits?
My piece of code works for smaller digits, but when I try a 1000 digits, i get the error:
OverflowError: (34, 'Result too large')
I'm thinking it may be on how I compute the fibonacci numbers, but i've tried several different methods, yet i get the same error.
Here's my code:
'''
What is the first term in the Fibonacci sequence to contain 1000 digits
'''
def fibonacci(n):
phi = (1 + pow(5, 0.5))/2 #Golden Ratio
return int((pow(phi, n) - pow(-phi, -n))/pow(5, 0.5)) #Formula: http://bit.ly/qDumIg
n = 0
while len(str(fibonacci(n))) < 1000:
n += 1
print n
Do you know what may the cause of this problem and how i could alter my code avoid this problem?
Thanks in advance.
The problem here is that only integers in Python have unlimited length, floating point values are still calculated using normal IEEE types which has a maximum precision.
As such, since you're using an approximation, using floating point calculations, you will get that problem eventually.
Instead, try calculating the Fibonacci sequence the normal way, one number (of the sequence) at a time, until you get to 1000 digits.
ie. calculate 1, 1, 2, 3, 5, 8, 13, 21, 34, etc.
By "normal way" I mean this:
/ 1 , n < 3
Fib(n) = |
\ Fib(n-2) + Fib(n-1) , n >= 3
Note that the "obvious" approach given the above formulas is wrong for this particular problem, so I'll post the code for the wrong approach just to make sure you don't waste time on that:
def fib(n):
if n <= 3:
return 1
else:
return fib(n-2) + fib(n-1)
n = 1
while True:
f = fib(n)
if len(str(f)) >= 1000:
print("#%d: %d" % (n, f))
exit()
n += 1
On my machine, the above code starts going really slow at around the 30th fibonacci number, which is still only 6 digits long.
I modified the above recursive approach to output the number of calls to the fib function for each number, and here are some values:
#1: 1
#10: 67
#20: 8361
#30: 1028457
#40: 126491971
I can reveal that the first Fibonacci number with 1000 digits or more is the 4782th number in the sequence (unless I miscalculated), and so the number of calls to the fib function in a recursive approach will be this number:
1322674645678488041058897524122997677251644370815418243017081997189365809170617080397240798694660940801306561333081985620826547131665853835988797427277436460008943552826302292637818371178869541946923675172160637882073812751617637975578859252434733232523159781720738111111789465039097802080315208597093485915332193691618926042255999185137115272769380924184682248184802491822233335279409301171526953109189313629293841597087510083986945111011402314286581478579689377521790151499066261906574161869200410684653808796432685809284286820053164879192557959922333112075826828349513158137604336674826721837135875890203904247933489561158950800113876836884059588285713810502973052057892127879455668391150708346800909439629659013173202984026200937561704281672042219641720514989818775239313026728787980474579564685426847905299010548673623281580547481750413205269166454195584292461766536845931986460985315260676689935535552432994592033224633385680958613360375475217820675316245314150525244440638913595353267694721961
And that is just for the 4782th number. The actual value is the sum of all those values for all the fibonacci numbers from 1 up to 4782. There is no way this will ever complete.
In fact, if we would give the code 1 year of running time (simplified as 365 days), and assuming that the machine could make 10.000.000.000 calls every second, the algorithm would get as far as to the 83rd number, which is still only 18 digits long.
Actually, althought the advice given above to avoid floating-point numbers is generally good advice for Project Euler problems, in this case it is incorrect. Fibonacci numbers can be computed by the formula F_n = phi^n / sqrt(5), so that the first fibonacci number greater than a thousand digits can be computed as 10^999 < phi^n / sqrt(5). Taking the logarithm to base ten of both sides -- recall that sqrt(5) is the same as 5^(1/2) -- gives 999 < n log_10(phi) - 1/2 log_10(5), and solving for n gives (999 + 1/2 log_10(5)) / log_10(phi) < n. The left-hand side of that equation evaluates to 4781.85927, so the smallest n that gives a thousand digits is 4782.
You can use the sliding window trick to compute the terms of the Fibonacci sequence iteratively, rather than using the closed form (or doing it recursively as it's normally defined).
The Python version for finding fib(n) is as follows:
def fib(n):
a = 1
b = 1
for i in range(2, n):
b = a + b
a = b - a
return b
This works when F(1) is defined as 1, as it is in Project Euler 25.
I won't give the exact solution to the problem here, but the code above can be reworked so it keeps track of n until a sentry value (10**999) is reached.
An iterative solution such as this one has no trouble executing. I get the answer in less than a second.
def fibonacci():
current = 0
previous = 1
while True:
temp = current
current = current + previous
previous = temp
yield current
def main():
for index, element in enumerate(fibonacci()):
if len(str(element)) >= 1000:
answer = index + 1 #starts from 0
break
print(answer)
import math as m
import time
start = time.time()
fib0 = 0
fib1 = 1
n = 0
k = 0
count = 1
while k<1000 :
n = fib0 + fib1
k = int(m.log10(n))+1
fib0 = fib1
fib1 = n
count += 1
print n
print count
print time.time()-start
takes 0.005388 s on my pc. did nothing fancy just followed simple code.
Iteration will always be better. Recursion was taking to long for me as well.
Also used a math function for calculating the number of digits in a number instead of taking the number in a list and iterating through it. Saves a lot of time
Here is my very simple solution
list = [1,1,2]
for i in range(2,5000):
if len(str(list[i]+list[i-1])) == 1000:
print (i + 2)
break
else:
list.append(list[i]+list[i-1])
This is sort of a "rogue" way of doing it, but if you change the 1000 to any number except one, it gets it right.
You can use the datatype Decimal. This is a little bit slower but you will be able to have arbitrary precision.
So your code:
'''
What is the first term in the Fibonacci sequence to contain 1000 digits
'''
from Decimal import *
def fibonacci(n):
phi = (Decimal(1) + pow(Decimal(5), Decimal(0.5))) / 2 #Golden Ratio
return int((pow(phi, Decimal(n))) - pow(-phi, Decimal(-n)))/pow(Decimal(5), Decimal(0.5)))
n = 0
while len(str(fibonacci(n))) < 1000:
n += 1
print n