def mystery11(n):
if n < 1: return n
def mystery12(n):
i = 1
while i < n:
i *= 2
return i
return mystery11(n/2) + mystery11(n/2) + mystery12(n-2)
I have a question about the code above. I completely understand that without the last recursive call to mystery12, the runtime of the code (mystery11) would be theta(n). But I don't believe that at each level theta(log(n)) work is being done.
At the first level, we do log(n), at the next level, we do 2log(n/2), then 4log(n/4)... but that doesn't look like log(n) at each level (it feels closer to 2log(n) at the second level and 4log(n) at the third level etc.)
I've also tried Wolfram Alpha, and I just get no solutions exist.
But works fine without the log(n) term.
So, is this solution correct theta(nlog(n))? And if not, what is the actual solution?
Ps. Apologies if anything in my post is not etiquette, this is my second time posting on Stackoverflow. Post a comment and I will fix it.
Since mystery12 is independent of any external functions / variables, let's look at it first.
After the j-th iteration of the while-loop, i = 2^j. Therefore the maximum number of loops is given by the equation 2^j >= n, or j = ceil(log2(n)) where ceil rounds up to the nearest integer.
Now for mystery11. Each call contains 2 recursive calls to mystery11 with argument n / 2, and a call to mystery12 with argument n - 2. This gives the time complexity recurrence relation (C is some positive constant):
You have already correctly deduced that the recursion depth is m ~ log n. The precise value is m = ceil(log2(n)). Using the fact that a number rounded up only differs from itself by less than 1, we can eliminate some of the rounding brackets:
Let's examine B first. A problem arises - the expression inside the logarithm can be negative. This is mitigated by the fact that the while-loop mystery12 never executes if n - 2 < 1 - i.e. its complexity can be truncated to O(1) in this edge case. Therefore, the sum is bounded from above by:
Where we used a Taylor expansion of log. Hence B can be ignored in our analysis as it is already overshadowed by the first term.
Now to examine A. This is a slightly tedious summation which I will use Wolfram Alpha to compute:
Therefore the overal time complexity of mystery11 is Θ(n), not Θ(n log n) as predicted.
Why is this? The reason lies in "each recursive call does log n work" - n here is not the same as the initial value of n passed to mystery11 (the n in the overall time complexity). At each recursion level n decreases exponentially, so:
We cannot naively multiply the amount of work done per call with the number of recursive calls.
This applies to complexity analysis in general.
I'm sorry I couldn't understand what your question is.
It seems not clear to me.
Whatever it is,
I did the math according to your 'mystery' code.
Let's say 'm1' is the mystery11, 'm2' is the mystery12.
Without m2,
time cost would be like this.
m1(n)
= 2*m1(n/2)
= 2*( m1(n/4) + m1(n/4) ) = 4*m1(n/4)
= .....
= 2^k * m1( n / 2^k )
For the k which makes 2^k n,
2^k = n.
Then time cost for m1(n) is n * m1(1) = n.
Time cost of m2 is log(n) obviously.
With m2,
time cost changes like this.
m1(n)
= 2*m1(n/2) + log(n)
= 2*( m1(n/4) + m1(n/4) + log(n) ) + log(n) = 4*m1(n/4) + ( log(n) + 2log(n) )
= ....
= 2^k * m1(n/2^k) + ( log(n) + 2log(n) + 4log(n) ... 2^(k-1)*log(n) )
= 2^k * m1(n/2^k) + log(n) * ∑( 2^(k-1) ) ( where k is from 1 to k )
= 2^k * m1(n/2^k) + log(n)/2 * ∑( 2^k )
= 2^k * m1(n/2^k) + log(n)/2 * ( 2^(k+1) - 1 )
= 2^k * m1(n/2^k) + 2^k * log(n) - log(n)/2
Just like the previous one,
for the k which makes 2^k n,
2^k * m1(n/2^k) + 2^k * log(n) - log(n)/2
= n * m1(1) + n*log(n) - log(n)/2 = n + n*log(n) - log(n)/2
I believe you can do the rest from here.
Ps. My apologies if it's not what you asked for.
Related
I am calculating the run time complexity of the below Python code as n^2 but it seems that it is incorrect. The correct answer shown is n(n-1)/2. Can anyone help help me in understanding why the inner loop is not running n*n times but rather n(n-1)/2?
for i in range(n):
for j in range(i):
val += 1
On the first run of the inner loop, when i = 0, the for loop does not execute at all. (There are zero elements to iterate over in range(0)).
On the second run of the inner loop, when i = 1, the for loop executes for one iteration. (There is one element to iterate over in range(1)).
Then, there are two elements in the third run, three in the fourth, following this pattern, up until i = n - 1.
The total number of iterations is 0 + 1 + ... + (n - 1). This summation has a well known form*: for any natural m, 0 + 1 + ... + m = m * (m + 1) / 2. In this case, we have m = n - 1, so there are n * (n - 1) / 2 iterations in total.
You're right that the asymptotic time complexity of this algorithm is O(n^2). However, given that you've said that the "correct answer" is n * (n - 1) / 2, the question most likely asked for the exact number of iterations (rather than an asymptotic bound).
* See here for a proof of this closed form.
def s_r(n):
if(n==1):
return 1
temp = 1
for i in range(int(n)):
temp += i
return temp * s_r(n/2) * s_r(n/2) * s_r(n/2) * s_r(n/2)
Using recursion tree what’s the Big Oh
And also how do we write this function into one recursive call.
I could only do the first part which I got O(n^2). This was one my exam questions which I want to know the answer and guidance to. Thank you
First, note that the program is incorrect: n/2 is floating point division in python (since Python3), so the program does not terminate unless n is a power of 2 (or eventually rounds to a power of 2). The corrected version of the program that works for all integer n>=1, is this:
def s_r(n):
if(n==1):
return 1
temp = 1
for i in range(n):
temp += i
return temp * s_r(n//2) * s_r(n//2) * s_r(n//2) * s_r(n//2)
If T(n) is the number of arithmetic operations performed by the function, then we have the recurrence relation:
T(1) = 0
T(n) = n + 4T(n//2)
T(n) is Theta(n^2) -- for n a power of 2 and telescoping we get: n + 4(n/2) + 16(n/4) + ... = 1n + 2n + 4n + 8n + ... + nn = (2n-1)n
We can rewrite the program to use Theta(log n) arithmetic operations. First, the temp variable is 1 + 0+1+...+(n-1) = n(n-1)/2 + 1. Second, we can avoid making the same recursive call 4 times.
def s_r(n):
return (1 + n * (n-1) // 2) * s_r(n//2) ** 4 if n > 1 else 1
Back to complexity, I have been careful to say that the first function does Theta(n^2) arithmetic operations and the second Theta(log n) arithmetic operations, rather than use the expression "time complexity". The result of the function grows FAST, so it's not practical to assume arithmetic operations run in O(1) time. If we print the length of the result and the time taken to compute it (using the second, faster version of the code) for powers of 2 using this...
import math
import timeit
def s_r(n):
return (1 + n * (n-1) // 2) * s_r(n//2) ** 4 if n > 1 else 1
for i in range(16):
n = 2 ** i
start = timeit.default_timer()
r = s_r(n)
end = timeit.default_timer()
print('n=2**%d,' % i, 'digits=%d,' % (int(math.log(r))+1), 'time=%.3gsec' % (end - start))
... we get this table, in which you can see the number of digits in the result grows FAST (s_r(2**14) has 101 million digits for example), and the time measured does not grow with log(n), but when n is doubled the time increases by something like a factor of 10, so it grows something like n^3 or n^4.
n=2**0, digits=1, time=6e-07sec
n=2**1, digits=1, time=1.5e-06sec
n=2**2, digits=5, time=9e-07sec
n=2**3, digits=23, time=1.1e-06sec
n=2**4, digits=94, time=2.1e-06sec
n=2**5, digits=382, time=2.6e-06sec
n=2**6, digits=1533, time=3.8e-06sec
n=2**7, digits=6140, time=3.99e-05sec
n=2**8, digits=24569, time=0.000105sec
n=2**9, digits=98286, time=0.000835sec
n=2**10, digits=393154, time=0.00668sec
n=2**11, digits=1572628, time=0.0592sec
n=2**12, digits=6290527, time=0.516sec
n=2**13, digits=25162123, time=4.69sec
n=2**14, digits=100648510, time=42.1sec
n=2**15, digits=402594059, time=377sec
Note that it's not wrong to say the time complexity of the original function is O(n^2) and the improved version of the code O(log n), it's just that these describe a measure of the program (arithmetic operations) and are not at all useful as an estimate of actual program running time.
As #kcsquared said in comments I also believe this function is O(n²). This code can be refactored into one function call by storing the result of recursion call, or just doing some math application. Also you can simplify the range sum, by using the built-in sum
def s_r(n):
if n == 1:
return 1
return (sum(range(int(n))) + 1) * s_r(n/2) ** 4
I'm working on a DP problem called house robber, I've solved the problem using a DP approach but my initial thought was to use the following recursive function:
def rec(self, start, possible):
if len(possible) == 0:
return start
money = start
for i, num in enumerate(possible):
nextMoney = self.rec(start + num, possible[i+2:])
money = max(money, nextMoney)
return money
def rob(self, nums: List[int]) -> int:
# Base Case
if(len(nums) == 0):
return 0
elif len(nums) == 1:
return nums[0]
path1 = self.rec(nums[0], nums[2:])
path2 = self.rec(nums[1], nums[3:])
# Recursion
return max(path1, path2)
My DP solution is O(n) but I'm struggling to determine the time complexity of the algorithm described above. My instinct says its of exponential order to the log(n) = O(n ^ log (n))
If anyone can point me in the right direction here that would be highly appreciated. Thanks.
Problem for reference: https://leetcode.com/problems/house-robber/
The code enumerates all subsets of 1..n with no two adjacent numbers. You do lots of slicing of possible which creates an O(n^2) cost per call, so the recurrence relation is:
T(n) = n^2 + sum(T(i) for i=0..n-2)
Subtracting T(n) from T(n-1):
T(n) - T(n-1) = n^2 - (n-1)^2 + T(n-2)
T(n) = 2n - 1 + T(n-1) + T(n-2)
Let U(n) = T(n) + 2n + 5, so T(n) = U(n) - 2n - 5. Substituting for T(n), T(n-1) and T(n-2) we get:
U(n) - 2n - 5 = 2n - 1 + U(n-1) - 2(n-1) - 5 + U(n-2) - 2(n-2) - 5
U(n) = U(n-1) + U(n-2) (simplifying)
so U(n) = Fib(n) (ie: fibonacci numbers), and T(n) = Fib(n) - 2n - 5.
So your runtime is Theta(Fib(n)), which is Theta(phi^n), where phi is the golden ratio.
[An interesting note is that if you removed the list slicing that causes the O(n^2) cost per call, the complexity class of your code would be the same -- the cost of slicing is lost in the otherwise exponential cost of the code].
I have two quantities a & b that are defined by recursion and through reference to another list of values x = [ x_1, x_2, ... x_N ], which will be an input to the program. The program will iterate over all the values in x and update a & b according to:
for n in range(1,N)
a[n] = a[n-1] * exp(+x[n]) + b[n-1] * exp(-x[n])
b[n] = b[n-1] * exp(+x[n]) + a[n-1] * exp(-x[n])
and starting values
a[0] = exp(+x[0])
b[0] = exp(-x[0])
The values in x are not big numbers (always <10) but N will be in the hundreds, and because of all the exponentials the final values of a & b will be very large. I'm worried that because of the form of the recursion where we are constantly multiplying exponentially large numbers with exponentially small ones and adding them this scheme will become quite numerically unstable.
Ideally I would calculate log(a) and log(b) instead to stop the values becoming too large. But because of the recursion scheme that's not possible, unless I compute something much messier like
log_a[n] = x[n] + log_a[n-1] + log( 1 + exp(-2*x[n] + log_b[n-1]-log_a[n-1] ) )
Is numerical stability something I am right to be concerned about here? And if so would something like the log based scheme help to stabilise it?
We can rewrite that first as:
for n in range(1,N)
a[n] = exp(log(a[n-1]) + x[n]) + exp(log(b[n-1]) - x[n])
b[n] = exp(log(b[n-1]) + x[n]) + exp(log(a[n-1]) - x[n]))
Then change our iteration variables:
for n in range(1,N)
log_a[n] = log(exp(log_a[n-1] + x[n]) + exp(log_b[n-1] - x[n]))
log_b[n] = log(exp(log_b[n-1] + x[n]) + exp(log_a[n-1] - x[n]))
Which can be computed more stably using np.logaddexp:
for n in range(1,N)
log_a[n] = np.logaddexp(log_a[n-1] + x[n], log_b[n-1] - x[n])
log_b[n] = np.logaddexp(log_b[n-1] + x[n], log_a[n-1] - x[n])
The implementation of logaddexp can be seen here
As far as I'm aware, all(?) recursive problems can be solved through dynamic programming. For example, the Fibonacci sequence could be expressed like so:
def fibo(n):
if n == 0:
return 0
elif n == 1:
return 1
return fibo(n-1) + fibo(n-2)
Or, iteratively:
n = 10
fibo_nums = [0, 1]
while len(fibo_nums) <= n:
fibo_nums.append(fibo_nums[-2] + fibo_nums[-1])
Presumably if you have a recursive problem you could perform a similar unpacking.
In Wikipedia this is one of the given algorithms to generate prime numbers:
def eratosthenes_sieve(n):
# Create a candidate list within which non-primes will be
# marked as None; only candidates below sqrt(n) need be checked.
candidates = [i for i in range(n + 1)]
fin = int(n ** 0.5)
# Loop over the candidates, marking out each multiple.
for i in range(2, fin + 1):
if not candidates[i]:
continue
candidates[i + i::i] = [None] * (n // i - 1)
# Filter out non-primes and return the list.
return [i for i in candidates[2:] if i]
I changed the algorithm slightly.
def eratosthenes_sieve(n):
# Create a candidate list within which non-primes will be
# marked as None; only candidates below sqrt(n) need be checked.
candidates = [i for i in range(n + 1)]
fin = int(n ** 0.5)
# Loop over the candidates, marking out each multiple.
candidates[4::2] = [None] * (n // 2 - 1)
for i in range(3, fin + 1, 2):
if not candidates[i]:
continue
candidates[i + i::i] = [None] * (n // i - 1)
# Filter out non-primes and return the list.
return [i for i in candidates[2:] if i]
I first marked off all the multiples of 2, and then I considered odd numbers only. When I timed both algorithms (tried 40.000.000) the first one was always better (albeit very slightly). I don't understand why. Can somebody please explain?
P.S.: When I try 100.000.000, my computer freezes. Why is that? I have Core Duo E8500, 4GB RAM, Windows 7 Pro 64 Bit.
Update 1: This is Python 3.
Update 2: This is how I timed:
start = time.time()
a = eratosthenes_sieve(40000000)
end = time.time()
print(end - start)
UPDATE: Upon valuable comments (especially by nightcracker and Winston Ewert) I managed to code what I intended in the first place:
def eratosthenes_sieve(n):
# Create a candidate list within which non-primes will be
# marked as None; only c below sqrt(n) need be checked.
c = [i for i in range(3, n + 1, 2)]
fin = int(n ** 0.5) // 2
# Loop over the c, marking out each multiple.
for i in range(fin):
if not c[i]:
continue
c[c[i] + i::c[i]] = [None] * ((n // c[i]) - (n // (2 * c[i])) - 1)
# Filter out non-primes and return the list.
return [2] + [i for i in c if i]
This algorithm improves the original algorithm (mentioned at the top) by (usually) 50%. (Still, worse than the algorithm mentioned by nightcracker, naturally).
A question to Python Masters: Is there a more Pythonic way to express this last code, in a more "functional" way?
UPDATE 2: I still couldn't decode the algorithm mentioned by nightcracker. I guess I'm too stupid.
The question is, why would it even be faster? In both examples you are filtering multiples of two, the hard way. It doesn't matter whether you hardcode candidates[4::2] = [None] * (n // 2 - 1) or that it gets executed in the first loop of for i in range(2, fin + 1):.
If you are interested in an optimized sieve of Eratosthenes, here you go:
def primesbelow(N):
# https://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188
#""" Input N>=6, Returns a list of primes, 2 <= p < N """
correction = N % 6 > 1
N = (N, N-1, N+4, N+3, N+2, N+1)[N%6]
sieve = [True] * (N // 3)
sieve[0] = False
for i in range(int(N ** .5) // 3 + 1):
if sieve[i]:
k = (3 * i + 1) | 1
sieve[k*k // 3::2*k] = [False] * ((N//6 - (k*k)//6 - 1)//k + 1)
sieve[(k*k + 4*k - 2*k*(i%2)) // 3::2*k] = [False] * ((N // 6 - (k*k + 4*k - 2*k*(i%2))//6 - 1) // k + 1)
return [2, 3] + [(3 * i + 1) | 1 for i in range(1, N//3 - correction) if sieve[i]]
Explanation here: Porting optimized Sieve of Eratosthenes from Python to C++
The original source is here, but there was no explanation. In short this primesieve skips multiples of 2 and 3 and uses a few hacks to make use of fast Python assignment.
You do not save a lot of time avoiding the evens. Most of the computation time within the algorithm is spent doing this:
candidates[i + i::i] = [None] * (n // i - 1)
That line causes a lot of action on the part of the computer. Whenever the number in question is even, this is not run as the loop bails on the if statement. The time spent running the loop for even numbers is thus really really small. So eliminating those even rounds does not produce a significant change in the timing of the loop. That's why your method isn't considerably faster.
When python produces numbers for range it uses a formula: start + index * step. Multiplying by two as you do in your case is going to be slightly more expensive then one as in the original case.
There is also quite possibly a small overhead to having a longer function.
Neither are of those are really significant speed issues, but they override the very small amount of benefit your version brings.
Its probably slightly slower because you are performing extra set up to do something that was done in the first case anyway (marking off multiples of two). That setup time might be what you see if it is as slight as you say
Your extra step is unnecessary and will actually traverse the whole collection n once doing that 'get rid of evens' operation rather than just operating on n^1/2.