Related
Firstly I looked into the following questions:
O(N+M) time complexity
Comparing complexity of O(n+m) and O(max(n,m))
Big-O of These Nested Loops
Is this function O(N+M) or O(N*M)?
How to find time complexity of an algorithm
However, I'm still not 100% confident. That said, I have the following python example code:
adjList = [[4,7,8,9],[7,7,5,6],[1,4,3],[2,9],[2,1,7]]
for i in adjList:
for j in i:
print "anything else"
I came to think this is an O(n+m) algorithm, here is my reasoning:
I have adjList which is a list of lists. The integers there are randomly picked for the sake of exemplification. This is actually an adjacency list where the vertex 1 is linked to the vertices 4, 7, 8 and 9 and so on.
I know that adjList[i][j] will return the j-th item from the i-th list. So adjList[0][2] is 8
The first for will loop through each of the i lists. If we have N lists this is an O(N).
The second for will loop through each of the j elements of a list. But in this case, j is not a fixed value, e.g., the first list has 4 elements (4,7,8,9) and the third one has 3 elements (1,4,3). So at the end the second for will loop M times, M being the sum of each of the different j values. So, M is the sum of elements of every list. Thus O(M)
In this example, the first for should loop 5 times and the second for should loop 16 times. A total of 21 loops. If I change the adjList to a single big list within a list like this:
adjList = [[4,7,8,9,7,7,5,6,1,4,3,2,9,2,1,7]]
It would still loop through the 16 elements in the second for plus 1 time for the first for.
Thereby I can say that the algorithm will loop N times plus M times. Where N is the number of lists in adjList and M is the sum of elements in each one of the lists inside adjList. So O(N+M)
So, where lies my doubt?
Anywhere I looked I've found examples of nested loops being O(N^2) or O(N*M). Even when people mentioned that they can be something else other than those I've found no example. I'm yet to find an example of O(N+M) nested loops. So I'm still in doubt if my reasoning is correct.
Part of me wonders if this is not a O(N*M) algorithm. But I wouldn't elaborate on this.
Thus my final questions remains: Is this reasoning correct and said algorithm is indeed O(N+M)? If not, care to show where my mistakes are?
Your big mistake is that you have not clearly identified M and N what mean.
For example:
Visiting all cells in an N x M matrix is O(N*M).
If you flatten that matrix into a list with P cells, visiting is O(P).
However P == N*M in this context ... so O(M*N) and O(P) mean the same thing ... in this context.
Looking at your reasoning, you seem to have conflated (your) M with the analog of my P. (I say analog because rectangular and ragged arrays are not identical.)
So, M is the sum of elements of every list.
That's not how I have used M. More importantly, it is not how the various other references you have looked at are using M. Specifically the ones that talk about an N x M matrix or an N x avge(M) ragged array. Hence your confusion.
Note that your M and N are not independent variables / parameters. If you scale a problem in N, that implicitly changes the value of M.
Hint: when reasoning about complexity, one way to be sure you get the correct is to go back to basics. Work out the formulae for counting the operations performed, and reasons about them rather than attempting to reason about how the "big O" notation composes.
You define N and M as follows:
Thereby I can say that the algorithm will loop N times plus M times. Where N is the number of lists in adjList and M is the sum of elements in each one of the lists inside adjList. So O(N+M)
By this definition, the algorithm is O(M)1. To understand why N vanishes, you need to consider the relationship between N and M. Suppose you have two lists, and you want to look at every possible pair of items from the two lists. We'll keep it simple:
list1 = [1, 2, 3]
list2 = [4, 5]
So you want to look at all six of these pairs:
pairs = [(1, 4), (2, 4), (3, 4), (1, 5), (2, 5), (3, 5)]
That's a total of 3 * 2 = 6. Now generalize that; say list1 has N items and list2 has M items. The total number of pairs is N * M, and so this will be an O(N * M) algorithm.
Now suppose that instead of looking at each pair, you just want to look at each item that is in one or the other list. Now you're just looking at all the values that appear in a concatenation of the two lists:
items = [1, 2, 3, 4, 5]
That's a total of 3 + 2 = 5 items. Now generalize; you'll get N + M, and so this will be an O(N + M) algorithm.
Given that, we should expect your case and the above case to be identical, if your case is indeed O(N + M). In other words, we should expect your case to involve looking at all the items from two different lists. But look:
all_lists = [[4,7,8,9],[7,7,5,6],[1,4,3],[2,9],[2,1,7]]
That's the same thing as:
list1 = [4,7,8,9]
list2 = [7,7,5,6]
list3 = [1,4,3]
list4 = [2,9]
list5 = [2,1,7]
Whereas in the O(N + M) case, there were only two lists, here, there are five lists! So this can't be O(N + M).
However, this should give you an idea of how to work out a better description. (Hint: it could include J, K, and L, in addition to M and N.)
The origin of your mistake is that in the first two examples, M and N are defined to be separate from one another, but your definitions of M and N overlap. In order for M and N to be summed or multiplied meaningfully, they need to be unrelated to one another. But in your definitions, the values of M and N are interrelated, and so in a sense, they repeat values that shouldn't be repeated.
Or, to put it in yet another way, suppose the sum of the lengths of all the inner lists adds up to M. If you have to take two steps instead of just one for each of those values, the result is still only a constant value C times M. And for any constant value C, C * O(M) is still O(M). So the work you do in the outer loop is already counted (up to a constant multiplier) by the O(M) term.
Notes:
1. Well, ok, not quite, as Stefan Pochmann points out. Because of a technicality, it might be more appropriate to say O(max(N,
M)) because if any inner lists are empty, you'll still have to visit them.
If your code looked like this:
for i in adjList:
<something significant 1>
for j in i:
<something significant 2>
Then I would agree with you. However <something significant 1> is missing (the internal work python does to execute the loop is not worth considering) and so there is no reason to include the O(N) part. Big O notation is not for counting every single thing, it's for showing how the algorithm scales as inputs get bigger. So we look at what matters, and that means your code should be considered O(M).
The reason nested loops are usually considered O(N*M) is because usually N is defined as the number of iterations of the outer loop (as you've done) and M is defined as the number of iterations of the inner loop per outer iteration, not in total. Therefore N*M by the common definition is equal to M in your definition.
EDIT: some are claiming that the time to loop should be considered, considering for example the case of a large number of empty lists. As the code below shows, it takes significantly longer just to construct such a nested list than to nested loop through it. And that's for a trivial construction, usually it would be more complicated. Therefore I do not believe it is worth considering the time to loop.
from time import time
start = time()
L = [[] for _ in range(10000000)]
construction_time = time() - start
start = time()
for sub in L:
for i in sub:
pass
loop_time = time() - start
print(construction_time / loop_time) # typically between 3 and 4
I would like to write a function that takes integer numbers x, y, L and R as parameters and returns True if x**y lies in the interval (L, R] and False otherwise.
I am considering several ways to write a conditional statement inside this function:
if L < x ** y <= R:
if x ** y > L and x ** y <= R:
if x ** y in range(L + 1, R + 1):
Why is option 1 the most efficient in terms of execution time ?
Both #1 and #3 avoid recalculating x ** y, where #2 must calculate it twice.
On Python 2, #3 will be terrible, because it must compute the whole contents of the range. On Python 3.2+, it doesn't have to (range is smart, and can properly determine mathematically whether an int appears in the range without actually iterating, in constant time), but it's at best equivalent to #1, since creating the range object at all has some overhead.
As tobias_k mentions in the comments, if x ** y produces a float, #3 will be slower (breaks the Python 3.2+ O(1) membership testing optimization, requiring an implicit loop over all values), and will get different results than #1 and #2 if the value is not equal to any int value in the range. That is, testing 3.5 in range(1, 5) returns False, and has to check 3.5 against 1, 2, 3, and 4 individually before it can even tell you that much.
Basically, stick to #1, it's going to be the only one that avoids redundant computations and avoids creating a ton of values for comparison on both Py 2 and Py3. #3 is not going to be much (if at all) slower on Python 3.2+, but it does involve creating a range object that isn't needed here, and won't be quite logically equivalent.
The first one has to evaluate x**y only once, so it should be faster than the second (also, more readable). The third one would have to loop over the iterator (in python 2, so it should be slower than both) or make two comparisons (in python 3, so it is no better than the first one). Keep the first one.
x = [1,2,3,4,5,6,7,8,9,10]
#Random list elements
for i in range(int(len(x)/2)):
value = x[i]
x[i] = x[len(x)-i-1]
x[len(x)-i-1] = value
#Confusion on efficiency
print(x)
This is a uni course for first year. So no python shortcuts are allowed
Not sure what counts as "a shortcut" (reversed and the "Martian Smiley" [::-1] being obvious candidates -- but does either count as "a shortcut"?!), but at least a couple small improvements are easy:
L = len(x)
for i in range(L//2):
mirror = L - i - 1
x[i], x[mirror] = x[mirror], x[i]
This gets len(x) only once -- it's a fast operation but there's no reason to keep repeating it over and over -- also computes mirror but once, does the swap more directly, and halves L (for the range argument) directly with the truncating-division operator rather than using the non-truncating division and then truncating with int. Nanoseconds for each case, but it may be considered slightly clearer as well as microscopically faster.
x = [1,2,3,4,5,6,7,8,9,10]
x = x.__getitem__(slice(None,None,-1))
slice is a python builtin object (like range and len that you used in your example)
__getitem__ is a method belonging to iterable types ( of which x is)
there are absolutely no shortcuts here :) and its effectively one line.
I am creating a fast method of generating a list of primes in the range(0, limit+1). In the function I end up removing all integers in the list named removable from the list named primes. I am looking for a fast and pythonic way of removing the integers, knowing that both lists are always sorted.
I might be wrong, but I believe list.remove(n) iterates over the list comparing each element with n. meaning that the following code runs in O(n^2) time.
# removable and primes are both sorted lists of integers
for composite in removable:
primes.remove(composite)
Based off my assumption (which could be wrong and please confirm whether or not this is correct) and the fact that both lists are always sorted, I would think that the following code runs faster, since it only loops over the list once for a O(n) time. However, it is not at all pythonic or clean.
i = 0
j = 0
while i < len(primes) and j < len(removable):
if primes[i] == removable[j]:
primes = primes[:i] + primes[i+1:]
j += 1
else:
i += 1
Is there perhaps a built in function or simpler way of doing this? And what is the fastest way?
Side notes: I have not actually timed the functions or code above. Also, it doesn't matter if the list removable is changed/destroyed in the process.
For anyone interested the full functions is below:
import math
# returns a list of primes in range(0, limit+1)
def fastPrimeList(limit):
if limit < 2:
return list()
sqrtLimit = int(math.ceil(math.sqrt(limit)))
primes = [2] + range(3, limit+1, 2)
index = 1
while primes[index] <= sqrtLimit:
removable = list()
index2 = index
while primes[index] * primes[index2] <= limit:
composite = primes[index] * primes[index2]
removable.append(composite)
index2 += 1
for composite in removable:
primes.remove(composite)
index += 1
return primes
This is quite fast and clean, it does O(n) set membership checks, and in amortized time it runs in O(n) (first line is O(n) amortized, second line is O(n * 1) amortized, because a membership check is O(1) amortized):
removable_set = set(removable)
primes = [p for p in primes if p not in removable_set]
Here is the modification of your 2nd solution. It does O(n) basic operations (worst case):
tmp = []
i = j = 0
while i < len(primes) and j < len(removable):
if primes[i] < removable[j]:
tmp.append(primes[i])
i += 1
elif primes[i] == removable[j]:
i += 1
else:
j += 1
primes[:i] = tmp
del tmp
Please note that constants also matter. The Python interpreter is quite slow (i.e. with a large constant) to execute Python code. The 2nd solution has lots of Python code, and it can indeed be slower for small practical values of n than the solution with sets, because the set operations are implemented in C, thus they are fast (i.e. with a small constant).
If you have multiple working solutions, run them on typical input sizes, and measure the time. You may get surprised about their relative speed, often it is not what you would predict.
The most important thing here is to remove the quadratic behavior. You have this for two reasons.
First, calling remove searches the entire list for values to remove. Doing this takes linear time, and you're doing it once for each element in removable, so your total time is O(NM) (where N is the length of primes and M is the length of removable).
Second, removing elements from the middle of a list forces you to shift the whole rest of the list up one slot. So, each one takes linear time, and again you're doing it M times, so again it's O(NM).
How can you avoid these?
For the first, you either need to take advantage of the sorting, or just use something that allows you to do constant-time lookups instead of linear-time, like a set.
For the second, you either need to create a list of indices to delete and then do a second pass to move each element up the appropriate number of indices all at once, or just build a new list instead of trying to mutate the original in-place.
So, there are a variety of options here. Which one is best? It almost certainly doesn't matter; changing your O(NM) time to just O(N+M) will probably be more than enough of an optimization that you're happy with the results. But if you need to squeeze out more performance, then you'll have to implement all of them and test them on realistic data.
The only one of these that I think isn't obvious is how to "use the sorting". The idea is to use the same kind of staggered-zip iteration that you'd use in a merge sort, like this:
def sorted_subtract(seq1, seq2):
i1, i2 = 0, 0
while i1 < len(seq1):
if seq1[i1] != seq2[i2]:
i2 += 1
if i2 == len(seq2):
yield from seq1[i1:]
return
else:
yield seq1[i1]
i1 += 1
The recursive formula for computing the number of ways of choosing k items out of a set of n items, denoted C(n,k), is:
1 if K = 0
C(n,k) = { 0 if n<k
c(n-1,k-1)+c(n-1,k) otherwise
I’m trying to write a recursive function C that computes C(n,k) using this recursive formula. The code I have written should work according to myself but it doesn’t give me the correct answers.
This is my code:
def combinations(n,k):
# base case
if k ==0:
return 1
elif n<k:
return 0
# recursive case
else:
return combinations(n-1,k-1)+ combinations(n-1,k)
The answers should look like this:
>>> c(2, 1)
0
>>> c(1, 2)
2
>>> c(2, 5)
10
but I get other numbers... don’t see where the problem is in my code.
I would try reversing the arguments, because as written n < k.
I think you mean this:
>>> c(2, 1)
2
>>> c(5, 2)
10
Your calls, e.g. c(2, 5) means that n=2 and k=5 (as per your definition of c at the top of your question). So n < k and as such the result should be 0. And that’s exactly what happens with your implementation. And all other examples do yield the actually correct results as well.
Are you sure that the arguments of your example test cases have the correct order? Because they are all c(k, n)-calls. So either those calls are wrong, or the order in your definition of c is off.
This is one of those times where you really shouldn't be using a recursive function. Computing combinations is very simple to do directly. For some things, like a factorial function, using recursion there is no big deal, because it can be optimized with tail-recursion anyway.
Here's the reason why:
Why do we never use this definition for the Fibonacci sequence when we are writing a program?
def fibbonacci(idx):
if(idx < 2):
return idx
else:
return fibbonacci(idx-1) + fibbonacci(idx-2)
The reason is because that, because of recursion, it is prohibitively slow. Multiple separate recursive calls should be avoided if possible, for the same reason.
If you do insist on using recursion, I would recommend reading this page first. A better recursive implementation will require only one recursive call each time. Rosetta code seems to have some pretty good recursive implementations as well.