We Keep Coding

What is the time complexity of this function that iterates through a list a creates a dictionary?

I have a function that rearranges an input list in a certain way and returns the output list. I am confused about what the time and space complexity of the function will be. Below is the code:
def rearrange_list(inp_list):
d = {}
output_list = []
for num in inp_list:
if num in d:
d[num] += 1
else:
d[num] = 0
output_list.append(num)
for k,v in d.items():
if v > 0:
for i in range(v):
output_list.append(k)
return output_list
This is my complexity analysis:
Time complexity: O(n + m2) where n is length of the input list and m is the size of dictionary
Space complexity: O(n) where n is the length of input list
The main confusion I have is should I consider iterating through the dictionary O(n) too since worst case we will have n items in the list, or should it be represent it by m like I did in my analysis since it can be anything from 0 to n?
Thank you in advance for your help!

Your time and space complexity are both Theta(n). While sometimes it can be useful for clarity to include terms in the time or space complexity that don't change the asymptotic value (a classic example being string searching algorithms), it doesn't make as much sense here.
While your claim of O(n + m^2) time complexity is technically correct as Big-O notation is an upper bound, you can show that O(n) is also an upper bound, since the dictionary has size at most n and we iterate over each key exactly once: there are n items read from input, at most n loop iterations for the dictionary, and n items appended to the output list.
If you wanted, you can calculate 'auxiliary' space required, which would be the amount of space needed but not counting the input or output arrays. Here, that would be Theta(m). You should note, however, that such analyses are fairly uncommon: by assumption, unless specified otherwise, space complexity analysis will include the size of the output.
To address a common confusion about why the second loop is still linear time with many duplicate values, let's look at an example.
The lines in question are:
for k, v in d.items():
if v > 0:
for i in range(v):
output_list.append(k)
Suppose our input list was [1, 1, 1, 1, 1, 1, 1, 2, 2, 2] (ten elements total: seven '1's and three '2's).
Then our dictionary.items() (which has the count of each element, minus one) would look like: [(key: 1, value: 6), (key: 2, value: 2)] (it's not really stored as a Python list of tuples internally, but these are the full contents of the items view-object).
Let's walk through that second loop's operation, line by line:
for k, v in [(key: 1, value: 6), (key: 2, value: 2)]:
# On our first iteration, so k = 1, v = 6.
if 6 > 0: # 1 operation
for i in range(6): # 6 operations
output_list.append(1) # 6 operations
# For the (k, v) pair of (1, 6), the inner-loop has run 6 times.
# Every time the inner-loop runs, the length of output_list
# increases by 1
# Second iteration of outer-loop runs again:
for k, v in [(key: 1, value: 6), (key: 2, value: 2)]:
# On our second iteration, so k = 2, v = 2.
if 2 > 0: # 1 operation
for i in range(2): # 2 operations
output_list.append(2) # 2 operations
# For the (k, v) pair of (1, 6), the inner loop has run 2 times.
# In total: 8 inner-loop iterations, and output_list has len 8
In informal complexity analysis, a 'standard' rule of thumb is that the run-time of a double-nested loop is often quadratic. This is because we're counting the total number of inner-loop iterations, for example
for i in range(n):
for j in range(n):
as
(n inner-loops per outer-loop) * (n outer-loops) = n^2 inner-loops
This assumption shouldn't be applied when the number of inner-loop iterations varies dramatically based on the state of the outer-loop. In our example, the inner-loop iterations is v, which depends on the outer loop.
To find the runtime here, we need a different way to count how many inner-loop iterations occur. Luckily, we can do that: in each inner-loop iteration, we append an element to output_list.
Since the final length of output_list is n, we know that the inner-loop has executed at most n times (technically, it's executed exactly n-m times, since the output_list already has size m after the earlier dictionary-initializing loop has terminated). Instead of incorrectly multiplying this by m, the number of outer-loop iterations, we should instead add the inner and outer loop iterations for a runtime of Theta(n+m) which is Theta(n).
Addendum: Comments have correctly pointed out that since Python dictionaries don't have an O(1) amortized worst-case lookup/insert time guarantee, so the first loop is, at best, Omega(m*n). While Python uses pseudo-random probing on an open-addressing table, this only ensures good 'average' performance. Thanks to Kelly Bundy for the highly informative discussion and corrections.
Unfortunately, while O(1) amortized worst-case lookup/insert hashing is possible, for example with Cuckoo hashing, in practice this is significantly slower on average than what's currently used in most standard libraries, and is unlikely to change in the near future.

Do a step by step breakdown of the algorithm:
First for loop (for num in inp_list).
It simply iterates over the list, which takes O(N) time, and the dictionary has size O(number of unique elements in list), which can be generalized as O(N) space (where N = length of list = max possible number of unique keys).
Second for loop (for k,v in d.items()).
This iterates over the keys in dict, which are O(number of unique elements in list) in count.
Third for loop (for i in range(v)).
Since summation of v would just be the count of duplicate elements in the list, it will have an upper bound of O(N).
A better approximation of the algorithm should be O(N) time and space, rather than the proposed O(n + m^2)

Time complexity
You can divide the code into two parts.
Part 1
for num in inp_list:
if num in d:
d[num] += 1
else:
d[num] = 0
output_list.append(num)
In the first line you iterate through inp_list, and in each iteration you call if num in d. What python does is searches through the keys of dictionary d, therefore the time complexity for this part is O(nm) where n is the size of inp_list and m is the size the number of unique values in inp_list.
Part 2
for k,v in d.items():
if v > 0:
for i in range(v):
output_list.append(k)
In this part you iterate through the size of the dictionary in the first row which is O(m). I am ignoring the nested for loop since the loop can be replaced by the following:
output_list = output_list + [k] * v
Which happens in O(1)
In conclusion, the time complexity should be O(nm + m) = O (m(n+1)) = O(nm).
Nevertheless, since d is a dictionary, the key search takes O(1) instead of O(m) (see more here), making the time complexity drop to O(n).
Space complexity
Since a dictionary with m keys is created (where m is the number of unique values in inp_list, the space complexity is O(n+m) < O(2n) = O(n)

What would the Big O notation be for alphabetically sorting each element in a nested list comprehension

# find all possible sorted substrings of s
substr = ["".join(sorted(s[i: j]))
for i in range(len(s))
for j in range(i + 1, len(s) + 1)]
I know the sorted() method is O(nlog(n)), and the finding of all possible substrings is O(n^2). However, the .join() is what is throwing me off. sorted() returns a list and .join() iterates over each element in the list to append it to a string. So it should be a linear operation.
Would my substring sorter therefore be O(n^2) for the nested loop * O(nlogn) for sorting each result * O(n) for joining? Ergo O(n^4logn)??
If so, would breaking up the operations make this more efficient? I had another implementation where I move the sorting of the substrings to a second list comprehension
substr = [s[i: j] for i in range(len(s))
for j in range(i + 1, len(s) + 1)]
sortedSubstr = ["".join(sorted(s)) for s in substr]
This would make it the O(n^2) list comprehension first + O(n)*O(nlogn) for the second list comprehension
Making the overall program now O(n^2logn)
Please correct me if I am wrong. Thanks

For the first algorithm time complexity is O(n^3*log(n)), because after two loop you are not making a join for each atomic action in sorted. You separately sort and join. So O(n) + O(n*log(n)) = O(n*log(n)), which multiplied by O(n^2) (nested loops) gives us O(n^3*log(n)).
About the second algorithm.
Calculation of substr gives us O(n^3): same O(n^2) for nested loops multiplied by O(n) for slicing s.
Note that len(substr) is O(n^2) — for each (i, j) from nested loops.
Calculation of sortedSubstr gives us O(n^3*log(n)): for each element of substr (whose count is O(n^2)) we call sorted. Each element's len is O(n), so sorted(s) gives us O(n*log(n)). So, samely, O(n^2) * O(n*log(n)) = O(n^3*log(n)).
Calculation of substr (O(n^3)) plus calculation of sortedSubstr (O(n^3*log(n))) yields O(n^3*log(n)).
So the same O(n^3*log(n)) for the second.

For the sake of analysis, let's replace the comprehensions with their equivalent loops as described in the docs.
Your first approach becomes
substr = []
for i in range(len(s)): # O(n)
for j in range(i + 1, len(s) + 1): # O(n)
substr.append("".join(sorted(s[i: j]))) # O(n*logn)
# Overall O(n^3 * logn)
Note that substr.append("".join(sorted(s[i: j]))) is not multiplicative, rather it is a sequential combination of the following operations (no actual assignments happen)
temp = s[i:j] # O(j-i), which worst case we can take O(n)
sorted_temp = sorted(temp) # O(n*logn)
joined_temp = "".join(sorted_temp) # O(n)
substr.append(joined_temp) # amortized O(1)
# Overall O(n*logn)
Now in the second approach the code becomes
substr = []
for i in range(len(s)): # O(n)
for j in range(i + 1, len(s) + 1): # O(n)
substr.append(s[i:j]) # O(n) for the slice
# first loop: O(n^3)
sortedSubstr = []
for s in substr: # O(n^2), we have appended n^2 times in the first loop
sortedSubstr.append("".join(sorted(s))) # O(n*logn)
# second loop: O(n^3 * logn)
# Overall O(n^3 * logn)
So as you can see, they should be the same time complexity. I almost missed substr length being n^2 rather than n, so that might be a pitfall.
References for time complexity of various operations:
Time complexity of string slice
Why is the time complexity of python's list.append() method O(1)?

In this expression:
"".join(sorted(s[i: j]))
the O(n) join is negligible, because you're doing it once, after you've done the O(nlogn) sort.
O(nlogn + n) = O((n+1)logn) = O(nlogn)
Doing that sort n^2 times gets you to O(n^3 logn), regardless of whether/when you do the join.

Python Time Complexity: For Loops

The code below uses nested loops to multiply each element of list a with all the other elements of list b.
I am conscious of the fact that the time complexity for each loop is O(n) and that here n is a fairly small value, but what if n was too large to be processed? That is if the lists a,b had too large the values? How could I alter my code's time complexity then for the same function applied.
a = [1,2,3]
b = [4,5,6]
new = []
if len(a) == len(b):
for x in a: # O(n)
for y in b: # O(n)
new.append((x*y))
print(new)

I don't see a way to improve your O(n^2) run-time. n counts multiplications so you could cache values. It would be the same complexity and your algorithm would be slower.

Which algorithm is faster to sort these number pairs?

I wrote two solutions in python.. It is supposed to take a list of numbers and sort the ones that add up to a sum, both these return the same pairs, but which one is more efficient? I'm not sure if using python's count method does more work behind the scene making the second one longer
numbers = [1, 2, 4, 4, 4, 4, 5, 7, 7, 8, 8, 8, 9]
match = []
for i in range(len(numbers)):
for j in range(len(numbers)):
if (i!=j):
if(numbers[i] + numbers[j] == sum):
match.append([numbers[i], numbers[j]])
match2 = []
for i in range(len(numbers)):
counterPart = abs(numbers[i] - sum)
numberOfCounterParts = numbers.count(counterPart)
if(numberOfCounterParts >= 1):
if(counterPart == numbers[i]):
for j in range(numbers.count(counterPart)-1):
match2.append([numbers[i], counterPart])
else:
for j in range(numbers.count(counterPart)):
match2.append([numbers[i], counterPart])
Is there an even better solution that I'm missing?

When comparing algorithms, you should compare their time complexities. Measuring the time is also a good idea, but heavily dependent on the input, which now is tiny.
The first algorithm takes:
O(N2)
because of the double for loop.
For the second algorithm, you should take into account that count() has a time complexity of O(N). You have on for loop, and in its body count() will be called twice, once after abs() and once in whichever body of the if-else statement you go into. As a result the time complexity is O(N) * 2 * O(N) = 2*O(N<sup>2</sup>), which yields:
O(N2)
That means that both algorithm have the same time complexity. As a result it now has meaning to measure the performance, by running many experiments and taking the average of the time measurements, with big enough input to reflect performance.

It's almost always useful to measure complexity of your algorithms.
Both of your algorithms has O(N^2) complexity, so there are almost interchangeable in terms of performance.
You may improve your algorithm by keeping a mapping of value-index pairs. It will reduce complexity to O(N), basically you'll have one loop.

you can run test yourself using the timeit module:
t1 = timeit(setup='from __main__ import sort1, numbers',
stmt='sort1(numbers)',
number=1)
t2 = timeit(setup='from __main__ import sort2, numbers',
stmt='sort2(numbers)',
number=1)
print(t1)
print(t2)
also note that sum is a built-in and therefore not a good name for a variable...
there are way better algorithms for that! especially considering you have duplicates in your list.
here is a faster version that will only give you the matches but not the multiplicity of the matches:
def sum_target(lst, target):
# make list unique
unique_set = set(lst)
unique_list = list(unique_set)
remainders = {number: target-number for number in unique_list}
print(remainders)
match = set()
for a, b in remainders.items():
if a == b and lst.count(a) >= 2:
match.add((a, b))
else:
if b in remainders:
match.add(frozenset((a, b)))
return match

Yes there is better algorithm which can be used if you know lower_bound and upper_bound of the data. Counting Sort which takes O(N) time and space is not constant (which depends on the range of upper bound and lower bound).
Refer Counting Sort
PS: Counting Sort is not a comparison based sort algorithm.
Refer below sample code:
def counting_sort(numbers, k):
counter = [0] * (k + 1)
for i in numbers:
counter[i] += 1
ndx = 0
for i in range(len(counter)):
while 0 < counter[i]:
numbers[ndx] = i
ndx += 1
counter[i] -= 1

What's a fast and pythonic/clean way of removing a sorted list from another sorted list in python?

I am creating a fast method of generating a list of primes in the range(0, limit+1). In the function I end up removing all integers in the list named removable from the list named primes. I am looking for a fast and pythonic way of removing the integers, knowing that both lists are always sorted.
I might be wrong, but I believe list.remove(n) iterates over the list comparing each element with n. meaning that the following code runs in O(n^2) time.
# removable and primes are both sorted lists of integers
for composite in removable:
primes.remove(composite)
Based off my assumption (which could be wrong and please confirm whether or not this is correct) and the fact that both lists are always sorted, I would think that the following code runs faster, since it only loops over the list once for a O(n) time. However, it is not at all pythonic or clean.
i = 0
j = 0
while i < len(primes) and j < len(removable):
if primes[i] == removable[j]:
primes = primes[:i] + primes[i+1:]
j += 1
else:
i += 1
Is there perhaps a built in function or simpler way of doing this? And what is the fastest way?
Side notes: I have not actually timed the functions or code above. Also, it doesn't matter if the list removable is changed/destroyed in the process.
For anyone interested the full functions is below:
import math
# returns a list of primes in range(0, limit+1)
def fastPrimeList(limit):
if limit < 2:
return list()
sqrtLimit = int(math.ceil(math.sqrt(limit)))
primes = [2] + range(3, limit+1, 2)
index = 1
while primes[index] <= sqrtLimit:
removable = list()
index2 = index
while primes[index] * primes[index2] <= limit:
composite = primes[index] * primes[index2]
removable.append(composite)
index2 += 1
for composite in removable:
primes.remove(composite)
index += 1
return primes

This is quite fast and clean, it does O(n) set membership checks, and in amortized time it runs in O(n) (first line is O(n) amortized, second line is O(n * 1) amortized, because a membership check is O(1) amortized):
removable_set = set(removable)
primes = [p for p in primes if p not in removable_set]
Here is the modification of your 2nd solution. It does O(n) basic operations (worst case):
tmp = []
i = j = 0
while i < len(primes) and j < len(removable):
if primes[i] < removable[j]:
tmp.append(primes[i])
i += 1
elif primes[i] == removable[j]:
i += 1
else:
j += 1
primes[:i] = tmp
del tmp
Please note that constants also matter. The Python interpreter is quite slow (i.e. with a large constant) to execute Python code. The 2nd solution has lots of Python code, and it can indeed be slower for small practical values of n than the solution with sets, because the set operations are implemented in C, thus they are fast (i.e. with a small constant).
If you have multiple working solutions, run them on typical input sizes, and measure the time. You may get surprised about their relative speed, often it is not what you would predict.

The most important thing here is to remove the quadratic behavior. You have this for two reasons.
First, calling remove searches the entire list for values to remove. Doing this takes linear time, and you're doing it once for each element in removable, so your total time is O(NM) (where N is the length of primes and M is the length of removable).
Second, removing elements from the middle of a list forces you to shift the whole rest of the list up one slot. So, each one takes linear time, and again you're doing it M times, so again it's O(NM).
How can you avoid these?
For the first, you either need to take advantage of the sorting, or just use something that allows you to do constant-time lookups instead of linear-time, like a set.
For the second, you either need to create a list of indices to delete and then do a second pass to move each element up the appropriate number of indices all at once, or just build a new list instead of trying to mutate the original in-place.
So, there are a variety of options here. Which one is best? It almost certainly doesn't matter; changing your O(NM) time to just O(N+M) will probably be more than enough of an optimization that you're happy with the results. But if you need to squeeze out more performance, then you'll have to implement all of them and test them on realistic data.
The only one of these that I think isn't obvious is how to "use the sorting". The idea is to use the same kind of staggered-zip iteration that you'd use in a merge sort, like this:
def sorted_subtract(seq1, seq2):
i1, i2 = 0, 0
while i1 < len(seq1):
if seq1[i1] != seq2[i2]:
i2 += 1
if i2 == len(seq2):
yield from seq1[i1:]
return
else:
yield seq1[i1]
i1 += 1