This question already has answers here:
Why does += behave unexpectedly on lists?
(9 answers)
+ and += operators are different? [duplicate]
(4 answers)
What exactly does += do?
(17 answers)
What does the += signify in python 3? [duplicate]
(3 answers)
Closed 1 year ago.
I was trying to submit my solution to a leetcode problem wherein x and y are lists and using
x = x + y
gave me a Time limit exceeded
whereas
using
x += y
passed the test cases and gave me AC.
What is the execution time difference between the two and the difference in the way both are executed?
For list objects,
temp = temp + []
Creates a new list, and takes linear time on the size of the resulting list (it scales linearly). Importantly, it re-creates the entire new list. If done in a loop, e.g.
x = []
for i in range(N):
x = x + [i]
The entire algorithm is quadratic time, O(N^2)
On the other hand, temp += [] works in-place. It does not create a new list. It is O(K) where K is the size of the list on the right, i.e. the number of elements added. This works this way because python list objects are implemented as array-lists which overallocate so you don't have to reallocate each time the list increases in size. Simply put, this means that appending an item to the end of the list is amortized constant time. Importantly, this makes:
x = []
for i in range(N):
x += [i]
linear time, i.e. O(N).
To see this behavior empirically, you could use the following script:
import pandas as pd
import matplotlib.pyplot as plt
import time
def concatenate(N):
result = []
for i in range(N):
result = result + [i]
def inplace(N):
result = []
for i in range(N):
result += [i]
def time_func(N, f):
start = time.perf_counter()
f(N)
stop = time.perf_counter()
return stop - start
NS = range(0, 100_001, 10_000)
inplc = [time_func(n, inplace) for n in NS]
concat = [time_func(n, concatenate) for n in NS]
df = pd.DataFrame({"in-place":inplc, "concat": concat}, index=NS)
df.plot()
plt.savefig('in-place-vs-new-list-loop.png')
Notice, at a N == 100_000, the concatenation version is taking over 10 seconds, whereas the in-place extend version takes 0.01 seconds... so it's several orders of magnitude slower, and the difference will keep growing dramatically (i.e. quadratically) as N increases.
To understand this behavior, here is an informal treatment of the time complexity:
For concat, at each iteration, x = x + [i] takes i amount of work, where i is the length of the resulting array. So the whole loop will be 0 + 1 + 2 + 3 + ... + N. Now, using the handy formula for the Nth partial sum of this well-known series the loop will require N*(N+1)/2 total work.
N*(N + 1) / 2 == N^2/2 + N/2 which is simply O(N^2)
On the other hand, the in-place extend version, on each iteration,
temp += [i]
Requires only 1 (constant) amount of work. So for the whole loop, it's just
1 + 1 + ... + 1 (N times)
So N total amount of work, so it is O(N)
The expression a = a + b does the following:
Allocate a new list big enough to hold both a and b.
Copy a to the new buffer.
Copy b to the new buffer.
Bind the name a to the new buffer (which is what list.__add__ returns).
The allocation and copies are inevitable in this case, regardless of the fact that b is empty.
The expression a += b is roughly equivalent to list.extend with an assignment at the end:
Extend the buffer of a by enough elements to hold b. This does not always involve a reallocation, because list growth is amortized to take O(n) time in the long run.
Copy b to the end of a.
Rebind the name a to the same object (which is what list.__iadd__ returns).
Notice that in this case, step is is a reallocation, so elements of a are copied only when the memory moves. Since b is empty in your case, nothing is reallocated or copied at all.
Related
I made a binary search algorithm, and I just was wondering if there is a way to increase the performance of memory usage / effective usage of memory and the speed of execution. Specifically, is there a way to quickly insert an item since I know that using the insert method requires shifting the indexes of all the rightward values by 1. This takes time, so is there a faster way to do this, maybe use a different data set that is mutable and index-able? Also, I made this with the intention that the values are all real numbers / floats or integers. Also, am I correct to assume I won't be using much more memory at any given point in the execution due to popping off the values and sorting them into the sorted list. Since A (length of original array) - B (number of values popped off) + B (number of values popped of from the original array that are moved into sorted array) = A.
def binary_sort(array: []):
""" Returns an organized version of the array """
number = 0
length = len(array)
sorted_array = []
while number < length:
value = array.pop()
start = 0
end = number
while start < end:
avg = (start + end) // 2
v = sorted_array[avg]
if v <= value:
start = 1 + avg
else:
end = avg
sorted_array.insert(start, value)
number += 1
return sorted_array
Why is try1 twice as fast as try2? Are these functions not both O(n)? Is the difference in execution time solely due to overhead?
I am new to programming and am teaching myself algorithms and data structures through Python.
According to the text I am following, the time complexity for the first function should be 3n (the three representing the three assignment statements), and for the second function n + n + n, or 3n.
What am I missing?
def try1(n):
start = time.time()
for i in range(n):
a = 1
b = 2
c = 3
end = time.time()
return c,end-start
def try2(n):
start = time.time()
for i in range(n):
a = 1
for i in range(n):
b = 2
for i in range(n):
c = 3
end = time.time()
return c,end-start
(1) If range(n) is actually calculating a list of integers, you are doing that three times as opposed to one time.
(2) In the first one you are cycling the loop variable through a list of n things one time, but in the second you are cycling it through a list of n things three times.
I would imagine the second could never be faster than the first, and in the absence of an optimizing compiler, I'd expect the second to be definitively slower than the first.
you might consider trying this experiment:
start = time.time()
for i in range(1000000):
for j in range(n):
break
end = time.time()
return c,end-start
By taking that time and dividing by a million, you should have a good idea of the raw cost of calling range(n). I wouldn't be surprised if this is the lion's share of the extra time you're seeing (compared to iterating through a list an extra two times).
If that's the case, you might also improve your second example by calculating x = range(n) first and then using for i in x three times.
So I'm trying to solve a challenge and have come across a dead end. My solution works when the list is small or medium but when it is over 50000. It just "time out"
a = int(input().strip())
b = list(map(int,input().split()))
result = []
flag = []
for i in range(len(b)):
temp = a - b[i]
if(temp >=0 and temp in flag):
if(temp<b[i]):
result.append((temp,b[i]))
else:
result.append((b[i],temp))
flag.remove(temp)
else:
flag.append(b[i])
result.sort()
for i in result:
print(i[0],i[1])
Where
a = 10
and b = [ 2, 4 ,6 ,8, 5 ]
Solution sum any two element in b which matches a
**Edit: ** Updated full code
flag is a list, of potentially the same order of magnitude as b. So, when you do temp in flag that's a linear search: it has to check every value in flag to see if that value is == temp. So, that's 50000 comparisons. And you're doing that once per loop in a linear walk over b. So, your total time is quadratic: 50,000 * 50,000 = 2,500,000,000. (And flag.remove is also linear time.)
If you replace flag with a set, you can test it for membership (and remove from it) in constant time. So your total time drops from quadratic to linear, or 50,000 steps, which is a lot faster than 2 billion:
flagset = set(flag)
for i in range(len(b)):
temp = a - b[i]
if(temp >=0 and temp in flagset):
if(temp<b[i]):
result.append((temp,b[i]))
else:
result.append((b[i],temp))
flagset.remove(temp)
else:
flagset.add(b[i])
flag = list(flagset)
If flag needs to retain duplicate values, then it's a multiset, not a set, which means you can implement with Counter:
flagset = collections.Counter(flag)
for i in range(len(b)):
temp = a - b[i]
if(temp >=0 and flagset[temp]):
if(temp<b[i]):
result.append((temp,b[i]))
else:
result.append((b[i],temp))
flagset[temp] -= 1
else:
flagset[temp] += 1
flag = list(flagset.elements())
In your edited code, you’ve got another list that’s potentially of the same size, result, and you’re sorting that list every time through the loop.
Sorting takes log-linear time. Since you do it up to 50,000 times, that’s around log(50;000) * 50,000 * 50,000, or around 30 billion steps.
If you needed to keep result in order throughout the operation, you’d want to use a logarithmic data structure, like a binary search tree or a skiplist, so you could insert a new element in the right place in logarithmic time, which would mean just 800.000 steps.
But you don’t need it in order until the end. So, much more simply, just move the result.sort out of the loop and do it at the end.
I have problems measuring complexity with python. Given the next two scripts:
1 def program1(L):
2 multiples = []
3 for x in L:
4 for y in L:
5 multiples.append(x*y)
6 return multiples
1 def program3(L1, L2):
2 intersection = []
3 for elt in L1:
4 if elt in L2:
5 intersection.append(elt)
6 return intersection
In the first one the best case (minimum steps to run the sript) is two considering an empty list L so are executed only the second and the sixth lines. The solution for the best case scenario: is 2.
In the worst case scenario L is a long list it goes through the loop for x in L n times.
The inner loop has three operations (assignment of a value to y, x*y, and list appending). So the inner loop executes 3*n times on each iteration of the outer loop. Thus the nested loop structure is executed n * (3*n + 1) = 3*n**2 + n times. Adding the second and the sixth line we get the solution 3n²+n+2.
But my question is: Where does comes from the number 1 in n(3n+1)?
According to me the solution is n(3n)+2 = 3n²+2 vs the right answer n(3n+1)+2 = 3n²+n+2.
Meanwhile in the second one the worst case scenario is n²+2n+2 but I don't understand why there is a quadratic term if there is only one loop.
According to you, there are three instructions in the innermost (y) loop of program1.
Assign to y.
Compute x*y.
Append to list.
By that same logic, there is one instruction in the outmost (x) loop:
Assign to x.
Perform inmost loop, see above.
That would make the outer loop:
n * (1 {assign to x} + n * 3 {assign, multiply, append})
Or:
n * (1 + 3n)
Adding the init/return instructions gives:
2 + n + 3n²
In program2, there is a similar situation with a "hidden loop":
2 instructions for init/return, plus ...
Then you run for elt in L1, which is going to be n iterations (n is size of L1). Your inner code is an if statement. In the worst case, the if body always runs. In the best case, it never runs.
The if condition is testing elt in L2, which is going to run an iterative function, type(L2).__contains__() on L2. The simple case will be an O(m) operation, where m is the length of L2. It is possible that L2 is not a list but some type where the in operation does not require a linear scan. For example, it might be a B-tree, or a dict, or a set, or who knows what? So you could assume that the best-case scenario is that elt in L2 is O(1) and the answer is no, while the worst-case is that elt in L2 is O(m) and the answer is yes.
Best case: 2 + n * (1 {assign to elt} + 1 {search L2})
Best case if L2 is a list: 2 + n * (1 {assign to elt} + m {search L2})
Worst case: 2 + n * (1 {assign to elt} + m {search L2} + 1 {append})
Which gives you 2 + 2n best case, 2 + n + nm best case if L2 is a list, and 2 + 2n + nm worst case.
You may be inclined to treat m as equal to n. That's your call, but if you're counting assignment statements, I'd argue against it.
def f2(L):
sum = 0
i = 1
while i < len(L):
sum = sum + L[i]
i = i * 2
return sum
Let n be the size of the list L passed to this function. Which of the following most accurately describes how the runtime of this function grow as n grows?
(a) It grows linearly, like n does.
(b) It grows quadratically, like n^2 does.
(c) It grows less than linearly.
(d) It grows more than quadratically.
I don't understand how you figure out the relationship between the runtime of the function and the growth of n. Can someone please explain this to me?
ok, since this is homework:
this is the code:
def f2(L):
sum = 0
i = 1
while i < len(L):
sum = sum + L[i]
i = i * 2
return sum
it is obviously dependant on len(L).
So lets see for each line, what it costs:
sum = 0
i = 1
# [...]
return sum
those are obviously constant time, independant of L.
In the loop we have:
sum = sum + L[i] # time to lookup L[i] (`timelookup(L)`) plus time to add to the sum (obviously constant time)
i = i * 2 # obviously constant time
and how many times is the loop executed?
it's obvously dependant on the size of L.
Lets call that loops(L)
so we got an overall complexity of
loops(L) * (timelookup(L) + const)
Being the nice guy I am, I'll tell you that list lookup is constant in python, so it boils down to
O(loops(L)) (constant factors ignored, as big-O convention implies)
And how often do you loop, based on the len() of L?
(a) as often as there are items in the list (b) quadratically as often as there are items in the list?
(c) less often as there are items in the list (d) more often than (b) ?
I am not a computer science major and I don't claim to have a strong grasp of this kind of theory, but I thought it might be relevant for someone from my perspective to try and contribute an answer.
Your function will always take time to execute, and if it is operating on a list argument of varying length, then the time it takes to run that function will be relative to how many elements are in that list.
Lets assume it takes 1 unit of time to process a list of length == 1. What the question is asking, is the relationship between the size of the list getting bigger vs the increase in time for this function to execute.
This link breaks down some basics of Big O notation: http://rob-bell.net/2009/06/a-beginners-guide-to-big-o-notation/
If it were O(1) complexity (which is not actually one of your A-D options) then it would mean the complexity never grows regardless of the size of L. Obviously in your example it is doing a while loop dependent on growing a counter i in relation to the length of L. I would focus on the fact that i is being multiplied, to indicate the relationship between how long it will take to get through that while loop vs the length of L. Basically, try to compare how many loops the while loop will need to perform at various values of len(L), and then that will determine your complexity. 1 unit of time can be 1 iteration through the while loop.
Hopefully I have made some form of contribution here, with my own lack of expertise on the subject.
Update
To clarify based on the comment from ch3ka, if you were doing more than what you currently have inside your with loop, then you would also have to consider the added complexity for each loop. But because your list lookup L[i] is constant complexity, as is the math that follows it, we can ignore those in terms of the complexity.
Here's a quick-and-dirty way to find out:
import matplotlib.pyplot as plt
def f2(L):
sum = 0
i = 1
times = 0
while i < len(L):
sum = sum + L[i]
i = i * 2
times += 1 # track how many times the loop gets called
return times
def main():
i = range(1200)
f_i = [f2([1]*n) for n in i]
plt.plot(i, f_i)
if __name__=="__main__":
main()
... which results in
Horizontal axis is size of L, vertical axis is how many times the function loops; big-O should be pretty obvious from this.
Consider what happens with an input of length n=10. Now consider what happens if the input size is doubled to 20. Will the runtime double as well? Then it's linear. If the runtime grows by factor 4, then it's quadratic. Etc.
When you look at the function, you have to determine how the size of the list will affect the number of loops that will occur.
In your specific situation, lets increment n and see how many times the while loop will run.
n = 0, loop = 0 times
n = 1, loop = 1 time
n = 2, loop = 1 time
n = 3, loop = 2 times
n = 4, loop = 2 times
See the pattern? Now answer your question, does it:
(a) It grows linearly, like n does. (b) It grows quadratically, like n^2 does.
(c) It grows less than linearly. (d) It grows more than quadratically.
Checkout Hugh's answer for an empirical result :)
it's O(log(len(L))), as list lookup is a constant time operation, independant of the size of the list.