We Keep Coding

Why is this 'for' loop going too long?

Below is a small code snippet that essentially tries to use a for loop to find all multiples of a = 3 that are less than 1000 and sum them together. For some reason the final value of mult is 2997, which is not within range(max). Thus, I'm confused as to why the final value of mult isn't 999 and why the for loop even executes after mult does hit a value of 999.
a = 3
max = 999
sum = 0 #cumulative sum of 'a' multiples
mult = 0 #current multiple of 'a'
k = 1 #counter variable
# --- Find all multiples of 'a' that are below 1000 ---
for mult in range(max):
mult = a*k
print(mult)
sum = sum + mult
k+=1
Also, it seems that changing the value of a directly affects how high mult will get before the loop is finished. More precisely, I've found that the final value of mult is equal to a*max, and I cannot think of a reason why this would be.
Edit: Nevermind, I realized that I intended to use a while loop but made the mistake of implementing a for loop instead. Sorry about this. Thank you to everyone who took the time to answer me.

Unlike other programming languages, for loops in Python strictly iterate over the given values.
I. e., with for mult in range(max):, range() defines what values should be assigned to mult on every loop run. It is not affected by any values you assign to mult later.
This concept is called iteration. range(max) returns an object which gives on every loop run the next value. In Python 2, this was just a list containing these values, in Python 3, it is an object which calculates the values "on demand".

It actually doesn't. It goes for exactly 999 times. The last output however is 2997, but as you can see in the output it always increases by 3. This is due to you reassigning mult to a*k.

Since mult is defined as the product of a (a constant equal 3) and k, initially equal to 1, incremented 998 times by 1 every time the loop passes, i.e. it is 999 at the end and 3 * 999 = 2997 in total.

If I am correct, then you want to print all the values between 0 and max and sum them right?
then the following code should help you.
for mult in range(max+1):
if mult % a == 0:
print mult
sum = sum + mult
print sum

Processing big list using python

So I'm trying to solve a challenge and have come across a dead end. My solution works when the list is small or medium but when it is over 50000. It just "time out"
a = int(input().strip())
b = list(map(int,input().split()))
result = []
flag = []
for i in range(len(b)):
temp = a - b[i]
if(temp >=0 and temp in flag):
if(temp<b[i]):
result.append((temp,b[i]))
else:
result.append((b[i],temp))
flag.remove(temp)
else:
flag.append(b[i])
result.sort()
for i in result:
print(i[0],i[1])
Where
a = 10
and b = [ 2, 4 ,6 ,8, 5 ]
Solution sum any two element in b which matches a
**Edit: ** Updated full code

flag is a list, of potentially the same order of magnitude as b. So, when you do temp in flag that's a linear search: it has to check every value in flag to see if that value is == temp. So, that's 50000 comparisons. And you're doing that once per loop in a linear walk over b. So, your total time is quadratic: 50,000 * 50,000 = 2,500,000,000. (And flag.remove is also linear time.)
If you replace flag with a set, you can test it for membership (and remove from it) in constant time. So your total time drops from quadratic to linear, or 50,000 steps, which is a lot faster than 2 billion:
flagset = set(flag)
for i in range(len(b)):
temp = a - b[i]
if(temp >=0 and temp in flagset):
if(temp<b[i]):
result.append((temp,b[i]))
else:
result.append((b[i],temp))
flagset.remove(temp)
else:
flagset.add(b[i])
flag = list(flagset)
If flag needs to retain duplicate values, then it's a multiset, not a set, which means you can implement with Counter:
flagset = collections.Counter(flag)
for i in range(len(b)):
temp = a - b[i]
if(temp >=0 and flagset[temp]):
if(temp<b[i]):
result.append((temp,b[i]))
else:
result.append((b[i],temp))
flagset[temp] -= 1
else:
flagset[temp] += 1
flag = list(flagset.elements())
In your edited code, you’ve got another list that’s potentially of the same size, result, and you’re sorting that list every time through the loop.
Sorting takes log-linear time. Since you do it up to 50,000 times, that’s around log(50;000) * 50,000 * 50,000, or around 30 billion steps.
If you needed to keep result in order throughout the operation, you’d want to use a logarithmic data structure, like a binary search tree or a skiplist, so you could insert a new element in the right place in logarithmic time, which would mean just 800.000 steps.
But you don’t need it in order until the end. So, much more simply, just move the result.sort out of the loop and do it at the end.

exchanging values of 2 maps 2D arrays in ipython,

I'm a beginner in python, and I wrote a code:
g=rand_image
for i in range(100):
for j in range (100):
G = np.fft.fft2(g)
theta = angle(G)
G_prim = image ** 0.5 * exp(1j * theta)
g_prim = np.fft.ifft2(G_prim)
for k in range(300):
for l in range(300):
if g_prim[k, l].imag > 1e-8 or g_prim[k, l] < 0:
g[k,l]=g[k,l]-(2-0.02*i)*g_prim[k,l]
else:
g[k ,l] = g_prim[k,l]
where I compare 2 images, rand_image and image. It takes 2hours to run such code. I would like to make it faster somehow replacing the 2 loops in the bottom (comparing 2 images with 300px*300px size) with maybe a map of 2 pictures and replacing certain values? Although I'm not sure if that is possible.
I nead to increment value of i and run the code multiple times for each i value.

^ ^
for i in range(100): | |
for j in range (100): | |
G = np.fft.fft2(g) #--- |
theta = angle(G) #-------
The first two statements in the j suite do not make use of i or j so move them out of the loops.
if g_prim[k, l].imag > 1e-8 or g_prim[k, l] < 0:
g[k,l]=g[k,l]-(2-0.02*i)*g_prim[k,l]
else:
g = g_prim
In this innermost loop suite you repeatedly assign different values to individual pixels of g if the condition is met.
g_prim relies on j so the condition ONLY relies on j
the calculation in the suite ONLY relies on i.
The value assigned makes use of the outermost loop's value, i but the condition does NOT. For every j that results in a True condition there will be 100 i's.
if the condition is met when j equals two then there will be one-hundred i,j pairs that meet the condition - (0,2), (1,2), ... (99,2)
each iteration of the k,l loop(s) will assign to g[k,l] when the condition is met.
the only i that matters is when i equals 99 because it will overwrite each previous calc when j was two.
The net affect is that only the terminal value (99) of i matters. You are doing a lot of wasted operations.
To make matters worse if the condition is not met, you overwrite ALL previous individual assignments making them useless.
It actually looks as if the entire outer loop is superfluous and can be discarded - i isn't used anywhere except that conditional suite - g[k,l]=g[k,l]-(2-0.02*i)*g_prim[k,l].
If your example is faithfully presented, the inner two (k,l) loops can be replaced using boolean indexing as #nio suggested.
Make a mask that represents your condition and its inverse
mask = np.logical_or(g_prim.imag > 1e-8, g_prim < 0)
not_mask = np.logical_not(mask)
Use the mask to make the assignments when the condition is met
#(2-0.02*i) .. i = 99!!
k = 2 - .02 * 99
g[mask] = g[mask] - k*g_prim[mask]
Assuming you did NOT intend to overwrite the previous conditional assignment,
use the inverse mask to make the rest of the assignments.
g[not_mask] = g_prim[not_mask]

Python Time Complexity (run-time)

def f2(L):
sum = 0
i = 1
while i < len(L):
sum = sum + L[i]
i = i * 2
return sum
Let n be the size of the list L passed to this function. Which of the following most accurately describes how the runtime of this function grow as n grows?
(a) It grows linearly, like n does.
(b) It grows quadratically, like n^2 does.
(c) It grows less than linearly.
(d) It grows more than quadratically.
I don't understand how you figure out the relationship between the runtime of the function and the growth of n. Can someone please explain this to me?

ok, since this is homework:
this is the code:
def f2(L):
sum = 0
i = 1
while i < len(L):
sum = sum + L[i]
i = i * 2
return sum
it is obviously dependant on len(L).
So lets see for each line, what it costs:
sum = 0
i = 1
# [...]
return sum
those are obviously constant time, independant of L.
In the loop we have:
sum = sum + L[i] # time to lookup L[i] (`timelookup(L)`) plus time to add to the sum (obviously constant time)
i = i * 2 # obviously constant time
and how many times is the loop executed?
it's obvously dependant on the size of L.
Lets call that loops(L)
so we got an overall complexity of
loops(L) * (timelookup(L) + const)
Being the nice guy I am, I'll tell you that list lookup is constant in python, so it boils down to
O(loops(L)) (constant factors ignored, as big-O convention implies)
And how often do you loop, based on the len() of L?
(a) as often as there are items in the list (b) quadratically as often as there are items in the list?
(c) less often as there are items in the list (d) more often than (b) ?

I am not a computer science major and I don't claim to have a strong grasp of this kind of theory, but I thought it might be relevant for someone from my perspective to try and contribute an answer.
Your function will always take time to execute, and if it is operating on a list argument of varying length, then the time it takes to run that function will be relative to how many elements are in that list.
Lets assume it takes 1 unit of time to process a list of length == 1. What the question is asking, is the relationship between the size of the list getting bigger vs the increase in time for this function to execute.
This link breaks down some basics of Big O notation: http://rob-bell.net/2009/06/a-beginners-guide-to-big-o-notation/
If it were O(1) complexity (which is not actually one of your A-D options) then it would mean the complexity never grows regardless of the size of L. Obviously in your example it is doing a while loop dependent on growing a counter i in relation to the length of L. I would focus on the fact that i is being multiplied, to indicate the relationship between how long it will take to get through that while loop vs the length of L. Basically, try to compare how many loops the while loop will need to perform at various values of len(L), and then that will determine your complexity. 1 unit of time can be 1 iteration through the while loop.
Hopefully I have made some form of contribution here, with my own lack of expertise on the subject.
Update
To clarify based on the comment from ch3ka, if you were doing more than what you currently have inside your with loop, then you would also have to consider the added complexity for each loop. But because your list lookup L[i] is constant complexity, as is the math that follows it, we can ignore those in terms of the complexity.

Here's a quick-and-dirty way to find out:
import matplotlib.pyplot as plt
def f2(L):
sum = 0
i = 1
times = 0
while i < len(L):
sum = sum + L[i]
i = i * 2
times += 1 # track how many times the loop gets called
return times
def main():
i = range(1200)
f_i = [f2([1]*n) for n in i]
plt.plot(i, f_i)
if __name__=="__main__":
main()
... which results in
Horizontal axis is size of L, vertical axis is how many times the function loops; big-O should be pretty obvious from this.

Consider what happens with an input of length n=10. Now consider what happens if the input size is doubled to 20. Will the runtime double as well? Then it's linear. If the runtime grows by factor 4, then it's quadratic. Etc.

When you look at the function, you have to determine how the size of the list will affect the number of loops that will occur.
In your specific situation, lets increment n and see how many times the while loop will run.
n = 0, loop = 0 times
n = 1, loop = 1 time
n = 2, loop = 1 time
n = 3, loop = 2 times
n = 4, loop = 2 times
See the pattern? Now answer your question, does it:
(a) It grows linearly, like n does. (b) It grows quadratically, like n^2 does.
(c) It grows less than linearly. (d) It grows more than quadratically.
Checkout Hugh's answer for an empirical result :)

it's O(log(len(L))), as list lookup is a constant time operation, independant of the size of the list.

Execute statement every N iterations in Python

I have a very long loop, and I would like to check the status every N iterations, in my specific case I have a loop of 10 million elements and I want to print a short report every millionth iteration.
So, currently I am doing just (n is the iteration counter):
if (n % 1000000==0):
print('Progress report...')
but I am worried I am slowing down the process by computing the modulus at each iteration, as one iteration lasts just few milliseconds.
Is there a better way to do this? Or shouldn't I worry at all about the modulus operation?

How about keeping a counter and resetting it to zero when you reach the wanted number? Adding and checking equality is faster than modulo.
printcounter = 0
# Whatever a while loop is in Python
while (...):
...
if (printcounter == 1000000):
print('Progress report...')
printcounter = 0
...
printcounter += 1
Although it's quite possible that the compiler is doing some sort of optimization like this for you already... but this may give you some peace of mind.

1. Human-language declarations for x and n:
let x be the number of iterations that have been examined at any given time.
let n be the multiple of iterations upon which your code will executed.
Example 1: "After x iterations, how many times was n done?"
Example 2: "It is the xth iteration and the action has occurred every nth time, so far."
2. What we're doing:
The first code block (Block A) uses only one variable, x (defined above), and uses 5 (an integer) rather than the variable n (defined above).
The second code block (Block B) uses both of the variables (x and n) that are defined above. The integer, 5, will be replaced by the variable, n. So, Block B literally performs an action at each nth iteration.
Our goal is to do one thing on every iteration and two things on every nth iteration.
We are going through 100 iterations.
m. Easy-to-understand attempt:
Block A, minimal variables:
for x in 100:
#what to do every time (100 times in-total): replace this line with your every-iteration functions.
if x % 5 == 0:
#what to do every 5th time: replace this line with your nth-iteration functions.
Block B, generalization.
n = 5
for x in 100:
#what to do every time (100 times in-total): replace this line with your every-iteration functions.
if x % n == 0:
#what to do every 5th time: replace this line with your nth-iteration functions.
Please, let me know if you have any issues because I haven't had time to test it after writing it here.
3. Exercises
If you've done this properly, see if you can use it with the turtle.Pen() and turtle.forward() function. For example, move the turtle forward 4 times and then right and forward once?
See if you can use this program with the turtle.circle() function. For example, draw a circle with radius+1 4 times and a circle of a new color with radiut+1 once?
Check out the reading (seen below) to attempt to improve the programs from exercise 1 and 2. I can't think of a good reason to be doing this: I just feel like it might be useful!
About modulo and other basic operators:
https://docs.python.org/2/library/stdtypes.html
http://www.tutorialspoint.com/python/python_basic_operators.htm
About turtle:
https://docs.python.org/2/library/turtle.html
https://michael0x2a.com/blog/turtle-examples

Is it really slowing down? You have to try and see for yourself. It won't be much of a slowdown, but if we're talking about nanoseconds it may be considerable. Alternatively you can convert one 10 million loop to two smaller loops:
m = 1000000
for i in range(10):
for i in range(m):
// do sth
print("Progress report")

It's difficult to know how your system will optimize your code without testing.
You could simplify the relational part by realizing that zero is evaluated as false.
if(not N % 10000000)
do stuff

Something like that ? :
for n in xrange(1000000,11000000,1000000):
for i in xrange(n-1000000,n):
x = 10/2
print 'Progress at '+str(i)
result
Progress at 999999
Progress at 1999999
Progress at 2999999
Progress at 3999999
Progress at 4999999
Progress at 5999999
Progress at 6999999
Progress at 7999999
Progress at 8999999
Progress at 9999999
.
EDIT
Better:
for n in xrange(0,10000000,1000000):
for i in xrange(n,n+1000000):
x = 10/2
print 'Progress at '+str(i)
And inspired from pajton:
m = 1000000
for n in xrange(0,10*m,m):
for i in xrange(n,n+m):
x = 10/2
print 'Progress at '+str(i+1)
I prefer this that I find more immediately readable than the pajton's solution.
It keeps the display of a value depending of i

I'd do some testing to see how much time your modulus calls are consuming. You can use timeit for that. If your results indicate a need for time reduction, another approach which eliminates your modulus calculation:
for m in xrange(m_min, m_max):
for n in xrange(n_min, n_max):
#do_n_stuff
print('Progress report...')

It's fast enough that I wouldn't worry about it.
If you really wanted to speed it up, you could do this to avoid the modulus
if (n == 1000000):
n = 0
print('Progress report...')

This makes the inner loop lean, and m does not have to be divisible by interval.
m = 10000000
interval = 1000000
i = 0
while i < m:
checkpoint = min(m, i+interval)
for j in xrange(i, checkpoint):
#do something
i = checkpoint
print "progress"

When I'm doing timing/reports based on count iterations, I just divide my counter by the desired iteration and determine if the result is an integer. So:
if n/1000000 == int(n/1000000):
print(report)