Reducing loops in python - python

I would like to implement the following snippet using just the inntermost for loop(the one which iterates 3 times),as this consumes a lot of time.
for i in arange(r):
for j in arange(c):
for k in arange(3):
if m[i,j]==n[i,j,k]:
new[i,j]=old[i,j,k]
Could anyone suggest a better method?

for k in range(3):
ind = m == n[:,:,k]
new[ind] = old[:,:,k][ind]

Look at using itertools.product - never used it with numpy arrays, but it might just work (and don't see why not)
for i, j, k in itertools.product(arange(r), arange(c), arange(3)):
if m[i,j]==n[i,j,k]:
new[i,j]=old[i,j,k]

Since arange(c) is computed for each i, and arange(3) for each couple (i, j), computing them once and for all outside the loop could save some time:
range_j = arange(c)
range_3 = arange(3)
for i in arange(r):
for j in range_j:
for k in range_3:
if m[i,j]==n[i,j,k]:
new[i,j]=old[i,j,k]
Of course this is only valid because these ranges are independent from i and j.

Related

Nested for loop python equation

I am new to python and coding and I have a problem where I need to use nested for loops to solve an equation where i^3 + j^3 + k^3 = N. Where N is 50 to 53 and i, j, k have a range of -800 to 800.
I do not understand how to use a nested for loop for this. Every example I have seen of for loops they are used for creating matrices or arrays.
The code I have tried output a very large amounts of falses.
for i in range (-800,801):
for j in range (-800,801):
for k in range(-800,801):
for N in range (50,54):
print(i**3+j**3+k**3==N,end=" ")
print()
Am I on the right track here? I got a large amount of false outputs so does that mean it ran it and is giving me a every possible outcome? I just need to know the numbers that exists that make the statement true
The nested loops are not your problem, they are already correct.
You need to print the values of i, j, k, and N only if they satisfy the equation. For this purpose of conditional execution of code, Python (as many other programming languages) has if statements.
You learn about them in the official Python tutorial or in any good beginners book about programming in Python, like Think Python or Automate the Boring Stuff with Python.
In your case, you can apply an if statement like this:
Instead of unconditionally printing
print(i**3+j**3+k**3==N,end=" ")
use
if i**3+j**3+k**3==N:
print(i, j, k, N)
I think what you want to do is check if
i^3+j^3+k^3==N
and only print i, j and k if the statement is true.
You can do that by adding:
for i in range(-800, 801):
for j in range(-800, 801):
for k in range(-800, 801):
for N in range(50, 54):
if i**3+j**3+k**3 == N:
print(i, j, k, N)
output:
-796 602 659 51
-796 659 602 51
This will give you the number of times the equation is correct:
count = 0
for i in range(-800,801):
for j in range(-800,801):
for k in range(-800,801):
for N in range(50,54):
if (i**3 + j**3 + k**3) == N:
count+= 1
print(count)

Python: Replacing multiple for loops, multiple iterators

I need your help. What is the most efficient way of substituting multiple nested for loops (over 5 nested loops into each other - 5 in the code example) if they all go through the same range (let's say from 1 to 10) and after the last for loop, at the bottom, i append every sum of the iterated items to the list. The code looks smth like this :
b=[]
for i in range(1, 11):
for j in range(1, 11):
for k in range(1, 11):
for l in range(1, 11):
for m in range(1, 11):
b.append(i+j+k+l+m)
It is obviously not memory-friendly and will take some time. As far as i know, itertools can not really help here either. So how could it be helped?
use itertools.product()
from itertools import product
b = [sum(item) for item in product(range(1, 11), repeat=5)]
You may want to make b generator, so that it is evaluated lazily
b = (sum(item) for item in product(range(1, 11), repeat=5))
I recommend using recursive programming. This happens when a function calls itself to do the same task with different parameters.
def someIteration(b, i, j, k, l, m):
for i in range(1,11):
b.append(i+j+k+l+m)
if (Certain task done):
return b
else:
someIteration(b, i, j, k, l, m)
I dont think this function will work directly, I recommend to do some research in recursive programming. It's a really interesting paradigm, mostly used in ai!
pros:
its very code efficient
cons:
when the stack is full, it will crash (this rarely happens, most of the time due to a bug.)

How to use parallel processing to run a for loop and some functions?

I have a an algorithm that has several for loops and some functions that sometimes need to be run in sequence or sometimes can be run in parallel. I am providing a pseudo-code and an example below.
Here, m is a mathematical model and I am trying to variables to the model m. The for loops are independent of each other (I have several such for loops not 2).
for i in range(1,N+1):
for d in range(1,delta+1):
for t in range(1,T+1):
for k in range(1,K+1):
z[(i,d,t,k)] = m.addVar(vtype = GRB.BINARY, name="z%d,%d,%d,%d" % (i,d,t,k))
for k in range(1,K+1):
for d in range(1,delta+1):
Q[(k,d)] = m.addVar(vtype = GRB.BINARY, name="Q%d,%d" % (k,d))
Once the model m is built completely i.e. all the for loops are completed I have the command that solves the optimization problem. This can only be done after the model is built completely. So the next command is:
z,Q = Solve(m)
Next, I am using other for loops to copy the results from model m. These cannot be used directly and must be copied in the way I have used.
for i in range(1,N+1):
for d in range(1,delta+1):
for t in range(1,T+1):
for k in range(1,K+1):
z_value[(i,d,t,k)] = z[(i,d,t,k)].X
for k in range(1,K+1):
for d in range(1,delta+1):
Q_value[(i,inst)] = Q[(k,d)].X
This portion is also independent of each other. I have more than two loops to run.
Is there a way that I can use parallel processing for these parts of my code. How do I do that?
You could reduce the nested loops to one loop using itertools.product for instance your first loop can be simplified to the below example using no nesting:
from itertools import product
for idtk in product(range(1,N+1), range(1,delta+1), range(1,T+1), range(1,K+1)):
#idtk is a tuple the same as (i, d, t, k)
z[idtk] = m.addVar(vtype = GRB.BINARY, name="z%d,%d,%d,%d" % idtk)

How to find last "K" indexes of vector satisfying condition (Python) ? (Analogue of Matlab's "find" )

Consider some vector:
import numpy as np
v = np.arange(10)
Assume we need to find last 2 indexes satisfying some condition.
For example in Matlab it would be written e.g.
find(v <5 , 2,'last')
answer = [ 3 , 4 ] (Note: Matlab indexing from 1)
Question: What would be the clearest way to do that in Python ?
"Nice" solution should STOP search when it finds 2 desired results, it should NOT search over all elements of vector.
So np.where does not seems to be "nice" in that sense.
We can easyly write that using "for", but is there any alternative way ?
I am afraid using "for" since it might be slow (at least it is very much so in Matlab).
This attempt doesn't use numpy, and it is probably not very idiomatic.
Nevertheless, if I understand it correctly, zip, filter and reversed are all lazy iterators that take only the elements that they really need. Therefore, you could try this:
x = list(range(10))
from itertools import islice
res = reversed(list(map(
lambda xi: xi[1],
islice(
filter(
lambda xi: xi[0] < 5,
zip(reversed(x), reversed(range(len(x))))
),
2
)
)))
print(list(res))
Output:
[3, 4]
What it does (from inside to outside):
create index range
reverse both array and indices
zip the reversed array with indices
filter the two (value, index)-pairs that you need, extract them by islice
Throw away the values, retain only indices with map
reverse again
Even though it looks somewhat monstrous, it should all be lazy, and stop after it finds the first two elements that you are looking for. I haven't compared it with a simple loop, maybe just using a loop would be both simpler and faster.
Any solution you'd find will iterate over the list even if the loop is 'hidden' inside a function.
The solution to your problem depends on the assumptions you can make e.g. is the list sorted?
for the general case I'd iterate over the loop starting at the end:
def find(condition, k, v):
indices = []
for i, var in enumerate(reversed(v)):
if condition(var):
indices.append(len(v) - i - 1)
if len(indices) >= k:
break
return indices
The condition should then be passed as a function, so you can use a lambda:
v = range(10)
find(lambda x: x < 5, 3, v)
will output
[4, 3, 2]
I'm not aware of a "good" numpy solution to short-circuiting.
The most principled way to go would be using something like Cython which to brutally oversimplify it adds fast loops to Python. Once you have set that up it would be easy.
If you do not want to do that you'd have to employ some gymnastics like:
import numpy as np
def find_last_k(vector, condition, k, minchunk=32):
if k > minchunk:
minchunk = k
l, r = vector.size - minchunk, vector.size
found = []
n_found = 0
while r > 0:
if l <= 0:
l = 0
found.append(l + np.where(condition(vector[l:r]))[0])
n_found += len(found[-1])
if n_found >= k:
break
l, r = 3 * l - 2 * r, l
return np.concatenate(found[::-1])[-k:]
This tries balancing loop overhead and numpy "inflexibility" by searching in chunks, which we grow exponentially until enough hits are found.
Not exactly pretty, though.
This is what I've found that seems to do this job for the example described (using argwhere which returns all indices that meet the criteria and then we find the last two of these as a numpy array):
ind = np.argwhere(v<5)
ind[-2:]
This searches through the entire array so is not optimal but is easy to code.

Optimizing the run time of the nested for loop

I am just getting started with competitive programming and after writing the solution to certain problem i got the error of RUNTIME exceeded.
max( | a [ i ] - a [ j ] | + | i - j | )
Where a is a list of elements and i,j are index i need to get the max() of the above expression.
Here is a short but complete code snippet.
t = int(input()) # Number of test cases
for i in range(t):
n = int(input()) #size of list
a = list(map(int, str(input()).split())) # getting space separated input
res = []
for s in range(n): # These two loops are increasing the run-time
for d in range(n):
res.append(abs(a[s] - a[d]) + abs(s - d))
print(max(res))
Input File This link may expire(Hope it works)
1<=t<=100
1<=n<=10^5
0<=a[i]<=10^5
Run-time on leader-board for C language is 5sec and that for Python is 35sec while this code takes 80sec.
It is an online judge so independent on machine.numpy is not available.
Please keep it simple i am new to python.
Thanks for reading.
For a given j<=i, |a[i]-a[j]|+|i-j| = max(a[i]-a[j]+i-j, a[j]-a[i]+i-j).
Thus for a given i, the value of j<=i that maximizes |a[i]-a[j]|+|i-j| is either the j that maximizes a[j]-j or the j that minimizes a[j]+j.
Both these values can be computed as you run along the array, giving a simple O(n) algorithm:
def maxdiff(xs):
mp = mn = xs[0]
best = 0
for i, x in enumerate(xs):
mp = max(mp, x-i)
mn = min(mn, x+i)
best = max(best, x+i-mn, -x+i+mp)
return best
And here's some simple testing against a naive but obviously correct algorithm:
def maxdiff_naive(xs):
best = 0
for i in xrange(len(xs)):
for j in xrange(i+1):
best = max(best, abs(xs[i]-xs[j]) + abs(i-j))
return best
import random
for _ in xrange(500):
r = [random.randrange(1000) for _ in xrange(50)]
md1 = maxdiff(r)
md2 = maxdiff_naive(r)
if md1 != md2:
print "%d != %d\n%s" % (md1, md2, r)
exit
It takes a fraction of a second to run maxdiff on an array of size 10^5, which is significantly better than your reported leaderboard scores.
"Competitive programming" is not about saving a few milliseconds by using a different kind of loop; it's about being smart about how you approach a problem, and then implementing the solution efficiently.
Still, one thing that jumps out is that you are wasting time building a list only to scan it to find the max. Your double loop can be transformed to the following (ignoring other possible improvements):
print(max(abs(a[s] - a[d]) + abs(s - d) for s in range(n) for d in range(n)))
But that's small fry. Worry about your algorithm first, and then turn to even obvious time-wasters like this. You can cut the number of comparisons to half, as #Brett showed you, but I would first study the problem and ask myself: Do I really need to calculate this quantity n^2 times, or even 0.5*n^2 times? That's how you get the times down, not by shaving off milliseconds.

Categories