We Keep Coding

Python nested loop optimization with function calls

I'm building a simple N-body integrator in python3.5, that implements the leapfrog timestepping as a position verlet.
In its essence it updates two float variables, x_tmp and v_tmp back and forth, where I need a function call to self.forces() to update v_tmp.
It is this function call that slows me down incredibly (I've profiled). The call does nothing out of the ordinary, just a few square roots and some adding and dividing numbers.
for t in range(self.max_timesteps):
#For all objects do the position verlet with generator expression / list comprehensions
x_tmp = [x_tmp[j] + 0.5*self.timestep*v_tmp[j] for j in range(self.num_objects)]
v_tmp = [v_tmp[j] + self.timestep*self.forces(x_tmp[j]) for j in range(self.num_objects)]
x_tmp = [x_tmp[j] + 0.5*self.timestep*v_tmp[j] for j in range(self.num_objects)]
if(t % self.outputtime == 0):
self.x_list[outputcounter] = x_tmp
self.v_list[outputcounter] = v_tmp
and the function self.forces() is
def forces(self,x):
r = np.sqrt((x[0])**2+(x[1])**2+(x[2])**2) # spherical radius
R = math.hypot(x[0], x[1]) # cylindrical radius
def _f1(r,x,y,z):
f = -G*self.Mb/(r*(r+self.rb)**2)
return np.array([f*x, f*y, f*z])
def _f2(R,x,y,z):
rr = math.hypot(self.disc_b,z)
arr = (self.disc_a+rr)
arrR = math.hypot(arr,R)
f = -G*self.Md/ arrR**3.
fz = f*(arr/rr)
return np.array([f*x,f*y,fz*z])
def _f3(r,x,y,z):
f = -self.Vh**2/(self.rh**2+r**2)
return np.array([f*x,f*y,f*z])
a = _f1(r,x[0],x[1],x[2]) + _f2(R,x[0],x[1],x[2]) + _f3(r,x[0],x[1],x[2])
return np.array((a[0], a[1], a[2]))
Now both lines in the upper code-block with x_tmp = ... in them scale well with num_objects (they nearly don't), but the line with v_tmp and the function call in it scales linearly with num_objects.
This is pretty bad. With max_timestes = 10^6 I get num_objects seconds runtime with this code, so if I want to compute 200 objects with this code, it takes me 200 seconds. This is completely unacceptable.
However I'm a bit at a loss for what to do here, as I've already optimized a few 2D-square-roots with math.hypot() and a couple of other things. But the forces()-call is still incredibly slow, an occurence that would never happen in C or equivalent.
So now I'm asking for help, is there anything obvious that I have overlooked in optimizing those function calls? Or can I quickly build a C-function to call which could speed things up.
Any ideas appreciated.

applying for loop such that counters are multiplied rather than being added in python

hello I am relatively new to python! Is there a way to do this using for loops in python?
This is a java implementation of something i want to do in python
for (i=1;i<20; i*= 2)
{System.out.println(i);}
Solution in while loop in python`
while i<20:
print i
i*=2
I cannot figure out a way to do this using for loops. Implemented it using while loop obviously, but still curious to know whether there is a method to do so or not

There are lots of ways to do this, e.g.
for i in range(5):
i = 2 ** i
print i
or using generators
from itertools import count, takewhile
def powers_of_two():
for i in count():
yield 2 ** i
for i in takewhile(lambda x: x < 20, powers_of_two()):
print i
But in the end, it depends on your use case what version gives the clearest and most readbale code. In most cases, you would probably just use a while-loop, since it's simple and does the job.

You think of for loops like they would be in other languages, like C, C++, Java, JavaScript etc.
Python for loops are different; they work on iterables, and you always have to read them like:
for element in iterable
instead of the C'ish
for(start_condition; continue_condition; step_statement)
Hence, you would need iterable to generate your products.
I like readability, so here's how I'd do it:
for a in (2**i for i in range(20)):
print a
But that mainly works because we mathematically know that the i'th element of your sequence is going to be 2**i.

There is not a real way to do this in Python. If you wanted to mimic the logic of that for loop exactly, then a manual while loop would definitely be the way to go.
Otherwise, in Python, you would try to find a generator or generator expression that produces the values of i. Depending on the complexity of your post loop expression, this may require an actual function.
In your case, it’s a bit simpler because the numbers you are looking for are the following:
1 = 2 ** 0
2 = 2 ** 1
4 = 2 ** 2
8 = 2 ** 3
...
So you can generate the numbers using a generator expression (2 ** k for k in range(x)). The problem here is that you would need to specify a value x which happens to be math.floor(math.log2(20)) + 1 (because you are looking for the largest number k for which 2 ** k < 20 is true).
So the full expression would be this:
for i in (2 ** k for k in range(math.floor(math.log2(20)) + 1)):
print(i)
… which is a bit messy, so if you don’t necessarily need the i to be those values, you could move it inside the loop body:
for k in range(math.floor(math.log2(20)) + 1):
i = 2 ** k
print(i)
But this still only fits your purpose. If you wanted a “real” C-for loop expression, you could write a generator function:
def classicForLoop (init, stop, step):
i = init
while i < stop:
yield i
i = step(i)
Used like this:
for i in classicForLoop(1, 20, lambda x: x * 2):
print(i)
Of course, you could also modify the generator function to take lambdas as the first and second parameter, but it’s a bit simpler like this.

Use range() function to define iteration length.You can directly use print() than system.out.println
Alexander mentioned it and re-iterating
for i in range(1,20):print(i*2)
You can also consider while loop here-
i=0
while (i<20):
print(2**i)
i=i+1
Remember indentation in python

nested for loop in need of possible optimisation

I have two lists of users (users1 and users2) and i am comparing them with the following code:
def lev(seq1, seq2):
oneago = None
thisrow = range(1, len(seq2) + 1) + [0]
for x in xrange(len(seq1)):
twoago, oneago, thisrow = oneago, thisrow, [0] * len(seq2) + [x + 1]
for y in xrange(len(seq2)):
delcost = oneago[y] + 1
addcost = thisrow[y - 1] + 1
subcost = oneago[y - 1] + (seq1[x] != seq2[y])
thisrow[y] = min(delcost, addcost, subcost)
return thisrow[len(seq2) - 1]
for x in users1_list:
for y in users2_list:
if 3 >= lev(x,y) > 1:
print x, "seems a lot like", y
Can i use list-comprehension to improve the nested for loop?

Can you use a list comprehension to improve the nested for loop?
In the lev function, I don't think so--at least not in the sense of "this is bad, and a list comprehension is the natural and direct thing that would clean it up."
Yes, you could use a list comprehension there, but several factors argue against comprehensions:
You're calculating a lot of things. This means there are many characters required for the resulting expressions (or subexpressions). It would be a very long comprehension expression, making quality formatting difficult and making it harder to hold all of the pieces in your head all at once.
You've nicely named the sub-expression components in ways that make logical sense. Spread out into multiple statements, the code is clear about how the deletion, addition, and substation costs are calculated. That's nice. It aids comprehension, esp. for you or someone else who comes back to this code after some time, and has to understand it all over again. If you shorten into a long expression to make a list comprehension neat, you'd remove the clarity of those subexpressions.
You do a lot of indexing. That is usually an anti-pattern / bad practice in Python, which has good "iterate over loop items" features. But there are algorithms--and this seems to be one of them--where indexing is the clear method of access. It's very consistent with what you will find in similar programs from other sources, or in reference materials. So using a more primitive indexing approach--something that often doesn't make sense in many Python contexts--works pretty well here.
In the second section, where you can loop over items not indices neatly, you do so. It's not like you're trying to avoid Pythonic constructs.
It does jump out at me that you're recalculating len(seq2) all the time, even though it seems to be a constant during this function. I'd calculate it once and reuse a stored value. And do you ever really use twoago? I didn't see it. So a revised snippet might be:
def lev(seq1, seq2):
oneago = None
len2 = len(seq2)
thisrow = range(1, len2 + 1) + [0]
for x in xrange(len(seq1)):
oneago, thisrow = thisrow, [0] * len2 + [x + 1]
for y in xrange(len2):
delcost = oneago[y] + 1
addcost = thisrow[y - 1] + 1
subcost = oneago[y - 1] + (seq1[x] != seq2[y])
thisrow[y] = min(delcost, addcost, subcost)
return thisrow[len2 - 1]
Finally, stackoverflow tends to be problem related. It has a sister site codereview that might be more appropriate for detailed code improvement suggestions (much as programmers is better for more theoretical programming questions).

>>> list1 = ['Bret', 'Jermaine', 'Murray']
>>> list2 = ['Jermaine', 'Murray', 'Mel']
If the entries in the lists are unique, it might make sense to convert them into sets. You could then see which things are common:
>>> set(list1).intersection(set(list2))
{'Jermaine', 'Murray'}
The union of both sets can be returned:
>>> set(list1).union(set(list2))
{'Bret', 'Jermaine', 'Mel', 'Murray'}
To measure the commonality between the two sets, you could calculate the Jaccard index (see http://en.wikipedia.org/wiki/Jaccard_index for more details):
>>> len(set(list1).intersection(set(list2))) / float(len(set(list1).union(set(list2))))
0.5
This the number of common elements divided by the total number of elements.

How many combinations are possible?

The recursive formula for computing the number of ways of choosing k items out of a set of n items, denoted C(n,k), is:
1 if K = 0
C(n,k) = { 0 if n<k
c(n-1,k-1)+c(n-1,k) otherwise
I’m trying to write a recursive function C that computes C(n,k) using this recursive formula. The code I have written should work according to myself but it doesn’t give me the correct answers.
This is my code:
def combinations(n,k):
# base case
if k ==0:
return 1
elif n<k:
return 0
# recursive case
else:
return combinations(n-1,k-1)+ combinations(n-1,k)
The answers should look like this:
>>> c(2, 1)
0
>>> c(1, 2)
2
>>> c(2, 5)
10
but I get other numbers... don’t see where the problem is in my code.

I would try reversing the arguments, because as written n < k.
I think you mean this:
>>> c(2, 1)
2
>>> c(5, 2)
10

Your calls, e.g. c(2, 5) means that n=2 and k=5 (as per your definition of c at the top of your question). So n < k and as such the result should be 0. And that’s exactly what happens with your implementation. And all other examples do yield the actually correct results as well.
Are you sure that the arguments of your example test cases have the correct order? Because they are all c(k, n)-calls. So either those calls are wrong, or the order in your definition of c is off.

This is one of those times where you really shouldn't be using a recursive function. Computing combinations is very simple to do directly. For some things, like a factorial function, using recursion there is no big deal, because it can be optimized with tail-recursion anyway.
Here's the reason why:
Why do we never use this definition for the Fibonacci sequence when we are writing a program?
def fibbonacci(idx):
if(idx < 2):
return idx
else:
return fibbonacci(idx-1) + fibbonacci(idx-2)
The reason is because that, because of recursion, it is prohibitively slow. Multiple separate recursive calls should be avoided if possible, for the same reason.
If you do insist on using recursion, I would recommend reading this page first. A better recursive implementation will require only one recursive call each time. Rosetta code seems to have some pretty good recursive implementations as well.

"'generator' object is not subscriptable" error

Why am I getting this error, from line 5 of my code, when attempting to solve Project Euler Problem 11?
for x in matrix:
p = 0
for y in x:
if p < 17:
currentProduct = int(y) * int(x[p + 1]) * int(x[p + 2]) * int(x[p + 3])
if currentProduct > highestProduct:
print(currentProduct)
highestProduct = currentProduct
else:
break
p += 1
'generator' object is not subscriptable

Your x value is is a generator object, which is an Iterator: it generates values in order, as they are requested by a for loop or by calling next(x).
You are trying to access it as though it were a list or other Sequence type, which let you access arbitrary elements by index as x[p + 1].
If you want to look up values from your generator's output by index, you may want to convert it to a list:
x = list(x)
This solves your problem, and is suitable in most cases. However, this requires generating and saving all of the values at once, so it can fail if you're dealing with an extremely long or infinite list of values, or the values are extremely large.
If you just needed a single value from the generator, you could instead use itertools.islice(x, p) to discard the first p values, then next(...) to take the one you need. This eliminate the need to hold multiple items in memory or compute values beyond the one you're looking for.
import itertools
result = next(itertools.islice(x, p))

As an extension to Jeremy's answer some thoughts about the design of your code:
Looking at your algorithm, it appears that you do not actually need truly random access to the values produced by the generator: At any point in time you only need to keep four consecutive values (three, with an extra bit of optimization). This is a bit obscured in your code because you mix indexing and iteration: If indexing would work (which it doesn't), your y could be written as x[p + 0].
For such algorithms, you can apply kind of a "sliding window" technique, demonstrated below in a stripped-down version of your code:
import itertools, functools, operator
vs = [int(v) for v in itertools.islice(x, 3)]
for v in x:
vs.append(int(v))
currentProduct = functools.reduce(operator.mul, vs, 1)
print(currentProduct)
vs = vs[1:]