I am trying to vectorize the following triple product operation on an N x N array called p below:
for j in range(len(p)):
for k in range(len(p)):
for l in range(len(p)):
h[j, k, l] = p[j, k] * p[k, l] * p[l, j] - p[j, l] * p[l, k] * p[k, j]
I thought numpy.einsum should be of use here, despite that I'm not actually summing over the repeated indices, but I haven't been able to pin it down. Thoughts?
Simply porting over those loop iterators as string notations, we would have an einsum based solution like so -
h = np.einsum('jk,kl,lj->jkl',p,p,p) - np.einsum('jl,lk,kj->jkl',p,p,p)
Being basically an expansion related question (as we are not reducing any axis), we can simply use NumPy broadcasting too by introducing new axes with None/np.newaxis at various places for allowing the expansion, like so -
h = p[...,None]*p*p[:,None,:].T - p[:,None,:]*p.T*p.T[...,None]
Related
I would like to know if there is a way to move all elements of a numpy array without iterating over each entry. The shift I desire is to relabel the indices by a fixed XOR operation, in the following form:
import numpy as np
N = 2
x = np.arange(2**(2 * N)).reshape(2**N, 2**N)
z = np.zeros((2**N, 2**N))
k = 1
for i in range(2**N):
for j in range(2**N):
z[i][j] = x[i ^ k][j ^ k]
The problem I have is that latter I wish to take huge values of N, which becomes a bottleneck if we wish to iterate over each entry. Any advice on how to perform the move in a single shot will be very much appreciated.
There is simply no reason to use a loop here at all. You are not shifting anything, and the problem is completely separable across the axes:
x = np.arange(2**(2 * N)).reshape(2**N, 2**N)
z = x[np.arange(2**N)[:, None] ^ k, np.arange(2**N) ^ k]
But what is x[a, b] for some a, b to begin with? From the original definition, you can see that it's a * 2**N + b. You can therefore plug in a = np.arange(2**N)[:, None] ^ k and b = np.arange(2**N) ^ k to avoid having to generate anything but the final result:
idx = np.arange(2**N) ^ k
z = (idx[:, None] << N) + idx
It's generally faster to substitute << N for * 2**N.
Allocation
None of the solutions shown here pre-allocate z. However, if you were to do that, as in the original question, you should be careful about the type.
z = np.zeros((2**N, 2**N))
This creates an array of floats by default, which is likely not what you want.
z = np.zeros((2**N, 2**N), dtype=int)
Adding an explicit dtype make the array into integers. However, the simplest method available to you is likely
z = np.zeros_like(x)
Since you are planning on filling in every single element, you don't need to waste time filling it with zeros first. np.empty presents a better option in this case:
z = np.empty((2**N, 2**N), dtype=int)
OR
z = np.empty_like(x)
I'm trying to improve the performance of some code I am translating from MATLAB to Python, and have found that using nested for loops like so:
arr1 = np.zeros((NN, MM))
for i in range(NN):
for j in range(MM):
arr1[i, j] = arr[i, j]/(1+np.abs(i-j))
Is causing the script to run for a couple seconds. Since I am going to be using this code on a large dataset this could potentially waste hours of computation time unnecessarily.
Is there any way to use vectorization/broadcasting to achieve the same affect?
You could precalculate the indices like this.
import numpy as np
arr1 = np.zeros((NN, MM))
Then use numpy indices function, which returns a 3d matrix where the first index indicates the either i or j
grid = np.indices(arr1.shape)
Then divide.
arr1 /= (1+np.abs(grid[0]-grid[1]))
If arr1 is np.zeros, then the whole operation is redundant, hehe.
But yes, it is possible to speed the operation up - if we focus on the denominator - (1+np.abs(i-j)), you can break it up this way:
J = np.tile(np.arange(MM), (NN, 1))
I = np.tile(np.reshape(np.arange(NN), (NN, 1)), (1, MM))
and then let
x = x / (1 + I - J)
Be mindful, however, that 1 + I - J will equal zero when I = J + 1.
i often need to calculate a matrix A[i,j] based on a given vector v[i] by:
A[i, j] = v[j] - v[i]
This is straightforward in a nested loop, but I'd like to vectorize it. So far I've only come up with the rather ugly solution of creating two matrizes additional, where v is repeated in each row/column and I therefore can use simple element-wise matrix addition.
Here a numpy example:
import numpy as np
length = 10
v = np.random.random(length)
vjMatrix = np.broadcast_to(v, (length, length))
viMatrix = np.transpose(vjMatrix)
A = vjMatrix - viMatrix
print(A)
However, I hope there is a more elegant solution, that I just fail to see. I was looking through a lot of threads, but haven't found anything particularly suitable.
Thanks!
If I understand you question correctly, you currently fill array A like:
import numpy as np
length = 100
np.random.seed(123)
v = np.random.rand(length)
vjMatrix = np.broadcast_to(v, (length, length))
viMatrix = np.transpose(vjMatrix)
A = vjMatrix - viMatrix
If this is what you want, you can replace the loop and the explicit creation of the v-matrices by broadcasting the vector v:
A_new = v - v[:, None]
print(np.all(A == A_new))
# Out: True
I need to turn the following summation into Python code:
This Summation
the summation length N is known.
I'm struggling with the second summation
I need something like this
for i in range(N):
(1 - (for j in range(N):
sum(x[i][j]))**2
But the second for loop is obviously not going to work (Syntax Error)
How can I iterate over j within the first for loop?
The result needs to be saved in a variable
Thank you very much
sum can take a generator expression, so you can make this a pretty clean one-liner with:
total = sum((1 - sum(x[i][j] for j in range(N)))**2 for i in range(N))
If N is such that you are actually summing all the elements (array is square and N == number of rows/columns) in the array you can do away with the ranges and just use:
sum((1 - sum(n for n in row))**2 for row in x)
which you can also do in numpy with something like:
x = np.array(x)
np.sum((1 - np.sum(x, axis=1))**2)
or if N is to take a slice of the matrix:
a = np.array(x)
N = 2
np.sum((1 - np.sum(a[0:N, 0:N], axis=1))**2)
sum = 0
for i in range(N):
sum_inside = 0
for j in range(N):
sum_inside += x[i][j]
sum += (1 - sum_inside)**2
I hope you aren't asking stack overflow to do your homework!
The summation is a simple nested comprehension:
sum(1 - sum(x[i,j] for j in range(1, N+1)) ** 2 for i in range(1, N+1))
Don't take my word for it though; it's a good idea to check the results. Here I am using x as a tuple-keyed dict.
Be aware that the image ranges from 1 to N, not from 0 to N-1. If x is a 2d array, then that would be okay if you are adjusting for zero-indexing.
#MarkMeyer has a more comprehensive answer than this one.
Regarding vectors / multi-dimensional arrays, we have recently decided it's a little more pythonic to represent them as tuple-keyed dicts instead of nested lists. For instance, with x above, we define it as follows:
x = {(i,j): fn(i,j) for j in range(1, N) for i in range(1, N)}
The major advantage is that there is no preference of index, and we are able to use filter/map for most functions.
I think this is may be a solution for your problem. Still x[i][j] is confusing.
N=int(input("Enter the value of N "))
Sum=0
for i in range(N):
for j in range(N):
Sum =Sum+( 1 - (sum(x[i][j]))**2)
print(Sum)
I have found lots of help on creating arrays with specific number values but I cannot seem to find anything to help me set up the array in the first or second problems.
I am not asking for the answers to this assignment, this is just my first Python assignment so I am a beginner and cannot figure out how to set up the arrays I need as I am not given numbers.
So far, I have found this to create an empty array:
import itertools
import numpy as np
my_array = np.empty([n, n])
And then set the value at coordinate i, j to to f(i, j).
for i, j in itertools.product(range(n), range(n)):
my_array[i, j] = f(i, j)
I just cannot seem to figure out how to actually apply this code to my question. Would sin(z) be my f(i, j)?
Yes, sin(zi,j) would be your f(i, j). It's probably more efficient to do without the loop, though:
np.sin((2 * np.pi) * (1 - np.random.random_sample((n, n))))