I have a 2D array shaped (1002,1004). For this question it could be generated via:
a = numpy.arange( (1002 * 1004) ).reshape(1002, 1004)
What I do is generate two lists. The lists are generated via:
theta = (61/180.) * numpy.pi
x = numpy.arange(a.shape[0]) #(1002, )
y = numpy.arange(a.shape[1]) #(1004, )
max_y_for_angle = int(y[-1] - (x[-1] / numpy.tan(theta)))
The first list is given by:
x_list = numpy.linspace(0, x[-1], len(x))
Note that this list is identical to x. However, for illustration purposes and to give a clear picture I declared this 'list'.
What I now want to do is create a y_list which is as long as x_list. I want to use these lists to determine the elements in my 2D array. After I determine and store the sum of the elements, I want to shift my y_list by one and determine the sum of the elements again. I want to do this for max_y_for_angle iterations. The code I have is:
sum_list = numpy.zeros(max_y_for_angle)
for idx in range(max_y_for_angle):
y_list = numpy.linspace((len(x) / numpy.tan(theta)) + idx, y[0] + idx , len(x))
elements = 0
for i in range(len(x)):
elements += a[x_list[i]][y_list[i]]
sum_list[idx] = elements
This operation works. However, as one might imagine this takes a lot of time due to the for loop within a for loop. The number of iterations of the for loops do not help as well. How can I speed things up? The operation now takes about 1 s. I'm looking for something below 200 ms.
Is it maybe possible to return a list of the 2D array elements when the inputs are x_list and y_list? I tried the following but this does not work:
a[x_list][y_list]
Thank you very much!
It's possible to return an array of elements form a 2d array by doing a[x, y] where x and y are both integer arrays. This is called advanced indexing or sometimes fancy indexing. In your question you mention lists a lot but never actually use any lists in your code, x_list and y_list are both arrays. Also, numpy multidimensional arrays are generally indexed a[i, j] even when when i and j are integers values.
Using fancy indexing along with some clean up of you code produced this:
import numpy
def line_sums(a, thata):
xsize, ysize = a.shape
tan_theta = numpy.tan(theta)
max_y_for_angle = int(ysize - 1 - ((xsize - 1) / tan_theta))
x = numpy.arange(xsize)
y_base = numpy.linspace(xsize / tan_theta, 0, xsize)
y_base = y_base.astype(int)
sum_list = numpy.zeros(max_y_for_angle)
for idx in range(max_y_for_angle):
sum_list[idx] = a[x, y_base + idx].sum()
return sum_list
a = numpy.arange( (1002 * 1004) ).reshape(1002, 1004)
theta = (61/180.) * numpy.pi
sum_list = line_sums(a, theta)
Hope that helps.
Related
The following function is written on Matlab. Now, I need to write an equivalent python function that will produce a similar output as Matlab. Can you help write the code, please?
function CORR=function_AutoCorr(tau,y)
% This function will generate a matrix, Where on-diagonal elements are autocorrelation and
% off-diagonal elements are cross-correlations
% y is the data set. e.g., a 10 by 9 Matrix.
% tau is the lag value. e.g. tau=1
Size=size(y);
N=Size(1,2); % Number of columns
T=Size(1,1); % length of the rows
for i=1:N
for j=1:N
temp1=0;
for t=1:T-tau
G=0.5*((y(t+tau,i)*y(t,j))+(y(t+tau,j)*y(t,i)));
temp1=temp1+G;
end
CORR(i,j)=temp1/(T-tau);
end
end
end
Assuming that y is a numpy Array, it would be pretty near something like (although I have not tested):
import numpy as np
def function_AutoCorr(tau, y):
Size = y.shape
N = Size[1]
T = Size[0]
CORR = np.zeros(shape=(N,N))
for i in range(N):
for j in range(N):
temp1 = 0
for t in range(T - tau):
G=0.5*((y[t+tau,i]*y[t,j])+(y[t+tau,j]*y[t,i]))
temp1 = temp1 + G
CORR[i, j] = temp1/(T - tau)
return CORR
y = np.array([[1,2,3], [4,5,6], [6,7,8], [13,14,15]])
print(y)
result = function_AutoCorr(1, y)
print(result)
The resulting CORR matrix for this example is:
If you want to run the function for different tau values, you could do, in Python:
result = [function_AutoCorr(tau, y) for tau in range(1, 11)]
The result will be a list of autocorrelation matrices, which are numpy arrays. This syntax is called a list comprehension.
You'll probably want to use NumPy. They even have a guide for Matlab users.
Here are some useful tips.
Defining a function
def auto_corr(tau, y):
"""Generate matrix of correlations"""
# Do calculations
return corr
Get the size of a numpy array
n_rows, n_cols = y.shape
Indexing
Indexing is 0-based and uses brackets ([]) instead of parentheses.
How can I write a recursive function to generate a vector X of size (1,n) as follows, where X_i is the i-th entry:
X_1 = Z_1 * E_1
X_i = max{B_(1,i) * X_1, ... , B_((i-1),i) * X_(i-1), Z_i} * E_i, i = 2,...,n,
where
Z = np.random.normal(0, 1,size = n)
E = np.random.lognormal(0, 1, size = n)
B = np.random.uniform(0,1,(n,n))
I do not have any experience with recursive functions, that is why I can not present any code with which I tried to solve this.
If you're working with numpy, then use all the power of numpy, not just the random module ;)
And if you work with vectors, then forget about recursion and use numpy's vectorised operations. For example, np.max gives you the maximum over an axis, np.dot gives you element-wise multiplication. You also have np.prod for the product of array elements over a given axis... Those are just examples that might fit your problem well. For a full documentation, https://docs.scipy.org/doc/numpy/
I got it, one does not need a recursion as #meowgoesthedog stated in the first comment.
import numpy as np
s=1000 # sample size
n=5
Z = np.random.normal(0, 1,size = (s,n))
B = np.random.uniform(0,1,(n,n))
E = np.random.lognormal(0, 1, size = (s,n))
X = np.zeros((s,n))
X[:,0] = Z[:,0]*E[:,0]
for k in range(s):
for l in range(1,n):
X[k,l] = max(np.max(X[k,:(l)] * B[:(l),l]), Z[k,l]) * E[k,l]
I'm trying to sum a two dimensional function using the array method, somehow, using a for loop is not outputting the correct answer. I want to find (in latex) $$\sum_{i=1}^{M}\sum_{j=1}^{M_2}\cos(i)\cos(j)$$ where according to Mathematica the answer when M=5 is 1.52725. According to the for loop:
def f(N):
s1=0;
for p1 in range(N):
for p2 in range(N):
s1+=np.cos(p1+1)*np.cos(p2+1)
return s1
print(f(4))
is 0.291927.
I have thus been trying to use some code of the form:
def f1(N):
mat3=np.zeros((N,N),np.complex)
for i in range(0,len(mat3)):
for j in range(0,len(mat3)):
mat3[i][j]=np.cos(i+1)*np.cos(j+1)
return sum(mat3)
which again
print(f1(4))
outputs 0.291927. Looking at the array we should find for each value of i and j a matrix of the form
mat3=[[np.cos(1)*np.cos(1),np.cos(2)*np.cos(1),...],[np.cos(2)*np.cos(1),...]...[np.cos(N+1)*np.cos(N+1)]]
so for N=4 we should have
mat3=[[np.cos(1)*np.cos(1) np.cos(2)*np.cos(1) ...] [np.cos(2)*np.cos(1) ...]...[... np.cos(5)*np.cos(5)]]
but what I actually get is the following
mat3=[[0.29192658+0.j 0.+0.j 0.+0.j ... 0.+0.j] ... [... 0.+0.j]]
or a matrix of all zeros apart from the mat3[0][0] element.
Does anybody know a correct way to do this and get the correct answer? I chose this as an example because the problem I'm trying to solve involves plotting a function which has been summed over two indices and the function that python outputs is not the same as Mathematica (i.e., a function of the form $$f(E)=\sum_{i=1}^{M}\sum_{j=1}^{M_2}F(i,j,E)$$).
The return statement is not indented correctly in your sample code. It returns immediately in the first loop iteration. Indent it on the function body instead, so that both for loops finish:
def f(N):
s1=0;
for p1 in range(N):
for p2 in range(N):
s1+=np.cos(p1+1)*np.cos(p2+1)
return s1
>>> print(f(5))
1.527247272700347
I have moved your code to a more numpy-ish version:
import numpy as np
N = 5
x = np.arange(N) + 1
y = np.arange(N) + 1
x = x.reshape((-1, 1))
y = y.reshape((1, -1))
mat = np.cos(x) * np.cos(y)
print(mat.sum()) # 1.5272472727003474
The trick here is to reshape x to a column and y to a row vector. If you multiply them, they are matched up like in your loop.
This should be more performant, since cos() is only called 2*N times. And it avoids loops (bad in python).
UPDATE (regarding your comment):
This pattern can be extended in any dimension. Basically, you get something like a crossproduct. Where every instance of x is matched up with every instance of y, z, u, k, ... Along the corresponding dimensions.
It's a bit confusing to describe, so here is some more code:
import numpy as np
N = 5
x = np.arange(N) + 1
y = np.arange(N) + 1
z = np.arange(N) + 1
x = x.reshape((-1, 1, 1))
y = y.reshape((1, -1, 1))
z = z.reshape((1, 1, -1))
mat = z**2 * np.cos(x) * np.cos(y)
# x along first axis
# y along second, z along third
# mat[0, 0, 0] == 1**2 * np.cos(1) * np.cos(1)
# mat[0, 4, 2] == 3**2 * np.cos(1) * np.cos(5)
If you use this for many dimensions, and big values for N, you will run into memory problems, though.
I have written the following code for creating a 2D array and filing the first element of each row. I am new to numpy. Is there a better way to do this?
y=np.zeros(N*T1).reshape(N,T1)
x = np.linspace(0,L,num = N)
for k in range(0,N):
y[k][0] = np.sin(PI*x[k]/L)
Yes, since numpy vectorizes operations, you can just do:
y[:,0] = np.sin(np.pi * x / L)
Note that y[:,0] grabs the first column of y (the : in the first coordinate essentially means "grab all rows", and the 0 in the second coordinate means "from the column at index 0" (ie the first column)). Since np.sin(np.pi * x / L) is also an array, you can assign the latter to the former directly.
This question is rather for codereview#stackexchange, but this snippet works!
import numpy as np
N = 1000 # arbitrary
T1 = 1000 # arbitrary
L = 10 # arbitrary
x = np.linspace(0,L,num = N)
# you don't need reshape here, give the size as a tuple!
y = np.zeros((N,T1))
# use a vectorized call here:
y[:,0] = np.sin(np.pi*x/L)
I have three separate 1d arrays of a list of numbers, their squares and cubes (created by a 'for' loop).
I would like these arrays to appear in three corresponding columns, however I have tried the column_stack function and python says its not defined. I have read about the vstack and hstack functions but am confused about which to use and what exactly they do.
My code so far reads;
import numpy
makearange = lambda a: numpy.arange(int(a[0]),int(a[1]),int(a[2]))
x = makearange(raw_input('Enter start,stop,increment: ').split(','))
y = numpy.zeros(len(x), dtype=int)
z = numpy.zeros(len(x), dtype=int)
for i in range(len(x)):
y[i] = x[i]**2
for i in range(len(x)):
z[i] = x[i]**3
print 'original array: ',x
print 'squared array: ',y
print 'cubed array: ', z
I would appreciate any advice
Why don't you define y and z directly?
y = x**2
z = x**3
and then simply:
stacked = np.column_stack((x,y,z))
which gives you a 2D array of shape len(x) * 3
import numpy
makearange = lambda a: numpy.arange(int(a[0]),int(a[1]),int(a[2]))
x = makearange(raw_input('Enter start,stop,increment: ').split(','))
a = np.zeros((len(x),3))
a[:,0] = x
a[:,1] = x**2
a[:,2] = x**3
When using arrays you should avoid for loops as much as possible, that's kind of the point of arrays.
a = np.zeros((len(x),3)) creates an array of length same as x and with 3 columns
a[:,i] is a reference to the 'i'th column of this array (i.e. select all values (denoted by :) along this (i) column)
I would strongly recommend you look at the Numpy Tutorial.
You do want column_stack. Have you tried:
w = numpy.column_stack((x,y,z))
print(w)