So I have a large list of points.
I have split those points up into the x coordinates and the y coordinates and then further split them into groups of 1000.
x = [points_Cartesian[x: x + 1000, 0] for x in range(0, len(points_Cartesian), 1000)]
(The y coordinates looks the same but with y instead of x.)
I am trying to turn the cartesian points into polar and to do so I must square every item in x and every item in y.
for sublist1 in x:
temp1 = []
for inte1 in sublist1:
temp1.append(inte1**2)
xSqua.append(temp1)
After that I add both of the Squared values together and square root them to get rad.
rad = np.sqrt(xSqua + ySqua)
The problem is, I started with 10,000 points and somewhere in this code it gets trimmed down to 1,000.
Does anyone know what the error is and how I fix it?
You're already using numpy. You can reshape matrices using numpy.reshape() and square the entire array elementwise using the ** operator on the entire array and your code will be much faster than iterating.
For example, let's say we have a 10000x3 points_cartesian
points_Cartesian = np.random.random((10000,2))
# reshape to 1000 columns, as many rows as required
xpts = points_Cartesian[:, 0].reshape((-1, 1000))
ypts = points_Cartesian[:, 1].reshape((-1, 1000))
# elementwise square using **
rad = np.sqrt(xpts**2 + ypts**2)
ang = np.arctan2(ypts, xpts)
Now rad and ang are 10x1000 arrays.
Related
I have a data array of multiple dimensions, with the last one being the distance. On the other hand, I have the distance vector r. For instance:
Data = np.ones((20, 30, 100))
r = np.linspace(10, 50, 100)
Finally, I also have a list of critical distance values called r0, such that (r0.shape == Data.shape[:-1]).all(). For instance,
r0 = np.random.random((20, 30))*40 + 10
I'm looking to replace values of Data by zero based on a condition on r and r0 corresponding to the first dimensions of Data. For example, I want for any i and j that:
Data[i, j, r>=r0[i,j]] = 0
Consider that Data can be a big array such that using loops is very long. My current workaround is:
r_temp = np.broadcast_to(np.expand_dims(r, list(np.arange(len(Data.shape)-1))), Data.shape)
Data[r_temp >= r0[..., None]] = 0
It is fast, but it consumes a lot of memory considering that I have to store the array r_temp, which can be problematic if Data starts to be big.
Any solution that does not necessitate to create and store r_temp ?
Note: for the creation of r_temp, see here.
I have two lists of matrices with different lengths.
e.g.
length of matrices in X:
(110,3)
(150,3)
(120,3)
length of matrices in Y:
(100,3)
(125,3)
My problem is I have a loop that will subtract matrices e.g.(X[0]-Y[0]) and then (X[0]-Y[1]) and so on. Since they have different shapes, I have to create a code that will slice the matrix.
for x in range (len(X)):
for y in range (len(Y)):
if len(X[x])> len(Y[y]):
X_a = len(X[x]) - len(Y[y])
X_len = len(X[x]) - X_a
Am I going somewhere here? Sorry. This has been confusing for me. :(
From the code it seems that when you want to subtract two matrices you want to do the subtraction up to the length of the smaller matrix. To do that you can have a function like:
def subtract_matricies(mat1, mat2):
minimum_len = min(len(mat1), len(mat2))
return mat1[:minimum_len]-mat2[:minimum_len]
then in your main loop use this function:
result=[]
for x in X:
for y in Y:
result.append(subtract_matricies(x,y))
I've an image of about 8000x9000 size as a numpy matrix. I also have a list of indices in a numpy 2xn matrix. These indices are fractional as well as may be out of image size. I need to interpolate the image and find the values for the given indices. If the indices fall outside, I need to return numpy.nan for them. Currently I'm doing it in for loop as below
def interpolate_image(image: numpy.ndarray, indices: numpy.ndarray) -> numpy.ndarray:
"""
:param image:
:param indices: 2xN matrix. 1st row is dim1 (rows) indices, 2nd row is dim2 (cols) indices
:return:
"""
# Todo: Vectorize this
M, N = image.shape
num_indices = indices.shape[1]
interpolated_image = numpy.zeros((1, num_indices))
for i in range(num_indices):
x, y = indices[:, i]
if (x < 0 or x > M - 1) or (y < 0 or y > N - 1):
interpolated_image[0, i] = numpy.nan
else:
# Todo: Do Bilinear Interpolation. For now nearest neighbor is implemented
interpolated_image[0, i] = image[int(round(x)), int(round(y))]
return interpolated_image
But the for loop is taking huge amount of time (as expected). How can I vectorize this? I found scipy.interpolate.interp2d, but I'm not able to use it. Can someone explain how to use this or any other method is also fine. I also found this, but again it is not according to my requirements. Given x and y indices, these generated interpolated matrices. I don't want that. For the given indices, I just want the interpolated values i.e. I need a vector output. Not a matrix.
I tried like this, but as said above, it gives a matrix output
f = interpolate.interp2d(numpy.arange(image.shape[0]), numpy.arange(image.shape[1]), image, kind='linear')
interp_image_vect = f(indices[:,0], indices[:,1])
RuntimeError: Cannot produce output of size 73156608x73156608 (size too large)
For now, I've implemented nearest-neighbor interpolation. scipy interp2d doesn't have nearest neighbor. It would be good if the library function as nearest neighbor (so I can compare). If not, then also fine.
It looks like scipy.interpolate.RectBivariateSpline will do the trick:
from scipy.interpolate import RectBivariateSpline
image = # as given
indices = # as given
spline = RectBivariateSpline(numpy.arange(M), numpy.arange(N), image)
interpolated = spline(indices[0], indices[1], grid=False)
This gets you the interpolated values, but it doesn't give you nan where you need it. You can get that with where:
nans = numpy.zeros(interpolated.shape) + numpy.nan
x_in_bounds = (0 <= indices[0]) & (indices[0] < M)
y_in_bounds = (0 <= indices[1]) & (indices[1] < N)
bounded = numpy.where(x_in_bounds & y_in_bounds, interpolated, nans)
I tested this with a 2624x2624 image and 100,000 points in indices and all told it took under a second.
Here is my problem : I manipulate 432*46*136*136 grids representing time*(space) encompassed in numpy arrays with numpy and python. I have one array alt, which encompasses the altitudes of the grid points, and another array temp which stores the temperature of the grid points.
It is problematic for a comparison : if T1 and T2 are two results, T1[t0,z0,x0,y0] and T2[t0,z0,x0,y0] represent the temperature at H1[t0,z0,x0,y0] and H2[t0,z0,x0,y0] meters, respectively. But I want to compare the temperature of points at the same altitude, not at the same grid point.
Hence I want to modify the z-axis of my matrices to represent the altitude and not the grid point. I create a function conv(alt[t,z,x,y]) which attributes a number between -20 and 200 to each altitude. Here is my code :
def interpolation_extended(self,temp,alt):
[t,z,x,y]=temp.shape
new=np.zeros([t,220,x,y])
for l in range(0,t):
for j in range(0,z):
for lat in range(0,x):
for lon in range(0,y):
new[l,conv(alt[l,j,lat,lon]),lat,lon]=temp[l,j,lat,lon]
return new
But this takes definitely too much time, I can't work this it. I tried to write it using universal functions with numpy :
def interpolation_extended(self,temp,alt):
[t,z,x,y]=temp.shape
new=np.zeros([t,220,x,y])
for j in range(0,z):
new[:,conv(alt[:,j,:,:]),:,:]=temp[:,j,:,:]
return new
But that does not work. Do you have any idea of doing this in python/numpy without using 4 nested loops ?
Thank you
I can't really try the code since I don't have your matrices, but something like this should do the job.
First, instead of declaring conv as a function, get the whole altitude projection for all your data:
conv = np.round(alt / 500.).astype(int)
Using np.round, the numpys version of round, it rounds all the elements of the matrix by vectorizing operations in C, and thus, you get a new array very quickly (at C speed). The following line aligns the altitudes to start in 0, by shifting all the array by its minimum value (in your case, -20):
conv -= conv.min()
the line above would transform your altitude matrix from [-20, 200] to [0, 220] (better for indexing).
With that, interpolation can be done easily by getting multidimensional indices:
t, z, y, x = np.indices(temp.shape)
the vectors above contain all the indices needed to index your original matrix. You can then create the new matrix by doing:
new_matrix[t, conv[t, z, y, x], y, x] = temp[t, z, y, x]
without any loop at all.
Let me know if it works. It might give you some erros since is hard for me to test it without data, but it should do the job.
The following toy example works fine:
A = np.random.randn(3,4,5) # Random 3x4x5 matrix -- your temp matrix
B = np.random.randint(0, 10, 3*4*5).reshape(3,4,5) # your conv matrix with altitudes from 0 to 9
C = np.zeros((3,10,5)) # your new matrix
z, y, x = np.indices(A.shape)
C[z, B[z, y, x], x] = A[z, y, x]
C contains your results by altitude.
I have a two-dimensional array that I want to fill up with values that represent powers but my problem lies in the speed of the code because the two-dimensional array is 100x100 size and I don't want to first initialize it with 100x100 list of zereos then fill up the list with values but rather fill up the 100x100 two-dimensional list by values directly. My code is shown down below
x_list = np.linspace(min_x, max_x, (max_x - min_x)+1)
y_list = np.linspace(min_y, max_y, (max_y - min_y)+1)
X, Y = np.meshgrid(x_list, y_list)
Y = Y[::-1]
Z = [[0 for x in range(len(x_list))] for x in range(len(y_list))] #Z is the two-dimensional list containing powers of reach position in the structure to be plotted
for each_axes in range(len(Z)):
for each_point in range(len(Z[each_axes])):
Z[len(Z)-1-each_axes][each_point] = power_at_each_point(each_point, each_axes)
#The method power_at_each_point is the one that calculates the values in the two-dimensional array Z
An example what I want to do is instead of doing what is shown below:
Z_old = [[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]
for each_axes in range(len(Z_old)):
for each_point in range(len(Z_old[each_axes])):
Z_old[len(Z_old)-1-each_axes][each_point] = power_at_each_point(each_point, each_axes)
I want now to not initialize the Z_old array with zeroes but rather fill it up with values while iterating through it which is going to be something like the written below although it's syntax is horribly wrong but that's what I want to reach in the end.
Z = np.zeros((len(x_list), len(y_list))) for Z[len(x_list) -1 - counter_1][counter_2] is equal to power_at_each_point(counter_1, counter_2] for counter_1 in range(len(x_list)) and counter_2 in range(len(y_list))]
plus the method of power_at_each_point is shown below with it's related methods if it helps you understand what I wanted to do:
#A method to calculate the power reached from one node to the other for contourf function
def cal_pow_rec_plandwall_contour(node_index_tx, receiver):
nodess_excel = xlrd.open_workbook(Node_file_location)
nodes_sheet = nodess_excel.sheet_by_index(0)
node_index_tx_coor = [nodes_sheet.cell_value(node_index_tx - 1, 3), nodes_sheet.cell_value(node_index_tx - 1, 4)] #just co-ordinates of a point
distance = cal_distance(node_index_tx_coor, receiver)
if distance == 0:
power_rec = 10 * (np.log10((nodes_sheet.cell_value(node_index_tx - 1, 0) * 1e-3)))
return power_rec #this is the power received at each position
else:
power_rec = 10 * (np.log10((nodes_sheet.cell_value(node_index_tx - 1, 0) * 1e-3))) - 20 * np.log10((4 * math.pi * distance * 2.4e9) / 3e8) - cal_wall_att([node_index_tx_coor, receiver])
return power_rec
def power_at_each_point(x_cord, y_coord): #A method to get each position in the structure and calculate the power reached at that position to draw the structure's contourf plot
fa = lambda xa: cal_pow_rec_plandwall_contour(xa, [x_cord, y_coord])
return max(fa(each_node) for each_node in range(1, len(Node_Positions_Ascending) + 1)) #Node_position_ascending is a list containing the co-ordinate positions of markers basically or nodes.
If someone could tell me how can I fill the two-dimensional array Z with values from the bottom of the top as I did right there without initially setting the two-dimensional array to zero first it would be much appreciated.
OK, first, you want to create a NumPy array, not a list of lists. This is almost always going to be significantly smaller, and a little faster to work on. And, more importantly, it opens the door to vectorizing your loops, which makes them a lot faster to work on. So, instead of this:
Z_old = [[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]
… do this:
Z_old = np.zeros((3, 5))
But now let's see whether we can vectorize your loop instead of modifying the values:
for each_axes in range(len(Z_old)):
for each_point in range(len(Z_old[each_axes])):
Z_old[len(Z_old)-1-each_axes][each_point] = each_point**2 + each_axes**2
The initial values of Z[…] aren't being used at all here, so we don't need to pre-fill them with 0, just as you suspected. What is being used at each point is r and c. (I'm going to rename your Z_old, each_axes, and each_point to Z, r, and c for brevity.) In particular, you're trying to set each Z[len(Z)-1-r, c] to r**2 + c**2.
First, let's reverse the negatives so you're setting each Z[r, c] to something—in this case, to (len(Z)-1-r)**2 + c**2.
That "something" is just a function on r and c values. Which we can get by creating aranges. In particular, arange(5) is just an array of the numbers 0, 1, 2, 3, 4, and arange(5)**2 is an array of the squares 0, 1, 4, 9, 16.
The only problem is that to get a 3x5 array out of this, we have to elementwise add two 2D arrays, a 3x1 array and a 1x5 array, vice-versa, but we've got two 1D arrays from arange. Well, we can reshape one of them:
Z_old = (3 - 1 - np.arange(3))**2 + (np.arange(5)**2).reshape((5, 1))
You can, of course, simplify this further (you obviously don't need 3 - 1, and you can just add a new axis without reshape), but hopefully this shows directly how it corresponds to your original code.