Here is my problem : I manipulate 432*46*136*136 grids representing time*(space) encompassed in numpy arrays with numpy and python. I have one array alt, which encompasses the altitudes of the grid points, and another array temp which stores the temperature of the grid points.
It is problematic for a comparison : if T1 and T2 are two results, T1[t0,z0,x0,y0] and T2[t0,z0,x0,y0] represent the temperature at H1[t0,z0,x0,y0] and H2[t0,z0,x0,y0] meters, respectively. But I want to compare the temperature of points at the same altitude, not at the same grid point.
Hence I want to modify the z-axis of my matrices to represent the altitude and not the grid point. I create a function conv(alt[t,z,x,y]) which attributes a number between -20 and 200 to each altitude. Here is my code :
def interpolation_extended(self,temp,alt):
[t,z,x,y]=temp.shape
new=np.zeros([t,220,x,y])
for l in range(0,t):
for j in range(0,z):
for lat in range(0,x):
for lon in range(0,y):
new[l,conv(alt[l,j,lat,lon]),lat,lon]=temp[l,j,lat,lon]
return new
But this takes definitely too much time, I can't work this it. I tried to write it using universal functions with numpy :
def interpolation_extended(self,temp,alt):
[t,z,x,y]=temp.shape
new=np.zeros([t,220,x,y])
for j in range(0,z):
new[:,conv(alt[:,j,:,:]),:,:]=temp[:,j,:,:]
return new
But that does not work. Do you have any idea of doing this in python/numpy without using 4 nested loops ?
Thank you
I can't really try the code since I don't have your matrices, but something like this should do the job.
First, instead of declaring conv as a function, get the whole altitude projection for all your data:
conv = np.round(alt / 500.).astype(int)
Using np.round, the numpys version of round, it rounds all the elements of the matrix by vectorizing operations in C, and thus, you get a new array very quickly (at C speed). The following line aligns the altitudes to start in 0, by shifting all the array by its minimum value (in your case, -20):
conv -= conv.min()
the line above would transform your altitude matrix from [-20, 200] to [0, 220] (better for indexing).
With that, interpolation can be done easily by getting multidimensional indices:
t, z, y, x = np.indices(temp.shape)
the vectors above contain all the indices needed to index your original matrix. You can then create the new matrix by doing:
new_matrix[t, conv[t, z, y, x], y, x] = temp[t, z, y, x]
without any loop at all.
Let me know if it works. It might give you some erros since is hard for me to test it without data, but it should do the job.
The following toy example works fine:
A = np.random.randn(3,4,5) # Random 3x4x5 matrix -- your temp matrix
B = np.random.randint(0, 10, 3*4*5).reshape(3,4,5) # your conv matrix with altitudes from 0 to 9
C = np.zeros((3,10,5)) # your new matrix
z, y, x = np.indices(A.shape)
C[z, B[z, y, x], x] = A[z, y, x]
C contains your results by altitude.
Related
I am doing a multiplication of two xarrays in python such as follows:
z = x * y where x and y are xarrays with dimensions [30,33,720,1440] for x and [33,720,1440] for y. The coordinates along dimensions 1, 2 and 3 for x match the coordinates along dimensions 0, 1 and 2 for y, and have the same dimension names (pressure, latitude, longitude). Strangely, the output, z has dimensions [30,33,630,1237].
I have figured out this is caused by one of the latitude arrays differing from the other by an extremely small amount, -1.42108547e-14, at 90 points, which is practically the no difference. A similar thing happens for the longitude. Any ideas on how to eliminate this difference or force xarray to ignore it? (I'd rather not do the multiplication with numpy).
In case you are wondering, I did try
x.assign_coords(lat=y.lat,lon=y.lon)
I don't know why that didn't work. Maybe its because one coordinate array has a different type (float32 vs float64)?
This works.
y_new = x[0,:,:,:].copy()
y_new.values = y.values
z = x * y_new
Let's say I have two arrays X and Y of floats (same length len) and a 2D array img (grayscale Image).
I want to calculate the values of img[X[i]][Y[i]] for i in {0..len} with a good approximation.
Is it enough to just convert X and Y to arrays of integers? Or is there a nice interpolation function that gives better results? (I looked for them but there are so many I got confused).
Thanks
import scipy.interpolate
y_interp = scipy.interpolate.interp1d(x, y)
print(y_interp(5.0))
scipy.interpolate.interp1d does linear interpolation by and can be customized to handle error conditions.
Just using python I would do something like this:
# Calculate tops (just in case the rounding goes over this)
x_top, y_top = len(img) - 1, len(img[0]) - 1
for x, y in zip(map(round, X), map(round, Y)):
x, y = min(x, x_top), min(y, y_top) # Check tops
val = img[x][y] # Here is your value
map(round, X) applies the function round to each element
zip takes to iterators and returns the elements in pairs
I want to generate a four dimensional array with dimensions (dim,N,N,N). The first component ndim =3 and N corresponds to the grid length. How can one elegantly generate such an array using python ?
here is my 'ugly' implementation:
qvec=np.zeros([ndim,N,N,N])
freq = np.arange(-(N-1)/2.,+(N+1)/2.)
x, y, z = np.meshgrid(freq[range(N)], freq[range(N)], freq[range(N)],indexing='ij')
qvec[0,:,:,:]=x
qvec[1,:,:,:]=y
qvec[2,:,:,:]=z
Your implementation looks good enough to me. However, here are some improvements to make it prettier:
qvec=np.empty([ndim,N,N,N])
freq = np.arange(-(N-1)/2.,+(N+1)/2.)
x, y, z = np.meshgrid(*[freq]*ndim, indexing='ij')
qvec[0,...]=x # qvec[0] = x
qvec[1,...]=y # qvec[1] = y
qvec[2,...]=z # qvec[2] = z
The improvements are:
Using numpy.empty() instead of numpy.zeros()
Getting rid of the range(N) indexing since that would give the same freq array
Using iterable unpacking and utilizing ndim
Using the ellipsis notation for dimensions (this is also not needed)
So, after incorporating all of the above points, the below piece of code would suffice:
qvec=np.empty([ndim,N,N,N])
freq = np.arange(-(N-1)/2.,+(N+1)/2.)
x, y, z = np.meshgrid(*[freq]*ndim, indexing='ij')
qvec[0:ndim] = x, y, z
Note: I'm assuming N is same since you used same variable name.
What I am trying to do is take a numpy array representing 3D image data and calculate the hessian matrix for every voxel. My input is a matrix of shape (Z,X,Y) and I can easily take a slice along z and retrieve a single original image.
gx, gy, gz = np.gradient(imgs)
gxx, gxy, gxz = np.gradient(gx)
gyx, gyy, gyz = np.gradient(gy)
gzx, gzy, gzz = np.gradient(gz)
And I can access the hessian for an individual voxel as follows:
x = 100
y = 100
z = 63
H = [[gxx[z][x][y], gxy[z][x][y], gxz[z][x][y]],
[gyx[z][x][y], gyy[z][x][y], gyz[z][x][y]],
[gzx[z][x][y], gzy[z][x][y], gzz[z][x][y]]]
But this is cumbersome and I can't easily slice the data.
I have tried using reshape as follows
H = H.reshape(Z, X, Y, 3, 3)
But when I test this by retrieving the hessian for a specific voxel the, the value returned from the reshaped array is completely different than the original array.
I think I could use zip somehow but I have only been able to find that for making lists of tuples.
Bonus: If there's a faster way to accomplish this please let me know, I essentially need to calculate the three eigenvalues of the hessian matrix for every voxel in the 3D data set. Calculating the hessian values is really fast but finding the eigenvalues for a single 2D image slice takes about 20 seconds. Are there any GPUs or tensor flow accelerated libraries for image processing?
We can use a list comprehension to get the hessians -
H_all = np.array([np.gradient(i) for i in np.gradient(imgs)]).transpose(2,3,4,0,1)
Just to give it a bit of explanation : [np.gradient(i) for i in np.gradient(imgs)] loops through the two levels of outputs from np.gradient calls, resulting in a (3 x 3) shaped tensor at the outer two axes. We need these two as the last two axes in the final output. So, we push those at the end with the transpose.
Thus, H_all holds all the hessians and hence we can extract our specific hessian given x,y,z, like so -
x = 100
y = 100
z = 63
H = H_all[z,y,x]
I am trying to set up some code for doing some numerical calculations on 3D vector fields such as electric or magnetic fields. I am having trouble setting up my meshes in the way I would like.
Consider this program:
import numpy as np
X1, Y1, Z1 = np.meshgrid(np.linspace(-10,10,10),np.linspace(-10,10,10),np.linspace(-10,10,10))
def scalarf(x,y,z):
return x**2 + y**2 + z**2
def vectorf(x,y,z):
return np.array([y,x,z])
def afunc(p,v):
return np.cross(p,v)
V = scalarf(X1,Y1,Z1)
F = vectorf(X1,Y1,Z1)
# This line clearly not working
F2 = afunc(F,np.array([1,0,0]))
print (V.shape)
print (F.shape)
The output of this gives (10,10,10) for V and (3,10,10,10) for F. So V is a 10x10x10 array of scalar values as intended. But for F what I wanted was a 10x10x10 array of 3 element arrays representing mathematical 3D vectors. Instead I have a 3 element array containing a 10x10x10 array as each of its elements. So I'm guessing I want a (10,10,10,3) shape. Ultimately I want to be able to (for ex) run functions like afun in the above. Again here the intention is that F2 would now be a new 10x10x10 array of vectors. At the moment it just fails because I guess its trying to perform a cross product with the 10x10x10 array and the fixed 3d vector in the function call.
Am I going about this in remotely the right way? Is there another way of creating a 3D array of vectors? BTW I have also tried using mgrids with much the same result.
Any help pointing me in the right direction much appreciated.
Use np.stack with the axis keyword
def vectorf(x, y, z):
return np.stack((y, x, z), axis = -1)