We Keep Coding

Fastest way to return interpolated points within a meshgrid

I have meshgrid points (X,Y,Z) and an array of points (xi,yi) in which I want to return the interpolated values zi.
The current approach would be:
from scipy import interpolate
ip = interpolate.interp2d(X, Y, Z); zi = ip(xi, yi)
However this process seems to be very slow for my dataset (xi,yi) size 1001
Is there a more efficient method?

I found that the RectBivariateSpline function was significantly faster.
ip = RectBivariateSpline(xl, yl, Zhat.T)(x, y,grid=False)

Finding the correct x and y widths of 2D array for Gaussian fit

I'd like to fit my 2D numpy array (image) data to a Gaussian. I've read a lot of examples using scipy.optimize, and I've tried but the fit has never been good -- this is probably because my background is non-zero, and sometimes I have other peaks too. I think it might be easier for me to simply generate a Gaussian that has the parameters of the correct peak. I already have the subpixel centroid coordinates x and y of the peak I want, and can easily get the amplitude of the peak with data[y][x], although I guess I would have to round the coordinates. What I'm stuck on now is the x and y widths. My Gaussian function looks like this:
import numpy as np
def gaussian_func(xy, x0, y0, width_x, width_y, amp): #x0 and y0 are the centroid coordinates
x = xy[0]
y = xy[1]
offset = np.min(data) #should this be a median value of the background instead?
a = 1/(2*width_x**2)
c = 1/(2*width_y**2)
exp_term = a*(x-x0)**2 + c*(y-y0)**2
return (offset + amp * np.exp(-exp_term)).ravel()
x, y = np.arange(0, np.shape(data)[1], 1), np.arange(0, np.shape(data)[0], 1)
xx, yy = np.meshgrid(x, y)
gaussian = gaussian_func((xx, yy), x0, y0, width_x, width_y, amp)
gaussian = np.reshape(gaussian, np.shape(data))
So I'm basically just confused on what to insert for width_x and width_y. I know these terms are supposed to be interchangeable with the standard deviations in x and y, but when I tried simply using np.std(data), I got bad results. Do the widths correspond to the actual physical widths of the peak? If so, how do I find those? Thanks!

Masking a 3D numpy array with a tilted disc

I'm working in Python2.7 with 3D numpy arrays, and trying to retrieve only pixels who fall on a 2D tilted disc.
Here is my code to plot the border of the disc (= a circle) I am interested in
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
#creating a 3d numpy array (empty in this example, but will represent a binary 3D image in my application)
space=np.zeros((40,40,20))
r = 8 #radius of the circle
theta = np.pi / 4 # "tilt" of the circle
phirange = np.linspace(0, 2 * np.pi) #to make a full circle
#center of the circle
center=[20,20,10]
#computing the values of the circle in spherical coordinates and converting them
#back to cartesian
for phi in phirange:
x = r * np.cos(theta) * np.cos(phi) + center[0]
y= r*np.sin(phi) + center[1]
z= r*np.sin(theta)* np.cos(phi) + center[2]
space[int(round(x)),int(round(y)),int(round(z))]=1
x,y,z = space.nonzero()
#plotting
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(x, y, z, zdir='z', c= 'red')
plt.show()
The plot gives the following figure :
which is a good start, but now I want a way to retrieve only the values of the pixels of space which are located in the disc defined by the circle : the ones in the pink zone in the following image (in my application, space will be a 3D binary image, here it is numpy.zeros() just to be able to plot and show you the disc I want):
How should I procede ?
I guess there is some numpy masking involved, an I understand how you would do it in 2D (like this question) but I'm having trouble applying this to 3D.

One easy way would be to calculate the normal vector to your disc plane. You can use your spherical coordinates for that. Be sure not to add the centre, set phi at zero and swap cos and sin theta, also stick a minus sign to the sin.
lets call that vector v. The plane is given by v0*x0 + v1*x1 + v2*x2 == c you can calculate c by inserting a point from your circle for x.
Next you can make a 2d grid for x0 and x1 and solve for x2. this gives you the height x2 as a function of the x0, x1 mesh. for these points you can calculate the distance from your disc centre and discard the points that are too far off. This you would indeed do using a mask.
Finally, depending on how precisely you want to plot you could round the x2 values to grid units, but for example for a surface plot I wouldn't do that.
To get a 3d mask as you describe you would round x2 and then starting from an all zero space set the disc pixels using space[x0, x1, x2] = True. This assumes that you have masked x0, x1, x2 as described earlier.

Well that is a math problem, you should ask it in the Mathematics Stack Exchange site.
From my perspective, you should first find the surface your disc is in, and do the area calculation within that surface, by, for example, the method you mentioned in the linked question.
numpy or matplotlib here definitely do not responsible for the projection, you do.
Without clearly point out which (or which kind of) surface they are in, and the equation does not guarantee it is a plane, the area does not mean anything.

Basemap streamplot blank sphere

I want to make a streamplot in Basemap module, but I get a blank sphere. Please help me resolve this problem. I use matplotlib 1.3 and ordinary streamplot is working fine.
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.basemap import Basemap
map = Basemap(projection='ortho',lat_0=45,lon_0=-100,resolution='l')
# draw lat/lon grid lines every 30 degrees.
map.drawmeridians(np.arange(0,360,30))
map.drawparallels(np.arange(-90,90,30))
# prepare grids
lons = np.linspace(0, 2*np.pi, 100)
lats = np.linspace(-np.pi/2, np.pi/2, 100)
lons, lats = np.meshgrid(lons, lats)
# parameters for vector field
beta = 0.0
alpha = 1.0
u = -np.cos(lats)*(beta - alpha*np.cos(2.0*lons))
v = alpha*(1.0 - np.cos(lats)**2)*np.sin(2.0*lons)
speed = np.sqrt(u*u + v*v)
# compute native map projection coordinates of lat/lon grid.
x, y = map(lons*180./np.pi, lats*180./np.pi)
# contour data over the map.
cs = map.streamplot(x, y, u, v, latlon = True, color = speed, cmap=plt.cm.autumn, linewidth=0.5)
plt.show()

I can't exactly tell you what's wrong, but from the matplotlib.streamplot manual:
matplotlib.pyplot.streamplot(x, y, u, v, density=1, linewidth=None,
color=None, cmap=None, norm=None, arrowsize=1, arrowstyle=u'-|>',
minlength=0.1, transform=None, zorder=1, hold=None)¶
Draws streamlines of a vector flow.
x, y : 1d arrays
an evenly spaced grid.
u, v : 2d arrays
x and y-velocities. Number of rows should match length of y, and the number of columns should match x.
Additionally from matplotlib.basemap.streamplot you can read that
If latlon keyword is set to True, x,y are intrepreted as longitude and latitude in degrees.
Which corresponds to the fact that x and y should be 1D arrays (lat, lon). However in your example x and y are
>>> np.shape(x)
(100, 100)
>>> np.shape(y)
(100, 100)
Then again you call the method map() "to compute native map projection coordinates of lat/lon grid" which is coincidentally the same as the name of your basemap.map. So it depends on which one do you want? Because both will return a value! (or better to say, both will return an error)
Aditionally check out the values you have in your u array. They are of range e-17. While other values are easily in the range e+30. IIRC the way you get streamlines is by solving a differential equation in which points you sent it as values are used as parameters at coordinates you sent. It's not hard to imagine a that while calculating something with these numbers a floating point round-off occurs and you suddenly start getting NaN or 0 values.
Try to scale your example better or if you want to pursue the solution to the end you can try and use np.seterr to get a more detailed idea where it fails.
Sorry I couldn't have been of a bigger help.

Using Radial Basis Functions to Interpolate a Function on a Sphere

First, a bit of background:
I am using spherical harmonics as an example of a function on the surface of a sphere like the front spheres in this image:
I produced one of these spheres, coloured according to the value of the harmonic function at points on its surface. I do this first for a very large number of points, so my function is very accurate. I've called this my fine sphere.
Now that I have my fine sphere, I take a relatively small number of points on the sphere. These are the points I wish to interpolate from, the training data, and I call them interp points. Here are my interp points, coloured to their values, plotted on my fine sphere.
Now, the goal of the project is to use these interp points to train a SciPy Radial Basis Function to interpolate my function on the sphere. I was able to do this using:
# Train the interpolation using interp coordinates
rbf = Rbf(interp.phi, interp.theta, harmonic13_coarse)
# The result of the interpolation on fine coordinates
interp_values = rbf(fine.phi, fine.theta)
Which produced this interpolation, plotted on the sphere:
Hopefully, through this last image, you can see my problem. Notice the line running through the interpolation? This is because the interpolation data has a boundary. The boundary is because I trained the radial basis function using spherical coordinates (boundaries at [0,pi] and [0,2pi]).
rbf = Rbf(interp.phi, interp.theta, harmonic13_coarse)
My goal, and why I'm posting this problem, is to interpolate my function on the surface of the sphere using the x,y,z Cartesian coordinates of the data on the sphere. This way, since spheres don't have boundaries, I won't have this boundary error like I do in spherical coordinates. However, I just can't figure out how to do this.
I've tried simply giving the Rbf function the x,y,z coordinates and the value of the function.
rbf=Rbf(interp.x, interp.y, interp.z, harmonic13_coarse)
interp_values=rbf(fine.x,fine.y,fine.z)
But NumPy throws me a Singular Matrix Error
numpy.linalg.linalg.LinAlgError: singular matrix
Is there any way for me to give Rbf my data sites in Cartesian coordinates, with the function values at each site and have it behave like it does with spherical coordinates but without that boundaries? From the Rbf documentation, there is the attribute norm for defining a different distance norm, could I have to use a spherical distance to get this to work?
I'm pretty much stumped on this. Let me know if you have any ideas for interpolating my function on a sphere without the boundaries of spherical coordinates.
Here is my code in full:
import matplotlib.pyplot as plt
from matplotlib import cm, colors
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
from scipy import special
from scipy.interpolate import Rbf
from collections import namedtuple
from mayavi import mlab
# Nice aliases
pi = np.pi
cos = np.cos
sin = np.sin
# Creating a sphere in Cartesian and Sphereical
# Saves coordinates as named tuples
def coordinates(r, n):
phi, theta = np.mgrid[0:pi:n, 0:2 * pi:n]
Coor = namedtuple('Coor', 'r phi theta x y z')
r = r
x = r * sin(phi) * cos(theta)
y = r * sin(phi) * sin(theta)
z = r * cos(phi)
return Coor(r, phi, theta, x, y, z)
# Creating a sphere
# fine is coordinates on a fine grid
# interp is coordinates on coarse grid for training interpolation
fine = coordinates(1, 100j)
interp = coordinates(1, 5j)
# Defining finection to colour sphere
# Here we are using a spherical harmonic
def harmonic(m, n, theta, phi):
return special.sph_harm(m, n, theta, phi).real
norm = colors.Normalize()
# One example of the harmonic function, for testing
harmonic13_fine = harmonic(1, 3, fine.theta, fine.phi)
harmonic13_coarse = harmonic(1, 3, interp.theta, interp.phi)
# Train the interpolation using interp coordinates
rbf = Rbf(interp.phi, interp.theta, harmonic13_coarse)
# The result of the interpolation on fine coordinates
interp_values = rbf(fine.phi, fine.theta)
rbf=Rbf(interp.x, interp.y, interp.z, harmonic13_coarse)
interp_values=rbf(fine.x,fine.y,fine.z)
#Figure of harmoinc function on sphere in fine cordinates
#Points3d showing interpolation training points coloured to their value
mlab.figure()
vmax, vmin = np.max(harmonic13_fine), np.min(harmonic13_fine)
mlab.mesh(fine.x, fine.y, fine.z, scalars=harmonic13_fine, vmax=vmax, vmin=vmin)
mlab.points3d(interp.x, interp.y, interp.z, harmonic13_coarse,
scale_factor=0.1, scale_mode='none', vmax=vmax, vmin=vmin)
#Figure showing results of rbf interpolation
mlab.figure()
vmax, vmin = np.max(harmonic13_fine), np.min(harmonic13_fine)
mlab.mesh(fine.x, fine.y, fine.z, scalars=interp_values)
# mlab.points3d(interp.x, interp.y, interp.z, scalars, scale_factor=0.1, scale_mode='none',vmax=vmax, vmin=vmin)
mlab.show()

The boundary you see is because you are mapping a closed surface (S2) to an open one (R2). One way or another, you will have boundaries. The local properties of the manifolds are compatible, so it works for most of the sphere, but not the global, you get a line.
The way around it is to use an atlas instead of a single chart. An atlas is a collection of overlapping charts. In the overlapping region, you need to define weights, a smooth function that goes from 0 to 1 on each chart. (Sorry, probably differential geometry was not what you were expecting to hear).
If you don't want to go all the way here, you can notice that your original sphere has an equator where the variance is minimal. You can then rotate your fine sphere and make it coincide with the line. It doesn't solve your problem, but it can certainly mitigate it.

You can change the standard distance:
def euclidean_norm(x1, x2):
return np.sqrt( ((x1 - x2)**2).sum(axis=0) )
by the sphere distance (see, for instance, this question Haversine Formula in Python (Bearing and Distance between two GPS points)).