We Keep Coding

Most efficient way to calculate point wise surface normal from a numpy grid

say we have a 2D grid that is projected on a 3D surface, resulting in a 3D numpy array, like the below image. What is the most efficient way to calculate a surface normal for each point of this grid?

I can give you an example with simulated data:
I showed your way, with three points. With three points you can always calculate the cross product to get the perpendicular vector based on the two vectors created from three points. Order does not matter.
I took the liberty to also add the PCA approach using predefined sklearn functions. You can create your own PCA, good exercise to understand what happens under the hood but this works fine. The benefit of the approach is that it is easy to increase the number of neighbors and you are still able to calculate the normal vector. It is also possible to select the neighbors within a range instead of N nearest neighbors.
If you need more explanation about the working of the code please let me know.
from functools import partial
import numpy as np
from sklearn.neighbors import KDTree
from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
# Grab some test data.
X, Y, Z = axes3d.get_test_data(0.25)
X, Y, Z = map(lambda x: x.flatten(), [X, Y, Z])
plt.plot(X, Y, Z, '.')
plt.show(block=False)
data = np.array([X, Y, Z]).T
tree = KDTree(data, metric='minkowski') # minkowki is p2 (euclidean)
# Get indices and distances:
dist, ind = tree.query(data, k=3) #k=3 points including itself
def calc_cross(p1, p2, p3):
v1 = p2 - p1
v2 = p3 - p1
v3 = np.cross(v1, v2)
return v3 / np.linalg.norm(v3)
def PCA_unit_vector(array, pca=PCA(n_components=3)):
pca.fit(array)
eigenvalues = pca.explained_variance_
return pca.components_[ np.argmin(eigenvalues) ]
combinations = data[ind]
normals = list(map(lambda x: calc_cross(*x), combinations))
# lazy with map
normals2 = list(map(PCA_unit_vector, combinations))
## NEW ##
def calc_angle_with_xy(vectors):
'''
Assuming unit vectors!
'''
l = np.sum(vectors[:,:2]**2, axis=1) ** 0.5
return np.arctan2(vectors[:, 2], l)
dist, ind = tree.query(data, k=5) #k=3 points including itself
combinations = data[ind]
# map with functools
pca = PCA(n_components=3)
normals3 = list(map(partial(PCA_unit_vector, pca=pca), combinations))
print( combinations[10] )
print(normals3[10])
n = np.array(normals3)
n[calc_angle_with_xy(n) < 0] *= -1
def set_axes_equal(ax):
'''Make axes of 3D plot have equal scale so that spheres appear as spheres,
cubes as cubes, etc.. This is one possible solution to Matplotlib's
ax.set_aspect('equal') and ax.axis('equal') not working for 3D.
Input
ax: a matplotlib axis, e.g., as output from plt.gca().
FROM: https://stackoverflow.com/questions/13685386/matplotlib-equal-unit-length-with-equal-aspect-ratio-z-axis-is-not-equal-to
'''
x_limits = ax.get_xlim3d()
y_limits = ax.get_ylim3d()
z_limits = ax.get_zlim3d()
x_range = abs(x_limits[1] - x_limits[0])
x_middle = np.mean(x_limits)
y_range = abs(y_limits[1] - y_limits[0])
y_middle = np.mean(y_limits)
z_range = abs(z_limits[1] - z_limits[0])
z_middle = np.mean(z_limits)
# The plot bounding box is a sphere in the sense of the infinity
# norm, hence I call half the max range the plot radius.
plot_radius = 0.5*max([x_range, y_range, z_range])
ax.set_xlim3d([x_middle - plot_radius, x_middle + plot_radius])
ax.set_ylim3d([y_middle - plot_radius, y_middle + plot_radius])
ax.set_zlim3d([z_middle - plot_radius, z_middle + plot_radius])
u, v, w = n.T
fig = plt.figure()
ax = fig.add_subplot(projection='3d')
# ax.set_aspect('equal')
# Make the grid
ax.quiver(X, Y, Z, u, v, w, length=10, normalize=True)
set_axes_equal(ax)
plt.show()

The surface normal for a point cloud is not well defined. One way to define them is from the surface normal of a reconstructed mesh using triangulation (which can introduce artefacts regarding you specific input). A relatively simple and fast solution is to use VTK to do that, and more specifically, vtkSurfaceReconstructionFilter and vtkPolyDataNormals . Regarding your needs, it might be useful to apply other filters.

How to use a smooth curve to link points approximately distributing in a circle?

I have a set of twelve points, which center at (0, 0) and distribute approximately in a circle, at the interval of 30 degrees, shown in the image.
The twelve points
I want to use a smooth curve to link (go through) them like the image below (I draw the red line by hand).
a hand-drawn curve in red
I want to make it in python or matlab. I have tried some interpolation methods for the upper half and lower half separately, and wanted to combine them as a complete curve. However, the results always overshoot.
Thank you for any suggestions!

I think the key here is to note that you have to consider it as a parametrized curve in 2d, not just a 1d to 2d function. Furthermore since it should be something like a circle, you need an interpolation method that supports periodic boundaries. Here are two methods for which this applies:
% set up toy data
t = linspace(0, 2*pi, 10);
t = t(1:end-1);
a = 0.08;
b = 0.08;
x = cos(t+a*randn(size(t))) + b*randn(size(t));
y = sin(t+a*randn(size(t))) + b*randn(size(t));
plot(x, y, 'ok');
% fourier interpolation
z = x+1i*y;
y = interpft(z, 200);
hold on
plot(real(y), imag(y), '-.r')
% periodic spline interpolation
z = [z, z(1)];
n = numel(z);
t = 1:n;
pp = csape(t, z, 'periodic');
ts = linspace(1, n, 200);
y = ppval(pp, ts);;
plot(real(y), imag(y), ':b');

Thank for suggestions from #flawr. According to the answer from #flawr, I implemented the periodic spline interpolation in python (still working on implementing fourier interpolation in python.). Here is the code:
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import CubicSpline
# set up toy data
t = np.linspace(0, 2*np.pi, 10)
t = t[0:-1]
a = 0.08
b = 0.08
x = np.cos(t + a * np.random.normal(size=len(t))) + b * np.random.normal(size=len(t))
y = np.sin(t + a * np.random.normal(size=len(t))) + b * np.random.normal(size=len(t))
plt.scatter(x, y)
# periodic spline interpolation
z = []
for idx in range(len(x)):
z.append(complex(x[idx], y[idx]))
z.append(complex(x[0], y[0]))
len_z = len(z)
t = [i for i in range(len_z)]
cs = CubicSpline(t, z, bc_type='periodic')
xs = np.linspace(0, len_z, 200)
y_new = cs(xs)
plt.plot(y_new.real, y_new.imag)
plt.show()

Test whether points are inside ellipses, without using Matplotlib?

I'm working on a Python-based data analysis. I have some x-y data points, and some ellipses, and I want to determine whether points are inside any of the ellipses. The way that I've been doing this works, but it's kludgy. As I think about distributing my software to other people, I find myself wanting a cleaner way.
Right now, I'm using matplotlib.patches.Ellipse objects. Matplotlib Ellipses have a useful method called contains_point(). You can work in data coordinates on a Matplotlib Axes object by calling Axes.transData.transform().
The catch is that I have to create a Figure and an Axes object to hold the Ellipses. And when my program runs, an annoying Matplotlib Figure object will get rendered, showing the Ellipses, which I don't actually need to see. I have tried several methods to suppress this output. I have succeeded in deleting the Ellipses from the Axes, using Axes.clear(), resulting in an empty graph. But I can't get Matplolib's pyplot.close(fig_number) to delete the Figure itself before calling pyplot.show().
Any advice is appreciated, thanks!

Inspired by how a carpenter draws an ellipse using two nails and a piece of string, here is a numpy-friendly implementation to test whether points lie inside given ellipses.
One of the definitions of an ellipse, is that the sum of the distances to the two foci is constant, equal to the width (or height if it would be larger) of the ellipse. The distance between the center and the foci is sqrt(a*a - b*b), where a and b are half of the width and height. Using that distance and rotation by the desired angle finds the locations of the foci. numpy.linalg.norm can be used to calculate the distances using numpy's efficient array operations.
After the calculations, a plot is generated to visually check whether everything went correct.
import numpy as np
from numpy.linalg import norm # calculate the length of a vector
x = np.random.uniform(0, 40, 20000)
y = np.random.uniform(0, 20, 20000)
xy = np.dstack((x, y))
el_cent = np.array([20, 10])
el_width = 28
el_height = 17
el_angle = 20
# distance between the center and the foci
foc_dist = np.sqrt(np.abs(el_height * el_height - el_width * el_width) / 4)
# vector from center to one of the foci
foc_vect = np.array([foc_dist * np.cos(el_angle * np.pi / 180), foc_dist * np.sin(el_angle * np.pi / 180)])
# the two foci
el_foc1 = el_cent + foc_vect
el_foc2 = el_cent - foc_vect
# for each x,y: calculate z as the sum of the distances to the foci;
# np.ravel is needed to change the array of arrays (of 1 element) into a single array
z = np.ravel(norm(xy - el_foc1, axis=-1) + norm(xy - el_foc2, axis=-1) )
# points are exactly on the ellipse when the sum of distances is equal to the width
# z = np.where(z <= max(el_width, el_height), 1, 0)
# now create a plot to check whether everything makes sense
from matplotlib import pyplot as plt
from matplotlib import patches as mpatches
fig, ax = plt.subplots()
# show the foci as red dots
plt.plot(*el_foc1, 'ro')
plt.plot(*el_foc2, 'ro')
# create a filter to separate the points inside the ellipse
filter = z <= max(el_width, el_height)
# draw all the points inside the ellipse with the plasma colormap
ax.scatter(x[filter], y[filter], s=5, c=z[filter], cmap='plasma')
# draw all the points outside with the cool colormap
ax.scatter(x[~filter], y[~filter], s=5, c=z[~filter], cmap='cool')
# add the original ellipse to verify that the boundaries match
ellipse = mpatches.Ellipse(xy=el_cent, width=el_width, height=el_height, angle=el_angle,
facecolor='None', edgecolor='black', linewidth=2,
transform=ax.transData)
ax.add_patch(ellipse)
ax.set_aspect('equal', 'box')
ax.autoscale(enable=True, axis='both', tight=True)
plt.show()

The simplest solution here is to use shapely.
If you have an array of shape Nx2 containing a set of vertices (xy) then it is trivial to construct the appropriate shapely.geometry.polygon object and check if an arbitrary point or set of points (points) is contained within -
import shapely.geometry as geom
ellipse = geom.Polygon(xy)
for p in points:
if ellipse.contains(geom.Point(p)):
# ...
Alternatively, if the ellipses are defined by their parameters (i.e. rotation angle, semimajor and semiminor axis) then the array containing the vertices must be constructed and then the same process applied. I would recommend using the polar form relative to center as this is the most compatible with how shapely constructs the polygons.
import shapely.geometry as geom
from shapely import affinity
n = 360
a = 2
b = 1
angle = 45
theta = np.linspace(0, np.pi*2, n)
r = a * b / np.sqrt((b * np.cos(theta))**2 + (a * np.sin(theta))**2)
xy = np.stack([r * np.cos(theta), r * np.sin(theta)], 1)
ellipse = affinity.rotate(geom.Polygon(xy), angle, 'center')
for p in points:
if ellipse.contains(geom.Point(p)):
# ...
This method is advantageous because it supports any properly defined polygons - not just ellipses, it doesn't rely on matplotlib methods to perform the containment checking, and it produces a very readable code (which is often important when "distributing [one's] software to other people").
Here is a complete example (with added plotting to show it working)
import shapely.geometry as geom
from shapely import affinity
import matplotlib.pyplot as plt
import numpy as np
n = 360
theta = np.linspace(0, np.pi*2, n)
a = 2
b = 1
angle = 45.0
r = a * b / np.sqrt((b * np.cos(theta))**2 + (a * np.sin(theta))**2)
xy = np.stack([r * np.cos(theta), r * np.sin(theta)], 1)
ellipse = affinity.rotate(geom.Polygon(xy), angle, 'center')
x, y = ellipse.exterior.xy
# Create a Nx2 array of points at grid coordinates throughout
# the ellipse extent
rnd = np.array([[i,j] for i in np.linspace(min(x),max(x),50)
for j in np.linspace(min(y),max(y),50)])
# Filter for points which are contained in the ellipse
res = np.array([p for p in rnd if ellipse.contains(geom.Point(p))])
plt.plot(x, y, lw = 1, color='k')
plt.scatter(rnd[:,0], rnd[:,1], s = 50, color=(0.68, 0.78, 0.91)
plt.scatter(res[:,0], res[:,1], s = 15, color=(0.12, 0.67, 0.71))
plt.show()

Generate profiles through a 2D array at an angle without altering pixels

I'd like to plot two profiles through the highest intensity point in a 2D numpy array, which is an image of a blob (i.e. a line through the semi-major axis, and another line through the semi-minor axis). The blob is rotated at an angle theta counterclockwise from the standard x-axis and is asymmetric.
It is a 600x600 array with a max intensity of 1 (at only one pixel) that is located right at the center at (300, 300). The angle rotation from the x-axis (which then gives the location of the semi-major axis when rotated by that angle) is theta = 89.54 degrees. I do not want to use scipy.ndimage.rotate because it uses spline interpolation, and I do not want to change any of my pixel values. But I suppose a nearest-neighbor interpolation method would be okay.
I tried generating lines corresponding to the major and minor axes across the image, but the result was not right at all (the peak was far less than 1), so maybe I did something wrong. The code for this is below:
import numpy as np
import matplotlib.pyplot as plt
from scipy import ndimage
def profiles_at_angle(image, axis, theta):
theta = np.deg2rad(theta)
if axis == 'major':
x_0, y_0 = 0, 300-300*np.tan(theta)
x_1, y_1 = 599, 300+300*np.tan(theta)
elif axis=='minor':
x_0, y_0 = 300-300*np.tan(theta), 599
x_1, y_1 = 300+300*np.tan(theta), -599
num = 600
x, y = np.linspace(x_0, x_1, num), np.linspace(y_0, y_1, num)
z = ndimage.map_coordinates(image, np.vstack((x,y)))
fig, axes = plt.subplots(nrows=2)
axes[0].imshow(image, cmap='gray')
axes[0].axis('image')
axes[1].plot(z)
plt.xlim(250,350)
plt.show()
profiles_at_angle(image, 'major', theta)
Did I do something obviously wrong in my code above? Or how else can I accomplish this? Thank you.
Edit: Here are some example images. Sorry for the bad quality; my browser crashed every time I tried uploading them anywhere so I had to take photos of the screen.
Figure 1: This is the result of my code above, which is clearly wrong since the peak should be at 1. I'm not sure what I did wrong though.
Figure 2: I made this plot below by just taking the profiles through the standard x and y axes, ignoring any rotation (this only looks good coincidentally because the real angle of rotation is so close to 90 degrees, so I was able to just switch the labels and get this). I want my result to look something like this, but taking the correction rotation angle into account.
Edit: It could be useful to run tests on this method using data very much like my own (it's a 2D Gaussian with nearly the same parameters):
image = np.random.random((600,600))
def generate(data_set):
xvec = np.arange(0, np.shape(data_set)[1], 1)
yvec = np.arange(0, np.shape(data_set)[0], 1)
X, Y = np.meshgrid(xvec, yvec)
return X, Y
def gaussian_func(xy, x0, y0, sigma_x, sigma_y, amp, theta, offset):
x, y = xy
a = (np.cos(theta))**2/(2*sigma_x**2) + (np.sin(theta))**2/(2*sigma_y**2)
b = -np.sin(2*theta)/(4*sigma_x**2) + np.sin(2*theta)/(4*sigma_y**2)
c = (np.sin(theta))**2/(2*sigma_x**2) + (np.cos(theta))**2/(2*sigma_y**2)
inner = a * (x-x0)**2
inner += 2*b*(x-x0)*(y-y0)
inner += c * (y-y0)**2
return (offset + amp * np.exp(-inner)).ravel()
xx, yy = generate(image)
image = gaussian_func((xx.ravel(), yy.ravel()), 300, 300, 5, 4, 1, 1.56, 0)
image = np.reshape(image, (600, 600))

This should do it for you. You just did not properly compute your lines.
theta = 65
peak = np.argwhere(image==1)[0]
x = np.linspace(peak[0]-100,peak[0]+100,1000)
y = lambda x: (x-peak[1])*np.tan(np.deg2rad(theta))+peak[0]
y_maj = np.linspace(y(peak[1]-100),y(peak[1]+100),1000)
y = lambda x: -(x-peak[1])/np.tan(np.deg2rad(theta))+peak[0]
y_min = np.linspace(y(peak[1]-100),y(peak[1]+100),1000)
del y
z_min = scipy.ndimage.map_coordinates(image, np.vstack((x,y_min)))
z_maj = scipy.ndimage.map_coordinates(image, np.vstack((x,y_maj)))
fig, axes = plt.subplots(nrows=2)
axes[0].imshow(image)
axes[0].plot(x,y_maj)
axes[0].plot(x,y_min)
axes[0].axis('image')
axes[1].plot(z_min)
axes[1].plot(z_maj)
plt.show()

Plot Surface instead of parametric curve

I am working on using the forward difference scheme for numerically solving the diffusion function in one dimension. My final plot of the solution should be a surface where the solution u(x,t) is plotted over a grid of x and t values. I have the problem solved, but I can't get the data to be plotted with the grid representation.
I can think of 2 ways to fix this:
1.) My x and t arrays should be one dimensional, but my u array should be a 2D array. Ultimately, I want a square matrix for u, but I am having a hard time coding that. Currently I have a 1D array for u. Here is the code where u is populated.
u = zeros(Nx+1) # unknown u at new time level
u_1 = zeros(Nx+1) # u at the previous time level
# Set initial condition u(x,0) = I(x)
for i in range(0, Nx+1):
#set initial u's to I(xi)
u_1[i] = 25-x[i]**2
for n in range(0, Nt):
# Compute u at inner mesh points
for i in range(1, Nx):
u[i] = u_1[i] + F*(u_1[i-1] - 2*u_1[i] + u_1[i+1])
2.) The above code returns a 1D array for u, is there a way to plot a 3D surface with 3 1D arrays for x,y,z?

Well..., there is a lot of information you haven't provided. For instance you said you wanted a x,y,z plot but haven't said what the x, y and z should be in the context of your plot. Also z is typically z(x,y).
The following recipe assumes a t and x, and u(t,x) as variables to be put into a surface. I imagine is not exactly your idea but it should be adaptable to your exercise:
EDIT: Also your code (which is in the function computeU in this recipe) had a loop for Nt that does not seem to do anything. I've removed it for the purpose of this example.
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
import matplotlib.pyplot as plt
import numpy as np
def computeU(Nx,x,F,Nt):
u = np.zeros(Nx+1) # unknown u at new time level
u_1 = np.zeros(Nx+1) # u at the previous time level
# Set initial condition u(x,0) = I(x)
for i in range(0, Nx+1):
#set initial u's to I(xi)
u_1[i] = 25-x[i]**2
#for n in range(0, Nt): # I'm not sure what this is doing. It has no effect.
# Compute u at inner mesh points
for i in range(1, Nx):
u[i] = u_1[i] + F*(u_1[i-1] - 2*u_1[i] + u_1[i+1])
return np.hstack((u[:,np.newaxis],u_1[:,np.newaxis]))
Nx = 10
F = 3
Nt = 5
x = np.arange(11)
t = np.arange(2)
X,Y = np.meshgrid(t,x)
Z = computeU(Nx,x,F,Nt)
fig = plt.figure()
ax = fig.gca(projection='3d')
surf = ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=cm.coolwarm,linewidth=0, antialiased=False)
plt.show()
Notice how I've used meshgrid to build new t,x (from 1D arrays) to be mapped against your stack of U arrays (which will have the same shape of X,Y - the new t,x). The result is this: