We Keep Coding

Gaussian RBF Visualization in Python

It is some days that I am trying to visualize the so-called kernel trick resulting from a RBF kernel transformation in a SVC model. Basically, I am trying to map a 2D space to a 3D space in order to let the viewer see how the kernel trick adds a dimension in order to linearly separate the space between two classes.
Following sklearn examples, I managed to plot a 2D example of the trick. However, I feel it is not enough to really grasp what it is happening behind the scences.
Below is what I managed to plot:
I would like to plot the same data on a three dimensional space, representing also the plane that splits the space between the two classes.
I am not asking for the actual code here. Rather, I would like to understand what goes on the axis for the third dimension. Indeed, I think that such axis should be equal to exp(-gamma||x-y||^2). However, due to my poor vector algebra skills, I do not know how to compute it.
Any help would be much appreciated.
Cheers!
UPDATE
The following allowed me to build a new matrix for a 3D plot:
def feature_map_2(X):
return np.asarray((X[:,0], X[:,1], np.exp( -gam*(( X[:,0]**2 + X[:,1]**2 -2*X[:,0]*X[:,1]))))).T
Z = feature_map_2(X)
Where gam = 1/n_features
Then, I computed the boundary as follows:
#SVM
clf = svm.NuSVC(kernel = 'linear', nu=0.5)
clf.fit(Z, y)
w = clf.coef_.flatten()
b = clf.intercept_.flatten()
# create x,y
xx, yy = np.meshgrid(np.linspace(-6,6), np.linspace(-2,2))
# calculate corresponding z
boundary = lambda xx, yy: (-w[0] * xx - w[1] * yy - b) * 1. /w[2]
However, results differ from what one might have expected looking at the 2D plot.

Do you mean SVM model?
https://jgreitemann.github.io/svm-demo
Visualizing 3D may be difficult because you will need to project to the screen bringing again to a 2D image.
To find the plane in the 3D space you simply apply your kernel to make your classes linearly separable and then apply a
Linear SVM
The equation w' x - b = 0 expressed in terms of scalars as w[0] * x[0] + w[1] * x[1] + w[2] * x[2] - b = 0, can be made parametric by choosing element of x (with non-zero coefficient). For instance if w[2] != 0 you can write the plane as.
(U, V, (b - w[0] * U - w[1] * V) / w[2])
And this may be used in common surface plot functions, for instance in python it would be like this
U, V = meshgrid(np.linspace(-1, 1, 100), np.linspace(-1, 1, 100))
plt.pcolormesh(U, V, (b - w[0] * U - w[1] * V) / w[2]);

Fool-proof algorithm for uniformly distributing points on a sphere's surface?

I've been trying to generate points on the surface of a sphere of radius "inner_radius", such that they're uniformly spread out. The algorithm works as expected for a radius of 1, but generates lesser than expected points for greater radii.
I have looked through similar questions on here, but they seem to be for generating points throughout the volume and not just on the surface of the sphere.
import numpy as np
PI=np.pi
def spherical_to_cartesian(pol_ang,azim_ang,radius): #This function converts given spherical coordinates (theta, phi and radius) to cartesian coordinates.
return np.array((radius*np.sin(pol_ang) * np.cos(azim_ang),
radius*np.sin(pol_ang) * np.sin(azim_ang),
radius*np.cos(pol_ang))
)
def get_electron_coordinates_list(inner_radius,electron_count):
#Algorithm used was mostly taken from https://www.cmu.edu/biolphys/deserno/pdf/sphere_equi.pdf . Explanations in code added by me.
electron_coordinate_list=[]
inner_area=4*(PI*inner_radius**2)
area_per_electron=inner_area/electron_count
pseudo_length_per_electron=np.sqrt(area_per_electron) #This is the side length of a square where the area of it is the area per electron on the sphere.
#Now, we need to get a value of angular space, such that angular space between electrons on latitude and longitude per electron is equal
#As a first step to obtaining this, we must make another value holding a whole number approximation of the ratio between PI and the pseudo_length. This will give the number of
#possible latitudes.
possible_count_of_lats=np.round(PI/pseudo_length_per_electron)
approx_length_per_electron_lat=PI/possible_count_of_lats #This is the length between electrons on a latitude
approx_length_per_electron_long=area_per_electron/approx_length_per_electron_lat #This is the length between electrons on a longitude
for electron_num_lat in range(int(possible_count_of_lats.item())): #The int(somenumpyvalue.item()) is used because Python cannot iterate over a numpy integer and it must be converted to normal int.
pol_ang=PI*(electron_num_lat+0.5)/possible_count_of_lats #The original algorithm recommended pol_ang=PI*(electron_num_lat+0.5)/possible_count_of_lats. The 0.5 appears to be added in order to get a larger number of coordinates.
#not sure if removing the 0.5 affects results. It didnt do so drastically, so what gives? Anyway, this gets the polar angle as PI*(latitudenumber)/totalnumberoflatitudes.
possible_count_of_longs=np.round(2*PI*np.sin(pol_ang)/approx_length_per_electron_long)
for electron_num_long in range(int(possible_count_of_longs.item())):
azim_ang=(2*PI)*(electron_num_long)/possible_count_of_longs #This gets the azimuthal angle as 2PI*longitudenumber/totalnumberoflongitudes
electron_coordinate=spherical_to_cartesian(pol_ang, azim_ang,inner_radius) #Converts the recieved spherical coordinates to cartesian so Manim can easily handle them.
electron_coordinate_list.append(electron_coordinate) #Add this coordinate to the electron_coordinate_list
print("Got coordinates: ",electron_coordinate) #Print the coordinate recieved.
print(len(electron_coordinate_list)," points generated.") #Print the amount of electrons will exist. Comment these two lines out if you don't need the data.
return electron_coordinate_list
get_electron_coordinates_list(1,100)
get_electron_coordinates_list(2,100)
Spherical_to_Cartesian() does nothing other than convert the spherical points to Cartesian.
For 100 points and radius 1, it generates 99 points.
But, only 26 points are made if the radius is 2 and 100 points are requested.

If you can generate points uniformly in the sphere's volume, then to get a uniform distribution on the sphere's surface, you can simply normalize the vectors so their radius equals the sphere's radius.
Alternatively, you can use the fact that independent identically-distributed normal distributions are rotationally-invariant. If you sample from 3 normal distributions with mean 1 and standard deviation 0, and then likewise normalize the vector, it will be uniform on the sphere's surface. Here's an example:
import random
def sample_sphere_surface(radius=1):
x, y, z = (random.normalvariate(0, 1) for i in range(3))
scalar = radius / (x**2 + y**2 + z**2) ** 0.5
return (x * scalar, y * scalar, z * scalar)
To be absolutely foolproof, we can handle the astronomically unlikely case of a division-by-zero error when x, y and z all happen to be zero:
def sample_sphere_surface(radius=1):
while True:
try:
x, y, z = (random.normalvariate(0, 1) for i in range(3))
scalar = radius / (x**2 + y**2 + z**2) ** 0.5
return (x * scalar, y * scalar, z * scalar)
except ZeroDivisionError:
pass

The element of area is, in polar coordinates, sinΘ dΘ dφ. Hence the azimuth angle can be uniformly distributed, while the inclination must be redistributed. Using the inverse transform sampling trick, Θ=arccos(u) where u is drawn uniformly will do.
Hence in Cartesian coordinates, (√(1-u²) cos v, √(1-u²) sin v, u) where u is drawn from [-1,1) and v from [0,2π).

Can't reproduce distance value between sources obtained with astropy

I have two sources with equatorial coordinates (ra, dec) and (ra_0, dec_0) located at distances r and r_0, and I need to calculate the 3D distance between them.
I use two approaches that should give the same result as far as I understand, but do not.
The first approach is to apply astropy's separation_3d function. The second approach is to use the expression that gives the distance between two sources with spherical coordinates:
as shown here.
In the MCVE below, the values returned are:
91.3427173002 pc
93.8470493776 pc
Shouldn't these two values be equal?
MCVE:
from astropy.coordinates import SkyCoord
from astropy import units as u
import numpy as np
# Define some coordinates and distances for the sources.
c1 = SkyCoord(ra=9.7*u.degree, dec=-50.6*u.degree, distance=1500.3*u.pc)
c2 = SkyCoord(ra=7.5*u.degree, dec=-47.6*u.degree, distance=1470.2*u.pc)
# Obtain astropy's distance between c1 & c2 coords.
print c1.separation_3d(c2)
# Obtain distance between c1 & c2 coords using explicit expression.
ra_0, dec_0, r_0 = c1.ra.radian, c1.dec.radian, c1.distance
ra, dec, r = c2.ra.radian, c2.dec.radian, c2.distance
alpha_delta_par = np.sin(dec) * np.sin(dec_0) * np.cos(ra - ra_0) +\
np.cos(dec) * np.cos(dec_0)
d_pc = np.sqrt(r**2 + r_0**2 - 2*r*r_0*alpha_delta_par)
print d_pc

This is a problem with coordinate systems, and the difference between declination (astral coordinates) and polar angle θ (spherical coordinates) :-)
Astral coordinates define declination as north of the celestial equator, while spherical coordinates define polar angle θ as downward from from vertical.
If you change your alpha_delta_par to account for this 90° difference by adding np.pi/2 to all your declination terms, you get
alpha_delta_par = np.sin(np.pi/2 + dec)*np.sin(np.pi/2 + dec0)*np.cos(ra - ra0) +\
np.cos(np.pi/2 + dec)*np.cos(np.pi/2 + dec0)
Which gives the correct result: 91.3427173002 pc.
Turns out physicists usually use the symbol θ as the polar angle and mathemeticians usually use φ; I went with θ because I followed my heart. I'm not making this up I swear.

Points on a geodesic line

I am working on a unit sphere. I am interested to place N points on a strait line over the surface of the sphere (geodesic) between two arbitrary points. The coordinate of these points are in spherical coordinate (radians).
How do I compute a set of N equally spaced points along such line. I would like to take the curvature of the sphere into account in my calculation.
I am using python 2.7.9

You may consider SLERP - spherical linear interpolation
P = P0*Sin(Omega*(1-t))/Sin(Omega) + P1*Sin(Omega * t)/Sin(Omega)
where Omega is central angle between start and end points (arc of great circle), t is parameter in range [0..1], for i-th point t(i) = i/N

Let us reason geometrically.
Convert the two given points to Cartesian coordinates.
The angle between the position vectors from the center to P0 and P1 is given by the dot product
cos A = P0.P1
Construct a linear combination of these:
P = (1-t).P0 + t.P1
The angle between P and P0 is given by the dot product with P normalized
cos a = cos kA/N = P.P0/|P| = ((1-t) + t.cos A)/ sqrt((1-t)² + 2.(1-t).t.cos A + t²)
Squaring and rewriting, you obtain a second degree equation in t:
cos²a.(1-t)² + 2.(1-t).t.cos²a.cos A + t².cos²a - (1-t)² - 2.(1-t).t.cos A - t².cos²A = 0
- sin²a.(1-t)² - 2.(1-t).t.sin²a.cos A - t².(cos²A - cos² a) = 0
t²(-sin²a + 2.sin²a.cos A - cos²A + cos²a) + 2.t.sin²a.(1 - cos A) - sin²a = 0
Solve the equation, compute the vector P from its definition and normalize it.
Then revert to spherical coordinates. Varying k between 1 and N-1 will give you the required intermediate points.
Alternatively, you can use the Rodrigue's rotation formula around an axis in 3D. The axis is given by the cross-product P0 x P1.

Retrieving the actual 3D coordinates of a point on a triangle that has been flattened to 2 dimensions

This is a bit of a complicated problem, so I'll do my best to break it down into chunks.
I'm writing a 3D Python library for the sake of learning / fun (as opposed to one that I'd intend for others to use). In the system I've developed, three-dimensional points are generally flattened to the image as follows:
Increasing the Z index by width moves the point halfway to the vanishing point in the center.
At Z = 0, the X and Y values correspond directly to the pixel at X, Y.
(There might be a name for this method, but if there is, I'm not familiar with it.)
In Python:
# vx and vy are the vanishing point's coordinates
def flatten_point(width, vx, vy, x, y, z):
distance = (x - vx, y - vy)
flat_distance = [d / (1 + float(z) / width) for d in distance]
return (vx + flat_distance[0], vx + flat_distance[1])
At this point, I'm able to create triangles somewhat efficiently by flattening its vertices and using barycentric coordinates to find and fill in the pixels that fall between those three points. This works well enough if I don't need to know anything about the actual points on the triangle that those pixels correspond to, but if I want to shade the triangle so that deeper points are drawn darker, I need to know what unflattened point on the triangle the pixel corresponds to.
joriki on math.stackexchange recommended using the barycentric coordinates as weights to find the original point. This did appear to work for awhile -- and it probably would work if I were using a linear depth system -- but it falls apart when the depths of the triangle's points differ by enough. The triangle appears to approach the greatest depth more quickly than it actually does, as if it were curved backwards.
So, in short: how can I reverse the point flattening function to get the actual 3D point of an arbitrary 2D pixel on a flattened triangle? Alternatively, if there is a better / more efficient way to flatten triangles without losing the depth of each pixel, that would work too.

You are right that the problem lies in your depth values not being linear. Fortunately, the solution is simple, but a little expensive if calculated per pixels.
Using your barycentric coordinates, rather than interpolating the three Z components directly, you need to interpolate their inverse and reinverse the result. This is called perspective correction.
Example for Z only :
def GetInterpolatedZ(triangle, u, v):
z0 = 1.0 / triangle[0].z
z1 = 1.0 / triangle[1].z
z2 = 1.0 / triangle[2].z
z = z0 + u * (z1-z0) + v * (z2-z0)
return 1.0/z
With triangle a list of three vectors and u and v the barycentric coordinates for triangle[1] and triangle[2] respectively. You will need to remap your Zs before and after the divisions if they are offset.
If you want to interpolate the actual X and Y coordinates, you do something similar. You will need to interpolate x/z and y/z and relinearize the result by multiplying by z.
def GetInterpolatedZ(tri, u, v):
t0 = Vec3(tri[0].x/tri[0].z, tri[0].y/tri[0].z, 1.0/tri[0].z)
t1 = Vec3(tri[1].x/tri[1].z, tri[1].y/tri[1].z, 1.0/tri[1].z)
t2 = Vec3(tri[2].x/tri[2].z, tri[2].y/tri[2].z, 1.0/tri[2].z)
inter = t0 + u * (t1-t0) + v * (t2-t0)
inter.z = 1.0 / inter.z
inter.x *= inter.z
inter.y *= inter.z
return inter
Again, tri is a list of the three vectors and u, v are the barycentric coordinates for tri[1], tri[2]. Vec3 is a regular 3 components Euclidean vector type.