We Keep Coding

Calculate the distance between fitted hyperplane and points

I'm trying to find the distance between a fitted hyperplane and five points. Most of the responses I've read use SVM, but I'm not trying to do a classification problem. I know there are probably multiple ways to do this in Python, but I'm a little stumped.
As an example here are my points:
[[ 163.3828172 169.65537306 144.69201418]
[-212.50951396 -167.06555958 56.69388025]
[-164.65129832 -163.42420063 -149.97008725]
[ 41.8704004 52.2538316 14.0683657 ]
[-128.38386078 -102.76840542 -303.4960438 ]]
To find the equation of a fitted plane I use SVD to compute the coefficients ax + by + cz - b = 0.
def fit_plane(points):
assert points.shape[1] == 3
centroid = points.mean(axis=0)
x = points - centroid[None, :]
U, S, Vt = np.linalg.svd(x.T # x)
#normal vector of best fitting plane is the left
#singular vector corresponding to the least singular value
normal = U[:, -1]
#calculate the distance from origin
origin_distance = normal # centroid
return np.hstack([normal, -origin_distance])
fit_plane(X)
Giving the equation:
-0.67449074x + 0.73767288y -0.03001614z -10.75632119 = 0
Now how do I calculate the distance between the points and the hyperplane? The answer I've seen used in conjunction with SVMs is d = |w^Tx +b|/||w||, but I don't know how to go from the equation I have already.

You can find the distance between an equation π and a point P by dropping a perpendicular N from P to π and get the point A where N and π intersect. The distance you are looking for is the distance between A and P.
This video explains the math of finding A (although it is about finding the reflection, finding A is part of it).

Python Linear Programming, Constrain Input to Zero or Range

In an Linear Program I am minimizing the distance between weighted input vectors and a target vector. I used Scipyto compute values for the weights I need. Currently they are between zero and one, but I'd like them to be zero if they are smaller than .2 for example, so x_i should be 0 or [.2; 1]. I was pointed to mixed integer linear programming but I still can't find any approach for my problem. How can I fix this?
tldr: i want to use (0,0) or (.3,1) as bounds for each x, how do i implement this?
Here is my SciPy code:
# minimize the distance between weighted input vectors and a target vector
def milp_objective_function(weights):
scaled_matrix = input_matrix * weights[:, np.newaxis] # scale input_matrix columns by weights
sum_vector = sum(scaled_matrix) # sum weighted_input_matrix columns
difference_vector = sum_vector - target_vector
return np.sqrt(difference_vector.dot(difference_vector)) # return the distance between the sum_vector and the target_vector
# sum of weights should equal 100%
def milp_constraint(weights):
return sum(weights) - 1
def main():
# bounds should be 0 or [.2; 1] -> mixed integer linear programming?
weight_bounds = tuple([(0, 1) for i in input_matrix])
# random guess, will implement later
initial_guess = milp_guess_weights()
constraint_obj = {'type': 'eq', 'fun': milp_constraint}
result = minimize(milp_objective_function, x0=initial_guess, bounds=weight_bounds, constraints=constraint_obj)

Variables that are in {0} ∪ [L,U] are called semi-continuous variables. Advanced MIP solvers have built-in support for these types of variables.
Note that SciPy does not have a MIP solver at all.
I also want to note that if your MIP solver does not support semi-continuous variables you can simulate them with binary variables:
L ⋅ δ(i) ≤ x(i) ≤ U ⋅ δ(i)
δ(i) ∈ {0,1}

How to solve Linear Programming Problem with more than one optimal solution using pulp

I want to solve an LPP which has more than one optimal solution. How can I do that?
For Example:
Maximize 2000x1 + 3000x2
subject to
6x1 + 9x2 ≤ 100
2x1 + x2 ≤ 20
x1, x2 ≥ 0
In this LPP problem, there is more than one optimal solution i.e (0,100/9) and (20/3,20/3). When I solve this problem using the pulp library, it gives me only a (0,100/9) solution. I want all the possible solutions.

There is a good discusion on this here.
There are two questions here: (i) How to find multiple optimal solutions to an LP, and (ii) Why you would want to.
Answering (ii) first - typically you might want to ask for all optimal solutions because there is some secondary or lower imporatance objective to pick between them (for example wanting to minimize the change from a current setting, or minimise risk in some way). If this is the case my personal recommendation would be to find a way of incorporating that lower order preference into your objective function (for example by adding in a term with a low weighting).
Answering (i) I find it helps to look at an LP graphically - and the example you have given works nicely for this (two variables means you can plot it - see plot on wolfram). Each of your constraints puts a lines on this inequality plot, and solutions can only be chosen on one side of that line.
Your objective function is like having a constant gradient across this feasible area and you are trying to find the highest spot. You can draw contours of your objective function by setting it to a specific value and drawing that line. What you'll find if you do this is that your objective function contours are parrallel to the top constraint line (your first constratin).
You can see this directly from the equation: 6x1 + 9x2 <= 100 divides down to 2x1 + 3x2 <= 100/3, and your objective divides down to have the same gradient. What this means is you can move along that top constraint from one corner to the other without changing the value of your objective function.
There are infinitely many optimal solutions which solve the equation:
2x1 + 3x2 == 100/3, between x1==0, and x1==20/3. You have already identified the solutions at the two corners.
If you want to find all the nodes which are equally optimal - for large problems there could be a large number of these - then the code below gives a basic implementation the method discussed here. When you run it the first time it will give you one of the corner solutions - you then need to add this node (set of variables and slacks which are zero) to A, and iterate until the objective degrades. You could put this within a loop. Note that as currently implemented this only works for problems with variables which have 0 lower bound, and are unbounded above.
import pulp as pulp
# Accounting:
# n structural varuables (n = 2)
# m constraints (m = 2)
# => No. of basics = 2 (no. of constraints)
# => No. of non-basics = 2 (no. of variables)
nb = 2
M = 100 # large M value - upper bound for x1, x2 * the slacks
model = pulp.LpProblem('get all basis', pulp.LpMaximize)
# Variables
x = pulp.LpVariable.dicts('x', range(2), lowBound=0, upBound=None, cat='Continuous')
# Non-negative Slack Variables - one for each constraint
s = pulp.LpVariable.dicts('s', range(2), lowBound=0, upBound=None, cat='Continuous')
# Basis variables (binary)
# one for each variable & one for each constraint (& so slack)
B_x = pulp.LpVariable.dicts('b_x', range(len(x)), cat='Binary')
B_s = pulp.LpVariable.dicts('b_s', range(len(s)), cat='Binary')
# Objective
model += 2000*x[0] + 3000*x[1]
# Constraints - with explicit slacks
model += 6*x[0] + 9*x[1] + s[0] == 100
model += 2*x[0] + x[1] + s[1] == 20
# No. of basics is correct:
model += pulp.lpSum(B_x) + pulp.lpSum(B_s) == nb
# Enforce basic and non-basic behaviour
for i in range(len(x)):
model += x[i] <= M*B_x[i]
for i in range(len(x)):
model += s[i] <= M*B_s[i]
# Cuts - already discovered solutions
A = []
# A = [[1, 1, 0, 0]]
# A = [[1, 1, 0, 0], [0, 1, 0, 1]]
for a in A:
model += (B_x[0]*a[0] + B_x[1]*a[1] +
B_s[0]*a[2] + B_s[1]*a[3]) <= nb - 1
model.solve()
print('Status:', pulp.LpStatus[model.status])
print('Objective:', pulp.value(model.objective))
for v in model.variables():
print (v.name, "=", v.varValue)

Here is an example of how to add a cut to a PuLP LP problem with multiple optimal solutions:
from pulp import *
# Define the LP problem
prob = LpProblem("LP Problem", LpMaximize)
# Define variables
x = LpVariable("x", lowBound=0)
y = LpVariable("y", lowBound=0)
# Define an objective function
prob += 2 * x + 3 * y
# Define constraints
prob += x + 2 * y <= 10
prob += x + y <= 5
# Solve the LP problem
prob.solve()
# Check if the LP problem has multiple solutions
if LpStatus[prob.status] == "Optimal":
if value(x) == 0:
# Add a cut to eliminate one of the solutions
prob += x >= 1
prob.solve()
# Print the result
print("x =", value(x))
print("y =", value(y))
In this example, after solving the LP problem, we check if it has an optimal solution and if variable x is equal to 0. If these conditions are true, it means that the problem has multiple solutions. To eliminate one of the solutions, we add a cut that requires variable x to be greater than or equal to 1. We then solve the LP problem again, and this time we should obtain a unique solution.

I'm testing a simple MILP program using cvxpy. However, the solver stops converging after I increase the number of variables slightly

The task is to place n number of non-overlapping segments into an interval, maximizing the total covered space inside the interval. This is a 1D toy problem to be extended for future "fitting boxes into a bigger box" kinds of problems.
The interval is defined as [x0, x1]. Each segment is defined by a starting xi and a width wi. The objective function is to maximize the sum of all widths, given some constraints.
There are straight forward constraints that keep all segments within the interval, and collisions constraints between segments to avoid overlaps. The collision constraints use binary helper variables.
import cvxpy as cp
# Problem set up
x0 = 2
x1 = 20
num_segments = 4
# Segment variables
X = cp.Variable(num_segments) # starting x's
W = cp.Variable(num_segments) # widths
# Constraints keeping segments in interval
constraints = [X + W <= x1, X >= x0, W >=1]
# Pairwise collision constraints
for i in range(num_segments-1):
for j in range(i+1, num_segments):
# Helper binary variables
sigma_r = cp.Variable(boolean=True)
sigma_l = cp.Variable(boolean=True)
# segment i can be on the left or the right of segment j
constraints += [X[i] - W[j] >= X[j]-4000*(1-sigma_r)]
constraints += [X[i] + W[i] <= X[j]+4000*(1-sigma_l)]
# require at least one sigma to be 1
constraints += [sigma_r + sigma_l >= 1]
# maximize sum of widths
objective = cp.Maximize(cp.sum(W))
prob = cp.Problem(objective, constraints)
prob.solve()
This works fine for num_segments up to 6, anything past that and the solver never converges. I feel this should not be a hard optimization for number of variables below 100. Any ideas what could be causing the issue? Should I be looking into other solvers?

Is this a proper implementation of point charge dynamics with python ODEint

Since learning about point charges in my physics II class this semester, I want to be able to investigate not only the static force and field distributions but the actual trajectories of movement of electrically charged particles. The first stage in doing this is to build a naive engine for simulating the dynamics of n individual point particles. I've implemented the solution using matrices in python and was hoping someone could comment on whether I've done so correctly. As I don't know what kind of dynamics to expect, I can't tell directly from the videos that my implementation of my equations is correct.
My Particular Problem
In particular, I cannot tell if in my calculation of Force magnitude I am computing the 1/r^(3/2) factor correctly. Why? because when I simulate a dipole and use $2/2$ as an exponent the particles start going in an elliptical orbit. which is what I would expect. However, when I use the correct exponent, I get this: Where is my code going wrong? What am I supposed to expect
I'll first write down the equations I'm using:
Given n charges q_1, q_2, ..., q_n, with masses m_1, m_2, ..., m_n located at initial positions r_1, r_2, ..., r_n, with velocities (d/dt)r_1, (d/dt)r_2, ..., (d/dt)r_n the force induced on q_i by q_j is given by
F_(j -> i) = k(q_iq_j)/norm(r_i-r_j)^{3/2} * (r_i - r_j)
Now, the net marginal force on particle $q_i$ is given as the sum of the pairwise forces
F_(N, i) = sum_(j != i)(F_(j -> i))
And then the net acceleration of particle $q_i$ just normalizes the force by the mass of the particle:
(d^2/dt^2)r_i = F_(N, i)/m_i
In total, for n particles, we have an n-th order system of differential equations. We will also need to specify n initial particle velocities and n initial positions.
To implement this in python, I need to be able to compute pairwise point distances and pairwise charge multiples. To do this I tile the q vector of charges and the r vector of positions and take, respectively, their product and difference with their transpose.
def integrator_func(y, t, q, m, n, d, k):
y = np.copy(y.reshape((n*2,d)))
# rj across, ri down
rs_from = np.tile(y[:n], (n,1,1))
# ri across, rj down
rs_to = np.transpose(rs_from, axes=(1,0,2))
# directional distance between each r_i and r_j
# dr_ij is the force from j onto i, i.e. r_i - r_j
dr = rs_to - rs_from
# Used as a mask to ignore divides by zero between r_i and r_i
nd_identity = np.eye(n).reshape((n,n,1))
# WHAT I AM UNSURE ABOUT
drmag = ma.array(
np.power(
np.sum(np.power(dr, 2), 2)
,3./2)
,mask=nd_identity)
# Pairwise q_i*q_j for force equation
qsa = np.tile(q, (n,1))
qsb = np.tile(q, (n,1)).T
qs = qsa*qsb
# Directional forces
Fs = (k*qs/drmag).reshape((n,n,1))
# Dividing by m to obtain acceleration vectors
a = np.sum(Fs*dr, 1)
# Setting velocities
y[:n] = np.copy(y[n:])
# Entering the acceleration into the velocity slot
y[n:] = np.copy(a)
# Flattening it out for scipy.odeint to work properly
return np.array(y).reshape(n*2*d)
def sim_particles(t, r, v, q, m, k=1.):
"""
With n particles in d dimensions:
t: timepoints to integrate over
r: n*d matrix. The d-dimensional initial positions of n particles
v: n*d matrix of initial particle velocities
q: n*1 matrix of particle charges
m: n*1 matrix of particle masses
k: electric constant.
"""
d = r.shape[-1]
n = r.shape[0]
y0 = np.zeros((n*2,d))
y0[:n] = r
y0[n:] = v
y0 = y0.reshape(n*2*d)
yf = odeint(
integrator_func,
y0,
t,
args=(q,m,n,d,k)).reshape(t.shape[0],n*2,d)
return yf