I need to minimize L_1(x) subject to Mx = y.
x is a vector with dimension b, y is a vector with dimension a, and M is a matrix with dimensions (a,b).
After some reading I determined to use scipy.optimize.minimize:
import numpy as np
from scipy.optimize import minimize
def objective(x): #L_1 norm objective function
return np.linalg.norm(x,ord=1)
constraints = [] #list of all constraint functions
for i in range(a):
def con(x,y=y,i=i):
return np.matmul(M[i],x)-y[i]
constraints.append(con)
#make constraints into tuple of dicts as required by scipy
cons = tuple({'type':'eq','fun':c} for c in constraints)
#perform the minimization with sequential least squares programming
opt = minimize(objective,x0 = x0,
constraints=cons,method='SLSQP',options={'disp': True})
First,
what can I use for x0? x is unknown, and I need an x0 which satisfies the constraint M*x0 = y: How can I find an initial guess which satisfies the constraint? M is a matrix of independent Gaussian variables (~N(0,1)) if that helps.
Second,
Is there a problem with the way I've set this up? When I use the true x (which I happen to know in the development phase) for x0, I expect it to return x = x0 quickly. Instead, it returns a zero vector x = [0,0,0...,0]. This behavior is unexpected.
Edit:
Here is a solution using cvxpy** solving min(L_1(x)) subject to Mx=y:
import cvxpy as cvx
x = cvx.Variable(b) #b is dim x
objective = cvx.Minimize(cvx.norm(x,1)) #L_1 norm objective function
constraints = [M*x == y] #y is dim a and M is dim a by b
prob = cvx.Problem(objective,constraints)
result = prob.solve(verbose=False)
#then clean up and chop the 1e-12 vals out of the solution
x = np.array(x.value) #extract array from variable
x = np.array([a for b in x for a in b]) #unpack the extra brackets
x[np.abs(x)<1e-9]=0 #chop small numbers to 0
Related
I have a set of linear equations that I am attempting to solve. I have five variables that are randomly assigned and sum to 1. I applied these variables row-wise to a matrix (i.e., Ax = B) However, the caveat is that one of my B variables needs to be maximized, subject to the constraint that the rest of my B variables are 0. My data is below:
import pandas as pd
A = [[0.031982, 0.02606, 0.055933, 0.004529, 0.064116],
[-0.000167, 0.181031, 0.145465, 0.120430, 0.114093],
[0.627812, 0.254377, 0.138211, 0.41156, -0.000801],
[-0.228139, 0.377169, 0.085415, 0.008888, -0.020791],
[-0.067697, -0.114113, 0.089583, 0.100222, -0.005291]]
B = [[maximized],
[0],
[0],
[0],
[0]]
x = [x1, x2, x3, x4, x5]
Note: 'maximized' is the value that I am attempting to maximize
What I've done so far:
import numpy as np
ABC=[]
A = DataFrame(A)
N=1000 #my attempt at maximizing by repeating the sequence N times and taking the largest value that results
for i in range(N):
x = np.random.rand(5) #creating random variables
x/= np.sum(x) #so that they sum to one
x=x*A.T[0] #I apply the variables to my data. I want to maximize the sum of the first column, so I transpose and take a slice that I sum below
x=x.sum()
ABC.append(x)
ABC = DataFrame(ABC)
A2=ABC.sort_values(by=0,ascending=False).head(1) #I sort by largest first and take the highest value and store in a new dataframe
maximized=np.array(A2) #I convert the dataframe back into an array
B = [[maximized],[0],[0],[0],[0]]
X = np.linalg.inv(A).dot(B)
X
Obviously this has a lot of error and isn't really achieving what I want. What I want to do is to run a maximization function that gives me the largest value and input that value into my matrix. I don't really know where to go from here, or what sort of maximization function applies in this case. If anyone has any ideas, that would be super appreciated!
If I understand you correctly, you have a given matrix A. Then you want to find a positive vector x with sum 1 such that if Ax = b then b is the all-zeroes vector except that the first entry is maximized, rather than zero.
Let A0 be the first row of matrix A and Ar be the rest. Then we can rephrase your problem as:
Find vector x such that its entries are non-negative, sum to 1, where Ar x = b, b = 0 and maximizing A0 x.
This is simply a standard linear programming instance (handling the 'sum-to-1' requirement by adding a single all-ones row to Ar and a 1 entry to b).
This isn't really a maximization problem I think. There's only one answer with the specified result. If our maximized number is c, then b = [[c,0,0,0,0]].T = c*[[1,0,0,0,0]].T Lets' set b_ as [[1,0,0,0,0]].T so b = c*b_
If we solve A # x_ = b_ by x_ = Ainv # b_, we get a single x_ matrix that still needs to be normalized to form x. If len_x_ = np.linalg.norm(x_), then:
A # (c * x_) = (c * b_) # multiplying `c` though
A # (c * x_) = b # substituting `b = c * b_`
c * x_ = x = x_ / len_x_ # therefore . . .
c = 1 / len_x_
c can only take one value if A is postive-definite, and thus x can only take one value:
b_ = np.array([[1,0,0,0,0]])
x_ = np.linalg.inv(A).dot(b_)
c = 1 / np.linalg.norm(x_)
b = b_ * c
x = x_ * c
Now, it'll get a lot more interesting if A isn't definite (i.e. it doesn't have an inverse). But I'm not sure that's within scope, since you're currently solving by inverting A.
I'm trying to write a program that can solve the general regression formula:
So I'm trying to implement this matrix equation, is there anyway to do this such as to let the user decide how big it can be, without me making more and more if conditions (so just one piece of code that collapses to the matrix that the user wishes for)?
Code:
#Solving the general matrix for the coefficients
if 3 == n:
a = np.array([[np.sum(np.multiply(FL[1],FL[1])),np.sum(np.multiply(FL[1],FL[2]))],
[np.sum(np.multiply(FL[1],FL[2])),np.sum(np.multiply(FL[2],FL[2]))]])
b = np.array([np.sum(np.multiply(FL[0],FL[1])),np.sum(np.multiply(FL[0],FL[2]))])
x = np.linalg.solve(a, b)
if 4 == n:
a = np.array([[np.sum(np.multiply(FL[1],FL[1])),np.sum(np.multiply(FL[1],FL[2])),np.sum(np.multiply(FL[1],FL[3]))],
[np.sum(np.multiply(FL[1],FL[2])),np.sum(np.multiply(FL[2],FL[2])),np.sum(np.multiply(FL[2],FL[3]))],
[np.sum(np.multiply(FL[1],FL[3])),np.sum(np.multiply(FL[2],FL[3])),np.sum(np.multiply(FL[3],FL[3]))]])
b = np.array([np.sum(np.multiply(FL[0],FL[1])),np.sum(np.multiply(FL[0],FL[2])),np.sum(np.multiply(FL[0],FL[3]))])
x = np.linalg.solve(a, b)
1 In this code Phi_0 corresponds to FL[i=1] and FL[0] corresponds to y.
You can make the algorithm independent of the order of the polynomial. The easiest way is using for loops, although these will be slow (since they don't exploit NumPy's vectorization).
Here is a reproducible example with random data:
import numpy as np
# Order of polynomial
n = 5
# Random seed for reproducibility
np.random.seed(1)
# Input arrays
phi = np.random.random((100,n))
y = np.random.random(100)
# Output arrays
a = np.zeros((n,n))
b = np.zeros(n)
for i in range(n):
b[i] = np.sum(y * phi[:,i])
for j in range(i,n):
# Exploit that matrix is diagonal
a[i,j] = a[j,i] = np.sum(phi[:,i] * phi[:,j])
# Coefficients array
x = np.linalg.solve(a,b)
How can I write a recursive function to generate a vector X of size (1,n) as follows, where X_i is the i-th entry:
X_1 = Z_1 * E_1
X_i = max{B_(1,i) * X_1, ... , B_((i-1),i) * X_(i-1), Z_i} * E_i, i = 2,...,n,
where
Z = np.random.normal(0, 1,size = n)
E = np.random.lognormal(0, 1, size = n)
B = np.random.uniform(0,1,(n,n))
I do not have any experience with recursive functions, that is why I can not present any code with which I tried to solve this.
If you're working with numpy, then use all the power of numpy, not just the random module ;)
And if you work with vectors, then forget about recursion and use numpy's vectorised operations. For example, np.max gives you the maximum over an axis, np.dot gives you element-wise multiplication. You also have np.prod for the product of array elements over a given axis... Those are just examples that might fit your problem well. For a full documentation, https://docs.scipy.org/doc/numpy/
I got it, one does not need a recursion as #meowgoesthedog stated in the first comment.
import numpy as np
s=1000 # sample size
n=5
Z = np.random.normal(0, 1,size = (s,n))
B = np.random.uniform(0,1,(n,n))
E = np.random.lognormal(0, 1, size = (s,n))
X = np.zeros((s,n))
X[:,0] = Z[:,0]*E[:,0]
for k in range(s):
for l in range(1,n):
X[k,l] = max(np.max(X[k,:(l)] * B[:(l),l]), Z[k,l]) * E[k,l]
I am trying to solve this differential equation as part of my assignment. I am not able to understand on how can i put the condition for u in the code. In the code shown below, i arbitrarily provided
u = 5.
2dx(t)dt=−x(t)+u(t)
5dy(t)dt=−y(t)+x(t)
u=2S(t−5)
x(0)=0
y(0)=0
where S(t−5) is a step function that changes from zero to one at t=5. When it is multiplied by two, it changes from zero to two at that same time, t=5.
def model(x,t,u):
dxdt = (-x+u)/2
return dxdt
def model2(y,x,t):
dydt = -(y+x)/5
return dydt
x0 = 0
y0 = 0
u = 5
t = np.linspace(0,40)
x = odeint(model,x0,t,args=(u,))
y = odeint(model2,y0,t,args=(u,))
plt.plot(t,x,'r-')
plt.plot(t,y,'b*')
plt.show()
I do not know the SciPy Library very well, but regarding the example in the documentation I would try something like this:
def model(x, t, K, PT)
"""
The model consists of the state x in R^2, the time in R and the two
parameters K and PT regarding the input u as step function, where K
is the infimum of u and PT is the delay of the step.
"""
x1, x2 = x # Split the state into two variables
u = K if t>=PT else 0 # This is the system input
# Here comes the differential equation in vectorized form
dx = [(-x1 + u)/2,
(-x2 + x1)/5]
return dx
x0 = [0, 0]
K = 2
PT = 5
t = np.linspace(0,40)
x = odeint(model, x0, t, args=(K, PT))
plt.plot(t, x[:, 0], 'r-')
plt.plot(t, x[:, 1], 'b*')
plt.show()
You have a couple of issues here, and the step function is only a small part of it. You can define a step function with a simple lambda and then simply capture it from the outer scope without even passing it to your function. Because sometimes that won't be the case, we'll be explicit and pass it.
Your next problem is the order of arguments in the function to integrate. As per the docs (y,t,...). Ie, First the function, then the time vector, then the other args arguments. So for the first part we get:
u = lambda t : 2 if t>5 else 0
def model(x,t,u):
dxdt = (-x+u(t))/2
return dxdt
x0 = 0
y0 = 0
t = np.linspace(0,40)
x = odeint(model,x0,t,args=(u,))
Moving to the next part, the trouble is, you can't feed x as an arg to y because it's a vector of values for x(t) for particular times and so y+x doesn't make sense in the function as you wrote it. You can follow your intuition from math class if you pass an x function instead of the x values. Doing so requires that you interpolate the x values using the specific time values you are interested in (which scipy can handle, no problem):
from scipy.interpolate import interp1d
xfunc = interp1d(t.flatten(),x.flatten(),fill_value="extrapolate")
#flatten cuz the shape is off , extrapolate because odeint will go out of bounds
def model2(y,t,x):
dydt = -(y+x(t))/5
return dydt
y = odeint(model2,y0,t,args=(xfunc,))
Then you get:
#Sven's answer is more idiomatic for vector programming like scipy/numpy. But I hope my answer provides a clearer path from what you know already to a working solution.
Let us assume I have an ODE with x'(t) = f(x) with the respective solution x(t) = ϕ(x(0),t) of a initial condition x(0). Now I intend to calculate numerically the equilibria as a function of their initial condition: eq(x0) := ϕ(x0, ∞). The ODEs are such that these equilibria exist unambiguously for all initial conditions (including eq = ∞).
My poor man's approach would be to integrate the ODE up to a late time and fetch that value (for brevity I do not show the plotting):
import numpy as np
from scipy.integrate import odeint
# ODE
def func(X,t):
return [ X[2]**2 * (X[0] - X[1]),
X[2]**3 * (X[0] + 3 * X[1]),
-X[2]**2]
# Forming a grid
n = 15
x0 = x1 = np.linspace(0,1,n)
x0_,x1_ = np.meshgrid(x0,x1)
eq = np.zeros([n,n,3])
t = np.linspace(0,100,1000)
x2 = 1
for i in range(n):
for j in range(n):
X = odeint(func,[x0_[j,i],x1_[j,i],x2], t)
eq[j,i,:] = X[-1,:]
Naive example above:
The problem with that approach is that you can never be sure if it converged. I know that you can just find the roots of f(x), but this would not yield the equilibria as a function of their initial conditions (You could trace them back, but since this function is not injective, you will not find values for all initial values). I somehow need a ODE solver which integrates until an equilibria is reached (or stops integrating if it goes beyond a limit). Do you have any ideas?