We Keep Coding

Using cvxpy to solve a lasso like problem

It's not entirely lasso because I add an extra constraint but I'm not sure how I'm supposed to solve a problem like the following using cvxpy
import cvxpy as cp
import numpy as np
A = np.random.rand(5000,1000)
v0 = np.random.rand(1000,1)
v = cp.Variable(v0.shape)
iota = np.ones(v0.shape)
lam = 1
objective = cp.Minimize( (A#(v-v0)).T#(A#(v-v0)) + lam * cp.abs(v).T # iota )
constraints = [v >= 0]
prob = cp.Problem(objective, constraints)
res = prob.solve()
I tried various versions of this but this is the one that most clearly shows what I'm trying to do. I get the error:
DCPError: Problem does not follow DCP rules. Specifically: The objective is not DCP. Its following subexpressions are not: ....
And then an error I don't undeerstand haha.

CVXPY is a modeling language for convex optimization. Therefore, there is a set of rules your problem must follow to ensure your problem is convex indeed. These are what cvxpy refers to DCP: Disciplined Convex Programming. As the error suggests, your objective is not DCP.
To be more precise, the problem is in the objective (A#(v-v0)).T#(A#(v-v0)): cvxpy don't know it is indeed convex (from program point of view, it's just few multiplications).
To ensure your problem is DCP, it's best to use cvxpy atomic functions.
Essentially you are modeling x^T * x (if x=A#(v-v0)), which is the squares of norm 2 of a vector. cp.norm2 is the way to ensure cvxpy will know the problem is convex.
change the line of the objective function to:
objective = cp.Minimize(cp.norm2(A # (v - v0)) ** 2 + lam * cp.abs(v).T # iota)
and it works.
(Also take a look at cvxpy example of lasso regression)

Bounded sympy symbol variables

I would like to solve a multivariate optimization problem by converting it to a system of non-linear equations and solving using sympy's "solve" function as so:
xopt = sympy.solve(gradf, x, set=True)
Trouble is, this particular equation has an infinite set of solutions and calling solve just freezes my computer.
If i could set lower and upper bounds for my symbolic variables, i.e introduce a set of constraints:
l1 <= x1 <= u1, l2 <= x2 <= u2, ..., ln <= xn <= un
... i could limit the set of solutions to a finite one, but im having a hard time finding out how to do this with sympy's API. Can anyone help out ?

Is there a way to setup and solve multiple numerical integration in scipy like in mathematica?

I would like to solve the following formulas numerically in Python;
In Mathematica, you can input multiple differential equations and solve it at the same time. Is there a way to do the similar thing with Scipy?
edit: yes, I have looked at scipy.integrate.odeint already, but I'm still not sure how I can solve multiple equations that correlates each other at the same time. Does anyone have suggestion for that?

Eventually I figured out myself and I'm writing down for other people who might be clueless like me;
in odeint, you give three parameters model, y, and t where model is a function that takes in y and t then return dydt, y is the initial value of y, and t is the variable you're trying to take integral over. If you have multiple differential equations that are dependent on each other, you can just pass in all of them to odeint. In my case,
t = np.linspace(0, 20) # range of t
y0 = [No_0, Na_0, Ni_0, Nn_0] # initial condition for each Ns
def model(y, t):
No, Na, Ni, Nn = y
dNodt = -k_oa * No
dNadt = k_oa * No - k_ai * Na
dNidt = k_ai * Na - k_in * Ni
dNndt = k_in * Ni
return [dNodt, dNadt, dNidt, dNndt]
y = odeint(model, y0, t)
You can define multiple differential equations you want to solve within the model you define, and pass in to odeint along with the initial values for all species.

Have a loot at SciPy's Integration package:
https://docs.scipy.org/doc/scipy/reference/integrate.html
specifically at odeint to solve systems of ODE's:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html#scipy.integrate.odeint

minimise function with constraints on non-optimisation variables

I need to optimise a function with a constraint on a variable which is calculated by solving a set of equations.
The optimisation parameters are the input variables to the system of equations and one of the calculated variables have a constraint.
Here is an extremely simplified example:
def opt(x):
x1, x2, x3 = x
z1 = x1 + x2 + x3
z2 = z1**2
.
.
.
z100 = f(x1, x2, x3, z1, ..., z99)
return some objective function
minimise opt(x)
s.t. z100 < a
I am familiar with scipy.optimize.minimize but I can not set a constraint on z100 and it is extremely difficult to calculate a function for z100 with only the variables x1, x2, x3.

This is very broad, but some remarks:
and it is extremely difficult to calculate a function for z100 with only the variables x1, x2, x3
Where is the problem? Just use the args-argument inside the call to minimize and you can use all the objects you want
I am familiar with scipy.optimize.minimize but I can not set a constraint on z100
This somewhat implicates, that z100 is depending on the optimization-variables (which makes it a constraint on optimization-variables opposed to the title; or there is no effect / reason to use it at all as we are only modifying opt-vars and no a-priori constraint-logic is affected)
Just introduce an auxiliary-variable y0 and constraints:
y0 < a
y0 == f(...)
But: this might contradict the solver's assumptions about smoothness (depending on f of course). So you probably should think again about your problem (and for us there is nothing to do as the structure of that problem is not posted)
And: eq-constraints sometimes are a bit of a hassle in regards to numerical-trouble

On ordinary differential equations (ODE) and optimization, in Python

I want to solve this kind of problem:
dy/dt = 0.01*y*(1-y), find t when y = 0.8 (0<t<3000)
I've tried the ode function in Python, but it can only calculate y when t is given.
So are there any simple ways to solve this problem in Python?
PS: This function is just a simple example. My real problem is so complex that can't be solve analytically. So I want to know how to solve it numerically. And I think this problem is more like an optimization problem:
Objective function y(t) = 0.8, Subject to dy/dt = 0.01*y*(1-y), and 0<t<3000
PPS: My real problem is:
objective function: F(t) = 0.85,
subject to: F(t) = sqrt(x(t)^2+y(t)^2+z(t)^2),
x''(t) = (1/F(t)-1)*250*x(t),
y''(t) = (1/F(t)-1)*250*y(t),
z''(t) = (1/F(t)-1)*250*z(t)-10,
x(0) = 0, y(0) = 0, z(0) = 0.7,
x'(0) = 0.1, y'(0) = 1.5, z'(0) = 0,
0<t<5

This differential equation can be solved analytically quite easily:
dy/dt = 0.01 * y * (1-y)
rearrange to gather y and t terms on opposite sides
100 dt = 1/(y * (1-y)) dy
The lhs integrates trivially to 100 * t, rhs is slightly more complicated. We can always write a product of two quotients as a sum of the two quotients * some constants:
1/(y * (1-y)) = A/y + B/(1-y)
The values for A and B can be worked out by putting the rhs on the same denominator and comparing constant and first order y terms on both sides. In this case it is simple, A=B=1. Thus we have to integrate
1/y + 1/(1-y) dy
The first term integrates to ln(y), the second term can be integrated with a change of variables u = 1-y to -ln(1-y). Our integrated equation therefor looks like:
100 * t + C = ln(y) - ln(1-y)
not forgetting the constant of integration (it is convenient to write it on the lhs here). We can combine the two logarithm terms:
100 * t + C = ln( y / (1-y) )
In order to solve t for an exact value of y, we first need to work out the value of C. We do this using the initial conditions. It is clear that if y starts at 1, dy/dt = 0 and the value of y never changes. Thus plug in the values for y and t at the beginning
100 * 0 + C = ln( y(0) / (1 - y(0) )
This will give a value for C (assuming y is not 0 or 1) and then use y=0.8 to get a value for t. Note that because of the logarithm and the factor 100 multiplying t y will reach 0.8 within a relatively short range of t values, unless the initial value of y is incredibly small. It is of course also straightforward to rearrange the equation above to express y in terms of t, then you can plot the function as well.
Edit: Numerical integration
For a more complexed ODE which cannot be solved analytically, you will have to try numerically. Initially we only know the value of the function at zero time y(0) (we have to know at least that in order to uniquely define the trajectory of the function), and how to evaluate the gradient. The idea of numerical integration is that we can use our knowledge of the gradient (which tells us how the function is changing) to work out what the value of the function will be in the vicinity of our starting point. The simplest way to do this is Euler integration:
y(dt) = y(0) + dy/dt * dt
Euler integration assumes that the gradient is constant between t=0 and t=dt. Once y(dt) is known, the gradient can be calculated there also and in turn used to calculate y(2 * dt) and so on, gradually building up the complete trajectory of the function. If you are looking for a particular target value, just wait until the trajectory goes past that value, then interpolate between the last two positions to get the precise t.
The problem with Euler integration (and with all other numerical integration methods) is that its results are only accurate when its assumptions are valid. Because the gradient is not constant between pairs of time points, a certain amount of error will arise for each integration step, which over time will build up until the answer is completely inaccurate. In order to improve the quality of the integration, it is necessary to use more sophisticated approximations to the gradient. Check out for example the Runge-Kutta methods, which are a family of integrators which remove progressive orders of error term at the cost of increased computation time. If your function is differentiable, knowing the second or even third derivatives can also be used to reduce the integration error.
Fortunately of course, somebody else has done the hard work here, and you don't have to worry too much about solving problems like numerical stability or have an in depth understanding of all the details (although understanding roughly what is going on helps a lot). Check out http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html#scipy.integrate.ode for an example of an integrator class which you should be able to use straightaway. For instance
from scipy.integrate import ode
def deriv(t, y):
return 0.01 * y * (1 - y)
my_integrator = ode(deriv)
my_integrator.set_initial_value(0.5)
t = 0.1 # start with a small value of time
while t < 3000:
y = my_integrator.integrate(t)
if y > 0.8:
print "y(%f) = %f" % (t, y)
break
t += 0.1
This code will print out the first t value when y passes 0.8 (or nothing if it never reaches 0.8). If you want a more accurate value of t, keep the y of the previous t as well and interpolate between them.

As an addition to Krastanov`s answer:
Aside of PyDSTool there are other packages, like Pysundials and Assimulo which provide bindings to the solver IDA from Sundials. This solver has root finding capabilites.

Use scipy.integrate.odeint to handle your integration, and analyse the results afterward.
import numpy as np
from scipy.integrate import odeint
ts = np.arange(0,3000,1) # time series - start, stop, step
def rhs(y,t):
return 0.01*y*(1-y)
y0 = np.array([1]) # initial value
ys = odeint(rhs,y0,ts)
Then analyse the numpy array ys to find your answer (dimensions of array ts matches ys). (This may not work first time because I am constructing from memory).
This might involve using the scipy interpolate function for the ys array, such that you get a result at time t.
EDIT: I see that you wish to solve a spring in 3D. This should be fine with the above method; Odeint on the scipy website has examples for systems such as coupled springs that can be solved for, and these could be extended.

What you are asking for is a ODE integrator with root finding capabilities. They exist and the low-level code for such integrators is supplied with scipy, but they have not yet been wrapped in python bindings.
For more information see this mailing list post that provides a few alternatives: http://mail.scipy.org/pipermail/scipy-user/2010-March/024890.html
You can use the following example implementation which uses backtracking (hence it is not optimal as it is a bolt-on addition to an integrator that does not have root finding on its own): https://github.com/scipy/scipy/pull/4904/files