I started learning machine learning in Python. I plotted the following graph following the author's notes:
One can notice that up until 3.5 weeks, there is a linear regression, and beyond it, there is a 2nd-order polynomial regression. The textbook question is: when will there be 100 000 requests/day?
The author says to use fsolve
reached_max = fsolve(myFunct-100000, x0=800)/(7*24)
The last point has x = 743 (hours). I am confused when the author says that we need to provide an initial starting position (e.g. 800). Why can it be any number after 743 and why does it have to be after 743? Thank you for your help!
The answer is that x0 will serve as an educated initial guess. In general, fsolve should work fine with its default value (x0 = 0), but in some cases will give a too far answer from the true root.
To better understand the math behind finding the root, I encourage to read about Newton's method which serves as good introduction. Of course, fsolve uses more complicated and efficient techniques, but this should be a good baseline.
Hope I was clear and understood, and this has helped you!
Related
I am trying to solve the following problem using fipy but I am a bit overwhelmed. Hoping someone can point me in the right direction.
C and q are the dependent variables and I am solving in 2 dimensions, time (t) and length (z).
Is there any example I can follow? Where should I be looking at in the documentation? Or is there a some other library better suited for this problem?
Thanks in advance.
examples.diffusion.mesh1D is the starting point for anybody using FiPy. It runs through the basic concepts of how to set up a problem and it covers the transient and diffusive behavior of your first equation.
examples.convection.robin illustrates a static convection-diffusion-source problem with a Robin boundary condition. This covers all the terms on the right-hand side of your first equation and your first boundary condition.
The two equations can either be "swept" in succession until they converge, or they can be coupled. examples.diffusion.coupled illustrates both approaches for a pair of diffusion equations.
If these aren't enough to get you started, please come back with specific questions about where you're having trouble.
I am looking for a definitive guide on formulating a CVXOPT quadratic programming problem with quadratic constraints. There are good documents provided here:
The problem statement I am dealing with is identical to the problem here:
What is the matrix G supposed to look like? I've formulated as a system of linear equations, but looking at examples this does not appear to be correct?
The best resource I've found is https://courses.csail.mit.edu/6.867/wiki/images/a/a7/Qp-cvxopt.pdf, but the links at the end are dead for more reading.
I have an ipython notebook trying to use this programming method but it continually fails: https://gist.github.com/jaredvacanti/62010beda0ccfc20d2eac3c900858e50
Edit: I have edited the data source file in the notebook to provide access to the real data used in this optimization problem.
The notebook you have posted seems to have it all figured out. The problem I had is that the source file for data is not available.
Now to your question:
What is the matrix G supposed to look like? I've formulated as a system of
linear equations, but looking at examples this does not appear to be correct?
Rewrite your "linear equations" into matrix form, i.e.
2x + 2y = 4
x - y = 1
is equivalent to
matrix([[2,2],[1,-1]]) * matrix([[x],[y]]) = matrix([[4],[1]])
where matrix is from cvxopt.
I have ran into a shape matching problem and one term which I read about is deterministic annealing. I want to use this method to convert discrete problems, e.g. travelling salesman problem to continuous problems which could assist of sticking in the local minima. I don't know whether there is already an implementation of this statistical method and also implementation of it seems a bit challenging for me because I couldn't completely understand what this method does and couldn't find enough documentations. Can somebody explain it more or introduce a library especially in python that got already implemented?
You can see explication on Simulated annealing. Also, take a look to scipy.optimize.anneal.
I'm trying to optimize a 4 dimensional function with scipy. Everything works so far, except that I'm not satisfied with the quality of the solution. Right now I have ground truth data, which I use to verify my code. What I get so far is:
End error: 1.52606896507e-05
End Gradient: [ -1.17291295e-05 2.60362493e-05 5.15347856e-06 -2.72388430e-05]
Ground Truth: [0.07999999..., 0.0178329..., 0.9372903878..., 1.7756283966...]
Reconstructed: [ 0.08375729 0.01226504 1.13730592 0.21389899]
The error itself sounds good, but as the values are totally wrong I want to force the optimization algorithm (BFGS) to do more steps.
In the documentation I found the options 'gtol' and 'norm' and I tried to set both to pretty small values (like 0.0000001) but it did not seem to change anything.
Background:
The problem is, that I try to demodulate waves, so I have sin and cos terms and potentially many local (or global) minima. I use bruteforce search to find a good starting point, witch helps a lot, but it currently seems that the most work is done by that brute force search, as the optimization uses often only one iteration step. So I'm trying to improve that part of the calculation somehow.
Many local minima + hardly any improvement after brute search, that sounds bad. It's hard to say something very specific with the level of detail you provide in the question, so here are vague ideas to try (basically, what I'd do if I suspect my minimizer gets stuck):
try manually starting the minimizer from a bunch of different initial guesses.
try using a stochastic minimizer. You're tagging a question scipy, so try basinhopping
if worst comes to worst, just throw random points in a loop, leave it to work over the lunch break (or overnight)
Also, waves, sines and cosines --- it might be useful to think if you can reformulate your problem in the Fourier space.
I found out that the gradient at the starting point is already very flat (values in 10^-5), so I tried to scale the gradient function witch I already provided. This seemed to be pretty effective, I could force the Algorithm to do much more steps and my results are far better now.
They are not perfect though, but a complete discussion of this is outside of the bounds of this question, so I might start a new one, where I describe the whole problem from bottom up.
I'm looking for a good library that will integrate stiff ODEs in Python. The issue is, scipy's odeint gives me good solutions sometimes, but the slightest change in the initial conditions causes it to fall down and give up. The same problem is solved quite happily by MATLAB's stiff solvers (ode15s and ode23s), but I can't use it (even from Python, because none of the Python bindings for the MATLAB C API implement callbacks, and I need to pass a function to the ODE solver). I'm trying PyGSL, but it's horrendously complex. Any suggestions would be greatly appreciated.
EDIT: The specific problem I'm having with PyGSL is choosing the right step function. There are several of them, but no direct analogues to ode15s or ode23s (bdf formula and modified Rosenbrock if that makes sense). So what is a good step function to choose for a stiff system? I have to solve this system for a really long time to ensure that it reaches steady-state, and the GSL solvers either choose a miniscule time-step or one that's too large.
If you can solve your problem with Matlab's ode15s, you should be able to solve it with the vode solver of scipy. To simulate ode15s, I use the following settings:
ode15s = scipy.integrate.ode(f)
ode15s.set_integrator('vode', method='bdf', order=15, nsteps=3000)
ode15s.set_initial_value(u0, t0)
and then you can happily solve your problem with ode15s.integrate(t_final). It should work pretty well on a stiff problem.
(See also Link)
Python can call C. The industry standard is LSODE in ODEPACK. It is public-domain. You can download the C version. These solvers are extremely tricky, so it's best to use some well-tested code.
Added: Be sure you really have a stiff system, i.e. if the rates (eigenvalues) differ by more than 2 or 3 orders of magnitude. Also, if the system is stiff, but you are only looking for a steady-state solution, these solvers give you the option of solving some of the equations algebraically. Otherwise, a good Runge-Kutta solver like DVERK will be a good, and much simpler, solution.
Added here because it would not fit in a comment: This is from the DLSODE header doc:
C T :INOUT Value of the independent variable. On return it
C will be the current value of t (normally TOUT).
C
C TOUT :IN Next point where output is desired (.NE. T).
Also, yes Michaelis-Menten kinetics is nonlinear. The Aitken acceleration works with it, though. (If you want a short explanation, first consider the simple case of Y being a scalar. You run the system to get 3 Y(T) points. Fit an exponential curve through them (simple algebra). Then set Y to the asymptote and repeat. Now just generalize to Y being a vector. Assume the 3 points are in a plane - it's OK if they're not.) Besides, unless you have a forcing function (like a constant IV drip), the MM elimination will decay away and the system will approach linearity. Hope that helps.
PyDSTool wraps the Radau solver, which is an excellent implicit stiff integrator. This has more setup than odeint, but a lot less than PyGSL. The greatest benefit is that your RHS function is specified as a string (typically, although you can build a system using symbolic manipulations) and is converted into C, so there are no slow python callbacks and the whole thing will be very fast.
I am currently studying a bit of ODE and its solvers, so your question is very interesting to me...
From what I have heard and read, for stiff problems the right way to go is to choose an implicit method as a step function (correct me if I am wrong, I am still learning the misteries of ODE solvers). I cannot cite you where I read this, because I don't remember, but here is a thread from gsl-help where a similar question was asked.
So, in short, seems like the bsimp method is worth taking a shot, although it requires a jacobian function. If you cannot calculate the Jacobian, I will try with rk2imp, rk4imp, or any of the gear methods.