I have a complicated non-linear problem I cannot copy here, where I have used IPOPT.exe so far (v 3.9.1) in a pyomo environment without a problem. Lately I experienced that the value of the objective function highly varies based on the settings:
if I decrease the tolerance the value gets more accurate at first (for some digits), but then it jumps to another value, which is 10% bigger than the previous
my optimization problem is the same for every run, except the value of one parameter (the hour I am evaluating the function in), still IPOPT can sometimes find the solution in 2 sec, sometimes I get infeasible result or max iteration reached (the parameter change is in a range that should not cause any anomaly in the system)
since it is a nasty problem I expected that ipopt randomly "jumps into" the problem, but it is not the case, it returns the same exit and value even if trying the problem several times (with for attempt: - try: - etc.).
also the result is different depending on if I use "exact" or "limited memory" hessian approximation.
since I do not like this behavior I wanted to use the multistart option but I cannot define an "executable" path with the option so I figured I change from the executable file to an installed solver.
I installed cyipopt and the ipopt to have everything and it works fine with another sample example but does not on my problem. It returns: ERROR: Solver (ipopt) returned non-zero return code (3221225501) without any feedback. I cannot figure out the cause.
So the questions would be:
how do I know which options to set and which is its proper value?
why do I keep always getting the same result even if I start the process completely new?
can I define an executable with the multistart option? and will it change anything or I should focus more on initializing the variables to a better value?
which is the easiest way to foster convergence?
I know it is not one issue but they belong together in the same context. Thanks for the answers in advance!
Related
I am working with some MIP models that only have binary variables as integer variables which represent optional elements in a network. Instead of solving the complete MIP I want to try out an algorithm where I do the following steps:
Solve the problem with all binaries fixed to zero
Solve the problem with all binaries fixed to one
Use the difference in objective as budget to remove some of the binaries with too expensive objective contribution from the pool of options and solve again; repeat this step until either no binaries remain or no binary is removed during the iteration
Solve the reduced MIP where some of the binaries are set to zero which reduces the number of practical binary variables
From my understanding this algorithm would benefit from using warmstarts as the last solution including the changes through variables fixings after the solve() calls would provide a feasible solution and bound for the model. I also use the deactivate()- and activate()-methods to change objectives and remove a constraint during step 2 and 3. For the constraint I also wrote code that sets the variables to a feasible solution after reactivating it.
When executing
results = optimizer.solve(self.pyM, warmstart=True, tee=True)
using Gurobi it seems that gurobi doesn't use the current variable values in the pyomo model but instead only uses the values from the last solve()-results without the changes afterwards (fixing variables to one/zero, adjusting values for the constraint).
I assume this because if I don't reactivate the constraint and run a model where no binaries can be removed the log reports a working MIP Start while when activating it does give the same output:
Read MIP start from file /tmp/tmp3lq2wbki.gurobi.mst
[some parameters settings and model characteristics]
User MIP start did not produce a new incumbent solution
User MIP start violates constraint c_e_x96944_ by 0.450000000
regardless if I comment out the code that adjusts the values or not. I also expect that code snippet to work properly as I tested it separately and checked with help of the display()-method that the value of the constraint body lies between both bounds. In Step 2 only the read MIP Start line from above is in the log, but no statement what happend to it.
Is it possible to tell Pyomo to use the values from the Pyomo model instead or to update the .mst-file with the updated pyomo model values?
I found this Gurobi-persistent Class
https://pyomo.readthedocs.io/en/stable/library_reference/solvers/gurobi_persistent.html
I tried
import pyomo.solvers.plugins.solvers.gurobi_persistent as gupyomo
[...]
test = gupyomo.GurobiPersistent(model = self.pyM)
for variable in adjustedVariables:
test.update_var(variable)
test.update()
but this neither produced output/errors nor changed the behaviour. Therefore I assume this is either not the correct way or I used it wrong.
Additional infos:
gurobi 9.0.2
pyomo 5.7.1
If specific parts of the code might be helpful I can provide them but I wasn't sure if and what part could be relevant for the question
So what seemed to work for me was instead using
optimizer.set_instance(self.pyM) at the start of my code and using optimizer.update_var(var[index]) whenever I change something like fixing the variable var[index].
, instead of above code.
I have an ipopt model that sometimes suffers from small numerical issues. I have a correction algorithm that can fix them, but may cause small violations of other inequalities. I need a way to determine whether the current solution is feasible without manually querying each variable bound and constraint. How can I do this within pyomo?
I know there's a way to log infeasible constraints to standard output, but this does not help me. I need my code to react dynamically to these infeasibilities, such as running a few more iterations post-correction.
More info:
I have a (relatively) small but highly nonlinear problem modeled in Pyomo. The model sometimes suffers from numerical issues with ipopt when certain variables are near zero. (I'm modeling unit vectors from full vectors via max*uvec=vec with some magnitude constraint, which becomes really messy when the magnitude of a vector is near-zero.) Fortunately, it is possible to compute everything in the model from a few key driving variables, so that small numerical infeasibilities in definition-type constraints (e.g. defining unit vectors) are easily resolved, but such resolution may cause small violations of the main problem constraints.
Some things I've tried:
The log_infeasible_constraints function from from pyomo.util.infeasible: only prints to standard output, and I cannot find any documentation on the function (to see if there are flags allowing it to be used for my needs). (The function returns None, so I can't e.g. simply check return string length.)
Update: I found the source code at https://github.com/Pyomo/pyomo/blob/master/pyomo/util/infeasible.py, which could be salvaged to create a (clunky) solution. However, I suspect this could still miss some things (e.g. other tolerance criteria) that could cause ipopt to consider a solution infeasible.
is_feasible = (results.solver.status == SolverStatus.ok) #(where opt=SolverFactory('ipopt'), and results=opt.solve(model))
This only works immediately after the solve. If the solver runs out of iterations or variables are changed post-solve, this gives no indication of current model feasibility.
(Currently, I blindly run the correction step after each solve since I can't query model feasibility.)
I am trying to run a differential equation system solver with GEKKO in Python 3 (via Jupyter notebook).
For bad initial parameters it of course immediately halts and says solution not found. Or immediately concludes with no changes done to the parameters / functions.
For other initial parameters the CPU (APM.exe in taskmanager) is busy for 30 seconds (depending on problem size), then it goes back down to 0% usage. But some calculation is still running and does not produce output. The only way to stop it (other than killing python completely) is to stop the APM.exe. I have not get a solution.
When doing this, I get the disp=True output me it has done a certain number of iterations (same for same initial parameters) and that:
No such file or directory: 'C:\\Users\\MYUSERNAME\\AppData\\Local\\Temp\\tmpfetrvwwwgk_model7\\options.json'
The code is very lengthy and depends on loading data from file. So I can not really provide a MWE.
Any ideas?
Here are a few suggestions on troubleshooting your application:
Sequential Simulation: If you are simulating a system with IMODE=4 then I would recommend that you switch to m.options.IMODE=7 and leave m.solve(disp=True) to see where the model is failing.
Change Solver: Sometimes it helps to switch solvers as well with m.options.SOLVER=1.
Short Time Step Solution: You can also try using a shorter time horizon initially with m.time=[0,0.0001] to see if your model can converge for even a short time step.
Coldstart: For optimization problems, you can try m.options.COLDSTART=2 to help identify equations or constraints that lead to an infeasible solution. It breaks the problem down into successive pieces that it can solve and tries to find the place that is causing convergence problems.
There are additional tutorials on simulation with Gekko and a troubleshooting guide (see #18) that helps you dig further into the application. When there are problems with convergence it is typically from one of the following issues:
Divide by zero: If you have an equation such as m.Equation(x.dt()==(x+1)/y) then replace it with m.Equation(y*x.dt()==x+1).
Non-continuously differentiable functions: Replace m.abs(x) with m.abs2(x) or m.abs3(x) that do not have a problem with x=0.
Infeasible: Remove any upper or lower bounds on variables.
Too Many Time Points: Try fewer time points or a shorter horizon initially to verify the model.
Let us know if any of this helps.
I am running nested optimization code.
sp.optimize.minimize(fun=A, x0=D, method="SLSQP", bounds=(E), constraints=({'type':'eq','fun':constrains}), options={'disp': True, 'maxiter':100, 'ftol':1e-05})
sp.optimize.minimize(fun=B, x0=C, method="Nelder-Mead", options={'disp': True})
The first minimization is the part of the function B, so it is kind of running inside the second minimization.
And the whole optimization is based on the data, there's no random number involved.
I run the exactly same code on two different computers, and get the totally different results.
I have installed different versions of anaconda, but
scipy, numpy, and all the packages used have the same versions.
I don't really think OS would matter, but one is windows 10 (64bit), and the other one is windows 8.1 (64 bit)
I am trying to figure out what might be causing this.
Even though I did not state the whole options, if two computers are running the same code, shouldn't the results be the same?
or are there any options for sp.optimize that default values are set to be different from computer to computer?
PS. I was looking at the option "eps". Is it possible that default values of "eps" are different on these computers?
You should never expect numerical methods to perform identically on different devices; or even different runs of the same code on the same device. Due to the finite precision of the machine you can never calculate the "real" result, but only numerical approximations. During a long optimization task these differences can sum up.
Furthermore, some optimazion methods use some kind of randomness on the inside to solve the problem of being stuck in local minima: they add a small, alomost vanishing noise to the previous calculated solution to allow the algorithm to converge faster in the global minimum and not being stuck in a local minimum or a saddle-point.
Can you try to plot the landscape of the function you want to minimize? This can help you to analyze the problem: If both of the results (on each machine) are local minima, then this behaviour can be explained by my previous description.
If this is not the case, you should check the version of scipy you have installed on both machines. Maybe you are implicitly using float values on one device and double values on the other one, too?
You see: there are a lot of possible explanations for this (at the first look) strange numerical behaviour; you have to give us more details to solve this.
I found that different versions of SciPy do or do not allow minimum and maximum bounds to be the same. For example, in SciPy version 1.5.4, a parameter with equal min and max bounds sends that term's Jacobian to nan, which brings the minimization to a premature stop.
I'm trying to minimize a function with lots of parameters (a little over 7000) using fmin_bfgs() or fmin_l_bfgs_b(). When I enter the command
opt_pars = fmin_l_bfgs_b(obj_f, pars, approx_grad=1)
(where obj_f is the function I'm trying to minimize and pars is the vector of initial parameters) the function just runs forever until python tells me that it has to terminate the program. There is never any output. I tried adding the argument maxfunc = 2 to see if it was getting anywhere at all and the same thing happened (ran forever then python terminated the program).
I'm just trying to figure out what could be going wrong with the function. It seems like maybe it's getting caught in a while loop or something. Has anyone encountered this problem? If not, I could also use some general debugging help here (as I'm relatively new to Python) on how to monitor what the function is doing.
And finally, maybe someone can recommend a different function or package for the task I'm attempting. I'm trying to fit a lasso regularized Poisson regression to sparse data with about 12 million observations of 7000 variables.
PS Sorry for not including the -log likelihood function I'm trying to minimize, but it would be completely uninterpretable.
Thanks a lot for any help!
Zach
Since you don't provide gradients to fmin_bfgs and fmin_l_bfgs_b, your objective function is evaluated len(x) > 7000 times each time the gradient is needed. If the objective function is slow to evaluate, that will add up.
The maxfun option doesn't apparently count the gradient estimation, so it's possible that it's actually not an infinite loop, just that it takes a very long time.
What do you mean by "python tells me that it has to terminate the program"?
Please in any case try to provide a reproducible test case here. It doesn't matter if the objective function is incomprehensible --- what is important is that people interested can reproduce the condition you encounter.
I don't see infinite loops problem on my system even for 7000 parameters. However, the function evaluation count was about 200000 for a simple 7000-parameter problem with l_bfgs_b and no gradient provided. Profile your code to see what such evaluation counts would mean for you. With gradient provided, it was 35 (+ 35 times the gradient). Providing a gradient may then help. (If the function is complicated, automatic differentiation may still work --- there are libraries for that in Python.)
Other optimization libraries for Python, see: http://scipy.org/Topical_Software (can't say which are the best ones, though --- ipopt or coin-or could be worth a try)
For reference: the L-BFGS-B implementation in Scipy is this one (and is written by guys who are supposed to know what they are doing):
http://users.eecs.northwestern.edu/~nocedal/lbfgsb.html
***
You can debug what is going on e.g. by using Python debugger pdb, python -m pdb your_script.py. Or just by inserting print statements inside it.
Also try to Google "debug python" and "profile python" ;)