We Keep Coding

Using ipopt within Pyomo, how can I query whether the current solution is feasible?

I have an ipopt model that sometimes suffers from small numerical issues. I have a correction algorithm that can fix them, but may cause small violations of other inequalities. I need a way to determine whether the current solution is feasible without manually querying each variable bound and constraint. How can I do this within pyomo?
I know there's a way to log infeasible constraints to standard output, but this does not help me. I need my code to react dynamically to these infeasibilities, such as running a few more iterations post-correction.
More info:
I have a (relatively) small but highly nonlinear problem modeled in Pyomo. The model sometimes suffers from numerical issues with ipopt when certain variables are near zero. (I'm modeling unit vectors from full vectors via max*uvec=vec with some magnitude constraint, which becomes really messy when the magnitude of a vector is near-zero.) Fortunately, it is possible to compute everything in the model from a few key driving variables, so that small numerical infeasibilities in definition-type constraints (e.g. defining unit vectors) are easily resolved, but such resolution may cause small violations of the main problem constraints.
Some things I've tried:
The log_infeasible_constraints function from from pyomo.util.infeasible: only prints to standard output, and I cannot find any documentation on the function (to see if there are flags allowing it to be used for my needs). (The function returns None, so I can't e.g. simply check return string length.)
Update: I found the source code at https://github.com/Pyomo/pyomo/blob/master/pyomo/util/infeasible.py, which could be salvaged to create a (clunky) solution. However, I suspect this could still miss some things (e.g. other tolerance criteria) that could cause ipopt to consider a solution infeasible.
is_feasible = (results.solver.status == SolverStatus.ok) #(where opt=SolverFactory('ipopt'), and results=opt.solve(model))
This only works immediately after the solve. If the solver runs out of iterations or variables are changed post-solve, this gives no indication of current model feasibility.
(Currently, I blindly run the correction step after each solve since I can't query model feasibility.)

GEKKO optimisation gets stuck (APM.exe)

I am trying to run a differential equation system solver with GEKKO in Python 3 (via Jupyter notebook).
For bad initial parameters it of course immediately halts and says solution not found. Or immediately concludes with no changes done to the parameters / functions.
For other initial parameters the CPU (APM.exe in taskmanager) is busy for 30 seconds (depending on problem size), then it goes back down to 0% usage. But some calculation is still running and does not produce output. The only way to stop it (other than killing python completely) is to stop the APM.exe. I have not get a solution.
When doing this, I get the disp=True output me it has done a certain number of iterations (same for same initial parameters) and that:
No such file or directory: 'C:\\Users\\MYUSERNAME\\AppData\\Local\\Temp\\tmpfetrvwwwgk_model7\\options.json'
The code is very lengthy and depends on loading data from file. So I can not really provide a MWE.
Any ideas?

Here are a few suggestions on troubleshooting your application:
Sequential Simulation: If you are simulating a system with IMODE=4 then I would recommend that you switch to m.options.IMODE=7 and leave m.solve(disp=True) to see where the model is failing.
Change Solver: Sometimes it helps to switch solvers as well with m.options.SOLVER=1.
Short Time Step Solution: You can also try using a shorter time horizon initially with m.time=[0,0.0001] to see if your model can converge for even a short time step.
Coldstart: For optimization problems, you can try m.options.COLDSTART=2 to help identify equations or constraints that lead to an infeasible solution. It breaks the problem down into successive pieces that it can solve and tries to find the place that is causing convergence problems.
There are additional tutorials on simulation with Gekko and a troubleshooting guide (see #18) that helps you dig further into the application. When there are problems with convergence it is typically from one of the following issues:
Divide by zero: If you have an equation such as m.Equation(x.dt()==(x+1)/y) then replace it with m.Equation(y*x.dt()==x+1).
Non-continuously differentiable functions: Replace m.abs(x) with m.abs2(x) or m.abs3(x) that do not have a problem with x=0.
Infeasible: Remove any upper or lower bounds on variables.
Too Many Time Points: Try fewer time points or a shorter horizon initially to verify the model.
Let us know if any of this helps.

Is there any way to do scipy.optimize.minimize (or something functionally equivalent) in parallel?

I have a multivariate optimization problem that I want to run. Each evaluation is quite slow. So obviously the ability to thread it out to multiple machines would be quite nice. I have no trouble writing the code to dispatch jobs to other machines. However, scipy.optimize.minimize calls each evaluation call sequentially; it won't give me another set of parameters to evaluate until the previous one returns.
Now, I know that the "easy" solution to this would be "run your evaluation task in a parallel manner - break it up". Indeed, while it is possible to do this to some extent, it only goes so far; the bandwidth rises the more you split it up until splitting it up further actually starts to slow you down. Having another axis in which to parallelize - aka, the minimization function itself - would greatly increase scaleability.
Is there no way to do this with scipy.optimize.minimize? Or any other utility that performs in a roughly functionally equivalent manner (trying to find as low of a minimum as possible)? Surely it's possible for a minimization utility to make use of parallelism, particularly on a multivariate optimization problem where there's many axes whose gradients relative to each other at given datapoints need to be examined.

Python cvxopt glpk ilp return first feasible solution

I am using cvxopt.glpk.ilp to solve a very complicated Mixed Integer Program. I was wondering if there is a way to get the program to terminate after finding the first solution? It takes too long and a feasible solution would work fine for my purposes.

If I understand you correctly, you are only interested in a feasible solution?
Then it should be sufficient to set the objective function to zero.
Here is a Wikipedia example for an integer linear program and the following Python code returns a feasible solution:
import cvxopt
import cvxopt.glpk
# original program maximizing the second variable
# c=cvxopt.matrix([0,-1],tc='d')
# modified program returning only a feasible solution
c=cvxopt.matrix([0,0],tc='d')
G=cvxopt.matrix([[-1,1],[3,2],[2,3],[-1,0],[0,-1]],tc='d')
h=cvxopt.matrix([1,12,12,0,0],tc='d')
(status, x)=cvxopt.glpk.ilp(c,G.T,h,I=set([0,1]))
print('status',status)
print('variables',x[0],x[1])
print('objective',sum(c.T*x))

If you're using PuLP (another python library like cvxopt) to invoke glpk to solve MIP, there is one parameter called maxtime. If you set maxtime=1 then what will solver do is, terminate search (almost) right after finding the first solution. I bet cvxopt should have some similar parameter for glpk since either PuLP or cvxopt is just a wrapper of those solvers.
Copy paste the description of maxtime parameter in Xpress which is another solver, but I think glpk should have something similar which you may need to find it out.
The maximum time in seconds that the Optimizer will run before it terminates, including the problem setup time and solution time. For MIP problems, this is the total time taken to solve all the nodes.
0 = No time limit.
n > 0 = If an integer solution has been found, stop MIP search after n seconds, otherwise continue until an integer solution is finally found.
n < 0 = Stop in LP or MIP search after -n seconds.

fmin_bfgs does not complete

I'm trying to minimize a function with lots of parameters (a little over 7000) using fmin_bfgs() or fmin_l_bfgs_b(). When I enter the command
opt_pars = fmin_l_bfgs_b(obj_f, pars, approx_grad=1)
(where obj_f is the function I'm trying to minimize and pars is the vector of initial parameters) the function just runs forever until python tells me that it has to terminate the program. There is never any output. I tried adding the argument maxfunc = 2 to see if it was getting anywhere at all and the same thing happened (ran forever then python terminated the program).
I'm just trying to figure out what could be going wrong with the function. It seems like maybe it's getting caught in a while loop or something. Has anyone encountered this problem? If not, I could also use some general debugging help here (as I'm relatively new to Python) on how to monitor what the function is doing.
And finally, maybe someone can recommend a different function or package for the task I'm attempting. I'm trying to fit a lasso regularized Poisson regression to sparse data with about 12 million observations of 7000 variables.
PS Sorry for not including the -log likelihood function I'm trying to minimize, but it would be completely uninterpretable.
Thanks a lot for any help!
Zach

Since you don't provide gradients to fmin_bfgs and fmin_l_bfgs_b, your objective function is evaluated len(x) > 7000 times each time the gradient is needed. If the objective function is slow to evaluate, that will add up.
The maxfun option doesn't apparently count the gradient estimation, so it's possible that it's actually not an infinite loop, just that it takes a very long time.
What do you mean by "python tells me that it has to terminate the program"?
Please in any case try to provide a reproducible test case here. It doesn't matter if the objective function is incomprehensible --- what is important is that people interested can reproduce the condition you encounter.
I don't see infinite loops problem on my system even for 7000 parameters. However, the function evaluation count was about 200000 for a simple 7000-parameter problem with l_bfgs_b and no gradient provided. Profile your code to see what such evaluation counts would mean for you. With gradient provided, it was 35 (+ 35 times the gradient). Providing a gradient may then help. (If the function is complicated, automatic differentiation may still work --- there are libraries for that in Python.)
Other optimization libraries for Python, see: http://scipy.org/Topical_Software (can't say which are the best ones, though --- ipopt or coin-or could be worth a try)
For reference: the L-BFGS-B implementation in Scipy is this one (and is written by guys who are supposed to know what they are doing):
http://users.eecs.northwestern.edu/~nocedal/lbfgsb.html
***
You can debug what is going on e.g. by using Python debugger pdb, python -m pdb your_script.py. Or just by inserting print statements inside it.
Also try to Google "debug python" and "profile python" ;)