Vector-valued Constraints in NLOpt - python

I'm trying to add some equality and inequality constraints to my minimization problem.
I am using nlopt Python API.
In particular I would like to add some vector-valued constraints.
My code looks like this example:
def eqconstr(result,x,grad):
if grad.size > 0:
print "gradient to be implemented"
for i in range(len(x)):
if condition: result[i] = 0.
initvect = # some initial guess of the parameters
opt = nlopt.opt(nlopt.LN_PRAXIS,len(initvect))
opt.set_min_objective(function)
tol = np.asarray([0.1]*len(initvect))
opt.add_equality_mconstraint(eqconstr, tol) # this is the call of the constraints (!!!)
opt.set_lower_bounds(-1.) # other parameters to set. not important for this question
opt.set_upper_bounds(1.)
opt.set_initial_step([0.1]*len(initvect))
opt.set_xtol_abs(10e-6)
opt.set_ftol_abs(10e-6)
res = opt.optimize(initvect)
This follows precisely the instructions in the nlopt wiki.
Now if I run this I get:
Traceback (most recent call last):
File "main.py", line 158, in <module>
opt.add_equality_mconstraint(eqconstr, tol)
File "/usr/local/lib/python2.7/dist-packages/nlopt.py", line 269, in add_equality_mconstraint
def add_equality_mconstraint(self, *args): return _nlopt.opt_add_equality_mconstraint(self, *args)
ValueError: nlopt invalid argument

Make sure that your function eqcontr has the same form as your objective function function. Maybe post it as well, so it'll be easy to understand. Also, I cannot see where condition is defined.

Related

NotImplementedError: Initial conditions produced too many solutions for constants [SymPy]

I attempted to solve this differential equation by adapting the code found here.
import sympy as sp
t = sp.symbols('t', real=True)
x = sp.Function('x', real=True)
diffeq = sp.Eq(x(t).diff(t), sp.sqrt(1 - x(t)**2))
res = sp.dsolve(diffeq, ics={x(0): 0})
The math.se post suggests that a piecewise function satisfies the equation, but perhaps my code does not add all the necessary assumptions. I get the following traceback:
Traceback (most recent call last):
File "/usr/lib/python3.10/idlelib/run.py", line 578, in runcode
exec(code, self.locals)
File "/home/galen/testing.py", line 7, in <module>
res = sp.dsolve(diffeq, ics={x(0): 0})
File "/usr/lib/python3/dist-packages/sympy/solvers/ode/ode.py", line 639, in dsolve
return _helper_simplify(eq, hint, hints, simplify, ics=ics)
File "/usr/lib/python3/dist-packages/sympy/solvers/ode/ode.py", line 694, in _helper_simplify
solved_constants = solve_ics([rv], [r['func']], cons(rv), ics)
File "/usr/lib/python3/dist-packages/sympy/solvers/ode/ode.py", line 808, in solve_ics
raise NotImplementedError("Initial conditions produced too many solutions for constants")
NotImplementedError: Initial conditions produced too many solutions for constants
How do I change my code to correctly solve the ODE?
First, the solver treats the equation as algebraic, that is, it solves x'²=1-x², and it does not get the piecewise solution that are possible, only the fully analytical.
For some reason returning a tuple of possible solutions is not implemented, you can do that manually, see https://stackoverflow.com/a/46979535/3088138
res = sp.dsolve(diffeq)
sol = res.rhs
print([ sol.subs(coeffs) for coeffs in sp.solve(sol.subs(t,0), dict=True) ])
which returns [sin(t), -sin(t)]. The second is not compatible with the ODE in its given form.

Theano scan function and argument number lstm

I am new to Neural Networks and I am trying to modify this code RNN-Classifier and instead of using the GRU_step, I would rather use an LSTM.
I added one extra parameter c_prev
def lstm_step(x, h_prev, c_prev, W_xz, W_hz, W_xm, W_hm):
and after applying all the LSTM equations I am returning them both (h and c)
My hidden vector looks like that:
hidden_vector, _ = theano.scan(
lstm_step,
sequences=input_vectors,
outputs_info=initial_hidden_vector,
non_sequences=[W_xz, W_hz, W_xm, W_hm]
)
hidden_vector = hidden_vector[-1]
I get an exception like this and don't understand why it does not see the c_prev as an existant parameter (or how/where can I feed it with some values, so that it's not empty?)
python rnnclassifier.py data/sentiment.train.txt data/sentiment.test.txt
Traceback (most recent call last):
File "rnnclassifier.py", line 167, in <module>
rnn_classifier = RnnClassifier(word2id_len, n_classes)
File "rnnclassifier.py", line 110, in __init__
non_sequences=[W_xz, W_hz, W_xm, W_hm]
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan.py",
line 773, in scan condition, outputs, updates =
scan_utils.get_updates_and_outputs(fn(*args))
TypeError: lstm_step() takes exactly 7 arguments (6 given)
I am new to this topic and would appreciate any help or advice! Thank you.

multi-threading in Python with pulp

I have a function which accepts a list R. In this function, I have defined an optimization problem using "pulp", This is my function:
import pulp
from multiprocessing.dummy import Pool as ThreadPool
def optimize(R):
variables = ["x1","x2","x3","x4"]
costs = {"x1":R[0], "x2":R[1], "x3":R[2], "x4":R[3]}
constraint = {"x1":5, "x2":7, "x3":4, "x4":3}
prob_variables = pulp.LpVariable.dicts("Intg",variables,
lowBound=0,
upBound=1,
cat=pulp.LpInteger)
prob = pulp.LpProblem("test1", pulp.LpMaximize)
# defines the constraints
prob += pulp.lpSum([constraint[i]*prob_variables[i] for i in variables]) <= 14
# defines the objective function to maximize
prob += pulp.lpSum([costs[i]*prob_variables[i] for i in variables])
pulp.GLPK().solve(prob)
# Solution
return pulp.value(prob.objective)
To get the output, I used a list as my input and the output is correct:
my_input = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
results =[]
for i in range(0,len(my_input)):
results.append(optimize(my_input[i]))
print("*"*20)
print(results)
But, I want to use multi-threading instead of the for loop. So, I used:
my_input = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
pool = ThreadPool(4)
results = pool.map(optimize, my_input)
But it gives me some errors:
Traceback (most recent call last):
File "/Users/Mohammad/PycharmProjects/untitled10/multi_thread.py", line 35, in <module>
results = pool.map(optimize, my_input)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/Users/Mohammad/PycharmProjects/untitled10/multi_thread.py", line 27, in optimize
pulp.GLPK().solve(prob)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/PuLP-1.6.1-py3.5.egg/pulp/solvers.py", line 179, in solve
return lp.solve(self)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/PuLP-1.6.1-py3.5.egg/pulp/pulp.py", line 1643, in solve
status = solver.actualSolve(self, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/PuLP-1.6.1-py3.5.egg/pulp/solvers.py", line 377, in actualSolve
raise PulpSolverError("PuLP: Error while executing "+self.path)
pulp.solvers.PulpSolverError: PuLP: Error while executing glpsol
Can anybody help me?
In my actual code, my_input list has the length of 27 (instead of 4 in the above code) and for each one, in my function I have to perform 80k optimizations (instead of one in the above code). So, multi-threading is a big help for me.
I have seen that class pulp.solvers.COIN_CMD has a threads argument, although the documentation is quite laconic. Taking a look at the code source, it seems to be indeed a way to provide threads to the solver.
If naming is indeed the issue, consider adding the desired name index for a given problem as an input argument to the function. Something like:
def optimize(tup): # here, tup contains (idx, R), so as to be callable using pool.map
...
prob = pulp.LpProblem('test'+str(idx), pulp.LpMaximize)
...
and then something like:
my_input = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
pool = ThreadPool(4)
results = pool.map(optimize, enumerate(my_input))

TypeError with Python DEAP mutFlipBit

I'm writing my first evolutionary algorithm with DEAP.
Everything is working, but the MultiFlipBit mutation operator.
When I attempt to mutate a Tree (Individual), I get the following error:
File "Genetic_Programming.py", line 92, in main
halloffame=hof, verbose=True)
offspring = varOr(population, toolbox, lambda_, cxpb, mutpb)
File "/Users/anaconda/lib/python2.7/site-packages/deap/algorithms.py", line 235, in varOr
ind, = toolbox.mutate(ind)
File "/Users/anaconda/lib/python2.7/site-packages/deap/tools/mutation.py", line 132, in mutFlipBit
individual[i] = type(individual[i])(not individual[i])
TypeError: __init__() takes exactly 4 arguments (2 given)
Here is the code:
pset = gp.PrimitiveSet("MAIN", 8)
pset.addPrimitive(operator.and_, 2)
pset.addPrimitive(operator.or_, 2)
pset.addPrimitive(operator.xor, 2)
pset.addPrimitive(operator.not_, 1)
creator.create("FitnessMax", base.Fitness, weights=(1.0,))
creator.create("Individual", gp.PrimitiveTree, fitness=creator.FitnessMax)
toolbox = base.Toolbox()
toolbox.register("expr", gp.genGrow, pset=pset, min_=1, max_=8)
toolbox.register("individual", tools.initIterate, creator.Individual, toolbox.expr)
toolbox.register("population", tools.initRepeat, list, toolbox.individual)
toolbox.register("compile", gp.compile, pset=pset)
def evalSymbReg(individual):
# Transform the tree expression in a callable function
print individual
ind = toolbox.compile(expr=individual)
# Evaluate the mean squared error between the expression
# and the real function : x**4 + x**3 + x**2 + x
performance=Genetic_V0.genetic_backtest(ind)
return performance,
toolbox.register("evaluate", evalSymbReg)
toolbox.register("select", tools.selTournament, tournsize=50)
toolbox.register("mate", gp.cxOnePoint)
#toolbox.register("expr_mut", gp.genGrow, min_=1, max_=4)
#toolbox.register("mutate", tools.mutFlipBit, expr=toolbox.expr_mut, pset=pset)
toolbox.register("mutate", tools.mutFlipBit, indpb=0.95)
def main():
nu=50
pop = toolbox.population(n=nu)
hof = tools.HallOfFame(3)
stats_fit = tools.Statistics(lambda ind: ind.fitness.values)
stats_size = tools.Statistics(len)
mstats = tools.MultiStatistics(fitness=stats_fit, size=stats_size)
mstats.register("avg", numpy.mean)
mstats.register("std", numpy.std)
mstats.register("min", numpy.min)
mstats.register("max", numpy.max)
pop, log = algorithms.eaMuPlusLambda(pop, toolbox, nu/2, nu/2, 0.85, 0.15, 200,stats=mstats,
halloffame=hof, verbose=True)
# print log
return pop, log, hof
if __name__ == "__main__":
main()
Thanks in advance for the help.
python version: 2.7
EDITED:
After a suggestion on how solving the problem, I've added to the DEAP mutation library a couple of "print" to better understand what is happening. Here is the same error, as per original question, but with some extra information:
individual is not_(ARG7)
individual[i] is <deap.gp.Primitive object at 0x10746c158>
(not individual[i]) is False
type(individual[i]) is <class 'deap.gp.Primitive'>
type(individual[i])(not individual[i]) is
Traceback (most recent call last):
File "Genetic_Programming.py", line 98, in <module>
main()
File "Genetic_Programming.py", line 92, in main
halloffame=hof, verbose=True)
File "/Users/giorgio/anaconda/lib/python2.7/site-packages/deap/algorithms.py", line 317, in eaMuPlusLambda
offspring = varOr(population, toolbox, lambda_, cxpb, mutpb)
File "/Users/giorgio/anaconda/lib/python2.7/site-packages/deap/algorithms.py", line 235, in varOr
ind, = toolbox.mutate(ind)
File "/Users/giorgio/anaconda/lib/python2.7/site-packages/deap/tools/mutation.py", line 136, in mutFlipBit
print "type(individual[i])(not individual[i]) is ", type(individual[i])(not individual[i])
TypeError: __init__() takes exactly 4 arguments (2 given)
Thanks again for any contributes
The error is complaining that you are trying to use a deap.tools.mutFlipBit function as a mutator on a indivual that is instance of deap.gp.PrimitiveTree. From the docs of mutFlipBit:
The individual is expected to be a sequence and the values of the attributes shall stay valid after the not operator is called on them.
So if the code decided to mutate the element i of individual, it is trying to evaluate deap.dp.Primitive(not ind[i]). And the error complains that the constructor of Primitive takes 4 arguments, self, name, args, ret instead of the passed self, (not ind[1]).
The cause of the error is twofold,
A mutBitFlip mutator is being a applied to an individual that consists of a list of both "symbolic values" and "operators", instead of a list of booleans as expected. So the code could try to negate an operator, e.g. fliping an and, which is not defined. However, the code will produce False because an object is evaluated to True in a boolean context, and will end up trying to create a new Primitive from a single argument which is a boolean value.
Even if mutFlipBit function could discriminate between a Primitive and a Terminal, and would only try to negate Terminals, deap will use symbolic terminals, because the primary target of a PrimitiveTree is not to find a single expressions equivalent to a value (i.e. constant), but rather a expressions, with arguments, equivalent to a function.
You should revisit what are you trying to evolve, an expression that maps to a boolean equation or a set of arguments to a boolean function. And based on that redesign your code.
INITIAL ANSWER:
Your code is not complete, since the line offspring = varOr(population, toolbox, lambda_, cxpb, mutpb) is not shown.
However, the problem is caused by the type of one of your invidivuals in your population variable. The code is assuming that your individuals are a list of booleans, so the code individual[i] = type(individual[i])(not individual[i]) would work:
x = True
print type(x)(not x) # Equivalent of bool(True) => False
The error says that, type(individual[i]) got the init of a class which expects 3 arguments, e.g. self + other 3 arguments.

python error using PaCal statistical package

I recently started exploring Python and have encountred a problem with a package named PaCal
Everything looks to be working fine except that I keep having this error anytime I want to print out some data (like in print A.mean() )
the error line is :
Traceback (most recent call last):
File "C:\Users\rmobenta\Desktop\tt.py", line 12, in <module>
print A.interval(0.95)
File "C:\Python27\lib\site-packages\pacal\distr.py", line 229, in interval
return self.quantile(p_lim), self.quantile(1.0 - p_lim)
File "C:\Python27\lib\site-packages\pacal\distr.py", line 215, in quantile
return self.get_piecewise_cdf().inverse(y)
File "C:\Python27\lib\site-packages\pacal\segments.py", line 1721, in inverse
x = findinv(segi.f, a = segi.a, b = segi.b, c = y, rtol = params.segments.cumint.reltol, maxiter = params.segments.cumint.maxiter) # TODO PInd, MInf
File "C:\Python27\lib\site-packages\pacal\utils.py", line 384, in findinv
return brentq(lambda x : fun(x) - c, a, b, **kwargs)
File "C:\Python27\lib\site-packages\scipy\optimize\zeros.py", line 414, in brentq
raise ValueError("rtol too small (%g < %g)" % (rtol, _rtol))
ValueError: rtol too small (1e-16 < 4.44089e-16)
I am using a two-line script that I got for a demo (given by the author of this package) and have no idea how to tackle this issue.
Here is the script:
from pacal import *
Y = UniformDistr(1, 2)
X = UniformDistr(3, 4)
A = atan(Y / X)
A.plot()
print A.mean()
print A.interval(0.95)
The problem comes from PaCal that defines in l.141 of params.py: segments.vumint.reltol = 1e-16.
This is the value passed as rtol in segments.py to the SciPy function brentq().
Finally it is compared to numpy.finfo(float).eps * 2 (l.413 and l.10 of scipy/optimize/zeros.py) and is unfortunately lesser.
So it could be a problem of PaCal implementation, not related to your code.
Note that the value you provided to interval() corresponds to the default value (l.222 of distr.py).
I think you should contact the PaCal developers to get more informations and probably open an issue.

Categories