I'm writing my first evolutionary algorithm with DEAP.
Everything is working, but the MultiFlipBit mutation operator.
When I attempt to mutate a Tree (Individual), I get the following error:
File "Genetic_Programming.py", line 92, in main
halloffame=hof, verbose=True)
offspring = varOr(population, toolbox, lambda_, cxpb, mutpb)
File "/Users/anaconda/lib/python2.7/site-packages/deap/algorithms.py", line 235, in varOr
ind, = toolbox.mutate(ind)
File "/Users/anaconda/lib/python2.7/site-packages/deap/tools/mutation.py", line 132, in mutFlipBit
individual[i] = type(individual[i])(not individual[i])
TypeError: __init__() takes exactly 4 arguments (2 given)
Here is the code:
pset = gp.PrimitiveSet("MAIN", 8)
pset.addPrimitive(operator.and_, 2)
pset.addPrimitive(operator.or_, 2)
pset.addPrimitive(operator.xor, 2)
pset.addPrimitive(operator.not_, 1)
creator.create("FitnessMax", base.Fitness, weights=(1.0,))
creator.create("Individual", gp.PrimitiveTree, fitness=creator.FitnessMax)
toolbox = base.Toolbox()
toolbox.register("expr", gp.genGrow, pset=pset, min_=1, max_=8)
toolbox.register("individual", tools.initIterate, creator.Individual, toolbox.expr)
toolbox.register("population", tools.initRepeat, list, toolbox.individual)
toolbox.register("compile", gp.compile, pset=pset)
def evalSymbReg(individual):
# Transform the tree expression in a callable function
print individual
ind = toolbox.compile(expr=individual)
# Evaluate the mean squared error between the expression
# and the real function : x**4 + x**3 + x**2 + x
performance=Genetic_V0.genetic_backtest(ind)
return performance,
toolbox.register("evaluate", evalSymbReg)
toolbox.register("select", tools.selTournament, tournsize=50)
toolbox.register("mate", gp.cxOnePoint)
#toolbox.register("expr_mut", gp.genGrow, min_=1, max_=4)
#toolbox.register("mutate", tools.mutFlipBit, expr=toolbox.expr_mut, pset=pset)
toolbox.register("mutate", tools.mutFlipBit, indpb=0.95)
def main():
nu=50
pop = toolbox.population(n=nu)
hof = tools.HallOfFame(3)
stats_fit = tools.Statistics(lambda ind: ind.fitness.values)
stats_size = tools.Statistics(len)
mstats = tools.MultiStatistics(fitness=stats_fit, size=stats_size)
mstats.register("avg", numpy.mean)
mstats.register("std", numpy.std)
mstats.register("min", numpy.min)
mstats.register("max", numpy.max)
pop, log = algorithms.eaMuPlusLambda(pop, toolbox, nu/2, nu/2, 0.85, 0.15, 200,stats=mstats,
halloffame=hof, verbose=True)
# print log
return pop, log, hof
if __name__ == "__main__":
main()
Thanks in advance for the help.
python version: 2.7
EDITED:
After a suggestion on how solving the problem, I've added to the DEAP mutation library a couple of "print" to better understand what is happening. Here is the same error, as per original question, but with some extra information:
individual is not_(ARG7)
individual[i] is <deap.gp.Primitive object at 0x10746c158>
(not individual[i]) is False
type(individual[i]) is <class 'deap.gp.Primitive'>
type(individual[i])(not individual[i]) is
Traceback (most recent call last):
File "Genetic_Programming.py", line 98, in <module>
main()
File "Genetic_Programming.py", line 92, in main
halloffame=hof, verbose=True)
File "/Users/giorgio/anaconda/lib/python2.7/site-packages/deap/algorithms.py", line 317, in eaMuPlusLambda
offspring = varOr(population, toolbox, lambda_, cxpb, mutpb)
File "/Users/giorgio/anaconda/lib/python2.7/site-packages/deap/algorithms.py", line 235, in varOr
ind, = toolbox.mutate(ind)
File "/Users/giorgio/anaconda/lib/python2.7/site-packages/deap/tools/mutation.py", line 136, in mutFlipBit
print "type(individual[i])(not individual[i]) is ", type(individual[i])(not individual[i])
TypeError: __init__() takes exactly 4 arguments (2 given)
Thanks again for any contributes
The error is complaining that you are trying to use a deap.tools.mutFlipBit function as a mutator on a indivual that is instance of deap.gp.PrimitiveTree. From the docs of mutFlipBit:
The individual is expected to be a sequence and the values of the attributes shall stay valid after the not operator is called on them.
So if the code decided to mutate the element i of individual, it is trying to evaluate deap.dp.Primitive(not ind[i]). And the error complains that the constructor of Primitive takes 4 arguments, self, name, args, ret instead of the passed self, (not ind[1]).
The cause of the error is twofold,
A mutBitFlip mutator is being a applied to an individual that consists of a list of both "symbolic values" and "operators", instead of a list of booleans as expected. So the code could try to negate an operator, e.g. fliping an and, which is not defined. However, the code will produce False because an object is evaluated to True in a boolean context, and will end up trying to create a new Primitive from a single argument which is a boolean value.
Even if mutFlipBit function could discriminate between a Primitive and a Terminal, and would only try to negate Terminals, deap will use symbolic terminals, because the primary target of a PrimitiveTree is not to find a single expressions equivalent to a value (i.e. constant), but rather a expressions, with arguments, equivalent to a function.
You should revisit what are you trying to evolve, an expression that maps to a boolean equation or a set of arguments to a boolean function. And based on that redesign your code.
INITIAL ANSWER:
Your code is not complete, since the line offspring = varOr(population, toolbox, lambda_, cxpb, mutpb) is not shown.
However, the problem is caused by the type of one of your invidivuals in your population variable. The code is assuming that your individuals are a list of booleans, so the code individual[i] = type(individual[i])(not individual[i]) would work:
x = True
print type(x)(not x) # Equivalent of bool(True) => False
The error says that, type(individual[i]) got the init of a class which expects 3 arguments, e.g. self + other 3 arguments.
Related
I need to calculate the friction factor using the Colebrook-Equation, but unlike most cases it needs to be over a length, so the Reynolds number is a list, not a float.
I basically used what this guy this guy recommended (fixed point iteration), specially because it will be part of a bigger programm which wasn't very fast when I did it in Matlab.
For my interpretation of that code I oriented myself at the example of the fixed_point documentation, because there they also pass arguments in form of lists into the function (or actually the solver). v is a list with 1000 elements.
k = 0.000045
d = 0.03
reynolds = d/10**(-6)*v
def f(x, Re):
LHS = -2*np.log10((2.51/(Re*np.sqrt(x))) + (k/(3.71*d)))
return 1/LHS**2
x0 = [0.02]*1000
Re = np.array(reynolds)
result = fixed_point(f,x0, args = Re)
print(result)
I made sure that the starting value and the Reynolds-argument have the same length and I still get an error message.
File " ... ", line 72, in <module>
result = fixed_point(f,x0, args = Re)
...
TypeError: f() takes 2 positional arguments but 1001 were given
I feel like I'm missing something fundamentally here although I'm sitting on it longer than this problem should be taking me.
Thanks for you help,
I've written a function that takes two arguments, one for no. dimensions and another for no. simulations. The function does exactly what is needed (calculating the volume of a unit hypersphere), however when I wish to plot the function over a range of dimensions it returns an error: ''list' object cannot be interpreted as an integer'.
My function is the following,
def hvolume(ndim, nsim):
ob = [np.random.uniform(0.0,1.0,(nsim, ndim))]
ob = np.concatenate(ob)
i = 0
res = []
while i <= nsim-1:
arr = np.sqrt(np.sum(np.square(ob[i])))
i += 1
res.append(arr)
N = nsim
n = ndim
M = len([i for i in res if i <= 1])
return ((2**n)*M/N)
The error traceback is:
Traceback (most recent call last):
File "<ipython-input-192-4c4a2c778637>", line 1, in <module>
runfile('H:/Documents/Python Scripts/Q4ATTEMPT.py', wdir='H:/Documents/Python Scripts')
File "C:\Users\u1708511\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 668, in runfile
execfile(filename, namespace)
File "C:\Users\u1708511\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "H:/Documents/Python Scripts/Q4ATTEMPT.py", line 20, in <module>
print(hvolume(d, 2))
File "H:/Documents/Python Scripts/Q4ATTEMPT.py", line 4, in hvolume
ob = [np.random.uniform(0.0,1.0,(nsim, ndim))]
File "mtrand.pyx", line 1307, in mtrand.RandomState.uniform
File "mtrand.pyx", line 242, in mtrand.cont2_array_sc
TypeError: 'list' object cannot be interpreted as an integer
I really have no idea where to go from here, and have searched thoroughly online for how to resolve this. Unfortunately I'm a beginner with this!
Any help is appreciated.
If you simply try your first line in the function;
ob = [np.random.uniform(0.0,1.0,(nsim, ndim))]
with a list as one of the variables like so;
[np.random.uniform(0.0,1.0,([1,2], 2))]
you will get the error:
TypeError: 'list' object cannot be interpreted as an integer
This is because the uniform command it looking for an integer, not a list. You will need to make a for loop if you would like to handle lists.
One pattern I use for situations like this would be to begin the function with a block to handle the case of if they're iterators. Something like this for example.
from collections import Iterator
def hvolume(ndim, nsim):
outputs = []
if isinstance(ndim, Iterator):
for ndim_arg in ndim:
outputs.append(hvolume(ndim_arg, nsim))
if isinstance(nsim, Iterator):
for nsim_arg in nsim:
outputs.append(hvolume(ndim, nsim_arg))
if len(outputs) == 0: # neither above is an Iterator
# ... the rest of the function but it appends to outputs
return outputs
Check the input parameters of your method "hvolume", it seems that you give a list either nsim or ndim, which should be both integer values. That makes the uniform throw a TypeError Exception.
I have been working on a project which uses Time Series for its Calculation.
I would want to have the the Floor and Celing of data (similar to the ones in Excel's FLOOR and CEILING for an entire column
I checked for custom numpy functions, but couldnt see anything which includes significance level
I defined custom functions
def ceil(x, s):
return s * math.ceil(float(x)/s)
def floor(x, s):
return s * math.floor(float(x)/s)
However I cannot use them simultaneously on an entire column
Because of which I need to iterate each row individually:
for i in symbols:
symbols[i]['PutStrike']=0
symbols[i]['CallStrike']=0
for counter in range(0,len(symbols[0])):
symbols[i]['PutStrike'][counter]=floor(symbols[i]['FUT'][counter],Strike_Diff[i])
symbols[i]['CallStrike'][counter]=ceil(symbols[i]['FUT'][counter],Strike_Diff[i])
return symbols
Which of course is not the correct approach along with being time consuming
What I want is something like this:
def CalculateIV(symbols):
for i in symbols:
symbols[i]['PutStrike']=0
symbols[i]['CallStrike']=0
symbols[i]['PutStrike']=floor(symbols[i]['FUT'],Strike_Diff[i])
symbols[i]['CallStrike']=ceil(symbols[i]['FUT'],Strike_Diff[i])
return symbols
However when I run, I get:
CalculateIV(abc)
Traceback (most recent call last):
File "<ipython-input-456-599f9aa19e37>", line 1, in <module>
CalculateIV(abc)
File "<ipython-input-452-190c395d86ed>", line 9, in CalculateIV
symbols[i]['PutStrike']=floor(symbols[i]['FUT'],Strike_Diff[i])
File "<ipython-input-260-8a88fc57ddf5>", line 2, in floor
return s * math.floor(float(x)/s)
File "C:\Users\jay\Anaconda2\lib\site-packages\pandas\core\series.py", line 93, in wrapper
"{0}".format(str(converter)))
TypeError: cannot convert the series to <type 'float'>
Can some one please suggest an alternative/quicker approach or any library which could ease this.
Thanks in Advance
Well it was easier than I thought
I had to use the vectorize function in numpy (np.vectorize)
def ceil(x, s):
return s * math.ceil(float(x)/s)
def floor(x, s):
return s * math.floor(float(x)/s)
vfloor=np.vectorize(floor)
vceil=np.vectorize(ceil)
Thus these functions are now vectorized.
I can straight away use this to process multiple dataframes within seconds.
def CalculateIV(symbols):
for i in symbols:
symbols[i]['PutStrike']=0
symbols[i]['CallStrike']=0
symbols[i]['PutStrike']=vfloor(symbols[i]['FUT'],Strike_Diff[i])
symbols[i]['CallStrike']=vceil(symbols[i]['FUT'],Strike_Diff[i])
return symbols
If pqr has multiple dataframes in it.
I can just use the below to gather the floor and ceil values
output=CalculateIV(pqr)
I have a function which accepts a list R. In this function, I have defined an optimization problem using "pulp", This is my function:
import pulp
from multiprocessing.dummy import Pool as ThreadPool
def optimize(R):
variables = ["x1","x2","x3","x4"]
costs = {"x1":R[0], "x2":R[1], "x3":R[2], "x4":R[3]}
constraint = {"x1":5, "x2":7, "x3":4, "x4":3}
prob_variables = pulp.LpVariable.dicts("Intg",variables,
lowBound=0,
upBound=1,
cat=pulp.LpInteger)
prob = pulp.LpProblem("test1", pulp.LpMaximize)
# defines the constraints
prob += pulp.lpSum([constraint[i]*prob_variables[i] for i in variables]) <= 14
# defines the objective function to maximize
prob += pulp.lpSum([costs[i]*prob_variables[i] for i in variables])
pulp.GLPK().solve(prob)
# Solution
return pulp.value(prob.objective)
To get the output, I used a list as my input and the output is correct:
my_input = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
results =[]
for i in range(0,len(my_input)):
results.append(optimize(my_input[i]))
print("*"*20)
print(results)
But, I want to use multi-threading instead of the for loop. So, I used:
my_input = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
pool = ThreadPool(4)
results = pool.map(optimize, my_input)
But it gives me some errors:
Traceback (most recent call last):
File "/Users/Mohammad/PycharmProjects/untitled10/multi_thread.py", line 35, in <module>
results = pool.map(optimize, my_input)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/Users/Mohammad/PycharmProjects/untitled10/multi_thread.py", line 27, in optimize
pulp.GLPK().solve(prob)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/PuLP-1.6.1-py3.5.egg/pulp/solvers.py", line 179, in solve
return lp.solve(self)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/PuLP-1.6.1-py3.5.egg/pulp/pulp.py", line 1643, in solve
status = solver.actualSolve(self, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/PuLP-1.6.1-py3.5.egg/pulp/solvers.py", line 377, in actualSolve
raise PulpSolverError("PuLP: Error while executing "+self.path)
pulp.solvers.PulpSolverError: PuLP: Error while executing glpsol
Can anybody help me?
In my actual code, my_input list has the length of 27 (instead of 4 in the above code) and for each one, in my function I have to perform 80k optimizations (instead of one in the above code). So, multi-threading is a big help for me.
I have seen that class pulp.solvers.COIN_CMD has a threads argument, although the documentation is quite laconic. Taking a look at the code source, it seems to be indeed a way to provide threads to the solver.
If naming is indeed the issue, consider adding the desired name index for a given problem as an input argument to the function. Something like:
def optimize(tup): # here, tup contains (idx, R), so as to be callable using pool.map
...
prob = pulp.LpProblem('test'+str(idx), pulp.LpMaximize)
...
and then something like:
my_input = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
pool = ThreadPool(4)
results = pool.map(optimize, enumerate(my_input))
I'm trying to add some equality and inequality constraints to my minimization problem.
I am using nlopt Python API.
In particular I would like to add some vector-valued constraints.
My code looks like this example:
def eqconstr(result,x,grad):
if grad.size > 0:
print "gradient to be implemented"
for i in range(len(x)):
if condition: result[i] = 0.
initvect = # some initial guess of the parameters
opt = nlopt.opt(nlopt.LN_PRAXIS,len(initvect))
opt.set_min_objective(function)
tol = np.asarray([0.1]*len(initvect))
opt.add_equality_mconstraint(eqconstr, tol) # this is the call of the constraints (!!!)
opt.set_lower_bounds(-1.) # other parameters to set. not important for this question
opt.set_upper_bounds(1.)
opt.set_initial_step([0.1]*len(initvect))
opt.set_xtol_abs(10e-6)
opt.set_ftol_abs(10e-6)
res = opt.optimize(initvect)
This follows precisely the instructions in the nlopt wiki.
Now if I run this I get:
Traceback (most recent call last):
File "main.py", line 158, in <module>
opt.add_equality_mconstraint(eqconstr, tol)
File "/usr/local/lib/python2.7/dist-packages/nlopt.py", line 269, in add_equality_mconstraint
def add_equality_mconstraint(self, *args): return _nlopt.opt_add_equality_mconstraint(self, *args)
ValueError: nlopt invalid argument
Make sure that your function eqcontr has the same form as your objective function function. Maybe post it as well, so it'll be easy to understand. Also, I cannot see where condition is defined.