I have a hyperbolic function and i need to find the 0 of it. I have tried various classical methods (bisection, newton and so on).
Second derivatives are continuous but not accessible analytically, so i have to exclude methods using them.
For the purpose of my application Newton method is the only one providing sufficient speed but it's relatively unstable if I'm not close enough to the actual zero. Here is a simple screenshot:
The zero is somewhere around 0.05. and since the function diverges at 0, if i take a initial guess value greater then the minimum location of a certain extent, then i obviously have problems with the asymptote.
Is there a more stable method in this case that would eventually offer speeds comparable to Newton?
I also thought of transforming the function in an equivalent better function with the same zero and only then applying Newton but I don't really know which transformations I can do.
Any help would be appreciated.
Dekker's or Brent's method should be almost as fast as Newton. If you want something simple to implement yourself, the Illinois variant of the regula-falsi method is also reasonably fast. These are all bracketing methods, so should not leave the domain if the initial interval is inside the domain.
def illinois(f,a,b,tol=1e-8):
'''regula falsi resp. false postion method with
the Illinois anti-stalling variation'''
fa = f(a)
fb = f(b)
if abs(fa)<abs(fb): a,fa,b,fb = b,fb,a,fa
while(np.abs(b-a)>tol):
c = (a*fb-b*fa)/(fb-fa)
fc = f(c)
if fa*fc < 0:
fa *= 0.5
else:
a, fa = b, fb
b, fb = c, fc
return b, fb
How about using log(x) instead of x?
For your case, #sams-studio's answer might work, and I would try that first. In a similar situation - also in multi-variate context - I used Newton-homotopy methods.
Basically, you limit the Newton step until the absolute value of y is descending.
The cheapest way to implement is that you half the Newton step if y increases from the last step. After a few steps, you're back at Newton with full second order convergence.
Disclamer: If you can bound your solution (you know a maximal x), the answer from #Lutz Lehmann would also be my first choice.
Related
I am very new to Pyomo, working on a use case where my objective function coefficient is dynamic & needs a min-max function.
Objective function = Max( sum (P * UC) - sum ( P - min(P)) * UC
where P is variable needs to be optimized and UC is function which is derived value based on some calculation.
I have few doubts
how to use min or max function in objective function, I have tried np.min or calling function but it gives error since function has if else condition
I have tried multiple things but none seems to be working. If someone can help me with dummy code that will be great.
Thanks in Advance.
Min could be implemented by defining a new variable min_P, which needs to be smaller than any of the P, expressed by constraints:
min_P <= P[i] for all i
This will make sure, that min_P is not larger than the smallest of the P. Then you can just use min_P in your objective function. I assume you know how to define constraints like this. This might result in an unbound variable problem, depending on how exactly you are optimizing, but this should put you on the right track.
The max case is analogous, if you define another value for the expression sum (P * UC) - sum ( P - min(P)).
It is not clear whether UC is a parameter or a variable itself (calculated in another constraint). In the latter case, the whole problem will be highly nonlinear and should be reconsidered.
I do not understand your AbstractModel vs ConcreteModel question. If you have the data available, use a ConcreteModel. Apart from that, see here.
I was wondering if i could make a multiple objective function in PuLP, by doing this Can I make a Min Z = max(a,b,c) in PuLP, however when using this code
ilp_prob = pulp.LpProblem("Miniimize Problem", pulp.LpMinimize)
x = []
if m >3:
return 1,1
for i in range(m):
temp = []
for j in range(len(jobs)):
temp += [pulp.LpVariable("x_%s_%s" %((i+1),(j+1)),0,1, cat = 'Binary')]
x+= [temp]
ilp_prob += max([pulp.lpSum([jobs[j]*x[i][j] for j in range(len(jobs))] for i in range(m))])
for i in range(len(jobs)):
ilp_prob += pulp.lpSum([x[j][i] for j in range(m)])==1
ilp_prob.solve()
It just returns all 1 in x[0], and all 0 in x[0].
I'm pretty sure you can't just use python's (!) max on pulp's internal expressions! Those solvers are working on a very specific problem-specification, LP standard form, where is no concept for that!
The exception would be if pulp would overload this max-function for it's data-structures (don't know if that's possible at all in python), but i'm pretty sure pulp does not support re-formulations like that (there is some needed; as again: the target is the Standard-form).
cvxpy for example does not overload, but introduces customized max-functions, which internally transform your problem.
That being said: i'm surprised your code runs without a critical error. But i'm too lazy to check pulps sources here.
Have a look at the usual LP/IP formulation-guides.
A first idea would be:
target: min (max(a,b,c))
reformulation:
introduce a new variable z
add constraints:
z >= a
z >= b
z >= c
assumption: the objective somehow want's to minimize z (maximizing will get you in trouble as the problem will get unbounded!)
this is the case here, as the final objective for our target would look like:
min(z)
Remark: One has to be careful that the problem will still be linear/convex (depending on the solver). In this case (our simple example; i did not check your whole model) i don't see a problem, but in more complex cases, min(max(complex_expression)) subjective to complex constraints, this might introduce non-convexity (and can't be solved by Conic solvers incl. LP-solvers).
And just throwing a keyword in the ring: your approach/objective sounds a bit like robust-optimization, where usually some worst-case scenario is optimized. Not all multi-objective optimization problems are treating multiple objective-components like that.
I have an optimization problem with constraints, but the COBYLA solver doesn't seem to respect the constraints I specify.
My optimization problem:
cons = ({'type':'ineq', 'fun':lambda t: t},) # all variables must be positive
minimize(lambda t: -stateEst(dict(zip(self.edgeEvents.keys(),t)), (0.1,)*len(self.edgeEvents), constraints=cons, method='COBYLA')
and stateEst is defined as:
def stateEst(t):
val = 0
for edge,nextState in self.edgeEvents.iteritems():
val += edge_probability(self,edge,ts) * estimates[nextState]
val += node_probability(self, edge.head, ts, edge_list=[edge])* cost
for node,nextState in self.nodeEvents.iteritems():
val += node_probability(self, node, ts) * \
(estimates[nextState] + cost*len([e for e in node.incoming if e in self.compEdges])
return val
The probability functions are only defined for positive t values. The dictionary is necessary because the probabilities are calculated with respect to the 'named' t-values.
When I run this, I notice that COBYLA tries a value of -0.025 for one of the t-values. Why is the optimization not respecting the constraints?
COBYLA is technically speaking an infeasible method, which means, that the iterates might not be always feasible in regards to your constraints! (it's only about the final convergence, where feasibility matters for these algorithms).
Using an objective-function which is not defined everywhere will be problematic. Maybe you are forced to switch to some feasible method.
Alternatively you could think about generalizing your objective, so that there are penalties introduced for negative t's. But this is problem-dependent and could introduce other problems as well (convergence; numeric-stability).
Try using L-BFGS-B, which is limited to bound-constraints, which is not a problem here (for your current problem!).
For something this simple, just redefine your function to take any real value by taking np.exp(t) or even t**2, then take the log (or square root) of the solution.
I have an objective function with several hundreds of quadratic terms which I would like to minimize; in this case I try to minimize the absolute distance between several variables. So the structure of my problem looks like this (highly simplified):
Minimize
obj: [ a^2 - 2 a * b + b^2 ] / 2
Subject To
c1: a + b >= 10
c2: a <= 100
End
I use the Python API to solve the problem in the following way:
import cplex
cpx = cplex.Cplex()
cpx.read('quadratic_obj_so.lp')
# use the dual simplex
cpx.parameters.lpmethod.set(cpx.parameters.lpmethod.values.dual)
cpx.solve()
print cpx.solution.get_values()[0:15]
print cpx.solution.status[cpx.solution.get_status()]
print cpx.solution.get_objective_value()
And for the above example I then receive (showing only iterations 16-18):
Itn Primal Obj Dual Obj Prim Inf Upper Inf Dual Inf
16 1.4492800e-19 -1.0579911e-07 3.81e-14 7.11e-15 5.17e-25
17 9.0580247e-21 -2.6449779e-08 1.91e-14 3.55e-15 2.33e-25
18 5.6612645e-22 -6.6124446e-09 5.45e-14 7.11e-15 6.46e-27
[73.11695794600045, 73.11695794603409]
optimal
0.0
so a and b are equal which makes sense since I try to minimize their distance and the constrains are clearly fulfilled.
However, my actual problem is far more complex and I receive:
Itn Primal Obj Dual Obj Prim Inf Upper Inf Dual Inf
92 1.4468496e+06 1.2138985e+06 1.80e+02 2.64e-12 5.17e-02
93 1.4468523e+06 1.2138969e+06 2.23e+02 2.17e-12 1.08e-02
94 1.4468541e+06 1.2138945e+06 2.93e+02 2.31e-12 5.62e-02
* 1.4457132e+06 1.2138598e+06 7.75e+00 7.61e-09 2.76e-02
num_best
1445714.46525
I have now several questions regarding the output which are closely connected:
1) Clearly, it is not the objective value for the dual simplex printed. Why is that, since I set the solver to be the dual simplex?!
2) How do I now access the results for the dual simplex? As the objective value is smaller I would be more interested in these results.
3) Does the num_best status guarantee that all the constrains are met i.e. is the solution valid but just not guaranteed to be optimal?
4) Primal Obj and Dual Obj differ quite a lot. Is there any strategy to minimize their difference?
To the best of my knowledge, get_objective_value always returns the best primal bound (regardless of the lpmethod).
Information about the dual solution can be retrieved with get_dual_values.
The num_best solution status means that a solution is available, but there is no proof of optimality (see here). This is probably the most important point with regards to the rest of the questions here.
You could try turning on the numerical emphasis parameter to see if that helps. There are also various tolerances you can adjust (e.g., optimality tolerance).
Note that all of the links I've used above are for the C Callable Library (which the Python API calls internally) for CPLEX 12.6.3.
I have a problem that is equal parts trig and Python. I am plotting a cosine over time interval [0,t] whose frequency changes (slightly) according to another cosine function. So what I'd expect to see is a repeating pattern of higher-to-lower frequency that repeats over the duration of the window [0,t].
Instead what I'm seeing is that over time a low-freq motif emerges in the cosine plot and repeats over time, each time becoming lower and lower in freq until eventually the cosine doesn't even oscillate properly it just "wobbles", for lack of a better term.
I don't understand how this is emerging over the course of the window [0,t] because cosine is (obviously) periodic and the function modulating it is as well. So how can "new" behavior emerge?? The behavior should be identical across all periods of the modulatory cosine that tunes the freq of the base cosine, right?
As a note, I'm technically using a modified cosine, instead of cos(wt) I'm using e^(cos(wt)) [called von mises eq or something similar].
Minimum needed Code:
cos_plot = []
for wind,pos_theta in zip(window,pos_theta_vec): #window is vec of time increments
# for ref: DBFT(pos_theta) = (1/(2*np.pi))*np.cos(np.radians(pos_theta - base_pos))
f = float(baserate+DBFT(pos_theta)) # DBFT() returns a val [-0.15,0.15] periodically depending on val of pos_theta
cos_plot.append(np.exp(np.cos(f*2*np.pi*wind)))
plt.plot(cos_plot)
plt.show()
What you are observing could depend on "aliasing", i.e. the emergence of low-frequency figures because of sampling of an high frequency function with a step that is too big.
(picture taken from the linked Wikipedia page)
If the issue is NOT aliasing consider that any function shape between -1 and 1 can be obtained with cos(f(x)*x) by simply choosing f(x).
For, consider any function -1 <= g(x) <= 1 and set f(x) = arccos(g(x))/x.
To look for the problem try plotting your "frequency" and see if anything really strange is present in it. May be you've a bug in DBFT.
In the interest of posterity, in case anyone ever needs an answer to this question:
I wanted a cosine whose frequency was a time-varying function freq(t). My mistake was simply evaluating this function at each time t like this: Acos(2pifreq(t)t). Instead you need to integrate freq(t) from 0 to t at each time point: y = cos(2%piintegral(f(t)) + 2%pi*f0*t + phase). The term for this procedure is a frequency sweep or chirp (not identical terms, but similar if you need to google/SO answers).
Thanks to those who responded with help :)
SB