We Keep Coding

How to define a custom cost function for Pulp library

I am trying to use a custom cost function for using the PuLP Python library(https://realpython.com/linear-programming-python/). However I am not sure how to use extract my optimization variables. The objective function has a model I am trying to tune that has both integer and continues parameters
def opt_funct(bins, reg):
return some_calcuation(bins, reg) #returns some integer
model = pulp.LpProblem(name="small-problem", sense=pulp.LpMinimize)
bins = pulp.LpVariable(name="bins", lowBound=1, upBound=100, cat='Integer')
reg = pulp.LpVariable(name="reg", lowBound=0.0001, upBound=1.0, cat='Continues')
model += opt_funct(bins, reg)
status = model.solve()
The problem seems to be that variables are in some PuLP variable format. How can I use the variables for my own functions?

Summarise the posterior of a single parameter from an array with arviz

I am estimating a model using the pyMC3 library in python. In my "real" model, there are four parameter arrays, two of which have over 170,000 parameters in them. Summarising this array of parameters is too computationally intensive on my computer. I have been trying to figure out if the summary function in arviz will allow me to only summarise one (or a small number) of parameters in the array. Below is a reprex where the same problem is present, though the model is a lot simpler. In the linear regression model below, the parameter array b has three parameters in it b[0], b[1], b[2]. I would like to know how to get the summary for just b[0] and b[1] or alternatively for just a single parameter, e.g., b[0].
import pandas as pd
import pymc3 as pm
import arviz as az
d = pd.read_csv("https://quantoid.net/files/mtcars.csv")
mpg = d['mpg'].values
hp = d['hp'].values
weight = d['wt'].values
with pm.Model() as model:
b = pm.Normal("b", mu=0, sigma=10, shape=3)
sig = pm.HalfCauchy("sig", beta=2)
mu = pm.Deterministic('mu', b[0] + b[1]*hp + b[2]*weight)
like = pm.Normal('like', mu=mu, sigma=sig, observed=mpg)
fit = pm.fit(10000, method='advi')
samp = fit.sample(1500)
with model:
smry = az.summary(samp, var_names = ["b"])
It looked like the coords argument to the summary() function would do it, but after googling around and finding a few examples, like the one here with plot_posterior() instead of summary(), I was unable to get something to work. In particular, I tried the following in the hopes that it would return the summary for b[0] and b[1].
with model:
smry = az.summary(samp, var_names = ["b"], coords={"b_dim_0": range(1)})
or this to return the summary of b[0]:
with model:
smry = az.summary(samp, var_names = ["b"], coords={"b_dim_0": [0]})
I suspect I am missing something simple (I'm an R user who dabbles occasionally with Python). Any help is greatly appreciated.
(BTW, I am using Python 3.8.0, pyMC3 3.9.3, arviz 0.10.0)

To use coords for this, you need to update to the development (which will still show 0.11.2 but has the code from github or any >0.11.2 release) version of ArviZ. Until 0.11.2, the coords argument in summary was not used to subset the data (like it did in all plotting functions) but instead it was only taken into account if the input was not already InferenceData in which case it was passed to the converter.
With older versions, you need to use xarray to subset the data before passing it to summary. Therefore you need to explicitly convert the trace to inferencedata beforehand. In the example above it would look like:
with model:
...
samp = fit.sample(1500)
idata = az.from_pymc3(samp)
az.summary(idata.posterior[["b"]].sel({"b_dim_0": [0]}))
Moreover, you may also want to indicate summary to compute only a subset of the stats/diagnostics as shown in the docstring examples.

Issue with Mixed Integer Non-Linear Programming problem in GEKKO

I am using GEKKO for solving an MINLP problem in Python that involves a co-simulation with PowerFactory, a power system simulation software. The MINLP problem objective function needs to call PowerFactory from within and execute particular tasks before returning the values to the MILP problem definition. In the equality constraint definition, I need to also use two Pandas data frames (tables with 10000x2 cells) to get particular values corresponding to the values of the decision variables in the constraint expression. But, I am getting errors while writing any code involving loops, or Pandas 'loc' or 'iloc' functions, within the objective/constraint functions due to some issues with the nature of GEKKO variables in these function calls. Any help in this regard would be much appreciated. The structure of the code is given below:
import powerfactory as pf
from gekko import GEKKO
import pandas as pd
# Execute the PF setup commands for Python
# Pandas dataframe in question
df1 = pd.read_csv("table1.csv")
df2 = pd.read_csv("table2.csv")
def constraint1(X):
P = 0
for i in range(length(X)):
V = X[i] * 1000
D = round(V, 1)
P += df1.loc[D, "ColumnName"]
return P
def objective(X):
for i in range(length(X)):
V = X[i] * 1000
D = round(V, 1)
I = df2.loc[D, "ColumnName"]
# A list with the different values of 'I' is generated for passing to PF
# Some PowerFactory based commands below, involving inputs, and extracting results from PF, using a list generated from the values of 'I'. PF returns some result values to the code
return results
# MINLP problem definition:
m = GEKKO(remote=False)
x = m.Array(m.Var, nInv, value=1.0, lb=0.533, ub=1.0)
m.Equation(constraint1(x) == 30)
m.Minimize(objective(x))
m.options.SOLVER = 3
m.solve(disp=False)
# Command for exporting the results to a .txt file
In another problem formulation, I am trying to run MINLP optimization problems within the objective and constraint function as a nested optimization problem. However, I am running into errors in that as well. The structure of the code is as follows:
import powerfactory as pf
from gekko import GEKKO
# Execute the PF setup commands for Python
def constraint1(X):
P = 0
for i in range(length(X)):
V = X[i] * 1000
# 2nd MINLP problem: Finds 'I' from value of 'V', a single element
# Calculate 'Pcal' from 'I'
P += Pcal
return P
def objective(X):
Iset = []
for i in range(length(X)):
V = X[i] * 1000
# 3rd MINLP problem: Finds 'I' from value of 'V'
# Appends 'I' to a 'Iset'
# 'Iset' list passed on to PF
# Some PowerFactory based commands below, involving inputs, and extracting results from PF, using the 'Iset' list. PF returns some result values to the code
return results
# Main MINLP problem definition:
m = GEKKO(remote=False)
x = m.Array(m.Var, nInv, value=1.0, lb=0.533, ub=1.0)
m.Equation(constraint1(x) == 30)
m.Minimize(objective(x))
m.options.SOLVER = 3
m.solve(disp=False)
# Command for exporting the results to a .txt file

Gekko does not have call-backs to external functions. This is because the solvers are gradient-based and a precursor steps is automatic differentiation to obtain exact 1st and 2nd derivatives in sparse form. Similar to Gekko and CoolProp, one option is to build an approximation to your external (in this case PowerFactory) model that the optimizer can use. Two options are the cubic spline (cspline) or the basis spline (bspline).
If you can't use these approximations then I recommend switching to a different platform for solving the optimization problem. Perhaps you could try scipy.optimize.minimize that can obtain gradients by finite difference and add branch and bound to solve the mixed integer portion.

Fitting parameters of a data set from a model

I am trying to fit the parameters of a transit light curve.
I have observed transit light curve data and I am using a .py in python that through 4 parameters (period, a(semi-major axis), inclination, planet radius) returns a model transit light curve. I would like to minimize the residual between these two light curves. This is what I am trying to do: First - Estimate a max likelihood using method = "L-BFGS-B" and then apply the mcmc using emcee to estimate the uncertainties.
The code:
p = lmfit.Parameters()
p.add_many(('per', 2.), ('inc', 90.), ('a', 5.), ('rp', 0.1))
per_b = [1., 3.]
a_b = [4., 6.]
inc_b = [88., 90.]
rp_b = [0.1, 0.3]
bounds = [(per_b[0], per_b[1]), (inc_b[0], inc_b[1]), (a_b[0], a_b[1]), (rp_b[0], rp_b[1])]
def residual(p):
v = p.valuesdict()
eclipse.criarEclipse(v['per'], v['a'], v['inc'], v['rp'])
lc0 = numpy.array(eclipse.getCurvaLuz()) (observed flux data)
ts0 = numpy.array(eclipse.getTempoHoras()) (observed time data)
c = numpy.linspace(min(time_phased[bb]),max(time_phased[bb]),len(time_phased[bb]),endpoint=True)
nn = interpolate.interp1d(ts0,lc0)
return nn(c) - smoothed_LC[bb] (residual between the model and the data)
Inside def residual(p) I make sure that both the observed data (time_phased[bb] and smoothed_LC[bb]) have the same size of the model transit light curve. I want it to give me the best fit values for the parameters (v['per'], v['a'], v['inc'], v['rp']).
I need your help and I appreciate your time and your attention. Kindest regards, Yuri.

Your example is incomplete, with many partial concepts and some invalid Python. This makes it slightly hard to understand your intention. If the answer below is not sufficient, update your question with a complete example.
It seems pretty clear that you want to model your data smoothed_LC[bb] (not sure what bb is) with a model for some effect of an eclipse. With that assumption, I would recommend using the lmfit.Model approach. Start by writing a function that models the data, just so you check and plot your model. I'm not entirely sure I understand everything you're doing, but this model function might look like this:
import numpy
from scipy import interpolate
from lmfit import Model
# import eclipse from somewhere....
def eclipse_lc(c, per, a, inc, p):
eclipse.criarEclipse(per, a, inc, rp)
lc0 = numpy.array(eclipse.getCurvaLuz()) # observed flux data
ts0 = numpy.array(eclipse.getTempoHoras()) # observed time data
return interpolate.interp1d(ts0,lc0)(c)
With this model function, you can build a Model:
lc_model = Model(eclipse_lc)
and then build parameters for your model. This will automatically name them after the argument names of your model function. Here, you can also give them initial values:
params = lc_model.make_params(per=2, inc=90, a=5, rp=0.1)
You wanted to place upper and lower bounds on these parameters. This is done by setting min and max parameters, not making an ordered array of bounds:
params['per'].min = 1.0
params['per'].max = 3.0
and so on. But also: setting such tight bounds is usually a bad idea. Set bounds to avoid unphysical parameter values or when it becomes evident that you need to place them.
Now, you can fit your data with this model. Well, first you need to get the data you want to model. This seems less clear from your example, but perhaps:
c_data = numpy.linspace(min(time_phased[bb]), max(time_phased[bb]),
len(time_phased[bb]), endpoint=True)
lc_data = smoothed_LC[bb]
Well: why do you need to make this c_data? Why not just use time_phased as the independent variable? Anyway, now you can fit your data to your model with your parameters:
result = lc_model(lc_data, params, c=c_data)
At this point, you can print out a report of the results and/or view or get the best-fit arrays:
print(result.fit_report())
for p in result.params.items(): print(p)
import matplotlib.pyplot as plt
plt.plot(c_data, lc_data, label='data')
plt.plot(c_data. result.best_fit, label='fit')
plt.legend()
plt.show()
Hope that helps...

How to define General deterministic function in PyMC

In my model, I need to obtain the value of my deterministic variable from a set of parent variables using a complicated python function.
Is it possible to do that?
Following is a pyMC3 code which shows what I am trying to do in a simplified case.
import numpy as np
import pymc as pm
#Predefine values on two parameter Grid (x,w) for a set of i values (1,2,3)
idata = np.array([1,2,3])
size= 20
gridlength = size*size
Grid = np.empty((gridlength,2+len(idata)))
for x in range(size):
for w in range(size):
# A silly version of my real model evaluated on grid.
Grid[x*size+w,:]= np.array([x,w]+[(x**i + w**i) for i in idata])
# A function to find the nearest value in Grid and return its product with third variable z
def FindFromGrid(x,w,z):
return Grid[int(x)*size+int(w),2:] * z
#Generate fake Y data with error
yerror = np.random.normal(loc=0.0, scale=9.0, size=len(idata))
ydata = Grid[16*size+12,2:]*3.6 + yerror # ie. True x= 16, w= 12 and z= 3.6
with pm.Model() as model:
#Priors
x = pm.Uniform('x',lower=0,upper= size)
w = pm.Uniform('w',lower=0,upper =size)
z = pm.Uniform('z',lower=-5,upper =10)
#Expected value
y_hat = pm.Deterministic('y_hat',FindFromGrid(x,w,z))
#Data likelihood
ysigmas = np.ones(len(idata))*9.0
y_like = pm.Normal('y_like',mu= y_hat, sd=ysigmas, observed=ydata)
# Inference...
start = pm.find_MAP() # Find starting value by optimization
step = pm.NUTS(state=start) # Instantiate MCMC sampling algorithm
trace = pm.sample(1000, step, start=start, progressbar=False) # draw 1000 posterior samples using NUTS sampling
print('The trace plot')
fig = pm.traceplot(trace, lines={'x': 16, 'w': 12, 'z':3.6})
fig.show()
When I run this code, I get error at the y_hat stage, because the int() function inside the FindFromGrid(x,w,z) function needs integer not FreeRV.
Finding y_hat from a pre calculated grid is important because my real model for y_hat does not have an analytical form to express.
I have earlier tried to use OpenBUGS, but I found out here it is not possible to do this in OpenBUGS. Is it possible in PyMC ?
Update
Based on an example in pyMC github page, I found I need to add the following decorator to my FindFromGrid(x,w,z) function.
#pm.theano.compile.ops.as_op(itypes=[t.dscalar, t.dscalar, t.dscalar],otypes=[t.dvector])
This seems to solve the above mentioned issue. But I cannot use NUTS sampler anymore since it needs gradient.
Metropolis seems to be not converging.
Which step method should I use in a scenario like this?

You found the correct solution with as_op.
Regarding the convergence: Are you using pm.Metropolis() instead of pm.NUTS() by any chance? One reason this could not converge is that Metropolis() by default samples in the joint space while often Gibbs within Metropolis is more effective (and this was the default in pymc2). Having said that, I just merged this: https://github.com/pymc-devs/pymc/pull/587 which changes the default behavior of the Metropolis and Slice sampler to be non-blocked by default (so within Gibbs). Other samplers like NUTS that are primarily designed to sample the joint space still default to blocked. You can always explicitly set this with the kwarg blocked=True.
Anyway, update pymc with the most recent master and see if convergence improves. If not, try the Slice sampler.