We Keep Coding

Using logarithmic in the objective function in IBM CPLEX

My objective function in IBM CPLEX is as follows:
objective = opt_model.sum(math.log(r_vars[0,0]*(3*w_vars[0]-1))+math.log(r_vars[1,0]*(3*w_vars[0]-1)))
opt_model.maximize(objective)
The variable w_vars can get a value in the range [0,1] and the value of r_vars can be in the range of [1,100]. But I am getting this error:
TypeError: must be real number, not QuadExpr
I assume the problem is the result of the parentheses for the math.log function. How can I use a log function in the objective function in IBM CPLEX? Or any thoughts on this?

What you could do is rely on cpo within Cplex
See
https://github.com/AlexFleischerParis/zoodocplex/blob/master/zoononlinear.py
For a tiny example
from docplex.cp.model import CpoModel
mdl = CpoModel(name='buses')
nbbus40 = mdl.integer_var(0,1000,name='nbBus40')
nbbus30 = mdl.integer_var(0,1000,name='nbBus30')
mdl.add(nbbus40*40 + nbbus30*30 >= 300)
#non linear objective
mdl.minimize(mdl.exponent(nbbus40)*500 + mdl.exponent(nbbus30)*400)
msol=mdl.solve()
print(msol[nbbus40]," buses 40 seats")
print(msol[nbbus30]," buses 30 seats")

How to view Gekko variables/parameters for debug purposes?

I have a fitting task where I am using GEKKO.
There are a lot of variables, arrays of variables, some variables that must contain arrays, and so on.
I didn't have success with the fitting,
so I need to do step-by-step verification of all parameters that I am providing for GEKKO and all the calculated intermediate values.
Is there a way to print out the values of each variable for debugging purposes?
Or to view the values of the variables in line-by-line execution?
for example, I have an array that is saved like a variable ro:
phi = model.Intermediate( c * ro) # phase shift
where c is some constant defined somewhere above in the model definition.
How can I view the values inside phi that will be used in the next steps?
I need to view/save all the values of all variables/constants/intermediates used during the model creation - before a try to solve. Is it possible?

Turn up the DIAGLEVEL to 2 or higher to produce diagnostic files in the run directory m.path.
from gekko import GEKKO
m = GEKKO(remote=False)
c = 2
x = m.Param(3,name='x')
ro = m.Var(value=4,lb=0,ub=10,name='ro')
y = m.Var()
phi = m.Intermediate(c*ro,name='phi')
m.Equation(y==phi**2+x)
m.Maximize(y)
m.options.SOLVER = 1
m.options.DIAGLEVEL=2
m.open_folder()
m.solve()
Here is a summary of the diagnostic files that are produced:
Variables, Equations, Jacobian, Lagrange Multipliers, Objective
apm_eqn.txt
apm_jac.txt
apm_jac_fv.txt
apm_lam.txt
apm_lbt.txt
apm_obj.txt
apm_obj_grad.txt
apm_var.txt
Solver Output and Options
APOPT.out
apopt_current_options.opt
Model File
gk_model0.apm
Data File
gk_model0.csv
Options Files
gk_model0.dbs
options.json
Specification File for FV, MV, SV, CV
gk_model0.info
Inputs to the Model
dbs_read.rpt
input_defaults.dbs
input_gk_model0.dbs
input_measurements.dbs
input_overrides.dbs
measurements.dbs
Results
rto.t0
results.csv
results.json
gk_model0_r_2022y12m04d08h12m28.509s.t0
Initialization Steps Before Solve
rto_1.t0
rto_2.t0
rto_3.t0
rto_3_eqn.txt
rto_3_eqn_var.txt
rto_3_var.t0
Reports After Solve
rto_4.t0
rto_4_eqn.txt
rto_4_eqn_var.txt
rto_4_var.t0
The files of interest for you are likely the rto* initialization files. The name changes based on the IMODE that you run. It is mpu* for your application for a Model Parameter Update with IMODE=2.

How to define a custom cost function for Pulp library

I am trying to use a custom cost function for using the PuLP Python library(https://realpython.com/linear-programming-python/). However I am not sure how to use extract my optimization variables. The objective function has a model I am trying to tune that has both integer and continues parameters
def opt_funct(bins, reg):
return some_calcuation(bins, reg) #returns some integer
model = pulp.LpProblem(name="small-problem", sense=pulp.LpMinimize)
bins = pulp.LpVariable(name="bins", lowBound=1, upBound=100, cat='Integer')
reg = pulp.LpVariable(name="reg", lowBound=0.0001, upBound=1.0, cat='Continues')
model += opt_funct(bins, reg)
status = model.solve()
The problem seems to be that variables are in some PuLP variable format. How can I use the variables for my own functions?

Passing Two List to a Python function

I am trying to run python package pyabc(Approximate Bayesian Computation) for model selection between two list of values i.e model_1=[2,3,4,5] and model_2=[3,4,2,5]. The main function of pyabc is ABCSMC which states that
Definition : ABCSMC(models: Union[List[Model], Model], parameter_priors:
Union[List[Distribution], Distribution, Callable], distance_function: Union[Distance,
Callable]=None, population_size: Union[PopulationStrategy, int]=100, summary_statistics:
Callable[[model_output], dict]=identity, model_prior: RV=None)
I don't know where to define and pass my two lists model_1 and model_2 in the below mentioned code. I tried it several times but not able to do it as I am new to Python. I am following an example and its code in mentioned below.
import os
import tempfile
import scipy.stats as st
import pyabc
# Define a gaussian model
sigma = .5
def model(parameters):
# sample from a gaussian
y = st.norm(parameters.x, sigma).rvs()
# return the sample as dictionary
return {"y": y}
# We define two models, but they are identical so far
models = [model, model]
# However, our models' priors are not the same.
# Their mean differs.
mu_x_1, mu_x_2 = 0, 1
parameter_priors = [
pyabc.Distribution(x=pyabc.RV("norm", mu_x_1, sigma)),
pyabc.Distribution(x=pyabc.RV("norm", mu_x_2, sigma))
]
abc = pyabc.ABCSMC(
models, parameter_priors,
pyabc.PercentileDistance(measures_to_use=["y"]))
db_path = ("sqlite:///" +
os.path.join(tempfile.gettempdir(), "test.db"))
history = abc.new(db_path, {"y": y_observed})
print("ABC-SMC run ID:", history.id)
# We run the ABC until either criterion is met
history = abc.run(minimum_epsilon=0.2, max_nr_populations=5)

Model selection in pyABC aims to decide among different model candidates which model describes a common set of observed data best. In your above code, you would thus typically use different models in the list of models models = [model1, model2]. I am not sure what you mean by model selection over lists?
For the underlying algorithm and problem that it tries to solve, see also the original paper https://doi.org/10.1093/bioinformatics/btp619.

statsmodels - printing summary of ARMA fit throws error

I want to fit an ARMA(p,q) model to simulated data, y, and check the effect of different estimation methods on the results. However, fitting a model to the same object like so
model = tsa.ARMA(y,(1,1))
results_mle = model.fit(trend='c', method='mle', disp=False)
results_css = model.fit(trend='c', method='css', disp=False)
and printing the results
print result_mle.summary()
print result_css.summary()
generates the following error
File "C:\Anaconda\lib\site-packages\statsmodels\tsa\arima_model.py", line 1572, in summary
smry.add_table_params(self, alpha=alpha, use_t=False)
File "C:\Anaconda\lib\site-packages\statsmodels\iolib\summary.py", line 885, in add_table_params
use_t=use_t)
File "C:\Anaconda\lib\site-packages\statsmodels\iolib\summary.py", line 475, in summary_params
exog_idx]
IndexError: index 3 is out of bounds for axis 0 with size 3
If, instead, I do this
model1 = tsa.ARMA(y,(1,1))
model2 = tsa.ARMA(y,(1,1))
result_mle = model1.fit(trend='c',method='css-mle',disp=False)
print result_mle.summary()
result_css = model2.fit(trend='c',method='css',disp=False)
print result_css.summary()
no error occurs. Is that supposed to be or a Bug that should be fixed?
BTW the ARMA process I generated as follows
from __future__ import division
import statsmodels.tsa.api as tsa
import numpy as np
# generate arma
a = -0.7
b = -0.7
c = 2
s = 10
y1 = np.random.normal(c/(1-a),s*(1+(a+b)**2/(1-a**2)))
e = np.random.normal(0,s,(100,))
y = [y1]
for t in xrange(e.size-1):
arma = c + a*y[-1] + e[t+1] + b*e[t]
y.append(arma)
y = np.array(y)

You could report this as a bug, even though it looks like a consequence of the current design.
Some attributes of the model change when the estimation method is changed, which should in general be avoided. Since both results instances access the same model, the older one is inconsistent with it in this case.
http://www.statsmodels.org/dev/pitfalls.html#repeated-calls-to-fit-with-different-parameters
In general, statsmodels tries to keep all parameters that need to change the model in the model.__init__ and not as arguments in fit, and attach the outcome of fit and results to the Results instance.
However, this is not followed everywhere, especially not in older models that gained new options along the way.
trend is an example that is supposed to go into the ARMA.__init__ because it is now handled together with the exog (which is an ARMAX model), but wasn't in pure ARMA. The estimation method belongs in fit and should not cause problems like these.
Aside: There is a helper function to simulate an ARMA process that uses scipy.signal.lfilter and should be much faster than an iteration loop in Python.