We Keep Coding

Constraint 'feasibility_cut[1]' does not have a proper value. Found 'True'

I'm new to python and pyomo, so I would kindly appreciate your help,
I'm currently having trouble trying to add a constraint to my mathematical model in Pyomo, the problem is while I try to add the "feasibility_cut", it says "Constraint 'feasibility_cut[1]' does not have a proper value. Found 'True' ", what I understand from this is that, pyomo sees this constraint as a logical comparative constraint, which is I don't know why!
Here is a part of the code that I think is necessary to see:
RMP = ConcreteModel()
RMP.ymp = Var(SND.E, within=Integers)
RMP.z = Var(within = Reals)
S1 = (len(SND.A), len(SND.K))
S2 = (len(SND.A), len(SND.A))
uBar= np.zeros(S1)
vBar=np.zeros(S2)
RMP.optimality_cut = ConstraintList()
RMP.feasibility_cut = ConstraintList()
expr2 = (sum(SND.Fixed_Cost[i,j]*RMP.ymp[i,j] for i,j in SND.E) + RMP.z)
RMP.Obj_RMP = pe.Objective(expr = expr2, sense = minimize)
iteration=0
epsilon = 0.01
while (UB-LB)>epsilon :
iteration = iteration +1
DSPsolution = Solver.solve(DSP)
for i in SND.A:
for k in SND.K:
uBar[i-1,k-1] = value(DSP.u[i,k])
for i,j in SND.E:
vBar[i-1,j-1] = value(DSP.v[i,j])
if value(DSP.Obj_DSP) == DSPinf:
RMP.feasCut.add()
else:
RMP.optimCut.add()
RMPsolution = solver.solve(RMP)
UB=min(UB,)
LB=max(LB,value(RMP.Obj_RMP))
if value(DSP.Obj_DSP) == DSPinf:
RMP.feasibility_cut.add( 0>= sum(-SND.Capacity[i,j]*vBar[i-1,j-1]*RMP.ymp[i,j] for i,j in
SND.E) + sum(uBar[i-1,k-1]*SND.New_Demand[k,i] for i in SND.A for k in SND.K if (k,i) in
SND.New_Demand) )
else:
RMP.optimality_cut.add( RMP.z >= sum(SND.Fixed_Cost[i,j]*RMP.ymp[i,j] for i,j in SND.E) +
sum(uBar[i-1,k-1]*SND.New_Demand[k,i] for i in SND.A for k in SND.K) -
sum(SND.Capacity[i,j]*vBar[i-1,j-1]*RMP.ymp[i,j] for i,j in SND.E) )

Welcome to the site.
A couple preliminaries... When you post code that generates an error, it is customary (and easier for those to help) if you post the entire code necessary to reproduce the error and the stack trace, or at least identify what line is causing the error.
So when you use a constraint list in pyomo, everything you add to it must be a legal expression in terms of the model variables, parameters, and other constants etc. You are likely getting the error because you are adding an expression that evaluates to True. So it is likely that an expression you are adding does not depend on a model variable. See the example below.
Also, you need to be careful mingling numpy and pyomo models, numpy arrays etc. can cause some confusion and odd errors. I'd recommend putting all data into the model or using pure python data types (lists, sets, dictionaries).
Here are 2 errors. You have an empty add() in your code too, which will throw an error.
In [1]: from pyomo.environ import *
In [2]: m = ConcreteModel()
In [3]: m.my_constraints = ConstraintList()
In [4]: m.X = Var()
In [5]: m.my_constraints.add(m.X >= 5) # good!
Out[5]: <pyomo.core.base.constraint._GeneralConstraintData at 0x7f8778cfa880>
In [6]: m.my_constraints.add() # error!
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-cf466911f3a1> in <module>
----> 1 m.my_constraints.add()
TypeError: add() missing 1 required positional argument: 'expr'
In [7]: m.my_constraints.add(3 <= 4) # error: trivial evaluation!
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-7-a0bec84404b0> in <module>
----> 1 m.my_constraints.add(3 <= 4)
...
ValueError: Invalid constraint expression. The constraint expression resolved to a trivial Boolean (True) instead of a Pyomo object. Please modify your rule to return Constraint.Feasible instead of True.
Error thrown for Constraint 'my_constraints[2]'
In [8]:

How to use the R 'with' operator in rpy2

I am doing an ordinal logistic regression, and following the guide here for the analysis: R Data Analysis Examples: Ordinal Logistic Regression
My dataframe (consult) looks like:
n raingarden es_score consult_case
garden_id
27436 7 0 3 0
27437 1 0 0 1
27439 1 1 1 1
37253 1 0 3 0
37256 3 0 0 0
I am at the part where I need to to create graph to test the proportional odds assumption, with the command in R as follows:
(s <- with(dat, summary(es_score ~ n + raingarden + consult_case, fun=sf)))
(es_score is an ordinal ranked score with values between 0 - 4; n is an integer; raingarden and consult_case, binary values of 0 or 1)
I have the sf function:
sf <- function(y) {
c('Y>=1' = qlogis(mean(y >= 1)),
'Y>=2' = qlogis(mean(y >= 2)),
'Y>=3' = qlogis(mean(y >= 3)))
}
in a utils.r file that I access as follows:
from rpy2.robjects.packages import STAP
with open('/file_path/utils.r', 'r') as f:
string = f.read()
sf = STAP(string, "sf")
And want to do something along the lines of:
R = ro.r
R.with(work_case_control, R.summary(formula, fun=sf))
The major problem is that the R withoperator is seen as a python keyword, so that even if I access it with ro.r.with it is still recognized as a python keyword. (As a side note: I tried using R's apply method instead, but got an error that TypeError: 'SignatureTranslatedAnonymousPackage' object is not callable ... I assume this is referring to my function sf?)
I also tried using the R assignment methods in rpy2 as follows:
R('sf = function(y) { c(\'Y>=1\' = qlogis(mean(y >= 1)), \'Y>=2\' = qlogis(mean(y >= 2)), \'Y>=3\' = qlogis(mean(y >= 3)))}')
R('s <- with({0}, summary(es_score~raingarden + consult_case, fun=sf)'.format(consult))
but ran into issues where the dataframe column names were somehow causing the error: RRuntimeError: Error in (function (file = "", n = NULL, text = NULL, prompt = "?", keep.source = getOption("keep.source"), :
<text>:1:19: unexpected symbol
1: s <- with( n raingarden
I could of course do this all in R, but I have a very involved ETL script in python, and would thus prefer to keep everything in python using rpy2 (I did try this using mord for scipy-learn to run my regreession, but it is pretty primitive).
Any suggestions would be most welcome right now.
EDIT
I tried various combinations #Parfait's suggestions, and qualifying the fun argument is syntactically incorrect, as per PyCharm interpreter (see image with red highlighting at end): ... it doesn't matter what the qualifier is, either, I always get an error
that SyntaxError: keyword can't be an expression.
On the other hand, with no qualifier, there is no syntax error: , but I do get the error TypeError: 'SignatureTranslatedAnonymousPackage' object is not callable when using the function sf as obtained:
from rpy2.robjects.packages import STAP
with open('/Users/gregsilverman/development/python/rest_api/rest_api/scripts/utils.r', 'r') as f:
string = f.read()
sf = STAP(string, "sf")
With that in mind, I created a package in R with the function sf, imported it, and tried various combos with the only one producing no error, being: print(base._with(consult_case_control, R.summary(formula, fun=gms.sf))) (gms is a reference to the package in R I made).
The output though makes no sense:
Length Class Mode
3 formula call
I am expecting a table ala the one on the UCLA site. Interesting. I am going to try recreating my analysis in R, just for the heck of it. I still would like to complete it in python though.

Consider bracketing the with call and be sure to qualify all arguments including fun:
ro.r['with'](work_case_control, ro.r.summary(formula, ro.r.summary.fun=sf))
Alternatively, import R's base package. And to avoid conflict with Python's named method with() translate the R name:
from rpy2.robjects.packages import importr
base = importr('base', robject_translations={'with': '_with'})
base._with(work_case_control, ro.r.summary(formula, ro.r.summary.fun=sf))
And be sure to properly create your formula. Consider using R's stats packages' as.formula to build from string. Notice too another translation is made due to naming conflict:
stats = importr('stats', robject_translations={'format_perc': '_format_perc'})
formula = stats.as_formula('es_score ~ n + raingarden + consult_case')

Modulo large prime - TypeError: unsupported operand type(s) for %: 'int' and 'str'

I am trying to partition a large string F to m blocks as follows:
import random
from largeprimes import generateRandom
def generateRandom(length):
t = random.randint(0, 2**length)
str = "{0:b}".format(t)
if (len(str) < length):
str.zfill(length)
return str
def divide_file_block (): #return bi: int
b = []
for i in range (0, m):
b_i = F[(i*blocklen) : ((i+1)* blocklen)]
temp = int(b_i, 2) % q
b.append(temp)
return b
F = generateRandom(102400)
m = 100
blocklen = len(F)/m
q = generateLargePrime(1024) # generateLargePrime is from https://langui.sh/2009/03/07/generating-very-large-primes/
print divide_file_block ()
Note: you shall copy the code from 1 to the current directory, remove the last print statement there and name it largpeprimes.py. This makes the generateLargePrime function importable.
When I test on a small example, it printed out correct result. But when I test on b_i and q having 1024 bits, it printed out the error:
temp = int(b_i, 2) % q
TypeError: unsupported operand type(s) for %: 'int' and 'str'
Could you please explain me why and give me a suggestion to show this problem. Thanks in advance.

The function generateLargePrime sometime returns string
Trying the code I run into the same problem.
Testing it in debugger I got the cause: q has a value of `'Failure after 1100.0 tries.'
This is definitely a string an causes the failures.
I would recommend modifying the generateLargePrime code to throw an exception instead of reporting the failure by returned value.
Few tips for detecting this kind of problems
print out the problematic value
This is the simplest (and probably most popular) quick solution.
something like
b_i = F[(i*blocklen) : ((i+1)* blocklen)]
print "q", q # here is all the magic
temp = int(b_i, 2) % q
would tell you the value
place assert into your code
b_i = F[(i*blocklen) : ((i+1)* blocklen)]
assert isinstance(q, int)
temp = int(b_i, 2) % q
would throw an exception as soon as q is not of type int
run the code in debugger
pdb comes with Python, I prefer ipdb which comes with IPython, both would help you.
have the failing code written as a script
Try to run it via Python interpreter
$ python failingscript.py
as it fails, you simply try once more, but instead of python use pdb or ipdb
$ ipdb failingscript.py
the debugger let you control running the code line by line. Usually I let it run by "c" (continue) command and it soon crashes at the problematic point. Then I use "l" (list) to see, what line of code we are at, and finally use "p" (print) command to print values of variables, which are making problems. This way I used "p q" and found, it is a string.
It takes a moment to learn pdb or ipdb, but it works as turbo resolver so it is definitely the skill to learn. Great tutorial is at PMotW

Generating LaTeX tables from R summary with RPy and xtable

I am running a few linear model fits in python (using R as a backend via RPy) and I would like to export some LaTeX tables with my R "summary" data.
This thread explains quite well how to do it in R (with the xtable function), but I cannot figure out how to implement this in RPy.
The only relevant thing searches such as "Chunk RPy" or "xtable RPy" returned was this, which seems to load the package in python but not to use it :-/
Here's an example of how I use RPy and what happens.
And this would be the error without bothering to load any data:
from rpy2.robjects.packages import importr
xtable = importr('xtable')
latex = xtable('')
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-131-8b38f31b5bb9> in <module>()
----> 1 latex = xtable(res_sum)
2 print latex
TypeError: 'SignatureTranslatedPackage' object is not callable
I have tried using the stargazer package instead of xtable and I get the same error.

Ok, I solved it, and I'm a bit ashamed to say that it was a total no-brainer.
You just have to call the functions as xtable.xtable() or stargazer.stargazer().

To easily generate TeX data from Python, I wrote the following function;
import re
def tformat(txt, v):
"""Replace the variables between [] in raw text with the contents
of the named variables. Between the [] there should be a variable name,
a colon and a formatting specification. E.g. [smin:.2f] would give the
value of the smin variable printed as a float with two decimal digits.
:txt: The text to search for replacements
:v: Dictionary to use for variables.
:returns: The txt string with variables substituted by their formatted
values.
"""
rx = re.compile(r'\[(\w+)(\[\d+\])?:([^\]]+)\]')
matches = rx.finditer(txt)
for m in matches:
nm, idx, fmt = m.groups()
try:
if idx:
idx = int(idx[1:-1])
r = format(v[nm][idx], fmt)
else:
r = format(v[nm], fmt)
txt = txt.replace(m.group(0), r)
except KeyError:
raise ValueError('Variable "{}" not found'.format(nm))
return txt
You can use any variable name from the dictionary in the text that you pass to this function and have it replaced by the formatted value of that variable.
What I tend to do is to do my calculations in Python, and then pass the output of the globals() function as the second parameter of tformat:
smin = 235.0
smax = 580.0
lst = [0, 1, 2, 3, 4]
t = r'''The strength of the steel lies between SI{[smin:.0f]}{MPa} and \SI{[smax:.0f]}{MPa}. lst[2] = [lst[2]:d].'''
print tformat(t, globals())
Feel free to use this. I put it in the public domain.
Edit: I'm not sure what you mean by "linear model fits", but might numpy.polyfit do what you want in Python?

To resolve your problem, please update stargazer to version 4.5.3, now available on CRAN. Your example should then work perfectly.

math syntax checker written in python

All I need is to check, using python, if a string is a valid math expression or not.
For simplicity let's say I just need + - * / operators (+ - as unary too) with numbers and nested parenthesis. I add also simple variable names for completeness.
So I can test this way:
test("-3 * (2 + 1)") #valid
test("-3 * ") #NOT valid
test("v1 + v2") #valid
test("v2 - 2v") #NOT valid ("2v" not a valid variable name)
I tried pyparsing but just trying the example: "simple algebraic expression parser, that performs +,-,*,/ and ^ arithmetic operations" I get passed invalid code and also trying to fix it I always get wrong syntaxes being parsed without raising Exceptions
just try:
>>>test('9', 9)
9 qwerty = 9.0 ['9'] => ['9']
>>>test('9 qwerty', 9)
9 qwerty = 9.0 ['9'] => ['9']
both test pass... o_O
Any advice?

This is because the pyparsing code allows functions. (And by the way, it does a lot more than what you need, i.e. create a stack and evaluate that.)
For starters, you could remove pi and ident (and possibly something else I'm missing right now) from the code to disallow characters.
The reason is different: PyParsing parsers won't try to consume the whole input by default. You have to add + StringEnd() (and import it, of course) to the end of expr to make it fail if it can't parse the whole input. In that case, pyparsing.ParseException will be raised. (Source: http://pyparsing-public.wikispaces.com/FAQs)
If you care to learn a bit of parsing, what you need can propably be built in less than thirty lines with any decent parsing library (I like LEPL).

You could try building a simple parser yourself to tokenize the string of the arithmetic expression and then build an expression tree, if the tree is valid (the leaves are all operands and the internal nodes are all operators) then you can say that the expression is valid.
The basic concept is to make a few helper functions to create your parser.
def extract() will get the next character from the expression
def peek() similar to extract but used if there is no whitespace to check the next character
get_expression()
get_next_token()
Alternatively if you can guarantee whitespace between characters you could use split() to do all the tokenizing.
Then you build your tree and evaluate if its structured correctly
Try this for more info: http://effbot.org/zone/simple-top-down-parsing.htm

Why not just evaluate it and catch the syntax error?
from math import *
def validateSyntax(expression):
functions = {'__builtins__': None}
variables = {'__builtins__': None}
functions = {'acos': acos,
'asin': asin,
'atan': atan,
'atan2': atan2,
'ceil': ceil,
'cos': cos,
'cosh': cosh,
'degrees': degrees,
'exp': exp,
'fabs':fabs,
'floor': floor,
'fmod': fmod,
'frexp': frexp,
'hypot': hypot,
'ldexp': ldexp,
'log': log,
'log10': log10,
'modf': modf,
'pow': pow,
'radians': radians,
'sin': sin,
'sinh': sinh,
'sqrt': sqrt,
'tan': tan,
'tanh': tanh}
variables = {'e': e, 'pi': pi}
try:
eval(expression, variables, functions)
except (SyntaxError, NameError, ZeroDivisionError):
return False
else:
return True
Here are some samples:
> print validSyntax('a+b-1') # a, b are undefined, so a NameError arises.
> False
> print validSyntax('1 + 2')
> True
> print validSyntax('1 - 2')
> True
> print validSyntax('1 / 2')
> True
> print validSyntax('1 * 2')
> True
> print validSyntax('1 +/ 2')
> False
> print validSyntax('1 + (2')
> False
> print validSyntax('import os')
> False
> print validSyntax('print "asd"')
> False
> print validSyntax('import os; os.delete("~\test.txt")')
> False # And the file was not removed
It's restricted to only mathematical operations, so it should work a bit better than a crude eval.

Adding parseAll=True to the call to parseString will convert this parser into a validator.

If you are interested in modifying a custom math evaluator engine written in Python so that it is a validator instead, you could start out with Evaluator 2.0 (Python 3.x) and Math_Evaluator (Python 2.x). They are not ready-made solutions but would allow you to fully customize whatever it is you are trying to do exactly using (hopefully) easy-to-read Python code. Note that "and" & "or" are treated as operators.