We Keep Coding

How do I only allow a list a maximum number of items in Python

I would like to apply a maximum number of items for a list, making sure that the code does not allow the function that appends to the list to add more than, for example, 3 items.
Function that appends to list:
transactions = []
def append_hash():
transactions.append(hash)
How do I not allow append_hash to add more than three hashes to the list: transactions without deleting any previous hashes?

A list is, by definition, of arbitrary size. You'll need a new type instead.
class BoundedListFullError(RuntimeError):
pass
class BoundedList:
def __init__(self, max_size, x=None):
if x is None:
x = []
self.values = []
self.values.extend(x)
self.max_size = max_size
def append(self, x):
if len(self.values) == self.max_size:
raise BoundedListFullError(self.max_size)
self.values.append(x)
def extend(self, xs):
if len(self.values) + len(xs) > self.max_size:
raise BoundedListFullError(self.max_size)
self.values.extend(xs)

You could just subclass list and modify the append method:
class MyStack(list):
def __init__(self, max_size, *args, **kwargs):
super().__init__(*args, **kwargs)
self.max_size = max_size
def append(self, value):
if len(self) >= self.max_size:
raise ValueError("NO!")
# Per #chepner's suggestion
super().append(value)
somestack = MyStack(3)
somestack.append(1)
somestack.append(2)
somestack.append(3)
somestack.append(4) # Raises ValueError

If you controll your code and ensure you only ever use your function:
transactions = []
def append_hash(h):
transaction = (transactions + [h])[:3]
or
def append_hash(h):
if len(transaction) < 3:
transaction.append(3)
# else:
# raise some error you need to choose/define
Neither of those will enforce it though - you can still modify the list without your function. You would need a seperate class - see chepners answer.
Adding a fourth hash will silently fail - if you want to raise an exception instead, the second solution can be adapted.

Can I limit the Python eval() result range?

Can I limit the value range of eval()?
eval('12345678**9') returns a large number
eval('1234567**8**9') neither returns nor throws an exception
I could live with #1 but not with #2. All I need is results in the range of 32bit integers. I fear there is no way to tell eval to stop calculating too large numbers, or is there?

I've written "calculators" before using ast to parse the string into a tree and then walk the tree. In this case, if you want to do some trickery, you can make this work:
import ast
import ctypes
import operator
def _pow(a, b):
if isinstance(a, (ctypes.c_int, ctypes.c_float, ctypes.c_double)):
a = float(a.value)
if isinstance(b, (ctypes.c_int, ctypes.c_float, ctypes.c_double)):
b = float(b.value)
return ctypes.c_double(a ** b)
def _wrap_bin_op(op):
def wrapper(a, b):
if isinstance(a, (ctypes.c_int, ctypes.c_float, ctypes.c_double)):
a = float(a.value)
if isinstance(b, (ctypes.c_int, ctypes.c_float, ctypes.c_double)):
b = float(b.value)
return ctypes.c_double(op(a, b))
return wrapper
def _wrap_unary_op(op):
def wrapper(a):
if isinstance(a, (ctypes.c_int, ctypes.c_float)):
a = float(a.value)
return ctypes.c_double(op(a))
return wrapper
_OP_MAP = {
ast.Add: _wrap_bin_op(operator.add),
ast.Sub: _wrap_bin_op(operator.sub),
ast.Pow: _wrap_bin_op(operator.pow),
ast.Mult: _wrap_bin_op(operator.mul),
ast.Div: _wrap_bin_op(operator.truediv),
ast.Invert: _wrap_unary_op(operator.neg),
}
class Calc(ast.NodeVisitor):
def visit_BinOp(self, node):
left = self.visit(node.left)
right = self.visit(node.right)
return _OP_MAP[type(node.op)](left, right)
def visit_Num(self, node):
if isinstance(node.n, int):
val = ctypes.c_int(node.n)
elif isinstance(node.n, float):
val = ctypes.c_double(node.n)
return val
def visit_Expr(self, node):
return self.visit(node.value)
#classmethod
def evaluate(cls, expression):
tree = ast.parse(expression)
calc = cls()
return calc.visit(tree.body[0])
print(Calc.evaluate('12345678**8'))
print(Calc.evaluate('5 * 8'))
Note that unlike eval, I'm specifically picking and choosing what operations I want to allow -- and I have control over how they behave. In this case, I'm doing all of my math with ctypes to avoid HUGE numbers. I'm also preventing integer __pow__ and forcing those arguments to become floats before raising to a specific power.

As John Coleman recommended, I'll add my own solution here. Thanks for the discussion, I learned a lot about pythons capabilities.
As I commented already:
I found a solution by making any number a float via concatenating '.0', eval('1234567.0**8.0**9.0') throws an exception, that's fine.
Here's the bigger context, where this evaluation is embedded:
import itertools
digits1to8 = list(str(i+1) for i in range(8)) #('1','2','3','4','5','6','7','8')
with open("expressions.txt", "w") as outfile:
for operators in itertools.product(['','.0+','.0-','.0*','.0/','.0**'], repeat=8):
calculation = zip(digits1to8,operators)
expression = (''.join(list(itertools.chain(*calculation))))+'9.0'
try:
out = str(eval(expression))+','
expression = expression.replace('.0','')
out = out.replace('.0,',',') + expression
if (not out.find('.')>0):
print(out, file=outfile)
except:
pass
beforehand I had ['','+','-','*','/','**'] instead of ['','.0+','.0-','.0*','.0/','.0**']. Overall this just is a little mathematical experimentation in response to https://www.youtube.com/watch?v=-ruC5A9EzzE

Something along these lines:
from math import log
def bounded_eval(expression, bits = 32):
nums = expression.split('**')
if len(nums) == 1:
val = eval(expression)
if log(val,2) > bits:
return "too large"
else:
return val
else:
base = nums[0]
power = '**'.join(nums[1:])
base = eval(base)
power = eval(power)
if power*log(base,2) > bits:
return "too large"
else:
return pow(base,power)
This does use eval() which is potentially a security risk, but if you are just calling it on arithmetical expressions that your own code generates then it isn't really a problem. Obviously, you can replace the code which returns "too large" by code which raises an error.

how to fix the _getitem__ method

def pnamedtuple(type_name, field_names, mutable=False):
pass
class type_name:
def __init__(self, x, y):
self.x = x
self.y = y
self._fields = ['x','y']
self._mutable = False
def get_x(self):
return self.x
def get_y(self):
return self.y
def __getitem__(self,i):
if i > 1 or i <0:
raise IndexError
if i == 0 or i == 'x':
return self.get_x():
if i == 1 or i == 'y':
return self.get_y():
the getitem method to overload the [] (indexing operator) for this class: an index of 0 returns the value of the first field name in the field_names list; an index of 1 returns the value of the second field name in the field_names list, etc. Also, the index can be a string with the named field. So, for p = Point(1,2) writing p.get_x(), or p[0]), or p['x'] returns a result of 1. Raise an IndexError with an appropriate message if the index is out of bounds int or a string that does not name a field.
I am not sure how to fix the getitme function. below is the bsc.txt
c-->t1 = Triple1(1,2,3)
c-->t2 = Triple2(1,2,3)
c-->t3 = Triple3(1,2,3)
# Test __getitem__ functions
e-->t1[0]-->1
e-->t1[1]-->2
e-->t1[2]-->3
e-->t1['a']-->1
e-->t1['b']-->2
e-->t1['c']-->3
^-->t1[4]-->IndexError
^-->t1['d']-->IndexError
^-->t1[3.2]-->IndexError
can someone tell how to fix my _getitem _ function to get the output in bsc.txt? many thanks.

You've spelled __getitem__ incorrectly. Magic methods require two __ underscores before and after them.
So you haven't overloaded the original __getitem__ method, you've simply created a new method named _getitem_.

Python 3 does not allow strings and integers to be compared with > or <; it's best to stick with == if you don't yet know the type of i. You could use isinstance, but here you can easily convert the only two valid integer values to strings (or vice versa), then work only on strings.
def __getitem__(self, i):
if i == 0:
i = "x"
elif i == 1:
i = "y"
if i == "x":
return self.get_x()
elif i == "y":
return self.get_y()
else:
raise IndexError("Invalid key: {}".format(i))

your function is interesting, but there are some issues with it:
In python 3 you can't compare string with numbers, so you first should check with == against know values and or types. For example
def __getitem__(self,i):
if i in {0,"x"}:
return self.x
elif i in {1,"y"}:
return self.y
else:
raise IndexError(repr(i))
But defined like that (in your code or in the example above) for an instance t1 this t1[X] for all string X others than "x" or "y" will always fail as you don't adjust it for any other value. And that is because
pnamedtuple looks like you want for it to be a factory like collections.namedtuple, but it fail to be general enough because you don't use any the arguments of your function at all. And no, type_name is not used either, whatever value it have is throw away when you make the class declaration.
how to fix it?
You need other ways to store the value of the fields and its respective name, for example a dictionary lets call it self._data
To remember how you called yours field, use the argument of your function, for instance self._fields = field_names
To accept a unknown number of arguments use * like __init__(self, *values) then verify that you have the same numbers of values and fields and build your data structure of point 1 (the dictionary)
Once that those are ready then __getitem__ become something like:
def __getitem__(self, key):
if key in self._data:
return self._data[key]
elif isintance(key,int) and 0 <= key < len(self._fields):
return self._data[ self._fields[key] ]
else:
raise IndexError( repr(key) )
or you can simple inherit from a appropriate namedtuple and the only thing you need to do is overwrite its __getitem__ like
def __getitem__(self,key):
if key in self._fields:
return getattr(self,key)
return super().__getitem__(key)

SQLAlchemy - Making a hybrid_property which utilizes multiple relationships queryable

I have a Flask-SQLAlchemy model that contains several relationships to tables for holding quantities in decimal, fraction, and integer:
class Packet(db.Model):
# ...
qty_decimal_id = db.Column(db.Integer, db.ForeignKey('qty_decimals.id'))
_qty_decimal = db.relationship('QtyDecimal', backref='_packets')
qty_fraction_id = db.Column(db.Integer, db.ForeignKey('qty_fractions.id'))
_qty_fraction = db.relationship('QtyFraction', backref='_packets')
qty_integer_id = db.Column(db.Integer, db.ForeignKey('qty_integers.id'))
_qty_integer = db.relationship('QtyInteger', backref='_packets')
# ...
The tables each contain a column named 'value' that contains the actual value, such that if I want to store an integer quantity of '100', I would store it in ._qty_integer.value. I have created a hybrid_property that currently gets whichever of these relationships is not null, and sets to a relevant relationship depending on what kind of data is detected by the setter:
#hybrid_property
def quantity(self):
retqty = None
for qty in [self._qty_decimal, self._qty_fraction, self._qty_integer]:
if qty is not None:
if retqty is None:
retqty = qty
else:
raise RuntimeError('More than one type of quantity'
' was detected for this packet. Only'
' one type of quantity may be set!')
return retqty.value
#quantity.setter
def quantity(self, value):
if is_decimal(value):
self.clear_quantity()
self._qty_decimal = QtyDecimal.query.filter_by(value=value).\
first() or QtyDecimal(value)
elif is_fraction(value):
if is_480th(value):
self.clear_quantity()
self._qty_fraction = QtyFraction.query.filter_by(value=value).\
first() or QtyFraction(value)
else:
raise ValueError('Fractions must have denominators'
' 480 is divisible by!')
elif is_int(value):
self.clear_quantity()
self._qty_integer = QtyInteger.query.filter_by(value=value).\
first() or QtyInteger(value)
else:
raise ValueError('Could not determine appropriate type for '
'quantity! Please make sure it is a integer, '
'decimal number, or fraction.')
The getter and setter for .quantity work perfectly as far as I can tell (it's always possible I missed some edge cases in my test suite) but I cannot, for the life of me, figure out how to implement a comparator or expression such that one could, for example, get all packets that have a quantity of 100, or all packets with a quantity of 1/4. (Units are implemented too, but irrelevant to this question.) As far as I can tell it's not doable as an expression, but it should be doable as a comparator. I can't figure out how to pull it off, though. So far this is my best attempt at a comparator:
class QuantityComparator(Comparator):
def __init__(self, qty_decimal, qty_fraction, qty_integer):
self.qty_decimal = qty_decimal
self.qty_fraction = qty_fraction
self.qty_integer = qty_integer
def __eq__(self, other):
if is_decimal(other):
return self.qty_decimal.value == other
elif is_fraction(other):
if is_480th(other):
return self.qty_fraction.value == other
else:
raise ValueError('Cannot query using a fraction with a '
'denominator 480 is not divisible by!')
elif is_int(other):
return self.qty_fraction.value == other
else:
raise ValueError('Could not parse query value'
' as a valid quantity!')
#quantity.comparator
def quantity(self):
return self.QuantityComparator(self._qty_decimal,
self._qty_fraction,
self._qty_integer)
Unsurprisingly, this does not work, when I try to run Packet.query.filter_by(quantity=) it raises an exception:
AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Packet._qty_fraction has an attribute 'value'
If the solution to getting this use case to work is in the SQLAlchemy docs, I've either missed it or (more likely) haven't wrapped my head around enough of SQLAlchemy yet to figure it out.
I have come up with a stopgap solution that at least lets users get Packets based on quantity:
#staticmethod
def quantity_equals(value):
if is_decimal(value):
return Packet.query.join(Packet._qty_decimal).\
filter(QtyDecimal.value == value)
elif is_fraction(value):
if is_480th(value):
return Packet.query.join(Packet._qty_fraction).\
filter(QtyFraction.value == value)
else:
raise ValueError('Fraction could not be converted to 480ths!')
elif is_int(value):
return Packet.query.join(Packet._qty_integer).\
filter(QtyInteger.value == value)
This works and gets me what looks to be the correct Query object, as shown by this test:
def test_quantity_equals(self):
pkt1 = Packet()
pkt2 = Packet()
pkt3 = Packet()
db.session.add_all([pkt1, pkt2, pkt3])
pkt1.quantity = Decimal('3.14')
pkt2.quantity = Fraction(1, 4)
pkt3.quantity = 100
db.session.commit()
qty_dec_query = Packet.quantity_equals(Decimal('3.14'))
self.assertIs(pkt1, qty_dec_query.first())
self.assertEqual(qty_dec_query.count(), 1)
qty_frac_query = Packet.quantity_equals(Fraction(1, 4))
self.assertIs(pkt2, qty_frac_query.first())
self.assertEqual(qty_frac_query.count(), 1)
qty_int_query = Packet.quantity_equals(100)
self.assertIs(pkt3, qty_int_query.first())
self.assertEqual(qty_int_query.count(), 1)
I can easily make similar dirty methods to substitute for other comparison operators, but I would think it's possible to do it in a custom comparator such as the aforementioned QuantityComparator, or to otherwise achieve the desired ability to use the .quantity property in query filters.
Can anybody help me get the QuantityComparator working, or point me in the right direction for figuring it out myself?
Edit: Solution
While I haven't actually solved the explicit question of making a hybrid_property with multiple relationships queryable, I have solved the core issue of representing a quantity that could either be int, float (I use float instead of Decimal here because no arithmetic is done the quantity) or Fraction in the database, and making it possible to use values in queries as they would be used by the value setter. The solution was to create a quantities table with rows for a floating point value, an integer numerator, an integer denominator, and a boolean representing whether or not the stored value should be interpreted as a decimal number (as opposed to a fraction) and create an sqlalchemy.ext.hybrid.Comparator for the hybrid property which compares against Quantity._float and Quantity.is_decimal:
class Quantity(db.Model):
# ...
_denominator = db.Column(db.Integer)
_float = db.Column(db.Float)
is_decimal = db.Column(db.Boolean, default=False)
_numerator = db.Column(db.Integer)
# ...
#staticmethod
def for_cmp(val):
"""Convert val to float so it can be used to query against _float."""
if Quantity.dec_check(val): # True if val looks like a decimal number
return float(val)
elif isinstance(val, str):
frac = Quantity.str_to_fraction(val)
else:
frac = Fraction(val)
return float(frac)
#hybrid_property
def value(self):
if self._float is not None:
if self.is_decimal:
return self._float
elif self._denominator == 1:
return self._numerator
else:
return Fraction(self._numerator, self._denominator)
else:
return None
class ValueComparator(Comparator):
def operate(self, op, other):
return and_(op(Quantity._float, Quantity.for_cmp(other)),
Quantity.is_decimal == Quantity.dec_check(other))
#value.comparator
def value(cls):
return Quantity.ValueComparator(cls)
#value.setter
def value(self, val):
if val is not None:
if Quantity.dec_check(val):
self.is_decimal = True
self._float = float(val)
self._numerator = None
self._denominator = None
else:
self.is_decimal = False
if isinstance(val, str):
frac = Quantity.str_to_fraction(val)
else:
frac = Fraction(val)
self._numerator = frac.numerator
self._denominator = frac.denominator
self._float = float(frac)
else:
self.is_decimal = None
self._numerator = None
self._denominator = None
self._float = None
Since I don't have the original model I asked this question for any longer, I can't readily go back and answer this question properly, but I'd imagine it could be done using join or select.

Mathematical equation manipulation in Python

I want to develop a GUI application which displays a given mathematical equation. When you click upon a particular variable in the equation to signify that it is the unknown variable ie., to be calculated, the equation transforms itself to evaluate the required unknown variable.
For example:
a = (b+c*d)/e
Let us suppose that I click upon "d" to signify that it is the unknown variable. Then the equation should be re-structured to:
d = (a*e - b)/c
As of now, I just want to know how I can go about rearranging the given equation based on user input. One suggestion I got from my brother was to use pre-fix/post-fix notational representation in back end to evaluate it.
Is that the only way to go or is there any simpler suggestion?
Also, I will be using not only basic mathematical functions but also trignometric and calculus (basic I think. No partial differential calculus and all that) as well. I think that the pre/post-fix notation evaluation might not be helpful in evaluation higher mathematical functions.
But that is just my opinion, so please point out if I am wrong.
Also, I will be using SymPy for mathematical evaluation so evaluation of a given mathematical equation is not a problem, creating a specific equation from a given generic one is my main problem.

Using SymPy, your example would go something like this:
>>> import sympy
>>> a,b,c,d,e = sympy.symbols('abcde')
>>> r = (b+c*d)/e
>>> l = a
>>> r = sympy.solve(l-r,d)
>>> l = d
>>> r
[(-b + a*e)/c]
>>>
It seems to work for trigonometric functions too:
>>> l = a
>>> r = b*sympy.sin(c)
>>> sympy.solve(l-r,c)
[asin(a/b)]
>>>
And since you are working with a GUI, you'll (probably) want to convert back and forth from strings to expressions:
>>> r = '(b+c*d)/e'
>>> sympy.sympify(r)
(b + c*d)/e
>>> sympy.sstr(_)
'(b + c*d)/e'
>>>
or you may prefer to display them as rendered LaTeX or MathML.

If you want to do this out of the box, without relying on librairies, I think that the problems you will find are not Python related. If you want to find such equations, you have to describe the heuristics necessary to solve these equations.
First, you have to represent your equation. What about separating:
operands:
symbolic operands (a,b)
numeric operands (1,2)
operators:
unary operators (-, trig functions)
binary operators (+,-,*,/)
Unary operators will obviously enclose one operand, binary ops will enclose two.
What about types?
I think that all of these components should derivate from a single common expression type.
And this class would have a getsymbols method to locate quickly symbols in your expressions.
And then distinguish between unary and binary operators, add a few basic complement/reorder primitives...
Something like:
class expression(object):
def symbols(self):
if not hasattr(self, '_symbols'):
self._symbols = self._getsymbols()
return self._symbols
def _getsymbols(self):
"""
return type: list of strings
"""
raise NotImplementedError
class operand(expression): pass
class symbolicoperand(operand):
def __init__(self, name):
self.name = name
def _getsymbols(self):
return [self.name]
def __str__(self):
return self.name
class numericoperand(operand):
def __init__(self, value):
self.value = value
def _getsymbols(self):
return []
def __str__(self):
return str(self.value)
class operator(expression): pass
class binaryoperator(operator):
def __init__(self, lop, rop):
"""
#type lop, rop: expression
"""
self.lop = lop
self.rop = rop
def _getsymbols(self):
return self.lop._getsymbols() + self.rop._getsymbols()
#staticmethod
def complementop():
"""
Return complement operator:
op.complementop()(op(a,b), b) = a
"""
raise NotImplementedError
def reorder():
"""
for op1(a,b) return op2(f(b),g(a)) such as op1(a,b) = op2(f(a),g(b))
"""
raise NotImplementedError
def _getstr(self):
"""
string representing the operator alone
"""
raise NotImplementedError
def __str__(self):
lop = str(self.lop)
if isinstance(self.lop, operator):
lop = '(%s)' % lop
rop = str(self.rop)
if isinstance(self.rop, operator):
rop = '(%s)' % rop
return '%s%s%s' % (lop, self._getstr(), rop)
class symetricoperator(binaryoperator):
def reorder(self):
return self.__class__(self.rop, self.lop)
class asymetricoperator(binaryoperator):
#staticmethod
def _invert(operand):
"""
div._invert(a) -> 1/a
sub._invert(a) -> -a
"""
raise NotImplementedError
def reorder(self):
return self.complementop()(self._invert(self.rop), self.lop)
class div(asymetricoperator):
#staticmethod
def _invert(operand):
if isinstance(operand, div):
return div(self.rop, self.lop)
else:
return div(numericoperand(1), operand)
#staticmethod
def complementop():
return mul
def _getstr(self):
return '/'
class mul(symetricoperator):
#staticmethod
def complementop():
return div
def _getstr(self):
return '*'
class add(symetricoperator):
#staticmethod
def complementop():
return sub
def _getstr(self):
return '+'
class sub(asymetricoperator):
#staticmethod
def _invert(operand):
if isinstance(operand, min):
return operand.op
else:
return min(operand)
#staticmethod
def complementop():
return add
def _getstr(self):
return '-'
class unaryoperator(operator):
def __init__(self, op):
"""
#type op: expression
"""
self.op = op
#staticmethod
def complement(expression):
raise NotImplementedError
def _getsymbols(self):
return self.op._getsymbols()
class min(unaryoperator):
#staticmethod
def complement(expression):
if isinstance(expression, min):
return expression.op
else:
return min(expression)
def __str__(self):
return '-' + str(self.op)
With this basic structure set up, you should be able to describe a simple heuristic to solve very simple equations. Just think of the simple rules you learned to solve equations, and write them down. That should work :)
And then a very naive solver:
def solve(left, right, symbol):
"""
#type left, right: expression
#type symbol: string
"""
if symbol not in left.symbols():
if symbol not in right.symbols():
raise ValueError('%s not in expressions' % symbol)
left, right = right, left
solved = False
while not solved:
if isinstance(left, operator):
if isinstance(left, unaryoperator):
complementor = left.complement
right = complementor(right)
left = complementor(left)
elif isinstance(left, binaryoperator):
if symbol in left.rop.symbols():
left = left.reorder()
else:
right = left.complementop()(right, left.rop)
left = left.lop
elif isinstance(left, operand):
assert isinstance(left, symbolicoperand)
assert symbol==left.name
solved = True
print symbol,'=',right
a,b,c,d,e = map(symbolicoperand, 'abcde')
solve(a, div(add(b,mul(c,d)),e), 'd') # d = ((a*e)-b)/c
solve(numericoperand(1), min(min(a)), 'a') # a = 1

Things have sure changed since 2009. I don't know how your GUI application is going, but this is now possible directly in IPython qtconsole (which one could embed inside a custom PyQt/PySide application, and keep track of all the defined symbols, to allow GUI interaction in a separate listbox, etc.)
(Uses the sympyprt extension for IPython)

What you want to do isn't easy. Some equations are quite straight forward to rearrange (like make b the subject of a = b*c+d, which is b = (a-d)/c), while others are not so obvious (like make x the subject of y = x*x + 4*x + 4), while others are not possible (especially when you trigonometric functions and other complications).
As other people have said, check out Sage. It does what you want:
You can solve equations for one variable in terms of others:
sage: x, b, c = var('x b c')
sage: solve([x^2 + b*x + c == 0],x)
[x == -1/2*b - 1/2*sqrt(b^2 - 4*c), x == -1/2*b + 1/2*sqrt(b^2 - 4*c)]

Sage has support for symbolic math. You could just use some of the equation manipulating functions built-in:
http://sagemath.org/