We Keep Coding

Trying to compare 3 wtforms fields with custom functions and geting recursion problem

I have 3 fields that I want to compare salary "from" field and "to" field and also there is fixed salary field. I have no idea how to do it, since there is no documentation how to do it, so i created custom function that look to each other and trying to se if they have a value.
def validate_salarylow(self, salarylow):
if self.validate_salary_fixed(self.salary_fixed) != "":
salarylow.data = int(0)
else:
try:
salarylow.data = int(salarylow.data)
except:
raise ValidationError("value is not a number")
return salarylow.data
def validate_salary_high(self, salary_high):
if self.validate_salary_fixed(self.salary_fixed) != "":
salary_high.data = int(0)
else:
try:
salary_high.data = int(salary_high.data)
except:
raise ValidationError("value is not a number")
return salary_high.data
def validate_salary_fixed(self, salary_fixed):
if self.validate_salary_high(self.salary_high) != "":
salary_fixed.data = int(0)
try:
salary_fixed.data = int(salary_fixed.data)
except:
raise ValidationError("value is not a number")
return salary_fixed.data
if I don't set if self.validate_salary_high(self.salary_high) != "": everything works fine. but when i set it I'm getting "RecursionError: maximum recursion depth exceeded" error.validate_salary_fixed function looks to validate_salary_high function and vice versa. I'm new in Python and flask and I'm sure there is easy solution, but I cant find it so I would appreciate if anyone could help.

My suggestion is to suppress the error message of the integer field by overwriting it. Thus, the types of the inputs do not have to be converted.
For validation I use two custom validators, one of which checks whether a range or a fixed value has been entered and the second checks the range for its limits. In addition, pre-built validators are used to prohibit negative values.
I'm not sure if you really need the field for the fixed salary, because it is possible to define a fixed value by narrowing the range.
from flask_wtf import FlaskForm
from wtforms import IntegerField
from wtforms.validators import (
NumberRange,
Optional,
StopValidation,
ValidationError
)
class OptionalIntegerField(IntegerField):
def process_data(self, value):
try:
super().process_data(value)
except ValueError:
pass
def process_formdata(self, valuelist):
try:
super().process_formdata(valuelist)
except ValueError:
pass
def validate_salary(form, field):
range_fields = [form.salary_low, form.salary_high]
if all(f.data is None for f in [form.salary_low, form.salary_high, form.salary_fixed]) or \
(form.salary_fixed.data is not None and any(f.data is not None for f in range_fields)) or \
(form.salary_fixed.data is None and any(f.data is None for f in range_fields)):
raise StopValidation('Either state a range from low to high or a fixed salary.')
def validate_salary_range(form, field):
if form.salary_low.data and form.salary_high.data and \
form.salary_low.data > form.salary_high.data:
raise ValidationError('The lower value should be less than or equal to the higher one.')
class SalaryForm(FlaskForm):
salary_low = OptionalIntegerField(
validators=[
validate_salary,
validate_salary_range,
Optional(),
NumberRange(min=0)
]
)
salary_high = OptionalIntegerField(
validators=[
validate_salary,
validate_salary_range,
Optional(),
NumberRange(min=0)
]
)
salary_fixed = OptionalIntegerField(
validators=[
validate_salary,
Optional(),
NumberRange(min=0)
]
)
app = Flask(__name__)
app.secret_key = 'your secret here'
#app.route('/', methods=['GET', 'POST'])
def index():
form = SalaryForm(request.form)
if form.validate_on_submit():
print(form.salary_low.data, ' - ', form.salary_high.data, '||', form.salary_fixed.data)
return render_template('index.html', **locals())

Let's take a look at your code:
Your function validate_salary_high calls validate_salary_fixed.
But when you go to your function validate_salary_fixed it calls validate_salary_high.
So you go back to your function validate_salary_high which calls validate_salary_fixed.
Now in your function validate_salary_fixed, you call validate_salary_high.
Your functions repeatedly call each other over and over again, forever, until your computer eventually throws an error - and this is exactly what is happening to you.
The way to get around this is to remove one of your recursive calls. More specifically you should either
remove your call to validate_salary_fixed in the function validate_salary_high
or remove your call to validate_salary_high in the function validate_salary_fixed
You should chose which function call to remove depending on the goal of your code (which I don't fully understand.) Good luck!

Look up items in list of dataclasses by value

I'm using python to filter data in elasticsearch based on request params provided. I've got a working example, but know it can be improved and am trying to think of a better way. The current code is like this:
#dataclass(frozen=True)
class Filter:
param: str
custom_es_field: Optional[str] = None
is_bool_query: bool = False
is_date_query: bool = False
is_range_query: bool = False
def es_field(self) -> str:
if self.custom_es_field:
field = self.custom_es_field
elif "." in self.param:
field = self.param.replace(".", "__")
else:
field = self.param
return field
filters = [
Filter(param="publication_year", is_range_query=True),
Filter(param="publication_date", is_date_query=True),
Filter(param="venue.issn").
...
]
def filter_records(filter_params, s):
for filter in filters:
# range query
if filter.param in filter_params and filter.is_range_query:
param = filter_params[filter.param]
if "<" in param:
param = param[1:]
validate_range_param(filter, param)
kwargs = {filter.es_field(): {"lte": int(param)}}
s = s.filter("range", **kwargs)
elif filter.param in filter_params and filter.is_bool_query:
....
The thing I think is slow is I am looping through all of the filters in order to use the one that came in as a request variable. I'm tempted to convert this to a dictionary so I can do filter["publication_year"], but I like having the extra methods available via the dataclass. Would love to hear any thoughts.

How to validate fields and get validated output

I'm trying to validate the values in the model and get as output the validated values. The only example in the documentation is the following code, which doesn't show how exactly this is done so I can't extend on it.
>>> from schematics.models import Model
>>> from schematics.types import StringType, BooleanType
>>> from schematics.exceptions import ValidationError
>>>
>>> class Signup(Model):
... name = StringType()
... call_me = BooleanType(default=False)
... def validate_call_me(self, data, value):
... if data['name'] == u'Brad' and data['call_me'] is True:
... raise ValidationError(u'He prefers email.')
... return value
...
>>> Signup({'name': u'Brad'}).validate()
>>> Signup({'name': u'Brad', 'call_me': True}).validate()
Traceback (most recent call last):
...
ModelValidationError: {'call_me': [u'He prefers email.']}
I made a version of it but removed data and value from arguments. Client is the name of my model. So when I do the following I get the desired result as output:
client.validate_client(client.to_native())
However,
first of all, this doesn't seem a clean way. client already has all the values and so I wouldn't need to do this.
Also, I like to get this to update the values of client as an outcome of validation.
For the first part, I did something line this:
def validate_client(self):
data = self.to_native()
...
return data
But I don't think this is the best way to do this and I am not sure about the second issue of updating the values. Is there a way to do this?
EDIT:
This is the code I have and I want the client value for employer to be set to 'unspecified' and client full_name to be set as specified in the function.
class LowerCaseEmailType(EmailType):
def convert(self, value, context=None):
value = super(LowerCaseEmailType, self).convert(value, context)
return value.lower()
class CleanedStringType(StringType):
converters = []
def __init__(self, **kwargs):
"""
This takes in all the inputs as String Type, but takes in an extra
input called converters.
Converters must be a list of functions, and each of those functions
must take in exactly 1 value , and return the transformed input
"""
if 'converters' in kwargs:
self.converters = kwargs['converters']
del kwargs['converters']
super(CleanedStringType, self).__init__(**kwargs)
def convert(self, value, context=None):
value = super(CleanedStringType, self).convert(value, context)
for func in self.converters:
value = func(value)
return value # will have a value after going through all the conversions in order
class Client(Model):
"""
Client Model
"""
first_name = CleanedStringType(required=False,
converters=[lambda x: re.sub(r"[!##$%&\(\)\^]+", ' ', x),
lambda x: x.strip()])
last_name = CleanedStringType(required=False,
converters=[lambda x: re.sub(r"[!##$%&\(\)\^]+", ' ', x),
lambda x: x.strip()])
full_name = CleanedStringType(required=False,
converters=[lambda x: re.sub(r"[!##$%&\(\)\^]+", ' ', x),
lambda x: x.strip()])
employer = StringType(required=False)
created = StringType(default=" ")
updated = StringType(default=" ")
email = LowerCaseEmailType(required=False)
def validate_client(self):
data = self.to_native()
if data.get('first_name') and data.get('last_name'):
data['full_name'] = ' '.join([data['first_name'], data['last_name']])
if data.get('full_name') is None:
if data.get('first_name') is None:
error_message = 'first name missing'
else:
error_message = 'last name missing'
logger.error('info: {} for {}'.format(error_message, data))
raise ValidationError(error_message)
if data.get('employer') is None:
logger.warning('info: employer missing for {}'.format(data))
data['employer'] = 'unspecified'
return data

I think you want pre_setattr. It is called with the value when you set a value of a model (code in question).
For the eymployer I think you want to set a default. To me this seems to behave exactly as you want.
I think fullname should not be on the model, because you can have inconsistent data where firstname + lastname != fullname. If you need it later you could implement that via #serialized.

Remove a specific letter from form input

Within clean(), I am attempting to check if the User included "#" in the SearchHashtagForm and, if so, remove it.
For example, assume that User enters "#foo" on the SearchHashtagForm:
EXPECTED 'search_text': "foo"
ACTUAL 'search_text': "#foo"
I suspect that the line: form_input = form_input[1:] doesn't work, but I'm not sure what else to use?
Views.py
class HashtagSearch(FormView):
""" FormView for user to enter hashtag search query """
def form_valid(self, form):
form.clean()
return super().form_valid(form)
Forms.py
class SearchHashtagForm(ModelForm):
""" ModelForm for user to search by hashtag """
def clean(self):
cleaned_data = self.cleaned_data
form_input = cleaned_data.get('search_text')
if form_input.startswith('#'): # if input starts with '#', remove it.
form_input = form_input[1:]
cleaned_data['search_text'] = form_input
return cleaned_data
SOLVED
In the interests of being succinct, I omitted the line: self.get_tweets(form) from form_valid() from the original query. I have provided by a more complete copy of the code, along with the solution, below:
As the solution, I removed clean() and instead included lstrip("#") (as suggested by below) in both get_success_url and get_tweets().
def get_success_url(self):
return '{}?search_text={}'.format(
reverse('mapping_twitter:results'),
self.request.POST.get('search_text').lower().lstrip("#"),
)
def form_valid(self, form):
self.get_tweets(form)
return super().form_valid(form)
def get_tweets(self, form):
...
search_filter = self.request.POST.get('search_text').lower().lstrip("#")
...
tweet_object, created = Hashtag.objects.get_or_create(search_text=search_filter)
...

You may use lstrip(), to remove the unwanted characters from the starting of your string as:
>>> tag = "#foo"
>>> tag.lstrip("#")
>>> "foo"
>>> tag = "foo"
>>> tag.lstrip("#")
>>> "foo"
>>> tag = "#foo#bar"
>>> tag.lstrip("#")
>>> "foo#bar"
This will also save you the extra method calls to check if the string starts with "#" or not, it implicitly handles it and does nothing if the tag does not start with the desired "#".

if form_input = form_input[1:] and form_input.startswith('#') don't throw errors,
Maybe you've got a normal string (else Idk why slicing and .startswith('#') should work) that starts with n invisible trailing characters before the # you see.
if thats the case, just try :
form_input = form_input[2:]
form_input = form_input[3:]
form_input = form_input[4:]
...
and see if you get reasonable results.
(print form_input and see what comes out)
if this does not work, you most likely have some werid datatype in form input.
obtain type and value of form input and add those to your question.

You can try this too:
''.join(filter(lambda x: x != '#', '#foo'))
# foo

You can use regular expression for this
import re
class SearchHashtagForm(ModelForm):
""" ModelForm for user to search by hashtag """
def clean(self):
cleaned_data = self.cleaned_data
form_input = cleaned_data.get('search_text')
re.sub('[#]','', form_input)
cleaned_data['search_text'] = form_input
return cleaned_data
This will replace all the characters in the string matching '#'. This solution is more robust since you can check for more characters like "%". for instance
re.sub('[#%]','', form_input)

SQLAlchemy - Making a hybrid_property which utilizes multiple relationships queryable

I have a Flask-SQLAlchemy model that contains several relationships to tables for holding quantities in decimal, fraction, and integer:
class Packet(db.Model):
# ...
qty_decimal_id = db.Column(db.Integer, db.ForeignKey('qty_decimals.id'))
_qty_decimal = db.relationship('QtyDecimal', backref='_packets')
qty_fraction_id = db.Column(db.Integer, db.ForeignKey('qty_fractions.id'))
_qty_fraction = db.relationship('QtyFraction', backref='_packets')
qty_integer_id = db.Column(db.Integer, db.ForeignKey('qty_integers.id'))
_qty_integer = db.relationship('QtyInteger', backref='_packets')
# ...
The tables each contain a column named 'value' that contains the actual value, such that if I want to store an integer quantity of '100', I would store it in ._qty_integer.value. I have created a hybrid_property that currently gets whichever of these relationships is not null, and sets to a relevant relationship depending on what kind of data is detected by the setter:
#hybrid_property
def quantity(self):
retqty = None
for qty in [self._qty_decimal, self._qty_fraction, self._qty_integer]:
if qty is not None:
if retqty is None:
retqty = qty
else:
raise RuntimeError('More than one type of quantity'
' was detected for this packet. Only'
' one type of quantity may be set!')
return retqty.value
#quantity.setter
def quantity(self, value):
if is_decimal(value):
self.clear_quantity()
self._qty_decimal = QtyDecimal.query.filter_by(value=value).\
first() or QtyDecimal(value)
elif is_fraction(value):
if is_480th(value):
self.clear_quantity()
self._qty_fraction = QtyFraction.query.filter_by(value=value).\
first() or QtyFraction(value)
else:
raise ValueError('Fractions must have denominators'
' 480 is divisible by!')
elif is_int(value):
self.clear_quantity()
self._qty_integer = QtyInteger.query.filter_by(value=value).\
first() or QtyInteger(value)
else:
raise ValueError('Could not determine appropriate type for '
'quantity! Please make sure it is a integer, '
'decimal number, or fraction.')
The getter and setter for .quantity work perfectly as far as I can tell (it's always possible I missed some edge cases in my test suite) but I cannot, for the life of me, figure out how to implement a comparator or expression such that one could, for example, get all packets that have a quantity of 100, or all packets with a quantity of 1/4. (Units are implemented too, but irrelevant to this question.) As far as I can tell it's not doable as an expression, but it should be doable as a comparator. I can't figure out how to pull it off, though. So far this is my best attempt at a comparator:
class QuantityComparator(Comparator):
def __init__(self, qty_decimal, qty_fraction, qty_integer):
self.qty_decimal = qty_decimal
self.qty_fraction = qty_fraction
self.qty_integer = qty_integer
def __eq__(self, other):
if is_decimal(other):
return self.qty_decimal.value == other
elif is_fraction(other):
if is_480th(other):
return self.qty_fraction.value == other
else:
raise ValueError('Cannot query using a fraction with a '
'denominator 480 is not divisible by!')
elif is_int(other):
return self.qty_fraction.value == other
else:
raise ValueError('Could not parse query value'
' as a valid quantity!')
#quantity.comparator
def quantity(self):
return self.QuantityComparator(self._qty_decimal,
self._qty_fraction,
self._qty_integer)
Unsurprisingly, this does not work, when I try to run Packet.query.filter_by(quantity=) it raises an exception:
AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Packet._qty_fraction has an attribute 'value'
If the solution to getting this use case to work is in the SQLAlchemy docs, I've either missed it or (more likely) haven't wrapped my head around enough of SQLAlchemy yet to figure it out.
I have come up with a stopgap solution that at least lets users get Packets based on quantity:
#staticmethod
def quantity_equals(value):
if is_decimal(value):
return Packet.query.join(Packet._qty_decimal).\
filter(QtyDecimal.value == value)
elif is_fraction(value):
if is_480th(value):
return Packet.query.join(Packet._qty_fraction).\
filter(QtyFraction.value == value)
else:
raise ValueError('Fraction could not be converted to 480ths!')
elif is_int(value):
return Packet.query.join(Packet._qty_integer).\
filter(QtyInteger.value == value)
This works and gets me what looks to be the correct Query object, as shown by this test:
def test_quantity_equals(self):
pkt1 = Packet()
pkt2 = Packet()
pkt3 = Packet()
db.session.add_all([pkt1, pkt2, pkt3])
pkt1.quantity = Decimal('3.14')
pkt2.quantity = Fraction(1, 4)
pkt3.quantity = 100
db.session.commit()
qty_dec_query = Packet.quantity_equals(Decimal('3.14'))
self.assertIs(pkt1, qty_dec_query.first())
self.assertEqual(qty_dec_query.count(), 1)
qty_frac_query = Packet.quantity_equals(Fraction(1, 4))
self.assertIs(pkt2, qty_frac_query.first())
self.assertEqual(qty_frac_query.count(), 1)
qty_int_query = Packet.quantity_equals(100)
self.assertIs(pkt3, qty_int_query.first())
self.assertEqual(qty_int_query.count(), 1)
I can easily make similar dirty methods to substitute for other comparison operators, but I would think it's possible to do it in a custom comparator such as the aforementioned QuantityComparator, or to otherwise achieve the desired ability to use the .quantity property in query filters.
Can anybody help me get the QuantityComparator working, or point me in the right direction for figuring it out myself?
Edit: Solution
While I haven't actually solved the explicit question of making a hybrid_property with multiple relationships queryable, I have solved the core issue of representing a quantity that could either be int, float (I use float instead of Decimal here because no arithmetic is done the quantity) or Fraction in the database, and making it possible to use values in queries as they would be used by the value setter. The solution was to create a quantities table with rows for a floating point value, an integer numerator, an integer denominator, and a boolean representing whether or not the stored value should be interpreted as a decimal number (as opposed to a fraction) and create an sqlalchemy.ext.hybrid.Comparator for the hybrid property which compares against Quantity._float and Quantity.is_decimal:
class Quantity(db.Model):
# ...
_denominator = db.Column(db.Integer)
_float = db.Column(db.Float)
is_decimal = db.Column(db.Boolean, default=False)
_numerator = db.Column(db.Integer)
# ...
#staticmethod
def for_cmp(val):
"""Convert val to float so it can be used to query against _float."""
if Quantity.dec_check(val): # True if val looks like a decimal number
return float(val)
elif isinstance(val, str):
frac = Quantity.str_to_fraction(val)
else:
frac = Fraction(val)
return float(frac)
#hybrid_property
def value(self):
if self._float is not None:
if self.is_decimal:
return self._float
elif self._denominator == 1:
return self._numerator
else:
return Fraction(self._numerator, self._denominator)
else:
return None
class ValueComparator(Comparator):
def operate(self, op, other):
return and_(op(Quantity._float, Quantity.for_cmp(other)),
Quantity.is_decimal == Quantity.dec_check(other))
#value.comparator
def value(cls):
return Quantity.ValueComparator(cls)
#value.setter
def value(self, val):
if val is not None:
if Quantity.dec_check(val):
self.is_decimal = True
self._float = float(val)
self._numerator = None
self._denominator = None
else:
self.is_decimal = False
if isinstance(val, str):
frac = Quantity.str_to_fraction(val)
else:
frac = Fraction(val)
self._numerator = frac.numerator
self._denominator = frac.denominator
self._float = float(frac)
else:
self.is_decimal = None
self._numerator = None
self._denominator = None
self._float = None
Since I don't have the original model I asked this question for any longer, I can't readily go back and answer this question properly, but I'd imagine it could be done using join or select.