We Keep Coding

Conditionally adding several filters to a SQLAlchemy query without duplicating code

I have a SQLAlchemy model:
class Ticket(db.Model):
__tablename__ = 'ticket'
id = db.Column(INTEGER(unsigned=True), primary_key=True, nullable=False,
autoincrement=True)
cluster = db.Column(db.VARCHAR(128))
#classmethod
def get(cls, cluster=None):
query = db.session.query(Ticket)
if cluster is not None:
query = query.filter(Ticket.cluster==cluster)
return query.one()
If I add a new column and would like to extend the get method, I have to add one if xxx is not None like this below:
#classmethod
def get(cls, cluster=None, user=None):
query = db.session.query(Ticket)
if cluster is not None:
query = query.filter(Ticket.cluster==cluster)
if user is not None:
query = query.filter(Ticket.user==user)
return query.one()
Is there any way I could make this more efficient? If I have too many columns, the get method would become so ugly.

As always, if you don't want to write something repetitive, use a loop:
#classmethod
def get(cls, **kwargs):
query = db.session.query(Ticket)
for k, v in kwargs.items():
query = query.filter(getattr(table, k) == v)
return query.one()
Because we're no longer setting the cluster=None/user=None as defaults (but instead depending on things that weren't specified by the caller simply never being added to kwargs), we no longer need to prevent filters for null values from being added: The only way a null value will end up in the argument list is if the user actually asked to search for a value of None; so this new code is able to honor that request should it ever take place.
If you prefer to retain the calling convention where cluster and user can be passed positionally (but the user can't search for a value of None), see the initial version of this answer.

Overwrite django choices output in graphene

I'm working with graphene and graphene-django and I have a problem with a IntegerField with choices. graphene create an Enum and the output is "A_1" if the value is 1; "A_2" if the value is 2 and so on. Example:
# model
class Foo(models.Model):
score = models.IntegerField(choices=((1, 1), (2, 2), (3, 3), (4, 4), (5, 5)))
# query
query {
foo {
score
}
}
# response
{
"data": {
"foo": {
"source": "A_1"
}
}
}
I found a function that convert the choices values.
def convert_choice_name(name):
name = to_const(force_text(name))
try:
assert_valid_name(name)
except AssertionError:
name = "A_%s" % name
return name
And assert_valid_name has this regular expression:
r'^[_a-zA-Z][_a-zA-Z0-9]*$'
Therefore, whatever begins with number, it will convert it to "A_...".
How I can overwrite this output?

The code comments say
GraphQL serializes Enum values as strings, however internally Enums
can be represented by any kind of type, often integers.
So for your particular case, you're not going to be able to replace the over-the-wire values with integers easily. But it may not matter if the actual value represented by the strings ("A_1") is still an integer internally and on the client end (from the field's description values.)
In general though you can replace the automatically generated field for the field with choices by defining an enum class and adding to the definition of the DjangoObjectType. Here's an example using the documentation Enum example...
class Episode(graphene.Enum):
NEWHOPE = 4
EMPIRE = 5
JEDI = 6
#property
def description(self):
if self == Episode.NEWHOPE:
return 'New Hope Episode'
return 'Other episode'
which you could then add to your DjangoObjectType like
class FooType(DjangoObjectType):
score = Episode()
class Meta:
model = Foo
Or if you want to get extra fancy you can generate the Enum field dynamically from your field's choices in Foo._meta.get_field('score').choices. See graphene_django.converter.convert_django_field_with_choices.

You can set convert_choices_to_enum to False in your Graphene-Django model which will leave them as integers.
class FooType(DjangoObjectType):
class Meta:
model = Foo
convert_choices_to_enum = False
There is more information on the setting here.

Having just bumped into this myself, an alternate is to define your field outside of the Meta (use only_fields) as just a graphene.Int , then you can provide your own resolver function and just return the value of the field, which will end up as a number.
My code snippet (the problem field was resource_type which is the Enum):
class ResourceItem(DjangoObjectType):
class Meta:
model = Resource
only_fields = (
"id",
"title",
"description",
"icon",
"target",
"session_reveal",
"metadata",
"payload",
)
resource_type = graphene.Int()
def resolve_resource_type(self, info):
return self.resource_type

Odoo - add custom field attribute?

Is there a way to add custom field attribute in Odoo? For example every field has attribute help where you can enter message explaining the field for the user. So I want to add custom attribute, so that would change the way field acts for all types of fields.
I want to add into Field class, so all fields would get that attribute. But it seems no matter what I do, Odoo does not see that such attribute was added.
If I simply add new custom attribute like:
some_field = fields.Char(custom_att="hello")
Then it is simply ignored. And I need it to be picked up by method fields_get, which can return wanted attribute value (info what it does:
def fields_get(self, cr, user, allfields=None, context=None, write_access=True, attributes=None):
""" fields_get([fields][, attributes])
Return the definition of each field.
The returned value is a dictionary (indiced by field name) of
dictionaries. The _inherits'd fields are included. The string, help,
and selection (if present) attributes are translated.
:param allfields: list of fields to document, all if empty or not provided
:param attributes: list of description attributes to return for each field, all if empty or not provided
"""
So calling it, does not return my custom attribute (it does return the ones originally defined by Odoo though).
I also tried updating _slots (with monkey patch or just testing by changing source code) attribute in Field class, but it seems it is not enough. Because my attribute is still ignored.
from openerp import fields
original_slots = fields.Field._slots
_slots = original_slots
_slots['custom_att'] = None
fields.Field._slots = _slots
Does anyone know how to properly add new custom attribute for field?

Assuming v9
The result of fields_get is a summary of fields defined on a model, the code shows that it will only add the attribute if the description was filled. It will fetch the description of the current field by calling field.get_description
So in order to ensure that your attribute gets inserted into this self.description_attrs you will need to add an attribute or method that starts with _description_customatt (customatt part from your example) and will return the required data.
I've not run any tests for this but you can look at the code for the fields and their attributes what they actually return. For instance the help attribute description (src)
def _description_help(self, env):
if self.help and env.lang:
model_name = self.base_field.model_name
field_help = env['ir.translation'].get_field_help(model_name)
return field_help.get(self.name) or self.help
return self.help

This is only something you can do if you run OpenERP/ODOO on your own server (in other words, not the cloud version whose code you cannot access).
You will need to modify the <base>/osv/fields.py file and add your changes to the field_to_dict function towards the bottom of the file (the base _column class already saves extra keyword arguments for you -- at least in version 7.0):
def field_to_dict(model, cr, user, field, context=None):
res = {'type': field._type}
...
...
for arg in ('string', 'readonly', ...) :
Somewhere in that long list of attributes you need to insert the name of the one you are interested in.
Alternatively, you could update _column.__init__ to save the names of the extra arguments, and field_to_dict to include them (untested):
diff -r a30d30db3cd9 osv/fields.py
--- a/osv/fields.py Thu Jun 09 17:18:29 2016 -0700
+++ b/osv/fields.py Mon Jun 13 18:11:26 2016 -0700
## -116,23 +116,24 ## class _column(object):
self._context = context
self.write = False
self.read = False
self.view_load = 0
self.select = select
self.manual = manual
self.selectable = True
self.group_operator = args.get('group_operator', False)
self.groups = False # CSV list of ext IDs of groups that can access this field
self.deprecated = False # Optional deprecation warning
- for a in args:
- if args[a]:
- setattr(self, a, args[a])
+ self._user_args = ()
+ for name, value in args:
+ setattr(self, name, value or False)
+ self._user_args += name
def restart(self):
pass
def set(self, cr, obj, id, name, value, user=None, context=None):
cr.execute('update '+obj._table+' set '+name+'='+self._symbol_set[0]+' where id=%s', (self._symbol_set[1](value), id))
def get(self, cr, obj, ids, name, user=None, offset=0, context=None, values=None):
raise Exception(_('undefined get method !'))
## -1559,20 +1560,22 ## def field_to_dict(model, cr, user, field
res['o2m_order'] = field._order or False
if isinstance(field, many2many):
(table, col1, col2) = field._sql_names(model)
res['m2m_join_columns'] = [col1, col2]
res['m2m_join_table'] = table
for arg in ('string', 'readonly', 'states', 'size', 'group_operator', 'required',
'change_default', 'translate', 'help', 'select', 'selectable', 'groups',
'deprecated', 'digits', 'invisible', 'filters'):
if getattr(field, arg, None):
res[arg] = getattr(field, arg)
+ for arg in field._user_args:
+ res[arg] = getattr(field, arg)
if hasattr(field, 'selection'):
if isinstance(field.selection, (tuple, list)):
res['selection'] = field.selection
else:
# call the 'dynamic selection' function
res['selection'] = field.selection(model, cr, user, context)
if res['type'] in ('one2many', 'many2many', 'many2one'):
res['relation'] = field._obj
res['domain'] = field._domain(model) if callable(field._domain) else field._domain

Combine several filters into one filter() with django-filters

I'm using django-filter app. There is however one problem I do not know how to solve. It's almost exactly the same thing as is described in django documentation:
https://docs.djangoproject.com/en/1.2/topics/db/queries/#spanning-multi-valued-relationships
I want to make a query where I select all Blogs that has an entry with both "Lennon" in headline and was published in 2008, eg.:
Blog.objects.filter(entry__headline__contains='Lennon',
entry__pub_date__year=2008)
Not to select Blogs that has an entry with "Lennon" in headline and another entry (possibly the same) that was published in 2008:
Blog.objects.filter(entry__headline__contains='Lennon').filter(
entry__pub_date__year=2008)
However, if I set up Filter such that there are two fields (nevermind __contains x __exact, just an example):
class BlogFilter(django_filters.FilterSet):
entry__headline = django_filters.CharFilter()
entry__pub_date = django_filters.CharFilter()
class Meta:
model = Blog
fields = ['entry__headline', 'entry__pub_date', ]
django-filter will generete the latter:
Blog.objects.filter(entry__headline__exact='Lennon').filter(
entry__pub_date__exact=2008)
Is there a way to combine both filters into a single filter field?

Well, I came with a solution. It is not possible to do using the regular django-filters, so I extended it a bit. Could've been improved, this is quick-n-dirty solution.
1st added a custom "grouped" field to django_filters.Filter and a filter_grouped method (almost copy of filter method)
class Filter(object):
def __init__(self, name=None, label=None, widget=None, action=None,
lookup_type='exact', required=False, grouped=False, **kwargs):
(...)
self.grouped = grouped
def filter_grouped(self, qs, value):
if isinstance(value, (list, tuple)):
lookup = str(value[1])
if not lookup:
lookup = 'exact' # we fallback to exact if no choice for lookup is provided
value = value[0]
else:
lookup = self.lookup_type
if value:
return {'%s__%s' % (self.name, lookup): value}
return {}
the only difference is that instead of creating a filter on query set, it returns a dictionary.
2nd updated BaseFilterSet qs method/property:
class BaseFilterSet(object):
(...)
#property
def qs(self):
if not hasattr(self, '_qs'):
qs = self.queryset.all()
grouped_dict = {}
for name, filter_ in self.filters.iteritems():
try:
if self.is_bound:
data = self.form[name].data
else:
data = self.form.initial.get(name, self.form[name].field.initial)
val = self.form.fields[name].clean(data)
if filter_.grouped:
grouped_dict.update(filter_.filter_grouped(qs, val))
else:
qs = filter_.filter(qs, val)
except forms.ValidationError:
pass
if grouped_dict:
qs = qs.filter(**grouped_dict)
(...)
return self._qs
The trick is to store all "grouped" filters in a dictionary and then use them all as a single filter.
The filter will look something like this then:
class BlogFilter(django_filters.FilterSet):
entry__headline = django_filters.CharFilter(grouped=True)
entry__pub_date = django_filters.CharFilter(grouped=True)
class Meta:
model = Blog
fields = ['entry__headline', 'entry__pub_date', ]

Set Django IntegerField by choices=... name

When you have a model field with a choices option you tend to have some magic values associated with human readable names. Is there in Django a convenient way to set these fields by the human readable name instead of the value?
Consider this model:
class Thing(models.Model):
PRIORITIES = (
(0, 'Low'),
(1, 'Normal'),
(2, 'High'),
)
priority = models.IntegerField(default=0, choices=PRIORITIES)
At some point we have a Thing instance and we want to set its priority. Obviously you could do,
thing.priority = 1
But that forces you to memorize the Value-Name mapping of PRIORITIES. This doesn't work:
thing.priority = 'Normal' # Throws ValueError on .save()
Currently I have this silly workaround:
thing.priority = dict((key,value) for (value,key) in Thing.PRIORITIES)['Normal']
but that's clunky. Given how common this scenario could be I was wondering if anyone had a better solution. Is there some field method for setting fields by choice name which I totally overlooked?

Do as seen here. Then you can use a word that represents the proper integer.
Like so:
LOW = 0
NORMAL = 1
HIGH = 2
STATUS_CHOICES = (
(LOW, 'Low'),
(NORMAL, 'Normal'),
(HIGH, 'High'),
)
Then they are still integers in the DB.
Usage would be thing.priority = Thing.NORMAL

As of Django 3.0, you can use:
class ThingPriority(models.IntegerChoices):
LOW = 0, 'Low'
NORMAL = 1, 'Normal'
HIGH = 2, 'High'
class Thing(models.Model):
priority = models.IntegerField(default=ThingPriority.LOW, choices=ThingPriority.choices)
# then in your code
thing = get_my_thing()
thing.priority = ThingPriority.HIGH

I'd probably set up the reverse-lookup dict once and for all, but if I hadn't I'd just use:
thing.priority = next(value for value, name in Thing.PRIORITIES
if name=='Normal')
which seems simpler than building the dict on the fly just to toss it away again;-).

Here's a field type I wrote a few minutes ago that I think does what you want. Its constructor requires an argument 'choices', which may be either a tuple of 2-tuples in the same format as the choices option to IntegerField, or instead a simple list of names (ie ChoiceField(('Low', 'Normal', 'High'), default='Low') ). The class takes care of the mapping from string to int for you, you never see the int.
class ChoiceField(models.IntegerField):
def __init__(self, choices, **kwargs):
if not hasattr(choices[0],'__iter__'):
choices = zip(range(len(choices)), choices)
self.val2choice = dict(choices)
self.choice2val = dict((v,k) for k,v in choices)
kwargs['choices'] = choices
super(models.IntegerField, self).__init__(**kwargs)
def to_python(self, value):
return self.val2choice[value]
def get_db_prep_value(self, choice):
return self.choice2val[choice]

Model's choices option accepts a sequence consisting itself of iterables of exactly two items (e.g. [(A, B), (A, B) ...]) to use as choices for this field.
In addition, Django provides enumeration types that you can subclass to define choices in a concise way:
from django.utils.translation import gettext_lazy as _
class ThingPriority(models.IntegerChoices):
LOW = 0, _('Low')
NORMAL = 1, _('Normal')
HIGH = 2, _('High')
class Thing(models.Model):
priority = models.IntegerField(default=ThingPriority.NORMAL, choices=ThingPriority.choices)
Django supports adding an extra string value to the end of this tuple to be used as the human-readable name, or label. The label can be a lazy translatable string.
# in your code
thing = get_thing() # instance of Thing
thing.priority = ThingPriority.LOW
Note: you can use that using ThingPriority.HIGH, ThingPriority.['HIGH'], or ThingPriority(0) to access or lookup enum members.
You need to import from django.utils.translation import gettext_lazy as _

I appreciate the constant defining way but I believe Enum type is far best for this task. They can represent integer and a string for an item in the same time, while keeping your code more readable.
Enums were introduced to Python in version 3.4. If you are using any lower (such as v2.x) you can still have it by installing the backported package: pip install enum34.
# myapp/fields.py
from enum import Enum
class ChoiceEnum(Enum):
#classmethod
def choices(cls):
choices = list()
# Loop thru defined enums
for item in cls:
choices.append((item.value, item.name))
# return as tuple
return tuple(choices)
def __str__(self):
return self.name
def __int__(self):
return self.value
class Language(ChoiceEnum):
Python = 1
Ruby = 2
Java = 3
PHP = 4
Cpp = 5
# Uh oh
Language.Cpp._name_ = 'C++'
This is pretty much all. You can inherit the ChoiceEnum to create your own definitions and use them in a model definition like:
from django.db import models
from myapp.fields import Language
class MyModel(models.Model):
language = models.IntegerField(choices=Language.choices(), default=int(Language.Python))
# ...
Querying is icing on the cake as you may guess:
MyModel.objects.filter(language=int(Language.Ruby))
# or if you don't prefer `__int__` method..
MyModel.objects.filter(language=Language.Ruby.value)
Representing them in string is also made easy:
# Get the enum item
lang = Language(some_instance.language)
print(str(lang))
# or if you don't prefer `__str__` method..
print(lang.name)
# Same as get_FOO_display
lang.name == some_instance.get_language_display()

class Sequence(object):
def __init__(self, func, *opts):
keys = func(len(opts))
self.attrs = dict(zip([t[0] for t in opts], keys))
self.choices = zip(keys, [t[1] for t in opts])
self.labels = dict(self.choices)
def __getattr__(self, a):
return self.attrs[a]
def __getitem__(self, k):
return self.labels[k]
def __len__(self):
return len(self.choices)
def __iter__(self):
return iter(self.choices)
def __deepcopy__(self, memo):
return self
class Enum(Sequence):
def __init__(self, *opts):
return super(Enum, self).__init__(range, *opts)
class Flags(Sequence):
def __init__(self, *opts):
return super(Flags, self).__init__(lambda l: [1<<i for i in xrange(l)], *opts)
Use it like this:
Priorities = Enum(
('LOW', 'Low'),
('NORMAL', 'Normal'),
('HIGH', 'High')
)
priority = models.IntegerField(default=Priorities.LOW, choices=Priorities)

Simply replace your numbers with the human readable values you would like. As such:
PRIORITIES = (
('LOW', 'Low'),
('NORMAL', 'Normal'),
('HIGH', 'High'),
)
This makes it human readable, however, you'd have to define your own ordering.

My answer is very late and might seem obvious to nowadays-Django experts, but to whoever lands here, i recently discovered a very elegant solution brought by django-model-utils: https://django-model-utils.readthedocs.io/en/latest/utilities.html#choices
This package allows you to define Choices with three-tuples where:
The first item is the database value
The second item is a code-readable value
The third item is a human-readable value
So here's what you can do:
from model_utils import Choices
class Thing(models.Model):
PRIORITIES = Choices(
(0, 'low', 'Low'),
(1, 'normal', 'Normal'),
(2, 'high', 'High'),
)
priority = models.IntegerField(default=PRIORITIES.normal, choices=PRIORITIES)
thing.priority = getattr(Thing.PRIORITIES.Normal)
This way:
You can use your human-readable value to actually choose the value of your field (in my case, it's useful because i'm scraping wild content and storing it in a normalized way)
A clean value is stored in your database
You have nothing non-DRY to do ;)
Enjoy :)

Originally I used a modified version of #Allan's answer:
from enum import IntEnum, EnumMeta
class IntegerChoiceField(models.IntegerField):
def __init__(self, choices, **kwargs):
if hasattr(choices, '__iter__') and isinstance(choices, EnumMeta):
choices = list(zip(range(1, len(choices) + 1), [member.name for member in list(choices)]))
kwargs['choices'] = choices
super(models.IntegerField, self).__init__(**kwargs)
def to_python(self, value):
return self.choices(value)
def get_db_prep_value(self, choice):
return self.choices[choice]
models.IntegerChoiceField = IntegerChoiceField
GEAR = IntEnum('GEAR', 'HEAD BODY FEET HANDS SHIELD NECK UNKNOWN')
class Gear(Item, models.Model):
# Safe to assume last element is largest value member of an enum?
#type = models.IntegerChoiceField(GEAR, default=list(GEAR)[-1].name)
largest_member = GEAR(max([member.value for member in list(GEAR)]))
type = models.IntegerChoiceField(GEAR, default=largest_member)
def __init__(self, *args, **kwargs):
super(Gear, self).__init__(*args, **kwargs)
for member in GEAR:
setattr(self, member.name, member.value)
print(Gear().HEAD, (Gear().HEAD == GEAR.HEAD.value))
Simplified with the django-enumfields package package which I now use:
from enumfields import EnumIntegerField, IntEnum
GEAR = IntEnum('GEAR', 'HEAD BODY FEET HANDS SHIELD NECK UNKNOWN')
class Gear(Item, models.Model):
# Safe to assume last element is largest value member of an enum?
type = EnumIntegerField(GEAR, default=list(GEAR)[-1])
#largest_member = GEAR(max([member.value for member in list(GEAR)]))
#type = EnumIntegerField(GEAR, default=largest_member)
def __init__(self, *args, **kwargs):
super(Gear, self).__init__(*args, **kwargs)
for member in GEAR:
setattr(self, member.name, member.value)