How do you create custom field lookups in Django?
When filtering querysets, django provides a set of lookups that you can use: __contains, __iexact, __in, and so forth. I want to be able to provide a new lookup for my manager, so for instance, someone could say:
twentysomethings = Person.objects.filter(age__within5=25)
and get back all the Person objects with an age between 20 and 30. Do I need to subclass the QuerySet or Manager class to do this? How would it be implemented?
A more flexible way to do this is to write a custom QuerySet as well as a custom manager. Working from ozan's code:
class PersonQuerySet(models.query.QuerySet):
def in_age_range(self, min, max):
return self.filter(age__gte=min, age__lt=max)
class PersonManager(models.Manager):
def get_query_set(self):
return PersonQuerySet(self.model)
def __getattr__(self, name):
return getattr(self.get_query_set(), name)
class Person(models.Model):
age = #...
objects = PersonManager()
This allows you to chain your custom query. So both these queries would be valid:
Person.objects.in_age_range(20,30)
Person.objects.exclude(somefield = some_value).in_age_range(20, 30)
Rather than creating a field lookup, best practice would be to create a manager method, that might look a little bit like this:
class PersonManger(models.Manager):
def in_age_range(self, min, max):
return self.filter(age__gte=min, age__lt=max)
class Person(models.Model):
age = #...
objects = PersonManager()
then usage would be like so:
twentysomethings = Person.objects.in_age_range(20, 30)
First, let me say that there is no Django machinery in place that's meant to publicly facilitate what you'd like.
(Edit - actually since Django 1.7 there is: https://docs.djangoproject.com/en/1.7/howto/custom-lookups/ )
That said, if you really want to accomplish this, subclass QuerySet and override the _filter_or_exclude() method. Then create a custom manager that only returns your custom QuerySet (or monkey-patch Django's QuerySet, yuck). We do this in neo4django to reuse as much of the Django ORM queryset code as possible while building Neo4j-specific Query objects.
Try something (roughly) like this, adapted from Zach's answer. I've left actual error handling for the field lookup parsing as an exercise for the reader :)
class PersonQuerySet(models.query.QuerySet):
def _filter_or_exclude(self, negate, *args, **kwargs):
cust_lookups = filter(lambda s: s[0].endswith('__within5'), kwargs.items())
for lookup in cust_lookups:
kwargs.pop(lookup[0])
lookup_prefix = lookup[0].rsplit('__',1)[0]
kwargs.update({lookup_prefix + '__gte':lookup[1]-5,
lookup_prefix + '__lt':lookup[1]+5})
return super(PersonQuerySet, self)._filter_or_exclude(negate, *args, **kwargs)
class PersonManager(models.Manager):
def get_query_set(self):
return PersonQuerySet(self.model)
class Person(models.Model):
age = #...
objects = PersonManager()
Final remarks - clearly, if you want to chain custom field lookups, this is going to get pretty hairy. Also, I'd normally write this a bit more functionally and use itertools for performance, but thought it was more clear to leave it out. Have fun!
As of Django 1.7, there is a simple way to implement it. Your example is actually very similar to the one from the documentation:
from django.db.models import Lookup
class AbsoluteValueLessThan(Lookup):
lookup_name = 'lt'
def as_sql(self, qn, connection):
lhs, lhs_params = qn.compile(self.lhs.lhs)
rhs, rhs_params = self.process_rhs(qn, connection)
params = lhs_params + rhs_params + lhs_params + rhs_params
return '%s < %s AND %s > -%s' % (lhs, rhs, lhs, rhs), params
AbsoluteValue.register_lookup(AbsoluteValueLessThan)
While registering, you can just use Field.register_lookup(AbsoluteValueLessThan) instead.
Related
How can I override a mongoengine's queryset method?
Specifically, I want to override .order_by(), but the closest I can get is to add another method ordered that would conditionally call .order_by():
class TransactionQuerySet(QuerySet):
def ordered(self, *args, target_data=None, **kwargs):
if target_data is None:
result = self.order_by(*args, **kwargs)
return result
else:
return 'transactions sorted by target data'
Ideally I would like this new method to be named the same as mongoengine's method - order_by - but if I do this I will exceed recursion depth when the queryset manager is called like Transaction.objects.order_by.
How can I do this?
To avoid recursion, you should call order_by method of the parent class. TransactionQuerySet should look like the following.
class TransactionQuerySet(QuerySet):
def order_by(self, *args, target_data=None):
if target_data is None:
return super().order_by(*args)
return "transactions sorted by target data"
Now if you call order_by on TransactionQuerySet object it won't fall in recursion.
class Transaction(Document):
title = StringField(max_length=100, required=True)
last_modified = DateTimeField(default=datetime.utcnow)
def __str__(self):
return self.title
meta = {"collection": "transaction_collection"}
with MongoClient("mongodb://localhost:27017") as client:
connection = client.get_database("transaction")
collection = connection.transaction_collection
transaction_set = TransactionQuerySet(Transaction, collection)
print(transaction_set.order_by("-title"))
OUTPUT
[<Transaction: ETHUSTD>, <Transaction: BTCUSTD>, ...]
I'm adding some extended functionallity to my models in Django. In order to not overload the model root interface (and file), i'm using some helpers as attributes of the model.
What i want is to group the methods and properties into this helpers, and what i want is something like:
class Product(models.Model):
downloads = DownloadsHelper()
# ....
pass
p = Product.objects.first()
p.downloads.files_count(p)
p.downloads.reset_permissions(p)
# ...
In order to not to have to pass the instance to the helper each time, I could use another approach.
class Product(models.Model):
def __init__(self, *args, **kwargs):
super(Product, self).__init__(*args, **kwargs)
self.downloads = DownloadsHelper(self)
self.shipping = ShippingHelper(self)
p = Product.objects.first()
p.downloads.files_count
p.downloads.reset_permissions()
And finally, a more python-generic/conceptual way to do this stuff would be like:
class Helper:
def __init__(self, helped):
self.helped = helped
class Helper1(Helper):
attribute_name = 'helloing'
def hola(self): print("hola")
class Helper2(Helper):
attribute_name = 'goodbying'
def chau(self): print("chau")
class Helped:
def __init__(self):
self._install_helpers()
def _install_helpers(self):
for helper in self.helpers:
setattr(self, helper.attribute_name, helper(self))
class MyHelped(Helped):
helpers = [Helper1, Helper2]
h = MyHelped()
h.helloing.hola()
h.goodbying.chau()
And the question is: Is this last approach a correct way /good practice to do the stuff from a pythonic-OOP and "Djangoid" point of view. Has this any problem?
Thanks for reading!
Maybe you did not show the full complexity for your methods, but from what you showed it could be simplified a bit.
You could use "direct/simple" inheritance, and if you have a lot of methods then for organization you could maybe prefix the method names with h1_ and h2_ to indicate which helper class they are from:
from django.db import models
class Helper1:
def h1_hola(self):
print("hola")
class Helper2:
def h2_chau(self):
print("chau")
class MyHelped(Helper1, Helper2, models.Model):
pass
h = MyHelped()
h.h1_hola()
h.h2_chau()
Would this simpler approach be sufficient for your specific needs?
I'm trying to initialize an objects field with a class that needs to know the type that is using it:
class Device(Model):
objects = AbstractManager(Device)
# the rest of the class here
This is how AbstractManager is defined:
class AbstractManager:
def __init__(self, cls: type):
self.cls = cls
def all(self):
result = []
for cls in self._get_subclasses():
result.extend(list(cls.objects.all()))
return result
def _get_subclasses(self):
return self.cls.__subclasses__()
So I can later call this and returns all() from all subclasses:
Device.objects.all()
The issue here is that I cannot use Device while initializing Device.objects, since Device is still not initialized.
As a work-around I'm initializing this outside of the class, but there's gotta be a better way:
class Device(Model):
objects = None
# the rest of the class here
Device.objects = AbstractManager(Device)
PD: I have a C#/C++ background, so maybe I'm thinking too much about this in a static-typing mindset, can't tell
You don't need to add any additional logic for this. Django allows you to access model class from manager using self.model attribute:
def _get_subclasses(self):
return self.model.__subclasses__()
You do not have to do that. Django will automatically call the contribute_to_class method, where it will pass the model, and for a manager, it will be stored in self.model. You can thus simply implement this as:
from django.db.models.manager import ManagerDescriptor
class AbstractManager(models.Manager):
def all(self):
result = []
for cls in self._get_subclasses():
result.extend(list(cls.objects.all()))
return result
def contribute_to_class(self, model, name):
self.name = self.name or name
self.model = model
setattr(model, name, AbstractManagerDescriptor(self))
model._meta.add_manager(self)
def _get_subclasses(self):
return self.model.__subclasses__()
class AbstractManagerDescriptor(ManagerDescriptor):
def __get__(self, instance, cls=None):
if instance is not None:
raise AttributeError("Manager isn't accessible via %s instances" % cls.__name__)
if cls._meta.swapped:
raise AttributeError(
"Manager isn't available; '%s.%s' has been swapped for '%s'" % (
cls._meta.app_label,
cls._meta.object_name,
cls._meta.swapped,
)
)
return cls._meta.managers_map[self.manager.name]
and add the manager as:
class Device(models.Model):
objects = AbstractManager()
That being said, I'm not sure that this is a good idea for two reasons:
you are returning a list, and normally .all() returns a QuerySet, you thus here "destroy" the laziness of the queryset, which can result in expensive querying; and
if one would use Device.objects.filter() for example, it would simply circumvent.
You might want to subclass the queryset, and then aim to implement that differently.
I am trying to create use the filter method with django-filters
See an example of the models below:
class Chicken(TimeStampedModel):
eggs = ForeignKey(Egg)
class Egg(TimeStampedModel):
hatched = BooleanField(default=False)
See an example of my current set up for the filter:
class ChickenFilter(FilterSet):
eggs__contains = ModelChoiceFilter(name="eggs", method='hatched_eggs', queryset=Eggs.objects.all())
def hatched_eggs(self, queryset, name, value):
print "We got eggs"
return queryset.filter(eggs__hatched=True)
The problem is that the method doesnt even print We got eggs when I hit the url. It just returns an empty queryset.
I did it like below:
In my urls I sent ?ids=1,2,3,4
class MyFilter(filters.FilterSet):
ids = django_filters.CharFilter(method='ids__in')
def ids__in(self, queryset, value, *args, **kwargs):
try:
if args:
ids = args[0].split(',')
ids = [int(_id) for _id in ids]
queryset = queryset.filter(id__in=ids)
except ValueError:
pass
return queryset
I was going through the same problem. My method was not being called.
So came to conclusion, I can make a workaround using:
Using custom methods in filter with django-rest-framework
class ChickenFilter(FilterSet):
eggs__contains = CharFilter(action='hatched_eggs')
def hatched_eggs(queryset, value):
print "We got eggs"
if value:
return queryset.filter(eggs__hatched=True)
return queryset
I have two related models:
class FirstModel(models.Model):
base_value = models.FloatField()
class SecondModel(models.Model):
parent = models.ForeignKey(FirstModel)
#property
def parent_value(self):
return self.parent.base_value
#property
def calculate(self):
return self.parent_value + 1
In general, SecondModel.calculate is mostly used in the context of its related FirstModel. However, I sometimes want to be able to call calculate with a temporary value as its parent_value. Something like this:
foo = SecondModel()
# would look in the database for the related FirstModel and add 1 to its base_value
foo.calculate
foo.parent_value = 10
foo.calculate # should return 11
Obviously you can't do this because the parent_value is a read-only property. I also have many different models similar to SecondModel that needs to have this kind of capability.
I've thought about and tried several things, but none have quite seemed to work:
1) Writing a Django proxy model - possible, but the number of objects is rather high, so I'd be writing a lot of similar code. Also, there appears to be a bug related to overriding properties: https://code.djangoproject.com/ticket/16176. But it'd look like this:
class ModelProxy(SecondModel):
class Meta:
proxy = True
def __init__(self, temp_value):
self.parent_value = temp_value
2) Overloading the parent_value property on the instance - like this:
foo = SecondModel()
setattr(foo, 'parent_value', 10)
but you can't do this because properties are members of the class, not the instance. And I only want the temporary value to be set for the instance
3) Metaclass or class generator? - Seems overly complicated. Also, I am uncertain what would happen if I used a metaclass to dynamically generate classes that are children of models.Model. Would I run into problems with the db tables not being in sync?
4) Rewriting the properties with proper getters and setters? - maybe the solution is to rewrite SecondModel so that the property can be set?
Any suggestions?
I believe a mixin would achieve what you want to do, and provide a simple and reusable way of supporting temporary values in your calculations. By mixing the below example into each model you want this behaviour on you can then:
Set a temporary parent value on each model
When calculate is called, it will check whether there is a property parent_value available, and if not it will use the temporary parent value in the calculation.
The code below should achieve what you are looking for - apologies I haven't been able to test it yet but it should be about right - please let me know if any problems that need editing.
class CalculateMixin(object):
#property
def temp_parent_value(self):
return self._temp_parent_value
#temp_parent_value.setter
def temp_parent_value(self, value):
self._temp_parent_value = value
#property
def calculate(self):
parent_value = self.parent_value if self.parent_value else self.temp_parent_value
return parent_value + 1
class SecondModel(models.Model, CalculateMixin):
parent = models.ForeignKey(FirstModel)
self.temp_parent_value = 'Whatever value you desire'
#property
def parent_value(self):
return self.parent.base_value
You can use the property setter:
class SecondModel(models.Model):
_base_value = None
parent = models.ForeignKey(FirstModel)
#property
def parent_value(self):
if self._base_value is None:
return self.parent.base_value
else:
return self._base_value
#parent_value.setter
def parent_value(self, value):
self._base_value = value
#property
def calculate(self):
return self.parent_value + 1
I think you can do what you need to using the mixin PropertyOverrideMixin shown below which, if some property value isn't available, then it will look for the same property prefixed with temp_. This will allow you to provide temporary values that can be used when the real property values can't be looked up.
Below is the mixin, some example models and a unit test to show how this can work. Hopefully this can be adapted for your problem! Finally it is worth mentioning that the properties here can be interchanged with normal object attributes and it should still all work.
from unittest import TestCase
class PropertyOverrideMixin(object):
def __getattribute__(self, name):
"""
Override that, if an attribute isn't found on the object, then it instead
looks for the same attribute prefixed with 'temp_' and tries to return
that value.
"""
try:
return object.__getattribute__(self, name)
except AttributeError:
temp_name = 'temp_{0}'.format(name)
return object.__getattribute__(self, temp_name)
class ParentModel(object):
attribute_1 = 'parent value 1'
class Model(PropertyOverrideMixin):
# Set our temporary property values
#property
def temp_attribute_1(self):
return 'temporary value 1'
#property
def temp_attribute_2(self):
return 'temporary value 2'
# Attribute 1 looks up value on its parent
#property
def attribute_1(self):
return self.parent.attribute_1
# Attribute 2 looks up a value on this object
#property
def attribute_2(self):
return self.some_other_attribute
class PropertyOverrideMixinTest(TestCase):
def test_attributes(self):
model = Model()
# Looking up attributes 1 and 2 returns the temp versions at first
self.assertEquals('temporary value 1', model.attribute_1)
self.assertEquals('temporary value 2', model.attribute_2)
# Now we set the parent, and lookup of attribute 1 works on the parent
model.parent = ParentModel()
self.assertEquals('parent value 1', model.attribute_1)
# now we set attribute_2, so this gets returned and the temporary ignored
model.some_other_attribute = 'value 2'
self.assertEquals('value 2', model.attribute_2)