We Keep Coding

Add a virtual field to a django query

I want to add an extra field to a query set in Django.
The field does not exist in the model but I want to add it to the query set.
Basically I want to add an extra field called "upcoming" which should return "True"
I already tried adding a #property method to my model class. This does not work because apparently django queries access the DB directly.
models.py
class upcomingActivity(models.Model):
title = models.CharField (max_length=150)
address = models.CharField (max_length=150)
Views.py
def get(self, request):
query = upcomingActivity.objects.all()
feature_collection = serialize('geojson', query ,
geometry_field='location',
fields= ( 'upcoming','title','address','pk' )
)

This answer is for the case that you do not want to add a virtual property to the model (the model remains as is).
To add an additional field to the queryset result objects:
from django.db.models import BooleanField, Value
upcomingActivity.objects.annotate(upcoming=Value(True, output_field=BooleanField())).all()
Each object in the resulting queryset will have the attribute upcoming with the boolean value True.
(Performance should be nice because this is easy work for the DB, and Django/Python does not need to do much additional work.)
EDIT after comment by Juan Carlos:
The Django serializer is for serializing model objects and thus will not serialize any non-model fields by default (because basically, the serializer in this case is for loading/dumping DB data).
See https://docs.djangoproject.com/en/2.2/topics/serialization/
Django’s serialization framework provides a mechanism for “translating” Django models into other formats.
See also: Django - How can you include annotated results in a serialized QuerySet?
From my own experience:
In most cases, we are using json.dumps for serialization in views and this works like a charm. You can prepare the data very flexibly for whatever needs arize, either by annotations or by processing the data (list comprehension etc.) before dumping it via json.
Another possibility in your situation could be to customize the serializer or the input to the serializer after fetching the data from the DB.

You can use a class function to return the upcoming value like this:
def upcoming(self):
is_upcoming = # some logic query or just basically set it to true.
return is_upcoming
then call it normally in your serializer the way you did it.
fields= ( 'upcoming','title','address','pk' )

Django: list all reverse relations of a model

I would like my django application to serve a list of any model's fields (this will help the GUI build itself).
Imagine the classes (ignore the fact that all field of Steps could be in Item, I have my reasons :-) )
class Item(models.Model):
name = models.CharField(max_length=100)
description = models.TextField()
class Steps(models.Model):
item = models.OneToOneField('Item', related_name='steps')
design = models.BooleanField(default=False)
prototype = models.BooleanField(default=False)
production = models.BooleanField(default=False)
Now, when I want to list a model's fields:
def get_fields(model):
return model._meta.fields + model._meta.many_to_many
But I would also like to get the list of "related" one-to-one foreign keys to my models. In my case Item.steps would not be in that list.
I have found that model._meta.get_all_field_names does include all the related fields.
But when I call Item._meta.get_field_by_name('steps') it returns a tuple holding a RelatedObject, which does not tell me instantly whether this is a single relation or a one-to-many (I want to list only reversed one-to-one relations).
Also, I can use this bit of code:
from django.db.models.fields.related import SingleRelatedObjectDescriptor
reversed_f_keys = [attr for attr in Item.__dict__.values() \
if isinstance(attr, SingleRelatedObjectDescriptor)]
But I'm not very satisfied with this.
Any help, idea, tips are welcome!
Cheers

This was changed (in 1.8 I think) and Olivier's answer doesn't work anymore. According to the docs, the new way is
[f for f in Item._meta.get_fields()
if f.auto_created and not f.concrete]
This includes one-to-one, many-to-one, and many-to-many.

I've found out that there are methods of Model._meta that can give me what I want.
my_model = get_model('app_name','model_name')
# Reverse foreign key relations
reverse_fks = my_model._meta.get_all_related_objects()
# Reverse M2M relations
reverse_m2ms = my_model._meta.get_all_related_many_to_many_objects()
By parsing the content of the relations, I can guess whether the "direct" field was a OneToOneField or whatever.

I was looking into this answer as a starting point to identify reversed relationships for a model instance.
So, I noticed that when you get all the fields using instance._meta.get_fields(), those that are direct relationships, which are 3 types (ForeignKey, ManyToMany, OneTone), their parent class (field.__class__.__bases__) is django.db.models.fields.related.ForeignKey.
However, those that are reverse relationships inherit from django.db.models.fields.reverse_related.ForeignObjectRel. And if you take a look at this class, it has:
auto_created = True
concrete = False
So you could identify those by the attributes mentioned in the top-rated answer or by asking isinstance(field, ForeignObjectRel.
Another thing I could notice is that those reverse relationships have a field attribute which points to the direct relationship generating that reverse relationship.
Additionally, in order to exclude the fields instantiating the through table, those have through and through_fields attributes

And what about this :
oneToOneFieldNames = [
field_name
for field_name in Item._meta.get_all_field_names()
if isinstance(
getattr(
Item._meta.get_field_by_name(field_name)[0],
'field',
None
),
models.OneToOneField
)
]
RelatedObject may have a Field attribute for relations. You just have to check if this is a OneToOne field and you can retrieve only what you want

if you are using Django Rest Framework, you could use something like that for your obj:
from rest_framework.utils import model_meta
info = model_meta.get_field_info(obj)
for field in obj.__class__.__dict__.keys():
if field in info.relations and info.relations[field].to_many and info.relations[field].reverse:
#print all reverse relations
print(field)

Altering django-filter default behaviour

This is a django-filter app specific guestion.
Has anyone tried to introduce conditions for the filters to query according to the condition?
Let me give an example:
Suppose we have a Product model. It can be filtered according to its name and price.
The default django-filter behaviour is that, as we use more filters and chain them together, they filter data using AND statements (it narrows the search).
I'd like to change this behaviour and add a ChoiceFilter, say with two options: AND as well as OR. From this point, the filter should work according to what a user have selected.
Eg. if a user query for products with name__startswith="Juice" OR price__lte=10.00, it should list all the products with names starting with Juice as well as products with price below 10.00.
Django-filter docs say that the filter can take an argument:
action
An optional callable that tells the filter how to handle the queryset. It recieves a
QuerySet and the value to filter on and should return a Queryset that is filtered
appropriately.
which seems to be what I am looking for, but the docs lacks any further explanation. Suggestions please?
#EDIT:
This is views.py:
def product_list(request):
f = ProductFilter(request.GET, queryset=Product.objects.all())
return render_to_response('my_app/template.html', {'filter': f})

Because of the way the final queryset is constructed, making each filter be ORed together is difficult. Essentially, the code works like this:
FilterSet, filterset.py line 253:
#property
def qs(self):
qs = self.queryset.all()
for filter_ in self.filters():
qs = filter_.filter(qs)
Filters, filters.py line 253:
def filter(self, qs):
return qs.filter(name=self.value)
Each filter can decide how to apply itself to the incoming queryset, and all filters, as currently implemented, filter the incoming queryset using AND. You could make a new set of filters that OR themselves to the incoming queryset, but there is no way of overriding the behaviour from the FilterSet side.

action won't cut it. This callback is used for particular filter field and only has access to that field's value.
The cleanest way would be to create multi-widget filter field, similar to RangeField. Check out the source.
So instead two date fields you use name, price and the logic type [AND|OR] as fields, this way you have access to all these values at once to use in custom queryset.
EDIT 1:
This is a little gist I wrote to show how to query two fields with selected operator.
https://gist.github.com/mariodev/6689472
Usage:
class ProductFilter(django_filters.FilterSet):
nameprice = NamePriceFilter()
class Meta:
model = Product
fields = ['nameprice']
It's actually not very flexible in terms of re-usage, but certainly can be re-factored to make it useful.

In order to make filters work with OR, you should make a subclass of FilterSet and override qs from Tim's answer like this:
#property
def qs(self):
qs = self.queryset.none()
for filter_ in self.filters():
qs |= filter_.filter(self.queryset.all())
I haven't tested this, but I think you got the idea. QuerySets support bitwise operations, so you can easily combine results of two filters with OR.

class FileFilterSet(django_filters.FilterSet):
class Meta:
model = File
fields = ['project']
def __init__(self, *args, **kwargs):
super(FileFilterSet, self).__init__(*args, **kwargs)
for name, field in self.filters.items():
if isinstance(field, ModelChoiceFilter):
field.extra['empty_label'] = None
field.extra['initial'] = Project.objects.get(pk=2)
# field.extra['queryset'] = Project.objects.filter(pk=2)
class FileFilter(FilterView):
model = File
template_name = 'files_list.html'
filterset_class = FileFilterSet

django return foreign key object from queryset?

So I have three models
class Post(....
class Project(....
# have a many to many relationship
class ProjectPost(....
post = ..... # foreignkey
project = .... # foreignkey
The data set I want to select is a list of Post objects given a Project object.
This is what I tried:
posts_list = ProjectPost.objects.filter(project=project_object).select_related("post")
But this returns a list of ProjectPost objects rather than a list of Post objects. What is the correct way of doing this?

You may want to use the ManyToManyField()
https://docs.djangoproject.com/en/dev/topics/db/examples/many_to_many/
You should do it like this:
class Post(models.Model):
pass
class Project(models.Model):
posts = models.ManyToManyField(Post)
And then, if you want to access the Posts of a Project, you can do
project_obj.posts.all()
You can use all the Queryset methods
If you want to access the projects of a posts you can do
post_obj.project_set.all()
Same as before, you can use all the Queryset methods.
If for any reason you want to do it that way, you could do:
post_list = ProjectPost.objects.filter(project=project_object).values('post')

Came across this problem myself recently and this was how I solved it. Would love it if someone could comment on whether my solution is efficient.
project_posts = ProjectPost.objects.filter(project=project_object).select_related("post")
posts_lists = map(lambda post: project.post, project_posts)
Objects in posts_lists are now of the correct type.

Tastypie Dehydrate reverse relation count

I have a simple model which includes a product and category table. The Product model has a foreign key Category.
When I make a tastypie API call that returns a list of categories /api/vi/categories/
I would like to add a field that determines the "product count" / the number of products that have a giving category. The result would be something like:
category_objects[
{
id: 53
name: Laptops
product_count: 7
},
...
]
The following code is working but the hit on my DB is heavy
def dehydrate(self, bundle):
category = Category.objects.get(pk=bundle.obj.id)
products = Product.objects.filter(category=category)
bundle.data['product_count'] = products.count()
return bundle
Is there a more efficient way to build this query? Perhaps with annotate ?

You can use prefetch_related method of QuerSet to reverse select_related.
Asper documentation,
prefetch_related(*lookups)
Returns a QuerySet that will automatically
retrieve, in a single batch, related objects for each of the specified
lookups.
This has a similar purpose to select_related, in that both are
designed to stop the deluge of database queries that is caused by
accessing related objects, but the strategy is quite different.
If you change your dehydrate function to following then database will be hit single time.
def dehydrate(self, bundle):
category = Category.objects.prefetch_related("product_set").get(pk=bundle.obj.id)
bundle.data['product_count'] = category.product_set.count()
return bundle
UPDATE 1
You should not initialize queryset inside dehydrate function. queryset should be always set in Meta class only. Please have a look at following example from django-tastypie documentation.
class MyResource(ModelResource):
class Meta:
queryset = User.objects.all()
excludes = ['email', 'password', 'is_staff', 'is_superuser']
def dehydrate(self, bundle):
# If they're requesting their own record, add in their email address.
if bundle.request.user.pk == bundle.obj.pk:
# Note that there isn't an ``email`` field on the ``Resource``.
# By this time, it doesn't matter, as the built data will no
# longer be checked against the fields on the ``Resource``.
bundle.data['email'] = bundle.obj.email
return bundle
As per official django-tastypie documentation on dehydrate() function,
dehydrate
The dehydrate method takes a now fully-populated bundle.data & make
any last alterations to it. This is useful for when a piece of data
might depend on more than one field, if you want to shove in extra
data that isn’t worth having its own field or if you want to
dynamically remove things from the data to be returned.
dehydrate() is only meant to make any last alterations to bundle.data.

Your code does additional count query for each category. You're right about annotate being helpfull in this kind of a problem.
Django will include all queryset's fields in GROUP BY statement. Notice .values() and empty .group_by() serve limiting field set to required fields.
cat_to_prod_count = dict(Product.objects
.values('category_id')
.order_by()
.annotate(product_count=Count('id'))
.values_list('category_id', 'product_count'))
The above dict object is a map [category_id -> product_count].
It can be used in dehydrate method:
bundle.data['product_count'] = cat_to_prod_count[bundle.obj.id]
If that doesn't help, try to keep similar counter on category records and use singals to keep it up to date.
Note categories are usually a tree-like beings and you probably want to keep count of all subcategories as well.
In that case look at the package django-mptt.