We Keep Coding

How to make Django values() exclude some fields?

I have multiple models which has fields like "created_at" "updated_at" which I don't want to get with objects.values().
Does Django has any way to exclude fields in values()?
I know people refer to defer(), but it doesn't return QuerySet<Dict> like values() instead returns QuerySet<Model>.
I tried objects.defer("created_at", "updated_at").values(), but it includes those 2 deferred fields in the resulting Dict.
I see defer().query only selecting the non-exluded fields in the SQL, but using defer(..).values() resets the deferred fields and selects all fields.
I cannot specify which field I want, since different model has different fields, I can only specity which fields I don't want. So I cannot use values('name', 'age', ...)
I'm planning to use a CustomeManager, which I can use in all model.
Example:
class CustomManager(models.Manager):
def values_excluded(self):
return self.values() # somehow exlude the fields and return QuerySet<Dict>
class ExampleModel(models.Model):
name = models.CharField(max_length=20)
age = models.IntegerField()
created_at = models.DateTimeField()
updated_at = models.DateTimeField()
objects = CustomManager()
ExampleModel.objects.values_excluded()
Is there any way in Django or do I have to manually delete those keys from the resulting Dict from values()?

esclude_fields = ['created_at', 'updated_at']
keys = [f.name for f in Model._meta.local_fields if f.name not in esclude_fields]
queryset.values(*keys)

This should work:
class CustomManager(models.Manager):
def values_excluded(self, *excluded_fields):
included_fields = [f.name for f in self.model._meta.fields if f.name not in excluded_fields]
return self.values(*included_fields)

Annotating a Subquery with queryset filtering methods from another model through Many to Many fields

I'm not sure how to make this possible, but I'm hoping to understand the intended method to do the following:
I have a simple model:
class Author(models.Model):
id = models.TextField(primary_key=True, default=uuid4)
name = models.TextField()
main_titles = models.ManyToManyField(
"Book",
through="BookMainAuthor",
related_name="main_authors",
)
objects = AuthorManager.from_queryset(AuthorQuerySet)()
class Book(models.Model):
id = models.TextField(primary_key=True, default=uuid4)
title = models.TextField()
genre = models.ForeignKey("Genre", on_delete=models.CASCADE)
objects = BookManager.from_queryset(BookQuerySet)()
class BookQuerySet(QuerySet):
def by_genre_scifi(self) -> QuerySet:
return self.filter(**self.LOOKUP_POPULAR_SCIFI)
I'd like to add a new QuerySet method for AuthorQuerySet to annotate Author objects using the method from BookQuerySet above. I've tried the following, which is incorrect:
class AuthorQuerySet(QuerySet):
def annotate_with_total_titles_by_genres(self) -> QuerySet:
main_titles_by_genre_sci_fi_query = Book.objects.filter(main_authors__in=[OuterRef('pk')]).all()
.by_genre_sci_fi()
.annotate(cnt=Count('*'))
.values('cnt')[:1]
return self.annotate(sci_fi_titles_total =
Subquery(main_titles_by_genre_sci_fi_query, output_field=IntegerField()))
Intended usage:
annotated_authors = Author.objects.filter(<some filter>).annotate_with_total_titles_by_genres()
There are additional fields in the lookup not shown in the model above, but the method here is working, and returns a BookQuerySet filtered by the lookup:
Book.objects.filter(main_authors__in=['some_author_id']).all().by_genre_sci_fi()
Similarly, I can run the subquery independently and get the count like so:
`Book.objects.filter(main_authors__in=['some_author_id']).all()
.by_genre_sci_fi()
.annotate(cnt=Count('*'))
.values('cnt')[:1]`
Out[1]: <BookQuerySet [{'cnt': 1}]>
However when I try to annotate using the AuthorQuerySet method above, I get None for every entry.
I wonder if there is an issue here with OuterRef and using in which will evaluate each character independently if it receives a string. If I try running it without the square parens:
ProgrammingError: syntax error at or near ""bookshop_author"" LINE 1: ...RE (U0."deleted_at" IS NULL AND U1."author_id" IN "bookshop_...

Find parent data using Django ORM

I have 3 tables
AllTests
ReportFormat
Parameter
AllTests has reportFormat as ForeignKey with one to one relation.
class AllTests(models.Model):
id=models.AutoField(primary_key=True)
name=models.CharField(max_length=200)
reportFormat=models.ForeignKey(ReportFormat)
.....
.....
class Meta:
db_table = 'AllTests'
Parameter table has reportFormat as ForeignKey with one too many relations. Means one report format has many parameters.
class Parameter(models.Model):
id=models.AutoField(primary_key=True)
name=models.CharField(max_length=200)
reportFormat=models.ForeignKey(ReportFormat)
.....
.....
class Meta:
db_table = 'AllTests
ReportFormat Table:-
class ReportFormat(models.Model):
id=models.AutoField(primary_key=True)
.....
.....
class Meta:
db_table = 'ReportFormat'
I want to make a query on the parameter model that return parameter data with the related test data. Please suggest me a better way for the same.
my current query is like this.
from django.db.models import (
Sum, Value, Count, OuterRef, Subquery
)
data = Parameter.objects.filter(isDisable=0).values('id', 'name', 'reportFormat_id').annotate(
test=Subquery(Alltsets.objects.filter(reportFormat_id=OuterRef('reportFormat_id').values('id', 'name')))
)

Since, report format column of both the tables refer to the same column of reportformat table, maybe you can directly relate them with something like below.
select * from parameter inner join alltests on parameter.reportformat=alltests.reportformat
Maybe in ORM something like this?
Parameter.objects.filter().annotate(alltests_id=F('reportformat__alltests__id'),alltests_name=F('reportformat__alltests__name'),....other fields)

Fetch all Parameter and AllTests objects. Find the parameter's alltests object using reportFormat field. Add the corresponding data to the parameter dict.
Solution:
parameter_qs = Parameter.objects.all().select_related('reportFormat')
alltests_qs = AllTests.objects.all().select_related('reportFormat')
data = []
for parameter in parameter_qs:
item = {
'id': parameter.id,
'name': parameter.name,
'alltests': {}
}
# fetch related AllTests object using `reportFormat` field
alltests_obj = None
for alltests in alltests_qs:
if parameter.reportFormat == alltests.reportFormat:
alltests_obj = alltests
break
if alltests_obj is not None:
item['alltests'] = {
'id': alltests_obj.id,
'name': alltests_obj.name
}
data.append(item)

Django model unique together both ways

Many questions already on this topic, but not what i'm searching for.
I have this Model:
class Options(TimeStampedModel)
option_1 = models.CharField(max_length=64)
option_2 = models.CharField(max_length=64)
class Meta:
unique_together = ('option_1', 'option_2')
Now I have a unique constraint on the fields.
Is there a way to also define this the other way around so that it doesn't matter what was option_1 and what was option_2
As example:
Options.create('spam', 'eggs') # Allowed
Options.create('spam', 'eggs') # Not allowed
Options.create('eggs', 'spam') # Is allowed but should not be
Thanks in advance!

I think a ManyToMany relation with a custom through table and an unique_together constraint on that table should do what you want.
Example code:
from django.db.models import Model, ForeignKey, ManyToManyField, CharField
class Option(Model):
name = CharField()
class Thing(TimeStampedModel):
options = ManyToManyField("Option", through="ThingOption")
class ThingOption(Model):
thing = ForeignKey(Thing)
option = ForeignKey(Option)
value = CharField()
class Meta:
unique_together = ('thing', 'option')
For Django 2.2+ it is recommended to use UniqueConstraint. In the docs there is a note stating unique_together may be deprecated in the future. See this post for its usage.

You can override create method, do something like
from django.db import models
class MyModelManager(models.Manager):
def create(self, *obj_data):
# Do some extra stuff here on the submitted data before saving...
# Ex- If obj_data[0]=="eggs" and obj_data[1]=="spam" is True don't allow it for your blah reason
# Call the super method which does the actual creation
return super().create(*obj_data) # Python 3 syntax!!
class MyModel(models.model):
option_1 = models.CharField(max_length=64)
option_2 = models.CharField(max_length=64)
objects = MyModelManager()

Django: Creating a non-destinct union of two querysets

I am writing an accouting app in django and there are Orders, which have a date when the invoice was created and an optional date when a credit note is created.
class Order(models.Model):
date_invoice_created = models.DateTimeField(null=True, blank=True)
date_credit_note_created = models.DateTimeField(null=True, blank=True)
I'm currently developing the view for our accountant, and she'd like to have both the invoice and the credit note on separate rows in the admin panel, sorted by theirs respective creation dates.
So basically I'd like to show the same model twice, in different row, sorted by different fields. In SQL, this would be something like:
SELECT id, create_date FROM (
SELECT id, date_invoice_created AS create_date, 'invoice' AS type FROM order
UNION
SELECT id, date_credit_note_created AS create_date, 'creditnote' AS type FROM order
) ORDER BY create_date
Don't mind my SQL-fu not being up-to-date, but I guess you understand what I mean.
So I've tried to get django to do this for me, by overriding the date in the second queryset, because django does not support the union of two extra'd querysets:
invoices = Order.objects.filter(date_invoice_created__isnull=False)
credit_notes = Order.filter_valid_orders(qs
).filter(
date_credit_note_created__isnull=False
).extra(
select={'date_invoice_created': 'date_credit_note_created'}
)
return (invoices | credit_notes).order_by('date_invoice_created')
unfortunately, the bit-wise-or operation for union always makes sure that the IDs are distinct, but I really want them not to be. How can I achieve to have a union with duplicate rows?

I have now found the solution to my problem using a SQL-View.
I've created a new migration (using south), which contains the above SQL query mentioned in the question as a view, which returns all rows twice, each with a create_date and type respectively for the credit note and the invoice.
accounting/migrations/00xx_create_invoice_creditnote_view.py:
class Migration(SchemaMigration):
def forwards(self, orm):
query = """
CREATE VIEW invoiceoverview_invoicecreditnoteunion AS
SELECT * FROM (
SELECT *,
date_invoice_created AS create_date,
'invoice' AS type
FROM accounting_order
WHERE date_invoice_created NOT NULL
UNION
SELECT *,
date_credit_note_created AS date,
'creditnote' AS type
FROM accounting_order
WHERE date_credit_note_created NOT NULL
);
"""
db.execute(query)
def backwards(self, orm):
query = """
DROP VIEW invoiceoverview_invoicecreditnoteunion;
"""
db.execute(query)
# ...
# the rest of the migration model
# ...
Then I've created a new model for this view, which has the Meta managed = False so that django uses the model without caring about it's creation. It has all the same fields as the original Order model, but also includes the two new fields from the SQL-View:
invoiceoverview/models.py:
class InvoiceCreditNoteUnion(models.Model):
""" This class is a SQL-view to Order, so that the credit note and
invoice can be displayed independently.
"""
class Meta:
managed = False # do not manage the model in the DB
# fields of the view
date = models.DateTimeField()
type = models.CharField(max_length=255)
# ...
# all the other fields of the original Order
# ...
Now I can use this model for the contrib.admin.ModelAdmin and display the appripriate content by checking the type field. e.g.:
class InvoiceAdmin(admin.ModelAdmin):
list_display = ['some_special_case']
def some_special_case(self, obj):
if obj.type == 'creditnote':
return obj.credit_note_specific field
else:
return obj.invoice_specific_field
admin.site.register(InvoiceCreditNoteUnion, InvoiceAdmin)
This finally allows me to use all the other features provided by the admin-panel, e.g. overriding the queryset method, sorting etc.