I am writing an accouting app in django and there are Orders, which have a date when the invoice was created and an optional date when a credit note is created.
class Order(models.Model):
date_invoice_created = models.DateTimeField(null=True, blank=True)
date_credit_note_created = models.DateTimeField(null=True, blank=True)
I'm currently developing the view for our accountant, and she'd like to have both the invoice and the credit note on separate rows in the admin panel, sorted by theirs respective creation dates.
So basically I'd like to show the same model twice, in different row, sorted by different fields. In SQL, this would be something like:
SELECT id, create_date FROM (
SELECT id, date_invoice_created AS create_date, 'invoice' AS type FROM order
UNION
SELECT id, date_credit_note_created AS create_date, 'creditnote' AS type FROM order
) ORDER BY create_date
Don't mind my SQL-fu not being up-to-date, but I guess you understand what I mean.
So I've tried to get django to do this for me, by overriding the date in the second queryset, because django does not support the union of two extra'd querysets:
invoices = Order.objects.filter(date_invoice_created__isnull=False)
credit_notes = Order.filter_valid_orders(qs
).filter(
date_credit_note_created__isnull=False
).extra(
select={'date_invoice_created': 'date_credit_note_created'}
)
return (invoices | credit_notes).order_by('date_invoice_created')
unfortunately, the bit-wise-or operation for union always makes sure that the IDs are distinct, but I really want them not to be. How can I achieve to have a union with duplicate rows?
I have now found the solution to my problem using a SQL-View.
I've created a new migration (using south), which contains the above SQL query mentioned in the question as a view, which returns all rows twice, each with a create_date and type respectively for the credit note and the invoice.
accounting/migrations/00xx_create_invoice_creditnote_view.py:
class Migration(SchemaMigration):
def forwards(self, orm):
query = """
CREATE VIEW invoiceoverview_invoicecreditnoteunion AS
SELECT * FROM (
SELECT *,
date_invoice_created AS create_date,
'invoice' AS type
FROM accounting_order
WHERE date_invoice_created NOT NULL
UNION
SELECT *,
date_credit_note_created AS date,
'creditnote' AS type
FROM accounting_order
WHERE date_credit_note_created NOT NULL
);
"""
db.execute(query)
def backwards(self, orm):
query = """
DROP VIEW invoiceoverview_invoicecreditnoteunion;
"""
db.execute(query)
# ...
# the rest of the migration model
# ...
Then I've created a new model for this view, which has the Meta managed = False so that django uses the model without caring about it's creation. It has all the same fields as the original Order model, but also includes the two new fields from the SQL-View:
invoiceoverview/models.py:
class InvoiceCreditNoteUnion(models.Model):
""" This class is a SQL-view to Order, so that the credit note and
invoice can be displayed independently.
"""
class Meta:
managed = False # do not manage the model in the DB
# fields of the view
date = models.DateTimeField()
type = models.CharField(max_length=255)
# ...
# all the other fields of the original Order
# ...
Now I can use this model for the contrib.admin.ModelAdmin and display the appripriate content by checking the type field. e.g.:
class InvoiceAdmin(admin.ModelAdmin):
list_display = ['some_special_case']
def some_special_case(self, obj):
if obj.type == 'creditnote':
return obj.credit_note_specific field
else:
return obj.invoice_specific_field
admin.site.register(InvoiceCreditNoteUnion, InvoiceAdmin)
This finally allows me to use all the other features provided by the admin-panel, e.g. overriding the queryset method, sorting etc.
Related
I have the following models:
class Customer(models.Model):
name = models.CharField(max_length=255)
email = models.EmailField(max_length = 255, default='example#example.com')
authorized_credit = models.IntegerField(default=0)
balance = models.IntegerField(default=0)
class Transaction(models.Model):
customer = models.ForeignKey(Customer, on_delete=models.CASCADE)
payment_amount = models.IntegerField(default=0) #can be 0 or have value
exit_amount = models.IntegerField(default=0) #can be 0 or have value
transaction_date = models.DateField()
I want to query for get all customer information and date of last payment.
I have this query in postgres that is correct, is just that i need:
select e.*, max(l.transaction_date) as last_date_payment
from app_customer as e
left join app_transaction as l
on e.id = l.customer_id and l.payment_amount != 0
group by e.id
order by e.id
But i need this query in django for an serializer. I try with that but return other query
In Python:
print(Customer.objects.filter(transaction__isnull=True).order_by('id').query)
>>> SELECT app_customer.id, app_customer.name, app_customer.email, app_customer.balance FROM app_customer
LEFT OUTER JOIN app_transaction
ON (app_customer.id = app_transaction.customer_id)
WHERE app_transaction.id IS NULL
ORDER BY app_customer.id ASC
But that i need is this rows
example
Whether you are working with a serializer or not you can reuse the same view/function for both the tasks.
First to get the transaction detail for the current customer object you have you have to be aware of related_name.related_name have default values but you can mention something unique so that you remember.
Change your model:
class Transaction(models.Model):
customer = models.ForeignKey(Customer, related_name="transac_set",on_delete=models.CASCADE)
related_names are a way in django to create reverse relationship from Customer to Transaction this way you will be able to do Customer cus.transac_set.all() and it will fetch all the transaction of cus object.
Since you might have multiple customers to get transaction details for you can use select_related() when querying this will hit the database least number of times and get all the data for you.
Create a function definition to get the data of all transaction of Customers:
def get_cus_transac(cus_id_list):
#here cus_id_list is the list of ids you need to fetch
cus_transac_list = Transaction.objects.select_related(customer).filter(id__in = cus_id_list)
return cus_transac_list
For your purpose you need to use another way that is the reason you needed related_name, prefetch_related().
Create a function definition to get the data of latest transaction of Customers: ***Warning: I was typing this answer before sleeping so there is no way the latest value of transaction is being fetched here.I will add it later but you can work on similar terms and get it done this way.
def get_latest_transac(cus_id_list):
#here cus_id_list is the list of ids you need to fetch
latest_transac_list = Customer.objects.filter(id__in = cus_id_list).prefetch_related('transac_set')
return latest_transac_list
Now coming to serializer,you need to have 3 serializers (Actually you need 2 only but third one can serialize Customer data + latest transaction that you need) ...one for Transaction and another for customer then the 3rd Serializer to combine them.
There might be some mistakes in code or i might have missed some details.As i have not checked it.I am assuming you know how to make serializers and views for the same.
One approach is to use subqueries:
transaction_subquery = Transaction.objects.filter(
customer=OuterRef('pk'), payment_amount__gt=0,
).order_by('-transaction_date')
Customer.objects.annotate(
last_date_payment=Subquery(
transaction_subquery.values('transaction_date')[:1]
)
)
This will get all customer data, and annotate with their last transaction date that has payment_amount as non-zero, in one query.
To solve your problem:
I want to query for get all customer information and date of last payment.
You can try use order by combine with distinct:
Customer.objects.prefetch_related('transaction_set').values('id', 'name', 'email', 'authorized_credit', 'balance', 'transaction__transaction_date').order_by('-transaction__transaction_date').distinct('transaction__transaction_date')
Note:
It only applies to PostgreSQL when distinct followed by parameters.
Usage of distinct: https://docs.djangoproject.com/en/3.2/ref/models/querysets/#distinct
I'm working in Django and having an issue displaying something properly in my Admin site. These are the models
class IndexSetSize(models.Model):
""" A time series of sizes for each index set """
index_set = models.ForeignKey(IndexSet, on_delete=models.CASCADE)
byte_size = models.BigIntegerField()
timestamp = models.DateTimeField()
class IndexSet(models.Model):
title = models.CharField(max_length=4096)
# ... some other stuff that isn't really important
def __str__(self):
return f"{self.title}"
It is displaying all the appropriate data I need, but, I want to display the sum of IndexSetSize, grouped by the index_set key and also grouped by the timestamp (There can be multiple occurrences of an IndexSet for a given timestamp, so I want to add up all the byte_sizes). Currently is just showing every single record. Additionally, I would prefer the total_size field to be sortable
Current Admin model looks like:
class IndexSetSizeAdmin(admin.ModelAdmin):
""" View-only admin for index set sizes """
fields = ["index_set", "total_size", "byte_size", "timestamp"]
list_display = ["index_set", "total_size", "timestamp"]
search_fields = ["index_set"]
list_filter = ["index_set__title"]
def total_size(self, obj):
""" Returns human readable size """
if obj.total_size:
return humanize.naturalsize(obj.total_size)
return "-"
total_size.admin_order_field = 'total_size'
def get_queryset(self, request):
queryset = super().get_queryset(request).select_related()
queryset = queryset.annotate(
total_size=Sum('byte_size', filter=Q(index_set__in_graylog=True)))
return queryset
It seems the proper way to do a group by in Django is to use .values(), although if I use that in get_queryset, an error is thrown saying Cannot call select_related() after .values() or .values_list(). I'm having trouble finding in the documentation if there's a 'correct' way to values/annotate/aggregate that will work correctly with get_queryset. It's a pretty simple sum/group by query I'm trying to do, but I'm not sure what the "Django way" is to accomplish it.
Thanks
I don't think you would be able to return the full queryset and group by index_set in get_queryset as you can't select all columns but group by an individual column in sql
SELECT *, SUM(index_size) FROM indexsetsize GROUP BY index_set // doesn't work
You could perform an extra query in the total_size method to get the aggregated value. However, this would perform the query for every row returned and slow your page load down.
def total_size(self, obj):
""" Returns human readable size """
return humanize.naturalsize(sum(IndexSetSize.objects.filter(
index_set=obj.index_set).values_list(
'byte_size', flat=True)))
total_size.admin_order_field = 'total_size'
It would be better to perform this annotation within the IndexSetAdmin as the index_set will already be grouped through the reverse foreign key. This will mean you can perform the annotation in get_queryset. I would also set the related_name on the foreign key on IndexSetSize so you can access the realted IndexSetSize objects from IndexSet using that name.
class IndexSetSize(models.Model):
index_set = models.ForeignKey(IndexSet, on_delete=models.CASCADE, related_name='index_set_sizes')
...
class IndexSetAdmin(admin.ModelAdmin):
...
def total_size(self, obj):
""" Returns human readable size """
if obj.total_size:
return humanize.naturalsize(obj.total_size)
return "-"
def get_queryset(self, request):
queryset = super().get_queryset(request).prefetch_related('index_set_sizes').annotate(
total_size=Sum('index_set_sizes__byte_size')).order_by('total_size')
return queryset
I defined 3 models related with M2M relationsships
class Suite(models.Model):
name = models.CharField(max_length=250)
title = models.CharField(max_length=250)
icon = models.CharField(max_length=250)
def __str__(self):
return self.title
class Role(models.Model):
name = models.CharField(max_length=250)
title = models.CharField(max_length=250)
suites = models.ManyToManyField(Suite)
services = models.ManyToManyField(Service)
Actions = models.ManyToManyField(Action)
users = models.ManyToManyField(User)
def __str__(self):
return self.title
In one of my views I tried to collect all the Suites related to an specific User. The user may be related to several Roles that can contain many Suites. And then filter Suites by name. But the filter seem to have no effects
queryset = Suite.objects.union(*(role.suites.all() for role in
self.get_user().role_set.all()))
repr(self.queryset)
'<QuerySet [<Suite: energia>, <Suite: waste 4 thing>]>'
self.queryset = self.queryset.filter(name="energia")
repr(self.queryset)
'<QuerySet [<Suite: energia>, <Suite: waste 4 thing>]>'
The query atribute inside the queryset not alter its content before executin the filter:
(SELECT "navbar_suite"."id", "navbar_suite"."name", "navbar_suite"."title", "navbar_suite"."icon" FROM "navbar_suite") UNION (SELECT "navbar_suite"."id", "navbar_suite"."name", "navbar_suite"."title", "navbar_suite"."icon" FROM "navbar_suite" INNER JOIN "navbar_role_suites" ON ("navbar_suite"."id" = "navbar_role_suites"."suite_id") WHERE "navbar_role_suites"."role_id" = 1)
(SELECT "navbar_suite"."id", "navbar_suite"."name", "navbar_suite"."title", "navbar_suite"."icon" FROM "navbar_suite") UNION (SELECT "navbar_suite"."id", "navbar_suite"."name", "navbar_suite"."title", "navbar_suite"."icon" FROM "navbar_suite" INNER JOIN "navbar_role_suites" ON ("navbar_suite"."id" = "navbar_role_suites"."suite_id") WHERE "navbar_role_suites"."role_id" = 1)
As stated in django docs, only count(), order_by(), values(), values_list() and slicing of union queryset is allowed. You can't filter on union queryset.
That means, you have to apply filters on queries before applying union on them.
Also, you can achieve your goal without even using union():
Suite.objects.filter(role_set__users=self.get_user(), name="energia")
You may need to adjust field name in filter if you've used related_name or related_query_name in definition of suites M2M field in Role model.
I had the same issue and ended up using the union query as a subquery so that the filters could work:
yourModelUnionSubQuerySet = YourModelQS1.union(YourModelQS2)
yourModelUnionQuerySet = YourModel.objects.filter(id__in=yourModelUnionSubQuerySet.values('id'))
There is a simple solution. Just use
self.queryset = self.queryset | <querySet you want to append>
instead of
self.queryset = self.queryset.union(<QuerySet you want to append>)
Worked for me. I hope this is understandable. After this you will be able to use filter.
I have the following queryset:
Pago.objects.filter(created__range=(self.inicio, self.fin)).values('tipo__nombre').annotate(monto=Sum('monto'))
And it is producing the following SQL:
SELECT "invoice_tipopago"."nombre", SUM("invoice_pago"."monto") AS "monto" FROM "invoice_pago" INNER JOIN "invoice_tipopago" ON ( "invoice_pago"."tipo_id" = "invoice_tipopago"."id" ) WHERE "invoice_pago"."created" BETWEEN 2015-01-01 00:00:00-06:00 AND 2015-06-04 14:18:00-06:00 GROUP BY "invoice_tipopago"."nombre", "invoice_pago"."modified", "invoice_pago"."created" ORDER BY "invoice_pago"."modified" DESC, "invoice_pago"."created" DESC
It is adding the extra modified and created columns that I am not specifying, and i would like to know how to avoid it. It should be noticed that Pago is a derived from the django-extensions TimeStampedModel class.
Thanks in advance.
I took a look at TimeStampedModeland it set a default ordering in the class meta:
class TimeStampedModel(models.Model):
""" TimeStampedModel
An abstract base class model that provides self-managed "created" and
"modified" fields.
"""
created = CreationDateTimeField(_('created'))
modified = ModificationDateTimeField(_('modified'))
class Meta:
get_latest_by = 'modified'
ordering = ('-modified', '-created',)
# ^^^^^^^^^^^^^^^^^^^^^^^^^^
abstract = True
see in github
You could overwrite that providing another filter to order by (like tipo__nombre or monto), ex:
Pago.objects.filter(
created__range=(self.inicio, self.fin)
).values(
'tipo__nombre'
).annotate(
monto=Sum('monto')
).order_by(
'tipo__nombre'
)
Hi I am trying to customize my inlines in django admin.
Here are my models:
class Row(models.Model):
name = models.CharField(max_length=255)
class Table(models.Model):
rows = models.ManyToManyField(Row, blank=True)
name = models.CharField(max_length=255)
def __unicode__(self):
return self.name
and my admin:
class RowInline(admin.TabularInline):
model = Table.rows.through
fields = ['name']
class TableAdmin(admin.ModelAdmin):
inlines = [
RowInline,
]
exclude = ('rows',)
However I get this error
ImproperlyConfigured at /admin/table_app/table/1/
'RowInline.fields' refers to field 'name' that is missing from the
form.
How is that possible ?
class RowInline(admin.TabularInline):
model = Table.rows.through
fields = ['name']
This presents a problem because Table.rows.through represents an intermediate model. If you would like to understand this better have a look at your database. You'll see an intermediate table which references this model. It is probably named something like apname_table_rows. This intermeditate model does not contain the field, name. It just has two foreign key fields: table and row. (And it has an id field.)
If you need the name it can be referenced as a readonly field through the rows relation.
class RowInline(admin.TabularInline):
model = Table.rows.through
fields = ['row_name']
readonly_fields = ['row_name']
def row_name(self, instance):
return instance.row.name
row_name.short_description = 'row name'
class TableAdmin(admin.ModelAdmin):
inlines = [
RowInline,
]
exclude = ('rows',)
Django can not display it as you expected. Because there is an intermediary join table which joins your tables. In your example:
admin.py:
class RowInline(admin.TabularInline):
model = Table.rows.through # You are not addressing directly Row table but intermediary table
fields = ['name']
As the above note, model in RowInline addressing following table in your database, not Your Row table and model
table: your-app-name_table_row
--------------------------------
id | int not null
table_id | int
row_id | int
You can think it like there is an imaginary table in your model that joins the two tables.
class Table_Row(Model):
table = ForeignKey(Table)
row = ForeignKey(Row)
So if you edit your Inline as following
class RowInline(admin.TabularInline):
model = Table.rows.through # You are not addressing directly Row table but intermediary table
fields = ['row', 'table']
you will not see any error or exception. Because your model in RowInline addresses an intermediary table and that table do have those fields. Django can virtualize the imaginary table Table_Row up to here and can handle this.
But we can use relations in admin, with using __. If your code do have a ForeignKey relation instead of ManyToManyField relation, then following be valid in your admin
class Row(models.Model):
name = models.CharField(max_length=255)
class Table(models.Model):
rows = models.ForeignKey(Row, blank=True)
name = models.CharField(max_length=255)
def __unicode__(self):
return self.name
and your admin:
class RowInline(admin.TabularInline):
model = Table
fields = ['rows__name']
Because you will have real Models and djnago can evaluate __ relation on them
But if you try that in your structure:
class Row(models.Model):
name = models.CharField(max_length=255)
class Table(models.Model):
rows = models.ManyToManyField(Row, blank=True)
name = models.CharField(max_length=255)
def __unicode__(self):
return self.name
and your admin:
class RowInline(admin.TabularInline):
model = Table.rows.through
fields = ['row__name']
it will raise Exception! Because you do not have real table in your model, and django can not evaluate __ relations on virtual models it designs on top of its head.
Conclusion:
In your Inlines addressing ManyToMany relations, you are dealing with an imaginary intermediary model and you can not use fields orexclude attributes on that because your imaginary model do not have those fields and django can not handle relations ober that imaginary table. Following will be acceptable
class RowInline(admin.TabularInline):
model = Table.rows.through
# No fields or exclude declarations in here
and django will display combo boxes for your virtual intermediary table options and add a fancy green + sign to add new records, but you will can not have inline fields to add new records directly to your database within the same single page. Djnago can not handle this on a single page.
You can try creating real intermediary table and show it using through, but thats totally a longer job and I do not test it to see its results.
Update: There is also the reason why django do not let something like that. Consider following:
Table | Table_Row | Row
-----------+-----------+---------
Start 5 | |
Step 1 5 | | 1
Step 2 5 | 5-1 | 1
At the beginning, You have a table with no related Rows, you want to add a row to the table... For joining a row with a table, you must first create a row so you execute step 1. After You do create your row, you can create Table_Row record to join these two. So in contains more than a single database insertion. Django crew may avoid such usage since it contains multiple inserts and operation is related more tables.
But this is just an assumption on the reason of the behavior.
In your admin.py try this
class RowInline(admin.TabularInline):
model = Table.rows.through
list_display = ('name',)
class TableAdmin(admin.ModelAdmin):
inlines = [
RowInline,
]
readonly_fields = ('rows',)