I have this model about Invoices which has a property method which refers to another model in order to get the cancelation date of the invoice, like so:
class Invoice(models.Model):
# (...)
#property
def cancel_date(self):
if self.canceled:
return self.records.filter(change_type = 'cancel').first().date
else:
return None
And in one of my views, i need to query every invoice that has been canceled after max_date or hasn't been canceled at all.
Like so:
def ExampleView(request):
# (...)
qs = Invoice.objects
if r.get('maxDate'):
max_date = datetime.strptime(r.get('maxDate'), r'%Y-%m-%d')
ids = list(map(lambda i: i.pk, filter(lambda i: (i.cancel_date == None) or (i.cancel_date > max_date), qs)))
qs = qs.filter(pk__in = ids) #Error -> django.db.utils.OperationalError: too many SQL variables
However, ids might give me a huge list of ids which causes the error too many SQL variables.
What's the smartest approach here?
EDIT:
I'm looking for a solution that does not involve adding cancel_date as a model field since invoice.records refers to another model where we store every date attribute of the invoice
Like so:
class InvoiceRecord(models.Model):
invoice = models.ForeignKey(Invoice, related_name = 'records', on_delete = models.CASCADE)
date = models.DateTimeField(default = timezone.now)
change_type = models.CharField(max_length = 32) # Multiple choices field
And every invoice might have more than one same date attribute. For example, one invoice might have two cancelation dates
You can annotate a Subquery() expression [Django docs] which will give you the date to do this:
from django.db.models import OuterRef, Q, Subquery
def ExampleView(request):
# (...)
qs = Invoice.objects.annotate(
cancel_date=Subquery(
InvoiceRecords.objects.filter(invoice=OuterRef("pk")).values('date')[:1]
)
)
if r.get('maxDate'):
max_date = datetime.strptime(r.get('maxDate'), r'%Y-%m-%d')
qs = qs.filter(Q(cancel_date__isnull=True) | Q(cancel_date__gt=max_date))
I would set cancel_date as database field when you set cancel flag. Then you can use single query:
qs = Invoice.objects.filter(Q(cancel_date__isnull=True) | Q(cancel_date__gt=max_date))
It's say cancel_date is NULL or greater than max_date
Not sure about your property cancel_date. It will return first record with change_type='cancel' which can be (don't know your code flow) other record then you call that property on.
Related
I have the following models:
class Customer(models.Model):
name = models.CharField(max_length=255)
email = models.EmailField(max_length = 255, default='example#example.com')
authorized_credit = models.IntegerField(default=0)
balance = models.IntegerField(default=0)
class Transaction(models.Model):
customer = models.ForeignKey(Customer, on_delete=models.CASCADE)
payment_amount = models.IntegerField(default=0) #can be 0 or have value
exit_amount = models.IntegerField(default=0) #can be 0 or have value
transaction_date = models.DateField()
I want to query for get all customer information and date of last payment.
I have this query in postgres that is correct, is just that i need:
select e.*, max(l.transaction_date) as last_date_payment
from app_customer as e
left join app_transaction as l
on e.id = l.customer_id and l.payment_amount != 0
group by e.id
order by e.id
But i need this query in django for an serializer. I try with that but return other query
In Python:
print(Customer.objects.filter(transaction__isnull=True).order_by('id').query)
>>> SELECT app_customer.id, app_customer.name, app_customer.email, app_customer.balance FROM app_customer
LEFT OUTER JOIN app_transaction
ON (app_customer.id = app_transaction.customer_id)
WHERE app_transaction.id IS NULL
ORDER BY app_customer.id ASC
But that i need is this rows
example
Whether you are working with a serializer or not you can reuse the same view/function for both the tasks.
First to get the transaction detail for the current customer object you have you have to be aware of related_name.related_name have default values but you can mention something unique so that you remember.
Change your model:
class Transaction(models.Model):
customer = models.ForeignKey(Customer, related_name="transac_set",on_delete=models.CASCADE)
related_names are a way in django to create reverse relationship from Customer to Transaction this way you will be able to do Customer cus.transac_set.all() and it will fetch all the transaction of cus object.
Since you might have multiple customers to get transaction details for you can use select_related() when querying this will hit the database least number of times and get all the data for you.
Create a function definition to get the data of all transaction of Customers:
def get_cus_transac(cus_id_list):
#here cus_id_list is the list of ids you need to fetch
cus_transac_list = Transaction.objects.select_related(customer).filter(id__in = cus_id_list)
return cus_transac_list
For your purpose you need to use another way that is the reason you needed related_name, prefetch_related().
Create a function definition to get the data of latest transaction of Customers: ***Warning: I was typing this answer before sleeping so there is no way the latest value of transaction is being fetched here.I will add it later but you can work on similar terms and get it done this way.
def get_latest_transac(cus_id_list):
#here cus_id_list is the list of ids you need to fetch
latest_transac_list = Customer.objects.filter(id__in = cus_id_list).prefetch_related('transac_set')
return latest_transac_list
Now coming to serializer,you need to have 3 serializers (Actually you need 2 only but third one can serialize Customer data + latest transaction that you need) ...one for Transaction and another for customer then the 3rd Serializer to combine them.
There might be some mistakes in code or i might have missed some details.As i have not checked it.I am assuming you know how to make serializers and views for the same.
One approach is to use subqueries:
transaction_subquery = Transaction.objects.filter(
customer=OuterRef('pk'), payment_amount__gt=0,
).order_by('-transaction_date')
Customer.objects.annotate(
last_date_payment=Subquery(
transaction_subquery.values('transaction_date')[:1]
)
)
This will get all customer data, and annotate with their last transaction date that has payment_amount as non-zero, in one query.
To solve your problem:
I want to query for get all customer information and date of last payment.
You can try use order by combine with distinct:
Customer.objects.prefetch_related('transaction_set').values('id', 'name', 'email', 'authorized_credit', 'balance', 'transaction__transaction_date').order_by('-transaction__transaction_date').distinct('transaction__transaction_date')
Note:
It only applies to PostgreSQL when distinct followed by parameters.
Usage of distinct: https://docs.djangoproject.com/en/3.2/ref/models/querysets/#distinct
I have a problem with filtering by boolean field in a subquery.
For example, I have two models: Good and Order.
class Good(models.Model):
objects = GoodQuerySet.as_manager()
class Order(models.Model):
good = models.FK(Good, related_name="orders")
is_completed = models.BooleanField(default=False)
I want to calculate how many completed orders has each good.
I implemented a method in Good's manager:
class GoodQuerySet(models.QuerySet):
def completed_orders_count(self):
subquery = Subquery(
Order.objects.filter(good_id=OuterRef("id"))
.order_by()
.values("good_id")
.annotate(c=Count("*"))
.values("c")
)
return self.annotate(completed_orders_count=Coalesce(subquery, 0))
This method counts all existing orders for a good, but it works when I call it like this:
Good.objects.completed_orders_count().first().completed_orders_count
To get the correct value of completed orders I tried to add filter is_completed=True. The final version looks like this:
class GoodQuerySet(models.QuerySet):
def completed_orders_count(self):
subquery = Subquery(
Order.objects.filter(good_id=OuterRef("id"), is_completed=True)
.order_by()
.values("good_id")
.annotate(c=Count("*"))
.values("c")
)
return self.annotate(completed_orders_count=Coalesce(subquery, 0))
If I try to call Good.objects.completed_orders_count().first().completed_orders_count I got an error:
django.core.exceptions.FieldError: Expression contains mixed types. You must set output_field.
I have a django tables2 table working pretty well where it only shows Querysets that have the correct Project Id. I need it to exclude Querysets that have duplicate user_id's though.
example:
user a, week 1, Project 1
user a, week 2, Project 1
user b, week 1, Project 1
display users that are assigned to Project 1. (needed result is table with rows a & b)
What I have so far:
#models.py
class Allocation(models.Model):
user_id = models.ForeignKey(Resource)
project_id = models.ForeignKey(Project)
week = models.DateField(default=timezone.now)
allocated_hours = models.IntegerField(default=0)
actual_hours = models.IntegerField(default=0)
#tables.py
class Project_Resource_table(tables.Table):
role = tables.Column(accessor='user_id.resource_type')
name = tables.Column(accessor='user_id.resource_name')
allocation = tables.Column(accessor='total_allocation')
class Meta:
model = Allocation
exclude = ('id', 'user_id', 'project_id', 'week', 'allocated_hours', 'actual_hours')
sequence = ('role', 'name', 'allocation')
#view (project_profile.py)
def project_properties(request, offset):
try:
offset = int(offset)
except ValueError:
raise Http404()
project = Project.objects.get(pk=offset)
table = Project_Resource_table(Allocation.objects.filter(project_id=offset).order_by('user_id'))
return render(request, '../templates/copacity/project_profile.html', {
'table': table,
})
I have tried using set() but to no avail. Also tried distinct() but does not work with sqlite3 db. Also tried calculating with a function, but kept getting "Expected table or queryset, not function". Any suggestions on what else to try?
You need to make sure you either pass a QuerySet or a list of dicts when creating the table. If django-tables2 complains about getting a function, you are giving it a function, rather than the value returned from that function.
I'm currently trying to find a way to do something with Django's (v1.10) ORM that I feel should be possible but I'm struggling to understand how to apply the documented methods to solve my problem.
Edit: So here's the sql that I've hacked together to return the data that I'd like from the dbshell, with a postgresql database now, after I realised that my original sqlite3 backed sql query was incorrect:
select
voting_bill.*,vv.vote
from
voting_bill
left join
(select
voting_votes.vote,voting_votes.bill_id
from
voting_bill
left join
voting_votes
on
voting_bill.id=voting_votes.bill_id
where
voting_votes.voter_id = (select id from auth_user where username='richard' or username is Null)
)
as
vv
on
voting_bill.id=vv.bill_id;
Here's the 'models.py' for my voting app:
from django.db import models
from django.contrib.auth.models import User
class Bill(models.Model):
name = models.CharField(max_length=255)
description = models.TextField()
result = models.BooleanField()
status = models.BooleanField(default=False)
def __str__(self):
return self.name
class Votes(models.Model):
vote = models.NullBooleanField()
bill = models.ForeignKey(Bill, related_name='bill',
on_delete=models.CASCADE,)
voter = models.ForeignKey(User, on_delete=models.CASCADE,)
def __str__(self):
return '{0} {1}'.format(self.bill, self.voter)
I can see that my sql works as I expect with the vote tacked onto the end, or a null if the user hasn't voted yet.
I was working to have the queryset in this format so that I can iterate over it in the template to produce a table and if the result is null I can instead provide a link which takes the user to another view.
I've read about select_related and prefetch_related, but as I said, I'm struggling to work out how I translate this to how I can do this in SQL.
Hope I correctly understood your problem. Try this:
votes = Votes.objects.filter(voter__username='django').select_related('bill')
You can use this. But I think you do not need select_related in this case.
bills_for_user = Bill.objects.filter(votes__voter__username='django').select_related('votes').distinct()
Now you can iterate your bills_for_user
for bill in bills_for_user:
bill_name = bill.name
bill_description = bill.description
bill_result = bill.result
bill_status = bill.status
# and there are several variants what you can with votes
bill_votes = bill.votes_set.all() # will return you all votes for this bill
bill_first_vote1 = bill.votes_set.first() # will return first element in this query or None if its empty
bill_first_vote2 = bill.votes_set.all()[0] # will return first element in this query or Error if its empty
bill_last_vote = bill.votes_set.last()[0] # will return last element in this query or None if its empty
# you can also filter it for example by voting
bill_positive_votes = bill.votes_set.filter(vote=True) # will return you all votes for this bill with 'vote' = True
bill_negative_votes = bill.votes_set.filter(vote=False) # will return you all votes for this bill with 'vote' = False
bill_neutral_votes = bill.votes_set.filter(vote=None) # will return you all votes for this bill with 'vote' = None
I have models called Stores and SaleItems which look some what like this.
class Stores(models.Model):
name = models.CharField(unique=True, max_length=20)
class SaleItems(models.Model):
sale_by = models.ForeignKey(Stores)
start_date = models.DateTimeField()
end_date = models.DateTimeField()
So I need to retrieve sale items based on the following conditions.
If its not my store, then leave out items having start_date greater than today and end_date lesser than today.
If its my store, then get items irrespective of start_date and end_date.
So in my views.py, this is how far I have come up.
class SaleItemsView(viewsets.ModelViewSet):
querys = SaleItems.objects.all()
def get_queryset(self):
#get my store id
store_id = self.request.query_params.get('store_id')
querys = SaleItems.objects\
.exclude(store__ne=store_id, end_date__lt=timezone.now())\
.exclude(store__ne=store_id, start_date__gt=timezone.now())
return querys
But it seems django doesn't have not equal operator any more. Is there any other way I can achieve this?
I think you would combine the records for each of your conditions using Qobjects, then do distinct() to remove duplicates:
now = timezone.now()
items_within_date_range = Q(start_date__lte=today, end_date__gte=today)
curr_store_items = Q(store=store_id)
result = SaleItems.objects.filter(items_within_date_range|curr_store_items).distinct()
The first query would get all items within time range for both your store and other stores, second query would get all items for your store, then combine then would get everything.