So I've been stuck with a design problem for the last couple of days and have sunk countless hours into it, to no avail.
My problem is that I wish return all the active Articles. I have made a method within the model, however am unable to use .filter(is_active=True)which would be the worlds best solution.
So now I have made the method into one long filter in the ArticleManager, the problem being that I cannot seem to figure out a way to count the current clicks in a way that can be useful to me. (The current_clicks method in the Article model is what I am aiming for).
Models.py
class ArticleManager(models.Manager):
def get_queryset(self):
return super(ArticleManager, self).get_queryset().filter(article_finish_date=None).filter(article_publish_date__lte=timezone.now())
#this is where i need something to the effect of .filter(article_max_clicks__gt=click_set.count())
class Article(models.Model):
article_name_text = models.CharField(max_length=200)
article_max_clicks = models.IntegerField(default=0)
article_creation_date = models.DateTimeField('date created')
article_publish_date = models.DateTimeField('date published', null=True, blank=True)
article_finish_date = models.DateTimeField('date finished', null=True, blank=True)
def __str__(self):
return self.article_name_text
def is_active(self):
if self.article_finish_date==None:
if self.article_publish_date <= timezone.now():
return self.current_clicks() < self.article_max_clicks
else:
return False
else:
return False
def current_clicks(self):
return self.click_set.count()
is_active.boolean = True
actives = ArticleManager()
class Click(models.Model):
click_article = models.ForeignKey(Article, on_delete=models.CASCADE)
click_user = models.ForeignKey(User, on_delete=models.CASCADE)
click_date = models.DateTimeField('date clicked')
def __str__(self):
return str(self.id) + " " + str(self.click_date)
This is how the clicks are created in views.py if this helps
article.click_set.create(click_article=article, click_user=user, click_date=timezone.now())
If anyone has any sort of idea of how abouts I should do this it would be greatly appreciated!
Many thanks in advance, just let me know if you need anymore information!
Django's annotate functionality is great for adding properties at the time of querying. From the docs -
Per-object summaries can be generated using the annotate() clause. When an annotate() clause is specified, each object in the QuerySet will be annotated with the specified values.
In order to keep your querying performance-minded, you can use this in your manager and not make the (possibly very slow) call of related objects for each of your Articles. Once you have an annotated property, you can use it in your query. Since Django only executes your query when objects are called, you can use this annotation instead of counting the click_set, which would call a separate query per related item. The current_clicks method may still be useful to you, but if calling it for multiple articles your queries will add up quickly and cause a big performance hit.
Please note - I added a related_name of clicks keyword arg to your click_article field in order to use it in place of 'click_set'.
In addition, you'll see the use of Q objects in the query below. What this allows us to do is chain together multiple filters together. These can be nested while using AND (,) / OR(|) operands. So, a reading of the Q objects below would be:
Find all articles where article publish date is before now AND (article has no finish date OR article finish date is after now)
from django.db.models.query import Q,Count
class ArticleManager(models.Manager):
def get_queryset(self):
return super(ArticleManager, self).get_queryset().filter(
Q(article_publish_date__lte=timezone.now()),
( Q(article_finish_date__isnull=True)|
Q(article_finish_date__gte=timezone.now())
).annotate(
click_count=Count('clicks')
).filter(
article_max_clicks__gt=click_count
)
class Article(models.Model):
actives = ArticleManager()
def current_clicks(self):
return self.clicks.count()
# Now you can call Article.actives.all() to get all active articles
class Click(models.Model):
click_article = models.ForeignKey(Article, on_delete=models.CASCADE, related_name='clicks') # added a related_name for more explicit calling of prefetch_related
Related
I have the following two models:
class URLResource(models.Model):
class ResourceType(models.TextChoices):
VID = 'VID', _('Video')
DOC = 'DOC', _('Document')
resource_type = models.CharField(
choices=ResourceType.choices,
default=ResourceType.UNK,
max_length=3)
title = models.CharField(max_length=280, null=True)
url = models.URLField()
created = models.DateTimeField(auto_now_add=True)
def __repr__(self):
return f'URLResource {self.resource_type}: {self.title} URL: {self.url}'
class Record(models.Model):
user = models.ForeignKey(
settings.AUTH_USER_MODEL,
...)
date = models.DateTimeField(default=now, blank=True)
# <resource>: I want to point to a resource somehow
def __repr__(self):
return f'Record {{ User: {self.user}, Resource: }}'
My use case is such that a lot of users can create Records and list them, but since the underlying URLResource is going to be same, I thought I could use many-to-one relationship from Records-to-URLResource so the underlying resources would remain the same. However, even though I have only one type of resource now i.e. URLResource, I also want to roll out other resources like VideoResource, PhysicalResource etc. Clearly, I can't simply use the many-to-one relationship. So, I figured I could use Django's contenttype framework, but I still can't get things to work.
I want to be able to do atleast these two things:
When creating a Record, I can easily assign it a URLResource (or any other Resource) (but only one resource).
I can easily access the URLResource fields. For example like Record.objects.get(user=x).resource.resource_type.
I feel like I'm chasing my tail here and so I've come to your fine folks to help me understand where I've screwed up and why my thinking on this must be flawed in some way.
I'm writing an API in DRF and while it doesn't have a lot of tables in the DB, there are many to many relationships in the DB which makes it feel complicated, or at least it makes it difficult to just view the db tables and intuitively understand what's related.
First my model. I'm getting this when attempting to POST a new object to the jobs model. I'm needing to validate that the requesting worker has the permission to create a job with the related target and workergroup
Models:
class Workers(models.Model):
class Meta:
ordering = ['id']
workerid = models.CharField(max_length=16, verbose_name="Worker ID", unique=True, default="Empty")
customer = models.ForeignKey(Customers, on_delete=models.DO_NOTHING, default=1)
workername = models.CharField(max_length=64, verbose_name="Worker Friendly Name", default="")
datecreated = models.DateTimeField(auto_now_add=True)
awsarn = models.CharField(max_length=60, verbose_name="ARN Name of Worker", blank=True, null=True)
customerrights = models.ManyToManyField(Customers, related_name="access_rights", default="")
class Targets(models.Model):
class Meta:
ordering = ['id']
customer = models.ForeignKey(Customers, on_delete=models.SET_NULL, default=1, null=True)
friendly_name = models.CharField(max_length=70, verbose_name="Target Friendly Name", unique=False)
hostname = models.CharField(max_length=120, verbose_name="Target Hostname", default="")
ipaddr = models.GenericIPAddressField(protocol='both', unpack_ipv4=True, default="", null=True)
class WorkerGroups(models.Model):
class Meta:
ordering = ['id']
name = models.CharField(max_length=60, default="Default Group")
workers = models.ManyToManyField(Workers)
class Jobs(models.Model):
target = models.ForeignKey(Targets, on_delete=models.DO_NOTHING)
datecreated = models.DateTimeField(auto_now_add=True)
startdate = models.DateTimeField()
enddate = models.DateTimeField(null=True)
frequency = models.TimeField(default='00:05')
workergroup = models.ForeignKey(WorkerGroups, on_delete=models.DO_NOTHING)
jobdefinition = models.ForeignKey(JobDefinitions, on_delete=models.DO_NOTHING)
In my serializers I have a JobSerializer which is referencing PrimaryKeyRelatedField classes which I think should have the effect of limiting the queryset which validates those related models. BTW, the customerrightsids are being built in the view and that seems to work fine for all other models.
Serializers:
def get_queryset(self):
print("In Custom TargetPK get_queryset")
queryset = Targets.objects.filter(customer_id__in=self.context['auth'].customerrightsids)
if isinstance(queryset, (QuerySet, Manager)):
queryset = queryset.all()
return queryset
class WorkerGroupPKSerializer(serializers.PrimaryKeyRelatedField):
def get_queryset(self):
print("In Custom WorkerGroupPK get_queryset")
queryset = WorkerGroups.objects.filter(workers__customer_id__in=self.context['auth'].customerrightsids)
if isinstance(queryset, (QuerySet, Manager)):
queryset = queryset.all()
return queryset
class JobSerializer(serializers.ModelSerializer):
workergroup = WorkerGroupPKSerializer(many=False) # Commenting this out removes error
target = TargetPKSerializer(many=False) # This seems to work fine even though it's similar to the line above
class Meta:
model = Jobs
fields = '__all__'
def create(self, validated_data):
print(self.context['auth'])
return super().create(validated_data)
There's nothing special in the create method of the viewset object. It's taking the request and passing a couple into the ViewSet and updating its context. I can share that if needed, but that doesn't seem to be where the issue is.
So finally, for the error. When I perform a POST to /jobs/ I'm getting the following:
Error:
MultipleObjectsReturned at /jobs/
get() returned more than one WorkerGroups -- it returned 2!
The error clearly states that I'm getting multiple WorkerGroups returned in a get() but I don't know where or how to resolve that in this case.
It is clearly a problem with the WorkerGroupPKSerializer. If I comment out the reference to it from the JobSerializer the error goes away. That stops the validation of that field though, so that's not a workable solution!
I'm not 100% sure I'm on the right track here, but it seems you might have a problem of duplicate/multiple results. You should try and use .distinct() on the Queryset in WorkerGroupPKSerializer (also, see Django docs). You used a ForeignKey on the customer property of the Worker model, so that makes it possible to have multiple Worker's belonging to the same WorkerGroup matching the filter query, thus returning the same WorkerGroup twice. So when the id of that WorkerGroup is POSTed, the get() will match two results, throwing that error.
Commenting out seems to clear the error, and this would be because of the fact that you're also commenting out many=False, thus, .get() is not called anymore. But as mentioned in the question, this would disable the custom queryset used to filter on.
Thanks to Sasja's answer. Here is the updated WorkerGroupPKSerializer that now works.
class WorkerGroupPKSerializer(serializers.PrimaryKeyRelatedField):
def get_queryset(self):
print("In Custom WorkerGroupPK get_queryset")
queryset = WorkerGroups.objects.filter(workers__customer_id__in=self.context['auth'].customerrightsids).distinct()
if isinstance(queryset, (QuerySet, Manager)):
queryset = queryset.all()
return queryset
I'm trying to improve the performance of one of my Django applications to make them run just a bit smoother, as part of a first iteration in improving what I currently have running. When doing some profiling I've noticed that I have a very high number of SQL queries being executed on a couple of pages.
The dashboard page for instance easily has 250+ SQL queries being executed. Further investigation pointed me to the following piece of code in my views.py:
for project in projects:
for historicaldata in project.historical_data_for_n_months_ago(i):
for key in ('hours', 'expenses'):
history_data[key] = history_data[key] + getattr(historicaldata, key)
Relevant function in models.py file:
def historical_data_for_n_months_ago(self, n=1):
n_year, n_month = n_months_ago(n)
try:
return self.historicaldata_set.filter(year=n_year, month=n_month)
except HistoricalData.DoesNotExist:
return []
As you can see, this will cause a lot of queries being executed for each project in the list. Originally this was set-up this way to keep functionality centrally at the model level and introduce convenience functions across the application.
What would be possible ways on how to reduce the number of queries being executed when loading this page? I was thinking on either removing the convince function and just working with select_related() in the view, but, it would still need a lot of queries in order to filter out records for a given year and month.
Thanks a lot in advance!
Edit As requested, some more info on the related models.
Project
class Project(models.Model):
name = models.CharField(max_length=200)
status = models.IntegerField(choices=PROJECT_STATUS_CHOICES, default=1)
last_updated = models.DateTimeField(default=datetime.datetime.now)
total_hours = models.DecimalField(default=0, max_digits=10, decimal_places=2)
total_expenses = models.DecimalField(default=0, max_digits=10, decimal_places=2)
def __str__(self):
return "{i.name}".format(i=self)
def historical_data_for_n_months_ago(self, n=1):
n_year, n_month = n_months_ago(n)
try:
return self.historicaldata_set.filter(year=n_year, month=n_month)
except HistoricalData.DoesNotExist:
return []
HistoricalData
class HistoricalData(models.Model):
project = models.ForeignKey(Project, on_delete=models.CASCADE)
person = models.ForeignKey(Person, on_delete=models.CASCADE)
year = models.IntegerField()
month = models.IntegerField()
hours = models.DecimalField(max_digits=10, decimal_places=2, default=0)
expenses = models.DecimalField(max_digits=10, decimal_places=2, default=0)
def __str__(self):
return "Historical data {i.month}/{i.year} for {i.person} ({i.project})".format(i=self)
I don't think looping through querysets is ever a good idea. So it would be better if you could find some other way. If you could elaborate your view function and exactly what its supposed be to done maybe I could help further.
If you want all the historical_data entries for a project (reverse related) you need to use prefetch_related. Since you want a specific portion of the historical data associated with said project you need to use it with Prefetch.
from django.db.models import Prefetch
Project.objects.prefetch_related(
Prefetch(
'historicaldata_set',
queryset=HistoricalData.objects.filter(year=n_year, month=n_month)
)
)
After that, you should be looping through this dataset in your django template (if you are using that). You can also pass it to a drf-serializer and that would also get your work done :)
I'm new to Django and I'm in the process of converting a PHP project into Python etc.
I'm trying to do something super simple, but I keep getting the following error:
AttributeError at /news/1/
'QuerySet' object has no attribute 'slug'
Here is my most of my Model to help explain:
class Article(models.Model):
title = models.CharField(max_length=200)
STATUS_CHOICES = ((1,'Published'), (2,'Hidden'), (3,'Draft'))
status = models.IntegerField(choices=STATUS_CHOICES,default=3)
pub_date = models.DateField('date published')
tags = TaggableManager()
header_width = models.IntegerField(default=1,blank=True,null=True)
header_height = models.IntegerField(default=1,blank=True,null=True)
header = models.ImageField(upload_to='news/',width_field='header_width',height_field='header_height',blank=True,null=True)
header_filter = models.BooleanField('Enable filter',default=1)
excerpt = HTMLField(blank=True)
body = HTMLField(blank=True)
custom_link_text = models.CharField(max_length=20,default="Read More")
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
slug = AutoSlugField(unique=True,max_length=200,populate_from='db_slug',default="",slugify=return_value)
def __str__(self):
return self.title
Im currently just testing to pull through the slug, so my view method currently looks like this:
def detail_redirect(request, pk):
a = Article.objects.all().filter(pk=pk)
return HttpResponse(a.slug)
# return redirect('news:detail',args=(a.slug,pk))
The plan is for this method to redirect to another URL in my application. It queries the DB via the primary key and fetches the Slug, which I then pass on to the redirect shortcut.
It seems to be something that should just work, but it's not. It's really frustrating. The object i query appears to return ok. Because of my __str__ method it returns the title. But any other attributes throw and error. Could it be todo with visibility such as being private or protected?
I'm hoping it's something simple I'm missing. Let me know if you require more code/detail to help explain.
Thanks for taking the time to look at my question.
filter always returns a queryset, which is a list-like object potentially consisting of many items. Querysets do not have model attributes, only their members do. You should use get instead:
a = Article.objects.get(pk=pk)
(Note you don't need all(), in either version of the code.)
I've got this Post model at the moment:
class Post(models.Model):
title = models.CharField(max_length = 140)
body = models.TextField()
date = models.DateTimeField()
def __unicode__(self):
return self.title
If I've got different parts of a website (or a forum rather) that contain different posts, e.g. Discussion about basketball, and Discussion about football, if I wanted to return just posts concerning basketball or just posts concerning football, is the easiest way to just make a specific basketball_post model/football_post model or is there a more efficient way? Should I perhaps be storing the values differently?
Thanks
Django has a really good tutorial. It is about making a Poll app. In the first chapter the thing you want is discussed. It is about a Question that can have multiple Choices.:
class Question(models.Model):
question_text = models.CharField(max_length=200)
pub_date = models.DateTimeField('date published')
class Choice(models.Model):
question = models.ForeignKey(Question)
choice_text = models.CharField(max_length=200)
votes = models.IntegerField(default=0)
The foreignKey creates a relation between two models. The same can be done for a blog:
class Category(models.Model):
title = models.CharField(max_length=200)
class Post(models.Model):
category = models.ForeignKey(Category) # This is the important part.
title = models.CharField(max_length=200)
body = models.TextField()
date = models.DateTimeField()
def __unicode__(self):
return self.title
The ForeignKey relation lets you do really nice things:
basketball_posts = Post.objects.filter(category_title='Basketball')
But before we all tell you how it is done, I really recommend to do the tutorial. It introduces you to all important Django concepts: https://docs.djangoproject.com/en/1.7/intro/tutorial01/
Update
If you have a fixed set of categories that are not likely to change, than you can hardcode them and use field choices:
class Post(models.Model):
FOOTBALL = 'F' # Variable name and db_value
CRICKET = 'C'
INTRODUCTION = 'I'
CATEGORY_CHOICES = (
(FOOTBALL, 'Soccer'), # Variable name and display value
(CRICKET, 'Cricket'),
(INTRODUCTION, 'Hello my name is'),
)
category = models.CharField(max_length=1,
choices=CATEGORY_CHOICES,
default=INTRODUCTION)
...
https://docs.djangoproject.com/en/dev/ref/models/fields/#choices
One of the advantages of this 'choice machinery' over a CharField without pre defined choices is that you are sure what values end up in your database. This lets you query them, without worrying if your data is sane:
Post.objects.filter(category=Post.CRICKET)
Use the extra table if you need the freedom to create new categories in the future. Use field choices if you don't want (or need) that freedom.
I would suggest to just add a field which makes the post relevant to that certain topic:
class Post(models.Model):
title = models.CharField(max_length = 140)
body = models.TextField()
date = models.DateTimeField()
type = models.CharField(max_length=20) #<--- new field: e.g 'basketball','hockey'..
def __unicode__(self):
return self.title
example query:
#basketball posts
qs = Post.objects.filter(type__icontains="basketball")
then you dont need to have multiple models which also would be redundant.
Assuming all of the posts are in the same format, you could add another field to your model like "type". Different discussion forums could send a different values for that field when the post is added.
type = models.CharField(max_length=140, choices=['Football', 'Basketball', 'Baseball'])
Storing this would make it easy to filter which posts are which.
Post.objects.filter(type = 'Football')
Assuming that one post can be about only one sport, the better approach would be to have a foreign key relation between a model that stores data about a post with another model that stores the data about sports.
Something like this
class Sport(models.Model):
name = models.CharField(max_length = 200)
description = models.TextField()
def __unicode__(self):
return self.name
class Post(models.Model):
title = models.CharField(max_length = 140)
body = models.TextField()
date = models.DateTimeField()
sport = models.ForeignKey(Sport)
def __unicode__(self):
return self.title
This gives you the advantage of isolating the 'Sport' and the 'Post' models.You can add as many sports as you want, without any posts referring to it.
One more advantage is that you can add relevant information to the relevant models.
Eg:Suppose you want to add the information about "how many players are there in a team for sport x?". You can easily achieve this by adding a field "number_of_players" in the 'Sport' model without affecting the 'Post' model.
If you had to do this in one model, 'Post', then it would create lot of issues in terms of data consistency and other undesirable things.
Also, the query will look something like this:
posts = Post.objects.filter(sport__name = "Basketball")
PS:If your requirement is that a post can be tagged to multiple sports, then you can use ManyToMany field instead of a simple foreign key.
https://docs.djangoproject.com/en/dev/topics/db/examples/many_to_many/
You could assign your posts tags or category, and filter on those.
If you use the model approach what happens when you add more sports? You'll need manually add the sports in your code, using a tags or category approach allows you to handle it in the db, and would then allow you to filter on the tags/categories in your system