Suppose I have the following function, which retrieves data from a server in the form of user-defined objects. For example, let's define the objects as PersonalEntry.
def retrieve_people()
// returns a list of all latest people objects, with fields equal to those in PersonEntry
def retrieve_books()
// returns a list of all latest book objects regardless of author, with fields equal to those in BookEntry, contains attribute,
Both user-defined classes has an .as_dict() method which returns all its attributes in a dictionary.
I would like to update the model whenever this function is called (ie. update the fields if the instance of that model already exists, else, define a new instance of the model). This is my current setup.
class PersonEntry(models.Model):
name = models.CharField(max_length = 50)
age = models.IntegerField()
biography = models.CharField(max_length = 200)
def update_persons():
try:
temp = retrieve_person()
for person in temp:
match = PersonEntry.objects.filter(name = person.name(), age = person.age())
match.update(**person.as_dict())
except DoesNotExist:
PersonEntry.create(**person.as_dict())
class BookEntry(models.Model):
author = models.ForeignKey(PersonEntry)
author_name = models.CharField(max_length = 50) //books return redundant info
author_age = models.IntegerField() //books return redundant info
title = models.CharField(max_length = 50)
summary = models.CharField(max_length = 200)
def update_books():
try:
temp = retrieve_books()
for book in temp:
match = BookEntry.objects.filter(title = temp.title())
match.update(**book.as_dict(), associate_person(book.author_age(), book.author_name()))
except DoesNotExist:
BookEntry.create(**book.as_dict(), associate_person(book.author_age(), book.author_name())
def associate_person(age, name):
return PersonEntry.get(name = name, age = age)
I suppose a more general question is, how do I update models with relationships if I have a function which returns data? Do I have a method in the model itself, or do I have it one level up (ie. move update_books to the Person model) I'm new to Django, so not really sure how the organization should be.
I confess I haven't completely grokked your question, but I'll take a punt that you should look into
Managers
Generally, in django, everything is done as lazily as possible - meaning nothing gets updated until you actually try to use it - so you don't update models/relationships as you go, rather you just declare what they are (perhaps with a manager) then it works it out the current value only when asked.
Methods returning a collection (or queryset) of some kind of a model, should be a part of the Managers. So in your case update_books should be in a custom manager for BookEntry and update_persons should be in custom manager for PersonEntry.
Also do not call the function retrieve_* from inside the model or manager. Call it in your application logic and then pass the result to the manager method unless that method itself is part of the manager/model.
You do not need a separate filter and update method. Depending on how you have retrieved the data and it has a pk you can directly do a save. See How Does Django Know when to create or update
Related
I am trying to build a single course platorm where I will only hold lessons units materials where only people with membership will be able to see it , however when I try to do retrieve Lesson.course_allowed_mem_types.all() I got the following error 'ManyToManyDescriptor' object has no attribute 'all' , how can I fix this simple error?
class Lesson(models.Model):
content_title = models.CharField(max_length=120)
content_text = models.CharField(max_length=200)
thumbnail = models.ImageField(upload_to='static/xxx/xxx/xxx/xxx')
link = models.CharField(max_length=200, null=True)
allowed_memberships = models.ManyToManyField(Membership)
def __str__(self):
return self.content_title
views
def get_context_data(self, **kwargs):
context = super(bootCamp, self).get_context_data(**kwargs)
context['lessons'] = Lesson.objects.all()
user_membership = UserMembership.objects.filter(user=self.request.user).first()
user_membership_type = user_membership.membership.membership_type
course_allowed_mem_types = Lesson.allowed_memberships.all()
context['course_allowed_mem_types'] = course_allowed_mem_types
return context
You can query many-to-many related field only for model instance, not model class. It's not really clear what exactly is "all concrete allowed membership objects for a Lesson class" (Lesson.allowed_memberships.all()).
Is it "all membership objects related to any of existing lesson objects" or is it "all membership objects that can be related to a lesson object"?
Those are different queries, and Lesson.allowed_memberships.all() does not imply either, it's incorrect usage.
If you want the former, something like this could work
Membership.objects.filter(lesson__in=Lesson.objects.all())
(You already have this as context['lessons'] so use that instead, just showing the idea)
I think,
One lesson may have many memberships. so you are selecting all lessons with all memberships Lesson.allowed_memberships.all() .
Try selecting a single lesson then retrieve associated members
lesson = Lessons.objects.filter(pk=1)
course_allowed_mem_types = lesson.allowed_memberships.all()
If you want to create custom list like type, it is always a good idea to inherit from collections.abc.Iterable. It provides common operations required to work on such container types.
You can't just call .all() on any object/type, that type definition actually has to have all() method defined in class or parent class.
e.g.
class ListLike:
def __init__(self):
...
def all(self):
return some_iterator
I'm getting a very slow lookup in my Django models.
I have two tables:
class Scan(models.Model):
scan_name = models.CharField(max_length=32, unique=True, validators=[alphanumeric_plus_validator])
class ScanProcessingInfo(models.Model):
scan_name = models.CharField(max_length=32)
processing_name = models.CharField(max_length=64)
in_progress = models.BooleanField(default=False)
When I perform the following operation to get a list of all Scan objects which have a ScanProcessingInfo for a specific processing_name:
scans = models.Scan.objects.all()
scan_set = []
for scan in scans:
if self.set_type_definition.test_scan(scan, self.arg1, self.arg2):
scan_set.append(scan)
(test_scan routes to)
def get_proc_info_been_done(scan, spd_name):
try:
proc_info = models.ScanProcessingInfo.objects.get(scan_name = scan.scan_name)
except models.ScanProcessingInfo.DoesNotExist:
proc_info = None
if proc_info == None:
return False
return not proc_info.in_progress
the request takes about 10 seconds. There are 300 Scans in total and 10 ScanProcessingInfos. The db backend is an RDS MySQL db. I also expect someone will tell me off for using strings for the cross-table identifiers, but I doubt that's the cause here.
I'm sure I'm doing something obvious wrong, but would appreciate a pointer, thank you.
I think what you're asking is how to get all Scans for which a matching ScanProcessingInfo exists.
The first thing to do is to declare the actual relationship. You don't need to change your database (you should, but you don't have to); you can use your existing underlying field, but just tell Django to treat it as a foreign key.
class ScanProcessingInfo(models.Model):
scan = models.ForeignKey('Scan', to_field='scan_name', db_field='scan_name', on_delete=models.DO_NOTHING)
Now you can use this relationship to get all the scans in one go:
scan_set = Scan.objects.exclude(scanprocessinginfo=None)
Edit
To get all matching objects with a specific attribute, use the double-underscore syntax:
scan_set = Scan.objects.filter(scanprocessinginfo__processing_name=spd_name)
Use Many-to-one relationship.
scan_name = ForeignKey(Scan, related_name='processing_infos',on_delete=models.CASCADE)
I have encountered an issue with the required option from ndb.Model when it is placed inside a Model specified as a structure from a StructuredProperty.
Here is the models definition :
class SubModel(ndb.Model):
submodel_id = ndb.StringProperty(required=True)
class MyModel(ndb.Model):
model_id = ndb.StringProperty(required = True)
submodels = ndb.StructuredProperty(SubModel, repeated=True)
def __init__(self, *args, **kwargs):
super(MyModel, self).__init__(*args, **kwargs)
if 'submodels' in kwargs.keys():
_submodels = []
for kwarg in kwargs['submodels']:
if isinstance(kwarg, SubModel):
_submodels.append(kwarg)
else:
t = SubModel(**kwarg)
_submodels.append(t)
self.submodels = _submodels
What I want to do is either to create a model with no submodels or to create a model in which every submodels have a required id, otherwise, it raises an error.
So when I test this model, I get these results :
args1 = {"model_id":"some_id","submodels":[{"submodel_id":null}]}
model1 = MyModel(**args1)
model1.put()
print model1.key.id()
# Everything works while submodel_id hasn't been set and it returns the key id from Google Datastore
args2 = {"model_id":null,"submodels":[{"submodel_id":"some_id"}]}
model2 = MyModel(**args2)
model2.put()
print model2.key.id()
# Google Datastore raises an error : Entity has uninitialized properties: model_id
Is this behavior standard behavior from ndb.StructuredProperty (meaning that no property options are considered from the structure specified) or should I implement what I want to do differently ?
What I also want to do is to make Submodel inherit from ndb.Expando instead of ndb.Model but since ndb.Expando inherits from ndb.Model, that would be an even more precise question...
You have required = True on 'model_id'. You are trying to put() an entity without defining one (null doesn't count).
Your put()s are on MyModel. Not sure why the submodel_id requirement in SubModel isn't being checked when putting on the parent model. Seems it should be checked. Can you check the datastore and see what has actually been put()? When you read back that entity, what do you get?
My main ask: Is there any way to change the behavior of a related lookup such as MyModel.objects.filter(relationship__field="value")?
Consider this setup. I've got a one-to-many relationship with a custom Manager that filters out Books with active=False
from django.db import models
class ActiveOnly(models.Manager):
use_for_related_fields = True
def get_queryset(self):
return super(ActiveOnly, self).get_queryset().filter(active=True)
class Author(models.Model):
name = models.TextField()
class Book(models.Model):
active = models.BooleanField()
author = models.ForeignKey(Author, related_name="books")
title = models.TextField()
objects = ActiveOnly()
And let's create some data:
jim = Author.objects.create(name="Jim")
ulysses = Book.objects.create(title="Ulysses", author=jim, active=True)
finnegans = Book.objects.create(title="Finnegan's Wake", author=jim, active=False)
bill = Author.objects.create(name="Bill")
hamlet = Book.objects.create(title="Hamlet", author=bill, active=False)
Essentially, I never want to have to deal with inactive Books. Here are some queries to test various scenarios.
>>> Book.objects.all().count() # expecting the 1 active book: good
1
>>> jim.books.all() # also expecting only 1: good
1
>>> Author.objects.filter(books__title="Hamlet").first().name
u'Bill'
# ^ this is what I don't want to see, because bill's only book has active=False.
# I want the queryset to return no results.
Is there any way to change the behavior of the books__* lookup to include the additional filter on active?
In Django 1.10 the Manager.use_for_related_fields is deprecated in favor of setting Meta.base_manager_name on the model. See the updated documentation for details:
Model._base_manager
Using managers for related object access
By default, Django uses an instance of the Model._base_manager manager
class when accessing related objects (i.e. choice.poll), not the
_default_manager on the related object. This is because Django needs to be able to retrieve the related object, even if it would otherwise
be filtered out (and hence be inaccessible) by the default manager.
If the normal base manager class (django.db.models.Manager) isn’t
appropriate for your circumstances, you can tell Django which class to
use by setting Meta.base_manager_name.
The warning, not do exclude objects in the BaseManager still stands!
To quote the documentation:
By default, Django uses an instance of a “plain” manager class when accessing related objects (i.e. choice.poll), not the default manager on the related object. This is because Django needs to be able to retrieve the related object, even if it would otherwise be filtered out (and hence be inaccessible) by the default manager.
If the normal plain manager class (django.db.models.Manager) is not appropriate for your circumstances, you can force Django to use the same class as the default manager for your model by setting the use_for_related_fields attribute on the manager class.
so you need to change your manager to:
class ActiveOnly(models.Manager):
use_for_related_fields = True
def get_queryset(self):
return super(ActiveOnly, self).get_queryset().filter(active=True)
Edit: I'm not sure if you can override the QuerySet class somewhere to get this feature, my only suggestion would be to add books__active=True to your queries.
I've added the code to a github repo with a test illustrating the issue if someone wants to take it for a spin. https://github.com/alecklandgraf/NonDeletableDjangoModel
Original post:
I think you may need to set the default_manager in your Book model, and I would add a second model manager to query the inactive books.
class Book(models.Model):
active = models.BooleanField()
author = models.ForeignKey(Author, related_name="books")
title = models.TextField()
objects = ActiveOnly()
# try adding the following...
default_manager = ActiveOnly()
inactive_objects = models.Manager()
I rebuilt your example and found it to be the same for Django 1.8 and 1.9. Of interest would be the queries that are created as you've illustrated. You can see the added "active" = True in the WHERE clause of the Book query but not the Author query.
I'll noodle on this for a while.
In [1]: Book.objects.filter(title="Hamlet")
Out[1]: []
In [2]: Author.objects.filter(books__title="Hamlet")
Out[2]: [<Author: Bill>]
In [3]: print Book.objects.filter(title="Hamlet").query
SELECT "library_book"."id", "library_book"."active", "library_book"."author_id", "library_book"."title" FROM "library_book" WHERE ("library_book"."active" = True AND "library_book"."title" = Hamlet)
In [4]: print Author.objects.filter(books__title="Hamlet").query
SELECT "library_author"."id", "library_author"."name" FROM "library_author" INNER JOIN "library_book" ON ("library_author"."id" = "library_book"."author_id") WHERE "library_book"."title" = Hamlet
I'm trying to hold a kind of table of contents structure in my database. Simplified example:
models.py
class Section (models.Model):
title = models.CharField(max_length=80)
order = models.IntegerField()
class SectionClickable(Section):
link = models.CharField(max_length=80)
class SectionHeading(Section):
background_color = models.CharField(max_length=6)
views.py
sections = Section.objects.filter(title="Hello!")
for section in sections:
if(section.sectionheading):
logger.debug("It's a heading")
I need to do some processing operations if it's a SectionHeading instance, but (as in the Django manual), accessing section.sectionheading will throw a DoesNotExist error if the object is not of type SectionHeading.
I've been looking into alternatives to this kind of problem, and I'm skimming over Generic Foreign Keys in the contenttypes package. However, this seems like it would cause even more headaches at the Django Admin side of things. Could anyone advise on a better solution than the one above?
Edit: I avoided abstract inheritence because of the order field. I would have to join the two QuerySets together and sort them by order
well you could check the type:
if isinstance(section, SectionHeading)
but duck typing is generally preferred
edit:
actually, that probably won't work. the object will be a Section. but you can look for the attribute:
if hasattr(section, 'sectionheading')
or
try:
do_something_with(section.sectionheading)
except AttributeError:
pass # i guess it wasn't one of those
The solution I came up using involved an extra field pointing to the (rather useful) ContentType class:
class Section(models.Model):
name = models.CharField(max_length=50)
content_type = models.ForeignKey(ContentType,editable=False,null=True)
def __unicode__(self):
try:
return self.as_leaf_class().__unicode__()
except:
return self.name
def save(self, *args, **kwargs):
if(not self.content_type):
self.content_type = ContentType.objects.get_for_model(self.__class__)
super(Section, self).save(*args, **kwargs)
def as_leaf_class(self):
content_type = self.content_type
model = content_type.model_class()
if(model == Section):
return self
return model.objects.get(id=self.id)
If you're going through "base" object, I think this solution is pretty nice and comfortable to work with.
I've been using something similar to what second suggests in his edit:
class SomeBaseModel(models.Model):
reverse_name_cache = models.CharField(_('relation cache'), max_length=10,
null=True, editable=False)
def get_reverse_instance(self):
try:
return getattr(self, self.reverse_name_cache)
except AttributeError:
for name in ['sectionclickable', 'sectionheading']:
try:
i = getattr(self, name)
self.reverse_name_cache = name
return i
except ObjectDoesNotExist:
pass
Now, this isn't exactly pretty, but it returns the subclass instance from a central place so I don't need to wrap other statements with try. Perhaps the hardcoding of subclass reverse manager names could be avoided but this approach was enough for my needs.
OP here.
While second's answer is correct for the question, I wanted to add that I believe multi-table inheritence is an inefficient approach for this scenario. Accessing the attribute of the sub-class model would cause a query to occur - thus requiring a query for every row returned. Ouch. As far as I can tell, select_related doesn't work for multi-table inheritence yet.
I also ruled out ContentTypes because it wouldn't do it elegantly enough and seemed to require a lot of queries also.
I settled on using an abstract class:
class Section (models.Model):
title = models.CharField(max_length=80)
order = models.IntegerField()
class Meta:
abstract=True
ordering=['order']
Queried both tables:
section_clickables = SectionClickable.objects.filter(video=video)
section_headings= SectionHeading.objects.filter(video=video)
and joined the two querysets together
#Join querysets http://stackoverflow.com/questions/431628/how-to-combine-2-or-more-querysets-in-a-django-view
s = sorted(chain(section_headings, section_clickables), key=attrgetter('order'))
Lastly I made a template tag to check the instance:
from my.models import SectionHeading, SectionClickable
#register.filter()
def is_instance(obj, c):
try:
return isinstance(obj, eval(c))
except:
raise ObjectDoesNotExist('Class supplied to is_instance could not be found. Import it in the template tag file.')
so that in my template (HamlPy) I could do this:
- if s|is_instance:"SectionClickable"
%span {{s.title}}
- if s|is_instance:"SectionHeading"
%span{'style':'color: #{{s.color}};'}
{{s.title}}
The result is that I only used two queries, one to get the SectionClickable objects and one for the SectionHeading objects