I'm trying to learn Google App Engine's NDB and I'm confused about the structure of models.
My situation is similar to a CMS platform with Post Types (like in WordPress), so I have "Blogs" and "Pages". All of these Post Types require the same set of attributes: Parent, Name, Slug, Template, Content, Status, and Date.
So far, I gather that I need to create a Model for these like this:
class Post(ndb.Expando):
parent = ndb.StringProperty()
name = ndb.StringProperty()
slug = ndb.StringProperty()
template = ndb.StringProperty()
content = ndb.StringProperty(indexed=False)
status = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
(I'm using Expando because I will be adding "unknown" attributes in my application)
But with this structure, all of my posts (in every Post Type) will be within the same "kind", so queries will take longer (if I'm not mistaken).
How can I create many Models (kinds) with the same attributes?
Do I copy & paste the above Model under different class names?
Is it possible to create new Models dynamically (similar to "Custom Post Types" in WordPress)? Does it work if I use ndb.Key('Blog', blogid) instead of declaring a Model?
Do I create a Model called class PostType(ndb.Model) that stores the "Post Types" and give them ancestors of Posts? (If I'm not mistaken, this would cause problems because updating a Post would "lock" the entire ancestor tree for a second or so)
My primary goal is efficiency. Thanks!
Updates:
As written by Dan and mgilson, adding sub-classes of the main Post class Model is a good way to solve this:
class Post(ndb.Expando):
parent = ndb.StringProperty()
name = ndb.StringProperty()
slug = ndb.StringProperty()
template = ndb.StringProperty()
content = ndb.StringProperty(indexed=False)
status = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
class Blog(Post):
pass
However, this requires writing the Models statically. Is there a way to accomplish this dynamically (without declaring them as Models beforehand)?
Update:
Following the advice given below, I decided to keep all of my entities under the same kind. I might decide later on to change this to subclasses (separate kinds for each "Post Type") if my queries get messy. Thank you all for your great advice!
How can I create many Models with the same attributes?
You can subclass:
class SpecialPost(Post):
"""Special post type that is a different kind than Post."""
Though it's often easy enough to use the same kind and just add an extra field that represents the kind of post which you can filter on in queries.
Is it possible to create new Models dynamically (similar to "Custom Post Types" in WordPress)? Does it work if I use ndb.Key('Blog', blogid) instead of declaring a Model?
I'm not 100% sure that I understand what you're asking here. You can dynamically create models the same way you can dynamically create classes in python (using type), but you probably don't want to be doing this. Getting those dynamically created models (and keeping track of their names) will probably end up giving you serious headaches.
Basically a simple example of subclassing, which #mgilson mentioned already.
class Post(ndb.Expando):
parent = ndb.StringProperty()
name = ndb.StringProperty()
slug = ndb.StringProperty()
template = ndb.StringProperty()
content = ndb.StringProperty(indexed=False)
status = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
class Blog(Post):
someint = ndb.IntegerProperty()
blog = Blog(status='new', someint=2)
key = blog.put()
print key.kind()
As for dynamically creating models, from the Model's Constructor doc:
An application won't normally call Model(), but is likely to call the
constructor of a class that inherits from Model. This creates a new
instance of this model, also known as an entity.
Even if possible (I didn't dig too deep inside ndb/models.py to say with certainty that it's not) it doesn't appear a clear thing. Personally I'd stay away from that and instead re-think the need for such dynamically created models.
Related
I am starting to create a webapp using Django and MongoDB. Everything is working fine when I create a model and save it into the Database. Now, I do a "Class.objects.get()" to get the object I need from my DB and I have one field called "media" which is a ListField(). I had tried doing either:
Concert.media.append(list)
or
Concert.media.extend(list)
and then
Concert.save()
This is my "Concert" object in my models.py:
class Concert(models.Model):
main_artist = models.CharField(max_length=50)
concert_id = models.AutoField(primary_key=True)
openers = ListField(EmbeddedModelField('Opener'))
concert_date = models.DateField()
slug = models.SlugField(unique=True)
media = ListField()
And when I go to see the results in does not update the object. No values where saved. If someone can help me I going to give a super cyber fist bump.
Concert is a class, not an instance. You can't save a class. You need to make an instance of the class and save that. Something like
c = Concert()
c.media.append(list)
c.save()
(btw, just as a note, list is a bad variable name because list is a type in python. Never use types as variable names (though everyone is guilty of this at one point or another, including me.))
When dealing whith model inheritance in django is it possible to remove a instance of model subclass, without removing the superclass itself?
Using the Django example, can you remove just the Resturaunt object and retain the Place object?
Yesterday I was looking for an answer to this question and I came up with this solution, which was enough for my problem but could be scaled up as needed.
Assuming you have a Restaurant and a Place django models, the way to delete a restaurant only without touching the row inside the Place's table is creating a "fake" Restaurant model like this:
class FakeRestaurant(models.Model):
place_ptr = models.PositiveIntegerField(db_column="place_ptr_id", primary_key=True)
serves_hot_dogs = models.BooleanField()
serves_pizza = models.BooleanField()
class Meta:
app_label = Restaurant._meta.app_label
db_table = Restaurant._meta.db_table
managed = False
Now, you can retrieve objects from that table as if it had no bound external relationship:
place = Place.objects.get(pk=1)
restaurant = Restaurant.objects.get(pk=1)
fake_restaurant = FakeRestaurant.objects.get(pk=1)
fake_restaurant.delete()
fake_restaurant and restaurant won't exist anymore, place will remain untouched.
Cheers,
Davide
In Django 1.9 parameter keep_parents was added to model delete() function, so to keep parents just call:
restaurant.delete(keep_parents=True)
Docs: https://docs.djangoproject.com/en/1.10/ref/models/instances/#django.db.models.Model.delete
UPDATE:
Apparently, this feature is not working properly in Django 1.9, please see the comments.
I'm building a personal project with Django, to train myself (because I love Django, but I miss skills). I have the basic requirements, I know Python, I carefully read the Django book twice if not thrice.
My goal is to create a simple monitoring service, with a Django-based web interface allowing me to check status of my "nodes" (servers). Each node has multiple "services". The application checks the availability of each service for each node.
My problem is that I have no idea how to represent different types of services in my database. I thought of two "solutions" :
single service model, with a "serviceType" field, and a big mess with the fields. (I have no great experience in database modeling, but this looks... "bad" to me)
multiple service models. i like this solution, but then I have no idea how I can reference these DIFFERENT services in the same field.
This is a short excerpt from my models.py file : (I removed everything that is not related to this problem)
from django.db import models
# Create your models here.
class service(models.Model):
port = models.PositiveIntegerField()
class Meta:
abstract = True
class sshService(service):
username = models.CharField(max_length=64)
pkey = models.TextField()
class telnetService(service):
username = models.CharField(max_length=64)
password = models.CharField(max_length=64)
class genericTcpService(service):
pass
class genericUdpService(service):
pass
class node(models.Model):
name = models.CharField(max_length=64)
# various fields
services = models.ManyToManyField(service)
Of course, the line with the ManyToManyField is bogus. I have no idea what to put in place of "*Service". I honestly searched for solutions about this, I heard of "generic relations", triple-join tables, but I did'nt really understand these things.
Moreover, English is not my native language, so coming to database structure and semantics, my knowledge and understanding of what I read is limited (but that's my problem)
For a start, use Django's multi-table inheritance, rather than the abstract model you have currently.
Your code would then become:
from django.db import models
class Service(models.Model):
port = models.PositiveIntegerField()
class SSHService(Service):
username = models.CharField(max_length=64)
pkey = models.TextField()
class TelnetService(Service):
username = models.CharField(max_length=64)
password = models.CharField(max_length=64)
class GenericTcpService(Service):
pass
class GenericUDPService(Service):
pass
class Node(models.Model):
name = models.CharField(max_length=64)
# various fields
services = models.ManyToManyField(Service)
On the database level, this will create a 'service' table, the rows of which will be linked via one to one relationships with separate tables for each child service.
The only difficulty with this approach is that when you do something like the following:
node = Node.objects.get(pk=node_id)
for service in node.services.all():
# Do something with the service
The 'service' objects you access in the loop will be of the parent type.
If you know what child type these will have beforehand, you can just access the child class in the following way:
from django.core.exceptions import ObjectDoesNotExist
try:
telnet_service = service.telnetservice
except (AttributeError, ObjectDoesNotExist):
# You chose the wrong child type!
telnet_service = None
If you don't know the child type beforehand, it gets a bit trickier. There are a few hacky/messy solutions, including a 'serviceType' field on the parent model, but a better way, as Joe J mentioned, is to use a 'subclassing queryset'. The InheritanceManager class from django-model-utils is probably the easiest to use. Read the documentation for it here, it's a really nice little bit of code.
I think one approach that you might consider is a "subclassing queryset". Basically, it allows you to query the parent model and it will return instances of the child models in the result queryset. It would let you do queries like:
models.service.objects.all()
and have it return to you results like the following:
[ <sshServiceInstance>, <telnetServiceInstance>, <telnetServiceInstance>, ...]
For some examples on how to do this, check out the links on the blog post linked below.
http://jazstudios.blogspot.com/2009/10/django-model-inheritance-with.html
However, if you use this approach, you shouldn't declare your service model as abstract as you do in the example. Granted, you will be introducing an extra join, but overall I've found the subclassing queryset to work pretty well for returning a mixed set of objects in a queryset.
Anyway, hope this helps,
Joe
If you are looking for generic foreign key relations you should check the Django contenttypes framework (built into Django). The docs pretty much explain how to use it and how to work with generic relations.
An actual service can only be on one node, right? In that case when not have a field
node = models.ForeignKey('node', related_name='services')
in the service class?
Say you have a concept of "user" records that you'd like to store in the data store.
class User (db.Model):
first_name = db.StringProperty()
last_name = db.StringProperty()
created = db.DateTimeProperty(auto_now_add=True)
twitter_oauth_token = db.StringProperty()
twitter_oauth_secret = db.StringProperty()
There are some fields you'd like to use almost ever time you use a user object, like first_name and last_name.
However, there are some fields you only have one use case for, for example, twitter_oauth_token and twitter_oauth_secret, and it's somewhat inefficient to bother serializing and deserializing these when they're not needed 95% of the time.
So if you split your model up:
class User (db.Model):
first_name = db.StringProperty()
last_name = db.StringProperty()
created = db.DateTimeProperty(auto_now_add=True)
class UserTwitterOauth(db.Model):
oauth_token = db.StringProperty(required=True)
oauth_secret = db.StringProperty(required=True)
created = db.DateTimeProperty(auto_now_add=True)
You can put a ReferenceProperty to the User in the UserTwitterOauth, but this would actually be one-to-many as there's nothing stopping there to being multiple UserTwitterOauth objects per User. You want there to be at most one UserTwitterOauth related to any User. How can you relate these models on a one-to-one basis?
In this specific case, your best option is probably to make the UserTwitterOauth entity a child of the User entity with a well-known key name, like so:
my_user = User(first_name="John", last_name="Smith")
my_user.put()
extra_info = UserTwitterOauth(parent=my_user, key_name="UserTwitterOauth")
extra_info.put()
You can add a straightforward method or property to the User class to make it easy to retrieve the additional information, and a class method to UserTwitterOauth to serve as a factory method, preserving the convention.
Incidentally, note that User is a dangerous name for an entity - the Users API has a class called User too, and unless you're very careful with your imports, you may end up referring to one when you intend to refer to the other.
A reference property to the user from the twitter access token is by far the easiest to maintain, in my view. It is true that the user could be referenced by many access tokens.
You will however find yourself doing things by convention a lot of times when working GAE.
EDIT preventing several access tokens referencing same user:
You can access referencing access tokens as a query via the User.usertwitteroauth_set property. If you want a more descriptive name, specify the parameter collection_namewhen setting up the ReferenceProperty. Say for example you want to remove any referencing access tokens before you add a new one, you could gather that logic as such:
class User(db.Model):
def set_access_token(self, access_token):
db.delete(self.twitter_access_tokens) # Think this should work, otherwise iterate over the query.
new_access_token.user = self
new_access_token.put()
class UserTwitterOauth(db.Model):
user = db.ReferenceProperty(User, collection_name = 'twitter_access_tokens')
Howdy. I'm working on migrating an internal system to Django and have run into a few wrinkles.
Intro
Our current system (a billing system) tracks double-entry bookkeeping while allowing users to enter data as invoices, expenses, etc.
Base Objects
So I have two base objects/models:
JournalEntry
JournalEntryItems
defined as follows:
class JournalEntry(models.Model):
gjID = models.AutoField(primary_key=True)
date = models.DateTimeField('entry date');
memo = models.CharField(max_length=100);
class JournalEntryItem(models.Model):
journalEntryID = models.AutoField(primary_key=True)
gjID = models.ForeignKey(JournalEntry, db_column='gjID')
amount = models.DecimalField(max_digits=10,decimal_places=2)
So far, so good. It works quite smoothly on the admin side (inlines work, etc.)
On to the next section.
We then have two more models
InvoiceEntry
InvoiceEntryItem
An InvoiceEntry is a superset of / it inherits from JournalEntry, so I've been using a OneToOneField (which is what we're using in the background on our current site). That works quite smoothly too.
class InvoiceEntry(JournalEntry):
invoiceID = models.AutoField(primary_key=True, db_column='invoiceID', verbose_name='')
journalEntry = models.OneToOneField(JournalEntry, parent_link=True, db_column='gjID')
client = models.ForeignKey(Client, db_column='clientID')
datePaid = models.DateTimeField(null=True, db_column='datePaid', blank=True, verbose_name='date paid')
Where I run into problems is when trying to add an InvoiceEntryItem (which inherits from JournalEntryItem) to an inline related to InvoiceEntry. I'm getting the error:
<class 'billing.models.InvoiceEntryItem'> has more than 1 ForeignKey to <class 'billing.models.InvoiceEntry'>
The way I see it, InvoiceEntryItem has a ForeignKey directly to InvoiceEntry. And it also has an indirect ForeignKey to InvoiceEntry through the JournalEntry 1->M JournalEntryItems relationship.
Here's the code I'm using at the moment.
class InvoiceEntryItem(JournalEntryItem):
invoiceEntryID = models.AutoField(primary_key=True, db_column='invoiceEntryID', verbose_name='')
invoiceEntry = models.ForeignKey(InvoiceEntry, related_name='invoiceEntries', db_column='invoiceID')
journalEntryItem = models.OneToOneField(JournalEntryItem, db_column='journalEntryID')
I've tried removing the journalEntryItem OneToOneField. Doing that then removes my ability to retrieve the dollar amount for this particular InvoiceEntryItem (which is only stored in journalEntryItem).
I've also tried removing the invoiceEntry ForeignKey relationship. Doing that removes the relationship that allows me to see the InvoiceEntry 1->M InvoiceEntryItems in the admin inline. All I see are blank fields (instead of the actual data that is currently stored in the DB).
It seems like option 2 is closer to what I want to do. But my inexperience with Django seems to be limiting me. I might be able to filter the larger pool of journal entries to see just invoice entries. But it would be really handy to think of these solely as invoices (instead of a subset of journal entries).
Any thoughts on how to do what I'm after?
First, inheriting from a model creates an automatic OneToOneField in the inherited model towards the parents so you don't need to add them. Remove them if you really want to use this form of model inheritance.
If you only want to share the member of the model, you can use Meta inheritance which will create the inherited columns in the table of your inherited model. This way would separate your JournalEntry in 2 tables though but it would be easy to retrieve only the invoices.
All fields in the superclass also exist on the subclass, so having an explicit relation is unnecessary.
Model inheritance in Django is terrible. Don't use it. Python doesn't need it anyway.