We Keep Coding

GAE NDB Confused about Models and duplicating attributes

I'm trying to learn Google App Engine's NDB and I'm confused about the structure of models.
My situation is similar to a CMS platform with Post Types (like in WordPress), so I have "Blogs" and "Pages". All of these Post Types require the same set of attributes: Parent, Name, Slug, Template, Content, Status, and Date.
So far, I gather that I need to create a Model for these like this:
class Post(ndb.Expando):
parent = ndb.StringProperty()
name = ndb.StringProperty()
slug = ndb.StringProperty()
template = ndb.StringProperty()
content = ndb.StringProperty(indexed=False)
status = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
(I'm using Expando because I will be adding "unknown" attributes in my application)
But with this structure, all of my posts (in every Post Type) will be within the same "kind", so queries will take longer (if I'm not mistaken).
How can I create many Models (kinds) with the same attributes?
Do I copy & paste the above Model under different class names?
Is it possible to create new Models dynamically (similar to "Custom Post Types" in WordPress)? Does it work if I use ndb.Key('Blog', blogid) instead of declaring a Model?
Do I create a Model called class PostType(ndb.Model) that stores the "Post Types" and give them ancestors of Posts? (If I'm not mistaken, this would cause problems because updating a Post would "lock" the entire ancestor tree for a second or so)
My primary goal is efficiency. Thanks!
Updates:
As written by Dan and mgilson, adding sub-classes of the main Post class Model is a good way to solve this:
class Post(ndb.Expando):
parent = ndb.StringProperty()
name = ndb.StringProperty()
slug = ndb.StringProperty()
template = ndb.StringProperty()
content = ndb.StringProperty(indexed=False)
status = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
class Blog(Post):
pass
However, this requires writing the Models statically. Is there a way to accomplish this dynamically (without declaring them as Models beforehand)?
Update:
Following the advice given below, I decided to keep all of my entities under the same kind. I might decide later on to change this to subclasses (separate kinds for each "Post Type") if my queries get messy. Thank you all for your great advice!

How can I create many Models with the same attributes?
You can subclass:
class SpecialPost(Post):
"""Special post type that is a different kind than Post."""
Though it's often easy enough to use the same kind and just add an extra field that represents the kind of post which you can filter on in queries.
Is it possible to create new Models dynamically (similar to "Custom Post Types" in WordPress)? Does it work if I use ndb.Key('Blog', blogid) instead of declaring a Model?
I'm not 100% sure that I understand what you're asking here. You can dynamically create models the same way you can dynamically create classes in python (using type), but you probably don't want to be doing this. Getting those dynamically created models (and keeping track of their names) will probably end up giving you serious headaches.

Basically a simple example of subclassing, which #mgilson mentioned already.
class Post(ndb.Expando):
parent = ndb.StringProperty()
name = ndb.StringProperty()
slug = ndb.StringProperty()
template = ndb.StringProperty()
content = ndb.StringProperty(indexed=False)
status = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
class Blog(Post):
someint = ndb.IntegerProperty()
blog = Blog(status='new', someint=2)
key = blog.put()
print key.kind()
As for dynamically creating models, from the Model's Constructor doc:
An application won't normally call Model(), but is likely to call the
constructor of a class that inherits from Model. This creates a new
instance of this model, also known as an entity.
Even if possible (I didn't dig too deep inside ndb/models.py to say with certainty that it's not) it doesn't appear a clear thing. Personally I'd stay away from that and instead re-think the need for such dynamically created models.

Django get_or_create does not return a usable Model object in clean method of ModelForm

Hello,
I have bound a ModelForm to one of my model that contains a ForeignKey to another model everything driven by a CreateView. What I want to achieve is to create the model object corresponding to the foreign key if it doesn't exist before the form is overall validated and the final object created in database.
Below the models I use:
class UmsAlerting(models.Model):
alert_id = models.IntegerField(primary_key=True, editable=False)
appli = models.ForeignKey('UmsApplication')
env = models.ForeignKey('UmsEnvironment')
contact = models.ForeignKey('UmsContacts')
custom_rule = models.ForeignKey('UmsCustomRules', null=True, blank=True)
class UmsApplication(models.Model):
appli_id = models.IntegerField(primary_key=True)
trigram_ums = models.CharField(max_length=4L)
class UmsContacts(models.Model):
contact_id = models.IntegerField(primary_key=True)
mail_addr = models.CharField(max_length=100L)
class UmsEnvironment(models.Model):
env_id = models.IntegerField(primary_key=True)
env_name = models.CharField(max_length=5L)
The model bound to the form is UmsAlerting. The model object I want to create if it doesn't exist is UmsContacts. I managed to use the field's clean method in my ModelForm of the contact field and use the get_or_create method like below:
def clean_contact(self):
data = self.cleaned_data['contact']
c, _ = UmsContacts.objects.get_or_create(mail_addr=data)
return c
It perfectly works when the contact is already in the database but when it needs to be created my form return a ValidationError on the contact field saying "This field cannot be null". If I submit the same form a second time without changing anything the UmsAlerting object is well created with no validation error.
My guess is that, for a reason I don't get, when get_or_create is used to create a UmsContacts object it cannot be used to create the new UmsAlerting object. So in clean_contact method the get is working and returns the UmsContacts object but the create part doesn't. It'd be like the UmsContacts object is saved when the whole form is validated but not before as I'd want it to.
Anyone could help me find out what is the problem ? Is using the clean method not the best idea ? Is there another strategy to use to take around this problem ?
Thanks in advance for your help.

It's probably because the object you are creating expects value for contact_id. If you use contact_id field for just setting object id -then you do not have to create it at all. Django takes care of Id's automatically.
Also. field clean method should return cleaned data not object. That creates whole lot more problems on its own.

Django dynamic class cannot be updated

I have a set of DB tables which store customer order counts per minute per day. Each month of the year is a different table in order to avoid excessively large tables. In order to retrieve this data in my Django webpage, I dynamically create a model class with db_table being populated based on the date received from an html form input. The problem is that when I resubmit the form with a new date value, the class does not update to what should be the new model, it still maintains the old value.
My models.py looks something like this:
class baseModel(models.Model):
id = models.CharField(max_length=40)
date = models.IntegerField()
minute = models.IntegerField()
totalorders = models.IntegerField()
class Meta:
abstract = True
managed = False
def getModel(type, yyyymm):
if type == 'duration':
class DurationClass(baseModel):
medianduration = models.IntegerField()
avgduration = models.IntegerField()
class Meta:
db_table='orderTable' + yyyymm
#debug statement
print db_table
return DurationClass
yyyymm is just a string, like '201204' for April 2012. So if I enter April 2012 into the input box it works fine but then if I change to March 2012, it still queries for April data. I can see from the debug statement that db_table is being properly updated, but for some reason it's not working. Do I need to deallocate the old dynamic model before allocating a new one or something? In view.py, I'm just doing this (not in a loop):
myModel = getModel('duration', startyyyymm)
QS = myModel.objects.using( ...etc
Many thanks for any ideas.

You have a problem about how python manage the creation of dynamic clases. I don't know exactly how python works, but it seems to be that the way you do it is not totally correct. I think it is because python classes are attached to one module, so the first time you execute "getModel" it creates the model as you expect. But, after that, every time you execute "getModel", as the class has always the same name, python can't create the same class at the same module, so it somehow returns you the same class you create the first time you call "getModel". (I hope you understand my English, although i might be wrong about how python dynamic classes creation works)
I search a little and make some tests before giving you an answer. It seems to be that the best way of creating a dynamic class is using "type" (python built-in method), so you can create one class per table (this classes must have a different name).
Here's an example of what you can do (it worked for me):
def getModel(type, yyyymm):
if type == 'duration':
newModelClass = type(
'newModelName', #It could be the table name you are going to search in. It must be different for every different db table you want to use. For example: 'orderTable' + yyyymm
(baseModel, ), #Base class for your new model
{
'medianduration' : models.IntegerField(), #New model's attribute
'avgduration' : models.IntegerField(), #New model's attribute
'__module__':__name__, #This is required. If not given, type raises a KeyError
'Meta': type(
'Meta',
(object,),
{
'db_table':'orderTable' + yyyymm, #Here you put the table name you want to use
'__module__':__name__,
}
)
}
)
return newModelClass
If i didn't make any copy/paste mistake, it should now work for you.
Again, sorry if i make any English mistake. I'm a little bit rusty at writing in English and in English in general :S
I hope it helps you. Althought i agree that your database should work fine without using multiple tables...

All you need is given the below link: https://code.djangoproject.com/wiki/DynamicModels

Create an object before calling the save method

I want to create an object in Django before calling the save method. This object will be created from a ForeignKey Value, I've changed the foreignkey field to look like an input field in order to write a value instead of selecting it.
I have 2 classes in 2 different model files
class Category(models.Model):
title = models.ForeignKey(Title, verbose_name="Title")
and
class Title(models.Model):
title = models.CharField("Title", primary_key=True, max_length=200)
When I create a category, I have to pick or write a title that already exists in the database and when I try to create a category with a new title I get this error :
Select a valid choice. That choice is not one of the available choices.
What I want to do is creating a title based on what I write in the ForeignKey field before creating the category so it can be used immediately.
I tried to redefine the save method to save the title object before saving the category but it didn't work.
Any help will be really appreciated.
Thank you

The save is performed after the form validation, you can make the category obj creation during the validation.
Have a look at the form fields' clean methods that you can override on django docs http://docs.djangoproject.com/en/dev/ref/forms/validation/#cleaning-and-validating-fields-that-depend-on-each-other

Thank you for your code, I've just tested it. But it's not exactly what I'm looking for, I will explain what I want to do.
Let's say that we have Category and Article classes in our model, each one has a title. To make this title reusable, I created another application that will manage fields, I created the class Title and I added it as foreignkey to Category and Article forms.
I switched the select box to an input field using raw_id_fields.
Now, when I create a category or an article, I have to select or write a title, when this title exists, it works perfectly but when it doesn't exist I want to create it before creating the category so it can use it.
I tried to do that in the save method, in the pre_save signal and in the clean method but I always get the error "Select a valid choice. That choice is not one of the available choices."
I'm using a hard coded solution to create the title now, I want just to see if it will work, these are the lines that I inserted in the different methods to create the title before creating the category :
t = Title(title = "MyTitle")
t.save()
I tried to create a Category with MyTitle as title but I get the same error, when I try to create another one using an existing title, it works and the title "MyTitle" is created. That's mean that the creation of the object happens after the form verification. What I want is just doing this before. The title object should be created before the verification.
Thank you very much for your help

You should probably consider putting the code to create Category entries in the model's manager:
class CategoryManager(Manager):
def create_category(category, title):
t = Title.objects.get_or_create(title=title)
return self.create(title=t)
class Category(models.Model):
title = models.ForeignKey(Title, verbose_name="Title")
objects = CategoryManager()
Then use create_category every time you want to create a Category.

converting django ForeignKey to a usable directory name

I'm working on a django app where the user will be able to upload documents of various kinds. The relevant part of my models.py is this:
class Materials(models.Model):
id = models.AutoField(primary_key=True)
id_presentations = models.ForeignKey(Presentations, db_column='id_Presentations', related_name = "materials_id_presentations") # Field name made lowercase.
materialpathname = 'documents/'
materialpathname += str(id_presentations)
document = models.FileField(db_column='Document', upload_to = materialpathname) # Field name made lowercase.
docname = models.CharField(max_length=40, db_column='DocName') # Field name made lowercase.
class Meta:
db_table = u'Materials'
My intention is for it to save the documents associated with a given presentation, in a subdirectory with the id number for that presentation (so if "Very Important Presentation" is on the database with id 3, it should store the associated materials at the location settings.MEDIA_ROOT/documents/3/whateverdocname.txt ).
However, while the above code "works", it creates a subdirectory that, instead of being named "3", is named <django.db.models.fields.related.ForeignKey object at 0x8e358ec>, or that kind of thing. I've tried using "id_presentations.name", "id_presentations.value", etc. but these attributes don't seem to exist. I can't seem to find a place where it gives a way to get at the integer value of the ForeignKey field, so that I can convert it to a string and use it as a subdirectory name.
Any help is greatly appreciated.

As of Django 1.0, the upload_to argument to FileFields can be a callable. If I'm understanding your intentions correctly, something like this should do the trick:
def material_path(instance, filename):
return 'documents/%d' % instance.id_presentations.id
class Materials(models.Model):
id_presentations = models.ForeignKey(Presentations)
document = models.FileField(upload_to=material_path)
docname = models.CharField(max_length=40)
That model has been simplified a little bit, but hopefully it illustrates the solution. If upload_to is a callable, then every time a file is uploaded Django will call the function, passing it two arguments: the instance to which the file was uploaded is attached and its original filename. You can generate the file path you want by pulling the ID of the presentation off of the instance in question.
More info:
http://docs.djangoproject.com/en/dev/ref/models/fields/#filefield

Provided that the "name" property is defined on your Presentation model, if you're working with a specific instance of the model then what you want should work. Like this:
from models import Materials
obj = Materials.objects.get([some criteria here])
name = obj.id_presentation.name
If you wanted to abstract that to a method on your model, you could do this:
class Materials(models.Model):
def id_presentation_name(self):
return self.id_presentation.name
If you want the database id of the object, you can access either object.id or object.pk.

Note: you can always find out what attributes and methods are available on an object in python by calling dir(object).