Howdy. I'm working on migrating an internal system to Django and have run into a few wrinkles.
Intro
Our current system (a billing system) tracks double-entry bookkeeping while allowing users to enter data as invoices, expenses, etc.
Base Objects
So I have two base objects/models:
JournalEntry
JournalEntryItems
defined as follows:
class JournalEntry(models.Model):
gjID = models.AutoField(primary_key=True)
date = models.DateTimeField('entry date');
memo = models.CharField(max_length=100);
class JournalEntryItem(models.Model):
journalEntryID = models.AutoField(primary_key=True)
gjID = models.ForeignKey(JournalEntry, db_column='gjID')
amount = models.DecimalField(max_digits=10,decimal_places=2)
So far, so good. It works quite smoothly on the admin side (inlines work, etc.)
On to the next section.
We then have two more models
InvoiceEntry
InvoiceEntryItem
An InvoiceEntry is a superset of / it inherits from JournalEntry, so I've been using a OneToOneField (which is what we're using in the background on our current site). That works quite smoothly too.
class InvoiceEntry(JournalEntry):
invoiceID = models.AutoField(primary_key=True, db_column='invoiceID', verbose_name='')
journalEntry = models.OneToOneField(JournalEntry, parent_link=True, db_column='gjID')
client = models.ForeignKey(Client, db_column='clientID')
datePaid = models.DateTimeField(null=True, db_column='datePaid', blank=True, verbose_name='date paid')
Where I run into problems is when trying to add an InvoiceEntryItem (which inherits from JournalEntryItem) to an inline related to InvoiceEntry. I'm getting the error:
<class 'billing.models.InvoiceEntryItem'> has more than 1 ForeignKey to <class 'billing.models.InvoiceEntry'>
The way I see it, InvoiceEntryItem has a ForeignKey directly to InvoiceEntry. And it also has an indirect ForeignKey to InvoiceEntry through the JournalEntry 1->M JournalEntryItems relationship.
Here's the code I'm using at the moment.
class InvoiceEntryItem(JournalEntryItem):
invoiceEntryID = models.AutoField(primary_key=True, db_column='invoiceEntryID', verbose_name='')
invoiceEntry = models.ForeignKey(InvoiceEntry, related_name='invoiceEntries', db_column='invoiceID')
journalEntryItem = models.OneToOneField(JournalEntryItem, db_column='journalEntryID')
I've tried removing the journalEntryItem OneToOneField. Doing that then removes my ability to retrieve the dollar amount for this particular InvoiceEntryItem (which is only stored in journalEntryItem).
I've also tried removing the invoiceEntry ForeignKey relationship. Doing that removes the relationship that allows me to see the InvoiceEntry 1->M InvoiceEntryItems in the admin inline. All I see are blank fields (instead of the actual data that is currently stored in the DB).
It seems like option 2 is closer to what I want to do. But my inexperience with Django seems to be limiting me. I might be able to filter the larger pool of journal entries to see just invoice entries. But it would be really handy to think of these solely as invoices (instead of a subset of journal entries).
Any thoughts on how to do what I'm after?
First, inheriting from a model creates an automatic OneToOneField in the inherited model towards the parents so you don't need to add them. Remove them if you really want to use this form of model inheritance.
If you only want to share the member of the model, you can use Meta inheritance which will create the inherited columns in the table of your inherited model. This way would separate your JournalEntry in 2 tables though but it would be easy to retrieve only the invoices.
All fields in the superclass also exist on the subclass, so having an explicit relation is unnecessary.
Model inheritance in Django is terrible. Don't use it. Python doesn't need it anyway.
Related
I am running Django on Heroku with zero-downtime feature. This means that during deployment there are two version of code running (old and new) on the same database. That's why we need to avoid any backward incompatible migrations.
It there a possibility to exclude a field from Django query on a given model?
Let say we have a model (version 1):
class Person(models.Model):
name = models.CharField()
address = models.TextField()
In some time in the future we want to move address to the separate table. We know that we should not delete a field for older code to work so Person model may look like (version 2):
class Person(models.Model):
name = models.CharField()
address = models.ForeignKey(Address)
_address = models.TextField(db_name='address')
This way if old code will query for address it will get it from Person table even if database has been migrated (it will be an old value, but let assume thats not a big issue).
How now I can safetly delete _address field? If we will deploy version 3 with _address field deleted then code for version 2 will still try to fetch _address on select, even if it's not used anywhere and will fail with "No such column" exception.
Is there a way to prevent this and mark some field as "non-fetchable" within the code for version 2? So version 2 will not delete field, but will not fetch it anymore and version 3 will delete field.
You can use custom object manager for defer your specific field/fields for all the queryset.
class CustomManager(models.Manager):
def get_queryset(self):
return super(CustomManager, self).get_queryset().defer('_address',)
class Person(models.Model):
name = models.CharField()
address = models.ForeignKey(Address)
_address = models.TextField(db_name='address')
objects = CustomManager()
after that in your any queryset against Person model will not include _address field in query by default.
Yes, you can do it:
QuerySet.defer():
"In some complex data-modeling situations, your models might contain a lot of fields, some of which could contain a lot of data (for example, text fields), or require expensive processing to convert them to Python objects. If you are using the results of a queryset in some situation where you don’t know if you need those particular fields when you initially fetch the data, you can tell Django not to retrieve them from the database." - docs
Entry.objects.defer("headline", "body")
OR
With django 1.8 onwards: use values_list. You can only include fields that you want. You can also use Queryset.only() and Queryset.defer() to refine your queryset queries. You can chain defer() calls as well
Entry.objects.values_list('id', 'headline')
I'm trying to learn Google App Engine's NDB and I'm confused about the structure of models.
My situation is similar to a CMS platform with Post Types (like in WordPress), so I have "Blogs" and "Pages". All of these Post Types require the same set of attributes: Parent, Name, Slug, Template, Content, Status, and Date.
So far, I gather that I need to create a Model for these like this:
class Post(ndb.Expando):
parent = ndb.StringProperty()
name = ndb.StringProperty()
slug = ndb.StringProperty()
template = ndb.StringProperty()
content = ndb.StringProperty(indexed=False)
status = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
(I'm using Expando because I will be adding "unknown" attributes in my application)
But with this structure, all of my posts (in every Post Type) will be within the same "kind", so queries will take longer (if I'm not mistaken).
How can I create many Models (kinds) with the same attributes?
Do I copy & paste the above Model under different class names?
Is it possible to create new Models dynamically (similar to "Custom Post Types" in WordPress)? Does it work if I use ndb.Key('Blog', blogid) instead of declaring a Model?
Do I create a Model called class PostType(ndb.Model) that stores the "Post Types" and give them ancestors of Posts? (If I'm not mistaken, this would cause problems because updating a Post would "lock" the entire ancestor tree for a second or so)
My primary goal is efficiency. Thanks!
Updates:
As written by Dan and mgilson, adding sub-classes of the main Post class Model is a good way to solve this:
class Post(ndb.Expando):
parent = ndb.StringProperty()
name = ndb.StringProperty()
slug = ndb.StringProperty()
template = ndb.StringProperty()
content = ndb.StringProperty(indexed=False)
status = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
class Blog(Post):
pass
However, this requires writing the Models statically. Is there a way to accomplish this dynamically (without declaring them as Models beforehand)?
Update:
Following the advice given below, I decided to keep all of my entities under the same kind. I might decide later on to change this to subclasses (separate kinds for each "Post Type") if my queries get messy. Thank you all for your great advice!
How can I create many Models with the same attributes?
You can subclass:
class SpecialPost(Post):
"""Special post type that is a different kind than Post."""
Though it's often easy enough to use the same kind and just add an extra field that represents the kind of post which you can filter on in queries.
Is it possible to create new Models dynamically (similar to "Custom Post Types" in WordPress)? Does it work if I use ndb.Key('Blog', blogid) instead of declaring a Model?
I'm not 100% sure that I understand what you're asking here. You can dynamically create models the same way you can dynamically create classes in python (using type), but you probably don't want to be doing this. Getting those dynamically created models (and keeping track of their names) will probably end up giving you serious headaches.
Basically a simple example of subclassing, which #mgilson mentioned already.
class Post(ndb.Expando):
parent = ndb.StringProperty()
name = ndb.StringProperty()
slug = ndb.StringProperty()
template = ndb.StringProperty()
content = ndb.StringProperty(indexed=False)
status = ndb.StringProperty()
date = ndb.DateTimeProperty(auto_now_add=True)
class Blog(Post):
someint = ndb.IntegerProperty()
blog = Blog(status='new', someint=2)
key = blog.put()
print key.kind()
As for dynamically creating models, from the Model's Constructor doc:
An application won't normally call Model(), but is likely to call the
constructor of a class that inherits from Model. This creates a new
instance of this model, also known as an entity.
Even if possible (I didn't dig too deep inside ndb/models.py to say with certainty that it's not) it doesn't appear a clear thing. Personally I'd stay away from that and instead re-think the need for such dynamically created models.
When dealing whith model inheritance in django is it possible to remove a instance of model subclass, without removing the superclass itself?
Using the Django example, can you remove just the Resturaunt object and retain the Place object?
Yesterday I was looking for an answer to this question and I came up with this solution, which was enough for my problem but could be scaled up as needed.
Assuming you have a Restaurant and a Place django models, the way to delete a restaurant only without touching the row inside the Place's table is creating a "fake" Restaurant model like this:
class FakeRestaurant(models.Model):
place_ptr = models.PositiveIntegerField(db_column="place_ptr_id", primary_key=True)
serves_hot_dogs = models.BooleanField()
serves_pizza = models.BooleanField()
class Meta:
app_label = Restaurant._meta.app_label
db_table = Restaurant._meta.db_table
managed = False
Now, you can retrieve objects from that table as if it had no bound external relationship:
place = Place.objects.get(pk=1)
restaurant = Restaurant.objects.get(pk=1)
fake_restaurant = FakeRestaurant.objects.get(pk=1)
fake_restaurant.delete()
fake_restaurant and restaurant won't exist anymore, place will remain untouched.
Cheers,
Davide
In Django 1.9 parameter keep_parents was added to model delete() function, so to keep parents just call:
restaurant.delete(keep_parents=True)
Docs: https://docs.djangoproject.com/en/1.10/ref/models/instances/#django.db.models.Model.delete
UPDATE:
Apparently, this feature is not working properly in Django 1.9, please see the comments.
I'm building a personal project with Django, to train myself (because I love Django, but I miss skills). I have the basic requirements, I know Python, I carefully read the Django book twice if not thrice.
My goal is to create a simple monitoring service, with a Django-based web interface allowing me to check status of my "nodes" (servers). Each node has multiple "services". The application checks the availability of each service for each node.
My problem is that I have no idea how to represent different types of services in my database. I thought of two "solutions" :
single service model, with a "serviceType" field, and a big mess with the fields. (I have no great experience in database modeling, but this looks... "bad" to me)
multiple service models. i like this solution, but then I have no idea how I can reference these DIFFERENT services in the same field.
This is a short excerpt from my models.py file : (I removed everything that is not related to this problem)
from django.db import models
# Create your models here.
class service(models.Model):
port = models.PositiveIntegerField()
class Meta:
abstract = True
class sshService(service):
username = models.CharField(max_length=64)
pkey = models.TextField()
class telnetService(service):
username = models.CharField(max_length=64)
password = models.CharField(max_length=64)
class genericTcpService(service):
pass
class genericUdpService(service):
pass
class node(models.Model):
name = models.CharField(max_length=64)
# various fields
services = models.ManyToManyField(service)
Of course, the line with the ManyToManyField is bogus. I have no idea what to put in place of "*Service". I honestly searched for solutions about this, I heard of "generic relations", triple-join tables, but I did'nt really understand these things.
Moreover, English is not my native language, so coming to database structure and semantics, my knowledge and understanding of what I read is limited (but that's my problem)
For a start, use Django's multi-table inheritance, rather than the abstract model you have currently.
Your code would then become:
from django.db import models
class Service(models.Model):
port = models.PositiveIntegerField()
class SSHService(Service):
username = models.CharField(max_length=64)
pkey = models.TextField()
class TelnetService(Service):
username = models.CharField(max_length=64)
password = models.CharField(max_length=64)
class GenericTcpService(Service):
pass
class GenericUDPService(Service):
pass
class Node(models.Model):
name = models.CharField(max_length=64)
# various fields
services = models.ManyToManyField(Service)
On the database level, this will create a 'service' table, the rows of which will be linked via one to one relationships with separate tables for each child service.
The only difficulty with this approach is that when you do something like the following:
node = Node.objects.get(pk=node_id)
for service in node.services.all():
# Do something with the service
The 'service' objects you access in the loop will be of the parent type.
If you know what child type these will have beforehand, you can just access the child class in the following way:
from django.core.exceptions import ObjectDoesNotExist
try:
telnet_service = service.telnetservice
except (AttributeError, ObjectDoesNotExist):
# You chose the wrong child type!
telnet_service = None
If you don't know the child type beforehand, it gets a bit trickier. There are a few hacky/messy solutions, including a 'serviceType' field on the parent model, but a better way, as Joe J mentioned, is to use a 'subclassing queryset'. The InheritanceManager class from django-model-utils is probably the easiest to use. Read the documentation for it here, it's a really nice little bit of code.
I think one approach that you might consider is a "subclassing queryset". Basically, it allows you to query the parent model and it will return instances of the child models in the result queryset. It would let you do queries like:
models.service.objects.all()
and have it return to you results like the following:
[ <sshServiceInstance>, <telnetServiceInstance>, <telnetServiceInstance>, ...]
For some examples on how to do this, check out the links on the blog post linked below.
http://jazstudios.blogspot.com/2009/10/django-model-inheritance-with.html
However, if you use this approach, you shouldn't declare your service model as abstract as you do in the example. Granted, you will be introducing an extra join, but overall I've found the subclassing queryset to work pretty well for returning a mixed set of objects in a queryset.
Anyway, hope this helps,
Joe
If you are looking for generic foreign key relations you should check the Django contenttypes framework (built into Django). The docs pretty much explain how to use it and how to work with generic relations.
An actual service can only be on one node, right? In that case when not have a field
node = models.ForeignKey('node', related_name='services')
in the service class?
I'd like to set up a ForeignKey field in a django model which points to another table some of the time. But I want it to be okay to insert an id into this field which refers to an entry in the other table which might not be there. So if the row exists in the other table, I'd like to get all the benefits of the ForeignKey relationship. But if not, I'd like this treated as just a number.
Is this possible? Is this what Generic relations are for?
This question was asked a long time ago, but for newcomers there is now a built in way to handle this by setting db_constraint=False on your ForeignKey:
https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.ForeignKey.db_constraint
customer = models.ForeignKey('Customer', db_constraint=False)
or if you want to to be nullable as well as not enforcing referential integrity:
customer = models.ForeignKey('Customer', null=True, blank=True, db_constraint=False)
We use this in cases where we cannot guarantee that the relations will get created in the right order.
EDIT: update link
I'm new to Django, so I don't now if it provides what you want out-of-the-box. I thought of something like this:
from django.db import models
class YourModel(models.Model):
my_fk = models.PositiveIntegerField()
def set_fk_obj(self, obj):
my_fk = obj.id
def get_fk_obj(self):
if my_fk == None:
return None
try:
obj = YourFkModel.objects.get(pk = self.my_fk)
return obj
except YourFkModel.DoesNotExist:
return None
I don't know if you use the contrib admin app. Using PositiveIntegerField instead of ForeignKey the field would be rendered with a text field on the admin site.
This is probably as simple as declaring a ForeignKey and creating the column without actually declaring it as a FOREIGN KEY. That way, you'll get o.obj_id, o.obj will work if the object exists, and--I think--raise an exception if you try to load an object that doesn't actually exist (probably DoesNotExist).
However, I don't think there's any way to make syncdb do this for you. I found syncdb to be limiting to the point of being useless, so I bypass it entirely and create the schema with my own code. You can use syncdb to create the database, then alter the table directly, eg. ALTER TABLE tablename DROP CONSTRAINT fk_constraint_name.
You also inherently lose ON DELETE CASCADE and all referential integrity checking, of course.
To do the solution by #Glenn Maynard via South, generate an empty South migration:
python manage.py schemamigration myapp name_of_migration --empty
Edit the migration file then run it:
def forwards(self, orm):
db.delete_foreign_key('table_name', 'field_name')
def backwards(self, orm):
sql = db.foreign_key_sql('table_name', 'field_name', 'foreign_table_name', 'foreign_field_name')
db.execute(sql)
Source article
(Note: It might help if you explain why you want this. There might be a better way to approach the underlying problem.)
Is this possible?
Not with ForeignKey alone, because you're overloading the column values with two different meanings, without a reliable way of distinguishing them. (For example, what would happen if a new entry in the target table is created with a primary key matching old entries in the referencing table? What would happen to these old referencing entries when the new target entry is deleted?)
The usual ad hoc solution to this problem is to define a "type" or "tag" column alongside the foreign key, to distinguish the different meanings (but see below).
Is this what Generic relations are for?
Yes, partly.
GenericForeignKey is just a Django convenience helper for the pattern above; it pairs a foreign key with a type tag that identifies which table/model it refers to (using the model's associated ContentType; see contenttypes)
Example:
class Foo(models.Model):
other_type = models.ForeignKey('contenttypes.ContentType', null=True)
other_id = models.PositiveIntegerField()
# Optional accessor, not a stored column
other = generic.GenericForeignKey('other_type', 'other_id')
This will allow you use other like a ForeignKey, to refer to instances of your other model. (In the background, GenericForeignKey gets and sets other_type and other_id for you.)
To represent a number that isn't a reference, you would set other_type to None, and just use other_id directly. In this case, trying to access other will always return None, instead of raising DoesNotExist (or returning an unintended object, due to id collision).
tablename= columnname.ForeignKey('table', null=True, blank=True, db_constraint=False)
use this in your program