I am running Django on Heroku with zero-downtime feature. This means that during deployment there are two version of code running (old and new) on the same database. That's why we need to avoid any backward incompatible migrations.
It there a possibility to exclude a field from Django query on a given model?
Let say we have a model (version 1):
class Person(models.Model):
name = models.CharField()
address = models.TextField()
In some time in the future we want to move address to the separate table. We know that we should not delete a field for older code to work so Person model may look like (version 2):
class Person(models.Model):
name = models.CharField()
address = models.ForeignKey(Address)
_address = models.TextField(db_name='address')
This way if old code will query for address it will get it from Person table even if database has been migrated (it will be an old value, but let assume thats not a big issue).
How now I can safetly delete _address field? If we will deploy version 3 with _address field deleted then code for version 2 will still try to fetch _address on select, even if it's not used anywhere and will fail with "No such column" exception.
Is there a way to prevent this and mark some field as "non-fetchable" within the code for version 2? So version 2 will not delete field, but will not fetch it anymore and version 3 will delete field.
You can use custom object manager for defer your specific field/fields for all the queryset.
class CustomManager(models.Manager):
def get_queryset(self):
return super(CustomManager, self).get_queryset().defer('_address',)
class Person(models.Model):
name = models.CharField()
address = models.ForeignKey(Address)
_address = models.TextField(db_name='address')
objects = CustomManager()
after that in your any queryset against Person model will not include _address field in query by default.
Yes, you can do it:
QuerySet.defer():
"In some complex data-modeling situations, your models might contain a lot of fields, some of which could contain a lot of data (for example, text fields), or require expensive processing to convert them to Python objects. If you are using the results of a queryset in some situation where you don’t know if you need those particular fields when you initially fetch the data, you can tell Django not to retrieve them from the database." - docs
Entry.objects.defer("headline", "body")
OR
With django 1.8 onwards: use values_list. You can only include fields that you want. You can also use Queryset.only() and Queryset.defer() to refine your queryset queries. You can chain defer() calls as well
Entry.objects.values_list('id', 'headline')
Related
Is there a way to set foreign key relationship using the integer id of a model? This would be for optimization purposes.
For example, suppose I have an Employee model:
class Employee(models.Model):
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
type = models.ForeignKey('EmployeeType')
and
EmployeeType(models.Model):
type = models.CharField(max_length=100)
I want the flexibility of having unlimited employee types, but in the deployed application there will likely be only a single type so I'm wondering if there is a way to hardcode the id and set the relationship this way. This way I can avoid a db call to get the EmployeeType object first.
Yep:
employee = Employee(first_name="Name", last_name="Name")
employee.type_id = 4
employee.save()
ForeignKey fields store their value in an attribute with _id at the end, which you can access directly to avoid visiting the database.
The _id version of a ForeignKey is a particularly useful aspect of Django, one that everyone should know and use from time to time when appropriate.
caveat: [ < Django 2.1 ]
#RuneKaagaard points out that employee.type is not accurate afterwards in recent Django versions, even after calling employee.save() (it holds its old value). Using it would of course defeat the purpose of the above optimisation, but I would prefer an accidental extra query to being incorrect. So be careful, only use this when you are finished working on your instance (eg employee).
Note: As #humcat points out below, the bug is fixed in Django 2.1
An alternative that uses create to create the object and save it to the database in one line:
employee = Employee.objects.create(first_name='first', last_name='last', type_id=4)
class BaseCommentAbstractModel(models.Model):
"""
An abstract base class that any custom comment models probably should
subclass.
"""
# Content-object field
content_type = models.ForeignKey(ContentType,
verbose_name=_('content type'),
related_name="content_type_set_for_%(class)s")
object_pk = models.TextField(_('object ID'))
content_object = generic.GenericForeignKey(ct_field="content_type", fk_field="object_pk")
# Metadata about the comment
site = models.ForeignKey(Site)
class Meta:
abstract = True
def get_content_object_url(self):
"""
Get a URL suitable for redirecting to the content object.
"""
return urlresolvers.reverse(
"comments-url-redirect",
args=(self.content_type_id, self.object_pk)
)
I have two questions related to this model code.
models.TextField(_('object ID')) Object ID which probably is the verbose Name of this TextField ,How does it reflect in the database?
Why do Django relies on field abstract of Meta innerclass instead of using abc(AbstractBaseClass) module?
That is indeed that verbose name. I assume you understand that _ is the call to ugettext_lazy which is for localizing strings. This is the verbose name of the field. The verbose name is not represented in the database. The name of the field in the database would be object_pk.
I'm not a django dev so I can't speak with authority, but some things are obvious. ABC is new in Python 2.6. This is an issue because as of the most recent release the minimum python version was finally moved to 2.5. This has been being bumped quite quickly as of late. For example it was only on django 1.2 that python 2.4 became required. Abstract models have existed at least as far back as django 1.0 and I think even further back then that (though I can't recall for sure) So even if ABC would be suitable (which I'm not sure it is since the behavior of models is kinda complex), it wouldn't be suitable for django at this point due to the required python version.
Additionally there is some complexity in managing classes that represent the database rather than just data structures. I'm not sure how much this impacts abstract models but for example you can't perform field hiding on django attributes that are Field instances.
I'm building a personal project with Django, to train myself (because I love Django, but I miss skills). I have the basic requirements, I know Python, I carefully read the Django book twice if not thrice.
My goal is to create a simple monitoring service, with a Django-based web interface allowing me to check status of my "nodes" (servers). Each node has multiple "services". The application checks the availability of each service for each node.
My problem is that I have no idea how to represent different types of services in my database. I thought of two "solutions" :
single service model, with a "serviceType" field, and a big mess with the fields. (I have no great experience in database modeling, but this looks... "bad" to me)
multiple service models. i like this solution, but then I have no idea how I can reference these DIFFERENT services in the same field.
This is a short excerpt from my models.py file : (I removed everything that is not related to this problem)
from django.db import models
# Create your models here.
class service(models.Model):
port = models.PositiveIntegerField()
class Meta:
abstract = True
class sshService(service):
username = models.CharField(max_length=64)
pkey = models.TextField()
class telnetService(service):
username = models.CharField(max_length=64)
password = models.CharField(max_length=64)
class genericTcpService(service):
pass
class genericUdpService(service):
pass
class node(models.Model):
name = models.CharField(max_length=64)
# various fields
services = models.ManyToManyField(service)
Of course, the line with the ManyToManyField is bogus. I have no idea what to put in place of "*Service". I honestly searched for solutions about this, I heard of "generic relations", triple-join tables, but I did'nt really understand these things.
Moreover, English is not my native language, so coming to database structure and semantics, my knowledge and understanding of what I read is limited (but that's my problem)
For a start, use Django's multi-table inheritance, rather than the abstract model you have currently.
Your code would then become:
from django.db import models
class Service(models.Model):
port = models.PositiveIntegerField()
class SSHService(Service):
username = models.CharField(max_length=64)
pkey = models.TextField()
class TelnetService(Service):
username = models.CharField(max_length=64)
password = models.CharField(max_length=64)
class GenericTcpService(Service):
pass
class GenericUDPService(Service):
pass
class Node(models.Model):
name = models.CharField(max_length=64)
# various fields
services = models.ManyToManyField(Service)
On the database level, this will create a 'service' table, the rows of which will be linked via one to one relationships with separate tables for each child service.
The only difficulty with this approach is that when you do something like the following:
node = Node.objects.get(pk=node_id)
for service in node.services.all():
# Do something with the service
The 'service' objects you access in the loop will be of the parent type.
If you know what child type these will have beforehand, you can just access the child class in the following way:
from django.core.exceptions import ObjectDoesNotExist
try:
telnet_service = service.telnetservice
except (AttributeError, ObjectDoesNotExist):
# You chose the wrong child type!
telnet_service = None
If you don't know the child type beforehand, it gets a bit trickier. There are a few hacky/messy solutions, including a 'serviceType' field on the parent model, but a better way, as Joe J mentioned, is to use a 'subclassing queryset'. The InheritanceManager class from django-model-utils is probably the easiest to use. Read the documentation for it here, it's a really nice little bit of code.
I think one approach that you might consider is a "subclassing queryset". Basically, it allows you to query the parent model and it will return instances of the child models in the result queryset. It would let you do queries like:
models.service.objects.all()
and have it return to you results like the following:
[ <sshServiceInstance>, <telnetServiceInstance>, <telnetServiceInstance>, ...]
For some examples on how to do this, check out the links on the blog post linked below.
http://jazstudios.blogspot.com/2009/10/django-model-inheritance-with.html
However, if you use this approach, you shouldn't declare your service model as abstract as you do in the example. Granted, you will be introducing an extra join, but overall I've found the subclassing queryset to work pretty well for returning a mixed set of objects in a queryset.
Anyway, hope this helps,
Joe
If you are looking for generic foreign key relations you should check the Django contenttypes framework (built into Django). The docs pretty much explain how to use it and how to work with generic relations.
An actual service can only be on one node, right? In that case when not have a field
node = models.ForeignKey('node', related_name='services')
in the service class?
I'd like to set up a ForeignKey field in a django model which points to another table some of the time. But I want it to be okay to insert an id into this field which refers to an entry in the other table which might not be there. So if the row exists in the other table, I'd like to get all the benefits of the ForeignKey relationship. But if not, I'd like this treated as just a number.
Is this possible? Is this what Generic relations are for?
This question was asked a long time ago, but for newcomers there is now a built in way to handle this by setting db_constraint=False on your ForeignKey:
https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.ForeignKey.db_constraint
customer = models.ForeignKey('Customer', db_constraint=False)
or if you want to to be nullable as well as not enforcing referential integrity:
customer = models.ForeignKey('Customer', null=True, blank=True, db_constraint=False)
We use this in cases where we cannot guarantee that the relations will get created in the right order.
EDIT: update link
I'm new to Django, so I don't now if it provides what you want out-of-the-box. I thought of something like this:
from django.db import models
class YourModel(models.Model):
my_fk = models.PositiveIntegerField()
def set_fk_obj(self, obj):
my_fk = obj.id
def get_fk_obj(self):
if my_fk == None:
return None
try:
obj = YourFkModel.objects.get(pk = self.my_fk)
return obj
except YourFkModel.DoesNotExist:
return None
I don't know if you use the contrib admin app. Using PositiveIntegerField instead of ForeignKey the field would be rendered with a text field on the admin site.
This is probably as simple as declaring a ForeignKey and creating the column without actually declaring it as a FOREIGN KEY. That way, you'll get o.obj_id, o.obj will work if the object exists, and--I think--raise an exception if you try to load an object that doesn't actually exist (probably DoesNotExist).
However, I don't think there's any way to make syncdb do this for you. I found syncdb to be limiting to the point of being useless, so I bypass it entirely and create the schema with my own code. You can use syncdb to create the database, then alter the table directly, eg. ALTER TABLE tablename DROP CONSTRAINT fk_constraint_name.
You also inherently lose ON DELETE CASCADE and all referential integrity checking, of course.
To do the solution by #Glenn Maynard via South, generate an empty South migration:
python manage.py schemamigration myapp name_of_migration --empty
Edit the migration file then run it:
def forwards(self, orm):
db.delete_foreign_key('table_name', 'field_name')
def backwards(self, orm):
sql = db.foreign_key_sql('table_name', 'field_name', 'foreign_table_name', 'foreign_field_name')
db.execute(sql)
Source article
(Note: It might help if you explain why you want this. There might be a better way to approach the underlying problem.)
Is this possible?
Not with ForeignKey alone, because you're overloading the column values with two different meanings, without a reliable way of distinguishing them. (For example, what would happen if a new entry in the target table is created with a primary key matching old entries in the referencing table? What would happen to these old referencing entries when the new target entry is deleted?)
The usual ad hoc solution to this problem is to define a "type" or "tag" column alongside the foreign key, to distinguish the different meanings (but see below).
Is this what Generic relations are for?
Yes, partly.
GenericForeignKey is just a Django convenience helper for the pattern above; it pairs a foreign key with a type tag that identifies which table/model it refers to (using the model's associated ContentType; see contenttypes)
Example:
class Foo(models.Model):
other_type = models.ForeignKey('contenttypes.ContentType', null=True)
other_id = models.PositiveIntegerField()
# Optional accessor, not a stored column
other = generic.GenericForeignKey('other_type', 'other_id')
This will allow you use other like a ForeignKey, to refer to instances of your other model. (In the background, GenericForeignKey gets and sets other_type and other_id for you.)
To represent a number that isn't a reference, you would set other_type to None, and just use other_id directly. In this case, trying to access other will always return None, instead of raising DoesNotExist (or returning an unintended object, due to id collision).
tablename= columnname.ForeignKey('table', null=True, blank=True, db_constraint=False)
use this in your program
Howdy. I'm working on migrating an internal system to Django and have run into a few wrinkles.
Intro
Our current system (a billing system) tracks double-entry bookkeeping while allowing users to enter data as invoices, expenses, etc.
Base Objects
So I have two base objects/models:
JournalEntry
JournalEntryItems
defined as follows:
class JournalEntry(models.Model):
gjID = models.AutoField(primary_key=True)
date = models.DateTimeField('entry date');
memo = models.CharField(max_length=100);
class JournalEntryItem(models.Model):
journalEntryID = models.AutoField(primary_key=True)
gjID = models.ForeignKey(JournalEntry, db_column='gjID')
amount = models.DecimalField(max_digits=10,decimal_places=2)
So far, so good. It works quite smoothly on the admin side (inlines work, etc.)
On to the next section.
We then have two more models
InvoiceEntry
InvoiceEntryItem
An InvoiceEntry is a superset of / it inherits from JournalEntry, so I've been using a OneToOneField (which is what we're using in the background on our current site). That works quite smoothly too.
class InvoiceEntry(JournalEntry):
invoiceID = models.AutoField(primary_key=True, db_column='invoiceID', verbose_name='')
journalEntry = models.OneToOneField(JournalEntry, parent_link=True, db_column='gjID')
client = models.ForeignKey(Client, db_column='clientID')
datePaid = models.DateTimeField(null=True, db_column='datePaid', blank=True, verbose_name='date paid')
Where I run into problems is when trying to add an InvoiceEntryItem (which inherits from JournalEntryItem) to an inline related to InvoiceEntry. I'm getting the error:
<class 'billing.models.InvoiceEntryItem'> has more than 1 ForeignKey to <class 'billing.models.InvoiceEntry'>
The way I see it, InvoiceEntryItem has a ForeignKey directly to InvoiceEntry. And it also has an indirect ForeignKey to InvoiceEntry through the JournalEntry 1->M JournalEntryItems relationship.
Here's the code I'm using at the moment.
class InvoiceEntryItem(JournalEntryItem):
invoiceEntryID = models.AutoField(primary_key=True, db_column='invoiceEntryID', verbose_name='')
invoiceEntry = models.ForeignKey(InvoiceEntry, related_name='invoiceEntries', db_column='invoiceID')
journalEntryItem = models.OneToOneField(JournalEntryItem, db_column='journalEntryID')
I've tried removing the journalEntryItem OneToOneField. Doing that then removes my ability to retrieve the dollar amount for this particular InvoiceEntryItem (which is only stored in journalEntryItem).
I've also tried removing the invoiceEntry ForeignKey relationship. Doing that removes the relationship that allows me to see the InvoiceEntry 1->M InvoiceEntryItems in the admin inline. All I see are blank fields (instead of the actual data that is currently stored in the DB).
It seems like option 2 is closer to what I want to do. But my inexperience with Django seems to be limiting me. I might be able to filter the larger pool of journal entries to see just invoice entries. But it would be really handy to think of these solely as invoices (instead of a subset of journal entries).
Any thoughts on how to do what I'm after?
First, inheriting from a model creates an automatic OneToOneField in the inherited model towards the parents so you don't need to add them. Remove them if you really want to use this form of model inheritance.
If you only want to share the member of the model, you can use Meta inheritance which will create the inherited columns in the table of your inherited model. This way would separate your JournalEntry in 2 tables though but it would be easy to retrieve only the invoices.
All fields in the superclass also exist on the subclass, so having an explicit relation is unnecessary.
Model inheritance in Django is terrible. Don't use it. Python doesn't need it anyway.