I have a model with as many as 20 fields. It is also referenced as ManytoMany in another model which references it using a through table. Let me put a scenario here showing my case.
class Class1(models.Model):
some_field = .....
myfield1 = models.ManyToManyField(Class2,through='Another')
......
class Another(models.Model):
class1 = models.ForeignKey(Class1, related_name='class1_class2')
class2 = models.ForeignKey(Class2, related_name='class1_class2')
"Another" is an admin inline field, using default Admin UI of Django. The problem is that if there are too many objects of "Another" which loads lot of other objects of class1 and class2, NGINX gives me 502: Bad Gateway.
I am not willing to increase the NGINX time, I have already done that many times. What I want to know is that, if there is a way I can say Django Admin to load the inlines only after all other contents are loaded, or say Lazy Load the inlines.
I have gone through almost every post that says Lazy Loading in Django, but it all applies to a particular view or a field, I found nothing close to what I need.
I would be very appreciable if anyone can shed some light on this.
Regards.
Using defer may give you what you need...
https://docs.djangoproject.com/en/dev/ref/models/querysets/#defer
defer
defer(*fields)
In some complex data-modeling situations, your models might contain a lot of fields, some of which could contain a lot of data (for example, text fields), or require expensive processing to convert them to Python objects. If you are using the results of a queryset in some situation where you don’t know if you need those particular fields when you initially fetch the data, you can tell Django not to retrieve them from the database.
This is done by passing the names of the fields to not load to defer():
Entry.objects.defer("headline", "body")
A queryset that has deferred fields will still return model instances. Each deferred field will be retrieved from the database if you access that field (one at a time, not all the deferred fields at once).
You can make multiple calls to defer(). Each call adds new fields to the deferred set:
Defers both the body and headline fields.
Entry.objects.defer("body").filter(rating=5).defer("headline")
The order in which fields are added to the deferred set does not matter. Calling defer() with a field name that has already been deferred is harmless (the field will still be deferred).
You can defer loading of fields in related models (if the related models are loading via select_related()) by using the standard double-underscore notation to separate related fields:
Blog.objects.select_related().defer("entry__headline", "entry__body")
If you want to clear the set of deferred fields, pass None as a parameter to defer():
Load all fields immediately.
my_queryset.defer(None)
Some fields in a model won’t be deferred, even if you ask for them. You can never defer the loading of the primary key. If you are using select_related() to retrieve related models, you shouldn’t defer the loading of the field that connects from the primary model to the related one, doing so will result in an error.
Related
I am looking for a reliable way to ensure that for a given instance of a django model, a specific field is written only once.
The code should run in both views or tasks using celery or even rq.
I am thinking about using the following snippet :
from django.db import transaction
from django.utils.timezone import now
...
def perform_writeonce(object_pk):
with transaction.atomic():
instance = MyModel.objects.get(pk=object_pk)
if instance.value is None:
instance.value = 'Value written now : {}'.format(now())
instance.save()
The objective is that value should only be written once and only if its value is None.
Are you confident in this snippet or is there something I missed ?
I am using multiple gunicorn instances and celery workers with a single postgresql database server.
I am trying to avoid locks at all costs, so any solution with application-land locks are no good.
Thanks for any help.
I am looking for a reliable way to ensure that for any given instance of a django model, a specific field is written only once.
The expected behavior is like the auto_now_add option to datefields.
auto_now_add does not ensure that a field is written only once. You can still write to that field if you really want to. It just doesn't show up by default in admin forms etc.
You can do the same for other fields by setting Field.editable
https://docs.djangoproject.com/en/1.10/ref/models/fields/#editable
Field.editable
If False, the field will not be displayed in the admin or any other
ModelForm. They are also skipped during model validation. Default is
True.
I am currently using Django framework including its Models mechanism to abstract the database schema declaration and general db access, which is working fine for most scenarios.
However, my application also requires tables to be created and accessed dynamically during runtime, which as far as I can see, is not supported by Django out of the box.
These tables usually have an identical structure, and can basically be abstracted by the same Model class, but Django doesn't let you change the underlying db_table of a certain model query, as it is declared on the Model class and not on the Manager.
My solution for this is to do this process whenever I need a new table to be created, populated and accessed:
Create and populate the table using raw sql
Add indexes to the table using raw sql
When I need to access the table (using django queryset api), I declare a new type dynamically and return it as the model for the query, by using this code:
table_name = # name of the table created by sql
model_name = '%d_%s' % (connection.tenant.id, table_name)
try:
model = apps.get_registered_model('myapp', model_name)
return model
except LookupError:
pass
logger.debug("no model exists for model %s, creating one" % model_name)
class Meta:
db_table = table_name
managed = False
attrs = {
'field1' : models.CharField(max_length=200),
'field2' : models.CharField(max_length=200),
'field3' : models.CharField(max_length=200)
'__module__': 'myapp.models',
'Meta':Meta
}
model = type(str(model_name), (models.Model,), attrs)
return model
Note that I do check if the model is already registered in django and I'm using an existing model in case it does. The model name is always unique for each table. Since I'm using multi tenants, the tenant name is also part of the model name to avoid conflict with similar tables declared on different schemas.
In case it's not clear: the tables created dynamically will and should be persisted permanently for future sessions.
This solution works fine for me so far.
However, the application will need to support a large number of these tables. i.e. 10,000 - 100,000 such tables(and corresponding model classes), with up to a million rows per table.
Assuming the underlying db is fine with this load, my questions are:
Do you see any problem with this solution, with and without regards to the expected scale ?
Anybody has a better solution for this scenario ?
Thanks.
There is a wiki page on creating models dynamically, although it has been a while since it was last updated:
DynamicModels Django
There are also a few apps that are designed for this use case, but I don't think any of them is being actively maintained:
Django Packages: Dynamic models
I understand that if you are already committed to Django this isn't very helpful, but this a use case for which Django isn't really good. It might be more costly to fight against the abstractions provided by Django's model layer, than to just use psycopg2 or whatever other adapter is appropriate for your data.
Depending on what sort of operations you are going to perform on your data, it may be also more reasonable to use a single model with an indexed field that allows you to distinguish in which table that row would be and then sharding the data by that column.
If you still need to do this, the general idea would be:
Create a metaclass that extends Django's ModelBase. This metaclass you would use as a factory for your actual models.
Consider stuff mentioned on that wiki page, like circumventing the app_label issue.
Generate and execute the sql for the creation of the model as also shown on the wiki page.
I'm really really confused about how django handles database relationships.
Originally I had an article model that contained a simple IntegerField for article_views, recently I'm trying to expand the definition of a article_view to contain it's own fields so I created a model for it. (IP, SESSION KEY etc..)
I'm at a bit of a loss regarding how to make the relationship, to me it makes the most sense to have a one-to-many field inside the article model, because an article can have many different views, but a view can only be part of one article.
all the implementations I'm seeing have this set up in a really weird reverse manner, what gives?
Unfortunately Django does not have a One-to-Many field. This is achieved by creating a ForeignKey on in this case the ArticleView model. When you want to easily access the article views in your template you can set the related_name on the ForeignKey.
class Article(models.Model):
# Article definition
class ArticleView(models.Model):
article = models.ForeignKey(Article, related_name='views')
In the template you can now use article.views.count() to get the number of views coupled to an account.
Please note that this creates a database query for each count you want. It would probably be better to have a queryset with annotate: Article.objects.annotate(num_views=Count('views'))
We have a django application that is, at its core, a series of webpages with Forms which our users fill out in order. (We'll call the particular series of pages with forms on them a "flow".)
We will be white-labeling this application for a Partner -- the Partner will want to add some fields and even add some webpages with their own new Forms. This may result in a new order in which the Forms are filled out. (A new "flow", in addition to changes to existing Forms/Models or new Forms/Models.)
What is the best way to extend our existing, simple Forms-and-Models structure to use different Forms and Models depending on the running instance of the app (e.g. an environment variable)? Some things we thought about:
implement something like get_user_model for every Model and Form use in the app, which would look at the current environment
implement a more generic key-value store so that we're not bound by the current implementation's field types (i.e., have the data field name be part of the data as well)
a data model which tracks this particular environment's "flow" and which models it needs to use
subclass existing Models and Forms for each new white-label implementation
Model Field injection may be what you are looking for, take a look of this article
The approach boils down to three concepts:
Dynamically adding fields to model classes Ensuring Django’s model
system respects the new fields
Getting the load ordering correct for the above to work
Mezzanine has done a beautiful job implementing this model field injection with dynamic extra models via EXTRA_MODEL_FIELDS
I'm currently working on a model that has been already built and i need to add some validation managment. (accessing to two fields and checking data, nothing too dramatic)
I was wondering about the exact difference between models and forms at a validation point of view and if i would be able to just make a clean method raising errors as in a formview in a model view ?
for extra knowledge, why are thoses two things separated ?
And finnaly, what would you do ? There are already some methods written for the model and i don't know yet if i would rewrite it to morph it into a form and simply add the clean() method + i don't exactly know how they work.
Oh, and everything is in the admin interface, havn't yet worked a lot on it since i started django not so long ago.
Thanks in advance,
You should use model (field) validation to make sure the returning datatype meets your database's requirements. Usually you won't need this because django's builtin fields do this for you, so unless you've built some custom field or know what you are doing you shouldn't change things.
Form validation is where you clean the user's input, you can add a clean method for every form field by adding a clean_FIELD(self) method, e.g.
class ContactForm(forms.Form):
# Everything as before.
...
def clean_recipients(self):
data = self.cleaned_data['recipients']
if "fred#example.com" not in data:
raise forms.ValidationError("You have forgotten about Fred!")
# Always return the cleaned data, whether you have changed it or
# not.
return data
Before a Form's main clean method is ran, it checks for a field level clean for each of its fields
Generally models represent business entities which may be stored in some persistent storage (usually relational DB). Forms are used to render HTML forms which may retreive data from users.
Django supports creating forms on the basis of models (using ModelForm class). Forms may be used to fetch data which should be saved in persistent storage, but that's not only the case - one may use forms just to get data to be searched in persistent storage or passed to external service, feed some application counters, test web browser engines, render some text on the basis of data entered by user (e.g. "Hello USERNAME"), login user etc.
Calling save() on model instance should guarantee that data will be saved in persistent storage if and only data is valid - that will provide consistent mechanism of validation of data before saving to persistent storage, regardless whether business entity is to be saved after user clicks "Save me" button on web page or in django interactive shell user will execute save() method of model instance.