The Problem
I'm working on a django project in which I need to share the same database models over multiple apps, for example when a user opens a page the page is displayed depending on user settings, which can be changed by the user in a different app than the one which displayes the page
At the moment I've made the browser app which contains most of my models (because it's the one I started with) and I've got other apps like watch_file and user_settings which then import
This is working but I'm noticing that I'm running into organization problems where I'm hopping from file to file to check models and I'm importing all over the place...
My potential Solution
I was thinking about making one big model file somewhere and just importing the models I need for every app in it's own model file, however, I read this question in which someone in the comments stated that generally this is not done
Also I red this question which said that sharing models between apps isn't a good idea at all in regards to organization
The question
In regards to it not being a good idea to make a general model file for all models my project is using, How would I go about organizing it so I don't have to import so many models from different files for an app to work?
Could I create a something like a helper app which isn't displayed but is only used to help other apps by having all models in it's models.py file and importing it frome here in other apps?
NB : when talking about "apps" in the following, I mean "project-specific apps", not "reusable apps".
It's perfectly legal (technically) and hardly avoidable in any non-trivial project to have inter-packages dependencies (Django "apps" are first and foremost python packages), so having one app importing models from another app is definitly not a problem in and by itself. Where it becomes a problem is when you start having circular dependencies (A depends on B which depends on C which depends on A...)
A pattern that usually works fine is to have a "core" app on which all other apps can depend but that must not depends on any other app (except contribs app or third-part apps of course), and a "main" app that can depend on any app but that no other app is allowed to depend on. The core app provides the "foundations" for your project - the basic building blocks that all the project will need - and the main app (which usually has no models at all) is responsible for "top-level integration" of everthing (IOW it's where you put everything that doesn't clearly belongs to another app and "bridges" the other apps features together).
Then you try to keep other apps mostly self-contained except for dependencies on contribs apps, third-part apps and the core app.
Also note that an app does not necessarily have models AND views AND templates AND templatetags etc etc - you can have simple apps defining a couple models or utils, a "frontend" app (somehow similar to the "main" app in concept actually) that's responsible for all your project's views, an "api" app providing a REST api over your other apps models, etc etc.
Related
After some years developing web apps using ruby on rails, I decided to give Django a try, however it seems that I'm missing something, which is how to structure large project, or any project in general.
For example, in rails we have a models folder which contains model classes, each in a separate ruby file, a controllers folder which contains controller classes, again each in a separate ruby file.
However, in Django it split the project into independent apps, which can be installed independently in other Django project, each app has a models.py file which contains all the models classes, a views.py file which contain all the views functions.
But then how to group functions in views like rails? That is one controller per each model.
In general how to structure my project when it contains one large app that can't be separated into multiple independent apps? I want for example to have a view index function for each model, but how to do this if all functions are in one file?
If my project is about selling cars for example. I should have index function that maps to /cars, another index function to map to /users, etc...
I searched the web but couldn't find a suitable answer.
It is unclear to me how to structure Django app, so any help will be appreciated.
As mentioned in #shanksfk's answer, Django is very flexible in folder arrangements. You don't have to follow the default app structure. When I create a purely backend Django project (with DRF), I usually have 3 base apps:
api - where modules, serializers, and urls are stored
core - the default app (the one that has the name of your Django project)
db - where models are stored
Then as I expand, I can add a folder dedicated for the helpers, utils, and possibly abstraction layers for external services. I recommend reading more about Domain-driven Design to get an idea on how to structure your project. You can also check other Django projects for inspiration:
django CMS
Baserow
Django API Domains
In short, Django is a Model-View-Template framework and Rails is a Model-View-Controller framework.
In Django we store controllers(sort of) in views.py for each specified app, while in MVC framework such as Rails store it in controllers. In Django, you also have to create your own HTML template separately which some people may find it tedious but its easier to work with other frameworks such as Vue or React due to that separation.
This is general comparison I found on the net.
However, to answer your questions on folder structure. Basically Django is very flexible on folder arrangements, it really depends how you want to design the project structure. Normally what I'd do is keep every app in the main folder (project folder). This way you wont mess with the venv setup
I have been using django for a while, but there is something im not quite clear on.
How big should a django app be. If for example a django app should only be user authentication or if it should be an entire website in one app.
If I have a project with several apps and each app is a whole website with a lot of code, is that the way it suppose to be or should all apps related to a single site within a project ?
Im thinking of creating one django project for each site, but im now wondering if I should be creating one project where each app is one site. Can anyone please comment on this, what it the preferred way to do it ?
In the django documentation one app is only used for a poll, so it seems to be that, according to the documentation, that each app should be some part of functionality on the site.
An app is a submodule of a project which contains functionality (views, models, urls etc) for a specific part of the larger site and is as decoupled from the other apps as possible. The project as a whole is the website and your apps make up the separate parts of functionality for your site.
If your sites are going to basically contain the same functionality, it might be worth looking into the Sites framework which django provides.
So taking the polls example further.
Lets say the website is a survey site. There would be the polls app, which would contain the relevant models and views for creating and recording poll results. Then you might need to view the data, so you could create an analysis app, which would store the views for displaying dashboards and contain functions for data processing. Then we could take things a bit further, and have users be able to log in and see their own results (and give us the chance to link users to poll results), so you would make an accounts app, which would hold views for logging in/out, maybe a profile page etc.
So each of these different parts of functionality would be separated out into distinct apps, which would make up the project (site) as a whole.
If the apps have been decoupled properly, you could reuse the different apps in other projects (e.g. the accounts app could be dropped into a new project do provide logging in/out functionality)
In my experience so far, an app should be a reusable entity. There many guiding principles for choosing what to go and what not in an app. Separating authentication is one example. A projcet is one big collection of may apps and a reusable app can be in many projects.
Nowadays, there is a trend to move to micro service architecture, which is next level of separation of functionality with each service doing best one thing.
Each project is a unit in itself, not an individual app. So you host the project, not the app. I recommend using different project for each site and using micro-service architecture. A lot depends on your existing codebase too.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
Note: I am not a proper python programmer... but I use python extensively. I do things like write classes with inheritance, use iterators and comprehension, etc. My point is that I do not have a full grasp of the language, e.g. what exactly constitutes an python object, why __init__.py is needed other than to specify a module, etc. In relation to Django, I have written multi-app sites (with the help of S.O.) and have really enjoyed Django's templating system, blocks, and how they can be nested. Now are my apps fully decoupled and reusable? That this is subject of this post.
I state this disclaimer because a lot of the Django resources seem to assume that one knows these things. This makes understanding some of the documentation and S.O. questions difficult for a person who is just an (subpower)-user. So please answer this question with that in mind.
Question
These questions are inspired by both the question When to create a new app with startapp in django? by #håkan and the answer given by #antti rasinen which links to James Bennett's 2008 PyCon presentation
A few key points from Bennett's presentation are:
sites are a collection of apps
an app does one thing and one thing well
Which directs me to his section "Project coupling kills re-use" that mentions:
Single module directly on Python path (registration, tagging, etc.)
Related modules under a package (ellington.events, ellington.podcasts, etc.)
Question 0
A "module" in this case is just an app made of other apps?
Question 1
(Apps with related functionality and shared models )
What should I do when apps share models?
In Barrett's slides he implies that user registration and user profiles are distinct and should be distinct apps. (He certainly states that profiles have nothing to do with user registration).
So if I wanted both, would my project have two apps like:
user-registration
user-profile
even though the app user-profile will need the user model from user-registration? Or do I make a single app (module):
user-app
registration
profile
which contains both?
Question 2
(Apps with distinct functions but shared models)
Extending the example from question 1, lets say that my main app (or some other app that is used by the main app) utilizes some aspect of the user model (e.g. recently active members if it was a chat site).
Clearly my main app gets this information from the user model. Does my main app now get bundled under the user-app module?
This may not be the best example, but the point is as follows:
I have two apps app-dependency and app-needs-dependency, where each app does its one thing and one thing well... It is just that app-needs-dependency needs information from app-dependency. What do I do in this case, if everything else about app-needs-dependency is completely decoupled from app-dependency (so it could be used in other projects)?
Question 3
(writing apps for flexibility)
Now I have my site with its couple of apps. Each app does its one thing and does it well. The main app serves as the landing page/ overview in this case.
I want all my other apps to use / inherit the static and template files of the main app.
Where do I store all the static files and templates? In the main app and set that as the default for the other apps? Or where should these static files / templates (e.g. base.css, base.html) go?
Do I make a copy of these files for each other app, so they can be run even though this is redundant?
Which makes my app more flexible?
Question 0
A "module" in the Python context is simply a file that contains definitions and statements. So "related modules under a package" really just means "split your code into separate files based on what the code is doing".
Describing it as "an app made of other apps" is to start confusing Django's concept of an app with Python's concept of a module (which, as stated above is just a file that houses some code).
Question 1
What should I do when apps share models?
You should still try and stick to the "apps do one thing and do it well" maxim. In this case separate profile and registration apps seems like a good idea - because they have quite different functions. A registration app is going to contain the logic for allowing users to register on your site. A profile app is all about what information you will store about a user.
There is nothing wrong with these two apps having a relationship to each other - see below.
Question 2
Let's say that my main app (or some other app that is used by the main app) utilizes some aspect of the user model (e.g. recently active members if it was a chat site). Clearly my main app gets this information from the user model. Does my main app now get bundled under the user-app?
No. It should still be a separate app, with links to the other app.
The user model is actually a good example. Django allows you to specify a custom user model that lets you store whatever additional data you want about a user.
Now, there are loads of third party apps out there that do things like registration, authentication, etc for users. They are designed to work with any user model, not just Django's default one. The way they do that is to use get_user_model() wherever they need to reference the User model, instead of directly importing django.contrib.auth.models.User.
This means that you can use those third party apps with whatever user model you have defined for your own project.
Django's get_user_model() utility is there to serve a very common use case. However the same principle can be extended to your own apps. If there is a dependency between your apps that you think should be swappable, then you can provide a way to swap it out - e.g., a setting/configuration that allows any other project using your app to specify an alternative.
There are hundreds of examples of this kind of configurability in the Django ecosystem. For example, Django itself ships with its own django.contrib.auth authentication app. However if you want to implement your own authentication logic, you don't have to reimplement the entire auth app again yourself (that would be a huge pain). Instead you specify an an authentication backend that it's auth app will use to authenticate. The auth app is designed to allow any project to swap out a core piece of its functionality with minimal effort.
So in your main app, you might define a setting that controls which profile model to use. This means that if someone else wants to use a different profile model, they simply change this setting and they're all set. They are no longer tied to your profile app.
For example - let's say you have a main app that has a view that displays some user data, but also provides a link to a registration view that is provided by a different app. You want anyone else to be able to use that app regardless of what registration app they are using. So you can make this view resuable like so:
In main/views.py:
from django.contrib.auth import get_user_model
from django.conf import settings
from django.urls import reverse
class UserDetailView(DetailView):
# First of all, we're using get_user_model so that a project
# can specify whatever user model it wants, and still use this
# view.
model = get_user_model()
def get_context_data(self, *args, *kwargs):
ctx = super().get_context_data(*args, **kwargs)
# We want to add a link to a registration view into this template context.
# But we want this to be configurable.
# Your REGISTRATION_URL would be something like 'profile:registration'
ctx['registration_link'] = reverse(settings.REGISTRATION_URL)
return ctx
Question 3
The main app serves as the landing page/ overview in this case. I want all my other apps to use / inherit the static and template files of the main app. Where do I store all the static files and templates?
You should store the templates in each respective app. If your main app is providing the base templates, then those should reside in the main app.
If your profile app is then providing a registration view, then the template for that should live in the profile app. There is nothing wrong with it extending the base template from the main app - this can easily be overridden by a project that wants to.
It's fine to make assumptions about how two apps are related to each other - as long as you're careful to allow overriding of those assumptions.
I have to admit your question is not a technical one but rather a conceptual and dogmatic one.
No answer is absolute and universally valid and every detail about how you project is structured and should behave can change the perspective.
As you wrote, each Django app does one thing and it does it well.
I would extend that to the point that each app should contain no more than one Model and at most, it's closets dependents.
Ex: the Product with it's Category, Color, Image
"What Changes together, stay together"
You will have plenty of logic to cover inside that app with only these ones.
Try to look at Django framework as a tool to create your project..this is the final goal...but if you want also to create reusable apps try to create them as independent as possible, or at least dependent to some Django features:
ex: a reusable app and totally independent would be an app that only requires User Model, Sessions, Groups included in Django. You get the idea of dependent but still autonomous app.
An app is part of a project after all...either here or in other part after you build it. Look at it as if it would be a simple function...can run alone or can depend on other functions result...at what point you keep everything inside one function and when you decide to split them in 2 separate ones.
So:
Question 0:
An app is the smallest piece that can run by it's own...having models, views, templates, urls, static files.
It can depend also on other apps...so answer is YES
Question 1:
Always keep things separate by functionality...
User Auth is dealing with user creation and their authentication
User Profile is dealing with personal data of the User
Question 2:
Nothing gets bundled. Everything stays at the same level as 2 different but dependents apps
Question 3:
You can do as you wish.
You can do static as a central place and templates specific for each app or everything central.
No right answer here but only what scales well for your project.
This is a great question and it covers all the questions associated to structuring the project I asked myself when i started working with Django.
Question 0:
Yes, in that case, a module is an app which consists of serveral apps (ellington.events, ellington.podcasts).
Question 1, Question 2, Question 3:
Django is a general purpose, full stack web framework. Since it is general purpose, a lot of it depends on your particular use case. You need not have an entire Django project follow a particular structure (if you want to achieve code reuse AND functional decoupling AND relational decoupling).
With that said, if you can prioritize what you want to achieve, you can go with one pattern over the other.
Let's take the example of Blog.
Code Reuse:
For achieving maximum code reuse, you have to identify what parts of your project is worthy of reuse. Once you have done that, you can set your project structure accordingly.
Project Structure:
BlogProject
-CommonApps
--AbstractUser(abstract class (just like it's java counterpart) )
--AbstractActivity
--AbstractComment
--AbstractArticle
-ProjectApps
--BlogUser (extends AbstractUser)
--BlogActivity (extends AbstractActivity)
--BlogComment (extends AbstractComment)
--BlogArticle (extends AbstractArticle)
The functionalities that can be shared across multiple projects should be implemented in abstract apps, and the ones specific to project can be implemented in Project apps.
Relational Decoupling:
You can create apps to represent the relations between two other apps, and implement all the functionality involving two different apps in that relation.
Project Structure:
BlogProject
-User
-UserActivityRelation
-Activity
-Article
-ArticleCommentRelation
-Comment
-UserCommentRelation
-and so on
Functional Decoupling:
This is the most common practice - create apps for particular functionality.
Project Structure:
BlogProject
-Article
-Activity
-User
-Comment
The point I am trying to make here is that the choice is yours. In more complex projects, it won't be so white and black.
You, depending on what an "app" means to you, and what you want it to do in a particular project and other projects, can decide on a particular structure.
Django keeps it abstract to give you the ability to do that.
I always go for an app setup that makes sense to that particular project. In a project not all apps are reusable. And having flexible/reusable apps does not make sense in all the cases.
As a general rule of thumb, Django devs say an App should be something whose functionality can be described with one sentence. But Django is designed so that you can bend the rules for your projects if you have to.
DISCLAIMER: Functional decoupling and relational decoupling aren't textbook terms. I just used them to describe what I meant here.
I like the Django ORM. It's simple, easy to use, and reasonably powerful.
I'm currently developing some internal sites for the VFX company I work for, for which I've used Django. In the meantime, we are developing other python applications and libraries to be used in various contexts in production. There's a number of places in which our core library needs to be interacting with some databases, and using an ORM like Django would really help things. I'm aware of other options like SqlAlchemy or PeeWee, but I'd like to see if Django will work since I use it on the websites and I like its API better.
Using Django as an ORM in a library is tricky (as I explored in a previous question), because Django expects to be used as a website with "apps". In a library, I might want to define any number of data models, which would exist in appropriate places in the library but not inside any Django app (as we're not using any other parts of the framework). So far so good.
I can create a baseclass for my models anywhere in the library as follows:
from django.db import models
from django.apps import apps
import django.conf
django.conf_settings.configure(
DATABASES = ...
)
apps.populate((__name__,))
class LibModel(models.Model):
class Meta:
abstract = True
app_label = __name__
Then anywhere in the library I can create my own models with this baseclass. Since I'm not relying on the "app" for the database names, I need to state them explicitly.
class SpecificModel(LibModel):
# fields go here
class Meta(LibModel.Meta):
db_table = "specific_model_table_name"
This gets around my concern of having to simulate the structure of an "app". The name property in the base class supplies Django with all it needs, and then Django quits whining about not finding an app. The other model files can live wherever they want.
However, there is a glaring use case where this all falls apart. Say that my Django web application wants to use some functionality from the company core python library, which now uses the Django ORM for various things. Since I make a call to django.conf.settings.configure in the library, Django is going to scream about defining the settings more than once when it tries to run the main application.
So basically, a library using the Django ORM is incompatible with Django. Wonderful.
Is there any way around this? I mean, it's a lovely ORM - is it really this impossible to use in a standalone modular way? Is the Django architecture utterly singleton in nature, making this impossible?
*Not a duplicate
I'm trying to have a company python library that uses Django as an ORM. Some of the things that could depend on it might be Django websites themselves. How do I get around Django's singleton insistence on only setting the settings config once? Or is it possible? None of these answers address this!
You can check if django has already been configured.
from django.apps import apps
from django.conf import settings
if not apps.ready:
settings.configure()
django.setup()
When starting Django application - core python library can be configured as separate app an be loaded on startup.
Also, check this answer on dynamic app loading at runtime.
A simple answer is how to initialize Django in a standalone application and do it compatible with Django applications.
import os
import django
if not 'DJANGO_SETTINGS_MODULE' in os.environ:
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysettings'
# # or without DJANGO_SETTINGS_MODULE directly
# from django.conf import settings
# settings.configure(DATABASES=... other...)
django.setup()
# this shouldn't be before DJANGO_SETTINGS_MODULE or settings.configure(...)
from myapp.models import MyModel
# this shouldn't be called before django.setup()
queryset = MyModel.objects.filter(...)
This is more compatible with Django then the answer by Oleg Russkin (where a risk of cyclic dependency at django.setup() is possible if the code is called inside inside a setup() initiated by another similar code or a normal Django project started by manage. It is similar to manage.py where django.setup() is also called internally by execute_from_command_line(sys.argv). The setup initializes all modules related to INSTALLED_APPS all urls modules and consequently all views etc. Many of them are lazy, but still. If any code called by setup() depends on this then neither the condition not apps.ready doesn't help. The setup() is not reentrant and the startup fails.)
Much more general answer
An important concept of Django is to support writing reusable parts of code ("applications" in Django terminology, that can be developed and tested independently. There can be also a tree of dependencies, but uncontrolled mutual dependencies should be avoided if possible) Reusable applications are expected that they can be easier combined to whole project ("project" in Django terminology is with all settings necessary to run it by Python.)
The only unavoidable and useful "singleton" in Django ORM are database connections django.db.connections and django.conf.settings especially INSTALLED_APPS. Only one connection should be used to the same database from the same process or thread.
Django is very configurable. An extreme example: It is possible to write a single file project where all code like settings, models, URL configs and views is defined in one file. That extreme that is probably useful only for some special tests or very short demos or as an exercise. It is even possible to define a project by one file with two "reusable" applications :-)
Django supports also "legacy databases" where the database structure is shared with existing non Django applications and models can be created by inspectdb command. Table names in such models are explicit and don't contain the app name. On the other hand the app name prefix is useful to prevent a possible conflict of the same table names from independent "applications". An important decision is if you can use it as a "legacy" database or a normal Django database.
You can decide between following two solutions or to combine them:
Use e.g. foo_models or bar.models and import all models from them e.g. to app.models and add only that "app" to INSTALLED_APPLICATIONS. This can be viable if it is only for one company and never otherwise and central name assigment is possible. (easiest but little naive)
Use some coarse separation of namespaces to several apps. You should probably use not more than one app with simple names without app name prefix.
Think ahead about migrations**. They will be probably very complicated and very soon impossible if you will create later more projects for the same database and different subsets of database tables without separating them to more apps and without app namespace.
There is really no "singleton" in Django ORM except of django.db.connections itself. If you use more databases you can direct some tables to a specific database by DATABASE_ROUTERS, even with two different models that use the same table name without a prefix.
I plan to build my project in Django framework. However, I noticed that all Django packages have models.py file. Now, let say I have a set of general purpose functions that I share between several apps in the project and I plan to put these functions definitions in a separate package (or app for that matter?). So, should I create an app "general" and copy-paste these functions into the models.py file? Or can I just create a general.py file in the "general" app directory and leave models.py empty? What is the "Django" way to do that?
Thanks.
models.py file is used to define the structure of database. So you should leave it for defining your database entries. You can make an app named generals and put general.py in that app, and from there you can use it by calling it in any app.
I usually create a utils.py file under my main app that is created from the django-admin.py when starting the project.
I plan to put these functions definitions in a separate package (or app for that matter?)
Before you decide to make this an app (and if you do decide to make it an app), I recommend you take a look at James Bennet keynote on Developing reusable apps and hist post on laying out an application. From one of his slides:
Should this be its own application?
Is it orthogonal to whatever else I’m doing?
Will I need similar functionality on other sites?
Yes? Then I should break it out into a separate application.
If you're cramming too much functionality in one single general purpose app, it might be better to split your general purpose app into multiple reusable apps.
Going back to your original question, Django is expecting a models.py file in every app. So you must have the file even if it's empty.
Inside your models.py, you should only have the application’s model classes. So, you wouldn't be following a best practice if you put inside models.py some miscellaneous code you want to reuse.
From the laying out an application post I mentioned before:
At the application level, I usually drop in a few more files depending on exactly what the application is going to be using:
If the application defines any custom manipulators, I put them in a file called forms.py instead of in the views file.
If there are multiple custom managers in the app, I put them in a file called managers.py instead of the models file.
If I’m defining any custom context processors, I put them in a file called context_processors.py.
If I’m setting up any custom dispatcher signals, they go in a file called signals.py.
If the application is setting up any syndication feeds, the feed classes go in a file called feeds.py. Similarly, sitemap classes go in sitemaps.py.
Middleware classes go in a file called middleware.py.
Any miscellaneous code which doesn’t clearly go anywhere else goes in a file or module called utils.
All of this does not answer directly your original question:
can I just create a general.py file in the "general" app directory and leave models.py empty?
But I hope this gives you additional information to make a decision that better fits your project and requirements.