We Keep Coding

Does Django's singleton architecture make it unworkable as a standalone ORM in a library?

I like the Django ORM. It's simple, easy to use, and reasonably powerful.
I'm currently developing some internal sites for the VFX company I work for, for which I've used Django. In the meantime, we are developing other python applications and libraries to be used in various contexts in production. There's a number of places in which our core library needs to be interacting with some databases, and using an ORM like Django would really help things. I'm aware of other options like SqlAlchemy or PeeWee, but I'd like to see if Django will work since I use it on the websites and I like its API better.
Using Django as an ORM in a library is tricky (as I explored in a previous question), because Django expects to be used as a website with "apps". In a library, I might want to define any number of data models, which would exist in appropriate places in the library but not inside any Django app (as we're not using any other parts of the framework). So far so good.
I can create a baseclass for my models anywhere in the library as follows:
from django.db import models
from django.apps import apps
import django.conf
django.conf_settings.configure(
DATABASES = ...
)
apps.populate((__name__,))
class LibModel(models.Model):
class Meta:
abstract = True
app_label = __name__
Then anywhere in the library I can create my own models with this baseclass. Since I'm not relying on the "app" for the database names, I need to state them explicitly.
class SpecificModel(LibModel):
# fields go here
class Meta(LibModel.Meta):
db_table = "specific_model_table_name"
This gets around my concern of having to simulate the structure of an "app". The name property in the base class supplies Django with all it needs, and then Django quits whining about not finding an app. The other model files can live wherever they want.
However, there is a glaring use case where this all falls apart. Say that my Django web application wants to use some functionality from the company core python library, which now uses the Django ORM for various things. Since I make a call to django.conf.settings.configure in the library, Django is going to scream about defining the settings more than once when it tries to run the main application.
So basically, a library using the Django ORM is incompatible with Django. Wonderful.
Is there any way around this? I mean, it's a lovely ORM - is it really this impossible to use in a standalone modular way? Is the Django architecture utterly singleton in nature, making this impossible?
*Not a duplicate
I'm trying to have a company python library that uses Django as an ORM. Some of the things that could depend on it might be Django websites themselves. How do I get around Django's singleton insistence on only setting the settings config once? Or is it possible? None of these answers address this!

You can check if django has already been configured.
from django.apps import apps
from django.conf import settings
if not apps.ready:
settings.configure()
django.setup()
When starting Django application - core python library can be configured as separate app an be loaded on startup.
Also, check this answer on dynamic app loading at runtime.

A simple answer is how to initialize Django in a standalone application and do it compatible with Django applications.
import os
import django
if not 'DJANGO_SETTINGS_MODULE' in os.environ:
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysettings'
# # or without DJANGO_SETTINGS_MODULE directly
# from django.conf import settings
# settings.configure(DATABASES=... other...)
django.setup()
# this shouldn't be before DJANGO_SETTINGS_MODULE or settings.configure(...)
from myapp.models import MyModel
# this shouldn't be called before django.setup()
queryset = MyModel.objects.filter(...)
This is more compatible with Django then the answer by Oleg Russkin (where a risk of cyclic dependency at django.setup() is possible if the code is called inside inside a setup() initiated by another similar code or a normal Django project started by manage. It is similar to manage.py where django.setup() is also called internally by execute_from_command_line(sys.argv). The setup initializes all modules related to INSTALLED_APPS all urls modules and consequently all views etc. Many of them are lazy, but still. If any code called by setup() depends on this then neither the condition not apps.ready doesn't help. The setup() is not reentrant and the startup fails.)
Much more general answer
An important concept of Django is to support writing reusable parts of code ("applications" in Django terminology, that can be developed and tested independently. There can be also a tree of dependencies, but uncontrolled mutual dependencies should be avoided if possible) Reusable applications are expected that they can be easier combined to whole project ("project" in Django terminology is with all settings necessary to run it by Python.)
The only unavoidable and useful "singleton" in Django ORM are database connections django.db.connections and django.conf.settings especially INSTALLED_APPS. Only one connection should be used to the same database from the same process or thread.
Django is very configurable. An extreme example: It is possible to write a single file project where all code like settings, models, URL configs and views is defined in one file. That extreme that is probably useful only for some special tests or very short demos or as an exercise. It is even possible to define a project by one file with two "reusable" applications :-)
Django supports also "legacy databases" where the database structure is shared with existing non Django applications and models can be created by inspectdb command. Table names in such models are explicit and don't contain the app name. On the other hand the app name prefix is useful to prevent a possible conflict of the same table names from independent "applications". An important decision is if you can use it as a "legacy" database or a normal Django database.
You can decide between following two solutions or to combine them:
Use e.g. foo_models or bar.models and import all models from them e.g. to app.models and add only that "app" to INSTALLED_APPLICATIONS. This can be viable if it is only for one company and never otherwise and central name assigment is possible. (easiest but little naive)
Use some coarse separation of namespaces to several apps. You should probably use not more than one app with simple names without app name prefix.
Think ahead about migrations**. They will be probably very complicated and very soon impossible if you will create later more projects for the same database and different subsets of database tables without separating them to more apps and without app namespace.
There is really no "singleton" in Django ORM except of django.db.connections itself. If you use more databases you can direct some tables to a specific database by DATABASE_ROUTERS, even with two different models that use the same table name without a prefix.

django programming - "create" methods in admin, model or view

Baseline: I'm building a django based application that is heavily using the admin interface as it spares me a lot of work in developing own CRUD routines. By now I came across several situations where i have models that hold some general information (say parents) and often have foreignkey-relations to derived models (say childs).
I realized that i sometimes implemented my routines to create child objects within the admin-class, sometimes within the model class(method being called from within some admin routine) or sometimes even within view-classes (e.g. as reaction to POST requests on some custom forms). It feels now, that my design is not very consistent (the effects of changing some model parameters being distributed over a lot of files) and i should refactor before it gets to big a mess.
So what is the best approach? Where should one concentrate methods that create/modify related objects (keeping in mind that i often want to give some feedback-messages related to process) ?

If your code is about a Model class, add it to models.py. This makes sense when the classes have to be added to database (migrations)
If your code is related to views, attach it to views.py. This makes sense when the code handles requests.
If your code is related to admin, attach it to admins.py. This makes sense when the code is related to admin interface.
If your code is generic, used in multiple places, refactor it into a separate file, and import that file elsewhere.
Your use case isn't exactly clear to me, so I'm taking a stab in the dark here - You can have the Models in models.py and create a separate file to create objects for models with child objects containing the code that's related to creating parent-child objects with given data. Then use this as an import in the admin and views wherever applicable.
Something like:
# foo in views.py and admin.py
def foo():
data = {} # get all data
make_parent_child(data) # create parent-child objects

Using Python Objects in Django Applications

I apologize if this seems like a stupid question but I'm still very much a novice Python/Django programmer. Is it normal to create Python objects in a Django application that aren't models that will be saved in the database?
I'm creating what's become a fairly large Django application and, to me, my code is really starting to "smell". What I mean is that my views are becoming very large because I'm taking a procedural rather than object-oriented approach. My intuition tells me that my code might be simpler, easier to test, and more robust in the long run if I were using more objects with their own attributes and behaviors rather than passing information from one function to the next in my views.
What's hanging me up is that these aren't objects I want to save in my database so I don't quite know if I should be using them and, if I should, where I'd put them. Is the approach I'm proposing typical in a Django application? If so, where would I store those objects with respect to the Django model/view/template structure? Also, are there any popular Django modules or libraries that do what I'm describing that I should study?
Thanks in advance for your response.

You can store your objects anywhere. There could be helper functions in your views file or models file or wherever. I prefer to put miscellaneous functions in a utils.py file but that is not a convention, just something I end up doing. I end up putting most of miscellaneous helper functions and base classes in a common app, and more specifically a common.utils file.
In one project I have lots of apps, and each app has an api client. The base class for the client resides in an app called common. Then each app then has their specific client in client.py file
project
common
client
app1
client
app2
client
Then in app1 client
from project.common.client import BaseClient
class ConcreteApp1Client(BaseClient):
pass
Then in my views or management commands or models or wherever the concrete client can be imported and used as normal. from project.app1.client import ConcreteApp1Client

Django also has class-based views if you feel certain variables could best be encapsulated in a class.
https://docs.djangoproject.com/en/dev/topics/class-based-views/

User defined classes in Web2Py

I am a beginner in Web2Py. I wish to develop a simple application, where the user should log in with a username and a password (no fancy stuff like login with facebook or OpenID etc.). Upon successful login, the user sees some interface, and performs certain operations. I imagined a User class and a JobStore class (which has certain methods defined, which the user should be able to call). There will be only one JobStore object for all users and sessions. When a user logs in, an User object gets created with a reference to the JobStore. The User class has methods like GetRights(), RequestJob(), MarkAsFinished(), etc. and JobStore has methods like GetUnfinished(), RemoveJobs(), etc. Structurally speaking, where am I supposed to have these classes, so that based on certain actions the user performs on the view, certain methods get called? Are these classes supposed to inherit from some standard classes used in Web2Py? I am trying to find an example, where some kind of object oriented approach is used in the controller, but have not found any so far. Also, is this the wrong approach in Web2Py app development? I am not trying to implement any complex business logic through these functions as of now; I am just trying to understand how traditional programming approach would map to Web2Py approach somehow.

You can define classes in model or controller files, but it would probably make most sense to put them in the app's /modules folder and import them where needed in your models and controllers. There is no need for your classes to inherit from web2py classes, though they can do so if desired. For more on using modules and importing, see here and here.
For an example of a heavily object-oriented approach, see the Movuca CMS. Most of the code is in the /modules folder.

How do you manage your Django applications?

I just wanted to try to build a project with django. Therefore I have a (basic) question on how to manage such a project. Since I cannot find any guidelines or so on how to split a project into applications.
Let's take a kind of SO as an example. Which applications would you use?
I'd say there should be the applications "users" and "questions". But what if there was a topic system with static articles, too. Maybe they also could receive votes.
How to build the apps structure then? One app for "questions", "votes" and "topics" or just one app "content"?
I have no idea what to do. Maybe it's because I know not very much about Django yet, but I'm interested either...

There aren't hard-and-fast rules, but I would say it's better to err on the side of more specialized applications. Ideally an application should handle just one functional concern: i.e. "tagging" or "commenting" or "auth/auth" or "posts." This type of design will also help you reuse available open source applications instead of reinventing the wheel (i.e. Django comes with auth and comments apps, django-tagging or django-taggable can almost certainly do what you need, etc).
Generic foreign keys can help you decouple applications such as tagging or commenting that might be applied to models from several other applications.

You should try and separate the project in as much applications as possible. For most projects an application will not contain more than 5 models. For example a project like SO would have separate applications for UsersProfiles, Questions, Tags (there's a ready one in django for this), etc. If there was a system with static pages that'd be a separate application too (there are ready ones for this purpose). You should also try and make your applications as generic as possible, so you may reuse them in other projects. There's a good presentation on reusable apps.

Just like any set of dependencies... try to find the most useful stand-alone aspects of the project and make those stand-alone apps. Other Django Apps will have higher level functionality, and reuse the parts of the lowest level apps that you have set up.
In my project, I have a calendar app with its own Event object in its models. I also have a carpool database set up, and for the departure time and the duration I use the calendar's Event object right in my RideShare tables. The carpooling database is calendar-aware, and gets all the nice .ics export and calendar views from the calendar app for 'free.'
There are some tricks to getting the Apps reusable, like naming the templates directory: project/app2/templates/app2/index.html. This lets you refer to app2/index.html from any other app, and get the right template. I picked that one up looking at the built-in reusable apps in Django itself. Pinax is a bit of a monster size-wise but it also demonstrates a nice reusable App structure.
If in doubt, forget about reusable apps for now. Put all your messages and polls in one app and get through one rev. You'll discover during the process what steps feel unnecessary, and could be broken out as something stand-alone in the future.

A good question to ask yourself when deciding whether or not to write an app is "could I use this in another project?". If you think you could, then consider what it would take to make the application as independent as possible; How can you reduce the dependancies so that the app doesn't rely on anything specific to a particular project.
Some of the ways you can do this are:
Giving each app its own urls.py
Allowing model types to be passed in as parameters rather than explicitly declaring what models are used in your views. Generic views use this principle.
Make your templates easily overridden by having some sort of template_name parameter passed in your urls.py
Make sure you can do reverse url lookups with your objects and views. This means naming your views in the urls.py and creating get_absolute_url methods on your models.
In some cases like Tagging, GenericForeignKeys can be used to associate a model in your app to any other model, regardless of whether it has ForeignKeys "looking back" at it.

I'll tell you how I am approaching such question: I usually sit with a sheet of paper and draw the boxes (functionalities) and arrows (interdependencies between functionalities). I am sure there are methodologies or other things that could help you, but my approach usually works for me (YMMV, of course).
Knowing what a site is supposed to be is basic, though. ;)