We Keep Coding

Django settings: sharing ORM with a standalone module

Introduction:
I am learning Django as I develop my first project using PyCharm with Python 3.8 and Postgresql.
I have a problem with the settings.
Environment description:
My aim is to have a Django project for a web site connected to a database (let's call it "web_db"), which is fed at regular intervals with data from an external source (another database, let's call it "source_db").
For this ETL process from one database to the other one, I am developing a Python module called "source2web". I read the content of source_db using psycopg2.
I am developing everything inside one PyCharm project.
Now, since I will be using Django ORM to access the content of web_db from the Django application server, I concluded I might as well use Django ORM in source2web to load my content into web_db.
This means I am trying to share some code between the Django project and source2web: the ORM models of Django.
source2web is simply launched as a python module ( python source2web.py).
Inside my PyCharm project, I have a src folder inside which are all my python modules, with a tree like this (folders are noted with parenthesis) :
(src)
|--(project_name)
|--(module1)
| |--source2web.py
|
|--(django)
|--(django_project)
|--(django_project)
|--(another_app)
The folders django_project/django_project and django_project/another_app have been created classicaly by Django with the commands django-admin startproject django_project and python3 manage.py startapp another_app.
The ORM models I want to share are the model classes inside the file django_project/another_app/models.py
The problem:
I tinkered to find a way to use the Django ORM models from source2web. I thought I had it : inside the code of source2web, I added these lines :
os.environ['DJANGO_SETTINGS_MODULE'] = 'project_name.django.django_project.django_project.settings'
import django
django.setup()
But then, I realize that the Django server and the standalone module source2web can both work, but not with the exact same content inside project_name.django.django_project.django_project.settings.py : the name of another_app in the list of INSTALLED_APPS must be different.
Django requires the app to be listed as just another_app
source2web requires the app to be listed as project_name.django.django_project.another_app
I am sure I could find a 'dirty way' to solve this, but I also think I am not the first one to face this problem (sharing Django ORM code between a Django project and a standalone program) and I hope there is cleaner way to manage this.
What I can think of is duplicating the module project_name.django.django_project.django_project.settings and then just change the settings call inside source2web ( os.environ['DJANGO_SETTINGS_MODULE'] = ... ) but I never like duplicating code and maybe there is a cleaner than that ?

I have found at least one decently clean workaround :
In the project_name.django.django_project.django_project.settings file, I have left the value of INSTALLED_APPS that work with Django, i.e., the list of default apps created with the Django Project plus another_app.
Then in the source2web module, I have changed my code for initializing Django ORM :
In module source2web, I added 2 lines :
import project_name.django.django_project.django_project.settings as djangosettings
djangosettings.INSTALLED_APPS = ['project_name.django.django_project.another_app',]
os.environ['DJANGO_SETTINGS_MODULE'] = 'project_name.django.django_project.django_project.settings'
import django`
django.setup()
This way, I leave untouched the settings for the Django server, and only override what's necessary in the source2web module (i.e. the value of INSTALLED_APPS, getting rid at the same time of all the default apps I don't need for my ETL process).
It works and seems robust enough.
Still, it brought me to question the whole project structure :
Because obviously, the whole problem is the result of the PyCharm project considering for source root src/, while the Django project considers for root project_name/django/django_project/ .
There should be a way to reconcile them, but how ?

Conditional Django Model Creation

I am writing a app for django which i am planning to publish. This app requires a Bolean Setting variable CONSUMER_REGISTRATION.
Aim of getting this variable is to decide whether to define ConsumerRegistrationModel or not.
This is what I did.
from django.db import models
from django.conf import settings
if getattr(settings, 'CONSUMER_REGISTRATION', False):
class ConsumerRegistration(models.Model):
...
Its working fine. The only Issue i am facing that developers will need to run makemigrations and migrate commands each time they change the variable in settings.
1- Can this work be automated ?. So if they change the variable then some code in django auto run the makemigrations and migrate commands.
2- Or is it perfectly fine to leave this work on developers ??
3- Also I want to ask that is it a good aproach to do this in django ?

The accepted answer doesn't really provide a way to do what the OP is asking, which is to conditionally declare a model.
People may have various reasons for wanting to do this, from not declaring a model at all, to declaring models differently based on settings (it is implied that if you are doing this: you are intend to run the same code base in different places using different settings).
One solution is to put the model in its own app, and conditionally include the app based on a setting:
# Set this your per-project settings:
CONSUMER_REGISTRATION = True
# Set this in the generic settings
INSTALLED_APPS = [...]
if CONSUMER_REGISTRATION:
INSTALLED_APPS.append('consumer_registration') # Models will be loaded.
There's nothing wrong with creating an app which just contains a model.
With regards to "automating" it, the table will be created if migrations are run when the setting is true. It will not delete the table if it is changed to false.

You could simply define the model without any conditionals and tweak your app logic so that instances of ConsumerRegistration model are only interacted with (i.e. created, updated etc.) when the 'CONSUMER_REGISTRATION' flag is set to True.
Running migrations every single time the value of 'CONSUMER_REGISTRATION' is changed would make much more mess than leaving ConsumerRegistration table empty.
As indicated by #dahrens, you could isolate the ConsumerRegistration model along with relevant logic in a separate app, which would only be installed as needed by developers.

Django Reusing models

I have Django application with 30+ models. I want to write an application that can take a snapshot of the data in some of the models. I want to write the models once and reuse them in each application so that if I maintain it in one place, the only difference being that when I call python manage.py syncdb the same table are created with different table prefixes.
Is there any way to do this?

This is exactly where the reusable app principle comes into play.
(as explained at the django website)

Django database error: missing table social_auth_usersocialauth when social_auth is not installed

I'm trying to deal with a very puzzling error in a Django app. When DEBUG=False, trying to delete a user (via user.delete()) gives this database error:
DatabaseError: relation "social_auth_usersocialauth" does not exist
LINE 1: ...", "social_auth_usersocialauth"."extra_data" FROM "social_au...
However, I do not have social_auth or anything by a similar name in INSTALLED_APPS, nor are there any such tables in my database, nor does any of my code reference anything of the sort (I ran a text search on 'social' in the entire project folder) - and again, this works fine when DEBUG=True. social_auth is installed on my system and on my PYTHONPATH, but I cannot see where this app is getting the idea it should be having social_auth's tables in its database, let alone why it only thinks so when DEBUG=False.
What possible pathways could my app be getting this table from and how could I convince it it's not supposed to be there?

The problem could be caused by saved generic relations realized by Django content types. Relations in Django are not only static, implemented by models and INSTALLED_APPS but also dynamic implemented by table django_content_type that saves mapping from a numeric id to app_label + model. An example of possible dynamic relationship is a permission or a comment. You can have or have not a permission to any table of any installed application. You can write a comment to everything e.g to an article, to a user to a comment itself without changing any model. This relation is realized by saving numeric id of ContentType related to that model (table) and a primary key of related object (row).
Django does not expect that someone can manipulate the database manually. If you use south for manipulation then if you after uninstalling an application then run syncdb, you are asked by south if you want automatically remove orphant content types. Then can be unused tables removed securely without beeing later referenced.
(Possible hack: delete from django_content_type where app_label='social_auth' but south is unfallible.)
Many parts of the question are still open.
Edit:
Why it was not the right way: All generic relations are from descendants to the parent and all data about the relation are saved in descendant. If the child app is removed from INSTALLED_APPS then django.db code can nevermore try to remove descendants because it can not recognize which columns contain the relation data.

This table is created by django-social-auth application.
Looks like you've added it to your project and haven't launched migrate (or syncdb).

How to add an Admin class to a model after syncdb?

I added some models in my models.py and I want to add an admin class to use a wysiwyg-editor in text-fields.
Well, I know that Django itself doesn't support migrations and I've used South, but it doesn't work either.
South doesn't "see" the change.
Could it be, that South just detects changes to fields, but not if I add a new class?
How can I tweak Django to detect such changes?

syncdb and South are only concerned with descendants of Model in apps listed in INSTALLED_APPS. Everything else is handled by Django directly.

You seem to be very confused, unfortunately. Of course Django reads the code in models.py - otherwise what would be the point of it? Django uses that code initially to define the model SQL when doing syncdb, but it doesn't modify existing database tables in subsequent calls to syncdb - hence the need for South.
But naturally, Django uses models.py and admin.py and all the other Python code to define its own configuration and state. (And note that admin classes are not defined in models.py but in admin.py.)
If you are not seeing changes, you will need to restart your server.

I'm fairly sure that if you follow the steps as outlined in the tutorial to create an admin app it'll just work. Migration isn't an issue as the admin app creates new tables rather than altering the existing one.