Related
I made a .env file in the same directory as my settings.py file and have some environmental variables in there such as: secret_key, database_name, etc. However, it doesn't seem to be reading the database name correctly in the .env file. I feel like I followed the docs, but still get the improperly configured error when pushing to Heroku. It does work when running the server locally though.
settings.py
from pathlib import Path
import os
from datetime import timedelta
import environ
env = environ.Env()
environ.Env.read_env()
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
# Take environment variables from .env file
environ.Env.read_env(os.path.join(BASE_DIR, '.env'))
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.0/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = env('SECRET_KEY')
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = env('DEBUG')
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': env('DATABASE_NAME'),
'USER': env('DATABASE_USER'),
'PASSWORD': env('DATABASE_PASSWORD'),
'HOST': env('DATABASE_HOST'),
'PORT': env('DATABASE_PORT'),
}
}
.env (example)
SECRET_KEY=django-insecure-vdihiodnsdkcndocndcndocdcoidcosjvodjv
DEBUG=True
DATABASE_NAME=vjiojjoj3oj3ioj3
DATABASE_USER=vdijvodivjdivfv
...
error
File "/app/project_name/settings.py", line 94, in <module>
'NAME': env('DATABASE_NAME'),
File "/app/.heroku/python/lib/python3.10/site-packages/environ/environ.py", line 175, in __call__
return self.get_value(
File "/app/.heroku/python/lib/python3.10/site-packages/environ/environ.py", line 371, in get_value
raise ImproperlyConfigured(error_msg)
django.core.exceptions.ImproperlyConfigured: Set the DATABASE_NAME environment variable
django-environ docs: https://github.com/joke2k/django-environ
EDIT: Okay it looks like pushing to Heroku with a .env file is not the way to go. Will be trying to link my github repo with heroku and configuring the vars in the settings. We'll see if that will do it.
As you may have guessed judging by your edit, Heroku doesn't support pushing .env files. This is because it uses an ephemeral filesystem.
Each dyno gets its own ephemeral filesystem, with a fresh copy of the most recently deployed code. [...] any files written will be discarded the moment the dyno is stopped or restarted. For example, this occurs any time a dyno is replaced due to application deployment and approximately once a day as part of normal dyno management.
You'd be better off using Heroku's built-in config var support. Here are some examples:
heroku config # View config vars
heroku config:set TEST=test # Sets TEST to "test"
heroku config:unset TEST # Reverses setting TEST
heroku config:get TEST # Returns value of TEST
I have been developing a basic app. Now at the deployment stage it has become clear I have need for both a local settings and production settings.
It would be great to know the following:
How best to deal with development and production settings.
How to keep apps such as django-debug-toolbar only in a development environment.
Any other tips and best practices for development and deployment settings.
The DJANGO_SETTINGS_MODULE environment variable controls which settings file Django will load.
You therefore create separate configuration files for your respective environments (note that they can of course both import * from a separate, "shared settings" file), and use DJANGO_SETTINGS_MODULE to control which one to use.
Here's how:
As noted in the Django documentation:
The value of DJANGO_SETTINGS_MODULE should be in Python path syntax, e.g. mysite.settings. Note that the settings module should be on the Python import search path.
So, let's assume you created myapp/production_settings.py and myapp/test_settings.py in your source repository.
In that case, you'd respectively set DJANGO_SETTINGS_MODULE=myapp.production_settings to use the former and DJANGO_SETTINGS_MODULE=myapp.test_settings to use the latter.
From here on out, the problem boils down to setting the DJANGO_SETTINGS_MODULE environment variable.
Setting DJANGO_SETTINGS_MODULE using a script or a shell
You can then use a bootstrap script or a process manager to load the correct settings (by setting the environment), or just run it from your shell before starting Django: export DJANGO_SETTINGS_MODULE=myapp.production_settings.
Note that you can run this export at any time from a shell — it does not need to live in your .bashrc or anything.
Setting DJANGO_SETTINGS_MODULE using a Process Manager
If you're not fond of writing a bootstrap script that sets the environment (and there are very good reasons to feel that way!), I would recommend using a process manager:
Supervisor lets you pass environment variables to managed processes using a program's environment configuration key.
Honcho (a pure-Python equivalent of Ruby's Foreman) lets you define environment variables in an "environment" (.env) file.
Finally, note that you can take advantage of the PYTHONPATH variable to store the settings in a completely different location (e.g. on a production server, storing them in /etc/). This allows for separating configuration from application files. You may or may not want that, it depends on how your app is structured.
By default use production settings, but create a file called settings_dev.py in the same folder as your settings.py file. Add overrides there, such as DEBUG=True.
On the computer that will be used for development, add this to your ~/.bashrc file:
export DJANGO_DEVELOPMENT=true
Or turn it on one time by prefixing your command:
DJANGO_DEVELOPMENT=true python manage.py runserver
At the bottom of your settings.py file, add the following.
# Override production variables if DJANGO_DEVELOPMENT env variable is true
if os.getenv('DJANGO_DEVELOPMENT') == 'true':
from settings_dev import * # or specific overrides
(Note that importing * should generally be avoided in Python)
By default the production servers will not override anything. Done!
Compared to the other answers, this one is simpler because it doesn't require updating PYTHONPATH, or setting DJANGO_SETTINGS_MODULE which only allows you to work on one django project at a time.
This is how I did it in 6 easy steps:
Create a folder inside your project directory and name it settings.
Project structure:
myproject/
myapp1/
myapp2/
myproject/
settings/
Create four python files inside of the settings directory namely __init__.py, base.py, dev.py and prod.py
Settings files:
settings/
__init__.py
base.py
prod.py
dev.py
Open __init__.py and fill it with the following content:
init.py:
from .base import *
# you need to set "myproject = 'prod'" as an environment variable
# in your OS (on which your website is hosted)
if os.environ['myproject'] == 'prod':
from .prod import *
else:
from .dev import *
Open base.py and fill it with all the common settings (that will be used in both production as well as development.) for example:
base.py:
import os
...
INSTALLED_APPS = [...]
MIDDLEWARE = [...]
TEMPLATES = [{...}]
...
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
MEDIA_ROOT = os.path.join(BASE_DIR, '/path/')
MEDIA_URL = '/path/'
Open dev.py and include that stuff which is development specific for example:
dev.py:
DEBUG = True
ALLOWED_HOSTS = ['localhost']
...
Open prod.py and include that stuff which is production specific for example:
prod.py:
DEBUG = False
ALLOWED_HOSTS = ['www.example.com']
LOGGING = [...]
...
Update
As ANDRESMA suggested in comments. Update BASE_DIR in your base.py file to reflect your updated path by adding another .parent to the end. For example:
BASE_DIR = Path(__file__).resolve().parent.parent.parent
I usually have one settings file per environment, and a shared settings file:
/myproject/
settings.production.py
settings.development.py
shared_settings.py
Each of my environment files has:
try:
from shared_settings import *
except ImportError:
pass
This allows me to override shared settings if necessary (by adding the modifications below that stanza).
I then select which settings files to use by linking it in to settings.py:
ln -s settings.development.py settings.py
I use the awesome django-configurations, and all the settings are stored in my settings.py:
from configurations import Configuration
class Base(Configuration):
# all the base settings here...
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
...
class Develop(Base):
# development settings here...
DEBUG = True
...
class Production(Base):
# production settings here...
DEBUG = False
To configure the Django project I just followed the docs.
Create multiple settings*.py files, extrapolating the variables that need to change per environment. Then at the end of your master settings.py file:
try:
from settings_dev import *
except ImportError:
pass
You keep the separate settings_* files for each stage.
At the top of your settings_dev.py file, add this:
import sys
globals().update(vars(sys.modules['settings']))
To import variables that you need to modify.
This wiki entry has more ideas on how to split your settings.
Here is the approach we use :
a settings module to split settings into multiple files for readability ;
a .env.json file to store credentials and parameters that we want excluded from our git repository, or that are environment specific ;
an env.py file to read the .env.json file
Considering the following structure :
...
.env.json # the file containing all specific credentials and parameters
.gitignore # the .gitignore file to exclude `.env.json`
project_name/ # project dir (the one which django-admin.py creates)
accounts/ # project's apps
__init__.py
...
...
env.py # the file to load credentials
settings/
__init__.py # main settings file
database.py # database conf
storage.py # storage conf
...
venv # virtualenv
...
With .env.json like :
{
"debug": false,
"allowed_hosts": ["mydomain.com"],
"django_secret_key": "my_very_long_secret_key",
"db_password": "my_db_password",
"db_name": "my_db_name",
"db_user": "my_db_user",
"db_host": "my_db_host",
}
And project_name/env.py :
<!-- language: lang-python -->
import json
import os
def get_credentials():
env_file_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
with open(os.path.join(env_file_dir, '.env.json'), 'r') as f:
creds = json.loads(f.read())
return creds
credentials = get_credentials()
We can have the following settings:
<!-- language: lang-py -->
# project_name/settings/__init__.py
from project_name.env import credentials
from project_name.settings.database import *
from project_name.settings.storage import *
...
SECRET_KEY = credentials.get('django_secret_key')
DEBUG = credentials.get('debug')
ALLOWED_HOSTS = credentials.get('allowed_hosts', [])
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
...
]
if DEBUG:
INSTALLED_APPS += ['debug_toolbar']
...
# project_name/settings/database.py
from project_name.env import credentials
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': credentials.get('db_name', ''),
'USER': credentials.get('db_user', ''),
'HOST': credentials.get('db_host', ''),
'PASSWORD': credentials.get('db_password', ''),
'PORT': '5432',
}
}
the benefits of this solution are :
user specific credentials and configurations for local development without modifying the git repository ;
environment specific configuration, you can have for example three different environments with three different .env.json like dev, stagging and production ;
credentials are not in the repository
I hope this helps, just let me know if you see any caveats with this solution.
I use the folloring file structure:
project/
...
settings/
settings/common.py
settings/local.py
settings/prod.py
settings/__init__.py -> local.py
So __init__.py is a link (ln in unix or mklink in windows) to local.py or can be to prod.py so the configuration is still in the project.settings module is clean and organized, and if you want to use a particular config you can use the environment variable DJANGO_SETTINGS_MODULE to project.settings.prod if you need to run a command for production environment.
In the files prod.py and local.py:
from .shared import *
DATABASE = {
...
}
and the shared.py file keeps as global without specific configs.
Use settings.py for production. In the same directory create settings_dev.py for overrides.
# settings_dev.py
from .settings import *
DEBUG = False
On a dev machine run your Django app with:
DJANGO_SETTINGS_MODULE=<your_app_name>.settings_dev python3 manage.py runserver
On a prod machine run as if you just had settings.py and nothing else.
ADVANTAGES
settings.py (used for production) is completely agnostic to the fact that any other environments even exist.
To see the difference between prod and dev you just look into a single location - settings_dev.py. No need to gather configurations scattered across settings_prod.py, settings_dev.py and settings_shared.py.
If someone adds a setting to your prod config after troubleshooting a production issue you can rest assured that it will appear in your dev config as well (unless explicitly overridden). Thus the divergence between different config files will be minimized.
building off cs01's answer:
if you're having problems with the environment variable, set its value to a string (e.g. I did DJANGO_DEVELOPMENT="true").
I also changed cs01's file workflow as follows:
#settings.py
import os
if os.environ.get('DJANGO_DEVELOPMENT') is not None:
from settings_dev import *
else:
from settings_production import *
#settings_dev.py
development settings go here
#settings_production.py
production settings go here
This way, Django doesn't have to read through the entirety of a settings file before running the appropriate settings file. This solution comes in handy if your production file needs stuff that's only on your production server.
Note: in Python 3, imported files need to have a . appended (e.g. from .settings_dev import *)
If you want to keep 1 settings file, and your development operating system is different than your production operating system, you can put this at the bottom of your settings.py:
from sys import platform
if platform == "linux" or platform == "linux2":
# linux
# some special setting here for when I'm on my prod server
elif platform == "darwin":
# OS X
# some special setting here for when I'm developing on my mac
elif platform == "win32":
# Windows...
# some special setting here for when I'm developing on my pc
Read more: How do I check the operating system in Python?
You want to be able to switch settings, secretes, environment variables and others based on the git branch that you are in and relying on different settings file is okay but in an enterprise situation you would like to hide all your sensitive information from the repo. It is not a best security best practice to expose all the environment variables, secrets of all environments (develop, staging, production, qa etc.,) to all the developers. The following should achieve 2.
isolation of settings as per their environment of deployment
hide sensitive information from git repo
My run.sh
#!/bin/bash
# default environment
export DJANGO_ENVIRONMENT="develop"
BRANCH=$(git rev-parse --abbrev-ref HEAD)
if [ $BRANCH == "main" ]; then
export DJANGO_ENVIRONMENT="production"
elif [ $BRANCH == "release/"* ]; then
export DJANGO_ENVIRONMENT="staging"
else
# for all other branches (feature, support, hotfix etc.,)
echo ''
fi
echo "
BRANCH: $BRANCH
ENVIRONMENT: $DJANGO_ENVIRONMENT
"
python3 myapp/manage.py makemigrations
python3 myapp/manage.py migrate --noinput
python3 myapp/manage.py runserver 0:8000
My vars.py (or secrets.py or whatever name) in the same folder as settings.py of django
vars = {
'develop': {
'environment': 'develop',
'SECRET_KEY': 'mysecretkey',
"DEBUG": "True"
},
'production': {
'environment': 'production',
'SECRET_KEY': 'mysecretkey',
"DEBUG": "False"
},
'staging': {
'environment': 'staging',
'SECRET_KEY': 'mysecretkey',
"DEBUG": "True"
}
}
then in settings.py just do the following
from . import vars # container environment specific vars
import os
DJANGO_ENVIRONMENT = os.getenv("DJANGO_ENVIRONMENT") # declared in run.sh
envs = vars.vars[DJANGO_ENVIRONMENT] # SECURITY WARNING: keep the secret key
used in production secret!
SECRET_KEY = envs["SECRET_KEY"]
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = envs["DEBUG"]
Let developers have their own vars.py in their local machine but during deployment your cicd pipeline can insert the actual vars.py with actual valures or some script should insert it. If you are using gitlab cicd then you can store the entire vars.py as an environment variable
This seems to have been answered, however a method which I use as combined with version control is the following:
Setup a env.py file in the same directory as settings on my local development environment that I also add to .gitignore:
env.py:
#!usr/bin/python
DJANGO_ENV = True
ALLOWED_HOSTS = ['127.0.0.1', 'dev.mywebsite.com']
.gitignore:
mywebsite/env.py
settings.py:
if os.path.exists(os.getcwd() + '/env.py'):
#env.py is excluded using the .gitignore file - when moving to production we can automatically set debug mode to off:
from env import *
else:
DJANGO_ENV = False
DEBUG = DJANGO_ENV
I just find this works and is far more elegant - with env.py it is easy to see our local environment variables and we can handle all of this without multiple settings.py files or the likes. This methods allows for all sorts of local environment variables to be used that we wouldn't want set on our production server. Utilising the .gitignore via version control we are also keeping everything seamlessly integrated.
For the problem of setting files, I choose to copy
Project
|---__init__.py [ write code to copy setting file from subdir to current dir]
|---settings.py (do not commit this file to git)
|---setting1_dir
| |-- settings.py
|---setting2_dir
| |-- settings.py
When you run django, __init__py will be ran. At this time , settings.py in setting1_dir will replace settings.py in Project.
How to choose different env?
modify __init__.py directly.
make a bash file to modify __init__.py.
modify env in linux, and then let __init__.py read this variable.
Why use to this way?
Because I don't like so many files in the same directory, too many files will confuse other partners and not very well for IDE.(IDE cannot find what file we use)
If you do not want to see all these details, you can divide project into two part.
make your small tool like Spring Initializr, just for setup your project.(do sth like copy file)
your project code
I'm using different app.yaml file to change configuration between environments in google cloud app engine.
You can use this to create a proxy connection in your terminal command:
./cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:1433
https://cloud.google.com/sql/docs/sqlserver/connect-admin-proxy#macos-64-bit
File: app.yaml
# [START django_app]
service: development
runtime: python37
env_variables:
DJANGO_DB_HOST: '/cloudsql/myproject:myregion:myinstance'
DJANGO_DEBUG: True
handlers:
# This configures Google App Engine to serve the files in the app's static
# directory.
- url: /static
static_dir: static/
# This handler routes all requests not caught above to your main app. It is
# required when static routes are defined, but can be omitted (along with
# the entire handlers section) when there are no static files defined.
- url: /.*
script: auto
# [END django_app]
I create a file named "production" in the working directory in production.
#settings.py
production = Path("production")
DEBUG = False
#if it's dev mode
if not production.is_file():
INSTALLED_APPS +=[
#apps_in_development_mode,
#...
]
DEBUG = True
#other settings to override the default production settings
You're probably going to use the wsgi.py file for production (this file is created automatically when you create the django project). That file points to a settings file. So make a separate production settings file and reference it in your wsgi.py file.
What we do here is to have an .ENV file for each environment. This file contains a lot of variables like ENV=development
The settings.py file is basically a bunch of os.environ.get(), like ENV = os.environ.get('ENV')
So when you need to access that you can do ENV = settings.ENV.
You would have to have a .env file for your production, testing, development.
This is my solution, with different environements for dev, test and prod
import socket
[...]
DEV_PC = 'PC059'
host_name = socket.gethostname()
if host_name == DEV_PC:
#do something
pass
elif [...]
I'm working with Django 1.11 and Apache 2.2. My SECRET_KEY suddenly stopped working, I'm getting an error "ImproperlyConfigured: The SECRET_KEY setting must not be empty." even though the SECRET_KEY is definitely set in my settings.py --
settings.py
SECRET_KEY = 'yaddayaddayadda'
I know that there are lot of similar questions and I've read many of the answers and I still don't understand what's going wrong here. It was working and to me it seemed like it suddenly stopped working. Deleting the *.pyc files did not help.
manage.py
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")
wsgi.py
path_1 = '/home/path/proj_website/mysite'
sys.path.append(path_1)
path_2 = '/home/path/proj_website/mysite/mysite'
sys.path.append(path_2)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")
Update
I think my real problem is that my production server is not registering any changes to my settings.py file. I always do "touch wsgi.py" when I make a change and I cleared out my *pyc files and it's still not working. Like I said in a comment below, when I do "python manage.py runserver" it runs without error. So I don't think it's a path issue or an issue with project files but something with the web server. Any other suggestions of what I can try?
secret key should be a string not a list
SECRET_KEY = 'yaddayaddayadda'
The resolution to what I put in the update is that I needed to restart the Apache server.
I have a Django application deployed to cloudControl. Configuration is standard and the push/deploy happens without (apparent) errors.
But the collectstatic step is not being executed: it fails silently (I see no -----> Collecting static files message). After the deploy, the static folder for the application is empty, so you get 500 Server Errors continually.
I can solve it changing the Procfile, but it is not consistent either:
web: python manage.py collectstatic --noinput; gunicorn app.wsgi:application --config gunicorn_cnf.py --bind 0.0.0.0:${PORT:-5000}`
collectstatic works as it should locally, and if I run cctrlapp app/deployment run "python manage.py collectstatic --noinput" no errors are shown either:
669 static files copied to '/srv/www/staticfiles/static', 669 post-processed.
But /srv/www/staticfiles/static is empty.
How can I know why collectstatic is not being executed in the push phase?
I've been able to debug the problem, using a custom python buildpack, so here is the answer for further reference.
The problem was in the settings.py file. The first thing I do in this file is to check if we are in a cloudControl environment or in a local environment. I do it looking for the CRED_FILE environment variable (not so different of what is suggested): if no variable is found, I load a local JSON file that mimics that credentials variable for development:
try:
cred_file = os.environ['CRED_FILE']
DEBUG = False
except KeyError:
cred_file = os.path.join(BASE_DIR, 'creds.json')
DEBUG = True
Once I know the environment, I can have different INSTALLED_APPS (requirements.txt files are slightly different in production and development, too) or change some settings.
Now the bad news: in the push phase there is no CRED_FILE available.
So I was trying to load apps that were not installed (because they were only in the development requirements file, like coverage or django-debug-toolbar) or use credentials that were not set (creds.json is, of course, not uploaded to the repository: only a TXT with dummy values is uploaded as a reference). That's why collectstatic was failing silently in the push phase.
Here is my solution (it will work as long as you have a dummy credentials file in your repo):
try:
cred_file = os.environ['CRED_FILE']
DEBUG = False
except KeyError:
if os.path.exists(os.path.join(BASE_DIR, 'creds.json')):
cred_file = os.path.join(BASE_DIR, 'creds.json')
DEBUG = True
else:
cred_file = os.path.join(BASE_DIR, 'creds.json.txt')
DEBUG = False
Credentials are not used by collectstatic, so you can have anything in the creds.json.txt file. Not very clean, but it works as expected now.
EDIT
As pointed by #pst in a comment there is an environment variable to know if the buildpack is running, so we could use that one too to load the desired credentials and set DEBUG.
if 'CRED_FILE' in os.environ:
cred_file = os.environ['CRED_FILE']
DEBUG = False
elif 'BUILDPACK_RUNNING' in os.environ:
cred_file = os.path.join(BASE_DIR, 'creds.json.txt')
DEBUG = False
else:
cred_file = os.path.join(BASE_DIR, 'creds.json')
DEBUG = True
I have been developing a basic app. Now at the deployment stage it has become clear I have need for both a local settings and production settings.
It would be great to know the following:
How best to deal with development and production settings.
How to keep apps such as django-debug-toolbar only in a development environment.
Any other tips and best practices for development and deployment settings.
The DJANGO_SETTINGS_MODULE environment variable controls which settings file Django will load.
You therefore create separate configuration files for your respective environments (note that they can of course both import * from a separate, "shared settings" file), and use DJANGO_SETTINGS_MODULE to control which one to use.
Here's how:
As noted in the Django documentation:
The value of DJANGO_SETTINGS_MODULE should be in Python path syntax, e.g. mysite.settings. Note that the settings module should be on the Python import search path.
So, let's assume you created myapp/production_settings.py and myapp/test_settings.py in your source repository.
In that case, you'd respectively set DJANGO_SETTINGS_MODULE=myapp.production_settings to use the former and DJANGO_SETTINGS_MODULE=myapp.test_settings to use the latter.
From here on out, the problem boils down to setting the DJANGO_SETTINGS_MODULE environment variable.
Setting DJANGO_SETTINGS_MODULE using a script or a shell
You can then use a bootstrap script or a process manager to load the correct settings (by setting the environment), or just run it from your shell before starting Django: export DJANGO_SETTINGS_MODULE=myapp.production_settings.
Note that you can run this export at any time from a shell — it does not need to live in your .bashrc or anything.
Setting DJANGO_SETTINGS_MODULE using a Process Manager
If you're not fond of writing a bootstrap script that sets the environment (and there are very good reasons to feel that way!), I would recommend using a process manager:
Supervisor lets you pass environment variables to managed processes using a program's environment configuration key.
Honcho (a pure-Python equivalent of Ruby's Foreman) lets you define environment variables in an "environment" (.env) file.
Finally, note that you can take advantage of the PYTHONPATH variable to store the settings in a completely different location (e.g. on a production server, storing them in /etc/). This allows for separating configuration from application files. You may or may not want that, it depends on how your app is structured.
By default use production settings, but create a file called settings_dev.py in the same folder as your settings.py file. Add overrides there, such as DEBUG=True.
On the computer that will be used for development, add this to your ~/.bashrc file:
export DJANGO_DEVELOPMENT=true
Or turn it on one time by prefixing your command:
DJANGO_DEVELOPMENT=true python manage.py runserver
At the bottom of your settings.py file, add the following.
# Override production variables if DJANGO_DEVELOPMENT env variable is true
if os.getenv('DJANGO_DEVELOPMENT') == 'true':
from settings_dev import * # or specific overrides
(Note that importing * should generally be avoided in Python)
By default the production servers will not override anything. Done!
Compared to the other answers, this one is simpler because it doesn't require updating PYTHONPATH, or setting DJANGO_SETTINGS_MODULE which only allows you to work on one django project at a time.
This is how I did it in 6 easy steps:
Create a folder inside your project directory and name it settings.
Project structure:
myproject/
myapp1/
myapp2/
myproject/
settings/
Create four python files inside of the settings directory namely __init__.py, base.py, dev.py and prod.py
Settings files:
settings/
__init__.py
base.py
prod.py
dev.py
Open __init__.py and fill it with the following content:
init.py:
from .base import *
# you need to set "myproject = 'prod'" as an environment variable
# in your OS (on which your website is hosted)
if os.environ['myproject'] == 'prod':
from .prod import *
else:
from .dev import *
Open base.py and fill it with all the common settings (that will be used in both production as well as development.) for example:
base.py:
import os
...
INSTALLED_APPS = [...]
MIDDLEWARE = [...]
TEMPLATES = [{...}]
...
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
MEDIA_ROOT = os.path.join(BASE_DIR, '/path/')
MEDIA_URL = '/path/'
Open dev.py and include that stuff which is development specific for example:
dev.py:
DEBUG = True
ALLOWED_HOSTS = ['localhost']
...
Open prod.py and include that stuff which is production specific for example:
prod.py:
DEBUG = False
ALLOWED_HOSTS = ['www.example.com']
LOGGING = [...]
...
Update
As ANDRESMA suggested in comments. Update BASE_DIR in your base.py file to reflect your updated path by adding another .parent to the end. For example:
BASE_DIR = Path(__file__).resolve().parent.parent.parent
I usually have one settings file per environment, and a shared settings file:
/myproject/
settings.production.py
settings.development.py
shared_settings.py
Each of my environment files has:
try:
from shared_settings import *
except ImportError:
pass
This allows me to override shared settings if necessary (by adding the modifications below that stanza).
I then select which settings files to use by linking it in to settings.py:
ln -s settings.development.py settings.py
I use the awesome django-configurations, and all the settings are stored in my settings.py:
from configurations import Configuration
class Base(Configuration):
# all the base settings here...
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
...
class Develop(Base):
# development settings here...
DEBUG = True
...
class Production(Base):
# production settings here...
DEBUG = False
To configure the Django project I just followed the docs.
Create multiple settings*.py files, extrapolating the variables that need to change per environment. Then at the end of your master settings.py file:
try:
from settings_dev import *
except ImportError:
pass
You keep the separate settings_* files for each stage.
At the top of your settings_dev.py file, add this:
import sys
globals().update(vars(sys.modules['settings']))
To import variables that you need to modify.
This wiki entry has more ideas on how to split your settings.
Here is the approach we use :
a settings module to split settings into multiple files for readability ;
a .env.json file to store credentials and parameters that we want excluded from our git repository, or that are environment specific ;
an env.py file to read the .env.json file
Considering the following structure :
...
.env.json # the file containing all specific credentials and parameters
.gitignore # the .gitignore file to exclude `.env.json`
project_name/ # project dir (the one which django-admin.py creates)
accounts/ # project's apps
__init__.py
...
...
env.py # the file to load credentials
settings/
__init__.py # main settings file
database.py # database conf
storage.py # storage conf
...
venv # virtualenv
...
With .env.json like :
{
"debug": false,
"allowed_hosts": ["mydomain.com"],
"django_secret_key": "my_very_long_secret_key",
"db_password": "my_db_password",
"db_name": "my_db_name",
"db_user": "my_db_user",
"db_host": "my_db_host",
}
And project_name/env.py :
<!-- language: lang-python -->
import json
import os
def get_credentials():
env_file_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
with open(os.path.join(env_file_dir, '.env.json'), 'r') as f:
creds = json.loads(f.read())
return creds
credentials = get_credentials()
We can have the following settings:
<!-- language: lang-py -->
# project_name/settings/__init__.py
from project_name.env import credentials
from project_name.settings.database import *
from project_name.settings.storage import *
...
SECRET_KEY = credentials.get('django_secret_key')
DEBUG = credentials.get('debug')
ALLOWED_HOSTS = credentials.get('allowed_hosts', [])
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
...
]
if DEBUG:
INSTALLED_APPS += ['debug_toolbar']
...
# project_name/settings/database.py
from project_name.env import credentials
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': credentials.get('db_name', ''),
'USER': credentials.get('db_user', ''),
'HOST': credentials.get('db_host', ''),
'PASSWORD': credentials.get('db_password', ''),
'PORT': '5432',
}
}
the benefits of this solution are :
user specific credentials and configurations for local development without modifying the git repository ;
environment specific configuration, you can have for example three different environments with three different .env.json like dev, stagging and production ;
credentials are not in the repository
I hope this helps, just let me know if you see any caveats with this solution.
I use the folloring file structure:
project/
...
settings/
settings/common.py
settings/local.py
settings/prod.py
settings/__init__.py -> local.py
So __init__.py is a link (ln in unix or mklink in windows) to local.py or can be to prod.py so the configuration is still in the project.settings module is clean and organized, and if you want to use a particular config you can use the environment variable DJANGO_SETTINGS_MODULE to project.settings.prod if you need to run a command for production environment.
In the files prod.py and local.py:
from .shared import *
DATABASE = {
...
}
and the shared.py file keeps as global without specific configs.
Use settings.py for production. In the same directory create settings_dev.py for overrides.
# settings_dev.py
from .settings import *
DEBUG = False
On a dev machine run your Django app with:
DJANGO_SETTINGS_MODULE=<your_app_name>.settings_dev python3 manage.py runserver
On a prod machine run as if you just had settings.py and nothing else.
ADVANTAGES
settings.py (used for production) is completely agnostic to the fact that any other environments even exist.
To see the difference between prod and dev you just look into a single location - settings_dev.py. No need to gather configurations scattered across settings_prod.py, settings_dev.py and settings_shared.py.
If someone adds a setting to your prod config after troubleshooting a production issue you can rest assured that it will appear in your dev config as well (unless explicitly overridden). Thus the divergence between different config files will be minimized.
building off cs01's answer:
if you're having problems with the environment variable, set its value to a string (e.g. I did DJANGO_DEVELOPMENT="true").
I also changed cs01's file workflow as follows:
#settings.py
import os
if os.environ.get('DJANGO_DEVELOPMENT') is not None:
from settings_dev import *
else:
from settings_production import *
#settings_dev.py
development settings go here
#settings_production.py
production settings go here
This way, Django doesn't have to read through the entirety of a settings file before running the appropriate settings file. This solution comes in handy if your production file needs stuff that's only on your production server.
Note: in Python 3, imported files need to have a . appended (e.g. from .settings_dev import *)
If you want to keep 1 settings file, and your development operating system is different than your production operating system, you can put this at the bottom of your settings.py:
from sys import platform
if platform == "linux" or platform == "linux2":
# linux
# some special setting here for when I'm on my prod server
elif platform == "darwin":
# OS X
# some special setting here for when I'm developing on my mac
elif platform == "win32":
# Windows...
# some special setting here for when I'm developing on my pc
Read more: How do I check the operating system in Python?
You want to be able to switch settings, secretes, environment variables and others based on the git branch that you are in and relying on different settings file is okay but in an enterprise situation you would like to hide all your sensitive information from the repo. It is not a best security best practice to expose all the environment variables, secrets of all environments (develop, staging, production, qa etc.,) to all the developers. The following should achieve 2.
isolation of settings as per their environment of deployment
hide sensitive information from git repo
My run.sh
#!/bin/bash
# default environment
export DJANGO_ENVIRONMENT="develop"
BRANCH=$(git rev-parse --abbrev-ref HEAD)
if [ $BRANCH == "main" ]; then
export DJANGO_ENVIRONMENT="production"
elif [ $BRANCH == "release/"* ]; then
export DJANGO_ENVIRONMENT="staging"
else
# for all other branches (feature, support, hotfix etc.,)
echo ''
fi
echo "
BRANCH: $BRANCH
ENVIRONMENT: $DJANGO_ENVIRONMENT
"
python3 myapp/manage.py makemigrations
python3 myapp/manage.py migrate --noinput
python3 myapp/manage.py runserver 0:8000
My vars.py (or secrets.py or whatever name) in the same folder as settings.py of django
vars = {
'develop': {
'environment': 'develop',
'SECRET_KEY': 'mysecretkey',
"DEBUG": "True"
},
'production': {
'environment': 'production',
'SECRET_KEY': 'mysecretkey',
"DEBUG": "False"
},
'staging': {
'environment': 'staging',
'SECRET_KEY': 'mysecretkey',
"DEBUG": "True"
}
}
then in settings.py just do the following
from . import vars # container environment specific vars
import os
DJANGO_ENVIRONMENT = os.getenv("DJANGO_ENVIRONMENT") # declared in run.sh
envs = vars.vars[DJANGO_ENVIRONMENT] # SECURITY WARNING: keep the secret key
used in production secret!
SECRET_KEY = envs["SECRET_KEY"]
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = envs["DEBUG"]
Let developers have their own vars.py in their local machine but during deployment your cicd pipeline can insert the actual vars.py with actual valures or some script should insert it. If you are using gitlab cicd then you can store the entire vars.py as an environment variable
This seems to have been answered, however a method which I use as combined with version control is the following:
Setup a env.py file in the same directory as settings on my local development environment that I also add to .gitignore:
env.py:
#!usr/bin/python
DJANGO_ENV = True
ALLOWED_HOSTS = ['127.0.0.1', 'dev.mywebsite.com']
.gitignore:
mywebsite/env.py
settings.py:
if os.path.exists(os.getcwd() + '/env.py'):
#env.py is excluded using the .gitignore file - when moving to production we can automatically set debug mode to off:
from env import *
else:
DJANGO_ENV = False
DEBUG = DJANGO_ENV
I just find this works and is far more elegant - with env.py it is easy to see our local environment variables and we can handle all of this without multiple settings.py files or the likes. This methods allows for all sorts of local environment variables to be used that we wouldn't want set on our production server. Utilising the .gitignore via version control we are also keeping everything seamlessly integrated.
For the problem of setting files, I choose to copy
Project
|---__init__.py [ write code to copy setting file from subdir to current dir]
|---settings.py (do not commit this file to git)
|---setting1_dir
| |-- settings.py
|---setting2_dir
| |-- settings.py
When you run django, __init__py will be ran. At this time , settings.py in setting1_dir will replace settings.py in Project.
How to choose different env?
modify __init__.py directly.
make a bash file to modify __init__.py.
modify env in linux, and then let __init__.py read this variable.
Why use to this way?
Because I don't like so many files in the same directory, too many files will confuse other partners and not very well for IDE.(IDE cannot find what file we use)
If you do not want to see all these details, you can divide project into two part.
make your small tool like Spring Initializr, just for setup your project.(do sth like copy file)
your project code
I'm using different app.yaml file to change configuration between environments in google cloud app engine.
You can use this to create a proxy connection in your terminal command:
./cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:1433
https://cloud.google.com/sql/docs/sqlserver/connect-admin-proxy#macos-64-bit
File: app.yaml
# [START django_app]
service: development
runtime: python37
env_variables:
DJANGO_DB_HOST: '/cloudsql/myproject:myregion:myinstance'
DJANGO_DEBUG: True
handlers:
# This configures Google App Engine to serve the files in the app's static
# directory.
- url: /static
static_dir: static/
# This handler routes all requests not caught above to your main app. It is
# required when static routes are defined, but can be omitted (along with
# the entire handlers section) when there are no static files defined.
- url: /.*
script: auto
# [END django_app]
I create a file named "production" in the working directory in production.
#settings.py
production = Path("production")
DEBUG = False
#if it's dev mode
if not production.is_file():
INSTALLED_APPS +=[
#apps_in_development_mode,
#...
]
DEBUG = True
#other settings to override the default production settings
You're probably going to use the wsgi.py file for production (this file is created automatically when you create the django project). That file points to a settings file. So make a separate production settings file and reference it in your wsgi.py file.
What we do here is to have an .ENV file for each environment. This file contains a lot of variables like ENV=development
The settings.py file is basically a bunch of os.environ.get(), like ENV = os.environ.get('ENV')
So when you need to access that you can do ENV = settings.ENV.
You would have to have a .env file for your production, testing, development.
This is my solution, with different environements for dev, test and prod
import socket
[...]
DEV_PC = 'PC059'
host_name = socket.gethostname()
if host_name == DEV_PC:
#do something
pass
elif [...]