Cannot use environment variables for settings in Django - python

In trying to find a place to store and save settings beyond settings.py and the database, I used an environment.json for environment variables. I import these in settings.py.
My problem is that when I try to change or store new values in my environment, env, settings.py does not notice the change - perhaps because the time and number of times settings.py is read by Django.
Is there a way I would be able to use my environment variables the way I want like attempted below?
# settings.py
import json
with open('/home/dotcloud/environment.json') as f:
env = json.load(f)
EMAIL_HOST = env.get('EMAIL_PORT', '500')
# views.py
import json
def site_configuration(request):
with open('/home/dotcloud/environment.json') as f:
env = json.load(f)
if request.method == 'POST':
os.environ['EMAIL_PORT'] = request.POST['email_port']
return render(request, ...)
# python manage.py shell demo
>>> import json
>>> with open('/home/dotcloud/environment.json') as f:
... env = json.load(f)
...
>>> project_settings.EMAIL_PORT
'500'
>>> env['EMAIL_PORT']
Traceback (most recent call last):
File "<console>", line 1, in <module>
KeyError: 'EMAIL_PORT'
>>> env['EMAIL_PORT'] = "123"
>>> env['EMAIL_PORT']
'123'
>>> project_settings.EMAIL_PORT
'500'
>>> project_settings.EMAIL_PORT == env['EMAIL_PORT']
False'
And if not, how else could I store changeable settings that are retrieved by settings.py somewhere in my Django project?

You might want to look into foreman (GitHub) or honcho (GitHub). Both of these look for a .env file in your current directory from which to load local environment variables.
My .env looks like this for most projects (I use dj-database-url for database configuration):
DATABASE_URL=sqlite://localhost/local.db
SECRET_KEY=<a secret key>
DEBUG=True
In your settings.py file, you can load these settings from os.environ like this:
import os
DEBUG = os.environ.get('DEBUG', False)
If there are required settings, you can assert their presence before trying to set them:
assert 'SECRET_KEY' in os.environ, 'Set SECRET_KEY in your .env file!'
SECRET_KEY = os.environ['SECRET_KEY']
I've been using this method of handling local settings for the last few projects I've started and I think it works really well. One caveat is to never commit your .env to source control. These are local settings that exist only for the current configuration and should be recreated for a different environment.

I see the question changed slightly, the original answers are still below but this one has a slightly different answer:
First, make sure you are using the right settings.py (print 'This file is being loaded' should do the trick).
Second, personally I would advise against using json files for config since it is less dynamic than Python files, but it should work regardless.
My recommended way of doing something like this:
create a base_settings.py file with your standard settings
create a settings.py which will be your default settings import. This file should have a from base_settings import * at the top to inherit the base settings.
If you want to have a custom settings file, dotcloud_settings.py for example, simply add the from dotcloud_settings import settings (or base_settings) and set the environment variable DJANGO_SETTINGS_MODULE to dotcloud_settings or your_project.dotcloud_settings depending on your setup.
Do note that you should be very careful with importing Django modules from these settings files. If any module does a from django.conf import settings it will stop parsing your settings after that point.
As for using json files, roughly the same principle of course:
Once again, make sure you don't have anything that imports django.conf.settings here
Make all of the variables within your json file global to your settings file:
import json
with open('/home/dotcloud/environment.json') as f:
env = json.load(f)
# A little hack to make all variables within our env global
globals().update(env)
Regardless though, I'd recommend turning this around and letting the settings file import this module instead.
Also, Django doesn't listen to environment variables by default (besides the DJANGO_SETTINGS_MODULE so that might be the problem too.

Related

Name error when trying to import API key from .env

I am trying to store my API Keys in a .env file
I created the file as a File containing settings for editor file type. Stored my APIKeys
TWILIO_ACCOUNT_SID=***
TWILIO_AUTH_TOKEN=***
TWIML_APPLICATION_SID=***
TWILIO_API_KEY=***
TWILIO_API_SECRET=***
Installed decouple, imported and used config to retrieve my API tokens in my settings.py file
from decouple import config
...
TWILIO_ACCOUNT_SID = config(TWILIO_ACCOUNT_SID)
TWILIO_AUTH_TOKEN = config(TWILIO_AUTH_TOKEN)
TWIML_APPLICATION_SID = config(TWIML_APPLICATION_SID)
TWILIO_API_KEY = config(TWILIO_API_KEY)
TWILIO_API_SECRET = config(TWILIO_API_SECRET)
I am however getting the error message:
TWILIO_ACCOUNT_SID = config(TWILIO_ACCOUNT_SID)
NameError: name 'TWILIO_ACCOUNT_SID' is not defined
You don't need to use the decouple library to read your environment variables.
Firstly download the .env plug-in supporter for PyCharm (if that's what you're using)
https://www.codestudyblog.com/cs2112pyc/1224021812.html
This will allow you to set and get variables from your file. Make sure your configuration has the correct .env file set.
my .env file has the variable set to:
TWILIO_ACCOUNT_SID=SUPER SECRET KEY
Then all you need to is:
import os
twilio_key = os.environ.get('TWILIO_ACCOUNT_SID')
print(twilio_key)
>>>SUPER SECRET KEY
Process finished with exit code 0

How to access Environment Variables in Django with Django Environ?

I'm a Node developer but I need to create a Django app (i'm totally beginner in Django).
I need to read some data from an API but ofc, I shouldn't hardcode the API url.
So having API_BASE_URL=api.domain.com in my .env file, in Node I would access the variables in my functions this way:
import ('dotenv/config');
import axios from 'axios';
baseUrl = process.env.API_BASE_URL;
function getApiData() {
return axios.get(baseUrl);
}
So how would be the Python/Django version of it?
Saying I have the function bellow:
import ???
def get_api_data():
url = ????
import environ
# reading .env file
environ.Env.read_env()
def get_api_data():
url = env('API_BASE_URL')
Let's say you have a .env file saved in the same directory as your manage.py file.
You can then go to settings.py and do:
from decouple import config
API_BASE_URL = config('API_BASE_URL')
Assuming your .env file looks like:
API_BASE_URL='some.url'

Detect If Flask App Is Running Locally

I would like certain parts of my code to not run while it is being run locally.
This is because, I am having trouble installing certain dependencies locally, for the code to run.
Specifically, memcache doesn't work for me locally.
#app.route('/some_url_route/')
#cache.cached(timeout=2000) #ignore this locally
def show_a_page():
How would the app somehow ignore the cache section of the code above, when running locally?
In my code I follow a Django-esq model and have a main settings.py file I keep all my settings in.
In that file putting DEBUG = True for your local environment (and False for production) I then use:
from settings import DEBUG
if DEBUG:
# Do this as it's development
else:
# Do this as it's production
So in your cache decorator include a similar line that only checks memcached if DEBUG=False
You can then load all these settings into your Flask setup as detailed in the configuration documentation.
If you're using Flask-Cache, then just edit the settings:
if app.debug:
app.settings.CACHE_TYPE = 'null' # the cache that doesn't cache
cache = Cache(app)
...
A better approach would be to have separate settings for production and development. I use a class-based approach:
class BaseSettings(object):
...
class DevelopmentSettings(BaseSettings):
DEBUG = True
CACHE_TYPE = 'null'
...
class ProductionSettings(BaseSettings):
CACHE_TYPE = 'memcached'
...
And then import the appropriate object when you setup your app (config.py is the name of the file which contains the settings):
app.config.from_object('config.DevelopmentConfig')

Configuring eulexistdb with python bringing errors in django setting module

I have following code written in python in order to communicate with ExistDB using eulexistdb module.
from eulexistdb import db
class TryExist:
def __init__(self):
self.db = db.ExistDB(server_url="http://localhost:8899/exist")
def get_data(self, query):
result = list()
qresult = self.db.executeQuery(query)
hits = self.db.getHits(qresult)
for i in range(hits):
result.append(str(self.db.retrieve(qresult, i)))
return result
query = '''
let $x:= doc("/db/sample/books.xml")
return $x/bookstore/book/author/text()
'''
a = TryExist()
response = a.get_data(query)
print response
I am amazed that this code runs fine in Aptana Studio 3 giving me the output I want, but when running from other IDE or using command "python.exe myfile.py" brings following error:
django.core.exceptions.ImproperlyConfigured: Requested setting EXISTDB_TIMEOUT, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
I used my own localsetting.py to solve the problem using following code:
import os
# must be set before importing anything from django
os.environ['DJANGO_SETTINGS_MODULE'] = 'localsettings'
... writing link for existdb here...
Then I get error as:
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
How do I configure the setting in Django to suit for ExistDB? Help me here please..
Never Mind. I found the answer with little research from this site. What I did was created a localsetting.py file with following configurations.
EXISTDB_SERVER_USER = 'user'
EXISTDB_SERVER_PASSWORD = 'admin'
EXISTDB_SERVER_URL = "http://localhost:8899/exist"
EXISTDB_ROOT_COLLECTION = "/db"
and in my main file myfile.py I used :
from localsettings import EXISTDB_SERVER_URL
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'localsettings.py'
and In the class TryExist I changed in __ init __() as:
def __init__(self):
self.db = db.ExistDB(server_url=EXISTDB_SERVER_URL)
PS: Using only os.environ['DJANGO_SETTINGS_MODULE'] = 'localsettings' brings the django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty..
The reason your code works in an IDE but not at the command line is probably that you have a difference in what Python environments are used to run your code.
I've done a couple of tests:
Virtualenv with eulexistdb installed but not Django. eulexistdb tries to load django.conf but fails and so does not try to get its configuration from a Django configuration. Ultimately, your code runs without error.
Virtualenv with 'eulexistdb*and* Django:eulexistdbtries to loaddjango.conf` and succeed. I then tries to get is configuration from the Django configuration but fails. I get the same error you describe in your question.
To prevent the error in the presence of a Django installation, the problem can be fixed by adding a Django configuration like you did in your accepted self-answer. But if the code you are writing does not otherwise use Django, that's a bit of a roundabout way to get your code to run. The most direct way to fix the problem is to simply add a timeout parameter to the code that creates the ExistDB instance:
self.db = db.ExistDB(
server_url="http://localhost:8080/exist", timeout=None)
If you do this, then there won't be any error. Setting the timeout to None leaves the default behavior in place but prevents eulexistdb from looking for a Django configuration.

Build Automation for Django

I'm pretty new to doing sysadmin stuff for my development and to the django framework. I want to have a different username/password for my local dev station and to my production environment.
I'm using dotcloud as the server. I can write a post install script (in python, bash, whatever) and it will execute it on every new push.
However I don't know how to go about this. Do I need to write this myself? is there a python/django build automation tools that will help me with that?
Clarification: how can I change debug=false in settings.py to true just on the server?
The django standard way is to use an environmanet variable DJANGO_SETTINGS_MODULE. Point it to different settings and let both import a common settings module for common things:
# settings_production.py
from settings_common import *
DEBUG = False
DATABASES = {...}
# settings_development.py
from settings_common import *
DEBUG = True
DATABASES = {...}
# settings_common.py
INSTALLED_APPS = (...) # etc
You can also use an alternative strategy of using one main settings and import names from another one depending on some system condition, like getting os.platform.node() or socket.gethostname() and switch over that value (or part of it).
reversed_hostname_parts = socket.gethostname().split('.').reverse()
host_specific = {
('com', 'dotcloud'): 'production',
('local'): 'dev',
}
for index in range(len(reversed_hostname_parts)):
identifier = tuple(reversed_hostname_parts[:index+1])
if identifier in host_specific:
extra_settings = host_specific[identifier]
break
else: # executed when the loop has not been `break`ed
extra_settings = 'dev' # any default value
if extra_settings == 'dev':
from development_settings import *
elif extra_settings == 'production':
from production_settings import *
EDIT: added link
See https://code.djangoproject.com/wiki/SplitSettings for other strategies.
I usually import my development settings at the end of production settings.py, if my project is residing on a local directory structure.
You can also store your DB settings and other settings that are different in production and development in a separate file and remove them from your SVN, Git of whatever you use.
Just add this at the end of your settings.py:
try:
from myapp.specific_settings import *
except ImportError:
pass
In this case, specific_settings will be different in production and development environment.
If you want to dynamically choose between development and production servers use this at the end of settings:
import os
directory = os.path.dirname(__file__)
if directory == '/home/yourname/development/':
from myapp.development_settings import *
else:
from myapp.production_settings import *
Note that I wrote this on top of my head and there might be some bugs in it. I will verify that when I go home.

Categories