In my settings.py i'm using django-environ like this:
import os
import environ
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
env = environ.Env(
SECRET_KEY=str,
)
env_path = os.path.join(BASE_DIR, '.env')
environ.Env.read_env('.env')
SECRET_KEY = env('SECRET_KEY')
My .env file looks like this
SECRET_KEY = ******************
However, when I run the app i get
django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY
environment variable
The .env file is found and lines are being read from it, so there's no problem with finding it, but somehow it doesn't work.
While following the execution thread i discovered that the regex for interpreting the .env lines is returning nothing
environ/environ.py
for line in content.splitlines():
m1 = re.match(r'\A(?:export )?([A-Za-z_0-9]+)=(.*)\Z', line)
if m1:
key, val = m1.group(1), m1.group(2)
m2 = re.match(r"\A'(.*)'\Z", val)
if m2:
val = m2.group(1)
m3 = re.match(r'\A"(.*)"\Z', val)
if m3:
val = re.sub(r'\\(.)', r'\1', m3.group(1))
cls.ENVIRON.setdefault(key, str(val))enter code here
re.match(r'\A(?:export )?([A-Za-z_0-9]+)=(.*)\Z', line) is returning none
Am i missing something here?
I found the answer by checking the regex responsible for interpreting the .env lines - \A(?:export )?([A-Za-z_0-9]+)=(.*)\Z
There should be no spaces between the key and the value
Problem:
KEY = VALUE
Good:
KEY=VALUE
Related
There's is very simple. I have a prob.py with environment variables:
SECRET_KEY = mykey
POSTGRES_PASSWORD=mypspwd
CONTACT = myemail#me.com
APIKEY_BOT = 11aa11a1-111a-1aa1-11a1-11aaa1a1a111
When I try to access this from my Django settings file, I get the following error:
File "/usr/src/app/prjbot/prob.py", line 4
APIKEY_BOT= 11aa11a1-111a-1aa1-11a1-11aaa1a1a111
^
SyntaxError: invalid syntax
I'm assuming that the issue is with the hyphens. I tried adding quotes around the API key, but that didn't work. What am I missing?
I found other stack overflow issues where users want the variable name to include a hyphen, but I didn't find this issue where the var is being defined with a string that includes hyphens.
You are using environmental variables incorrectly. Environmental variables are designed to sit outside of your code in a file named .env.
You can then setup your Python code to import the variables from the .env file.
.env
SECRET_KEY = mykey
POSTGRES_PASSWORD=mypspwd
CONTACT = myemail#me.com
APIKEY_BOT = 11aa11a1-111a-1aa1-11a1-11aaa1a1a111
prob.py
Use python-decouple to import your variables.
from decouple import config
SECRET_KEY = config('SECRET_KEY')
POSTGRES_PASSWORD = config('POSTGRES_PASSWORD')
CONTACT = config('CONTACT')
APIKEY_BOT = config('APIKEY_BOT')
For example, now I have this .ini file:
[DEFAULT]
db_user = user_test
db_password = psw_test
db_host = localhost
db_name = db_test
...
I want to do:
export ENV_DB_USER=user_test
export ENV_DB_PSW=psw_test
export ENV_DB_HOST=localhost
export ENV_DB_NAME=db_test
Then I want to do somethings like this in .ini file:
[DEFAULT]
db_user = ${ENV_DB_USER}
db_password = ${ENV_DB_PSW}
db_host = ${ENV_DB_HOST}
db_name = ${ENV_DB_NAME}
...
I have try to use %()s syntax but unfortunately not working.
Where I'm doing wrong?
Thanks :)
Unfortunately they are not supported right now - you have to load them at runtime instead of in the ini file itself. A common approach in my apps is to define a render-config script which takes a site.ini.in template and renders it out to site.ini using jinja2 with env vars available to it. It's not very satisfying but it does work well.
import jinja2
import os
def multiline(val):
lines = (l.strip() for l in val.strip().split('\n'))
return '\n ' + '\n '.join(lines)
def main(cli, args):
env = jinja2.Environment(
loader=jinja2.FileSystemLoader(os.getcwd()),
undefined=jinja2.StrictUndefined,
)
env.filters['multiline'] = multiline
template = env.get_template(args.template)
result = template.render({
'env': os.environ,
})
with open(args.config_file, 'w', encoding='utf8') as fp:
fp.write(result)
When my settings.py has ALLOWED_HOSTS = [] - the server works fine. But When I do ALLOWED_HOSTS = config('ALLOWED_HOSTS') - it gives the error:
DisallowedHost at /admin/login/
Invalid HTTP_HOST header: '127.0.0.1:8000'. You may need to add '127.0.0.1' to ALLOWED_HOSTS.
However when I go into python shell I'm successfully able to import ALLOWED_HOSTS - and it prints [] which is the correct value.
Any reason why I still get the error?
PS: The config is a feature of the python-decouple package - used to store sensitive values.
The python-decouple readme suggests thay you have to cast the ALLOWED_HOSTS to a list. The example it gives is:
os.environ['ALLOWED_HOSTS'] = '.localhost, .herokuapp.com'
config('ALLOWED_HOSTS', cast=lambda v: [s.strip() for s in v.split(',')])
You appear to be doing config('ALLOWED_HOSTS'), so perhaps your ALLOWED_HOSTS is being evaluated as a string '[]' instead of an empty list [].
In this case django expected a list in ALLOWED_HOSTS, but environment variables are just a plan text, you have to convert before use it, fortunately python decouple has a optional parameter called cast
Consider the following .env file:
.env
ALLOWED_HOSTS=127.0.0.1, .herokuapp.com
DEBUG=True
PORT=5403
To access these values in your setting.py you have to import the util class Csv, it will convert a plan string in a list, but pay attention in your .env file or environment variable don't forget to separate the values with comma and space (, ).
For built-in types you don't need to import any aditional class, you can use int, str, bool...
setting.py
from decouple import config, Csv
ALLOWED_HOSTS = config('ALLOWED_HOSTS', default=[], cast=Csv())
DEBUG = config('DEBUG', default=False, cast=bool)
PORT= config('PORT', cast=int)
I am new to pyramid.
The Issue is I am not able to figure out how app specific settings (key value pairs) work in pyramid.
This is what I have done after various google searches and other stackoverflow answers:
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
if '__file__' in global_config:
settings.update(
load_sensitive_settings(global_config['__file__'], global_config))
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
# config.add_static_view('static', 'static', cache_max_age=3600)
# config.add_route('home', '/')
config.add_route(
'tags',
'/tags', request_method='POST', accept='application/json', )
config.scan()
return config.make_wsgi_app()
def load_sensitive_settings(configurationPath, defaultByKey):
'Load sensitive settings from hidden configuration file'
# configFolder, configName = os.path.split(configurationPath)
# sensitivePath = os.path.join(configFolder, '.' + configName)
sensitivePath = configurationPath
settings = {}
configParser = ConfigParser.ConfigParser(defaultByKey)
if not configParser.read(sensitivePath):
log.warn('Could not open %s' % sensitivePath)
return settings
settings.update(configParser.items('custom'))
return settings
I have a file where I try to fetch settings like this:
from pyramid.threadlocal import get_current_registry
settings = get_current_registry().settings
value = settings['my_key']
But I always get settings object as None.
This is how I am defining my custom settings in development.ini
[custom]
my_key = ''
This is how I start my server in develpoment
pserve development.ini
I have read that request.settings can give me settings, But that approach is not feasible for me as my key contains the name of a file which is 1.5GBs and it has to be present in memory all the time. It takes around 5 minutes to load that file in server, hence cannot load the file on demand.
Please advice.
Thanks a lot for all the help in advance.
Update:
Thanks to all the answers provided, I finally made it work.
This is how my main function looks like:
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
if '__file__' in global_config:
init_config(global_config['__file__'])
And I made a config file, this is how my config file looks like:
import ConfigParser
settings = dict()
def init_config(filename):
config = ConfigParser.ConfigParser()
config.read(filename)
settings_dict = config.items('custom')
settings.update(settings_dict)
Now wherever I want settings, I just do:
from projectname.config import settings
settings.get('my_key')
And I put my app specific settings (development/production.py) like this
[custom]
my_key = value
Regards
HM
Easiest way is putting your settings to the app main section with dot separated names. Example:
[app:main]
websauna.site_name = Trees
websauna.site_tag_line = Enjoy
websauna.site_url = http://localhost:6543
websauna.site_author = Trees team
Then you can do:
my_settings_value = request.registry.settings.get("websauna.site_name", "Default value)
WSGI pipeline does not bring you settings from other sections and you need to reparse the INI file with ConfigParser if you want to access the other sections (as far as I know).
If you need to load a lot of data during development time just store a filename in settings and load the file when you need to access the data, so that you don't slow the web server startup.
Here is my working solution:
config.ini
[APP.CONFIG]
url = http://....
[SMTP.CONFIG]
smtp.server = ...
smtp.port = 25
smtp.login = ...
smtp.password = ...
smtp.from = ...
[DB.CONFIG]
db.database=...
db.host=...
db.port=..
db.user=...
db.password=...
config.py
import configparser
config = configparser.ConfigParser()
config._interpolation = configparser.ExtendedInterpolation()
config.read(encoding='utf-8', filenames=['path to file/config.ini'])
smtp = config['SMTP.CONFIG']
db = config['DB.CONFIG']
mail = config['APP.CONFIG']
And how i use it in APP
from config import db
host = db['db.host']
If, like me, you are using PasteDeploy with Pyramid, the Pyramid docs here explain how you can use a [DEFAULT] section in your .ini configuration file to hold your custom parameters.
You might also benefit from reading the documentation on .ini files, since it gives some snippets which make it all much clearer.
In trying to find a place to store and save settings beyond settings.py and the database, I used an environment.json for environment variables. I import these in settings.py.
My problem is that when I try to change or store new values in my environment, env, settings.py does not notice the change - perhaps because the time and number of times settings.py is read by Django.
Is there a way I would be able to use my environment variables the way I want like attempted below?
# settings.py
import json
with open('/home/dotcloud/environment.json') as f:
env = json.load(f)
EMAIL_HOST = env.get('EMAIL_PORT', '500')
# views.py
import json
def site_configuration(request):
with open('/home/dotcloud/environment.json') as f:
env = json.load(f)
if request.method == 'POST':
os.environ['EMAIL_PORT'] = request.POST['email_port']
return render(request, ...)
# python manage.py shell demo
>>> import json
>>> with open('/home/dotcloud/environment.json') as f:
... env = json.load(f)
...
>>> project_settings.EMAIL_PORT
'500'
>>> env['EMAIL_PORT']
Traceback (most recent call last):
File "<console>", line 1, in <module>
KeyError: 'EMAIL_PORT'
>>> env['EMAIL_PORT'] = "123"
>>> env['EMAIL_PORT']
'123'
>>> project_settings.EMAIL_PORT
'500'
>>> project_settings.EMAIL_PORT == env['EMAIL_PORT']
False'
And if not, how else could I store changeable settings that are retrieved by settings.py somewhere in my Django project?
You might want to look into foreman (GitHub) or honcho (GitHub). Both of these look for a .env file in your current directory from which to load local environment variables.
My .env looks like this for most projects (I use dj-database-url for database configuration):
DATABASE_URL=sqlite://localhost/local.db
SECRET_KEY=<a secret key>
DEBUG=True
In your settings.py file, you can load these settings from os.environ like this:
import os
DEBUG = os.environ.get('DEBUG', False)
If there are required settings, you can assert their presence before trying to set them:
assert 'SECRET_KEY' in os.environ, 'Set SECRET_KEY in your .env file!'
SECRET_KEY = os.environ['SECRET_KEY']
I've been using this method of handling local settings for the last few projects I've started and I think it works really well. One caveat is to never commit your .env to source control. These are local settings that exist only for the current configuration and should be recreated for a different environment.
I see the question changed slightly, the original answers are still below but this one has a slightly different answer:
First, make sure you are using the right settings.py (print 'This file is being loaded' should do the trick).
Second, personally I would advise against using json files for config since it is less dynamic than Python files, but it should work regardless.
My recommended way of doing something like this:
create a base_settings.py file with your standard settings
create a settings.py which will be your default settings import. This file should have a from base_settings import * at the top to inherit the base settings.
If you want to have a custom settings file, dotcloud_settings.py for example, simply add the from dotcloud_settings import settings (or base_settings) and set the environment variable DJANGO_SETTINGS_MODULE to dotcloud_settings or your_project.dotcloud_settings depending on your setup.
Do note that you should be very careful with importing Django modules from these settings files. If any module does a from django.conf import settings it will stop parsing your settings after that point.
As for using json files, roughly the same principle of course:
Once again, make sure you don't have anything that imports django.conf.settings here
Make all of the variables within your json file global to your settings file:
import json
with open('/home/dotcloud/environment.json') as f:
env = json.load(f)
# A little hack to make all variables within our env global
globals().update(env)
Regardless though, I'd recommend turning this around and letting the settings file import this module instead.
Also, Django doesn't listen to environment variables by default (besides the DJANGO_SETTINGS_MODULE so that might be the problem too.