I have sensitive data (database passwords) in settings.py and I was advised to upload my Django project to a github repository before pushing it to Heroku on their "Getting Started with Django on Heroku". If I put settings.py in .gitignore, then presumably it won't get deployed with my project. How can I prevent settings.py from being exposed but still get it deployed with my project ?
You can use environment variables (with heroku config:add SECRET=my-secret) to setup sensitive data and retrieve them in your settings with:
SECRET = os.environ.get('SECRET', 'my-default-secret-key')
If you don't want to be able to start your app without having set up some data, use this syntax instead:
SECRET = os.environ['SECRET']
It will raise an exception if you didn't set the SECRET environment variable.
You should use a tool designed for factoring out sensitive data. I use YamJam https://pypi.python.org/pypi/yamjam/ . It allows all the advantages of the os.environ method but is simpler -- you still have to set those environ variables, you'll need to put them in a script/ rc file somewhere. YamJam eliminates these questions and stores these config settings in a config store outside of the project. This allows you to have different settings for dev, staging and production.
from YamJam import yamjam
secret = yamjam()['myproject']['secret']
Is the basic usage. And like the os.environ method, it is not framework specific, you can use it with Django or any other app/framework. I've tried them all, multiple settings.py files, brittle logic of if/then and environment wrangling. In the end, I switched to yamjam and haven't regretted it.
Related
Hi I am relatively new to programming and building my first flask project and I haven't been able to figure out if I should prefer accessing environment variables by using dotenv / load_dotenv or using them from a config.py file.
I understand the config route is more flexible but my question is specifically to do with environment variables.
Is there a best practice here? [I am building a simple app that will be hosted externally]
Best practices dictate that any value which is secret should not be hard-coded into any files which persist with the project or are checked into source control. Your config file is very likely to be saved in source control, so it should not store secrets, but instead load them from the environment variables set at execution time of the app. For example, let's say you are configuring an SMTP relay:
MAIL_PORT is a value that is not secret and not likely to change so it is a good candidate to be set in your config file.
MAIL_PASSWORD is a secret value that you do not want saved in your project's repository, so it should be loaded from the host's environment variables.
In this example, your config file might contain entries that look something like this:
MAIL_PORT = 465
MAIL_PASSWORD = os.environ.get('MAIL_PASSWORD')
Beyond evaluating whether or not a config value is a secret, also consider how often the value will change and how hard it would be to make that change. Anything hard-coded into your config file will require changing the file and adding a new commit to your source control, possibly even triggering a full CI/CD pipeline process. If the value were instead loaded from environment variables then this value could be changed by simply stopping the application, exporting the new value as an environment variable, and restarting the application.
Dotenv files are simply a convenience for grouping a number of variables together and auto-loading them to be read by your configuration. A .env file is not always used as these values can be manually exported when the app is invoked or handled by another system responsible for starting or scaling your application. Do not check .env or .flaskenv files into your source control.
I want to configure Sentry in a Django app to report errors using different environments, like staging and production. This way I can configure alerting per environment.
How can I configure different environments for Raven using different Django settings? The environment variable is not listed at the Raven Python client arguments docs, however I can find the variable in the raven-python code.
If you are setting environment as a constant within Django settings, you can set the environment argument when initializing the raven-python client.
You're correct—our docs didn't include the environment argument. I've updated them to include it. Thanks for raising the issue.
You can use different settings for different branches. You have your main one, with all shared settings. And for develop branch you have dev.py settings and for production you have your prod.py. And while deploying your app you just specify which settings are meant to be used. If not you can also use GitPython package. Where you make something like this:
if branch in ['develop']:
DEBUG = True
RAVEN_CONFIG = {
'dsn': 'your_link_to_raven',
}
else:
#some other settings
I have a Flask application that I want to deploy to Amazon Elastic Beanstalk, but have the following problem:
Our configuration contains a lot of secret API keys, so we don't have it checked into Git.
Elastic Beanstalk only deploys the files that are committed to Git, so our config doesn't get uploaded as part of the deployment process.
Elastic Beanstalk offers a way to set and read environment variables - but Flask typically expects environment variables to be set in Python files that are imported at runtime.
Is there a way we can get our configuration file automatically uploaded to AWS EB alongside the source code when we deploy it? I considered having it in a file that gets created via configuration in .ebextensions/... but that would just need to be in Git anyway, defeating the object.
Or do we have to (a) convert all our code to read configuration from environment variables, and (b) create some sort of pre-startup script that injects the values in the current config scripts into the environment? It wouldn't be ideal to have to maintain 2 totally different ways of reading configuration data into our app depending on whether it's running on AWS or not.
I have seen this question/answer - How to specify sensitive environment variables at deploy time with Elastic Beanstalk - and I don't think that adequately addresses the issue because it is not very scalable to large numbers of config options nor deals with the fact that Flask typically expects its config in a different format.
If you are doing a 12 factor application, the simplest way to handle this kind of configuration is to lookup environment variables by default, and fall back on your hard-coded configurations:
from os import environ
from flask.config import Config
class EnvConfiguration(Config):
"""Configuration sub-class that checks its environment
If no environment variable is available,
it falls back to the hard-coded value provided"""
def __getitem__(self, key):
environ.get(key, super(EnvConfig, self).__getitem__(key))
You can then override either Flask.config_class (v1+) or Flask.make_config to ensure that the configuration used is the one you want:
class CustomApp(Flask):
# 1.0+ way
config_class = EnvConfiguration
# 0.8+ way (you only need one of these)
def make_config(self, instance_relative=False):
config = super(CustomApp, self).make_config(instance_relative=instance_relative)
config = EnvConfig(**config)
return config
I deploy my project to GAE over Github. There is some foreign API-key which I don't want to save in repository and make them public. Is it possible to set an environment variable for a project in GAE control panel so I can catch it in my application?
You can store your keys in datastore. Later when you need them in the code, you can fetch them from datastore and cache them by memcache.
I prefer using the Datastore for keys like this. See the code in my answer at Securely storing environment variables in GAE with app.yaml
That code auto-generates placeholder values that you can then update from the developer console. Also, it uses the ndb library, so reading the keys is fast.
You can define environment variables in configuration file for App Engine application. In case of Python, it is app.yaml
env_variables:
MY_ENV_VAR: 'some_value'
You can find more details here.
There is no such a thing like project parameters that can be defined in Developers Console at the moment.
I have a Flask-based python app that needs a bunch of configuration information (e.g. database connection parameters) on app start.
In my nginx configuration, I can provide parameters using uwsgi_param, as shown in this SO question.
However my problem is that the request.environ object is not available outside of a request handler, so I can't use it at application start. Furthermore, I'm trying to provide a mostly deployment-agnostic configuration mechanism so that I can test using Flask's embedded server during development and use parameters passed by uWSGI in production.
What's the best way to get configuration into my application so that it's accessible via os.environ or something similar?
Look at http://flask.pocoo.org/docs/config/#development-production. You always can have development, test and production config, you also can get some settings from OS envarment or specific file.
For example I have config module and import some secret settings (which don't want store with source code) from another module. Then on deploy I just replace file with secret settings. Probably better use for this OS envarement.