Based on the Configuration Handling Documents for Flask the section of Configuring from Files mentions a possibility to configure the App using files however it provides no example or mention of files that are not Python Files.
Is it possible to configure apps via files like config.yml or config.toml?
My Current flask app has configurations for two distinct databases and since I am using flask-restplus there are additional configurations for Swagger documentations.
Snippet:
from flask import Flask
app = Flask(__name__)
def configure_app(flask_app):
# MongoDB Setting
flask_app.config['MONGO_URI'] = 'mongodb://user:password#mongo_db_endpoint:37018/myDB?authSource=admin'
flask_app.config['MONGO_DBNAME'] = 'myDB'
# InfluxDB Setting
flask_app.config['INFLUXDB_HOST'] = 'my_influxdb_endpoint'
flask_app.config['INFLUXDB_PORT'] = 8086
flask_app.config['INFLUXDB_USER'] = 'influx_user'
flask_app.config['INFLUXDB_PASSWORD'] = 'influx_password'
flask_app.config['INFLUXDB_SSL'] = True
flask_app.config['INFLUXDB_VERIFY_SSL'] = False
flask_app.config['INFLUXDB_DATABASE'] = 'IoTData'
# Flask-Restplus Swagger Configuration
flask_app.config['RESTPLUS_SWAGGER_UI_DOC_EXPANSION'] = 'list'
flask_app.config['RESTPLUS_VALIDATE'] = True
flask_app.config['RESTPLUS_MASK_SWAGGER'] = False
flask_app.config['ERROR_404_HELP'] = False
def main():
configure_app(app)
if __name__ == "__main__":
main()
I would like to avoid setting large number of Environment Variables and wish to configure them using a config.toml file?
How is this achieved in flask?
You can use the .cfg files and from_envvar to achieve this. Create config file with all your environment variables.
my_config.cfg
MONGO_URI=mongodb://user:password#mongo_db_endpoint:37018
..
..
ERROR_404_HELP=False
Then set the env var APP_ENVS=my_config.cfg. Now all you need to do is use from_envvars given by Flask.
def configure_app(flask_app):
flask_app.config.from_envvar('APP_ENVS')
# configure any other things
# register blue prints if you have any
Quoting from documentation:
Configuring from Data Files
It is also possible to load configuration from a file in a format of
your choice using from_file(). For example to load from a TOML file:
import toml
app.config.from_file("config.toml", load=toml.load)
Or from a JSON file:
import json
app.config.from_file("config.json", load=json.load)
EDIT: The above feature is new for v2.0.
Link to the documentation reference:
Class Flask.config, method from_file(filename, load, silent=False)
Related
The parameter is not read from files other than the run.py . run.py is the Flask startup script
from run.py :
from flask import Flask
app = Flask(__name__)
print(app.config["PARAMETER"])
From any other python file in project following error is returned :
KeyError: 'PARAMETER'
How to read configuration file from other locations within project other than startup script file ?
Update :
Reading config file :
app.config.from_pyfile(os.path.join(".", "myconfig.conf"))
Contents of myconfig.conf :
PARAMETER = "test"
I think you just simply missed loading parameters.
If you do inline loadign of parameters, you would specifiy it as:
from flask import Flask
app = Flask(__name__)
app.config['PARAMETER_1'] = 'test'
assert app.config["PARAMETER_1"] == 'test'
The code above works, now how can one load similar data from file?
# prepare file
from pathlib import Path
Path("myconfig.conf").write_text("PARAMETER_2 = 'test2'\n")
# load it
app.config.from_pyfile("myconfig.conf")
# test it
assert app.config["PARAMETER_2"] == 'test2'
You can't print the PARAMETER key after the creation of the app because there is no any key into its configuration object which its name is PARAMETER. In order to do it, first create the app, import a simple object which holds the configuration and then use app.config.from_object(Config)
#config.py
class Config(object):
PARAMETER = 'my-parameter-value'
#you can add another keys
SECRET_KEY = os.environ.get('SECRET_KEY') or 'you-will-never-guess'
#__init__.py
from config import Config
app = Flask(__name__);
app.config.from_object(Config)
I have a Sanic application, and want to retrieve app.config from a blueprint as it holds MONGO_URL, and I will pass it to a repository class from the blueprint.
However, I could not find how to get app.config in a blueprint. I have also checked Flask solutions, but they are not applicable to Sanic.
My app.py:
from sanic import Sanic
from routes.authentication import auth_route
from routes.user import user_route
app = Sanic(__name__)
app.blueprint(auth_route, url_prefix="/auth")
app.blueprint(user_route, url_prefix="/user")
app.config.from_envvar('TWEETBOX_CONFIG')
app.run(host='127.0.0.1', port=8000, debug=True)
My auth blueprint:
import jwt
from sanic import Blueprint
from sanic.response import json, redirect
from domain.user import User
from repository.user_repository import UserRepository
...
auth_route = Blueprint('authentication')
mongo_url = ?????
user_repository = UserRepository(mongo_url)
...
#auth_route.route('/signin')
async def redirect_user(request):
...
The Sanic way...
Inside a view method, you can access the app instance from the request object. And, therefore access your configuration.
#auth_route.route('/signin')
async def redirect_user(request):
configuration = request.app.config
2021-10-10 Update
There are two newer ways to get to the configuration values (or, perhaps more accuratlely getting the application instance from which you can get the configuration). The first version might be more on point to answering the question of how to get to the config from the blueprint. However, the second option is probably the preferred method since it is precisely intended for this kind of use.
Alternative #1
Blueprints have access to the Sanic applications they are attached to beginning with v21.3.
Therefore, if you have a blueprint object, you can trace that back to the application instance, and therefore also the config value.
app = Sanic("MyApp")
bp = Blueprint("MyBlueprint")
app.blueprint(bp)
assert bp.apps[0] is app
The Blueprint.apps property is a set because it is possible to attach a single blueprint to multiple applications.
Alternative #2
Sanic has a built-in method for retrieving an application instance from the global scope beginning in v20.12. This means that once an application has been instantiated, you can retrieve it using: Sanic.get_app().
app = Sanic("MyApp")
assert Sanic.get_app() is app
This method will only work if there is a single Sanic instance available. If you have multiple application instances, you will need to use the optional name argument:
app1 = Sanic("MyApp")
app2 = Sanic("MyOtherApp")
assert Sanic.get_app("MyApp") is app1
I would suggest a slightly different approach, based on the 12 Factor App (very interesting read which, among others, provides a nice guideline on how to protect and isolate your sensitive info).
The general idea is to place your sensitive and configuration variables in a file that is going to be gitignored and therefore will only be available locally.
I will try to present the method I tend to use in order to be as close as possible to the 12 Factor guidelines:
Create a .env file with your project variables in it:
MONGO_URL=http://no_peeking_this_is_secret:port/
SENSITIVE_PASSWORD=for_your_eyes_only
CONFIG_OPTION_1=config_this
DEBUG=True
...
(Important) Add .env and .env.* on your .gitignore file, thus protecting your sensitive info from been uploaded to GitHub.
Create an env.example (be careful not to name it with a . in the beginning, because it will get ignored).
In that file, you can put an example of the expected configuration in order to be reproducible by simply copy, paste, rename to .env.
In a file named settings.py, use decouple.config to read your config file into variables:
from decouple import config
MONGO_URL = config('MONGO_URL')
CONFIG_OPTION_1 = config('CONFIG_OPTION_1', default='')
DEBUG = config('DEBUG', cast=bool, default=True)
...
Now you can use these variables wherever is necessary for your implementation:
myblueprint.py:
import settings
...
auth_route = Blueprint('authentication')
mongo_url = settings.MONGO_URL
user_repository = UserRepository(mongo_url)
...
As a finisher, I would like to point out that this method is framework (and even language) agnostic so you can use it on Sanic as well as Flask and everywhere you need it!
I think you can create a config.py to save your configuration, just like
config.py
config = {
'MONGO_URL':'127.0.0.1:27017'
}
and use it in app.py
from config import config
mongo_url = config['MONGO_URL']
There is a variable named current_app in Flask. You can use current_app.config["MONGO_URL"].
But I am not familiar with Sanic.
I am using Mongoengine(version: 0.9.0 ) with Django(version: 1.8).
This is my settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.dummy'
}
}
MONGO_DBNAME = "mydatabasename"
MONGO_HOSTNAME = "localhost"
connect(MONGO_DBNAME, host=MONGO_HOSTNAME)
I want to have fixtures for the application. I have created initial_data.json in myapp/fixtures/ location.
When I run the command python manage.py dumpdata , I get the following error :
CommandError: Unable to serialize database: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
Questions:
1) Any workaround for this problem ?
2) Is there any other way to load the initial data ?
References at this link
Thank you
Mongoengine itsn a backend(in django terminology). Its has own models (schemas) and DOM (like ORM in docuemnt db's) but it dont have a Django backend adapters.
You can use it. But there is issue while workind with out-of-box Django solution like Tests, Fixtures, etc.
You need to write your own loader, sadenly but true.
I see 2 options here:
You can try to use Django MongoDB Engine
You can write your own loader for mongodb
Ill write my own fixture loader for tests.
I have a json file where mapped all fixture file ill need to load to db.
So a fast example here:
import bson
import os
from django.conf import settings
from mongoengine.connection import get_db
def _get_db(self):
self.db = get_db()
def _load_fixtures(self, clear_before_load=True):
"""
Load to db a fixtures from folder fixtures/{{DB_NAME}}/{{COLLECTION_NAME}} before each test.
In file fixtures.json mapped collection name and file name for it.
"""
fixture_path = lambda file_name: os.path.join(settings.FIXTURES_DIR, self.db.name, file_name)
with open(settings.COLLECTION_FIXTURES_PATH) as file_object:
db_collections = loads(file_object.read())
for collection_name, filename in db_collections.items():
collection = self.db[collection_name]
if clear_before_load:
collection.remove()
path = fixture_path(filename)
if os.path.exists(path) and os.path.isfile(path):
with open(path, 'r') as raw_data:
collection_data = bson.decode_all(raw_data.read())
for document in collection_data:
collection.save(document)
There is no support for fixtures on mongoengine, and I don't think the mongoengine team is continuing the plugin as of version 0.9.0.
What I ended up doing to load initial data for mongoDB is to create a script called startup.py in my project folder.
startup.py:
from {{app}}.models import Sample
def init():
if Sample.objects(name="test").count() == 0: # a flag to prevent initial data repetition
Sample(name="test").save()
Next is to run this script on Django's startup. The entry point of Django project is when DJANGO_SETTINGS_MODULE is first loaded at wsgi.py:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{{project_name}}.settings")
import {{project_name}}.startup as startup
startup.init()
application = get_wsgi_application()
With this setup, when you run python manage.py runserver, the init() on startup.py will run and the data you set will be inserted to the DB.
Hope this helps.
I am new to pyramid.
The Issue is I am not able to figure out how app specific settings (key value pairs) work in pyramid.
This is what I have done after various google searches and other stackoverflow answers:
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
if '__file__' in global_config:
settings.update(
load_sensitive_settings(global_config['__file__'], global_config))
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
# config.add_static_view('static', 'static', cache_max_age=3600)
# config.add_route('home', '/')
config.add_route(
'tags',
'/tags', request_method='POST', accept='application/json', )
config.scan()
return config.make_wsgi_app()
def load_sensitive_settings(configurationPath, defaultByKey):
'Load sensitive settings from hidden configuration file'
# configFolder, configName = os.path.split(configurationPath)
# sensitivePath = os.path.join(configFolder, '.' + configName)
sensitivePath = configurationPath
settings = {}
configParser = ConfigParser.ConfigParser(defaultByKey)
if not configParser.read(sensitivePath):
log.warn('Could not open %s' % sensitivePath)
return settings
settings.update(configParser.items('custom'))
return settings
I have a file where I try to fetch settings like this:
from pyramid.threadlocal import get_current_registry
settings = get_current_registry().settings
value = settings['my_key']
But I always get settings object as None.
This is how I am defining my custom settings in development.ini
[custom]
my_key = ''
This is how I start my server in develpoment
pserve development.ini
I have read that request.settings can give me settings, But that approach is not feasible for me as my key contains the name of a file which is 1.5GBs and it has to be present in memory all the time. It takes around 5 minutes to load that file in server, hence cannot load the file on demand.
Please advice.
Thanks a lot for all the help in advance.
Update:
Thanks to all the answers provided, I finally made it work.
This is how my main function looks like:
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
if '__file__' in global_config:
init_config(global_config['__file__'])
And I made a config file, this is how my config file looks like:
import ConfigParser
settings = dict()
def init_config(filename):
config = ConfigParser.ConfigParser()
config.read(filename)
settings_dict = config.items('custom')
settings.update(settings_dict)
Now wherever I want settings, I just do:
from projectname.config import settings
settings.get('my_key')
And I put my app specific settings (development/production.py) like this
[custom]
my_key = value
Regards
HM
Easiest way is putting your settings to the app main section with dot separated names. Example:
[app:main]
websauna.site_name = Trees
websauna.site_tag_line = Enjoy
websauna.site_url = http://localhost:6543
websauna.site_author = Trees team
Then you can do:
my_settings_value = request.registry.settings.get("websauna.site_name", "Default value)
WSGI pipeline does not bring you settings from other sections and you need to reparse the INI file with ConfigParser if you want to access the other sections (as far as I know).
If you need to load a lot of data during development time just store a filename in settings and load the file when you need to access the data, so that you don't slow the web server startup.
Here is my working solution:
config.ini
[APP.CONFIG]
url = http://....
[SMTP.CONFIG]
smtp.server = ...
smtp.port = 25
smtp.login = ...
smtp.password = ...
smtp.from = ...
[DB.CONFIG]
db.database=...
db.host=...
db.port=..
db.user=...
db.password=...
config.py
import configparser
config = configparser.ConfigParser()
config._interpolation = configparser.ExtendedInterpolation()
config.read(encoding='utf-8', filenames=['path to file/config.ini'])
smtp = config['SMTP.CONFIG']
db = config['DB.CONFIG']
mail = config['APP.CONFIG']
And how i use it in APP
from config import db
host = db['db.host']
If, like me, you are using PasteDeploy with Pyramid, the Pyramid docs here explain how you can use a [DEFAULT] section in your .ini configuration file to hold your custom parameters.
You might also benefit from reading the documentation on .ini files, since it gives some snippets which make it all much clearer.
I would like certain parts of my code to not run while it is being run locally.
This is because, I am having trouble installing certain dependencies locally, for the code to run.
Specifically, memcache doesn't work for me locally.
#app.route('/some_url_route/')
#cache.cached(timeout=2000) #ignore this locally
def show_a_page():
How would the app somehow ignore the cache section of the code above, when running locally?
In my code I follow a Django-esq model and have a main settings.py file I keep all my settings in.
In that file putting DEBUG = True for your local environment (and False for production) I then use:
from settings import DEBUG
if DEBUG:
# Do this as it's development
else:
# Do this as it's production
So in your cache decorator include a similar line that only checks memcached if DEBUG=False
You can then load all these settings into your Flask setup as detailed in the configuration documentation.
If you're using Flask-Cache, then just edit the settings:
if app.debug:
app.settings.CACHE_TYPE = 'null' # the cache that doesn't cache
cache = Cache(app)
...
A better approach would be to have separate settings for production and development. I use a class-based approach:
class BaseSettings(object):
...
class DevelopmentSettings(BaseSettings):
DEBUG = True
CACHE_TYPE = 'null'
...
class ProductionSettings(BaseSettings):
CACHE_TYPE = 'memcached'
...
And then import the appropriate object when you setup your app (config.py is the name of the file which contains the settings):
app.config.from_object('config.DevelopmentConfig')