The parameter is not read from files other than the run.py . run.py is the Flask startup script
from run.py :
from flask import Flask
app = Flask(__name__)
print(app.config["PARAMETER"])
From any other python file in project following error is returned :
KeyError: 'PARAMETER'
How to read configuration file from other locations within project other than startup script file ?
Update :
Reading config file :
app.config.from_pyfile(os.path.join(".", "myconfig.conf"))
Contents of myconfig.conf :
PARAMETER = "test"
I think you just simply missed loading parameters.
If you do inline loadign of parameters, you would specifiy it as:
from flask import Flask
app = Flask(__name__)
app.config['PARAMETER_1'] = 'test'
assert app.config["PARAMETER_1"] == 'test'
The code above works, now how can one load similar data from file?
# prepare file
from pathlib import Path
Path("myconfig.conf").write_text("PARAMETER_2 = 'test2'\n")
# load it
app.config.from_pyfile("myconfig.conf")
# test it
assert app.config["PARAMETER_2"] == 'test2'
You can't print the PARAMETER key after the creation of the app because there is no any key into its configuration object which its name is PARAMETER. In order to do it, first create the app, import a simple object which holds the configuration and then use app.config.from_object(Config)
#config.py
class Config(object):
PARAMETER = 'my-parameter-value'
#you can add another keys
SECRET_KEY = os.environ.get('SECRET_KEY') or 'you-will-never-guess'
#__init__.py
from config import Config
app = Flask(__name__);
app.config.from_object(Config)
Related
Based on the Configuration Handling Documents for Flask the section of Configuring from Files mentions a possibility to configure the App using files however it provides no example or mention of files that are not Python Files.
Is it possible to configure apps via files like config.yml or config.toml?
My Current flask app has configurations for two distinct databases and since I am using flask-restplus there are additional configurations for Swagger documentations.
Snippet:
from flask import Flask
app = Flask(__name__)
def configure_app(flask_app):
# MongoDB Setting
flask_app.config['MONGO_URI'] = 'mongodb://user:password#mongo_db_endpoint:37018/myDB?authSource=admin'
flask_app.config['MONGO_DBNAME'] = 'myDB'
# InfluxDB Setting
flask_app.config['INFLUXDB_HOST'] = 'my_influxdb_endpoint'
flask_app.config['INFLUXDB_PORT'] = 8086
flask_app.config['INFLUXDB_USER'] = 'influx_user'
flask_app.config['INFLUXDB_PASSWORD'] = 'influx_password'
flask_app.config['INFLUXDB_SSL'] = True
flask_app.config['INFLUXDB_VERIFY_SSL'] = False
flask_app.config['INFLUXDB_DATABASE'] = 'IoTData'
# Flask-Restplus Swagger Configuration
flask_app.config['RESTPLUS_SWAGGER_UI_DOC_EXPANSION'] = 'list'
flask_app.config['RESTPLUS_VALIDATE'] = True
flask_app.config['RESTPLUS_MASK_SWAGGER'] = False
flask_app.config['ERROR_404_HELP'] = False
def main():
configure_app(app)
if __name__ == "__main__":
main()
I would like to avoid setting large number of Environment Variables and wish to configure them using a config.toml file?
How is this achieved in flask?
You can use the .cfg files and from_envvar to achieve this. Create config file with all your environment variables.
my_config.cfg
MONGO_URI=mongodb://user:password#mongo_db_endpoint:37018
..
..
ERROR_404_HELP=False
Then set the env var APP_ENVS=my_config.cfg. Now all you need to do is use from_envvars given by Flask.
def configure_app(flask_app):
flask_app.config.from_envvar('APP_ENVS')
# configure any other things
# register blue prints if you have any
Quoting from documentation:
Configuring from Data Files
It is also possible to load configuration from a file in a format of
your choice using from_file(). For example to load from a TOML file:
import toml
app.config.from_file("config.toml", load=toml.load)
Or from a JSON file:
import json
app.config.from_file("config.json", load=json.load)
EDIT: The above feature is new for v2.0.
Link to the documentation reference:
Class Flask.config, method from_file(filename, load, silent=False)
in /instance/app.cfg I've configured :
test=test
In my flask file app.py :
with app.open_instance_resource('app.cfg') as f:
config = f.read()
print('config' , type(config))
Which prints config <class 'bytes'>
Reading the flask doc it does not detail how to read values from configuration files, how is this achieved ?
can config be read a dictionary instead of bytes ?
Update :
app.py :
# Shamelessly copied from http://flask.pocoo.org/docs/quickstart/
from flask import Flask
app = Flask(__name__)
import os
ac = app.config.from_pyfile(os.path.join('.', 'conf/api.conf'), silent=True)
logging_configuration = app.config.get('LOGGING')
if ac:
print(logging.config.dictConfig(ac))
#app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run()
api.conf :
myvar=tester
returns error :
/./conf/api.conf", line 1, in <module>
myvar=tester
NameError: name 'tester' is not defined
Update 2 :
app.py :
from flask import Flask
app = Flask(__name__)
import os
from logging.config import dictConfig
app.config.from_pyfile(os.path.join('.', 'conf/api.conf'), silent=True)
logging_configuration = app.config.get('LOGGING')
if logging_configuration:
print(dictConfig(logging_configuration))
#app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run()
api.conf :
LOGGING="tester"
returns error :
ValueError: dictionary update sequence element #0 has length 1; 2 is required
Reading the flask doc it does not detail how to read values from configuration files, how is this achieved ?
You can read about it in flask's doc here (title "configuring-from-files")
open_instance_resource is only a shortcut to make deal with files which are located in "instance folder" (a special place where you can store deploy specific files). It's not supposed to be a way to get your config as a dict.
Flask stores his config variable(app.config) as a dict object. You can update it via a bunch of methods: from_envvar, from_pyfile, from_object etc. Look at the source code
One of the typical ways how people read config files in flask-based apps:
app = Flask('your_app')
...
app.config.from_pyfile(os.path.join(basedir, 'conf/api.conf'), silent=True)
...
After that, you can use your dict-like config object as you want:
...
logging_configuration = app.config.get('LOGGING')
if logging_configuration:
logging.config.dictConfig(logging_configuration)
...
from flask import Flask
app = Flask(__name__)
import os
app.config.from_pyfile(os.path.join('.', 'conf/api.conf'), silent=True)
#app.route('/')
def hello_world():
return 'Hello World! {}'.format(app.config.get('LOGGING'))
if __name__ == '__main__':
app.run()
If you do app.config.from_pyfile('app.cfg') you can obtain your config as a dictionary by dict(app.config).
However, this dictionary will contain the whole configuration for your app not only those variables which were set by the configuration file.
Has anyone tried this snippet flask config based static folder code cnippet?
The code:
import flask
class MyFlask(flask.Flask):
#property
def static_folder(self):
if self.config.get('STATIC_FOLDER') is not None:
return os.path.join(self.root_path,
self.config.get('STATIC_FOLDER'))
#static_folder.setter
def static_folder(self, value):
self.config.get('STATIC_FOLDER') = value
# Now these are equivalent:
app = Flask(__name__, static_folder='foo')
app = MyFlask(__name__)
app.config['STATIC_FOLDER'] = 'foo'
In my case in complains about this line:
self.config.get('STATIC_FOLDER') = value
The error message: Can't assign to function call
Does anyone how to set the static_folder from the config.py file in Flask?
Okay, I assume you want to use a custom path to the static folder for whatever reason. I wanted to do the same for the sake of better app modularity.
Here's my app folder structure:
instance/
core/
|_templates/
|_static/
|_views.py
run.py
config.py
As you can see, my static folder is inside the core folder.
In run.py, you can do the following:
app = Flask(__name__, static_url_path=None)
if __name__ == '__main__':
app.config.from_object('config')
# config file has STATIC_FOLDER='/core/static'
app.static_url_path=app.config.get('STATIC_FOLDER')
# set the absolute path to the static folder
app.static_folder=app.root_path + app.static_url_path
print(app.static_url_path)
print(app.static_folder)
app.run(
host=app.config.get('HOST'),
port=app.config.get('PORT'),
threaded=True
)
This is what I did, and it works perfectly fine. I'm using flask 0.12.
I don't know anything about that snippet, but
some_function(...) = some_value
is never valid Python (Python doesn't have l-values). It looks like config has a dict-like interface, so the offending line should probably just be
self.config['STATIC_FOLDER'] = value
Probably a copy-and-paste error from the getter definition above the setter.
app = Flask(__name__, static_url_path="/STATIC_FOLDER", static_folder='STATIC_FOLDER')
Yes, In one of my projects I am using/setting a custom path for STATIC_FOLDER. You can set the path to STATIC_FOLDER in config.py like below:
STATIC_PATH = '<project-name>/<path-to-static-folder>/'
ex:
STATIC_PATH = 'myApp/static/'
If you can write your project structure then I can answer it as per your requirements.
FYI if you want your directory to be outside of the server directory the best solutions I found so far are either to make a copy of your directory into the server directory before startup in your main(), or to create a symlink.
I am using Mongoengine(version: 0.9.0 ) with Django(version: 1.8).
This is my settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.dummy'
}
}
MONGO_DBNAME = "mydatabasename"
MONGO_HOSTNAME = "localhost"
connect(MONGO_DBNAME, host=MONGO_HOSTNAME)
I want to have fixtures for the application. I have created initial_data.json in myapp/fixtures/ location.
When I run the command python manage.py dumpdata , I get the following error :
CommandError: Unable to serialize database: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
Questions:
1) Any workaround for this problem ?
2) Is there any other way to load the initial data ?
References at this link
Thank you
Mongoengine itsn a backend(in django terminology). Its has own models (schemas) and DOM (like ORM in docuemnt db's) but it dont have a Django backend adapters.
You can use it. But there is issue while workind with out-of-box Django solution like Tests, Fixtures, etc.
You need to write your own loader, sadenly but true.
I see 2 options here:
You can try to use Django MongoDB Engine
You can write your own loader for mongodb
Ill write my own fixture loader for tests.
I have a json file where mapped all fixture file ill need to load to db.
So a fast example here:
import bson
import os
from django.conf import settings
from mongoengine.connection import get_db
def _get_db(self):
self.db = get_db()
def _load_fixtures(self, clear_before_load=True):
"""
Load to db a fixtures from folder fixtures/{{DB_NAME}}/{{COLLECTION_NAME}} before each test.
In file fixtures.json mapped collection name and file name for it.
"""
fixture_path = lambda file_name: os.path.join(settings.FIXTURES_DIR, self.db.name, file_name)
with open(settings.COLLECTION_FIXTURES_PATH) as file_object:
db_collections = loads(file_object.read())
for collection_name, filename in db_collections.items():
collection = self.db[collection_name]
if clear_before_load:
collection.remove()
path = fixture_path(filename)
if os.path.exists(path) and os.path.isfile(path):
with open(path, 'r') as raw_data:
collection_data = bson.decode_all(raw_data.read())
for document in collection_data:
collection.save(document)
There is no support for fixtures on mongoengine, and I don't think the mongoengine team is continuing the plugin as of version 0.9.0.
What I ended up doing to load initial data for mongoDB is to create a script called startup.py in my project folder.
startup.py:
from {{app}}.models import Sample
def init():
if Sample.objects(name="test").count() == 0: # a flag to prevent initial data repetition
Sample(name="test").save()
Next is to run this script on Django's startup. The entry point of Django project is when DJANGO_SETTINGS_MODULE is first loaded at wsgi.py:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{{project_name}}.settings")
import {{project_name}}.startup as startup
startup.init()
application = get_wsgi_application()
With this setup, when you run python manage.py runserver, the init() on startup.py will run and the data you set will be inserted to the DB.
Hope this helps.
I am new to pyramid.
The Issue is I am not able to figure out how app specific settings (key value pairs) work in pyramid.
This is what I have done after various google searches and other stackoverflow answers:
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
if '__file__' in global_config:
settings.update(
load_sensitive_settings(global_config['__file__'], global_config))
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
# config.add_static_view('static', 'static', cache_max_age=3600)
# config.add_route('home', '/')
config.add_route(
'tags',
'/tags', request_method='POST', accept='application/json', )
config.scan()
return config.make_wsgi_app()
def load_sensitive_settings(configurationPath, defaultByKey):
'Load sensitive settings from hidden configuration file'
# configFolder, configName = os.path.split(configurationPath)
# sensitivePath = os.path.join(configFolder, '.' + configName)
sensitivePath = configurationPath
settings = {}
configParser = ConfigParser.ConfigParser(defaultByKey)
if not configParser.read(sensitivePath):
log.warn('Could not open %s' % sensitivePath)
return settings
settings.update(configParser.items('custom'))
return settings
I have a file where I try to fetch settings like this:
from pyramid.threadlocal import get_current_registry
settings = get_current_registry().settings
value = settings['my_key']
But I always get settings object as None.
This is how I am defining my custom settings in development.ini
[custom]
my_key = ''
This is how I start my server in develpoment
pserve development.ini
I have read that request.settings can give me settings, But that approach is not feasible for me as my key contains the name of a file which is 1.5GBs and it has to be present in memory all the time. It takes around 5 minutes to load that file in server, hence cannot load the file on demand.
Please advice.
Thanks a lot for all the help in advance.
Update:
Thanks to all the answers provided, I finally made it work.
This is how my main function looks like:
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
if '__file__' in global_config:
init_config(global_config['__file__'])
And I made a config file, this is how my config file looks like:
import ConfigParser
settings = dict()
def init_config(filename):
config = ConfigParser.ConfigParser()
config.read(filename)
settings_dict = config.items('custom')
settings.update(settings_dict)
Now wherever I want settings, I just do:
from projectname.config import settings
settings.get('my_key')
And I put my app specific settings (development/production.py) like this
[custom]
my_key = value
Regards
HM
Easiest way is putting your settings to the app main section with dot separated names. Example:
[app:main]
websauna.site_name = Trees
websauna.site_tag_line = Enjoy
websauna.site_url = http://localhost:6543
websauna.site_author = Trees team
Then you can do:
my_settings_value = request.registry.settings.get("websauna.site_name", "Default value)
WSGI pipeline does not bring you settings from other sections and you need to reparse the INI file with ConfigParser if you want to access the other sections (as far as I know).
If you need to load a lot of data during development time just store a filename in settings and load the file when you need to access the data, so that you don't slow the web server startup.
Here is my working solution:
config.ini
[APP.CONFIG]
url = http://....
[SMTP.CONFIG]
smtp.server = ...
smtp.port = 25
smtp.login = ...
smtp.password = ...
smtp.from = ...
[DB.CONFIG]
db.database=...
db.host=...
db.port=..
db.user=...
db.password=...
config.py
import configparser
config = configparser.ConfigParser()
config._interpolation = configparser.ExtendedInterpolation()
config.read(encoding='utf-8', filenames=['path to file/config.ini'])
smtp = config['SMTP.CONFIG']
db = config['DB.CONFIG']
mail = config['APP.CONFIG']
And how i use it in APP
from config import db
host = db['db.host']
If, like me, you are using PasteDeploy with Pyramid, the Pyramid docs here explain how you can use a [DEFAULT] section in your .ini configuration file to hold your custom parameters.
You might also benefit from reading the documentation on .ini files, since it gives some snippets which make it all much clearer.