We Keep Coding

Config parameter not reading from other locations

The parameter is not read from files other than the run.py . run.py is the Flask startup script
from run.py :
from flask import Flask
app = Flask(__name__)
print(app.config["PARAMETER"])
From any other python file in project following error is returned :
KeyError: 'PARAMETER'
How to read configuration file from other locations within project other than startup script file ?
Update :
Reading config file :
app.config.from_pyfile(os.path.join(".", "myconfig.conf"))
Contents of myconfig.conf :
PARAMETER = "test"

I think you just simply missed loading parameters.
If you do inline loadign of parameters, you would specifiy it as:
from flask import Flask
app = Flask(__name__)
app.config['PARAMETER_1'] = 'test'
assert app.config["PARAMETER_1"] == 'test'
The code above works, now how can one load similar data from file?
# prepare file
from pathlib import Path
Path("myconfig.conf").write_text("PARAMETER_2 = 'test2'\n")
# load it
app.config.from_pyfile("myconfig.conf")
# test it
assert app.config["PARAMETER_2"] == 'test2'

You can't print the PARAMETER key after the creation of the app because there is no any key into its configuration object which its name is PARAMETER. In order to do it, first create the app, import a simple object which holds the configuration and then use app.config.from_object(Config)
#config.py
class Config(object):
PARAMETER = 'my-parameter-value'
#you can add another keys
SECRET_KEY = os.environ.get('SECRET_KEY') or 'you-will-never-guess'
#__init__.py
from config import Config
app = Flask(__name__);
app.config.from_object(Config)

Fixtures for Mongoengine with Django not working

I am using Mongoengine(version: 0.9.0 ) with Django(version: 1.8).
This is my settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.dummy'
}
}
MONGO_DBNAME = "mydatabasename"
MONGO_HOSTNAME = "localhost"
connect(MONGO_DBNAME, host=MONGO_HOSTNAME)
I want to have fixtures for the application. I have created initial_data.json in myapp/fixtures/ location.
When I run the command python manage.py dumpdata , I get the following error :
CommandError: Unable to serialize database: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
Questions:
1) Any workaround for this problem ?
2) Is there any other way to load the initial data ?
References at this link
Thank you

Mongoengine itsn a backend(in django terminology). Its has own models (schemas) and DOM (like ORM in docuemnt db's) but it dont have a Django backend adapters.
You can use it. But there is issue while workind with out-of-box Django solution like Tests, Fixtures, etc.
You need to write your own loader, sadenly but true.
I see 2 options here:
You can try to use Django MongoDB Engine
You can write your own loader for mongodb
Ill write my own fixture loader for tests.
I have a json file where mapped all fixture file ill need to load to db.
So a fast example here:
import bson
import os
from django.conf import settings
from mongoengine.connection import get_db
def _get_db(self):
self.db = get_db()
def _load_fixtures(self, clear_before_load=True):
"""
Load to db a fixtures from folder fixtures/{{DB_NAME}}/{{COLLECTION_NAME}} before each test.
In file fixtures.json mapped collection name and file name for it.
"""
fixture_path = lambda file_name: os.path.join(settings.FIXTURES_DIR, self.db.name, file_name)
with open(settings.COLLECTION_FIXTURES_PATH) as file_object:
db_collections = loads(file_object.read())
for collection_name, filename in db_collections.items():
collection = self.db[collection_name]
if clear_before_load:
collection.remove()
path = fixture_path(filename)
if os.path.exists(path) and os.path.isfile(path):
with open(path, 'r') as raw_data:
collection_data = bson.decode_all(raw_data.read())
for document in collection_data:
collection.save(document)

There is no support for fixtures on mongoengine, and I don't think the mongoengine team is continuing the plugin as of version 0.9.0.
What I ended up doing to load initial data for mongoDB is to create a script called startup.py in my project folder.
startup.py:
from {{app}}.models import Sample
def init():
if Sample.objects(name="test").count() == 0: # a flag to prevent initial data repetition
Sample(name="test").save()
Next is to run this script on Django's startup. The entry point of Django project is when DJANGO_SETTINGS_MODULE is first loaded at wsgi.py:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{{project_name}}.settings")
import {{project_name}}.startup as startup
startup.init()
application = get_wsgi_application()
With this setup, when you run python manage.py runserver, the init() on startup.py will run and the data you set will be inserted to the DB.
Hope this helps.

How to add your app specific settings in pyramid rest framework?

I am new to pyramid.
The Issue is I am not able to figure out how app specific settings (key value pairs) work in pyramid.
This is what I have done after various google searches and other stackoverflow answers:
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
if '__file__' in global_config:
settings.update(
load_sensitive_settings(global_config['__file__'], global_config))
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
# config.add_static_view('static', 'static', cache_max_age=3600)
# config.add_route('home', '/')
config.add_route(
'tags',
'/tags', request_method='POST', accept='application/json', )
config.scan()
return config.make_wsgi_app()
def load_sensitive_settings(configurationPath, defaultByKey):
'Load sensitive settings from hidden configuration file'
# configFolder, configName = os.path.split(configurationPath)
# sensitivePath = os.path.join(configFolder, '.' + configName)
sensitivePath = configurationPath
settings = {}
configParser = ConfigParser.ConfigParser(defaultByKey)
if not configParser.read(sensitivePath):
log.warn('Could not open %s' % sensitivePath)
return settings
settings.update(configParser.items('custom'))
return settings
I have a file where I try to fetch settings like this:
from pyramid.threadlocal import get_current_registry
settings = get_current_registry().settings
value = settings['my_key']
But I always get settings object as None.
This is how I am defining my custom settings in development.ini
[custom]
my_key = ''
This is how I start my server in develpoment
pserve development.ini
I have read that request.settings can give me settings, But that approach is not feasible for me as my key contains the name of a file which is 1.5GBs and it has to be present in memory all the time. It takes around 5 minutes to load that file in server, hence cannot load the file on demand.
Please advice.
Thanks a lot for all the help in advance.
Update:
Thanks to all the answers provided, I finally made it work.
This is how my main function looks like:
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
if '__file__' in global_config:
init_config(global_config['__file__'])
And I made a config file, this is how my config file looks like:
import ConfigParser
settings = dict()
def init_config(filename):
config = ConfigParser.ConfigParser()
config.read(filename)
settings_dict = config.items('custom')
settings.update(settings_dict)
Now wherever I want settings, I just do:
from projectname.config import settings
settings.get('my_key')
And I put my app specific settings (development/production.py) like this
[custom]
my_key = value
Regards
HM

Easiest way is putting your settings to the app main section with dot separated names. Example:
[app:main]
websauna.site_name = Trees
websauna.site_tag_line = Enjoy
websauna.site_url = http://localhost:6543
websauna.site_author = Trees team
Then you can do:
my_settings_value = request.registry.settings.get("websauna.site_name", "Default value)
WSGI pipeline does not bring you settings from other sections and you need to reparse the INI file with ConfigParser if you want to access the other sections (as far as I know).
If you need to load a lot of data during development time just store a filename in settings and load the file when you need to access the data, so that you don't slow the web server startup.

Here is my working solution:
config.ini
[APP.CONFIG]
url = http://....
[SMTP.CONFIG]
smtp.server = ...
smtp.port = 25
smtp.login = ...
smtp.password = ...
smtp.from = ...
[DB.CONFIG]
db.database=...
db.host=...
db.port=..
db.user=...
db.password=...
config.py
import configparser
config = configparser.ConfigParser()
config._interpolation = configparser.ExtendedInterpolation()
config.read(encoding='utf-8', filenames=['path to file/config.ini'])
smtp = config['SMTP.CONFIG']
db = config['DB.CONFIG']
mail = config['APP.CONFIG']
And how i use it in APP
from config import db
host = db['db.host']

If, like me, you are using PasteDeploy with Pyramid, the Pyramid docs here explain how you can use a [DEFAULT] section in your .ini configuration file to hold your custom parameters.
You might also benefit from reading the documentation on .ini files, since it gives some snippets which make it all much clearer.

Python/Django: Get AttributeError when rendering html to pdf using xhtml2pdf

Here you can see the errors I get.
I am trying to render a pdf from a html template using Django.
This worked pretty well locally when I was fiddling about with it.
Managed to get a .pdf file from it.
Now I'm stuck with this error claiming that the uri has no attribute encode.
Has anyone had a similar error while using this python package? If so, how did you manage to fix it?
from __future__ import print_function
import easy_pdf
from deliveries.models import Delivery
from django.conf import settings
from easy_pdf.views import PDFTemplateView
import os
import sys
from django.core.wsgi import get_wsgi_application
from scripts import messages as msg
basename = os.path.splitext(os.path.basename(__file__))[0]
def run(*args, **kwargs):
if len(args) < 1 or args[0] is None:
print(msg.GeneralMessages.ERROR_DELIVERY_MISSING, file=sys.stderr)
sys.exit(1)
if not str(args[0]).isdigit():
logger.error(msg.GeneralMessages.ERROR_DELIVERY_INVALID)
sys.exit(1)
delivery = Delivery.get(args[0])
application = get_wsgi_application()
context = ''
view = ''
encoding = 'utf-8'
print(context)
view = easy_pdf.rendering.render_to_pdf('report.html', context)
if view:
f = open('report.pdf', 'wb')
f.write(view)
f.close()
print ('Report has been exported as .pdf')
if not settings.configured:
settings.configure(
DEBUG=True,
TIMEZONE="UTC",
INSTALLED_APPS=["easy_pdf", "django.contrib.staticfiles"],
TEMPLATE_DIRS=["../../deliveries/templates"],
ROOT_URLCONF=basename,
WSGI_APPLICATION="{}.application".format(basename),
STATIC_URL='/deliveries/static'
)

HTML file response in Pyramid

I am trying to adapt the to-do list app in the Pyramid tutorial here to create a Python hangman game. My directory structure is as follows:
/tasks
tasks.py
/static
custom.css
/templates
main.html
I want to have a single view main.html (i.e., just the game title, the current picture with blanks, score, etc., and a button for each letter to guess) that gets updated every time the user chooses a letter.
I figured I could do this by dynamically creating a new HTML display with each button press, similar to the jpg serving method shown here. But upon running python tasks.py and opening http://localhost:8080/# in the browser, it gives a server error and: "OSError: [Errno 2] No such file or directory: '/templates/main.html' in the terminal.
Here's my tasks.py:
import os
import logging
from pyramid.config import Configurator
from pyramid.events import NewRequest
from pyramid.events import ApplicationCreated
from pyramid.exceptions import NotFound
from pyramid.httpexceptions import HTTPFound
from pyramid.response import FileResponse
from pyramid.view import view_config
from wsgiref.simple_server import make_server
logging.basicConfig()
log = logging.getLogger(__file__)
here = os.path.dirname(os.path.abspath(__file__))
##view_config(route_name='home', renderer='main.mako')
#def main_page(request):
# print request #just checking
# return {}
#view_config(route_name='home')
def main_page(request):
response = FileResponse(os.path.join(here,'/templates/main.html'), \
request=request, content_type='text/html')
return response
if __name__ == '__main__':
settings = {}
settings['reload_all'] = True
settings['debug_all'] = True
settings['mako.directories'] = os.path.join(here, 'templates')
config = Configurator(settings=settings)
config.include('pyramid_mako')
config.add_route('home', '/')
config.add_static_view('static', os.path.join(here, 'static'))
config.scan()
app = config.make_wsgi_app()
server = make_server('0.0.0.0', 8080, app)
print 'server made'
server.serve_forever()
Considering the here declaration, I don't see how the file can't be found. As you can see from the commented out view_config, I tried using my initial HTML file (renamed main.mako) as the renderer at first, which worked fine and displayed the main page nicely.
But I don't think I can use that to update dynamically as desired. What am I missing here? Maybe I need to add something else to the settings dictionary?

You getting this error because you are using absolute path /templates/main.html.
Simple example how os.path.join works:
>>> os.path.join('/some/path/to/project', '/templates/main.html')
'/templates/main.html'
>>> os.path.join('/some/path/to/project', 'templates/main.html')
'/some/path/to/project/templates/main.html'
So try change os.path.join(here,'/templates/main.html') to os.path.join(here,'templates/main.html').