We Keep Coding

Avoiding importing application factory into module needing application context

This question is an extension on my previous one here. I was suggested to put more to explain the problem. As the heading says, I am trying to find a way to avoid importing the application factory (create_app function) into a module that needs application context and were "import current_app as app" is not sufficient.
My problem is I have a circular import problem due to this create_app function which I need to pass in order to get the app_context.
In my __ini__.py, I have this:
# application/__init__.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_restful import Api
from application.resources.product import Product, Products
from application.resources.offer import Offer, Offers # HERE IS THE PROBLEM
api = Api()
db = SQLAlchemy()
api.add_resource(Product, "/product/<string:name>") # GET, POST, DELETE, PUT to my local database
api.add_resource(Products, "/products") # GET all products from my local database
api.add_resource(Offer, "/offer/<int:id>") # POST call to the external Offers API microservise
api.add_resource(Offers, "/offers") # GET all offers from my local database
def create_app(config_filename=None):
""" Initialize core application. """
app = Flask(__name__, instance_relative_config=False)
app.config.from_object("config.Config")
db.init_app(app)
api.init_app(app)
with app.app_context():
db.create_all()
return app
The problem is in this line:
from application.resources.offer import Offer, Offers # HERE IS THE PROBLEM
because in that module, I have:
#application/resources/offer.py
from flask_restful import Resource
from application.models.offer import OfferModel # IMPORTING OFFER MODEL
which in turn imports application/models/offer.py where I have the critical part:
#application/models/offer.py
import requests
# from flask import current_app as app
from application import create_app # THIS CAUSES THE CIRCULAR IMPORT ERROR
from sqlalchemy.exc import OperationalError
app = create_app() # I NEED TO CREATE THE APP IN ORDER TO GET THE APP CONTEXT BECASE IN THE CLASS I HAVE SOME FUNCTIONS THAT NEED IT
class OfferModel(db.Model):
""" Data model for offers. """
# some code to instantiate the class... + other methods..
# THIS IS ONE OF THE METHODS THAT NEED APP_CONTEXT OR ELSE IT WILL ERROR OUT
#classmethod
def update_offer_price(cls):
""" Call offers api to get new prices. This function will run in a separated thread in a scheduler. """
with app.app_context():
headers = {"Bearer": app.config["MS_API_ACCESS_TOKEN"]}
for offer_id in OfferModel.offer_ids:
offers_url = app.config["MS_API_OFFERS_BASE_URL"] + "/products/" + str(offer_id) + "/offers"
res = requests.get(offers_url, headers=headers).json()
for offer in res:
try:
OfferModel.query.filter_by(offer_id=offer["id"]).update(dict(price=offer["price"]))
db.session.commit()
except OperationalError:
print("Database does not exists.")
db.session.rollback()
I have tried to use from flask import current_app as app to get the context, it did not work. I don't know why it was not sufficient to pass current_app as app and get the context because it now forces me to pass the create_app application factory which causes the circular import problem.

Your update_offer_price method needs database interaction and an access to the configuration. It gets them from the application context but it works only if your Flask application is initialized. This method is run in a separate thread so you create the second instance of Flask application in this thread.
Alternative way is getting standalone database interaction and configuration access outside the application context.
Configuration
Configuration does not seem a problem as your application gets it from another module:
app.config.from_object("config.Config")
So you can directly import this object to your offer.py:
from config import Config
headers = {"Bearer": Config.MS_API_ACCESS_TOKEN}
Database access
To get standalone database access you need to define your models via SQLAlchemy instead of flask_sqlalchemy. It was already described in this answer but I post here the essentials. For your case it may look like this. Your base.py module:
from sqlalchemy import MetaData
from sqlalchemy.ext.declarative import declarative_base
metadata = MetaData()
Base = declarative_base(metadata=metadata)
And offer.py module:
import sqlalchemy as sa
from .base import Base
class OfferModel(Base):
id = sa.Column(sa.Integer, primary_key=True)
# Another declarations
The produced metadata object is used to initialize your flask_sqlalchemy object:
from flask_sqlalchemy import SQLAlchemy
from application.models.base import metadata
db = SQLAlchemy(metadata=metadata)
Your models can be queried outside the application context but you need to manually create database engine and sessions. For example:
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from config import Config
from application.models.offer import Offer
engine = create_engine(Config.YOUR_DATABASE_URL)
# It is recommended to create a single engine
# and use it afterwards to bind database sessions to.
# Perhaps `application.models.base` module
# is better to be used for this declaration.
def your_database_interaction():
session = Session(engine)
offers = session.query(Offer).all()
for offer in offers:
# Some update here
session.commit()
session.close()
Note that with this approach you can't use your models classes for queriing, I mean:
OfferModel.query.all() # Does not work
db.session.query(OfferModel).all() # Works

ok so this is how I solved it. I made a new file endpoints.py where I put all my Api resources
# application/endpoints.py
from application import api
from application.resources.product import Product, Products
from application.resources.offer import Offer, Offers
api.add_resource(Product, "/product/<string:name>") # GET, POST, DELETE, PUT - calls to local database
api.add_resource(Products, "/products") # GET all products from local database.
api.add_resource(Offer, "/offer/<int:id>") # POST call to the Offers API microservice.
api.add_resource(Offers, "/offers") # GET all offers from local database
Then in init.py I import it at the very bottom.
# aplication/__init__.py
from flask import Flask
from flask_restful import Api
from db import db
api = Api()
def create_app():
app = Flask(__name__, instance_relative_config=False)
app.config.from_object("config.Config")
db.init_app(app)
api.init_app(app)
with app.app_context():
from application import routes
db.create_all()
return app
from application import endpoints # importing here to avoid circular imports
It is not very pretty but it works.

How to access flask config variables outside application context according to my project structure?

spinngod.py - flask app starter code
from app import create_app
import sys
run_profile = str(sys.argv[1]) if len(sys.argv) >= 2 else 'development'
app = create_app(run_profile)
print("App Root Path:" + app.root_path)
if __name__ == '__main__':
print sys.path
app.run(debug=True, host='0.0.0.0')
app/init.py - creates flask app
def create_app(profile_name):
print "currently active profile:" + profile_name
app = Flask(__name__)
############# configurations ####################
app.config.from_object(config[profile_name])
configure_app(app)
configure_app_logger(app)
#################### blueprint registration and rest_plus namespace additions ###############
from api_1_0 import api as api_1_0_blueprint
from api_1_0.restplus import api_restplus
# ************************************************** #
api_restplus.init_app(api_1_0_blueprint)
api_restplus.add_namespace(application_namespace)
api_restplus.add_namespace(pipeline_template_namespace)
api_restplus.add_namespace(loadbalancer_namespace)
api_restplus.add_namespace(servergroup_namespace)
api_restplus.add_namespace(task_namespace)
# ************************************************** #
app.register_blueprint(api_1_0_blueprint)
##############################################################
return app
I want to access flask config variables defined in config.py in some other files which are outside application context. The app configuration depends on which profile it is started with (dev,stage or production) which is being passed from command line as an arg.
The only way that I can think of accessing config variables outside app context is to set profile (dev,stage or prod) as an environment variable and
then import directly from config file.
The second way that I tried was to move creation of flask app in app/init.py outside method.
This is how I am trying to access config variables in another class.
import requests
class Client(object):
def __init__(self):
from app import app
print "fjaijflkajsf" + app.config['SPINNAKER_BASE_URL']
pass
Is there a way better of doing this in flask ?

From the docs:
Rather than passing the application around to each function, the current_app and g proxies are accessed instead.
The Flask application object has attributes, such as config, that are useful to access within views and CLI commands. However, importing the app instance within the modules in your project is prone to circular import issues.
Flask solves this issue with the application context. Rather than referring to an app directly, you use the the current_app proxy, which points to the application handling the current activity.
You import current_app like this:
from flask import current_app
and then access the config or other attributes like this:
config = current_app.config
Example:
src/application.py (where config is set in the context)
create_app():
app = Flask('app')
app.config.from_object(some_class)
return app
src/module/another_module.py
from flask import current_app
def function_that_requires_config():
config = current_app.config
Alternative:
src/application.py (where config is set in the context)
APP = create_app(os.environ.get('FLASK_ENV'))
src/module/another_module.py
from src.application import APP
def function_that_requires_config():
config_value = APP.config.get(config_key, default_value)

Not sure if it is good to put it here as it may not respond to the question directly, but here is the cleanest way i've figured to use config values outside of requests, without having to pass config as a param.
The solution is actually pretty simple, juste consider the part of your code as a flask_extension.
my exemple will be the use of external api, with ROOT_URL in the config file, and i don't want to make api call from within my routes, so the api is in its own module.
in my create_app fuction:
from flask import Flask
from .api import api
from .configmodule import Config
from .model import db
def create_app(environment):
app = Flask(__name__)
app.config.from_object(Config.get_config(environment))
db.init_app(app)
api.init_app(app) # here i use api.init_app the same way i do for sqlalchemy
and in api/init.py
class Api:
def init_app(self, app):
self.config = app.config
api = Api()
and in any files in my api modude i can now write
from . import api
def foo():
print(api.config.get("API_ROOT_URL"))
this can even be improved if you feel the need to access some other global app vars from your module.

Can't get Pyramid to recognize Mongo

I am trying to get the Pyramid Web framework to handle a request using Mongo but I am a relative newbie to both. I cannot get my view to recognize a database attached to a request.
In development.ini:
###
# configure mongodb
###
mongo_uri = mongodb://localhost:27017/nomad
The __init__.py imports and main function:
# imports for Mongodb
from urllib.parse import urlparse
from gridfs import GridFS
from pymongo import MongoClient
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
config = Configurator(settings=settings)
init_includes(config)
init_routing(config)
db_url = urlparse(settings['mongo_uri'])
config.registry.db = MongoClient(
host=db_url.hostname,
port=db_url.port,
)
def add_db(request):
db = config.registry.db[db_url.path[1:]]
if db_url.username and db_url.password:
db.authenticate(db_url.username, db_url.password)
return db
def add_fs(request):
return GridFS(request.db)
config.add_request_method(add_db, 'db', reify=True)
config.add_request_method(add_fs, 'fs', reify=True)
config.scan()
return config.make_wsgi_app()
In jobscontroller.py, which is the handler view making the request:
import pyramid_handlers
from nomad.controllers.base_controller import BaseController
class JobsController(BaseController):
#pyramid_handlers.action(renderer='templates/jobs/index.pt')
def index(request):
all_jobs = request.db['jobs'].find()
return {'all_jobs': all_jobs}
I get an error:
all_jobs = request.db['jobs'].find()
AttributeError: 'JobsController' object has no attribute 'db'
I am using Pyramid handlers to manage routing and views, and I know that all of this works because all my routes resolve and deliver web pages. It's only the jobs controller that's funky, and only after I tried adding that request.db call.
Can someone help me understand what's going on?

You're not referring to the request - you're referring to the object itself (usually named self, but you have named it request - which would work if it was just a function and not a method on an object). Since you're inside an object of a class, the first parameter is always the object itself:
class JobsController(BaseController):
#pyramid_handlers.action(renderer='templates/jobs/index.pt')
def index(self, request):
all_jobs = request.db['jobs'].find()
return {'all_jobs': all_jobs}

Flask extension not registered in app.extensions

I want to access some extensions registered on my Flask app. I tried using app.extensions, but some of the extensions that I initialized aren't in the dict.
from flask import current_app
current_app.extensions.get(plugin_name)
extensions.py:
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from flask_mail import Mail
from flask_debugtoolbar import DebugToolbarExtension
from flask_images import Images
from flask_redis import FlaskRedis
from flask_plugins import PluginManager
db = SQLAlchemy()
migrate = Migrate()
mail = Mail()
toolbar = DebugToolbarExtension()
images = Images()
redis = FlaskRedis()
plugin_manager = PluginManager()
main.py:
def configure_extensions(app):
db.init_app(app)
migrate.init_app(app, db)
mail.init_app(app)
toolbar.init_app(app)
images.init_app(app)
redis.init_app(app)
plugin_manager.init_app(
app,
plugin_folder="app/plugins",
plugin_import_path='app.plugins',
)
When I look at current_app.extensions, I only see some of the extensions. plugin_manager and toolbar are not in the list.
>>> for key in app.extensions: print(key)
migrate
mail
images
sqlalchemy
redis
Why aren't some extensions registered in app.extensions? Am I initializing them incorrectly? How can I get the extensions?

Extensions aren't required to add themselves to app.extensions, it's provided as an optional convenience. Apparently, those extensions don't add themselves. If you want that, consider submitting a patch to the relevant projects.
You don't need to use app.extensions to access the extension instances. Instead, import the instances you created, such as from myapp.extensions import db. That's the point of being able to create them separately from initializing them. When used in an app context (such as during a request), the objects will know what app to use.

Flask - working outside of application context

I am testing a Flask application and am receiving a "working outside of application context" error. My file directory is as follows:
api
app.py
__init__.py
models
__init__.py
user.py
resources
__init__.py
deals.py
stores.py
common
__init__.py
calculations.py
decorators.py
My app.py file looks like the following:
import os
from flask import Flask, jsonify, url_for, redirect, request, g, current_app
from flask_pymongo import PyMongo
from flask_restful import Api, Resource
from flask_httpauth import HTTPTokenAuth
from flask.ext.sqlalchemy import SQLAlchemy
from flask.ext.httpauth import HTTPBasicAuth
from resources.deals import Deals
from resources.stores import Stores
from models.user import User
USERDBFILE=os.path.join(os.path.join(os.path.dirname(os.path.realpath(__file__)),'database'),'db.sqlite')
#Deals database
app = Flask(__name__)
app.config["MONGO_DBNAME"] = "database"
mongo = PyMongo(app,config_prefix='MONGO')
app.db = mongo
#User database
app.config['SECRET_KEY'] = 'SECRET KEY'
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///db.sqlite'
app.config['SQLALCHEMY_COMMIT_ON_TEARDOWN'] = True
app.dbuser = SQLAlchemy(app)
#App url
app.APP_URL = "http://127.0.0.1:5000"
#Setup authorization
auth = HTTPTokenAuth(scheme='Token')
#Setup the app
api = Api(app)
api.add_resource(Deals, '/deals', '/Deals/<string:type>/<string:id>',endpoint="dealType")
api.add_resource(Stores, '/stores', '/Stores/<string:type>/<string:id>',endpoint="type")
if __name__ == "__main__":
if not os.path.exists(USERDBFILE):
app.dbuser.create_all()
app.run(debug=True)
My users.py file is as follows:
from flask import current_app
import os
from flask import Flask, abort, request, jsonify, g, url_for
from flask.ext.sqlalchemy import SQLAlchemy
from flask.ext.httpauth import HTTPBasicAuth
from passlib.apps import custom_app_context as pwd_context
from itsdangerous import (TimedJSONWebSignatureSerializer
as Serializer, BadSignature, SignatureExpired)
class User(current_app.dbuser.Model):
__tablename__ = 'user_api'
id = current_app.dbuser.Column(current_app.dbuser.Integer,primary_key=True)
date_created = current_app.dbuser.Column(current_app.dbuser.DateTime,default=current_app.dbuser.func.current_timestamp())
date_modified = current_app.dbuser.Column(current_app.dbuser.DateTime,default=current_app.dbuser.func.current_timestamp(),
onupdate=current_app.dbuser.func.current_timestamp())
# User Name
name = current_app.dbuser.Column(current_app.dbuser.String(128),nullable=False)
# Identification Data: email & password
email = current_app.dbuser.Column(current_app.dbuser.String(128),nullable=False,unique=True)
password = current_app.dbuser.Column(current_app.dbuser.String(192),nullable=False)
company = current_app.dbuser.Column(current_app.dbuser.String(128),nullable=False,unique=True)
# Authorization Data: role & status
role = current_app.dbuser.Column(current_app.dbuser.String(32),nullable=False,default='user')
status = current_app.dbuser.Column(current_app.dbuser.Boolean,nullable=False,default=True)
hourly_limit = current_app.dbuser.Column(current_app.dbuser.Integer,nullable=False,default=100)
daily_limit = current_app.dbuser.Column(current_app.dbuser.Integer,nullable=False,default=2400)
monthly_limit = current_app.dbuser.Column(current_app.dbuser.Integer,nullable=False,default=2400)
admin = current_app.dbuser.Column(current_app.dbuser.Boolean,nullable=False,default=True)
def hash_password(self, password):
self.password_hash = pwd_context.encrypt(password)
def verify_password(self, password):
return pwd_context.verify(password, self.password_hash)
def generate_auth_token(self, expiration=600):
s = Serializer(current_app.config['SECRET_KEY'], expires_in=expiration)
return s.dumps({'id': self.id})
#staticmethod
def verify_auth_token(token):
s = Serializer(current_app.config['SECRET_KEY'])
try:
data = s.loads(token)
except SignatureExpired:
return None # valid token, but expired
except BadSignature:
return None # invalid token
user = User.query.get(data['id'])
return user
I run the file in the same directory as app.py using
python app.py
But it returns the following error:
File "app.py", line 13, in <module>
from models.user import User
File "/Users/toby/api/api/models/user.py", line 10, in <module>
class User(current_app.dbuser.Model):
File "/Users/toby/api/venv/lib/python3.4/site-packages/werkzeug/local.py", line 343, in __getattr__
return getattr(self._get_current_object(), name)
File "/Users/toby/api/venv/lib/python3.4/site-packages/werkzeug/local.py", line 302, in _get_current_object
return self.__local()
File "/Users/toby/api/venv/lib/python3.4/site-packages/flask/globals.py", line 34, in _find_app
raise RuntimeError('working outside of application context')
RuntimeError: working outside of application context
If I move the contents of the user.py file into the app.py file and change the inheritance from current_app.dbuser.Model to app.dbuser.Model it seems to work fine. Does anyone know what I am doing wrong?

Flask-Sqlalchemy binds some sqlalchemy concepts like the session, the engine, and the declaritive base to the flask app. This is convenient because you only have one thing to instantiate at your uwsgi entry point (the app object) but a pain when testing- because you have to instantiate the app object.
EDIT- I am leaving the part about testing below, but I reread your question and realized you're not actually trying to test anything.
You don't have access to the 'current_app' object at import time (when you are trying to initialize your sqlalchemy models). Instead, you have to actually import the app object from your app file. This of course means you have to worry about circular dependencies...
I have a method called 'register_routes' that gets called after I initialize the app object that imports models and views files that require access to the app object at import time.
#at the bottom of app.py
def register_models(app):
from models import User
register_models(app)
# in models.py
from app import app
class User(app.dbuser.Model):
...
EDIT- the below discusses this issue with respect to unit testing
Flask-Testing is a project that attempts to solve these problems, and is almost certainly appropriate for a beginner in this area- it provides a test class to inherit from that will set up your flask app before test cases and tear it down after. (As you come to understand the various globals and what they do you may want to move away from this... but it is very helpful for getting started!)
If you don't want to do that, you need to create an app and initialize an app context before doing anything with your flask-sqlalchemy models. This may just be
app = myapp.create()
with app.test_request_context():
# do some testing...
You will probably want to refresh this in between methods, otherwise global state will leak between test cases.

Basically, flask uses quite a lot of global variables like current_app, request etc. which only exist when a flask app is instantiated and running and in various states.
You have used current_app in the definition of the User object which is evaluated as soon as the file is imported by Python. You need to ensure you only use values like this when an app is already running.
You could move the instantiation of the User class until after the app exists, but I think the root problem is why are you using current_app.dbuser.Boolean rather than say sqlalchemy.types.Boolean?
I'm not a great expert on flask.ext.sqlalchemy, but my guess is you don't need to load those definitions of things like Column and Boolean from the particular instance of the app you have. Using the static definitions from sqlalchemy would prevent you from having a dependency from the User class to the app.