I currently have a Flask app with the following structure:
deploy/
api/
customer/
models.py
contact/
models.py
campaign/
models.py
activity/
models.py
__init__.py
database.py
tests/
test_api.py
Because I have so many models, I have split them out into separate files which is causing me issues when I try and initialise my DB.
In each one of my models, I do the following to make Flask-SQLAlchemy's Model call available:
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class User(db.Model):
...
Doing it this way means there's a different db object for every model which makes it impossible to initialise my app properly.
How can I structure my project so I can have one db object that is used by all my models, tests and my create_app() function in __init__.py?
Here's one way:
myapp/
__init__.py
database.py
app.py
model1/
__init__.py
models.py
model2/
__init__.py
models.py
In the database.py:
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy()
In the various models.py files:
from ..database import db
class User(db.Model):
...
This makes use of explicit relative imports
Related
Following this tutorial of using SQLAlchemy and Postgresql, I based myself for creating a file structure of models, views, templates etc..
requirements.txt
run.py
website/
__init__.py
views/
models/
users.py
static/
templates/
As you can see, inside models I have users.py
from app import db
class Users(db.Model):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
user = db.Column(db.String(), index=True, unique=True)
password = db.Column(db.String(128))
def __init__(self, user, password):
self.user = user
self.password= password
def __repr__(self):
return f"<User {self.user}>"
My init.py
from flask import Flask, render_template
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = "postgresql://postgres:postgres#localhost:5432/project"
db = SQLAlchemy(app)
migrate = Migrate(app, db)
#Routes
#app.route("/")
def index():
return render_template('index.html')
Now, I used these commands to create user table flask db init, flask db migrate, flask db upgrade. OK, after this, was created a table named public.alembic_version, but as you see, Users didnt.
My last tried, was insert this import from models import users between:
migrate = Migrate(app, db)
from models import users
#app.route("/")
def index():
return render_template('index.html')
But the message is ModuleNotFoundError: No module named 'models' I thought that I'm using a reserved word, even changing folder, page name, the error keeps, just change module name.
Update:
Final result, this example work to me
run.py
from website import app
from website.models.users import User
app.run(host="0.0.0.0",debug=True,port=5000)
Update
requirements.txt
run.py
website/
__init__.py
views/
models.py
static/
templates/
What I changed is remove Models Folder e create a file models.py and adding the class Users, Purchase, anything that I want...
I took these imports and include in top of my models.py...
from website import app
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
import psycopg2
app.config['SQLALCHEMY_DATABASE_URI'] = "postgres://postgres:password#ipaddress:5432/databasename"
db = SQLAlchemy(app)
migrate = Migrate(app, db)
In __init__.py added in end of line, after blueprints imports, if I wanted do it, from website.models import Users
from flask import Flask, render_template
app = Flask(__name__)
from website.models import Usuario
The only problem I can find is that you add an import statement (from models import users) in your last attached part of the code. However, you have not created an __init__.py inside your models folder, to declare it as a module. So, that you can import it later as a module like you have created an __init__.py file in your main directory ("website/"). But how you import the models.py file in your run.py file? In your run.py file you should import the models like from website.models.users import Users.
I have two working projects, where I also use FlaskSQLAlchemy & PostgreSQL, but I have set as the SQLALCHEMY_DATABASE_URI = 'postgres://postgres:[password]#localhost:5432/[database_name]'. It works perfectly fine by putting postgres instead of postgresql for the engine configuration.
Reference: Flask-SQLAlchemy import/context issue
UPDATE
Well, you have put your business login (routes, ...) in the init.py file. If you want it to run, I think you should probably put it in the run.py file. Leave the __init__.py files empty.
Furthermore, in your app.py file import your models like this: from .website.models import Users. This requires that you have three __init__.py files. One in your main directory where app.py is located, one in the website folder, and one in the models folder.
*I hope that helps. Please let me know if this works, and if there is anything else I can help you with.
Trying to run the tutorial here: http://flask-sqlalchemy.pocoo.org/2.1/quickstart/ using my app
I have looked at the circular imports problem but I don't think that's it. I'm an absolute beginner to python and flask (and sqlalchemy). My app currently runs, but the database part doesn't
This is the current setup:
mysite
|- __init__.py
|- flask_app.py
|- models.py
|- views.py
init.py
from flask import Flask
app = Flask(__name__)
flask_app.py
from flask import Flask, request, url_for
import random
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql:// -- database uri --'
... app continues here
models.py
from app import app
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
class Foo(db.Model):
... model continues here
views.py
from app import app,models
... views continue here, still not using anything from models
when I run from mysite import db in the python console I get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'db'
Declare your db object in __init__.py. The stuff that is declared in __init__.py defines what can be imported under mysite/.
See: What is __init__.py for?
Also consider moving to the application factory pattern.
For example in __init__.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
def create_app():
app = Flask(__name__)
app.config['DEBUG'] = True
... more application config ...
db.init_app(app)
return app
Then in flask_app.py:
from mysite import create_app, db
app = create_app()
if __name__ == '__main__':
app.run()
I point this out because you instantiate the app object twice in the code you've shown. Which is definitely wrong.
I use supervisor to run my app. It is structured as follows:
My app layout
my_app
__init__.py
my_app
__init__.py
startup
create_app.py
create_users.py
common_settings.py
core
__init__.py
models.py
views.py
Outer __init__.py
from my_app import app
Inner __init__.py
from flask import Flask
from flask_script import Manager
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__) # The WSGI compliant web application object
db = SQLAlchemy(app) # Setup Flask-SQLAlchemy
manager = Manager(app) # Setup Flask-Script
from my_app.startup.create_app import create_app
create_app()
create_app.py
def create_app(extra_config_settings={}):
"""
Initialize Flask applicaton
"""
# ***** Initialize app config settings *****
# Read common settings from 'app/startup/common_settings.py' file
app.config.from_object('app.startup.common_settings')
# Read environment-specific settings from file defined by OS environment variable 'ENV_SETTINGS_FILE'
app.config.from_envvar('ENV_SETTINGS_FILE')
# Load all blueprints with their manager commands, models and views
# Setup Flask-User to handle user account related forms
from my_app.core.models import User
# Setup Flask-User
db_adapter = SQLAlchemyAdapter(db, User) # Setup the SQLAlchemy DB Adapter
user_manager = UserManager(db_adapter, app) # Init Flask-User and bind to app
from my_app import core
return app
my_app/core/__init__.py
from . import models
from . import views
views.py
from my_app import db, app
'''
Register a new user
'''
#app.route('/register', methods = ['POST'])
def register_user():
user_manager = app.user_manager
db_adapter = user_manager.db_adapter
I was trying to follow an example I found online.
I'm creating the variables db_adapter and user_manager in create_app(). Are these the same ones being used in my views.py?
If anyone has any suggestions or links to examples that I can follow to structure my project, it would be greatly appreciated.
Thanks.
Assuming that that's how Flask-User works (sets the user_manager attribute on app), this is trivial to determine, just compare them in the create_app function when you still have a direct reference to the objects.
db_adapter = SQLAlchemyAdapter(db, User)
user_manager = UserManager(db_adapter, app)
assert db_adapter is user_manager.db_adapter
assert user_manager is app.user_manager
However, your entire project layout doesn't make much sense. You should be creating the entire app inside the create_app factory. You should not have an __init__.py file at the top level, that's the project folder not the package. You should use current_app within views to access the app, since it will only be created at runtime by the factory. You should create a manage.py file at the project level to use the factory.
my_project/
my_app/
__init__.py
models.py
views.py
defaults.py
instance/
config.py
manage.py
__init__.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
def create_app():
app = Flask(__name__, instance_relative_config=True)
app.config.from_object('my_app.defaults')
app.config.from_pyfile('config.py')
db.init_app(app)
from my_app.views import bp
app.register_blueprint(bp)
return app
models.py:
from my_app import db
class User(db.Model):
...
views.py:
from flask import Blueprint, render_template
from my_app.models import User
bp = Blueprint('app', __name__)
#bp.route('/')
def index():
return render_template('index.html')
manage.py:
#!/usr/bin/env python
from flask_script import Manager
from my_app import create_app
Manager(create_app).run()
I have a running Flask application that is set up according to a combination of best practices we found online and in Miguel Grinberg's "Flask Web Development" book.
We now need a second Python application, that is NOT a web app, and that needs access to the same models as the Flask application. We wanted to re-use the same models ofcourse, so both apps can benefit from the shared code.
We have removed dependencies on the flask-sqlalchemy extention (which we used before, when we had just the Flask application). And replaced it with the SQLalchemy Declarative extension described here, which is a bit simpler (Flask-SQLalchemy adds a few specific things to standard SQLAlchemy)
In line with the example we have created a database.py file in the root. In our case there are two things different from the Declarative extension example: I put the engine and session in a class, because all of our models use db.session, instead of db_session, and I pass a dictionary with configuration values to the init(), so that I can re-use this database.py from both Flask as well as another application, using a different configuration. it looks like this:
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
class Database(object):
def __init__(self, cfg):
self.engine = create_engine(cfg['SQLALCHEMY_DATABASE_URI'], convert_unicode=True)
self.session = scoped_session(sessionmaker(autocommit=False, autoflush=False, bind=self.engine))
class Model(object):
pass
Base = declarative_base()
So now we come to the actual problem. Flask creates a dictionary-like object containing configuration options, and adds them as a property to the app-instance. It loads them from an instance folder, a config.py in the root of the site and from environment variables. I need to pass in the configuration dictionary from Flask, so I need Flask to FIRST load and assemble the configuration, and after that initialise the database, and have a (configured) db object in the root of the app file. However, we follow the Application factory pattern, so we can use different configurations for different situations (test, production, development).
This means our app/__init__.py looks something like this (simplified):
from flask import Flask
from database import Database
from flask.ext.mail import Mail
from flask_bcrypt import Bcrypt
from config import config
mail = Mail()
bcrypt = Bcrypt()
def create_app(config_name):
app = Flask(__name__, instance_relative_config=True)
if not config_name:
config_name = 'default'
app.config.from_object(config[config_name])
app.config.from_pyfile('config.py')
config[config_name].init_app(app)
db = Database(app.config)
mail.init_app(app)
bcrypt.init_app(app)
#app.teardown_appcontext
def shutdown_session(exception=None):
db.session.remove()
from main import main as main_blueprint
app.register_blueprint(main_blueprint)
return app
But the db (that the models import from ..), now needs to be inside the create_app() function, because that's where Flask loads the configuration. If I would instantiate the db object outside of the create_app() function, it will be importable from the models, but it is not configured!
an example model looks like this, and as you can see, it expects a "db" in the root of the app:
from . base_models import areas
from sqlalchemy.orm import relationship, backref
from ..utils.helper_functions import newid
from .. import db
class Areas(db.Model, areas):
"""Area model class.
"""
country = relationship("Countries", backref=backref('areas'))
def __init__(self, *args, **kwargs):
self.area_id = newid()
super(Areas, self).__init__(*args, **kwargs)
def __str__(self):
return u"{}".format(self.area_name).encode('utf8')
def __repr__(self):
return u"<Area: '{}'>".format(self.area_name).encode('utf8')
So my question is, how can I have a db instance that can be configured externally (by either Flask or another app), and still use the Application Factory Pattern?
edit: The code-example was incorrect, it had an import for Flask-SQLalchemy which was replaced by from database import Database. Sorry for any confusion.
The Flask-SQLAlchemy extension, like most Flask extensions, should be created outside the factory, then initialized in the factory using init_app. This is so that you can use the db object before an app is created.
db = SQLAlchemy()
def create_app():
app = Flask(__name__)
db.init_app(app)
return app
Your Flask app, like any properly designed Python project, should be an installable package. This is simple to do: make sure your project layout makes sense, then add a basic setup.py file.
project/
my_flask_package/
__init__.py # at the most basic, this contains create_app and db
setup.py
from setuptools import setup, find_packages
setup(
name='my_flask_package',
version='1.0',
packages=find_packages(),
install_requires=['flask', 'flask-sqlalchemy'],
)
$ python setup.py sdist
Now you can install your Flask app, along with it's database, for use in other projects. Install and import it in your second project's virtualenv, then create and push an app to initialize it.
$ pip install my_flask_package-1.0.tar.gz
from my_flask_package import db, create_app
create_app().app_context().push()
db.session.query(...)
If you're concerned about overhead involved in creating your application, you could add arguments to the create_app function to control what gets initialized. For most cases this shouldn't be an issue though.
I ran into the same problem.
If you turn on "SQLALCHEMY_ECHO" you'll likely see that a new transaction is started but the corresponding COMMIT/ ROLLBACK is missing.
For what i found out, it has something to do with two SQLAlchemy instances which you also create, once in your model file and once in your web.py. Most likely it's because you interact with your web.py's session and if you query your models there is some context switched which will receive the COMMIT.
I fixed the issue by importing "db" from models and then init it by calling db.init_app(app). According to the logs, committing now works fine.
The #app.teardown_appcontext shouldn't be necessary as it is set up in Flask-SQLAlchemy's SQLAlchemy class (https://github.com/mitsuhiko/flask-sqlalchemy/blob/master/flask_sqlalchemy/init.py)
You can easily share. I will show how. Considering this Flask app:
.
├── config.py
├── db
│ └── test.db
├── do_somenthing2.py ============> Here is run some script 2
├── do_something.py ============> Here is run some script
├── machinelearning
│ ├── models
│ │ ├── restore.py
│ │ ├── train.py
│ │ └── utils.py
│ └── save
│ └── test.ckpt
├── runserver.py ============> Here is run your app
├── test.py
└── web
├── __init__.py
├── api
│ ├── __init__.py
│ ├── app.py ============> Here is app = Flask(__name__)
│ ├── client.py
│ ├── models.py ==========> Here is db = SQLAlchemy(app)
│ ├── sample.json
│ └── utils.py
└── frontend
├── __init__.py
└── routes.py
runserver.py
import os
from config import DEBUG
from web.api.app import app
from web.api.client import *
if __name__ == "__main__":
app.run(debug=DEBUG)
OK. Now you want do use the same models to do another thing. For example: train a machine, serve and save in database (ORM) using the same models.
You can import the app and use the app.test_request_context(). Like that:
do_something.py
from web.api.app import app
from web.api.models import db, user
def do_something():
q = db.session.query(User)\
.filter(User.Name.ilike('Andre'))
for i in q.all():
print (i.Name)
with app.test_request_context():
do_something()
do_something2.py (real example)
from web.api.app import app
from web.api.models import *
def save(df):
passengers = []
records = df.to_dict('records')
for row in records:
p = Passenger(row)
passengers.append(p)
for p in passengers:
db.session.add(p)
db.session.commit()
from ml.models import train, restore
with app.test_request_context():
print ('Trainning model. It will take a while... (~ 5 minutos)')
train.run()
print ('Saving model...')
save(restore.run())
print ('Saved!')
Many answers recommend use (Importing files from different folder):
import sys
sys.path.append('../')
But I disagree when you have a Flask app and other scripts, because you will get crazy solving the relative references.
The approach to install your Flask app, along with it's database, for use in other projects, is another option.
Here you can find a documentation about the packages and modules.
Packages are a way of structuring Python’s module namespace by using
“dotted module names”. For example, the module name A.B designates a
submodule named B in a package named A. Just like the use of modules
saves the authors of different modules from having to worry about each
other’s global variable names, the use of dotted module names saves
the authors of multi-module packages like NumPy or Pillow from having
to worry about each other’s module names.
Lets say I want to build a project Facebook
I need a project structure like
facebook/
__init__.py
feed/
__init__.py
models.py
business.py
views.py
chat/
__init__.py
models.py
business.py
views.py
games/
__init__.py
models.py
business.py
views.py
common/
common.py
runserver.py
How can I structure this well so that when I run
python facebook/runserver.py
It loads views from all my apps internally?
I want to keep this structure because extending the project further is more natural way
I am trying to follow their advice, but don't really understand where I need to write
from flask import Flask
app = Flask(__name__)
and how to import all views from all apps at one place, please help
If lets say I write the above code in facebook/__init__.py, then how in facebook/feed/views.py, I can do
from facebook import app
Use blueprints. Each one of your sub-applications should be a blueprint, and you load every one of them inside your main init file.
Answering your second question
from flask import Flask
app = Flask(__name__)
You should put this into facebook/__init__.py
BTW, my runserver.py and settings.py always resides one level under facebook/.
Like this:
facebook/
__init__.py
feed/
__init__.py
models.py
business.py
views.py
chat/
__init__.py
models.py
business.py
views.py
games/
__init__.py
models.py
business.py
views.py
common/
common.py
runserver.py
settings.py
Content of runserver.py:
from facebook import app
app.run()
I suppose the content of settings.py should not be explained.
Content of facebook/__init__.py:
from flask import Flask
app = Flask(__name__)
app.config.from_object('settings')
from blog.views import blog #blog is blueprint, I prefer to init them inside views.py file
app.register_blueprint(blog,url_prefix="/blog")
I have tried blueprints and came up with a solution which works for me, let me know if you have other ideas.
Project Structure
facebook/
runserver.py
feed/
__init__.py
views.py
chat/
__init__.py
views.py
Code
# create blueprint in feed/__init__.py
from flask import Blueprint
feed = Blueprint('feed', __name__)
import views
# create blueprint in chat/__init__.py
from flask import Blueprint
chat = Blueprint('chat', __name__)
import views
# add views (endpoints) in feed/views.py
from . import feed
#feed.route('/feed')
def feed():
return 'feed'
# add views (endpoints) in chat/views.py
from . import chat
#chat.route('/chat')
def chat():
return 'chat'
# register blueprint and start flask app
from flask import Flask
from feed import feed
from chat import chat
app = Flask(__name__)
app.register_blueprint(feed)
app.register_blueprint(chat)
app.run(debug=True)
In Action
* Running on http://127.0.0.1:5000/
# Hit Urls
http://127.0.0.1:5000/feed # output feed
http://127.0.0.1:5000/chat # output chat