We Keep Coding

db.create_all() not generating db

I'm trying to test Flask with SQLAlchemy and I stumbeld accross this problem. First, I have to note that I read all of the related threads and none of them solves my problem. I have a problem that db.create_all() doesn't generate the table I defined. I have model class in file person.py:
from website import db
class Person(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String, nullable=False)
password = db.Column(db.String)
width = db.Column(db.Integer)
height = db.Column(db.Integer)
agent = db.Column(db.String)
user_data_dir = db.Column(db.String)
And in my website.py which is the file from where I launch the app:
from flask import Flask, jsonify, render_template, request
from flask_sqlalchemy import SQLAlchemy
# create the extension
db = SQLAlchemy()
def start_server(host, port, debug=False):
from person import Person
# create the app
app = Flask(__name__,
static_url_path='',
static_folder='web/static',
template_folder='web/templates')
# configure the SQLite database, relative to the app instance folder
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///database0.db"
# initialize the app with the extension
db.init_app(app)
print('initialized db')
print('creating tables...')
with app.app_context():
db.create_all()
db.session.add(Person(username="example33"))
db.session.commit()
person = db.session.execute(db.select(Person)).scalar()
print('persons')
print(person.username)
if __name__ == '__main__':
start_server(host='0.0.0.0', port=5002, debug=True)
I think the problem might be that the Person class is not importing properly, because when I put the class inside the start_server function it executes fine and creates the table, but I don't know why this is happening. I followed all the advice and imported it before everything, and also I share the same db object between the 2 files

There is probably a better way to do this but this is the only way I could get this to work. You need to create a models.py file or w.e you wanna call it. Then all your database stuff goes in there. The db engine, ALL your models and a function to initialize it all. The reason is, you are having import issues where Person is imported but not fully and so the db doesn't have it in its metadata.
models.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Person(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String, nullable=False)
password = db.Column(db.String)
width = db.Column(db.Integer)
height = db.Column(db.Integer)
agent = db.Column(db.String)
user_data_dir = db.Column(db.String)
# All other models
def initialize_db(app: Flask):
db.init_app(app)
with app.app_context():
db.create_all()
main.py
from flask import Flask
import models
def start_server(host, port, debug=False):
app = Flask(__name__)
# configure the SQLite database, relative to the app instance folder
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///database0.db"
# initialize the app with the extension
models.initialize_db(app)
db = models.db
with app.app_context():
db.session.add(models.Person(username="example33"))
db.session.commit()
person = db.session.execute(db.select(models.Person)).scalar()
print('persons')
print(person.username)
if __name__ == '__main__':
start_server(host='0.0.0.0', port=5002, debug=True)

I am reading the documentation,
which explains that the function will
Create all tables stored in this metadata.
That leads me to believe Person is not associated with the db metadata.
You mentioned
when I put the class inside the start_server function it ... creates the table
Your from person import Person is nice enough,
but I suspect we wanted a simple import person.
In many apps the idiom would be import models.
Failing that, you may be able to point
create_all in the right direction
with this optional parameter:
tables – Optional list of Table objects, which is a subset of the total tables in the MetaData
Please let us know
what technical approach worked for you.

Flask_SQLAlchemy modularization issues due ORM

I am trying to build an API using Flask. For database actions I use flask_sqlalchemy.
In my main file, the flask app is initalized. I pass the resulting instance to another file where the configuration is set and to my database module that handles database operations.
main.py:
app = flask.Flask(__name__) # initialize flask app
#initialize modules with app
config.init(app)
database.init(app)
The problem is, the relations I use in the database are in a seperate file and it needs the db object to declare the classes for ORM.
My idea was to declare db and initialize it later in an init function, but that doesn't work in this case, because the db object is undefined when the pythonfile is loaded by an import.
relations.py
db: SQLAlchemy
def init(db):
Relations.db = db
class Series(db.Model):
"""Representation of a series
"""
id = db.Column(db.String(255), primary_key=True)
title = db.Column(db.String(255))
class User(db.Model):
"""Representation of a user
"""
id = db.Column(db.INT, primary_key=True)
name = db.Column(db.String(255))
class Subscription(db.Model):
"""Representation of a subscription
"""
series_id = db.Column(db.INT, primary_key=True)
user_id = db.Column(db.String(255), primary_key=True)
My database module uses the way and it works fine(init.py file):
db: SQLAlchemy
def init(app):
database.db = SQLAlchemy(app)
# handle database operations...
One approach to solve the issue is just using another instance in the relations.py like that:
app = flask.Flask(__name__)
db = SQLAlchemy(app)
# declare classes...
I tried it out and it workes, but that is not a nice way to solve this and leads to other problems.
Importing it from main does also not work because of circular import.
I have no idea how to smoothly solve this without removing modularization. I would be thankful for any inputs. If I should add any further information, just let me know.

I would create the app variable in your main.py file but leave out the initializing part. From there you call a function from init.py to basically set up the database. That is what I did for my last flask project.
Main.py:
from init import create_app
app = create_app()
if __name__ == '__main__':
app.run(debug=True)
Init.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
DB_NAME = "database.db"
def create_app():
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = f'sqlite:///{DB_NAME}'
db.init_app(app)
create_database(app)
#Other operations ...
return app
Relations.py
from init import db
#all your classes ...
db.create_all()
So now you can import the db object to your relations.py file from the init.py.

Flask_SQLAlchemy, db.create_all() is unable to "see" my tables when imported though a service class

The intent: Refactor my code into MVC (this is just the model/database part), and have the server create the database with tables on first run if the database or tables does not exist.
This works when using a "flat" file with all the classes and functions defined in that file, but after moving out the functions into a service class and the models into their own folder with model classes, the db.create_all() function does not seem to be able to detect the table class correctly any more.
Example structure, (minimum viable problem):
server.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db = SQLAlchemy(app)
def main():
# Intentionally moved into the main() function to prevent import loops
from services.users import UserService
users = UserService(db)
db.create_all()
app.run(debug=True)
if __name__ == '__main__':
main()
services\users.py
# Class used to access users data using a session
from models.users import Users
class UserService:
def __init__(self, db):
self.db = db
def get_all(self):
return self.db.session.query(Users).all()
def get(self, uid):
return self.db.session.query(Users).get(uid)
def add(self, json):
user = Users(email=json['email'], password=json['password'])
self.db.session.add(user)
self.db.session.commit()
return user
models\users.py
# The actual model
from server import db
class Users(db.Model):
_id = db.Column("id", db.Integer, primary_key=True)
email = db.Column(db.Text)
password = db.Column(db.Text)
Result: The database is created, but it is just an empty file with no tables inside of it.
I have also tried placing the db.create_all() inside the service class def __init__(self, db) (grasping at straws here), both as a self reference and as an argument reference. Neither have worked.
I am sure it is something obvious I am missing, but I have boiled down my project to just the bare minimum and still fail to see why it is not working - so I have to ask. How can I get the db.create_all() to detect my table classes correctly and actually create the required tables, while using this code structure (or something similar, in case I have misunderstood MVC)?

The problem is that server.py is executed twice
when it's imported in models/users.py
when server.py is called to run the app
Each execution generates a new db instance. The db imported by the model file adds the models to its metadata, the db created when the app is run has empty metadata.
You can confirm this by printing id(db) and db.metadata.tables at the end of models/users.py and just before the call to db.create_all() in the main function.
You need to structure your code so that only one db gets created. For example, you could move the app configuration and creation code into its own module, mkapp.py (feel free to come up with a better name):
mkapp.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config['SQLALCHEMY_ECHO'] = True
db = SQLAlchemy(app)
And in server.py do
from mkapp import app, db
and in models/users.py do
from mkapp import db
As a bonus, this should also remove the import cycle.
I don't use flask much, so this solution can probably be improved on. For example, having a function create app and db and memoise the results might be better than creating them in top-level module code.

How to properly run consecutive tests querying a Flask-SQLAlchemy database?

I'm setting up unit-testing for a Flask project using SQLAlchemy as ORM. For my tests I need to setup a new test database every time I run a single unit-test. Somehow, I cannot seem to run consecutive tests that query the database, even though if I run these tests in isolation they succeed.
I use the flask-testing package, and follow their documentation here.
Here is a working example to illustrate the problem:
app.py:
from flask import Flask
def create_app():
app = Flask(__name__)
return app
if __name__ == '__main__':
app = create_app()
app.run(port=8080)
database.py:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
models.py:
from database import db
class TestModel(db.Model):
"""Model for testing."""
__tablename__ = 'test_models'
id = db.Column(db.Integer,
primary_key=True
)
test/__init__.py:
from flask_testing import TestCase
from app import create_app
from database import db
class BaseTestCase(TestCase):
def create_app(self):
app = create_app()
app.config.update({
'SQLALCHEMY_DATABASE_URI': 'sqlite:///:memory:',
'SQLALCHEMY_TRACK_MODIFICATIONS': False,
'TESTING': True
})
db.init_app(app)
return app
def setUp(self):
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all()
test/test_app.py:
from models import TestModel
from test import BaseTestCase
from database import db
test_model = TestModel()
class TestApp(BaseTestCase):
"""WebpageEnricherController integration test stubs"""
def _add_to_db(self, record):
db.session.add(record)
db.session.commit()
self.assertTrue(record in db.session)
def test_first(self):
"""
This test runs perfectly fine
"""
self._add_to_db(test_model)
result = db.session.query(TestModel).first()
self.assertIsNotNone(result, 'Nothing in the database')
def test_second(self):
"""
This test runs fine in isolation, but fails if run consecutively
after the first test
"""
self._add_to_db(test_model)
result = db.session.query(TestModel).first()
self.assertIsNotNone(result, 'Nothing in the database')
if __name__ == '__main__':
import unittest
unittest.main()
So, I can run TestApp.test_first and TestApp.test_second fine if run in isolation. If I run them consecutively, the first test passes, but the second test fails with:
=================================== FAILURES ===================================
_____________________________ TestApp.test_second ______________________________
self = <test.test_app.TestApp testMethod=test_second>
def test_second(self):
"""
This test runs fine in isolation, but fails if run consecutively
after the first test
"""
self._add_to_db(test_model)
result = db.session.query(TestModel).first()
> self.assertIsNotNone(result, 'Nothing in the database')
E AssertionError: unexpectedly None : Nothing in the database
Something is going wrong in the database setup and teardown, but I cannot figure out what. How do I set this up correctly?

The answer is that you are leaking state between one test and the next by reusing a single TestModel instance defined once in the module scope (test_model = TestModel()).
The state of that instance at the commencement of the first test is transient:
an instance that’s not in a session, and is not saved to the database;
i.e. it has no database identity. The only relationship such an object
has to the ORM is that its class has a mapper() associated with it.
The state of the object at commencement of the second test is detached:
Detached - an instance which corresponds, or previously corresponded,
to a record in the database, but is not currently in any session. The
detached object will contain a database identity marker, however
because it is not associated with a session, it is unknown whether or
not this database identity actually exists in a target database.
Detached objects are safe to use normally, except that they have no
ability to load unloaded attributes or attributes that were previously
marked as “expired”.
This kind of interdependence between tests is almost always a bad idea. You could use make_transient() on the object at the end of every test:
class BaseTestCase(TestCase):
...
def tearDown(self):
db.session.remove()
db.drop_all()
make_transient(test_model)
Or you should construct a new TestModel instance for each test:
class BaseTestCase(TestCase):
...
def setUp(self):
db.create_all()
self.test_model = TestModel()
class TestApp(BaseTestCase):
...
def test_xxxxx(self):
self._add_to_db(self.test_model)
I think the latter is the better choice as there is no danger of any other leaky state getting carried between tests.

Writting py test for sqlalchemy app

I am trying convert unit test into py test. I am using the unit test example
class TestCase(unittest.TestCase):
def setUp(self):
app.config['TESTING'] = True
app.config['CSRF_ENABLED'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir,
'test.db')
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all()
I am not sure, What should be its py test version.

I searched high and low for a well explained solution to use SqlAlchemy without Flask-SQLAlchemy and run tests with Pytest, so here's how i have achieved this:
Set up your engine & Session objects as per the docs. (I have opted for sessionmaker as i want to check in my app if the session is still available in the Flask's request thread pool, see: https://dev.to/nestedsoftware/flask-and-sqlalchemy-without-the-flask-sqlalchemy-extension-3cf8
Import your Base object from wherever you've created it in your app. This will create all the tables in your database defined by the engine.
Now we want to Yield a Session back to your unit tests. The idea is to setup before calling Yield & teardown after. Now, in your test you can create a table and populate it with some rows of data etc.
Now we must close the Session, this is important!
Now by calling Base.metadata.drop_all(bind=engine) we drop all the tables in the database ( we can define a table(s) to drop if required, default is: tables=None)
engine = create_engine(create_db_connection_str(config), echo=True)
Session = scoped_session(sessionmaker(bind=engine))
#pytest.fixture(scope="function") # or "module" (to teardown at a module level)
def db_session():
Base.metadata.create_all(engine)
session = Session()
yield session
session.close()
Base.metadata.drop_all(bind=engine)
Now we can pass the function scoped fixture to each unit test:
class TestNotebookManager:
"""
Using book1.mon for this test suite
"""
book_name = "book1"
def test_load(self, client: FlaskClient, db_session) -> None:
notebook = Notebook(name=self.book_name)
db_session.add(book)
db_session.commit()
rv = client.get(f"/api/v1/manager/load?name={self.name}")
assert "200" in rv.status

First off, py.test should just run the existing unittest test case. However the native thing to do in py.test is use a fixture for the setup and teardown:
import pytest
#pytest.fixture
def some_db(request):
app.config['TESTING'] = True
app.config['CSRF_ENABLED'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir, 'test.db')
db.create_all()
def fin():
db.session.remove()
db.drop_all()
request.addfinalizer(fin)
def test_foo(some_db):
pass
Note that I have no idea about SQLAlchemy and whether there are better ways of handling it's setup and teardown. All this example demonstrates is how to turn the setup/teardown methods into a fixture.