We Keep Coding

SQL Alchemy Won't Drop Tables at end of Test due to MetaData Lock on db

I am working on testing my flask app model. I'm using mysql 5.7, sqlalchemy and pytest.
Within my model, I have a CRUD mixin that I used to manage creating, updating and deleting. Whenever I try to access the object in the Mixin before returning the object to the test function, SQLAlchemy hangs at db.drop_all in my tear down. When I look in mysql at PROCESSLIST, it shows 1 sleep query and 1 query waiting for table metadata lock.
I can fix this by calling db.session.commit in the create method in the mixin before returning the object. However, if I call it at the test teardown (or in the main test function), it doesn't work. I'd prefer not to add an extra commit just to make my tests work as it doesn't feel correct. Does anyone know why this is happening or have any suggested fixes?
models.py
class CRUDMixin(object):
#classmethod
def create(cls, **kwargs):
instance = cls(**kwargs)
saved_instance = instance.save()
# do stuff with saved_instance (i.e. add to full text search engine)
# db.drop_all in teardown works if add db.session.commit() here
return saved_instance
def save(self, commit=True):
db.session.add(self)
if commit:
try:
db.session.commit()
except Exception:
db.session.rollback()
raise
return self
class User(CRUDMixin, db.model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
conftest.py
#pytest.fixture(scope='session')
def app():
app = create_app(TestConfig)
ctx = app.app_context()
ctx.push()
yield app
ctx.pop()
#pytest.fixture(scope='session')
def test_db(app):
db.drop_all()
db.create_all()
# add test db information
yield db
db.session.remove()
db.drop_all() # test hangs within drop all
#pytest.fixture(scope='function')
def db_session(test_db):
connection = db.engine.connect()
transaction = connection.begin()
options = dict(bind=connection, binds={})
session = db.create_scoped_session(options)
db.session = session
yield db
db.session.remove() # tables won't drop if I put db.session.commit before the remove call
transaction.rollback()
connection.close() # even though connection closes, mysql still shows process
test_models.py
class TestUser(object):
def test_add_new(self, db_session):
u = User.create(name='test_name')
assert u.name == 'test_name'
# if I put db.session.commit() here, tables won't drop

How to properly run consecutive tests querying a Flask-SQLAlchemy database?

I'm setting up unit-testing for a Flask project using SQLAlchemy as ORM. For my tests I need to setup a new test database every time I run a single unit-test. Somehow, I cannot seem to run consecutive tests that query the database, even though if I run these tests in isolation they succeed.
I use the flask-testing package, and follow their documentation here.
Here is a working example to illustrate the problem:
app.py:
from flask import Flask
def create_app():
app = Flask(__name__)
return app
if __name__ == '__main__':
app = create_app()
app.run(port=8080)
database.py:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
models.py:
from database import db
class TestModel(db.Model):
"""Model for testing."""
__tablename__ = 'test_models'
id = db.Column(db.Integer,
primary_key=True
)
test/__init__.py:
from flask_testing import TestCase
from app import create_app
from database import db
class BaseTestCase(TestCase):
def create_app(self):
app = create_app()
app.config.update({
'SQLALCHEMY_DATABASE_URI': 'sqlite:///:memory:',
'SQLALCHEMY_TRACK_MODIFICATIONS': False,
'TESTING': True
})
db.init_app(app)
return app
def setUp(self):
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all()
test/test_app.py:
from models import TestModel
from test import BaseTestCase
from database import db
test_model = TestModel()
class TestApp(BaseTestCase):
"""WebpageEnricherController integration test stubs"""
def _add_to_db(self, record):
db.session.add(record)
db.session.commit()
self.assertTrue(record in db.session)
def test_first(self):
"""
This test runs perfectly fine
"""
self._add_to_db(test_model)
result = db.session.query(TestModel).first()
self.assertIsNotNone(result, 'Nothing in the database')
def test_second(self):
"""
This test runs fine in isolation, but fails if run consecutively
after the first test
"""
self._add_to_db(test_model)
result = db.session.query(TestModel).first()
self.assertIsNotNone(result, 'Nothing in the database')
if __name__ == '__main__':
import unittest
unittest.main()
So, I can run TestApp.test_first and TestApp.test_second fine if run in isolation. If I run them consecutively, the first test passes, but the second test fails with:
=================================== FAILURES ===================================
_____________________________ TestApp.test_second ______________________________
self = <test.test_app.TestApp testMethod=test_second>
def test_second(self):
"""
This test runs fine in isolation, but fails if run consecutively
after the first test
"""
self._add_to_db(test_model)
result = db.session.query(TestModel).first()
> self.assertIsNotNone(result, 'Nothing in the database')
E AssertionError: unexpectedly None : Nothing in the database
Something is going wrong in the database setup and teardown, but I cannot figure out what. How do I set this up correctly?

The answer is that you are leaking state between one test and the next by reusing a single TestModel instance defined once in the module scope (test_model = TestModel()).
The state of that instance at the commencement of the first test is transient:
an instance that’s not in a session, and is not saved to the database;
i.e. it has no database identity. The only relationship such an object
has to the ORM is that its class has a mapper() associated with it.
The state of the object at commencement of the second test is detached:
Detached - an instance which corresponds, or previously corresponded,
to a record in the database, but is not currently in any session. The
detached object will contain a database identity marker, however
because it is not associated with a session, it is unknown whether or
not this database identity actually exists in a target database.
Detached objects are safe to use normally, except that they have no
ability to load unloaded attributes or attributes that were previously
marked as “expired”.
This kind of interdependence between tests is almost always a bad idea. You could use make_transient() on the object at the end of every test:
class BaseTestCase(TestCase):
...
def tearDown(self):
db.session.remove()
db.drop_all()
make_transient(test_model)
Or you should construct a new TestModel instance for each test:
class BaseTestCase(TestCase):
...
def setUp(self):
db.create_all()
self.test_model = TestModel()
class TestApp(BaseTestCase):
...
def test_xxxxx(self):
self._add_to_db(self.test_model)
I think the latter is the better choice as there is no danger of any other leaky state getting carried between tests.

Access mongo instance in Flask Unit Tests

I have a simple create_app function in app/__init__.py:
from flask import Flask
from flask_bootstrap import Bootstrap
from flask_login import LoginManager
from flask_pymongo import PyMongo
from .user_management import User
from config import app_config
login_manager = LoginManager()
mongo = PyMongo()
....
def create_app(config):
app = Flask(__name__, instance_relative_config=True, static_folder='static')
login_manager.init_app(app)
login_manager.login_message = 'You must be logged in to view this page'
login_manager.login_view = 'auth.login'
Bootstrap(app)
app.config.from_object(app_config[config])
app.config.from_pyfile('config.py')
if app.testing:
mongo.init_app(app, config_prefix='MONGO2')
else:
mongo.init_app(app)
....
return app
And my config:
class Config():
DEBUG = False
MONGO_HOST = 'localhost'
MONGO_PORT = 27017
....
class DevelopmentConfig(Config):
DEBUG = True
DEVELOPMENT = True
class TestingConfig(Config):
TESTING = True
DEBUG = False
CSRF_ENABLED = False
MONGO2_DBNAME = 'test'
....
app_config = {
'testing': TestingConfig,
'development': DevelopmentConfig,
'production': ProductionConfig
}
Throughout the app, I import the mongo instance from this file make use of it throughout the app. However, I can not find a find to set up a new mongo instance that uses the 'test' database without using the app context, as I'm doing here
My unit test file looks like this.
from app import create_app
import unittest
from app import mongo
class TestCase(unittest.TestCase):
def setUp(self):
app = create_app('testing')
self.app = app
def tearDown(self):
pass
def test_mongo(self):
with self.app.app_context():
assert mongo.db.name == 'test'
if __name__ == '__main__':
unittest.main()
This does not seem like the way to go at all. It also makes it impossible to use the app.test_client(). What is the proper way to instantiate the test database in a flask test setting?

I'm not sure how to do that via config files, but I have another approach that might interest you.
When I have been creating unit tests to my flask / mongo app, I have just simply read sys.args (in my '__init__.py') and decided from there if I want to use test DB or the actual DB, like this:
if "__init__.py" == sys.argv[0]:
db = LoginManager()
else:
db = TestDB() # if running unit tests
I don't know is this the most robust way to work with unit tests, but it is most certainly is super simple way to do it, and does the job.
If you are interested about the whole setup, check out my example app.

You will need to re-create the mongo object in each different unit test that needs to access a different database defined in your config. Here is how I did it and to get a real test I wrote a value to the database and made sure I could read it back two different ways:
import uuid
from datetime import datetime
from flask_testing import TestCase
from flask_pymongo import PyMongo
from project import app
class TestMongoDev(TestCase):
def create_app(self):
app.config.from_object('project.config.DevelopmentConfig')
return app
def test_mongo_development(self):
mongo = PyMongo(app)
testval = str(uuid.uuid4())
inserted_id = mongo.db.test.insert_one({
'testedAt': datetime.now(),
'testval': testval
}).inserted_id
self.assertTrue(mongo.db.name == 'dev')
self.assertTrue(mongo.db.test.find_one({'testval': testval})['testval'] == testval)
self.assertTrue(mongo.db.test.find_one({'_id': inserted_id})['testval'] == testval)
class TestMongoTest(TestCase):
def create_app(self):
app.config.from_object('project.config.TestingConfig')
return app
def test_mongo_testing(self):
mongo = PyMongo(app)
testval = str(uuid.uuid4())
inserted_id = mongo.db.test.insert_one({
'testedAt': datetime.now(),
'testval': testval
}).inserted_id
self.assertTrue(mongo.db.name == 'test')
self.assertTrue(mongo.db.test.find_one({'testval': testval})['testval'] == testval)
self.assertTrue(mongo.db.test.find_one({'_id': inserted_id})['testval'] == testval)
class TestMongoProd(TestCase):
def create_app(self):
app.config.from_object('project.config.ProductionConfig')
return app
def test_mongo_production(self):
mongo = PyMongo(app)
testval = str(uuid.uuid4())
inserted_id = mongo.db.test.insert_one({
'testedAt': datetime.now(),
'testval': testval
}).inserted_id
self.assertTrue(mongo.db.name == 'prod')
self.assertTrue(mongo.db.test.find_one({'testval': testval})['testval'] == testval)
self.assertTrue(mongo.db.test.find_one({'_id': inserted_id})['testval'] == testval)

Writting py test for sqlalchemy app

I am trying convert unit test into py test. I am using the unit test example
class TestCase(unittest.TestCase):
def setUp(self):
app.config['TESTING'] = True
app.config['CSRF_ENABLED'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir,
'test.db')
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all()
I am not sure, What should be its py test version.

I searched high and low for a well explained solution to use SqlAlchemy without Flask-SQLAlchemy and run tests with Pytest, so here's how i have achieved this:
Set up your engine & Session objects as per the docs. (I have opted for sessionmaker as i want to check in my app if the session is still available in the Flask's request thread pool, see: https://dev.to/nestedsoftware/flask-and-sqlalchemy-without-the-flask-sqlalchemy-extension-3cf8
Import your Base object from wherever you've created it in your app. This will create all the tables in your database defined by the engine.
Now we want to Yield a Session back to your unit tests. The idea is to setup before calling Yield & teardown after. Now, in your test you can create a table and populate it with some rows of data etc.
Now we must close the Session, this is important!
Now by calling Base.metadata.drop_all(bind=engine) we drop all the tables in the database ( we can define a table(s) to drop if required, default is: tables=None)
engine = create_engine(create_db_connection_str(config), echo=True)
Session = scoped_session(sessionmaker(bind=engine))
#pytest.fixture(scope="function") # or "module" (to teardown at a module level)
def db_session():
Base.metadata.create_all(engine)
session = Session()
yield session
session.close()
Base.metadata.drop_all(bind=engine)
Now we can pass the function scoped fixture to each unit test:
class TestNotebookManager:
"""
Using book1.mon for this test suite
"""
book_name = "book1"
def test_load(self, client: FlaskClient, db_session) -> None:
notebook = Notebook(name=self.book_name)
db_session.add(book)
db_session.commit()
rv = client.get(f"/api/v1/manager/load?name={self.name}")
assert "200" in rv.status

First off, py.test should just run the existing unittest test case. However the native thing to do in py.test is use a fixture for the setup and teardown:
import pytest
#pytest.fixture
def some_db(request):
app.config['TESTING'] = True
app.config['CSRF_ENABLED'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir, 'test.db')
db.create_all()
def fin():
db.session.remove()
db.drop_all()
request.addfinalizer(fin)
def test_foo(some_db):
pass
Note that I have no idea about SQLAlchemy and whether there are better ways of handling it's setup and teardown. All this example demonstrates is how to turn the setup/teardown methods into a fixture.

Setting up database for testing in Flask

I am developing my first Flask app. It is my side project, so I focus on good practises and design and take my time. I am a bit stuck on testing - I found some examples in docs and here on SO, but they either do not apply to my app or do not seem Pythonic/well designed.
The relevant pieces of code are:
# application module __init__.py
def create_app(config):
app = Flask(__name__)
app.config.from_object('config.%s' % config.title())
return app
config = os.getenv('CONFIG', 'development')
app = create_app(config)
db = SQLAlchemy(app)
# config.py
class Testing(Base):
TESTING = True
SQLALCHEMY_DATABASE_URI = \
'sqlite:///' + os.path.join(_basedir, 'testing.sqlite')
# models.py
class User(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(60), unique=True, nullable=False)
password_hash = db.Column(db.String(60), nullable=False)
# testing.py
class TestCase(unittest.TestCase):
def setUp(self):
self.app = create_app('testing')
# TODO: create connection to testing db
def tearDown(self):
# TODO: abort transaction
pass
The question is: how to implement setUp and tearDown so that in my tests I can use my models and connection do testing database? If I just import db, it would work on development database.
If it helps anything, I do not need to create testing db from scratch, I use Flask-Migrate and tests can assume the testing db is initialized and empty.
Any comments are welcome, I do not mind refactoring if my design is flawed.

It looks like you should just be able to run CONFIG=Testing python -m unittest discover and have everything just work. The only think you may want to change is, instead of calling create_app in your tests, simply import it from __init__.py:
# testing.py
from . import config, db
class TestCase(unittest.TestCase):
def setUp(self):
self.app = create_app(config)
# db is properly set up to use the testing config
# but any *changes* you make to the db object
# (e. g. monkey-patching) will persist between tests
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all()
See here for an example.