I'm setting up unit-testing for a Flask project using SQLAlchemy as ORM. For my tests I need to setup a new test database every time I run a single unit-test. Somehow, I cannot seem to run consecutive tests that query the database, even though if I run these tests in isolation they succeed.
I use the flask-testing package, and follow their documentation here.
Here is a working example to illustrate the problem:
app.py:
from flask import Flask
def create_app():
app = Flask(__name__)
return app
if __name__ == '__main__':
app = create_app()
app.run(port=8080)
database.py:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
models.py:
from database import db
class TestModel(db.Model):
"""Model for testing."""
__tablename__ = 'test_models'
id = db.Column(db.Integer,
primary_key=True
)
test/__init__.py:
from flask_testing import TestCase
from app import create_app
from database import db
class BaseTestCase(TestCase):
def create_app(self):
app = create_app()
app.config.update({
'SQLALCHEMY_DATABASE_URI': 'sqlite:///:memory:',
'SQLALCHEMY_TRACK_MODIFICATIONS': False,
'TESTING': True
})
db.init_app(app)
return app
def setUp(self):
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all()
test/test_app.py:
from models import TestModel
from test import BaseTestCase
from database import db
test_model = TestModel()
class TestApp(BaseTestCase):
"""WebpageEnricherController integration test stubs"""
def _add_to_db(self, record):
db.session.add(record)
db.session.commit()
self.assertTrue(record in db.session)
def test_first(self):
"""
This test runs perfectly fine
"""
self._add_to_db(test_model)
result = db.session.query(TestModel).first()
self.assertIsNotNone(result, 'Nothing in the database')
def test_second(self):
"""
This test runs fine in isolation, but fails if run consecutively
after the first test
"""
self._add_to_db(test_model)
result = db.session.query(TestModel).first()
self.assertIsNotNone(result, 'Nothing in the database')
if __name__ == '__main__':
import unittest
unittest.main()
So, I can run TestApp.test_first and TestApp.test_second fine if run in isolation. If I run them consecutively, the first test passes, but the second test fails with:
=================================== FAILURES ===================================
_____________________________ TestApp.test_second ______________________________
self = <test.test_app.TestApp testMethod=test_second>
def test_second(self):
"""
This test runs fine in isolation, but fails if run consecutively
after the first test
"""
self._add_to_db(test_model)
result = db.session.query(TestModel).first()
> self.assertIsNotNone(result, 'Nothing in the database')
E AssertionError: unexpectedly None : Nothing in the database
Something is going wrong in the database setup and teardown, but I cannot figure out what. How do I set this up correctly?
The answer is that you are leaking state between one test and the next by reusing a single TestModel instance defined once in the module scope (test_model = TestModel()).
The state of that instance at the commencement of the first test is transient:
an instance that’s not in a session, and is not saved to the database;
i.e. it has no database identity. The only relationship such an object
has to the ORM is that its class has a mapper() associated with it.
The state of the object at commencement of the second test is detached:
Detached - an instance which corresponds, or previously corresponded,
to a record in the database, but is not currently in any session. The
detached object will contain a database identity marker, however
because it is not associated with a session, it is unknown whether or
not this database identity actually exists in a target database.
Detached objects are safe to use normally, except that they have no
ability to load unloaded attributes or attributes that were previously
marked as “expired”.
This kind of interdependence between tests is almost always a bad idea. You could use make_transient() on the object at the end of every test:
class BaseTestCase(TestCase):
...
def tearDown(self):
db.session.remove()
db.drop_all()
make_transient(test_model)
Or you should construct a new TestModel instance for each test:
class BaseTestCase(TestCase):
...
def setUp(self):
db.create_all()
self.test_model = TestModel()
class TestApp(BaseTestCase):
...
def test_xxxxx(self):
self._add_to_db(self.test_model)
I think the latter is the better choice as there is no danger of any other leaky state getting carried between tests.
Related
The intent: Refactor my code into MVC (this is just the model/database part), and have the server create the database with tables on first run if the database or tables does not exist.
This works when using a "flat" file with all the classes and functions defined in that file, but after moving out the functions into a service class and the models into their own folder with model classes, the db.create_all() function does not seem to be able to detect the table class correctly any more.
Example structure, (minimum viable problem):
server.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db = SQLAlchemy(app)
def main():
# Intentionally moved into the main() function to prevent import loops
from services.users import UserService
users = UserService(db)
db.create_all()
app.run(debug=True)
if __name__ == '__main__':
main()
services\users.py
# Class used to access users data using a session
from models.users import Users
class UserService:
def __init__(self, db):
self.db = db
def get_all(self):
return self.db.session.query(Users).all()
def get(self, uid):
return self.db.session.query(Users).get(uid)
def add(self, json):
user = Users(email=json['email'], password=json['password'])
self.db.session.add(user)
self.db.session.commit()
return user
models\users.py
# The actual model
from server import db
class Users(db.Model):
_id = db.Column("id", db.Integer, primary_key=True)
email = db.Column(db.Text)
password = db.Column(db.Text)
Result: The database is created, but it is just an empty file with no tables inside of it.
I have also tried placing the db.create_all() inside the service class def __init__(self, db) (grasping at straws here), both as a self reference and as an argument reference. Neither have worked.
I am sure it is something obvious I am missing, but I have boiled down my project to just the bare minimum and still fail to see why it is not working - so I have to ask. How can I get the db.create_all() to detect my table classes correctly and actually create the required tables, while using this code structure (or something similar, in case I have misunderstood MVC)?
The problem is that server.py is executed twice
when it's imported in models/users.py
when server.py is called to run the app
Each execution generates a new db instance. The db imported by the model file adds the models to its metadata, the db created when the app is run has empty metadata.
You can confirm this by printing id(db) and db.metadata.tables at the end of models/users.py and just before the call to db.create_all() in the main function.
You need to structure your code so that only one db gets created. For example, you could move the app configuration and creation code into its own module, mkapp.py (feel free to come up with a better name):
mkapp.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config['SQLALCHEMY_ECHO'] = True
db = SQLAlchemy(app)
And in server.py do
from mkapp import app, db
and in models/users.py do
from mkapp import db
As a bonus, this should also remove the import cycle.
I don't use flask much, so this solution can probably be improved on. For example, having a function create app and db and memoise the results might be better than creating them in top-level module code.
I am working on testing my flask app model. I'm using mysql 5.7, sqlalchemy and pytest.
Within my model, I have a CRUD mixin that I used to manage creating, updating and deleting. Whenever I try to access the object in the Mixin before returning the object to the test function, SQLAlchemy hangs at db.drop_all in my tear down. When I look in mysql at PROCESSLIST, it shows 1 sleep query and 1 query waiting for table metadata lock.
I can fix this by calling db.session.commit in the create method in the mixin before returning the object. However, if I call it at the test teardown (or in the main test function), it doesn't work. I'd prefer not to add an extra commit just to make my tests work as it doesn't feel correct. Does anyone know why this is happening or have any suggested fixes?
models.py
class CRUDMixin(object):
#classmethod
def create(cls, **kwargs):
instance = cls(**kwargs)
saved_instance = instance.save()
# do stuff with saved_instance (i.e. add to full text search engine)
# db.drop_all in teardown works if add db.session.commit() here
return saved_instance
def save(self, commit=True):
db.session.add(self)
if commit:
try:
db.session.commit()
except Exception:
db.session.rollback()
raise
return self
class User(CRUDMixin, db.model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
conftest.py
#pytest.fixture(scope='session')
def app():
app = create_app(TestConfig)
ctx = app.app_context()
ctx.push()
yield app
ctx.pop()
#pytest.fixture(scope='session')
def test_db(app):
db.drop_all()
db.create_all()
# add test db information
yield db
db.session.remove()
db.drop_all() # test hangs within drop all
#pytest.fixture(scope='function')
def db_session(test_db):
connection = db.engine.connect()
transaction = connection.begin()
options = dict(bind=connection, binds={})
session = db.create_scoped_session(options)
db.session = session
yield db
db.session.remove() # tables won't drop if I put db.session.commit before the remove call
transaction.rollback()
connection.close() # even though connection closes, mysql still shows process
test_models.py
class TestUser(object):
def test_add_new(self, db_session):
u = User.create(name='test_name')
assert u.name == 'test_name'
# if I put db.session.commit() here, tables won't drop
I'm porting the tests of a Flask application from unittest to pytest. For tests that need a database I added a fixture that returns a DB session
The DB is an SQLAlchemy object that runs a PostgreSQL database
import pytest
from app import create_app, db
from app.models import Project, Client
#pytest.fixture(scope='function')
def db_session():
app = create_app('testing')
app_context = app.app_context()
app_context.push()
db.create_all()
yield db.session
db.session.remove()
db.drop_all()
app_context.pop()
def test_getJson_withOneProjectSet_returnsBasicClientJson(db_session):
testClient = Client()
db_session.add(testClient)
testProject = Project()
db_session.add(testProject)
testClient.projects.append(testProject)
db_session.commit()
clientJson = testClient.getJson()
assert len(clientJson['projects']) == 1
def test_getJson_withThreeProjectsSet_returnsBasicClientJson(db_session):
testClient = Client()
db_session.add(testClient)
testProject1 = Project()
db_session.add(testProject1)
testProject2 = Project()
db_session.add(testProject2)
testProject3 = Project()
db_session.add(testProject3)
testClient.projects.append(testProject1)
testClient.projects.append(testProject2)
testClient.projects.append(testProject3)
db_session.commit()
clientJson = testClient.getJson()
assert len(clientJson['projects']) == 3
When running the tests, the first one passes but the second one returns an integrity error: sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "project_code_key"
Apparently the DB is not cleaned up after finishing a test-function
The fixture is a port from a working unittest setup/teardown function that has no problems with integrity errors:
# def setUp(self):
# self.app = create_app('testing')
# self.app_context = self.app.app_context()
# self.app_context.push()
# db.create_all()
#
# def tearDown(self):
# db.session.remove()
# db.drop_all()
# self.app_context.pop()
try put scope="module" instead of function, scope module means use this db for all tests then drop it
I am trying convert unit test into py test. I am using the unit test example
class TestCase(unittest.TestCase):
def setUp(self):
app.config['TESTING'] = True
app.config['CSRF_ENABLED'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir,
'test.db')
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all()
I am not sure, What should be its py test version.
I searched high and low for a well explained solution to use SqlAlchemy without Flask-SQLAlchemy and run tests with Pytest, so here's how i have achieved this:
Set up your engine & Session objects as per the docs. (I have opted for sessionmaker as i want to check in my app if the session is still available in the Flask's request thread pool, see: https://dev.to/nestedsoftware/flask-and-sqlalchemy-without-the-flask-sqlalchemy-extension-3cf8
Import your Base object from wherever you've created it in your app. This will create all the tables in your database defined by the engine.
Now we want to Yield a Session back to your unit tests. The idea is to setup before calling Yield & teardown after. Now, in your test you can create a table and populate it with some rows of data etc.
Now we must close the Session, this is important!
Now by calling Base.metadata.drop_all(bind=engine) we drop all the tables in the database ( we can define a table(s) to drop if required, default is: tables=None)
engine = create_engine(create_db_connection_str(config), echo=True)
Session = scoped_session(sessionmaker(bind=engine))
#pytest.fixture(scope="function") # or "module" (to teardown at a module level)
def db_session():
Base.metadata.create_all(engine)
session = Session()
yield session
session.close()
Base.metadata.drop_all(bind=engine)
Now we can pass the function scoped fixture to each unit test:
class TestNotebookManager:
"""
Using book1.mon for this test suite
"""
book_name = "book1"
def test_load(self, client: FlaskClient, db_session) -> None:
notebook = Notebook(name=self.book_name)
db_session.add(book)
db_session.commit()
rv = client.get(f"/api/v1/manager/load?name={self.name}")
assert "200" in rv.status
First off, py.test should just run the existing unittest test case. However the native thing to do in py.test is use a fixture for the setup and teardown:
import pytest
#pytest.fixture
def some_db(request):
app.config['TESTING'] = True
app.config['CSRF_ENABLED'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir, 'test.db')
db.create_all()
def fin():
db.session.remove()
db.drop_all()
request.addfinalizer(fin)
def test_foo(some_db):
pass
Note that I have no idea about SQLAlchemy and whether there are better ways of handling it's setup and teardown. All this example demonstrates is how to turn the setup/teardown methods into a fixture.
I am developing my first Flask app. It is my side project, so I focus on good practises and design and take my time. I am a bit stuck on testing - I found some examples in docs and here on SO, but they either do not apply to my app or do not seem Pythonic/well designed.
The relevant pieces of code are:
# application module __init__.py
def create_app(config):
app = Flask(__name__)
app.config.from_object('config.%s' % config.title())
return app
config = os.getenv('CONFIG', 'development')
app = create_app(config)
db = SQLAlchemy(app)
# config.py
class Testing(Base):
TESTING = True
SQLALCHEMY_DATABASE_URI = \
'sqlite:///' + os.path.join(_basedir, 'testing.sqlite')
# models.py
class User(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(60), unique=True, nullable=False)
password_hash = db.Column(db.String(60), nullable=False)
# testing.py
class TestCase(unittest.TestCase):
def setUp(self):
self.app = create_app('testing')
# TODO: create connection to testing db
def tearDown(self):
# TODO: abort transaction
pass
The question is: how to implement setUp and tearDown so that in my tests I can use my models and connection do testing database? If I just import db, it would work on development database.
If it helps anything, I do not need to create testing db from scratch, I use Flask-Migrate and tests can assume the testing db is initialized and empty.
Any comments are welcome, I do not mind refactoring if my design is flawed.
It looks like you should just be able to run CONFIG=Testing python -m unittest discover and have everything just work. The only think you may want to change is, instead of calling create_app in your tests, simply import it from __init__.py:
# testing.py
from . import config, db
class TestCase(unittest.TestCase):
def setUp(self):
self.app = create_app(config)
# db is properly set up to use the testing config
# but any *changes* you make to the db object
# (e. g. monkey-patching) will persist between tests
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all()
See here for an example.