I am using Django 1.8. I have been writing tests for my Django API in one long file called test_api.py. The structure of the file has been as follows:
def setUpModule():
management.call_command('loaddata', 'frontend/fixtures/data.json',
verbosity=0)
management.call_command('my_custom_command')
def tearDownModule():
management.call_command('flush', verbosity=0, interactive=False)
class TestAPIBNFViews(TestCase):
def test_api_view_list_all(self):
url = '/api/1.0/bnf_code'
# do testing
def test_api_view_query(self):
# more testint
The fixtures and management command being loaded once before all the tests run, and so far this has worked great.
Now however the file is getting long and unwieldy, and I want to split it into multiple files. I've created multiple files called test_list and test_query and given each a setUpModule section as above.
However, firstly this isn't DRY, and secondly when I run python manage.py test, lots of the tests fail with duplicate foreign key errors like:
ProgrammingError: relation "frontend_sha_id" already exists
I guess this isn't surprising, since the tests are trying to create the test database multiple times.
However, if I remove setUpModule from all but the first test (listed alphabetically by filename), the other tests fail because they can't see any data.
How can I run setUpModule once, before all the tests run, and still keep the tests in separate files for convenience?
Instead of using a global setUpModule for both test classes, you can alternatively use setUpTestData once at each TestCase class. From Django documentation: Testing tools:
The class-level atomic block ... allows the creation of initial data at the class level, once for the whole TestCase.
Related
I want to write pytest unit test in Kedro 0.17.5. They need to perform integrity checks on dataframes created by the pipeline.
These dataframes are specified in the catalog.yml and already persisted successfully using kedro run. The catalog.yml is in conf/base.
I have a test module test_my_dataframe.py in src/tests/pipelines/my_pipeline/.
How can I load the data catalog based on my catalog.yml programmatically from within test_my_dataframe.py in order to properly access my specified dataframes?
Or, for that matter, how can I programmatically load the whole project context (including the data catalog) in order to also execute nodes etc.?
For unit testing, we test just the function which we are testing, and everything external to the function we should mock/patch. Check if you really need kedro project context while writing the unit test.
If you really need project context in test, you can do something like following
from kedro.framework.project import configure_project
from kedro.framework.session import KedroSession
with KedroSession.create(package_name="demo", project_path=Path.cwd()) as session:
context = session.load_context()
catalog = context.catalog
or you can also create pytest fixture to use it again and again with scope of your choice.
#pytest.fixture
def get_project_context():
session = KedroSession.create(
package_name="demo",
project_path=Path.cwd()
)
_activate_session(session, force=True)
context = session.load_context()
return context
Different args supported by KedroSession create you can check it here https://kedro.readthedocs.io/en/0.17.5/kedro.framework.session.session.KedroSession.html#kedro.framework.session.session.KedroSession.create
To read more about pytest fixture you can refer to https://docs.pytest.org/en/6.2.x/fixture.html#scope-sharing-fixtures-across-classes-modules-packages-or-session
I am writing unit-tests with pytest-django for a django app. I want to make my tests more performant and doing so is requiring me to keep data saved in database for a certain time and not drop it after one test only. For example:
#pytest.mark.django_db
def test_save():
p1 = MyModel.objects.create(description="some description") # this object has the id 1
p1.save()
#pytest.mark.django_db
def test_modify():
p1 = MyModel.objects.get(id=1)
p1.description = "new description"
What I want is to know how to keep both tests separated but both will use the same test database for some time and drop it thereafter.
I think what you need are pytest fixtures. They allow you yo create objects (stored in database if needed) that will be used during tests. You can have a look at pytest fixtures scope that you can set so that the fixture is not deleted from database and reloading for each test that requires it but instead is created once for a bunch of tests and deleted afterwards.
You should read the documentation of pytest fixtures (https://docs.pytest.org/en/6.2.x/fixture.html) and the section dedicated to fixtures' scope (https://docs.pytest.org/en/6.2.x/fixture.html#scope-sharing-fixtures-across-classes-modules-packages-or-session).
I have a class which has a side effect at import time: it populates a field with a value from a deployed infrastructure. It basically boils down to this:
class MyModel:
class Meta:
table_name = get_param("table_name")
I want to use instances of MyModel in offline tests, so I need to make sure that boto3 doesn't actually run the production get_param code. I've done it by mocking out get_param in these tests:
with patch("….get_param"):
from … import MyModel
def test_…
Clunky, but it sort of works. The problem is that if I run pytest with this test file first and then an acceptance test file (which does need to use the real get_param), the acceptance test ends up with a mocked version of get_param (reversing the file sequence makes both tests pass). The acceptance test runs fine on its own, but I'd rather not pull apart the test suite based on such a weird premise. Is there some way to avoid this namespace pollution?
Looks like I don't understand how Python imports work.
I implemented scenario support for my pytest-based tests, and it works well.
However, one of my fixtures initialises the database with clean tables, and the second scenario runs the test with a dirty database. How can I get the database fixture re-initialised or refreshed in subsequent scenarios?
To be clear, I want to see this:
scenario 1
test_demo1 gets a fresh DB, and the test writes to the DB
test_demo2 does not re-init the DB, but sees the changes made by test_demo1
scenario 2
test_demo1 gets a fresh DB again, and the test writes to the DB
test_demo2 does not re-init the DB, but sees the changes made by test_demo1 only in scenario 2.
def pytest_runtest_setup(item):
if hasattr(item.cls, "scenarios") and "first" in item.keywords:
# what to do here?
#pytest.mark.usefixtures("db")
class TestSampleWithScenarios:
scenarios = [scenario1, scenario2]
#pytest.mark.first
def test_demo1(self, db):
# db is dirty here in scenario2
pass
def test_demo2(self, db):
pass
I'm currently digging through the pytest sources to find an answer and I will post here once I have something.
I've found a workaround. Have a regular test which uses the DB fixture, parametrize with the scenarios, and call the tests in the class directly:
#pytest.mark.parametrize(["scenario"], scenarios)
def test_sample_with_scenarios(db, scenario):
TestSampleWithScenarios().test_demo1(db, scenario)
TestSampleWithScenarios().test_demo2(db, scenario)
I've got a module pagetypes.py that extracts a couple of constants (I shouldn't really use word constant here) from the db for later reuse:
def _get_page_type_(type):
return PageType.objects.get(type=type)
PAGE_TYPE_MAIN = _get_page_type_('Main')
PAGE_TYPE_OTHER = _get_page_type_('Other')
then somewhere in views I do:
import pagetypes
...
print pagetypes.PAGE_TYPE_MAIN #simplified
It all works fine when db has those records and I make sure it does... unless this code is under test. In that case I want to load those records into db via fixtures. The problem with that is that fixtures are not loaded (even syncdb is not run) by the time pagetypes module is imported resulting in _get_page_type_ call failing with:
psycopg2.ProgrammingError: relation "pagetype" does not exist
Test runner always tries to import pagetypes module, because it is imported by view that is under test.
How do I get around this problem?
I was thinking of lazily loading pagetype constants PAGE_TYPE_MAIN, PAGE_TYPE_OTHER, but then I want it to fail early if those records are not in the db (or fixtures if under test), so I don't really know how to implement this.
I was also thinking of object level caching and just call PageType.objects.get(type=type) whenever constant is used/called, but wouldn't that be an overkill? Calling orm without cache would result in too many db calls, which I want to prevent.
It must be something very simple, but I can't work it out. ;-)
I would use the functions instead of the constants, but memoize them:
_cache = {}
def get_page_type(type_name):
if type_name not in _cache:
_cache[type_name] = PageType.objects.get(type=type_name)
return _cache[type_name]
So now you'd call get_page_type('Main') directly when necessary.