According to "https://docs.pytest.org/en/stable/getting-started.html" in Pytest when grouping tests inside classes is that each test has a unique instance of the class. Having each test share the same class instance would be very detrimental to test isolation and would promote poor test practices.What does that mean ? This is outlined below:
content of test_class_demo.
class TestClassDemoInstance:
def test_one(self):
assert 0
def test_two(self):
assert 0
Imagine you are testing a user account system, where you can create users and change passwords. You need to have a user before you can change its password, and you don't want to repeat yourself, so you could naively structure the test like this:
class TestUserService:
def test_create_user(self):
# Store user_id on self to reuse it in subsequent tests.
self.user_id = UserService.create_user("timmy")
assert UserService.get_user_id("timmy") == self.user_id
def test_set_password(self):
# Set the password of the user we created in the previous test.
UserService.set_password(self.user_id, "hunter2")
assert UserService.is_password_valid(self.user_id, "hunter2")
But by using self to pass data from one test case to the next, you have created several problems:
The tests must be run in this order. First test_create_user, then test_set_password.
All tests must be run. You can't re-run just test_set_password independently.
All previous tests must pass. If test_create_user fails, we can no longer meaningfully run test_set_password.
Tests can no longer be run in parallel.
So to prevent this kind of design, pytest wisely decided that each test gets a brand new instance of the test class.
Just to add to Thomas answer, please be aware that currently there is an error in the documentation you mentioned. As described in this open issue on pytest Github, in what is shown below the explanation, both tests are demonstrated to share the same instance of the class, at 0xdeadbeef. This seems to demonstrate the opposite of what is stated (that the instances are not the same).
Related
I am relatively new to pytest, so I understand the simple use of fixtures that looks like that:
#pytest.fixture
def example_data():
return "abc"
and then using it in a way like this:
def test_data(self, example_data):
assert example_data == "abc"
I am working on a django app and where it gets confusing is when I try to use fixtures to create django objects that will be used for the tests.
The closest solution that I've found online looks like that:
#pytest.fixture
def test_data(self):
users = get_user_model()
client = users.objects.get_or_create(username="test_user", password="password")
and then I am expecting to be able to access this user object in a test function:
#pytest.mark.django_db
#pytest.mark.usefixtures("test_data")
async def test_get_users(self):
# the user object should be included in this queryset
all_users = await sync_to_async(User.objects.all)()
.... (doing assertions) ...
The issue is that when I try to list all the users I can't find the one that was created as part of the test_data fixture and therefore can't use it for testing.
I noticed that if I create the objects inside the function then there is no problem, but this approach won't work for me because I need to parametrize the function and depending on the input add different groups to each user.
I also tried some type of init or setup function for my testing class and creating the User test objects from there but this doesn't seem to be pytest's recommended way of doing things. But either way that approach didn't work either when it comes to listing them later.
Is there any way to create test objects which will be accessible when doing a queryset?
Is the right way to manually create separate functions and objects for each test case or is there a pytest-way of achieving that?
I am writing unit-tests with pytest-django for a django app. I want to make my tests more performant and doing so is requiring me to keep data saved in database for a certain time and not drop it after one test only. For example:
#pytest.mark.django_db
def test_save():
p1 = MyModel.objects.create(description="some description") # this object has the id 1
p1.save()
#pytest.mark.django_db
def test_modify():
p1 = MyModel.objects.get(id=1)
p1.description = "new description"
What I want is to know how to keep both tests separated but both will use the same test database for some time and drop it thereafter.
I think what you need are pytest fixtures. They allow you yo create objects (stored in database if needed) that will be used during tests. You can have a look at pytest fixtures scope that you can set so that the fixture is not deleted from database and reloading for each test that requires it but instead is created once for a bunch of tests and deleted afterwards.
You should read the documentation of pytest fixtures (https://docs.pytest.org/en/6.2.x/fixture.html) and the section dedicated to fixtures' scope (https://docs.pytest.org/en/6.2.x/fixture.html#scope-sharing-fixtures-across-classes-modules-packages-or-session).
I need to:
Avoid to use save() in tests
Use #pytest.mark.django_db on all tests inside this class
Create a number of trx fixtures (10/20) to act like false data.
import pytest
from ngg.processing import (
elab_data
)
class TestProcessing:
#pytest.mark.django_db
def test_elab_data(self, plan,
obp,
customer,
bac,
col,
trx_1,
trx_2,
...):
plan.save()
obp.save()
customer.save()
bac.save()
col.save()
trx.save()
elab_data(bac, col)
Where the fixtures are simply models like that:
#pytest.fixture
def plan():
plan = Plan(
name = 'test_plan',
status = '1'
)
return plan
I don't find this way really clean. How would you do that?
TL;DR
test.py
import pytest
from ngg.processing import elab_data
#pytest.mark.django_db
class TestProcessing:
def test_elab_data(self, plan, obp, customer, bac, col, trx_1, trx_2):
elab_data(bac, col)
conftest.py
#pytest.fixture(params=[
('test_plan', 1),
('test_plan2', 2),
])
def plan(request, db):
name, status = request.param
return Plan.objects.create(name=name, status=status)
I'm not quite sure if I got it correctly
Avoid to use save() in tests
You may create objects using instance = Model.objects.create() or just put instance.save() in fixtures.
As described at note section here
To access the database in a fixture, it is recommended that the fixture explicitly request one of the db, transactional_db or django_db_reset_sequences fixtures.
and at fixture section here
This fixture will ensure the Django database is set up. Only required for fixtures that want to use the database themselves. A test function should normally use the pytest.mark.django_db() mark to signal it needs the database.
you may want to use db fixture in you records fixtures and keep django_db mark in your test cases.
Use #pytest.mark.django_db on all tests inside this class
To mark whole classes you may use decorator on classes or pytestmark variable as described here.
You may use pytest.mark decorators with classes to apply markers to all of its test methods
To apply marks at the module level, use the pytestmark global variable
Create a number of trx fixtures (10/20) to act like false data.
I didn't quite get what you were trying to do but still would assume that it is one of the following things:
Create multiple objects and pass them as fixtures
In that case you may want to create a fixture that will return generator or list and use whole list instead of multiple fixtures
Test case using different variants of fixture but with one/few at a time
In that case you may want to parametrize your fixture so it will return different objects and test case will run multiple times - one time per one variant
I have an issue with a my unit tests and the way django manages transactions.
In my code I have a function:
def send():
autocommit = transaction.set_autocommit(False)
try:
# stuff
finally:
transaction.rollback()
transaction.set_autocommit(autocommit)
In my test I have:
class MyTest(TransactionTestCase):
def test_send(self):
send()
The issue I am having is that my test_send passes successfully but not 80% of my other tests.
It seems the transaction of the other tests are failing
btw I am using py.test to run my tests
EDIT:
To make things more clear when I run my tests with only
myapp.test.test_module.py it runs fine and all 3 tests passes but when I run all my tests most of the fails, will try to produce a test app
Also all my tests passes with the default test runner from django
EDIT2:
Here is A minimal example to test this issue:
class ManagementTestCase(TransactionTestCase):
def test_transfer_ubl(self, MockExact):
pass
class TestTestCase(TestCase):
def test_1_user(self):
get_user_model().objects.get(username="admin")
self.assertEqual(get_user_model().objects.all().count(), 1)
Bear in mind there is a datamigration that adds an "admin" user (the TestTestCase succeeds alone but not when the ManagmentTestCase is run before)
It seems autocommit has nothing to do with it.
The TestCase class wraps the tests inside two atomic blocks. Therefore it is not possible to use transaction.set_autocommit() or transaction.rollback() if you are inheriting from TestCase.
As the docs say, you should use TransactionTestCase if you are testing specific database transaction behaviour.
having autocommit = transaction.set_autocommit(False) inside the send function feels wrong. Disabling the transaction is done here presumably for testing purposes, but the rule of thumb is to keep your test logic outside your code.
As #Alasdair has pointed out, django docs states "Django’s TestCase class also wraps each test in a transaction for performance reasons."
It is not clear from your question whether you're testing specific database transaction logic or not, if that is the case then #Alasdair's answer of using the TransactionTestCase is the way to go.
Otherwise, removing the transaction context switch from around the stuff inside your send function should help.
Since you mentioned pytest as your test runner, I would also recommend making use of pytest. Pytest-django plugin comes with nice features such selectively setting some of your tests to require transactions, using markers.
pytest.mark.django_db(transaction=False)
If installing a plugin is too much, then you could roll your own transaction manage fixture. Like
#pytest.fixture
def no_transaction(request):
autocommit = transaction.set_autocommit(False)
def rollback():
transaction.rollback()
transaction.set_autocommit(True)
request.addfinalizer(rollback)
Your test_send will then require the no_transaction fixture.
def test_send(no_transaction):
send()
For those who still looking for a solution, serialized_rollback option is a way to go:
class ManagementTestCase(TransactionTestCase):
serialized_rollback = True
def test_transfer_ubl(self, MockExact):
pass
class TestTestCase(TestCase):
def test_1_user(self):
get_user_model().objects.get(username="admin")
self.assertEqual(get_user_model().objects.all().count(), 1)
from the docs
Django can reload that data for you on a per-testcase basis by setting the serialized_rollback option to True in the body of the TestCase or TransactionTestCase, but note that this will slow down that test suite by approximately 3x.
Unfortunately, pytest-django still missing this feature.
I've never written any tests in my life, but I'd like to start writing tests for my Django projects. I've read some articles about tests and decided to try to write some tests for an extremely simple Django app or a start.
The app has two views (a list view, and a detail view) and a model with four fields:
class News(models.Model):
title = models.CharField(max_length=250)
content = models.TextField()
pub_date = models.DateTimeField(default=datetime.datetime.now)
slug = models.SlugField(unique=True)
I would like to show you my tests.py file and ask:
Does it make sense?
Am I even testing for the right things?
Are there best practices I'm not following, and you could point me to?
my tests.py (it contains 11 tests):
# -*- coding: utf-8 -*-
from django.test import TestCase
from django.test.client import Client
from django.core.urlresolvers import reverse
import datetime
from someproject.myapp.models import News
class viewTest(TestCase):
def setUp(self):
self.test_title = u'Test title: bąrekść'
self.test_content = u'This is a content 156'
self.test_slug = u'test-title-bareksc'
self.test_pub_date = datetime.datetime.today()
self.test_item = News.objects.create(
title=self.test_title,
content=self.test_content,
slug=self.test_slug,
pub_date=self.test_pub_date,
)
client = Client()
self.response_detail = client.get(self.test_item.get_absolute_url())
self.response_index = client.get(reverse('the-list-view'))
def test_detail_status_code(self):
"""
HTTP status code for the detail view
"""
self.failUnlessEqual(self.response_detail.status_code, 200)
def test_list_status_code(self):
"""
HTTP status code for the list view
"""
self.failUnlessEqual(self.response_index.status_code, 200)
def test_list_numer_of_items(self):
self.failUnlessEqual(len(self.response_index.context['object_list']), 1)
def test_detail_title(self):
self.failUnlessEqual(self.response_detail.context['object'].title, self.test_title)
def test_list_title(self):
self.failUnlessEqual(self.response_index.context['object_list'][0].title, self.test_title)
def test_detail_content(self):
self.failUnlessEqual(self.response_detail.context['object'].content, self.test_content)
def test_list_content(self):
self.failUnlessEqual(self.response_index.context['object_list'][0].content, self.test_content)
def test_detail_slug(self):
self.failUnlessEqual(self.response_detail.context['object'].slug, self.test_slug)
def test_list_slug(self):
self.failUnlessEqual(self.response_index.context['object_list'][0].slug, self.test_slug)
def test_detail_template(self):
self.assertContains(self.response_detail, self.test_title)
self.assertContains(self.response_detail, self.test_content)
def test_list_template(self):
self.assertContains(self.response_index, self.test_title)
I am not perfect in testing but a few thoughts:
Basically you should test every function, method, class, whatever, that you have written by yourself.
This implies that you don't have to test functions, classes, etc. which the framework provides.
That said, a quick check of your test functions:
test_detail_status_code and test_list_status_code:
Ok to check whether you have configured the routing properly or not. Even more important when you provide your own implementation of get_absolute_url().
test_list_numer_of_items:
Ok if a certain number of items should be returned by the view. Not necessary if the number is not important (i.e. arbitrary).
test_detail_template and test_list_template:
Ok to check whether template variables are correctly set.
All the other functions: Not necessary.
What your are basically testing here is whether the ORM worked properly, whether lists work as expected and whether object properties can be accessed (or not). As long as you don't change e.g. the save() method of a model and/or provide your custom logic, I would not test this. You should trust the framework developers that this works properly.
You only should have to test what you have (over)written.
The model classes are maybe a special case. You basically have to test them, as I said, if you provide custom logic. But you should also test them against your requirements. E.g. it could be that a field is not allowed to be null (or that it has to be a certain datatype, like integer). So you should test that storing an object fails, if it has a null value in this field.
This does not test the ORM for correctly following your specification but test that the specification still fulfills your requirements. It might be that you change the model and change some settings (by chance or because you forgot about the requirements).
But you don't have to test e.g. methods like save() or wether you can access a property.
Of course when you use buggy third party code... well things can be different. But as Django uses the test framework itself to verify that everything is working, I would assume it is working.
To sum up:
Test against your requirements, test your own code.
This is only my point of view. Maybe others have better proposals.
Break your tests into two completely separate kinds.
Model tests. Put these in your models.py file with your model. These tests will exercise the methods in your model classes. You can do simple CRUD (Create, Retrieve, Update, Delete) to simply prove that your model works. Don't test every attribute. Do test field defaults and save() rules if you're curious.
For your example, create a TestNews class that creates, gets, updates and deletes a News item. Be sure to test the default date results. This class should be short and to the point. You can, if your application requires it, test various kinds of filter processing. Your unit test code can (and should) provide examples of the "right" way to filter News.
UI Tests. Put these in a separate tests.py file. These tests will test the view functions and templates.
Name the TestCase with the "condition" you're creating. "TestNotLoggedIn". "TestLoggedIn". "TestNoValidThis". "TestNotAllowedToDoThat". Your setUp will do the login and any other steps required to establish the required condition.
Name each test method with the action and result. "test_get_noquery_should_list", "test_post_should_validate_with_errors", "test_get_query_should_detail".