We Keep Coding

How can I fail tests in a "teardown" fixture properly in pytest?

I have a test framework that is performing multiple asserts and catching them (inherited someone else's code). The proprietary results report is correct, but as you may have guessed, pytest will mark these as passed:
test_something.py::TestSomething::test_that_should_fail PASSED
I've added an autouse fixture as such:
#pytest.fixture(autouse=True)
def run_after_tests(self):
yield
if self.actually_failed():
pytest.fail("Yay, failing when failures occur is cool!")
This solution works okay, except that it seems like the clean up happens after the test has already been marked as PASSED and a duplicate test is shown with an error.
Now pytest results look like this:
test_something.py::TestSomething::test_that_should_fail PASSED
test_something.py::TestSomething::test_that_should_fail ERROR
Is there a way to delay the evaluation of the test so it doesn't say it has passed?
I know this is a really stupid way of doing things and performing test evaluation at cleanup is not recommended, but there are too many tests that have been written this way and spending weeks to refactor the test is not feasible.
An alternative solution I've thought of is to write a decorator and then sed all the test cases and add it to the functions; but this is going to be my plan B if fixtures can't solve this.
Thanks!

As you've discovered, a fixture is a separate object from the test itself.
You'll need to modify the appropriate pytest hook. I haven't personally tested, but I believe placing the following code into your projects conftest.py will give you your desired result.
def check_for_failure(output) -> bool:
# define me
#pytest.hookimpl(hookwrapper=True)
def pytest_runtest_call(item):
output = yield
if check_for_failure(output):
pytest.fail()

Pytest class scope parametrization

I have a couple of fixtures that do some initialization that is rather expensive. Some of those fixtures can take parameters, altering their behaviour slightly.
Because these are so expensive, I wanted to do initialisation of them once per test class. However, it does not destroy and reinit the fixtures on the next permutation of parameters.
See this example: https://gist.github.com/vhdirk/3d7bd632c8433eaaa481555a149168c2
I would expect that StuffStub would be a different instance when DBStub is recreated for parameters 'foo' and 'bar'.
Did I misunderstand something? Is this a bug?

I've recently encountered the same problem and wanted to share another solution. In my case the graph of fixtures that required regenerating for each parameter set was very deep and it's not so easy to control. An alternative is to bypass the pytest parametrization system and programmatically generate the test classes like so:
import pytest
import random
def make_test_class(name):
class TestFoo:
#pytest.fixture(scope="class")
def random_int(self):
return random.randint(1, 100)
def test_someting(self, random_int):
assert random_int and name == "foo"
return TestFoo
TestFooGood = make_test_class("foo")
TestFooBad = make_test_class("bar")
TestFooBad2 = make_test_class("wibble")
You can see from this that three tests are run, one passes (where "foo" == "foo") the other two fail, but you can see that the class scope fixtures have been recreated.

This is not a bug. There is no relation between the fixtures so one of them is not going to get called again just because the other one was due to having multiple params.
In your case db is called twice because db_factory that it uses has 2 params. The stuff fixture on the other hand is called only once because stuff_factory has only one item in params.
You should get what you expect if stuff would include db_factory as well without actually using its output (db_factory would not be called more than twice):
#pytest.fixture(scope="class")
def stuff(stuff_factory, db_factory):
return stuff_factory()

difference between fixture and yield_fixture in pytest

I am going through pytest fixtures, and the following looks pretty similar, latest works pretty similar.
Yes, the readability is better in yield_fixure, however could someone let me know what exactly is the difference.
which should I use, in cases like mentioned below?
#pytest.fixture()
def open_browser(request):
print("Browser opened")
def close_browser():
print("browser closed")
request.addfinalizer(close_browser)
return "browser object"
#pytest.yield_fixture()
def open_browser():
print("Browser opened")
yield "browser object"
print("browser closed")
def test_google_search(open_browser):
print(open_browser)
print("test_google_search")

The only difference is in readability. I think (though I'm not 100% sure) the underlying behavior is identical (i.e. the cleanup after the yield statement is run as a finalizer). I always prefer using yield fixtures for cleanup, since it's more readable.
If you're using pytest <3.0, you'll still need to use pytest.yield_fixture to get that behavior. But if you're able to use pytest 3.0+, pytest.yield_fixture is deprecated and you can use pytest.fixture to get the same yield_fixture behavior.
Here are the explanatory docs:
Since pytest-3.0, fixtures using the normal fixture decorator can use
a yield statement to provide fixture values and execute teardown code,
exactly like yield_fixture in previous versions.
Marking functions as yield_fixture is still supported, but deprecated
and should not be used in new code.

addfinalizer has two key differences over yield:
It is possible to register multiple finalizer functions.
Finalizers will always be called regardless if the fixture setup
code raises an exception. This is handy to properly close all
resources created by a fixture even if one of them fails to be
created/acquired
From the pytest docs

How do I find out if the current test is the last to be run?

How can I find out from the current test if its the last to be run? (Python unittest / nosetests)
I have some specific fixture teardown to be done at the very end of the test run and it would be a lot easier if on a test by test basis I could just determine:
if last_test:
hard_fixture_teardown()
else:
soft_fixture_teardown()
I have a package teardown which would work perfectly but it seems very messy passing the fixture information back to the __init__.teardown_package().

You can use a combination of TestCase.tearDown() and TestCase.tearDownClass() to achieve this. tearDown() is called for each test method while tearDownClass() is called after all tests in the class have run.

As stated here unit test are not meant to have an order, unit tests depending on the order are either not well conceived or just have to be merged in a monolithic one. (merging separate functions in a single test is the accepted answer )
[edit after comment]
If the order is not important you can do this (quite messy, imo we are still forcing the boundaries of how unit tests should be used)
in every test package you put:
def tearDownModule():
xxx = int(os.getenv('XXX', '0')) + 1
if xxx == NUMBER_OF_TEST_PACKAGES:
print "hard tear down"
else:
print "not yet"
os.environ['XXX'] = str(xxx)
with NUMBER_OF_TEST_PACKAGES imported from somewhere global.
also if the order is not important I suppose that when used the fixture is only readed and not modified, if so you can setup that as a class method
#classmethod
def setUpClass(cls):
print "I'll take a lot of time here"

python unittest - how to reference a callable function

OK, I know it's going to be obvious, but I cannot work out how to write a test for an internal function. Here's a trivial piece of code to illustrate the problem.
def high(x, y):
def low(x):
return x*2
return y*low(x)
class TestHigh(unittest.TestCase):
def test_high(self):
self.assertEqual(high(1,2),4)
def test_low(self):
self.assertEqual(low(3),6)
results in
Exception: NameError: global name 'low' is not defined
In the "real" case I want to be able to test the lower level function in isolation to make sure all the paths are exercised, which is cumbersome when testing only from the higher level.

low is nested within the high function, so it's not accessible from outside the function. The equivalent for your function would be high(3,1)

You write tests to ensure that the publicly visible interface performs according to its specification. You should not attempt to write tests for internal functionality that is not exposed.
If you cannot fully test low() through the results of high() then the untested parts of low() cannot matter to anything outside.

BAD: Try making a class and adding the functions as methods (or staticfunctions) to it.
(I'll leave this here as a reference for what NOT to do.)
GOOD: Write module level functions or accept that you can't test it if you nest it.