We Keep Coding

Monkey patching env variables for pytest

I have an app.py file with variables being loaded from my template.yaml file.
KINESIS_VALID = os.environ['KINESIS_VALID']
KINESIS_INVALID = os.environ['KINESIS_INVALID']
I've created a conftest.py file in my tests directory and set a function to monkeypatch test variables:
import pytest
#pytest.fixture(autouse=True)
def env_setup(monkeypatch):
monkeypatch.setenv('KINESIS_VALID', 'kinesis-valid-test')
monkeypatch.setenv('KINESIS_INVALID', 'kinesis-invalid-test)
My test file is in tests/unit/test_handler.py and the conftest.py is in the same directory as test_handler.py.
When I run the test, I'm getting KeyyError on KINESIS_VALID from where it's being defined in app.py (KINESIS_VALID = os.environ['KINESIS_VALID'])
The variables are being defined in app.py and I added a conftest.py to the directory app.py is in but the same error occurs. Not sure if I'm missing something in configuration or I'm not defining either the test variables or app.py variables correctly.

Import path in pytest

I have Python Flask project with following structure of my project (I removed uneccesary things):
server
src
service
__init__.py
User.py
tests
pytest.ini
service
test_user.py
Where pytest.ini contains:
[pytest]
python_paths = ../src
And test_user.py:
from service.User import UserService
def test_empty_db():
service = UserService()
users = service.get_all() # Get all users from DB.
assert len(users) = 0 # There should be users.
Now, I would like to run this test. When I run pytest or pytest server from root of project, everything is ok. However, when I want to run the specified file pytest server/tests/service/test_user.py error appears:
from services.User import UserService
E ModuleNotFoundError: No module named 'service'
Is there any way to fix it?

You can try to make your imports more absolute inside your test_user.py like: from server.src.service.User import …
Or you can also try adding a __init__.py to your tests directory.

Your error message is showing services.User while you code is showing service.User, could it be a typo?

pytest return ModuleNotFoundError when module imported in test file import another module within the same directory of the imported module

I am sorry if the title takes some time to understand. So here is the folder structure:
falcon_tut/
falcon_tut/
app.py
images.py
__init__.py
tests/
test_app.py
__init__.py
And some codes
####################
# app.py
####################
from images import Resource
images = Resource()
api = application = falcon.API()
api.add_route('/images', images)
# ... few more codes
####################
# test_app.py
####################
import falcon
from falcon import testing
import ujson
import pytest
from falcon_tut.app import api
#pytest.fixture
def client():
return testing.TestClient(api)
def test_list_images(client):
doc = {
'images': [
{
'href': '/images/1eaf6ef1-7f2d-4ecc-a8d5-6e8adba7cc0e.png'
}
]
}
response = client.simulate_get('/images')
result_doc = ujson.loads(response.content)
assert result_doc == doc
assert response.status == falcon.HTTP_OK
It works fine when running with python falcon_tut/app.py and curl it with response of 200 and payload of the images
Until running pytest tests/ from the project root it output this:
ImportError while importing test module ../falcon_tut/tests/test_app.py
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_app.py:6: in <module>
from falcon_tut.app import api
E ModuleNotFoundError: No module named 'falcon_tut'
I tried creating __init__.py at the project root but it still output the same error above
Python version 3.7.0, with falcon 1.4.1, cpython 0.28.5, pytest 3.7.3, and instead of using gunicorn, I am using bjoern 2.2.2
I am trying out the python falcon framework and encounter the error at the testing part.
==========UPDATE===========
The reason why pytest could not found the module is because sys.path does not have ../falcon_tut/falcon_tut exists.
When I ran pytest and edit those 2 files and printing out sys.path, it only has [../falcon_tut/tests, ../falcon_tut, ..]. The workaround to this is to append the path to the package to sys.path. So here is the edited app.py
#############
# app.py
#############
import sys
# this line is just example, please rewrite this properly if you wants to use this workaround
# sys_path[1] only applied to my situation, again this is just example to know that it works
# the idea is to make sure the path to your module exists in sys.path
# in this case, I appended ../falcon_tut/falcon_tut to sys.path
# so that now ../falcon_tut/falcon_tut/images.py can be found by pytest
sys.path.insert(0, '{}/falcon_tut'.format(sys_path[1]))
# body codes...

How do you properly integrate unit tests for file parsing with pytest?

I'm trying to test file parsing with pytest. I have a directory tree that looks something like this for my project:
project
project/
cool_code.py
setup.py
setup.cfg
test/
test_read_files.py
test_files/
data_file1.txt
data_file2.txt
My setup.py file looks something like this:
from setuptools import setup
setup(
name = 'project',
description = 'The coolest project ever!',
setup_requires = ['pytest-runner'],
tests_require = ['pytest'],
)
My setup.cfg file looks something like this:
[aliases]
test=pytest
I've written several unit tests with pytest to verify that files are properly read. They work fine when I run pytest from within the "test" directory. However, if I execute any of the following from my project directory, the tests fail because they cannot find data files in test_files:
>> py.test
>> python setup.py pytest
The test seems to be sensitive to the directory from which pytest is executed.
How can I get pytest unit tests to discover the files in "data_files" for parsing when I call it from either the test directory or the project root directory?

One solution is to define a rootdir fixture with the path to the test directory, and reference all data files relative to this. This can be done by creating a test/conftest.py (if not already created) with some code like this:
import os
import pytest
#pytest.fixture
def rootdir():
return os.path.dirname(os.path.abspath(__file__))
Then use os.path.join in your tests to get absolute paths to test files:
import os
def test_read_favorite_color(rootdir):
test_file = os.path.join(rootdir, 'test_files/favorite_color.csv')
data = read_favorite_color(test_file)
# ...

One solution is to try multiple paths to find the files.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from coolprogram import *
import os
def test_file_locations():
"""Possible locations where test data could be found."""
return(['./test_files',
'./tests/test_files',
])
def find_file(filename):
""" Searches for a data file to use in tests """
for location in test_file_locations():
filepath = os.path.join(location, filename)
if os.path.exists(filepath):
return(filepath)
raise IOError('Could not find test file.')
def test_read_favorite_color():
""" Test that favorite color is read properly """
filename = 'favorite_color.csv'
test_file = find_file(filename)
data = read_favorite_color(test_file)
assert(data['first_name'][1] == 'King')
assert(data['last_name'][1] == 'Arthur')
assert(data['correct_answers'][1] == 2)
assert(data['cross_bridge'][1] == True)
assert(data['favorite_color'][1] == 'green')

One way is to pass a dictionary of command name and custom command class to cmdclass argument of setup function.
Another way is like here, posted it here for quick reference.
pytest-runner will install itself on every invocation of setup.py. In some cases, this causes delays for invocations of setup.py that will never invoke pytest-runner. To help avoid this contingency, consider requiring pytest-runner only when pytest is invoked:
pytest = {'pytest', 'test', 'ptr'}.intersection(sys.argv)
pytest_runner = ['pytest-runner'] if needs_pytest else []
# ...
setup(
#...
setup_requires=[
#... (other setup requirements)
] + pytest_runner,
)
Make sure all the data you read in your test module is relative to the location of setup.py directory.
In OP's case data file path would be test/test_files/data_file1.txt,
I made a project with same structure and read the data_file1.txt with some text in it and it works for me.

Optimal file structure organization of Python module unittests?

Sadly I observed that there are are too many ways to keep your unittest in Python and they are not usually well documented.
I am looking for an "ultimate" structure, one would accomplish most of the below requirements:
be discoverable by test frameworks, including:
pytest
nosetests
tox
the tests should be outside the module files and in another directory than the module itself (maintenance), probably in a tests/ directory at package level.
it should be possible to just execute a test file (the test must be able to know where is the module that is supposed to test)
Please provide a sample test file that does a fake test, specify filename and directory.

Here's the approach I've been using:
Directory structure
# All __init__.py files are empty in this example.
app
package_a
__init__.py
module_a.py
package_b
__init__.py
module_b.py
test
__init__.py
test_app.py
__init__.py
main.py
main.py
# This is the application's front-end.
#
# The import will succeed if Python can find the `app` package, which
# will occur if the parent directory of app/ is in sys.path, either
# because the user is running the script from within that parect directory
# or because the user has included the parent directory in the PYTHONPATH
# environment variable.
from app.package_a.module_a import aaa
print aaa(123, 456)
module_a.py
# We can import a sibling module like this.
from app.package_b.module_b import bbb
def aaa(s, t):
return '{0} {1}'.format(s, bbb(t))
# We can also run module_a.py directly, using Python's -m option, which
# allows you to run a module like a script.
#
# python -m app.package_a.module_a
if __name__ == '__main__':
print aaa(111, 222)
print bbb(333)
module_b.py
def bbb(s):
return s + 1
test_app.py
import unittest
# From the point of view of testing code, our working modules
# are siblings. Imports work accordingly, as seen in module_a.
from app.package_a.module_a import aaa
from app.package_a.module_a import bbb
class TestApp(unittest.TestCase):
def test_aaa(self):
self.assertEqual(aaa(77, 88), '77 89')
def test_bbb(self):
self.assertEqual(bbb(99), 100)
# Simiarly, we can run our test modules directly as scripts using the -m option,
# or using nose.
#
# python -m app.test.test_app
# nosetests app/test/test_app.py
if __name__ == '__main__':
unittest.main()