I am very new to python, coming from a php background and cant figure out the best way to organise my code.
Currently I am working through the project euler exercises to learn python. I would like to have a directory for my solution to the problem and a directory that mirrors this for tests.
So ideally:
Problem
App
main.py
Tests
maintTest.py
Using php this is very easy as i can just require_once the correct file, or amend the include_path.
How can this be achieved in python? Obviously this is a very simplistic example - therefore some advice on how this is approached on a larger scale would also be extremely grateful.
This depends on which test runner you want to use.
pytest
I recently learned to like pytest.
It has a section about how to organize the code.
If you can not import your main into the code then you can use the tricks below.
unittest
When I use unittest I do it like this:
with import main
Problem
App
main.py
Tests
test_main.py
test_main.py
import sys
import os
import unittest
sys.path.append(os.path.join(os.path.dirname(__file__), 'App'))
import main
# do the tests
if __name__ == '__main__':
unittest.run()
or with import App.main
Problem
App
__init__.py
main.py
Tests
test.py
test_main.py
test.py
import sys
import os
import unittest
sys.path.append(os.path.dirname(__file__))
test_main.py
from test import *
import App.main
# do the tests
if __name__ == '__main__':
unittest.run()
I have always loved nosetests, so here is my solution:
Problem
App
__init__.py
main.py
Tests
__init__.py
tests.py
Then, open the command prompt, CD to /path/to/Problem and type:
nosetests Tests
it will automatically recognize and run the tests. However, read this:
Any python source file, directory or package that matches the
testMatch regular expression (by default: (?:^|[b_.-])[Tt]est) will be
collected as a test (or source for collection of tests).
[...]
Within a test directory or package, any python source file matching testMatch will be examined for test cases. Within a test module, functions and classes whose names match testMatch and TestCase subclasses with any name will be loaded and executed as tests.
This basically means that your tests (both your files and function/methods/classes) have to begin with the "test" or "Test" word.
More on Nosetests usage here: Basic Usage.
Related
We have a number of python components, that are deployed independently of each other (a serverless type environment). I'm struggling to get python imports working in a way that's compatible with this, while also running pytest across all of the components.
With the following file/directory structure:
pipelines
- components
- componentA
__init__.py
main.py
otherfile.py
test_main.py
- componentB
__init__.py
main.py
The resulting file structure on the cloud platform for componentA:
main.py
otherfile.py
(__init__.py appears to be removed by the platform)
In order to import code from otherfile.py in componentA, we write
from otherfile import some_func
which works on the cloud platform and when running main.py directly, but then fails with pytest ("ModuleNotFoundError"). And vice-versa, using
from .otherfile import some_func
or
from componentA.otherfile import some_func
works great for pytest but not for production ("ImportError: attempted relative import with no known parent package").
I realise we could try and fudge something on the server so the function is nested in another folder, but that will be very kludgy to achieve.
Is there a way we can make these two play nicely together?
I am working on developing unittests for a project that has been already completed, however I am having a hard time running my unittests without modifying the original code. The module I am trying to test has other dependencies in the same folder that will not import when the unittests are run. Here is what my directory looks like:
root
|--main_folder
|--module1.py
|--module2.py
|--tests
|--test_module1.py
The original code in module1.py successfully imports module2.py on its own like this: from module2 import Practices where Practices is a function from module2.
The issue I am running into is that in order to run test_module1.py (which I am doing by calling python3 -m unittest from the root directory), I have to modify module1.py itself such that it says: from main_folder.module2 import Practices.
If I run the test file without modifying module1.py, I get the error ModuleNotFoundError: No module named 'module2'.
Ideally I cannot modify the code in this way, and I am trying to find a way to make my tests work without touching the application itself. How should I go about this? module1.py runs normally when I run the application without modifying the file, however modifying it so that the tests work breaks the main application. What can I do to make my tests independent of the code for the main app?
(For some more background, the test_module1.py file works by calling from main_folder.module1 import fun1 where fun1 is the function I am trying to test)
Try running your tests using one of the following commands (replacting the actual paths):
if your tests import the modules "from main_folder import ..."
env PYTHONPATH=/root python3 -m unittest
or if your tests import directly "import module1":
env PYTHONPATH=/root/main_folder python3 -m unittest
As a side note, you might need to have existing
main_folder/__init__.py
file, to get the main_folder recognized as package, depending of the python version you're using. If you currently don't have such file, try creating it (empty, no need to put code inside it) and check if the issue persists.
I am writing a simple package in python. In the root directory, my structure currently is,
my_package
init.py
main_code.py
tests
init.py
test_main_code.py
I want to import all functions from main_code.py in test_main_code.py. To achieve this, I did
import sys
import os
sys.path.append(os.path.dirname(os.path.realpath(__file__)) + '\..' + '\my_package')
from main_code import *
This runs smoothly on my system. But when I push the code to github and tests are run on travis-ci, all tests fail and return NameError. It says given function name not defined. How do I make sure that tests run on travis-ci as well ?
Paths could be different if you run the test somewhere else, so I wouldn't trust any import related to a particular location.
In any case, I always try to run the tests from the root of the project.
Then, you can simply try:
from my_package.main_code import *
I am struggling to successfully import my project to the test-suite in my project, as well as being able to run the program from the command-line. I've been able to run my test-suite for some time, under the impression that if the tests work, so does the command-line stuff--evidently this isn't the case. I do not yet intend on using my program as a library. The api.py acts is the entry-point for the program.
I have a project with the following structure (the same directory hierarchy as requests):
myapp/
myapp/
__init__.py
api.py # depends on commands.py
commands.py # depends on utils.py
utils.py
tests/
context.py
test_api.py # depends on api.py
test_commands.py # depends on commands.py, utils.py
In the file context.py I have a path modification adding myapp to the PYTHONPATH, so I can successfully run the tests on my code. Here is the contents of that file
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
import myapp
I've tried imaginable import combination I can think of. Far too many to list! I have also perused the Python reference import system page, and this tutorial.
How should I import my dependencies?
Turns out this was the correct layout, I mistook the error for something else. Although for future reference, relative imports in Python 3 must be explicit: when in the myapp package directory you can't say import commands, you must instead import it as from . import commands. This was defined in PEP 328 also see this SO post on the topic. Run your package with python -m mutil.api not python ./mutil/api.py as the latter won't give the interpreter context of the current path.
I have several test modules saved in \tests. I then load them in the main class prior to running the main program by specifying
from tests.ClassTests1 import *
from tests.ClassTests2 import *
...
unittest.main()
Is there any way to instruct unittest.main() to run all the files in \tests without going through endless repetition of importing them as above? E.g. I tried from tests import * and it does not work.
EDIT: I am after the programmatic discovery not the discover via the command-line.
Many thanks!
Thanks goes to John Gordon for his links. Here's what made it work for me:
from tests.ClassTests1 import *
# Run test cases first
suite = unittest.TestLoader().discover('tests', pattern='ClassTests*.py')
result = unittest.TextTestRunner(verbosity=2).run(suite)
However, I still need to import at least one test module, and I'm not sure if I'm doing this right as it discovers the entire test hierarchy of tests.
From https://docs.python.org/2/library/unittest.html, section 25.3.3. Test Discovery:
Unittest supports simple test discovery. In order to be compatible with test discovery, all of the test files must be modules or packages importable from the top-level directory of the project (this means that their filenames must be valid identifiers).
Test discovery is implemented in TestLoader.discover(), but can also be used from the command line. The basic command-line usage is:
cd project_directory
python -m unittest discover