Imports break VSCode testing with pytest - python

I have a project where I want to VS Code's discover tests and other testing features to make testing easier. I have a problem that imports in test files break when I try to discover tests.
I have a file structure like so:
project\
__init__.py
package1\
module1.py
__init__.py
tests\
test.py
__init__.py
In test.py I have a line:
import project.package1.module1 as module1
I run my project by calling python -m project in the root folder, and I am able to run tests successfully by calling python -m pytest project from the root folder.
When I run VS Code's "discover tests" feature or try to step through a file with the debugger, I receive an error 'ModuleNotFoundError: No module named project'.
Does anyone know how to solve this problem?

I had the same issue. The solution that worked for me was to introduce a .envfile that holds my PYTHONPATH entries, relative to my workspace folder.
PYTHONPATH="path1:path2:pathN"
Then I added a line to my workspace settings that specifies the location of my .env file.
// ...
"python.envFile": "${workspaceFolder}/.env",
// ...

I had the same issue where I was able to run pytest and python -m pytest successfully in the terminal within VSCode but the discovery was failing. My solution was to implement the failing import in the following way
import sys
sys.path.insert(0, '/full/path/to/package1/')
from package1.module1 import module1
Note that VSCode was opened with the project folder being the root.

Next solution works for Linux and Windows,
import sys
from pathlib import Path
sys.path.insert(0, str(Path('package1/').resolve()))
It's based on #Chufolon answer. My StackOverflow reputation doesn't allow me to just comment on his answer. I prefer his solution because in the .env there could be sensitive information (passwords, ...) that shouldn't be shared (omit it in .gitignore file) for security reasons; and also because __init__.py is shared by default through Git.

Related

python import packages that know nothing about my system configuration

I have a project I am working on, let's call it Project, which lives in the directory Project somewhere wholly unknown to me (really it lives both on my local system and on a couple Docker build systems). In that project, I have some source files, source/module1.py and source/module2.py. I also have some example files, some test files, and an init.py So my directory looks something like this:
Project
__init__.py
/source
module1.py
module2.py
/test
testRunner.py
/examples
awesomeExample.py
However, module1 needs some stuff from module2. My naive self thought this could be done by putting an import statement in module1:
import module2
# Do some other interesting stuff
And this works, but only when I am running / importing module 1 from the source directory. If I am, for example, running some unit tests in another directory test/testRunner.py, either from the test directory or in the main Project directory, the import will fail. Same with trying to use it when running an example in the examples directory.
So here is my problem: in general, I don't know where the calling script lives. It might be in the examples directory, it might be in the test directory, or it might be in the main Project directory (for example when trying to import stuff with an init.py). How do I ensure that module1 can always import module2 in each of these scenarios?
I am not looking for a solution like "add all those directories to your python path". Initially I just added Project to my python path on my local machine, and then did all my imports relative to that (import Project.source.module2), but this (predictably) caused my builds to fail on the Docker instances. I don't just want this to work on my local machine, but also on the Docker instances I'm using to build and test this software, and on any user's machine that subsequently installs it (i.e. by doing a pip install Project. What is the most robust way to make sure this dependency is satisfied? How can I make sure module1 can import module2 regardless of where module1 itself is imported from? Any python 3.x.x solution is welcome.
I figured out a way to do it (credit here) - it's a little inelegant, but extremely robust. Works on my local machine independent of whether using an import statement or running a script directly, as well as my build servers which use github actions and Travis CI.
Basically, I added a file in the source directory, called context.py with the following contents:
import os
import sys
fileLocation = os.path.dirname(os.path.abspath(__file__))
sourceLocation = os.path.abspath(os.path.join(fileLocation, '..', 'source/'))
sys.path.insert(0, sourceLocation)
This finds out the current file being executed from python, and then uses that to add to the python path. And then in my module1.py file, at the top I have:
import context
import module2
Now, whenever module1 is imported, it successfully imports module2. More elegant answers or comments on why this works and in which cases it might fail are appreciated.

pytest cannot import module while python can

I am working on a package in Python. I use virtualenv. I set the path to the root of the module in a .pth path in my virtualenv, so that I can import modules of the package while developing the code and do testing (Question 1: is it a good way to do?). This works fine (here is an example, this is the behavior I want):
(VEnvTestRc) zz#zz:~/Desktop/GitFolders/rc$ python
Python 2.7.12 (default, Jul 1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from rc import ns
>>> exit()
(VEnvTestRc) zz#zz:~/Desktop/GitFolders/rc$ python tests/test_ns.py
issued command: echo hello
command output: hello
However, if I try to use PyTest, I get some import error messages:
(VEnvTestRc) zz#zz:~/Desktop/GitFolders/rc$ pytest
=========================================== test session starts ============================================
platform linux2 -- Python 2.7.12, pytest-3.0.5, py-1.4.31, pluggy-0.4.0
rootdir: /home/zz/Desktop/GitFolders/rc, inifile:
collected 0 items / 1 errors
================================================== ERRORS ==================================================
________________________________ ERROR collecting tests/test_ns.py ________________________________
ImportError while importing test module '/home/zz/Desktop/GitFolders/rc/tests/test_ns.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_ns.py:2: in <module>
from rc import ns
E ImportError: cannot import name ns
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================= 1 error in 0.09 seconds ==========================================
(VEnvTestRc) zz#zz:~/Desktop/GitFolders/rc$ which pytest
/home/zz/Desktop/VirtualEnvs/VEnvTestRc/bin/pytest
I am a bit puzzled, it looks like this indicates an import error, but Python does it fine so why is there a problem specifically with PyTest? Any suggestion to the reason / remedy (Question 2)? I googled and stack-overflowed the 'ImportError: cannot import' error for PyTest, but the hits I got were related to missing python path and remedy to this, which does not seem to be the problem here. Any suggestions?
Found the answer:
DO NOT put a __init__.py file in a folder containing TESTS if you plan on using pytest. I had one such file, deleting it solved the problem.
This was actually buried in the comments to the second answer of PATH issue with pytest 'ImportError: No module named YadaYadaYada' so I did not see it, hope it gets more visibility here.
I can't say I understand why this works, but I had the same problem and the tests work fine if I run python -m pytest.
I'm in a virtualenv, with pytest also available globally:
(proj)tom#neon ~/dev/proj$ type -a python
python is /home/tom/.virtualenvs/proj/bin/python
python is /usr/bin/python
(proj)tom#neon ~/dev/proj$ python -V
Python 3.5.2
(proj)tom#neon ~/dev/proj$ type -a pytest
pytest is /home/tom/.virtualenvs/proj/bin/pytest
pytest is /usr/bin/pytest
(proj)tom#neon ~/dev/proj$ pytest --version
This is pytest version 3.5.0, imported from /home/tom/.virtualenvs/proj/lib/python3.5/site-packages/pytest.py
I just solved this by removing the __init__.py in my project root:
.
├── __init__.py <--- removed
├── models
│   ├── __init__.py
│   ├── address.py
│   ├── appointment.py
│   └── client.py
├── requirements.txt
├── setup.cfg
├── tests
│   ├── __init__.py
│   ├── models
│   │   ├── __init__.py
│   │   ├── appointment_test.py
│   │   └── client_test.py
│   └── other_test.py
└── script.py
I had the same problem but for another reason than the ones mentioned:
I had py.test installed globally, while the packages were installed in a virtual environment.
The solution was to install pytest in the virtual environment. (In case your shell hashes executables, as Bash does, use hash -r, or use the full path to py.test)
Had a similar issue and it worked when I added __init__.py file under tests directory.
Simply put an empty conftest.py file in the project root directory, because when pytest discovers a conftest.py, it modifies sys.path so it can import stuff from the conftest module.
A general directory structure can be:
Root
├── conftest.py
├── module1
│ ├── __init__.py
│ └── sample.py
└── tests
└── test_sample.py
In my case I am working in a container and unfortuantely pytest has the tendency to use python2.7 rather than my python3 interpreter of choice.
In my case this worked:
python3 -m pytest
My folder structure
/
app/
-module1.py
-module2.py
-tests/
--test_module1.py
--test_module2.py
requirements.txt
README.md
ANOTHER SUGGESTION
I explored this question and various others on SO and elsewhere... all the stuff about adding (or removing) an empty __init__.py in and/or conftest.py in various parts of the project directory structure, all the stuff about PYTHONPATH, etc., etc.: NONE of these solutions worked for me, in what is actually a very simple situation, and which shouldn't be causing any grief.
I regard this as a flaw in pytest's current setup. In fact I got a message recently from someone on SO who clearly knew his stuff. He said that pytest is not designed to work with (as per Java/Groovy/Gradle) separate "src" and "test" directory structures, and that test files should be mingled in amongst the application directories and files. This perhaps goes some way to providing an explanation ... however, tests, particularly integration/functional tests, don't always necessarily correspond neatly to particular directories, and I think pytest should give users more choice in this regard.
Structure of my project:
project_root
src
core
__init__.py
__main__.py
my_other_file.py
tests
basic_tests
test_first_tests.py
The import problem posed: very simply, __main__.py has a line import my_other_file. This (not surprisingly) works OK when I just run the app, i.e. run python src/core from the root directory.
But when I run pytest with a test which imports __main__.py I get
ModuleNotFoundError: No module named 'my_other_file'
on the line in __main__.py that tries to import "my_other_file". Note that the problem here, in my case, is that, in the course of a pytest test, one application file fails to find another application file in the same package.
USING PYTHONPATH
After a lot of experimentation, and putting an __init__.py file and a confest.py file in just about every directory I could find (I think the crucial files were __init__.py added to "tests" and "basic_tests", see above directory structure), and then setting PYTHONPATH to as follows
PYTHONPATH=D:\My Documents\software projects\EclipseWorkspace\my_project\src\core
... I found it worked. Imports in the testing files had to be tweaked a bit, the general pattern being from core import app, project, but the test files were able to "see" core, and crucially there was no need to mess around with the import commands in the app files.
HOWEVER... for some reason the tests now run much slower using this method! Compared to my solution below, where the core package can be seen to be loaded just once, my suspicion is that the PYTHONPATH solution probably results in vast amounts of code being reloaded again and again. I can't yet confirm this, but I can't see any other explanation.
THE ALTERNATIVE
My alternative fairly simple solution is this:
1 - in __init__.py in that application package (i.e. directory "core"), put the following two lines:
import pathlib, sys
sys.path.append(str(pathlib.Path(__file__).parent))
NB normally there isn't anything in an __init__.py of course. It turns out, as I confirmed by experimentation, that pytest usually (see update below) executes __init__.py in this situation, after pytest has done whatever it has done to mess up the sys.path entries.
2 - UPDATE 2022-01:
The original solution I had found involved putting a conftest.py file in the application directory(ies) - without which things didn't work. This is obviously undesirable. I find that another solution is to put this code in your conftest.py file in your root directory:
def pytest_configure(config):
import src.core # NB this causes `src/core/__init__.py` to run
# set up any "aliases" (optional...)
import sys
sys.modules['core'] = sys.modules['src.core']
... indeed, from my experiments, the effect of putting conftest.py in the application directory seems to be that pytest then runs __init__.py in that directory. This appears to imply that the module is being imported...
(previous suggestion:)
yes, you also HAVE to include an empty "conftest.py" file in the directory "core". Hopefully this should be the only conftest.py you'll need: I experimented with all this, and putting one in the root directory was not necessary (nor did it solve the problem without the suggested code in __init__.py).
3 - finally, in my test function, before calling core.__main__ in my example, I have to import the file I know is about to be imported:
import core.my_other_file
import core.__main__
If you do this in the first test in your file, you will find that sys.modules is set up for all other tests in that file. Better yet, put import core.my_other_file at the very start of the file, before the first test. Unfortunately, from core import * does not seem to work.
Later: this method has some idiosyncracies and limitations. For example, although the -k switch works OK to filter in/out tests or entire files, if you do something like pytest tests/tests_concerning_module_x, it appears that core.__init__.py does NOT get run... so the files in the core module are once again mutually unimportable during testing. Other limitations will probably come to light...
As I say, I regard this as a flaw in pytest's setup. I have absolutely no idea what pytest does to establish a common-sense setup for sys.path, but it's obviously getting it wrong. There should be no need to rely on PYTHONPATH, or whatever, and if indeed this is the "official" solution, the documentation on this subject is sorely lacking.
NB this suggestion of mine has a problem: by adding to sys.path every time pytest runs __init__.py in a module, it means that this new path thereafter becomes permanent in sys.path, both during testing and, more worryingly, during runs of the application itself, if there is anything which actually calls __init__.py. (Incidentally, just going python src/core (as in my example) does NOT cause this to happen. But other things might.)
To cater for this I have a clunky but effective solution:
import pathlib, sys, traceback
frame_list = traceback.extract_stack()
if len(frame_list) > 2:
path_parts = pathlib.Path(frame_list[2].filename).parts
if len(path_parts) > 2:
module_dir_path_str = str(pathlib.Path(__file__).parent)
if sys.platform.startswith('win'):
if path_parts[-3:-1] == ('Scripts', 'pytest.exe'):
sys.testing_context = True
sys.path.append(module_dir_path_str)
elif sys.platform.startswith('lin'):
if path_parts[-3:-1] == ('_pytest', 'config'):
sys.testing_context = True
sys.path.append(module_dir_path_str)
This is based on my examination of the stacktrace when pytest runs something, in a W10 context and Linux Mint 20 context. It means that in an application run there will be no messing with sys.path.
Of course, this may break with future versions of pytest. My version is 6.2.5.
This problem will happen if you have a tests.py file and a tests folder with tests/__init__.py.
During the collection pytest finds the folder, but when it tries to import the test files from the folder, tests.py file will cause the import problem.
To fix simply remove the tests.py file and put all your tests inside the tests/ folder.
For your specific case the fix will be precisely:
Remove the file /home/zz/Desktop/GitFolders/rc/tests.py
Make sure /home/zz/Desktop/GitFolders/rc/tests/__init__.py is present
I solved my problem by setting the PYTHONPATH in Environment Variables for the specific configuration I'm running my tests with.
While you're viewing the test file on PyCharm:
Ctrl + Shift + A
Type Edit Configurations
Set your PYTHONPATH under Environment > Environment variables.
UPDATE
Move into your project's directory root
Create a virtual environment
Activate your newly created virtual environment
Set the variable $PYTHONPATH to the root of your project and export it:
export PYTHONPATH=$(pwd)
Do not remove the __init__.py from the tests/ directory or from the src/ directory.
Also note:
The root of your directory is not a python module so do NOT add an __init__.py
conftest.py is not necessary in the root of your project.
The $PYTHONPATH var will only be available during the current terminal/console session; so you will need to set this each time.
(You can follow the steps previous to this update if you are working in pycharm).
I had a similar issue, exact same error, but different cause. I was running the test code just fine, but against an old version of the module. In the previous version of my code one class existed, while the other did not. After updating my code, I should have run the following to install it.
sudo pip install ./ --upgrade
Upon installing the updated module running pytest produced the correct results (because i was using the correct code base).
Please check here: https://docs.pytest.org/en/documentation-restructure/background/pythonpath.html
I has an issue with pytest (that was solved using python -m pytest); the error was
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.9/site-packages/...
I found the problem was missing __init__.py in tests/ and tests/subfolders.
In my case, the import error occurred because the package is pointing to another package/directory with the same name and its path is one level above the folder I actually wanted.
I think this also explains why some people need to remove _ init _.py while others need to add back.
I just put print(the_root_package.__path__) (after import the_root_package) in both python console and pytest scripts to compare the difference
BOTTOM LINE: When you do python, the package you import may be different from the package when you run pytest.
I had placed all my tests in a tests folder and was getting the same error. I solved this by adding an __init__.py in that folder like so:
.
|-- Pipfile
|-- Pipfile.lock
|-- README.md
|-- api
|-- app.py
|-- config.py
|-- migrations
|-- pull_request_template.md
|-- settings.py
`-- tests
|-- __init__.py <------
|-- conftest.py
`-- test_sample.py
Install the packages into Your virtual environment.
Then start a new shell and source Your virtual environment again.
As of pytest 7.0, you can now add pythonpath in pytest.ini. No need to add __init__.py or conftest.py in your root directory.
[pytest]
minversion = 7.0
addopts = --cov=src
pythonpath = src
testpaths =
tests
You can run pytest without any parameters.
https://docs.pytest.org/en/7.0.x/reference/reference.html#confval-pythonpath
Here's a medium article! describing the problem!
The issue is which pytest you are using and your use of a virtual environment.
If you have installed pytest system-wide, in other words, outside of a virtual environment, pytest has a nasty habit of only looking outside your virtual environment for modules! If your project is using a module that is only installed in your virtual environment, and you’re using a system-wide pytest, it won’t find the module, even if you’ve activated the virtual environment.1
Here’s the step-by-step:1
Exit any virtual environment
Use Pip to uninstall pytest
Activate your project’s virtual environment
Install pytest inside the virtual environment
pytest will now find your virtual-environment-only packages!
The answer above not work for me. I just solved it by appending the absolute path of the module which not found to the sys.path at top of the test_xxx.py (your test module), like:
import sys
sys.path.append('path')
I was getting this using VSCode. I have a conda environment. I don't think the VScode python extension could see the updates I was making.
python c:\Users\brig\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\testing_tools\run_adapter.py discover pytest -- -s --cache-clear test
Test Discovery failed:
I had to run pip install ./ --upgrade
I was experiencing this issue today and solved it by calling python -m pytest from the root of my project directory.
Calling pytest from the same location still caused issues.
My Project dir is organized as:
api/
- server/
- tests/
- test_routes.py
- routes/
- routes.py
- app.py
The module routes was imported in my test_routes.py as: from server.routes.routes import Routes
Hope that helps!
I disagree with the posts saying that you must remove any __init__.py files. What you must instead do is alter the sys.path.
Run an experiment where you print sys.path when running the code normally.
Then print sys.path while running the code via pytest. I think you will find that there is a difference between these two paths, hence why pytest breaks.
To fix this, insert the path from the first experiment at the 0th index of the second.
Let '/usr/exampleUser/Documents/foo' be the first element of print(sys.path) for experiment 1.
Below is code that should fix your issue:
import sys
sys.path[0] = '/usr/exampleUser/Documents/foo'
Put this at the top of your file, before your actual import statement.
Source: I was dealing with this myself and the above process solved it.
Another special case:
I had the problem using tox. So my program ran fine, but unittests via tox kept complaining.
After installing packages (needed for the program) you need to additionally specify the packages used in the unittests in the tox.ini
[testenv]
deps =
package1
package2
...
If it is related to python code that was originally developed in python 2.7 and now migrated into python 3.x than the problem is probably related to an import issue.
e.g.
when importing an object from a file: base that is located in the same directory this will work in python 2.x:
from base import MyClass
in python 3.x you should replace with base full path or .base
not doing so will cause the above problem.
so try:
from .base import MyClass
Yet another massive win for Python's import system. I think the reason there is no consensus is that what works probably depends on your environment and the tools you are using on top of it.
I'm using this from VS Code, in the test explorer under Windows in a conda environment, Python 3.8.
The setup I have got to work is:
mypkg/
__init__.py
app.py
view.py
tests/
test_app.py
test_view.py
Under this setup intellisense works and so does test discovery.
Note that I originally tried the following, as recommended here.
src/
mypkg/
__init__.py
app.py
view.py
tests/
test_app.py
test_view.py
I could find no way of getting this to work from VS Code because the src folder just blew the mind of the import system. I can imagine there is a way of getting this to work from the command line. As a relatively new convert to Python programming it gives me a nostalgic feeling of working with COM, but being slightly less fun.
I find the answer in there :Click here
If you have other project structure, place the conftest.py in the package root dir (the one that contains packages but is not a package itself, so does not contain an init.py)
update PYTHONPATH till src folder
export PYTHONPATH=/tmp/pycharm_project_968/src
The TestCase directory must be a Python Package
For anyone who tried everything and still getting error,I have a work around.
In the folder where pytest is installed,go to pytest-env folder.
Open pyvenv.cfg file.
In the file change include-system-site-packages from false to true.
home = /usr/bin
include-system-site-packages = true
version = 3.6.6
Hope it works .Don't forget to up vote.
My 2 cents on this: pytest will fail at chance if you are not using virtual environments. Sometimes it will just work, sometimes not.
Therefore, the solution is:
remove pytest with pip uninstall
create your venv
activate your venv
pip install your project path in editable mode, so it will be treated by pytest as a module (otherwise, pytest wont find your internal imports). You will need a setup.py file for that
install your packages, including pytest
finally, run your tests
The code, using windows PowerShell:
pip uninstall pytest
python.exe -m venv my_env
.\my_env\Scripts\activate
(my_env) pip install -e .
(my_env) pip install pytest pytest-html pandas numpy
Then finally
(my_env) pytest --html="my_testing_report.html"
An example of setup.py, for pip install -e:
import setuptools
setuptools.setup(
name='my_package',
version='devel',
author='erickfis',
author_email='erickfis#gmail.com',
description='My package',
long_description='My gooood package',
packages=setuptools.find_packages(),
classifiers=[
'Programming Language :: Python :: 3',
'Operating System :: OS Independent',
],
include_package_data=True
)
If you run Pytest from a terminal:
Run pytest with the --import-mode=append command-line flag.
Argument description in the official documentation: https://docs.pytest.org/en/stable/pythonpath.html
UPD:
Before I also wrote how to do the same if you use PyCharm, but community does not like extendend answers, so I removed additional information that probably was helpful to someone who have a similar issue.

Why doesn't my current directory show up in the path using pytest on Windows?

I have the following folder structure;
myapp\
myapp\
__init__.py
tests\
test_myapp.py
and my pwd is
C:\Users\wwerner\programming\myapp\
I have the following test setup:
import sys
import pprint
def test_cool():
pprint.pprint(sys.path)
assert False
That produces the following paths:
['C:\\Users\\wwerner\\programming\\myapp\\tests',
'C:\\Users\\wwerner\\programming\\envs\\myapp\\Scripts',
'C:\\Windows\\system32\\python34.zip',
'C:\\Python34\\DLLs',
'C:\\Python34\\lib',
'C:\\Python34',
'C:\\Users\\wwerner\\programming\\envs\\myapp',
'C:\\Users\\wwerner\\programming\\envs\\myapp\\lib\\site-packages']
And when I try to import myapp I get the following error:
ImportError: No module named 'myapp'
So it looks like it's not adding the current directory to my path.
By changing my import line to look like this:
import sys
sys.path.insert(0, '.')
import myapp
I am then able to import myapp with no problems.
Why does my current directory not show up in the path when running pytest? Is my only workaround to insert . into the sys.path? (I'm using Python 3.4 if it matters)
Ahah!
After comparing the layout of my cookiecutter repo, it turns out to be way more simple (and better) than that.
tests/
__init__.py
test_myapp.py
A simple addition of the __init__.py file to my test dir allows me to run py.test from my main directory.
Using an installable package
If you have an installable package (setup.py or pyproject.toml file with a build-system defined) then you want to test against the installed package.
pip install --editable .
pytest
The simplest possible way to make the project shown in the question into an installable package would be by adding this setup.py:
from setuptools import setup
setup(
name="myapp",
version="0.1",
packages=["myapp"],
)
This will put the myapp code at /path/to/myapp/.venv/lib/python3.XY/site-packages, which is in the sys.path of the virtual environment. Now myapp can be imported from the site-packages dir, just as it would be for a user installation. It is neither necessary nor desirable for the current working directory to be present on sys.path during test execution.
Not using an installable package
The project shown in the question does not have any installer, so it can't be installed. It can still be tested by making sure the project root (i.e. the directory which contains both myapp and tests as subdirectories) is present on sys.path.
The best way to do this is to use python -m pytest, rather than invoking the bare pytest command. When you use python -m pytest it adds the current working directory to the start of sys.path. That's the normal Python behavior when executing a package as __main__ (documented here) and it's also a documented usage for pytest - see Invoking pytest versus python -m pytest.
Why does adding an __init__.py to the tests subdirectory (not) work?
The directory structure shown in the question is the "Tests outside application code" pattern, documented here. This is also the directory structure I recommend, since it creates a clear distinction between library/application code and test code.
It's not recommended to add __init__.py files inside the test directories when using a "Tests outside application code" structure, since the test files aren't intended to be "packaged" (e.g. test files do not really need to import from other test files, and they do not need to be installed at all for end users of your package).
The reason adding a myapp/__init__.py actually allows myapp to be imported by pytest, as described in Wayne's answer is actually an accident due to the way test discovery appends sys.path during the test collection phase. This is described as "problematic" in the docs
... this introduces a subtle problem: in order to load the test modules from the tests directory, pytest prepends the root of the repository to sys.path, which adds the side-effect that now mypkg is also importable
They go on to strongly recommend using the src-layout if you intend to have __init__.py files inside test directories, to avoid this confusion of the import system.
But perhaps the best reason not to rely on this side-effect is that pytest collection actually can work in multiple modes (see import modes), and Wayne's answer relies upon pytest using the default "prepend" mode. It is currently mentioned that a future version will switch to "importlib" mode as default:
We intend to make importlib the default in future releases.
The accepted answer does not work with pytest --import-mode=importlib and so will stop working altogether at some stage.
sys.path automatically has the script's directory in it, and not the current working directory.
I am guessing that your script in placed in tests directory. Based on this assumption, your code should look like this:
import sys
import os
ROOT_DIR = os.path.dirname(os.path.dirname(__file__))
sys.path.append(ROOT_DIR)
import myapp # Should work now
Use the environment variable PYTHONPATH.
In Windows:
set PYTHONPATH=.
py.test
In Unix:
PYTHONPATH=. py.test

Py.test No module named *

I have a folder structure like this
App
--App
--app.py
--Docs
--Tests
--test_app.py
In my test_app.py file, I have a line to import my app module. When I run py.test on the root folder, I get this error about no module named app. How should I configure this?
Working with Python 3 and getting the same error on a similar project layout, I solved it by adding an __init__ file to my tests module.
$ touch tests/__init__.py
I'm not great at packaging and importing, but I think that this helps pytest work out where the target App module is located.
I already had an __init__.py file in the /App/App directory and wanted to run tests from the project root without any path-mangling magic:
python -m pytest tests
The output immediately looks like this:
➟ python -m pytest tests
====================================== test session starts ======================================
platform linux -- Python 3.5.1, pytest-2.9.0, py-1.4.31, pluggy-0.3.1
rootdir: /home/andrew/code/app, inifile:
plugins: teamcity-messages-1.17
collected 46 items
... lines omitted ...
============================= 44 passed, 2 skipped in 1.61 seconds ==============================
I had a similar problem and had to delete __init__.py from the root and add an __init__.py to the tests folder.
So you are running py.test from /App. Are you sure /App/App is in your $PYTHONPATH?
If it's not, code that tries to import app will fail with such a message.
EDIT0: including the info from my comment below, for completeness.
An attempt to import app will only succeed if it was executed inside /App/App, which is not the case here. You probably want to make /App/App a package by putting __init__.py inside it, and change your import to qualify app as from App import app.
EDIT1: by request, adding further explanation from my second comment below.
By putting __init__.py inside /App/App, that directory becomes a package. Which means you can import from it, as long as it - the directory - is visible in the $PYTHONPATH. I.e. you can do from App import app if /App is in the $PYTHONPATH. Your current working directory gets automatically added to $PYTHONPATH, so when you run a script from /App, the import will work.
Running pytest with the python -m pytest command helps with this exact thing.
Since your current package is not yet in your $PYTHONPATH or sys.path - pytest gets this error.
By using python -m pytest you automatically add the working directory into sys.path for running pytest. Their documentation also mentions:
This is almost equivalent to invoking the command line script pytest
I also got same error while running test cases for my app located as below
myproject
--app1
--__init.py__
--test.py
--app2
--__init.py__
--test.py
--__init.py__
I deleted my myproject's init.py file to run my test cases.
TL;DR
You might as well add an empty conftest.py file to your root app folder.
(if you take a look at the question folder structure, that would be the same level as the "Tests" folder, not inside of it).
More info:
Pytest looks for conftest.py files inside all your project folders.
conftest.py provides configuration for the file tree pytest finds it in. Because pytest somehow scans all subdirectories starting from conftest.py folder, it should find packages/modules outside the tests folder (as long as a conftest.py file is in your app root folder).
Eventually, you might want to write some code in your empty conftest.py, specially to share fixtures among different tests files, in order to avoid duplicate code. Afterall, the DRY principle (Don't Repeat Yourself) should also be follwed when writing tests.
Adding __init.py__ to the tests folder also should help pytest to find modules throughout your application. However, note that Python 3.3+ has implicit namespace packages that allow it to create a packages without an __init__.py file. That been said, creating __init__.py files for this specific purpose seems more like a a workaround for pytest than a python requirement. More about that in: Is __init__.py not required for packages in Python 3.3+
I got the similar issue. And after trying multiple things including installing pytest on virtual environment and adding/removing __init__.py file from the package, none worked for me.
Solution that worked for me is(windows solution):
Added Project folder(not package folder) to python path(set PYTHONPATH=%PYTHONPATH%;%CD%)
Ran my script from Project Folder and boom, it worked.
I hit the same issue.
my-app
--conf
--my-app
--tests
I set the __init__.py files. I added a conftest.py ( for sharing pytest.fixtures ). I added this to my Poetry file ( pyproject.toml:
[tool.pytest.ini_options]
pythonpath = [
"."
Turned out it was my use of hyphens and not underscores ! Noo...
# pytest can't find Module
--my-app
# works
--my_app
What worked for me: I had to make absolute imports in my test file and call python -m test in the root folder.
This worked for me:
Went to parent app, and pip install -e . (install a local and editable app).

Python import src modules when running tests

My source files are located under src and my test files are located under tests. When I want to run a test file, say python myTest.py, I get an import error: "No module named ASourceModule.py".
How do I import all the modules from source needed to run my tests?
You need to add that directory to the path:
import sys
sys.path.append('../src')
Maybe put this into a module if you are using it a lot.
If you don't want to add the source path to each test file or change your PYTHONPATH, you can use nose to run the tests.
Suppose your directory structure is like this:
project
package
__init__.py
module.py
tests
__init__.py
test_module.py
You should import the module normally in the test_module.py (e.g. from package import module). Then run the tests by running nosetests in the project folder. You can also run specific tests by doing nosetests tests/test_module.py.
The __init__.py in the tests directory is necessary if you want to run the tests from inside it.
You can install nose easily with easy_install or pip:
easy_install nose
or
pip install nose
nose extends unittest in a lot more ways, to learn more about it you can check their website: https://nose.readthedocs.org/en/latest/
On my system (Windows 10), I was required to do something like this:
import sys
import os
sys.path.append(os.path.dirname(os.path.realpath(__file__)) + "/../src")
Appending the relative directory directly to sys.path did not work
The best (most manageable) solution appears to be using a virtualenv and setuptools/distribute to install a development copy of your (src) package. That way your tests execute against a fully "installed" system.
In the pystest docs there is a section on "good practices" explaining this approach, see here.
For those using Pytest:
Make sure src is recognized as a package by putting an empty__init__.py inside.
Put an empty conftest.py in the project folder.
Make sure there is no __init__.py in the test directory.
Looks like this:
project
conftest.py
src
__init__.py
module.py
test
test_module.py
And make sure module.py contains the line:
from src import module
Source: Pytest docs

Categories