Pytest test discovery is failing. The UI states:
Test discovery error, please check the configuration settings for the tests
The output window states:
Test Discovery failed:
Error: Traceback (most recent call last):
File "C:\Users\mikep\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\run_adapter.py", line 16, in <module>
main(tool, cmd, subargs, toolargs)
File "C:\Users\mikep\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter\__main__.py", line 90, in main
parents, result = run(toolargs, **subargs)
File "C:\Users\mikep\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter\pytest.py", line 43, in discover
raise Exception('pytest discovery failed (exit code {})'.format(ec))
Exception: pytest discovery failed (exit code 3)
Here are my settings:
{
"python.pythonPath": ".venv\\Scripts\\python.exe",
"python.testing.pyTestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pyTestEnabled": true
}
I can run pytest from the command line successfully FWIW.
I spent ages trying to decipher this unhelpful error after creating a test that had import errors. Verify that your test suite can actually be executed before doing any deeper troubleshooting.
pytest --collect-only is your friend.
This is not a complete answer as I do not know why this is happening and may not relate to your problem, depending how you have your tests structured.
I resolved this issue by putting an __init__.py file in my tests folder
E.G.:
├───.vscode
│ settings.json
│
├───app
│ myapp.py
│
└───tests
test_myapp.py
__init__.py
this was working a few days ago without this but the python extension was recently updated. I am not sure if this is the intended behavior or a side effect of how discoveries are now being made
https://github.com/Microsoft/vscode-python/blob/master/CHANGELOG.md
Use Python code for discovery of tests when using pytest. (#4795)
I just thought I would add my answer here as this might well affect someone who uses a .env file for their project's environment settings since it is such a common configuration for 12 factor apps.
My example assumes that you're using pipenv for your virtual environment management and that you have a .env file at the project's root directory.
My vscode workspace settings json file looks like below. The crucial line for me here was "python.envFile": "${workspaceFolder}/.env",
{
"python.pythonPath": ".venv/bin/python",
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.linting.pycodestyleEnabled": false,
"python.linting.flake8Enabled": false,
"python.linting.pylintPath": ".venv/bin/pylint",
"python.linting.pylintArgs": [
"--load-plugins=pylint_django",
],
"python.formatting.provider": "black",
"python.formatting.blackArgs": [
"--line-length",
"100"
],
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestEnabled": true,
"python.testing.pytestPath": ".venv/bin/pytest",
"python.envFile": "${workspaceFolder}/.env",
"python.testing.pytestArgs": [
"--no-cov"
],
}
I hope this saves someone the time I spent figuring this out.
In my case the problem with vscode being unable to discover tests was the coverage module being enabled in the setup.cfg (alternatively this could be a pytest.ini), i.e.
addopts= --cov <path> -ra
which caused the test discovery to fail due to low coverage. The solution was to remove that line from the config file.
Also, as suggested in the documentation:
You can also configure testing manually by setting one and only one of the following settings to true: python.testing.unittestEnabled, python.testing.pytestEnabled, and python.testing.nosetestsEnabled.
In settings.json you can also disable coverage using --no-cov flag:
"python.testing.pytestArgs": ["--no-cov"],
EDIT:
Slightly related - in case of more complex projects it might be also necessary to change the rootdir parameter (inside your settings.json) when running tests with pytest (in case of ModuleNotFoundError):
"python.testing.pytestArgs": [
"--rootdir","${workspaceFolder}/<path-to-directory-with-tests>"
],
Seems like a bug in the latest version of VS Code Python extension. I had the same issue, then I downgraded the Python extension to 2019.3.6558 and then it works again. So we should go to our VS Code extensions list, select the Python extension and "Install another version..." from the setting of that extension.
I hope this works for you too.
I resolved the issue by upgrading pytest to the latest version: 4.4.1 with "pip install --upgrade pytest". I was apparently running an old version 3.4.2
Before begin the tests discovery, check that python.testing.cwd points correctly to your tests dir and your python.testing.pytestEnabled is set to true.
Once those requirements are set correctly, run tests discovery and its output (see OUTPUT window). You should see something like this:
python $HOME/.vscode/extensions/ms-python.python-$VERSION/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir $ROOT_DIR_OF_YOUR_PROJECT -s --cache-clear
Test Discovery failed:
Error: ============================= test session starts ==============================
<SETTINGS RELATED TO YOUR MACHINE, PYTHON VERSION, PYTEST VERSION,...>
collected N items / M errors
...
It's important to highlight here the last line: collected N items / M errors. The following lines will contain info about the tests discovered by pytest. So your tests are discoverable but there are errors related to their correct execution.
The following lines will contain the errors which in most of the cases will be related to an incorrect import.
Check that all your dependencies have been downloaded previously. If you are working with specific versions of dependencies (on therequirements.txt file you have something like your_package == X.Y.Z) be sure that it's the right version that you need to.
If you are having trouble with pytest I was struggling with the discovery test part.
Reading some open issue (other) in vscode I found a workaround using the test-adapter extension
market_place_link
The extention works like a charm. (and solve my discovery problem)
In my case, it was the vscode python extension problem.
I switched the test platform from pytest and switched to it back again, and the tests got discovered.
It seems that when testing is universally enabled for all python projects and it fails to discover tests at the beginning, it fails forever!
Test files need to be named test_*.py or *_test.py for pytest collection to work.
Then run pytest --collect-only at the command line to make sure all of the tests are found. Then magically, the flask icon in VSCode suddenly shows the test files and their tests.
As a noob, I was putting my unit tests inside the module files, since I was following the pattern of unittest, and then running pytest *.py. That doesn't work with pytest, and I didn't find a command line argument override the naming convention.
I searched in the settings for "python" and found this:
Switching to pytest automatically detect my tests:
This error is so frustrating..
In my case the error fixed by modifying the python.pythonPath parameter in settings.json (found inside .vscode folder under project root directory), to the path which I obtained using which python in terminal (e.g. /usr/local/var/pyenv/shims/python)
I use pyenv with python3.9, and my error said previously:
Error: Process returned an error: /Users/"user-name"/.pyenv/shims/python:
line 21: /usr/local/Cellar/pyenv/1.2.23/libexec/pyenv: No such file or
directory
at ChildProcess.
(/Users/"user-name"/.vscode/extensions/littlefoxteam.vscode-python-test-adapter-0.6.8/out/src/processRunner.js:35:36)
at Object.onceWrapper (events.js:422:26) at ChildProcess.emit
(events.js:315:20) at maybeClose (internal/child_process.js:1021:16)
at Process.ChildProcess._handle.onexit
(internal/child_process.js:286:5)
In my case, I had to make sure that I was using the right interpreter where the libraries where installed (I used a virtual environment) but the interpreter was pointing to the globally installed python. After changing the interpreter to that of the virtual environment, it worked.
Looking at https://docs.pytest.org/en/latest/usage.html#possible-exit-codes it seems pytest itself is falling over do to some internal error.
I had this same issue and tracked it down to my pytest.ini file. Removing most of the addopts from that file fixed the issue.
I had this problem and struggle with it for hours. I think that there is a special resolution for every other platform configuration. My platform is:
VSCode: 1.55.0 (Ubuntu)
Pytest: 6.2.3
MS Python extension (ms-python.python 2021.3.680753044)
Python Test Explorer for Visual Studio Code (littlefoxteam.vscode-python-test-adapter - 0.6.7)
The worst thing has that the tool itself does not have a standard output (at least that I know of or could find easily on the internet).
In the end, the problem was
--no-cov
parameters that was not recognized by the VSCode testing explorer tool (that I copied from some page on the internet) and the error was showed by the extension littlefoxteam.vscode-python-test-adapter and it may help you to find where things are broken.
In my case, same problem appeared each time when flake8 linter reported errors. Even one error was enough to fail VS Code test discovery.
So the fix is either disable the linter or fix linter errors.
I use setup described here.
2021-12-22
Please find below the settings I used to get pytest working in VsCode after much frustration. I found many helpful pieces of advice here and elsewhere on the internet, but none were complete enough to spare me a bit of cursing. I hope the following will help someone out. This setup allows me to run testing visually from the test explorer extension and also from the integrated terminal. I am using a src format in my workspace and Conda for environment management. The settings related to the terminal setup keep me from having to manually enable my Conda environment or set the python path. Possibly people who have been using VSCODE for more than two days could add something nice to make this better and/or more complete.
##### Vscode info:
Version: 1.63.2 (Universal)
Commit: 899d46d82c4c95423fb7e10e68eba52050e30ba3
Date: 2021-12-15T09:37:28.172Z (1 wk ago)
Electron: 13.5.2
Chromium: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Darwin x64 20.6.0
##### Testing Extension:
Python Test Explorer for Visual Studio Code
extension installed, v. 0.7.0
##### Pytest version
pytest 6.2.5
##### Directory Structure:
workspace
.env
./src
__init__.py
code1.py
code2.py
./tests
__init__.py
test_code1.py
test_code2.py
##### .env file (in root, but see the "python.envFile" setting in settings.json)
PYTHONPATH=src
#####. settings.json
{
"workbench.colorTheme": "Visual Studio Dark",
"editor.fontFamily": " monospace, Menlo, Monaco, Courier New",
"python.testing.unittestEnabled": false,
"python.testing.cwd": ".",
"terminal.integrated.inheritEnv": true,
"python.envFile": "${workspaceFolder}/.env",
"python.defaultInterpreterPath":
"~/anaconda3/envs/mycurrentenv/bin/python",
"pythonTestExplorer.testFramework": "pytest",
"python.testing.pytestEnabled": true,
"python.testing.pytestArgs": [
"tests"
],
"python.terminal.activateEnvironment": true,
"python.terminal.activateEnvInCurrentTerminal": true,
"terminal.integrated.env.osx": {
"PYTHONPATH": "${workspaceFolder}/src:${env:PYTHONPATH}"
}
}
Here is generic way to get Django tests to run with full vscode support
Configure python tests
Choose unittest
Root Directory
test*.py
Then each test case will need to look like the following:
from django.test import TestCase
class views(TestCase):
#classmethod
def setUpClass(cls):
import django
django.setup()
def test_something(self,):
from user.model import something
...
Any functions you want to import have to be imported inside the test case (like shown). The setUpClass runs before the test class is setup and will setup your django project. Once it's setup you can import functions inside the test methods. If you try to import models/views at the top of your script, it will raise an exception since django isn't setup. If you have any other preinitialization that needs to run for your django project to work, run it inside setUpClass. Pytest might work the same way, I haven't tested it.
There are several causes for this.
The simple answer is, it is not perfect. Furthermore, Python is not native to VS Code; Python extensions may not be playing nice either.
I have observed some mitigating measures, though.
NB VS Code didn't "design" test discovery; that is the job of the testing framework. Make sure you have a basic grasp on how it works.
Troubleshooting
For every "battery" of tests, VS Code will gladly tell you what went wrong, right in the Test view panel. To see this, expand the bar with the name of the project, and hover your mouse over the line that is revealed. Look at the traceback.
In the image above, the error is "No module named src". This is more interesting than it sounds. If I import a file from another file, that import path may not be visible to the test discovery mechanism. It may be running from a different "cwd". The best you can do is try to figure out the topmost path of your project, and either add or remove path qualifiers. Until it works.
Main causes
From time to time, VS Code will lose info on the project test configuration. Use command window (Ctrl + Shift + P) to either:
(re)scan tests
(re)configure test spec for the project <-- important!
restart Python Language Server
Python is supposed to remove the need for the empty __init__.py; it seems VS Code loves those. They should be in each folder that leads to a test folder. Not every folder, but the top-down path.
Depending on what you select as the "root test folder" in the test configuration, it might have different meaning based on what VS Code thinks is the root of your project. That probably goes for any specific folder too.
import. VS Code doesn't like syntax errors. It is possible some syntax errors will not get highlighted in the code.
This (unfortunately) goes for all the imports in your file. But you shouldn't test invalid code anyway, right?
Minor buggy behaviors
Running some other visible test might help refresh the discovery process.
VS Code should automatically (by setting) refresh tests on each Save operation. But it doesn't hurt to refresh it manually.
TLDR
Look at the error items in the test panel
Re-configure project test discovery parameters from time to time
Make sure you don't have syntax errors (visible or not)
Create empty __init__.pys in each folder of your project that leads to the test folder
Clean up your import logic
P.S. The Test view has been extensively worked on and improved much over the course of 1 year. Expect changes in behavior.
Related
I have been having issues with MyPy in VSCode. I usually have my venv activated, Pylance as the language server and the MyPy extension (to perform check on editor change event).
MyPy.
MyPy is installed on the current venv, is enabled, linting in enabled, and mypy is using the active interpreter, in my settings.json I have:
{
"python.languageServer": "Pylance",
"python.linting.enabled": true,
"python.linting.mypyEnabled": true,
"mypy.runUsingActiveInterpreter": true
}
MyPy does complain about "library stubs" not found, for packages that are not typed, it also points to some errors, but some very explicit ones are not being shown.
An example:
# file_a.py
def myfunc() -> dict:
return {}
# file_b.py
from file_a import myfunc
from file_z import func_dont_exist
# No error shown here
abc: bool = myfunc()
xyz: str = func_dont_exist()
# This points to an error, highlight the "1"
abc = "abc" + 1
If I run mypy . on the root, also no errors. I have mypy.ini file on the root, ignoring the migrations folder, .venv and usiing SQLMyPy Plugin.
If I disable the ignore_errors mypy point to a bunch of errors on the migration folders (expected), which tells me that the daemon is working.
I have updated my VSCode, tried to disable every extension I have, reloaded, restarted, checked the console, and nothing.
Running out of ideas on how to go by debugging this issue, every single question I find goes in the lines of: enable mypy on the settings, check the path, enable linting etc, but I've already done all of that.
Any ideas?
I had trouble with mypy in the past, like loading up mypy plugin. It was resolved when I started vscode from terminal with the right python environment activated
conda activate my-env
cd my/vs-code/project/dir
code .
Worth a try
I have PyTest setup in vs-code but none of the tests are being found even though running pytest from the command line works fine.
(I'm developing a Django app on Win10 using MiniConda and a Python 3.6.6 virtual env. VS Code is fully updated and I have the Python and Debugger for Chrome extensions installed)
Pytest.ini:
[pytest]
DJANGO_SETTINGS_MODULE = callsign.settings
python_files = tests.py test_*.py *_tests.py
vs-code workspace settings:
{
"folders": [
{
"path": "."
}
],
"settings": {
"python.pythonPath": "C:\\ProgramData\\Miniconda3\\envs\\callsign\\python.exe",
"python.unitTest.unittestEnabled": false,
"python.unitTest.nosetestsEnabled": false,
"python.unitTest.pyTestEnabled": true,
"python.unitTest.pyTestArgs": ["--rootdir=.\\callsign", "--verbose"]
}
}
Finally, the output from the Python Test Log inside VS code:
============================= test session starts =============================
platform win32 -- Python 3.6.6, pytest-4.1.1, py-1.7.0, pluggy-0.8.1
Django settings: callsign.settings (from ini file)
rootdir: c:\Users\benhe\Projects\CallsignCopilot\callsign, inifile: pytest.ini
plugins: django-3.4.5
collected 23 items
<Package c:\Users\benhe\Projects\CallsignCopilot\callsign\transcription>
<Module test_utils.py>
<Function test_n_digits>
<Function test_n_alpha>
<Function test_n_hex>
<Function test_n_digits_in_range>
<Function test_v1_audiofilename>
<Function test_v2_audiofilename>
<Function test_v1_bad_int_filename>
<Function test_v1_bad_non_int_filename>
<Function test_bad_format>
<Function test_no_format>
<Function test_too_many_segments>
<Function test_too_few_segments>
<Function test_good_v2_filename>
<Function test_bad_year_v2_filename>
<Function test_bad_month_v2_filename>
<Function test_bad_day_v2_filename>
<Function test_bad_date_v2_filename>
<Function test_bad_short_serial_v2_filename>
<Function test_bad_long_serial_v2_filename>
<Function test_good_v3_filename>
<Function test_good_lowercase_block_v3_filename>
<Function test_bad_non_alpha_block_v3_filename>
<Function test_real_filenames>
======================== no tests ran in 1.12 seconds =========================
Am I missing any steps to get vs-code to find the tests?
If anyone comes across this post-2020, this issue in the vscode-python repo saved my life. Basically, just do the following:
Uninstall the Python extension
Delete the file that contains the extension from your ~/.vscode folder (mine looked like ms-python.python-[YEAR].[MONTH].[VERSION])
Reinstall the extension
Worked like a charm.
EDIT: I downgraded to Pytest 4.0.1 after reading issue 3911 and Test Discovery now works.
Me too. When I blow away .pytest_cache and rerun Python: Discover Unit Tests, I see that the freshly generated .pytest_cache/v/cache/nodeids contains all the tests, but I still get the dialog complaining about No tests discovered.
Python 3.7.2
macOS 10.13.6
VS Code 1.30.2
Python Extension 2018.12.1
Pytest 4.1.0
.vscode/settings.json:
{
"python.linting.enabled": false,
"python.unitTest.unittestEnabled": false,
"python.unitTest.nosetestsEnabled": false,
"python.unitTest.pyTestEnabled": true,
"python.pythonPath": "venv3/bin/python"
}
Tests are in a top-level subdirectory called test. Running pytest manually works.
Another thing to check, if vscode fails to discover the tests, is to make sure it doesn't do so because of the coverage module being enabled. In my case the test cases were being discovered correctly but the discovery eventually kept failing due to low test coverage, as described here. So first, make sure that the tests could actually be collected by pytest as suggested here:
pytest --collect-only
and then make sure you're not forcing coverage check (e.g. in setup.cfg) with e.g.
addopts= --cov <path> -ra
In September 2021, I was able to get VS code to find the test directory again by downgrading the VS Code Python extension from version 2021.9.1191016588 to version v2021.8.1159798656. To downgrade, right-click the extension and click "Install another version..."
The following steps worked for me (assuming latest PyTest installed in Visual Studio Code):
Make sure there are no errors (bottom left of Visual Studio Code)
In Visual Studio Code from File menu choose: File > Close Folder (i.e. close current project folder) and close Visual Studio Code.
Re-open Visual Studio Code and choose: File > Open Folder (re-open the folder that contains your project/test files).
Choose Testing and should have visibility of tests from here ...
VScode needs to specify python configure:
Open Command Palette by Ctrl + shift + p
and write this >python: configure tests And follow the program directions
Note: You may need (Mostly you will not need) to do this first before following the previous steps
My Python plugin and Test Explorer works just fine.
In my case naming the class without test_.py was the issue. In other words, naming the test file without it starting with "test_", not "tests_", makes it so the explorer does not see the issue. Ex: test_.py works but tests_.py, 12345abcde.py don't work.
An obvious thing that caught me...
The test folder hierarchy needs __init__.py files in each folder.
I refactored my tests introducing an extra folder layer and forgot to add __init__.py files.
Command line didn't complain but vscode showed no tests found
In my case the problem was that I had a syntax error in a source file which caused the pytest discovery to fail. Also, note that you get the traceback from Output (tab) -> Python (dropdown).
If everything here fails and you checked all the basic configurations, try specifying the path to your test folder in your workspace's settings.ini file like so. The three configurations below it are from the majority of other solutions.
"python.testing.pytestArgs": [
"${workspaceFolder}/tests"
],
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestEnabled": true,
I am trying to run a specific test method with pytest on Python 3.6 with the following command:
pytest sanity_test.py::test_check
(as mentioned in pytest docs)
and it fails with the following error message:
ERROR: not found: <rootdir>/tests/automation/sanity/sanity_test.py::test_check
(no name '<rootdir>/tests/automation/sanity/sanity_test.py::test_check' in any of [<DoctestModule 'tests/automation/sanity/sanity_test.py'>, <Module
'tests/automation/sanity/sanity_test.py'>])
when I run the whole test file the tests are running properly with:
pytest sanity_test.py
When I tried running a specific method in isolated project it works perfectly.
Any reason why pytest can't find specifically my test methods on my project?
The file is very simple and looks like this:
def test_check():
pass
Thank to hoefling, I recognized that I had --doctest-modules option in my setup.cfg file. Removing this option fixed the problem.
hoefling opened an issue in pytest for that problem.
I am new to pytest and I have python2.6 installed on my setup.
I installed pytest and the testcases get executed properly. I installed couple of plugins like pytest-timeout, putest-xdist etc but these plugins does not load when I run the cases. For timeout, I get following error: py.test: error: unrecognized arguments: --timeout
Same steps followed with python2.7 works.
Any idea how this can be solved or alteast steps to debug to know what exactly is causing the issue.
Unfortunately pytest < 3.0 "hides" the ImportError happening when failing to import a plugin. If you remove all plugin arguments but add -rw, you should be able to see what exactly is going wrong in the warning summary.
In your conftest.py file just add the following line after the imports:
pytest_plugins = 'pytest_timeout'
It should solve your issue.
I'm trying to run code coverage over my program's unit tests. I'm using mock in the tests, which means I have to use python3 and up. I've installed coverage.py using pip:
pip install coverage
The installation worked and coverage is working preperly. The issue is that when I'm trying to run coverage over my unit tests it runs with python2.6 and fails on import mock although my script starts with #!/usr/bin/python3:
coverage run ./my_tests.py
Traceback (most recent call last):
File "./my_tests.py", line 9, in module
from unittest.mock import patch
ImportError: No module named mock
Is there a way to configure coverage to run with python3? Is there a version of coverage which works with python3 by default?
You apparently have 2.6 as your default python. Or at least, you installed the coveragepy module in the 2.6 tree, which put 'coverage' in python26/Scripts, which then runs coveragepy with 2.6. However, the module works with both 2.x and 3.x if you explicitly run it with one or the other instead of just the default.
I happened to have 'installed' coveragepy by cloning it in my dev directory. I also wrote a cover.bat for my particular need, which is to test new and patched idlelib files in my python repository clone before committing them. Here is my file. Of particular relevance to your question are the lines that begin with %py%. I set that to my repository build of 3.4, but you could just as easily point it to installed 3.4 or even make it an input.
#echo off
rem Usage: cover fileName [test_ suffix] # proper case required by coveragepy
rem filename without .py, 2nd parameter if test is not test_filename
setlocal
set py=34\pcbuild\python_d
set src=idlelib.%1
if "%2" EQU "" set tst=34/Lib/idlelib/idle_test/test_%1.py
if "%2" NEQ "" set tst=34/Lib/idlelib/idle_test/test_%2.py
%py% coveragepy run --pylib --source=%src% %tst%
%py% coveragepy report --show-missing
%py% coveragepy html
htmlcov\34_Lib_idlelib_%1.html
rem Above opens new report; htmlcov\index.html displays report index