We Keep Coding

Python unit tests not discovered in VSCode

I've written a python test file called scraping_test.py, with a single test class, using unittest, called TestScrapingUtils
"""Tests for the scraping app"""
import unittest
from bs4 import BeautifulSoup as bs4
from mosque_scraper.management.commands import scraping_utils
from mosque_scraper.selectors import MOSQUE_INFO_ROWS_SELECTOR
class TestScrapingUtils(unittest.TestCase):
"""Test scraping_utils.py """
def setup(self):
"""Setup McSetupface."""
pass
def test_get_keys_from_row(self):
""" Test that we extract the correct keys from the supplied rows."""
test_page_name = "test_page.html"
with open(test_page_name) as test_page_file:
test_mosque = bs4(test_page_file, 'html.parser')
rows = test_mosque.select(MOSQUE_INFO_ROWS_SELECTOR)
field_dict = scraping_utils.get_fields_from_rows(rows)
self.assertDictEqual(field_dict, {})
My settings for unit tests are:
{
"python.unitTest.unittestEnabled": true,
"python.unitTest.unittestArgs": [
"-v",
"-s",
".",
"-p",
"*test.py"
]
}
It looks like it should work, but when I click to run the tests in VSCode it says that no tests were discovered:
No tests discovered, please check the configuration settings for the tests.
How do I make it work?

You have to run it once by using shortcut key shift+ctrl p, and type "Python run all unit tests".
It won't show up in the editor until it was successfully executed at least once or use the discover unit test method.
However one thing catch me many times is that the Python file has to be a valid Python file. The intellisense in VS Code for Python is not complex(compare to Javascript or Typescript), and it won't highlight any syntax error. You can verify that by force it to run all unit test and observe the Python Test Log window.

What caught me is that the __init__.py file must be created in every subdirectory, from the root folder specified with -s option (in the example, the current directory ".") to the subdirectory where the test module is located. Only then was I able to discover tests successfully.
In the question example, both project_dir/ and project_dir/scraping_app/ should contain __init__.py. This is assuming that settings.json is located in project_dir/.vscode and the tests are run from project_dir/ directory.
Edit: Alternatively, use "-s", "./scraping_app/" as the root test directory so you don't have to put __init__.py to project_dir/.

Instead of file name 'scraping_test.py' it shall be 'test_scraping.py'
string shall start from 'test' prefix

I had the same error with a slightly different configuration. (I am posting this here because this is the question that comes up when you search for this error.)
In addition to what was said above, it is also important to not use periods in the test file names (e.g. use module_test.py instead of module.test.py).

You can add the DJANGO_SETTINGS_MODULE variable and django.setup() inside the __init__.py file of tests package.
import os
import django
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'your_app.settings')
django.setup()

In my case, the problem was that my test was importing a module which was reading an environment variable using os.environ['ENV_NAME']. If the variable does not exist, it throws an error. But VS Code does not log anything (or at least I couldn't find it).
So, the reason was that my .env file was NOT in the workspace root. So I had to add "python.envFile": "${workspaceFolder}/path/to/.env" to the settings.json file.
After that, the test was discovered successfully.

Also had this issue.
for me the issue was, make sure there are no errors, and comment out all code in files that rely on pytest, just for the initial load up.

Another issue that causes the unit tests not be discovered is when using a conda environment that contains an explicit dependency on the conda package itself. This happens when the enviroment.yml contains the line:
- conda
Removing this line and creating the environment from scratch makes the unit tests discoverable. I have created a bug report in Github for this: https://github.com/microsoft/vscode-python/issues/19643
(This is my second solution to this issue; I decided to create another answer since this is entirely different from the previous one.)

This is my first time using unittest in vscode. I found that the file names cannot contain spaces and dots. and cannot start with numbers.
for the dots, I guess anything after a dot is considered by the extension by unittest.
for the spaces, I guess they do not use "" to surround the filename.

For me Discovering the unit tests did the trick.
SHIFT+CTRL+P and execute "Python: Discover unit tests"
After running this I get the "Run Test|Debug Test" over each test function.

Python script runs perfect in PyCharm but in terminal don't

I really don't understand... If you need screenshots of some settings please tell me because I really don't know why it works in PyCharm but not outside Pycharm...
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
import unittest
from datetime import datetime
class MYMaster(unittest.TestCase):
def Test_login(self):
<<<<< MY CODE >>>>>
if __name__ == '__main__':
unittest.main()
If I right click in PyCharm on line class MYMaster(unittest.TestCase): and select option Run 'Unittest in MYMaster' it will send this code
C:\Users\MyNameIs\AppData\Local\Programs\Python\Python35-32\python.exe"C:\Program Files (x86)\JetBrains\PyCharm 4.5.4\helpers\pycharm\utrunner.py" C:\Users\MyNameIs\PycharmProjects\untitled\MyProject\MyMain.py::MYMaster true
Testing started at 15:42 ...
Process finished with exit code 0
Empty test suite.
If I right click on def Test_login(self):and select option Run 'Unittest TestLogin' it will send this code (But it runs entire code and give results).
C:\Users\MyNameIs\AppData\Local\Programs\Python\Python35-32\python.exe "C:\Program Files (x86)\JetBrains\PyCharm 4.5.4\helpers\pycharm\utrunner.py" C:\Users\MyNameIs\PycharmProjects\untitled\MyProject\MyMain.py::MYMaster::Test_login true
Testing started at 15:50 ...
Process finished with exit code 0
Now I decided to open the MyMain.py in Python IDLE where I clicked Run module and this are the results:
= RESTART: C:\Users\MyNameIs\PycharmProjects\untitled\MyProject\MyMain.py =
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
In Pycharm under Tool>Python Integrated Tools Default test runner =
Unittests Docstring format = reStructuredText Checked checkbox Analyze
Python code in docstrings

When you run unittests in Pycharm, Pycharm provides a wrapper that executes other code before your code.
To execute unittests from the terminal, you should use this command, provided that you have python set in your PATH:
python -m unittest /path/to/script_with_tests.py
You should also make sure that your functions' names start with test, ie test_login.
From unittest documentation:
A testcase is created by subclassing unittest.TestCase. The three individual tests are defined with methods whose names start with the letters test. This naming convention informs the test runner about which methods represent tests.

Is it possible to run all unit test?

I have two module with two different classes and their corresponding test classes.
foo.py
------
class foo(object):
def fooMethod(self):
// smthg
bar.py
------
class bar(object):
def barMethod(self):
// smthg
fooTest.py
------
class fooTest(unittest.TestCase):
def fooMethodTest(self):
// smthg
barTest.py
------
class barTest(unittest.TestCase):
def barMethodTest(self):
// smthg
In any, test and source module, file, I erased the if __name__ == "__main__": because of increasing coherency and obeying object-oriented ideology.
Like in Java unit test, I'm looking for creating a module to run all unittest. For example,
runAllTest.py
-------------
class runAllTest(unittest.TestCase):
?????
if __name__ == "__main__":
?????
I looked for search engine but didn't find any tutorial or example. Is it possible to do so? Why? or How?
Note: I'm using eclipse and pydev distribution on windows machine.

When running unit tests based on the built-in python unittest module, at the root level of your project run
python -m unittest discover <module_name>
For the specific example above, it suffices to run
python -m unittest discover .
https://docs.python.org/2/library/unittest.html

You could create a TestSuite and run all your tests in it's if __name__ == '__main__' block:
import unittest
def create_suite():
test_suite = unittest.TestSuite()
test_suite.addTest(fooTest())
test_suite.addTest(barTest())
return test_suite
if __name__ == '__main__':
suite = create_suite()
runner=unittest.TextTestRunner()
runner.run(suite)
If you do not want to create the test cases manually look at this quesiton/answer, which basically creates the test cases dynamically, or use some of the features of the unittest module like test discovery feature and command line options ..

I think what you are looking for is the TestLoader. With this you can load specific tests or modules or load everything under a given directory. Also, this post has some useful examples using a TestSuite instance.
EDIT: The code I usually have in my test.py:
if not popts.tests:
suite = unittest.TestLoader().discover(os.path.dirname(__file__)+'/tests')
#print(suite._tests)
# Print outline
lg.info(' * Going for Interactive net tests = '+str(not tvars.NOINTERACTIVE))
# Run
unittest.TextTestRunner(verbosity=popts.verbosity).run(suite)
else:
lg.info(' * Running specific tests')
suite = unittest.TestSuite()
# Load standard tests
for t in popts.tests:
test = unittest.TestLoader().loadTestsFromName("tests."+t)
suite.addTest(test)
# Run
unittest.TextTestRunner(verbosity=popts.verbosity).run(suite)
Does two things:
If -t flag (tests) is not present, find and load all tests in directory
Else, load the requested tests one-by-one

I think you could just run the following command under the folder where your tests files are located:
python -m unittest
as mentioned here in the doc that "when executed without arguments Test Discovery is started"

With PyDev right click on a folder in Eclipse and choose "Run as-> Python unit-test". This will run all tests in that folder (the names of the test files and methods have to start with "test_".)

You are looking for nosetests.
You might need to rename your files; I'm not sure about the pattern nose uses to find the test files but, personally, I use *_test.py. It is possible to specify a custom pattern which your project uses for test filenames but I remember being unable to make it work so I ended up renaming my tests instead.
You also need to follow PEP 328 conventions to work with nose. I don't use IDEs with Python but your IDE may already follow it---just read the PEP and check.
With a PEP 328 directory/package structure, you can run individual tests as
nosetests path.to.class_test
Note that instead of the usual directory separators (/ or \), I used dots.
To run all tests, simply invoke nosetests at the root of your project.

Good way to collect programmatically generated test suites in nose or pytest

Say I've got a test suite like this:
class SafeTests(unittest.TestCase):
# snip 20 test functions
class BombTests(unittest.TestCase):
# snip 10 different test cases
I am currently doing the following:
suite = unittest.TestSuite()
loader = unittest.TestLoader()
safetests = loader.loadTestsFromTestCase(SafeTests)
suite.addTests(safetests)
if TARGET != 'prod':
unsafetests = loader.loadTestsFromTestCase(BombTests)
suite.addTests(unsafetests)
unittest.TextTestRunner().run(suite)
I have major problem, and one interesting point
I would like to be using nose or py.test (doestn't really matter which)
I have a large number of different applications that are exposing these test
suites via entry points.
I would like to be able to aggregate these custom tests across all installed
applications so I can't just use a clever naming convention. I don't
particularly care about these being exposed through entry points, but I
do care about being able to run tests across applications in
site-packages. (Without just importing... every module.)
I do not care about maintaining the current dependency on
unittest.TestCase, trashing that dependency is practically a goal.
EDIT This is to confirm that #Oleksiy's point about passing args to
nose.run does in fact work with some caveats.
Things that do not work:
passing all the files that one wants to execute (which, weird)
passing all the modules that one wants to execute. (This either executes
nothing, the wrong thing, or too many things. Interesting case of 0, 1 or
many, perhaps?)
Passing in the modules before the directories: the directories have to come
first, or else you will get duplicate tests.
This fragility is absurd, if you've got ideas for improving it I welcome
comments, or I set up
a github repo with my
experiments trying to get this to work.
All that aside, The following works, including picking up multiple projects
installed into site-packages:
#!python
import importlib, os, sys
import nose
def runtests():
modnames = []
dirs = set()
for modname in sys.argv[1:]:
modnames.append(modname)
mod = importlib.import_module(modname)
fname = mod.__file__
dirs.add(os.path.dirname(fname))
modnames = list(dirs) + modnames
nose.run(argv=modnames)
if __name__ == '__main__':
runtests()
which, if saved into a runtests.py file, does the right thing when run as:
runtests.py project.tests otherproject.tests

For nose you can have both tests in place and select which one to run using attribute plugin, which is great for selecting which tests to run. I would keep both tests and assign attributes to them:
from nose.plugins.attrib import attr
#attr("safe")
class SafeTests(unittest.TestCase):
# snip 20 test functions
class BombTests(unittest.TestCase):
# snip 10 different test cases
For you production code I would just call nose with nosetests -a safe, or setting NOSE_ATTR=safe in your os production test environment, or call run method on nose object to run it natively in python with -a command line options based on your TARGET:
import sys
import nose
if __name__ == '__main__':
module_name = sys.modules[__name__].__file__
argv = [sys.argv[0], module_name]
if TARGET == 'prod':
argv.append('-a slow')
result = nose.run(argv=argv)
Finally, if for some reason your tests are not discovered you can explicitly mark them as test with #istest attribute (from nose.tools import istest)

This turned out to be a mess: Nose pretty much exclusively uses the
TestLoader.load_tests_from_names function (it's the only function tested in
unit_tests/test_loader)
so since I wanted to actually load things from an arbitrary python object I
seemed to need to write my own figure out what kind of load function to use.
Then, in addition, to correctly get things to work like the nosetests script
I needed to import a large number of things. I'm not at all certain that this
is the best way to do things, not even kind of. But this is a stripped down
example (no error checking, less verbosity) that is working for me:
import sys
import types
import unittest
from nose.config import Config, all_config_files
from nose.core import run
from nose.loader import TestLoader
from nose.suite import ContextSuite
from nose.plugins.manager import PluginManager
from myapp import find_test_objects
def load_tests(config, obj):
"""Load tests from an object
Requires an already configured nose.config.Config object.
Returns a nose.suite.ContextSuite so that nose can actually give
formatted output.
"""
loader = TestLoader()
kinds = [
(unittest.TestCase, loader.loadTestsFromTestCase),
(types.ModuleType, loader.loadTestsFromModule),
(object, loader.loadTestsFromTestClass),
]
tests = None
for kind, load in kinds.items():
if isinstance(obj, kind) or issubclass(obj, kind):
log.debug("found tests for %s as %s", obj, kind)
tests = load(obj)
break
suite = ContextSuite(tests=tests, context=obj, config=config)
def main():
"Actually configure the nose config object and run the tests"
config = Config(files=all_config_files(), plugins=PluginManager())
config.configure(argv=sys.argv)
tests = []
for group in find_test_objects():
tests.append(load_tests(config, group))
run(suite=tests)

If your question is, "How do I get pytest to 'see' a test?", you'll need to prepend 'test_' to each test file and each test case (i.e. function). Then, just pass the directories you want to search on the pytest command line and it will recursively search for files that match 'test_XXX.py', collect the 'test_XXX' functions from them and run them.
As for the docs, you can try starting here.
If you don't like the default pytest test collection method, you can customize it using the directions here.

If you are willing to change your code to generate a py.test "suite" (my definition) instead of a unittest suite (tech term), you may do so easily. Create a file called conftest.py like the following stub
import pytest
def pytest_collect_file(parent, path):
if path.basename == "foo":
return MyFile(path, parent)
class MyFile(pytest.File):
def collect(self):
myname="foo"
yield MyItem(myname, self)
yield MyItem(myname, self)
class MyItem(pytest.Item):
SUCCEEDED=False
def __init__(self, name, parent):
super(MyItem, self).__init__(name, parent)
def runtest(self):
if not MyItem.SUCCEEDED:
MyItem.SUCCEEDED = True
print "good job, buddy"
return
else:
print "you sucker, buddy"
raise Exception()
def repr_failure(self, excinfo):
return ""
Where you will be generating/adding your code into your MyFile and MyItem classes (as opposed to the unittest.TestSuite and unittest.TestCase). I kept the naming convention of MyFile class that way, because it is intended to represent something that you read from a file, but of course you can basically decouple it (as I've done here). See here for an official example of that. The only limit is that in the way I've written this foo must exist as a file, but you can decouple that too, e.g. by using conftest.py or whatever other file name exist in your tree (and only once, otherwise everything will run for each files that matches -- and if you don't do the if path.basename test for every file that exists in your tree!!!)
You can run this from command line with
py.test -whatever -options
or programmactically from any code you with
import pytest
pytest.main("-whatever -options")
The nice thing with py.test is that you unlock many very powerful plugings such as html report

python nosetests equivalent of unittest Testsuite in the test file

In nosetests, I know that you can specify which tests you want to run via a nosetests config file as such:
[nosetests]
tests=testIWT_AVW.py:testIWT_AVW.tst_bynd1,testIWT_AVW.py:testIWT_AVW.tst_bynd3
However, the above just looks messy and becomes harder to maintain when a lot of tests are added, especially without being able to use linebreaks. I found it a lot more convenient to be able to specify which tests I want to run using unittests TestSuite feature. e.g.
def custom_suite():
suite = unittest.TestSuite()
suite.addTest(testIWT_AVW('tst_bynd1'))
suite.addTest(testIWT_AVW('tst_bynd3'))
return suite
if __name__=="__main__":
runner = unittest.TextTestRunner()
runner.run(custom_suite())
Question: How do I specify which tests should be run by nosetests within my .py file? Thanks.
P.S. If there is a way to specify tests via a nosetest config file that doesn't force all tests to be written on one line I would be open to it as well, as a second alternative

I'm not entirely sure whether you want to run the tests programmatically or from the command line. Either way this should cover both:
import itertools
from nose.loader import TestLoader
from nose import run
from nose.suite import LazySuite
paths = ("/path/to/my/project/module_a",
"/path/to/my/project/module_b",
"/path/to/my/project/module_c")
def run_my_tests():
all_tests = ()
for path in paths:
all_tests = itertools.chain(all_tests, TestLoader().loadTestsFromDir(path))
suite = LazySuite(all_tests)
run(suite=suite)
if __name__ == '__main__':
run_my_tests()
Note that the nose.suite.TestLoader object has a number of different methods available for loading tests.
You can call the run_my_tests method from other code or you can run this from the command line with a python interpreter, rather than through nose. If you have other nose configuration, you may need to pass that in programmatically as well.

If I'm correctly understanding your question, you have several options here:
you can mark your tests with special nose decorators: istest and nottest. See docs
you can mark tests with tags
you can join test cases in test suites. I haven't used it by myself, but it seems that you have to override nose's default test discovery to respect your test suites (see docs)
Hope that helps.