I've written tests to be run by pytest in a vscode project. The configuration file .vscode/settings.json allow passing additional command line parameters to pytest using:
"python.testing.pytestArgs": [
"test/",
"--exitfirst",
"--verbose"
],
How can I also pass custom script arguments to the test script itself? like invoking pytest from the command line as:
pytest --exitfirst --verbose test/ --test_arg1 --test_arg2
After much experimentation I finally found how to do it. What I needed was to pass user name and password to my script in order to allow the code to log into a test server. My test looked like this:
my_module_test.py
import pytest
import my_module
def login_test(username, password):
instance = my_module.Login(username, password)
# ...more...
conftest.py
import pytest
def pytest_addoption(parser):
parser.addoption('--username', action='store', help='Repository user')
parser.addoption('--password', action='store', help='Repository password')
def pytest_generate_tests(metafunc):
username = metafunc.config.option.username
if 'username' in metafunc.fixturenames and username is not None:
metafunc.parametrize('username', [username])
password = metafunc.config.option.password
if 'password' in metafunc.fixturenames and password is not None:
metafunc.parametrize('password', [password])
Then in my settings file I can use:
.vscode/settings.json
{
// ...more...
"python.testing.autoTestDiscoverOnSaveEnabled": true,
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestEnabled": true,
"python.testing.pytestArgs": [
"--exitfirst",
"--verbose",
"test/",
"--username=myname",
"--password=secret",
// ...more...
],
}
An alternative way is to use pytest.ini file instead:
pytest.ini
[pytest]
junit_family=legacy
addopts = --username=myname --password=secret
If this is just for the debugger then you can specify things in "args" in your launch.json file. See https://code.visualstudio.com/docs/python/debugging#_args for more details.
Related
While my practice I try to build my hybrid test framework. Here is link to GitHub repository of My framework
Its structure is
...root/
...Base/
...Locators/
...Pages/
...Tests/
...conftest.py
...pytest.ini
I would like to make possibility to pass configurations from terminal command or from conftest.py file
But each time I run command:
pytest search_test.py --browser chrome --server server_ip/wd/hub
I get error:
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --browser chrome --server server_ip/wd/hub/wd/hub
inifile: path to inifile
rootdir: path to root directory
here is content of my conftest.py file:
import pytest
import configparser
from selenium import webdriver
def environment_options(parser):
parser.addoption('--browser', '-B', dest='BROWSER', choises=['chrome', 'chr', 'firefox', 'ff'],
help=f"possible values are: {['chrome', 'chr', 'firefox', 'ff']}")
parser.addoption('--server', '-S', dest="SERVER")
#pytest.fixture(scope='class')
def environment_configuration(request):
read_config = configparser.ConfigParser()
# checking if browser was input from console or from config file from section Environment and assigment
# it to browser_name variable
browser_name = request.config.getoption(
"BROWSER") or read_config.get("Environments", "browser")
# checking if remote server was input from console or from config file from section Environment and assigment
# it to remote_server variable
remote_server = request.config.getoption(
"SERVER") or read_config.get("Environments", "remote_server")
try:
request.cls.driver = webdriver.Remote(
command_executor=remote_server,
desired_capabilities={
"browserName": browser_name})
except BaseException:
print("check browser or remote server configs")
yield request.cls.driver
request.cls.driver.close()
request.cls.driver.quit()
need to change name of function from environment_options to pytest_addoption and everything starts to work :-)
I have a problem with setting report name and folder with it dynamically in Python's pytest.
For example: I've run all pytest's tests # 2020-03-06 21:50 so I'd like to have my report stored in folder 20200306 with name report_2150.html. I want it to be automated and triggered right after the tests are finished.
I'm working in VS Code and I'm aiming to share my work with colleagues with no automation experience so I'm aiming to use it as "click test to start".
My project structure:
webtools/
|── .vscode/
|──── settings.json
|── drivers/
|── pages/
|── reports/
|── tests/
|──── __init__.py
|──── config.json
|──── conftest.py
|──── test_1.py
|──── test_2.py
|── setup.py
Code samples:
settings.json
{
"python.linting.pylintEnabled": false,
"python.linting.flake8Enabled": true,
"python.linting.enabled": true,
"python.pythonPath": "C:\\Users\\user\\envs\\webtools\\Scripts\\python.exe",
"python.testing.pytestArgs": [
"tests",
"--self-contained-html",
"--html=./reports/tmp_report.html"
],
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestEnabled": true,
"python.testing.unittestArgs": [
"-v",
"-s",
"./tests",
"-p",
"test_*.py"
]
}
config.json
{
"browser": "chrome",
"wait_time": 10
}
conftest.py
import json
import pytest
from datetime import datetime
import time
import shutil
import os
from selenium import webdriver
from selenium.webdriver import Chrome
CONFIG_PATH = 'tests/config.json'
DEFAULT_WAIT_TIME = 10
SUPPORTED_BROWSERS = ['chrome', 'explorer']
#pytest.fixture(scope='session')
def config():
# Read the JSON config file and returns it as a parsed dict
with open(CONFIG_PATH) as config_file:
data = json.load(config_file)
return data
#pytest.fixture(scope='session')
def config_browser(config):
# Validate and return the browser choice from the config data
if 'browser' not in config:
raise Exception('The config file does not contain "browser"')
elif config['browser'] not in SUPPORTED_BROWSERS:
raise Exception(f'"{config["browser"]}" is not a supported browser')
return config['browser']
#pytest.fixture(scope='session')
def config_wait_time(config):
# Validate and return the wait time from the config data
return config['wait_time'] if 'wait_time' in config else DEFAULT_WAIT_TIME
#pytest.fixture
def browser(config_browser, config_wait_time):
# Initialize WebDriver
if config_browser == 'chrome':
driver = webdriver.Chrome(r"./drivers/chromedriver.exe")
elif config_browser == 'explorer':
driver = webdriver.Ie(r"./drivers/IEDriverServer.exe")
else:
raise Exception(f'"{config_browser}" is not a supported browser')
# Wait implicitly for elements to be ready before attempting interactions
driver.implicitly_wait(config_wait_time)
# Maximize window for test
driver.maximize_window()
# Return the driver object at the end of setup
yield driver
# For cleanup, quit the driver
driver.quit()
#pytest.fixture(scope='session')
def cleanup_report():
timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
os.chdir("./reports")
os.mkdir(timestamp)
yield
shutil.move("./tmp_report.html", "./%s/test_report.html" % timestamp)
In current situation the report is created as tmp_report.html in the reports folder, but I don't know how I can force running cleanup_report() after all tests are completed and tmp_report.html is present and complete in folder. For checking if complete I assume I'd have to verify if all html tags have their closing (or at least <html> one).
Can somebody help me with that? If you need some further code portions I'll provide them as soon as possible.
Thank you in advance!
You can customize the plugin options in a custom impl of the pytest_configure hook. Put this example code in a conftest.py file in your project root dir:
from datetime import datetime
from pathlib import Path
import pytest
#pytest.hookimpl(tryfirst=True)
def pytest_configure(config):
# set custom options only if none are provided from command line
if not config.option.htmlpath:
now = datetime.now()
# create report target dir
reports_dir = Path('reports', now.strftime('%Y%m%d'))
reports_dir.mkdir(parents=True, exist_ok=True)
# custom report file
report = reports_dir / f"report_{now.strftime('%H%M')}.html"
# adjust plugin options
config.option.htmlpath = report
config.option.self_contained_html = True
If you want to completely ignore what's passed from command line, remove the if not config.option.htmlpath: condition.
If you want to stick with your current impl, notice that on fixtures teardown, pytest-html hasn't written the report yet. Move the code from cleanup_report to a custom impl of the pytest_sessionfinish hook to ensure pytest-html has already written the default report file:
#pytest.hookimpl(trylast=True)
def pytest_sessionfinish(session, exitstatus):
shutil.move(...)
I have a long run test, which lasts 2 days, which I don't want to include in a usual test run. I also don't want to type command line parameters, that would deselect it and other tests at every usual test run. I would prefer to select a default-deselected test, when I actually need it. I tried renaming the test from test_longrun to longrun and use the command
py.test mytests.py::longrun
but that does not work.
Alternatively to the pytest_configure solution above I had found pytest.mark.skipif.
You need to put pytest_addoption() into conftest.py
def pytest_addoption(parser):
parser.addoption('--longrun', action='store_true', dest="longrun",
default=False, help="enable longrundecorated tests")
And you use skipif in the test file.
import pytest
longrun = pytest.mark.skipif("not config.getoption('longrun')")
def test_usual(request):
assert False, 'usual test failed'
#longrun
def test_longrun(request):
assert False, 'longrun failed'
In the command line
py.test
will not execute test_longrun(), but
py.test --longrun
will also execute test_longrun().
try to decorate your test as #pytest.mark.longrun
in your conftest.py
def pytest_addoption(parser):
parser.addoption('--longrun', action='store_true', dest="longrun",
default=False, help="enable longrundecorated tests")
def pytest_configure(config):
if not config.option.longrun:
setattr(config.option, 'markexpr', 'not longrun')
This is a slightly different way.
Decorate your test with #pytest.mark.longrun:
#pytest.mark.longrun
def test_something():
...
At this point you can run everything except tests marked with that using -m 'not longrun'
$ pytest -m 'not longrun'
or if you only want to run the longrun marked tests,
$ pytest -m 'longrun'
But, to make -m 'not longrun' the default, in pytest.ini, add it to addopts:
[pytest]
addopts =
-m 'not longrun'
...
If you want to run all the tests, you can do
$ pytest -m 'longrun or not longrun'
I am using pytest to run tests in multiple environments and I wanted to include that information (ideally) in an ini style config file. I would also like to override parts or all of the configuration at the command line as well. I tried using the hook pytest_addoption in my conftest.py like so:
def pytest_addoption(parser):
parser.addoption("--hostname", action="store", help="The host")
parser.addoption("--port", action="store", help="The port")
#pytest.fixture
def hostname(request):
return request.config.getoption("--hostname")
#pytest.fixture
def port(request):
return request.config.getoption("--port")
Using this I can add the configuration info at the command line, but not in a config file. I also tried adding
[pytest]
addopts = --hostname host --port 311
to my pytest.ini file, but that didn't work. Is there a way to do this without building my own plugin? Thanks for your time.
The parser object does have an addini method as well that you can use to specify configuration options through an ini file.
Here is the documentation for it: https://pytest.org/latest/writing_plugins.html?highlight=addini#_pytest.config.Parser.addini
addini(name, help, type=None, default=None)[source]
registers an ini-file option.
Name: name of the ini-variable
Type: type of the variable, can be pathlist, args, linelist or bool.
Default: default value if no ini-file option exists but is queried.
The value of ini-variables can be retrieved via a call to config.getini(name).
Where and how does py.test look for fixtures? I have the same code in 2 files in the same folder. When I delete conftest.py, cmdopt cannot be found running test_conf.py (also in same folder. Why is sonoftest.py not searched?
# content of test_sample.py
def test_answer(cmdopt):
if cmdopt == "type1":
print ("first")
elif cmdopt == "type2":
print ("second")
assert 0 # to see what was printed
content of conftest.py
import pytest
def pytest_addoption(parser):
parser.addoption("--cmdopt", action="store", default="type1",
help="my option: type1 or type2")
#pytest.fixture
def cmdopt(request):
return request.config.getoption("--cmdopt")
content of sonoftest.py
import pytest
def pytest_addoption(parser):
parser.addoption("--cmdopt", action="store", default="type1",
help="my option: type1 or type2")
#pytest.fixture
def cmdopt(request):
return request.config.getoption("--cmdopt")
The docs say
http://pytest.org/latest/fixture.html#fixture-function
pytest finds the test_ehlo because of the test_ prefix. The test function needs a function argument named smtp. A matching fixture
function is discovered by looking for a fixture-marked function named
smtp.
smtp() is called to create an instance.
test_ehlo() is called and fails in the last line of the test function.
py.test will import conftest.py and all Python files that match the python_files pattern, by default test_*.py. If you have a test fixture, you need to include or import it from conftest.py or from the test files that depend on it:
from sonoftest import pytest_addoption, cmdopt
Here is the order and where py.test looks for fixtures (and tests) (taken from here):
py.test loads plugin modules at tool startup in the following way:
by loading all builtin plugins
by loading all plugins registered through setuptools entry points.
by pre-scanning the command line for the -p name option and loading the specified plugin before actual command line parsing.
by loading all conftest.py files as inferred by the command line invocation (test files and all of its parent directories). Note that
conftest.py files from sub directories are by default not loaded at
tool startup.
by recursively loading all plugins specified by the pytest_plugins variable in conftest.py files
I had the same issue and spent a lot of time to find out a simple solution, this example is for others that have a similar situation as I had.
conftest.py:
import pytest
pytest_plugins = [
"some_package.sonoftest"
]
def pytest_addoption(parser):
parser.addoption("--cmdopt", action="store", default="type1",
help="my option: type1 or type2")
#pytest.fixture
def cmdopt(request):
return request.config.getoption("--cmdopt")
some_package/sonoftest.py:
import pytest
#pytest.fixture
def sono_cmdopt(request):
return request.config.getoption("--cmdopt")
some_package/test_sample.py
def test_answer1(cmdopt):
if cmdopt == "type1":
print ("first")
elif cmdopt == "type2":
print ("second")
assert 0 # to see what was printed
def test_answer2(sono_cmdopt):
if sono_cmdopt == "type1":
print ("first")
elif sono_cmdopt == "type2":
print ("second")
assert 0 # to see what was printed
You can find a similar example here: https://github.com/pytest-dev/pytest/issues/3039#issuecomment-464489204
and other here https://stackoverflow.com/a/54736376/6655459
Description from official pytest documentation: https://docs.pytest.org/en/latest/reference.html?highlight=pytest_plugins#pytest-plugins
As a note that the respective directories referred to in
some_package.test_sample" need to have __init__.py files for the plugins to be loaded by pytest