We Keep Coding

Testing with pytest: import that works on GitLab doesn't work in VS Code (and vice versa)

TL;DR: How can I set up my GitLab test pipeline so that the tests also run locally on VS Code?
I'm very new to GitLab pipelines, so please forgive me if the question is amateurish. I have a GitLab repo set up online, and I'm using VS Code to develop locally. I've created a new pipeline, I want to make sure all my unit tests (written with PyTest) run anytime I make a commit.
The issue is, that even though I use the same setup.py file for both places (obviously), I can't get both VS Code testing and the GitLab pipeline test to work at the same time. The issue is, I'm doing an import for my tests, and if I import like
...
from external_workforce import misc_tools
# I want to test functions in this misc_tools module
...
Then it works on GitLab, but not on VS Code, as VS Code gives an error when I'm doing test discovery, namely: ModuleNotFoundError: No module named 'external_workforce'. However, this works on GitLab. But if I import (in my test_tools.py file, see location below) like this:
...
from hr_datapool.external_workforce import misc_tools
...
It works in VS Code, but now GitLab is doing a crazy on me saying ModuleNotFoundError: No module named 'hr_datapool'.
I think the relevant info might be the following, please ask for more if more info is needed!
My file structure is:
.
|__ requirements.txt
setup.py
hr_datapool
|__ external_workforce
|__ __init__.py
misc_tools.py
tests
|__ test_tools.py
|__ other_module
...
In my pipeline editor (the .gitlab-ci.yml file) I have:
image: python:3.9.7
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
cache:
paths:
- .cache/pip
- venv/
before_script:
- python --version # For debugging
- pip install virtualenv
- virtualenv venv
- source venv/bin/activate
- pip install -r requirements.txt
test:
script:
- pytest --pyargs hr_datapool #- python setup.py test
run:
script:
- python setup.py bdist_wheel
artifacts:
paths:
- dist/*.whl
And finally, my setup.py is:
import re
from unittest import removeResult
from setuptools import setup, find_packages
with open('requirements.txt') as f:
requirements = f.read().splitlines()
for req in ['wheel', 'bar']:
requirements.append(req)
setup(
name='hr-datapool',
version='0.1',
...
packages=find_packages(),
install_requires=requirements,
)
Basically, the question is: How can I set up my GitLab test pipeline so that the tests also run locally on VS Code? Thank you!
UPDATE:
Adding the full trace coming from VS Code:
> conda run -n base --no-capture-output --live-stream python ~/.vscode/extensions/ms-python.python-2022.2.1924087327/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.2.1924087327/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir "." -s --cache-clear hr_datapool
cwd: .
[ERROR 2022-2-23 9:2:4.500]: Error discovering pytest tests:
[r [Error]: ============================= test session starts ==============================
platform darwin -- Python 3.9.7, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /Users/myuser/Documents/myfolder
plugins: anyio-2.2.0
collected 0 items / 1 error
==================================== ERRORS ====================================
_____ ERROR collecting hr_datapool/external_workforce/tests/test_tools.py ______
ImportError while importing test module '/Users/myuser/Documents/myfolder/hr_datapool/external_workforce/tests/test_tools.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../opt/anaconda3/lib/python3.9/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
hr_datapool/external_workforce/tests/test_tools.py:2: in <module>
from external_workforce import misc_tools
E ModuleNotFoundError: No module named 'external_workforce'
=========================== short test summary info ============================
ERROR hr_datapool/external_workforce/tests/test_tools.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
===================== no tests collected, 1 error in 0.08s =====================
Traceback (most recent call last):
File "/Users/myuser/.vscode/extensions/ms-python.python-2022.2.1924087327/pythonFiles/get_output_via_markers.py", line 26, in <module>
runpy.run_path(module, run_name="__main__")
File "/Users/myuser/opt/anaconda3/lib/python3.9/runpy.py", line 268, in run_path
return _run_module_code(code, init_globals, run_name,
File "/Users/myuser/opt/anaconda3/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/Users/myuser/opt/anaconda3/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/myuser/.vscode/extensions/ms-python.python-2022.2.1924087327/pythonFiles/testing_tools/run_adapter.py", line 22, in <module>
main(tool, cmd, subargs, toolargs)
File "/Users/myuser/.vscode/extensions/ms-python.python-2022.2.1924087327/pythonFiles/testing_tools/adapter/__main__.py", line 100, in main
parents, result = run(toolargs, **subargs)
File "/Users/myuser/.vscode/extensions/ms-python.python-2022.2.1924087327/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 44, in discover
raise Exception("pytest discovery failed (exit code {})".format(ec))
Exception: pytest discovery failed (exit code 2)
ERROR conda.cli.main_run:execute(33): Subprocess for 'conda run ['python', '/Users/myuser/.vscode/extensions/ms-python.python-2022.2.1924087327/pythonFiles/get_output_via_markers.py', '/Users/A111086670/.vscode/extensions/ms-python.python-2022.2.1924087327/pythonFiles/testing_tools/run_adapter.py', 'discover', 'pytest', '--', '--rootdir', '/Users/myuser/Documents/myfolder', '-s', '--cache-clear', 'hr_datapool']' command failed. (See above for error)
at ChildProcess.<anonymous> (/Users/myuser/.vscode/extensions/ms-python.python-2022.2.1924087327/out/client/extension.js:32:39235)
at Object.onceWrapper (events.js:422:26)
at ChildProcess.emit (events.js:315:20)
at maybeClose (internal/child_process.js:1048:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)]

The PYTHONPATH caused the problem.
As external_workforce parent folder path -> the path of hr_datapool in the PYTHONPATH when you are using GitLab. While hr_datapool parent folder path in the PYTHONPATH when you are using VSCode.
Are you running the test in the terminal on the VSCode? And have you added this in the settings.json file?
"terminal.integrated.env.windows": {
"PYTHONPATH": "${workspaceFolder};"
},
Then you can execute pytest in the terminal on the VSCode. But you have not configured this in GitLab instead of adding hr-datapool( - pytest --pyargs hr_datapool or setup( name='hr-datapool',), so you will get the error message.

How to pass options from PyTest to Twisted.trial?

I have some twisted.trial tests in my project, that is tests inheriting from twisted.trial.unittest.TestCase.
I need to pass some trial options to my test, specifically it is --reactor option of twisted.trial command line utility. Is there some way for me to pass it to pytest? My thinking is: I add something to pytest.ini, and pytest would somehow launch my trial unittest testcase with this option. Is that possible at the moment?
Sample to reproduce this. Take the following unit test:
# test_reactor.py
from twisted.trial.unittest import TestCase
class CrawlTestCase(TestCase):
def test_if_correct_reactor(self):
from twisted.internet import reactor
from twisted.internet.asyncioreactor import AsyncioSelectorReactor
assert isinstance(reactor, AsyncioSelectorReactor)
Now run it with trial with --reactor flag
python -m twisted.trial --reactor=asyncio test_reactor
test_reactor
CrawlTestCase
test_if_correct_reactor ... [OK]
-------------------------------------------------------------------------------
Ran 1 tests in 0.042s
PASSED (successes=1)
Now run it without --reactor flag
python -m twisted.trial test_reactor
test_reactor
CrawlTestCase
test_if_correct_reactor ... [ERROR]
===============================================================================
[ERROR]
Traceback (most recent call last):
File "/home/pawel/.../test_reactor.py", line 8, in test_if_correct_reactor
assert isinstance(reactor, AsyncioSelectorReactor)
builtins.AssertionError:
test_reactor.CrawlTestCase.test_if_correct_reactor
-------------------------------------------------------------------------------
Ran 1 tests in 0.081s
FAILED (errors=1)
Now run it with pytest
py.test test_reactor.py
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.9.4, pytest-6.2.3, py-1.11.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/pawel/../aaomw, configfile: pytest.ini
plugins: Faker-8.1.3, hypothesis-6.10.1, benchmark-3.4.1
collected 1 item
test_reactor.py F
Question is: how do I pass something to pytest so that it passes it to trial? Is there something I can put in pytest.ini so that reactor is passed to twisted trial?
If what I'm trying to do is not possible, please provide proof that it is not possible, this is also possibly accepted answer, perhaps something needs to be changed in pytest to make this kind of thing possible.

After installing pytest-twisted plugin I get --reactor flag and a proper option of installing and launching reactor.

pytest is magically working / not working

Two different Terminal windows open. Both set to the same dir. The second one was created by doing a "New tab" while in the first one
In the first one:
me $ pytest test_MakeInfo.py
================================================================================ test session starts =================================================================================
platform darwin -- Python 3.7.4, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /Users/me/Documents/workspace-vsc/Pipeline/src/python
plugins: arraydiff-0.3, remotedata-0.3.2, doctestplus-0.4.0, openfiles-0.4.0
collected 12 items
test_MakeInfo.py ............ [100%]
================================================================================= 12 passed in 0.87s =================================================================================
me $ which pytest
/Users/me/opt/anaconda3/bin/pytest
In the second one:
me $ pytest test_MakeInfo.py
================================================================================ test session starts =================================================================================
platform darwin -- Python 3.7.4, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /Users/me/Documents/workspace-vsc/Pipeline/src/python
plugins: arraydiff-0.3, remotedata-0.3.2, doctestplus-0.4.0, openfiles-0.4.0
collected 0 items / 1 error
======================================================================================= ERRORS =======================================================================================
_________________________________________________________________________ ERROR collecting test_MakeInfo.py __________________________________________________________________________
ImportError while importing test module '/Users/me/Documents/workspace-vsc/Pipeline/src/python/test_MakeInfo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Users/me/opt/anaconda3/lib/python3.7/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test_MakeInfo.py:6: in <module>
from MakeInfo import main, makeInfo, makeTumorInfo, _getNormalTumorInfo
E ModuleNotFoundError: No module named 'MakeInfo'
============================================================================== short test summary info ===============================================================================
ERROR test_MakeInfo.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================== 1 error in 0.17s ==================================================================================
me $ which pytest
/Users/me/opt/anaconda3/bin/pytest
What environment variables should I be looking at for differences? Because so far as I can tell everything's the same between the two

You want to check your PYTHONPATH and PATH environment variables.
Depending on the shell you use, they may not have been set the same when opening a new tab.
For example, in bash, you could append the required directory to your path using ~/.bash_profile or ~/.bashrc

pytest 3.1.0 does not detect plugin and does not report errors about it

I've setup pytest (3.1.0) and pytest-django (3.1.2) to run some tests. This works great on Ubuntu 17.04 and OSX 10.12.2, but not on our test environment (Ubuntu 12.04.5).
When reviewing pytest --version, the plugin is detected on the environments where everything is working as expected, but on the test server the plugin is not listed. This results in errors like ImproperlyConfigured: Requested setting CACHES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings. Output for a working environment:
$ venv/bin/pytest --version
This is pytest version 3.1.0, imported from /home/me/code/myproject/myproject_device/venv/local/lib/python2.7/site-packages/pytest.pyc
setuptools registered plugins:
pytest-django-3.1.2 at /home/me/code/myproject/myproject_device/venv/local/lib/python2.7/site-packages/pytest_django/plugin.py
When searching for details, I'm finding a number of pytest issues (f.i. https://github.com/pytest-dev/pytest/issues/1307) regarding suppression of import errors for pytest plugins, but these refer to changes in behaviour for pytest 3, where errors or warnings would be visible for these kind of problems. But I don't see any error other than the ImproperlyConfigured exception from django itself. Full output:
$ ./vagrant_venv/bin/pytest --version
This is pytest version 3.1.0, imported from /vagrant/vagrant_venv/local/lib/python2.7/site-packages/pytest.pyc
$ ./vagrant_venv/bin/pytest -rw -s src
============================= test session starts ==============================
platform linux2 -- Python 2.7.12, pytest-3.1.0, py-1.4.33, pluggy-0.4.0
rootdir: /vagrant, inifile: pytest.ini
collected 36 items / 1 errors
==================================== ERRORS ====================================
_________ ERROR collecting src/myproject/myproject/tests/test_views.py _________
src/myproject/myproject/tests/test_views.py:6: in <module>
from ui.views import controlview
src/myproject/ui/views/__init__.py:2: in <module>
from controlviewdata import ControlViewData
src/myproject/ui/views/controlviewdata.py:5: in <module>
from django.views.decorators.csrf import csrf_exempt
vagrant_venv/local/lib/python2.7/site-packages/django/views/decorators/csrf.py:1: in <module>
from django.middleware.csrf import CsrfViewMiddleware, get_token
vagrant_venv/local/lib/python2.7/site-packages/django/middleware/csrf.py:14: in <module>
from django.utils.cache import patch_vary_headers
vagrant_venv/local/lib/python2.7/site-packages/django/utils/cache.py:26: in <module>
from django.core.cache import caches
vagrant_venv/local/lib/python2.7/site-packages/django/core/cache/__init__.py:34: in <module>
if DEFAULT_CACHE_ALIAS not in settings.CACHES:
vagrant_venv/local/lib/python2.7/site-packages/django/conf/__init__.py:46: in __getattr__
self._setup(name)
vagrant_venv/local/lib/python2.7/site-packages/django/conf/__init__.py:40: in _setup
% (desc, ENVIRONMENT_VARIABLE))
E ImproperlyConfigured: Requested setting CACHES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 7.25 seconds ============================
I can reproduce this in a vagrant box for ubuntu/precise64 without issues. I tried upgrading python 2.7 to a newer version, but made no difference either.
So how do I track down the error that prevents the pytest-django plugin to load?

py.test: ImportError - cannot import ----

I am using someone else's code, available on GitHub. To run their code I created a virtualenv and installed all the dependencies listed - both python libraries and clones of other repositories. When I proceed to run the included tests, I get an ImportError:
Namespace(all=False, regr=False, sci=False, unit=True)
[localhost] local: py.test -x -v engine/test
==================================================================================== test session starts =====================================================================================
platform linux2 -- Python 2.7.6, pytest-2.8.2, py-1.4.31, pluggy-0.3.1 -- /home/compomics/local/METASPACE/SM_distributed/SM_engine/bin/python
cachedir: engine/test/.cache
rootdir: /home/compomics/local/METASPACE/SM_distributed/engine/test, inifile:
collecting 6 items / 1 errors
=========================================================================================== ERRORS ===========================================================================================
_______________________________________________________________________ ERROR collecting test_formula_img_validator.py _______________________________________________________________________
engine/test/test_formula_img_validator.py:7: in <module>
from engine.formula_img_validator import filter_sf_images,get_compute_img_measures, ImgMeasures
engine/formula_img_validator.py:7: in <module>
from pyIMS.image_measures import measure_of_chaos, isotope_image_correlation, isotope_pattern_match
E ImportError: cannot import name measure_of_chaos
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================== 1 error in 0.99 seconds ===================================================================================
Fatal error: local() encountered an error (return code 2) while executing 'py.test -x -v engine/test'
However, if I open the python interpreter and try to do the exact same imports, it does it just fine without any error. Similar questions suggested:
adding an empty __init__.py to the test directory
making sure pytest is installed in the virtualenv
I did both these things, and the error persists.
I added to the beginning of the test script:
import os
print(os.environ["PYTHONPATH"].split(os.pathsep))
print(os.listdir("."))
and confirmed that the folder from where I'm trying to import is indeed in the resulting list.
Not sure how to proceed. Would appreciate any help I can get :)

In file formula_img_validator.py change
from pyIMS.image_measures import measure_of_chaos,isotope_image_correlation, isotope_pattern_match
to
from engine.pyIMS.image_measures import measure_of_chaos, isotope_image_correlation, isotope_pattern_match
That'll solve the problem. For complete solution go to GitHub for new updated code.

there was a conflict with other library
EDIT - this was my own stupidity for not remembering I had cloned a previous version of the dependent repos, which was also on my path, and that did not include the function this code was trying to load. Sorry for not having deleted the question when I noticed, I couldn't for the life of me find the delete button :)