This is a broad question because no one seems to have found a solution to it as yet so I think asking to see a working example might prove more useful. So here goes:
Has anyone run a nosetests on a python project using imports of multiple files/packages?
What I mean is, do you have a directory listing such as:
project/
|
|____app/
|___main.py
|___2ndFile.py
|___3rdFile.py
|____tests/
|____main_tests.py
Where your main.py imports multiple files and you perform a nosetests from the project file of utilizing a test script in the main_tests.py file? If so please can you screen shot your import section both of all your main files and your main_tests.py file?
This seems to be a major issue in nosetests, with no apparent solution:
Nosetests Import Error
A test running with nosetests fails with ImportError, but works with python command
https://github.com/nose-devs/nose/issues/978
https://github.com/nose-devs/nose/issues/964
You can't have python modules starting with a digit, so 2ndFile.py, 3rdFile.py won't actually work (rename them).
You'll need an __init__.py inside the app directory, for it to be considered a package, so add that (it can be empty file).
You don't need an __init__.py in the tests directory!
The import statements in main_tests.py should look like from app.main import blah
The absolute path of the project directory needs to be in your sys.path. To achieve this, set an environment variable: export PYTHONPATH=/path/to/project
Now running nosetests should work.
Related
I have a project I am working on, let's call it Project, which lives in the directory Project somewhere wholly unknown to me (really it lives both on my local system and on a couple Docker build systems). In that project, I have some source files, source/module1.py and source/module2.py. I also have some example files, some test files, and an init.py So my directory looks something like this:
Project
__init__.py
/source
module1.py
module2.py
/test
testRunner.py
/examples
awesomeExample.py
However, module1 needs some stuff from module2. My naive self thought this could be done by putting an import statement in module1:
import module2
# Do some other interesting stuff
And this works, but only when I am running / importing module 1 from the source directory. If I am, for example, running some unit tests in another directory test/testRunner.py, either from the test directory or in the main Project directory, the import will fail. Same with trying to use it when running an example in the examples directory.
So here is my problem: in general, I don't know where the calling script lives. It might be in the examples directory, it might be in the test directory, or it might be in the main Project directory (for example when trying to import stuff with an init.py). How do I ensure that module1 can always import module2 in each of these scenarios?
I am not looking for a solution like "add all those directories to your python path". Initially I just added Project to my python path on my local machine, and then did all my imports relative to that (import Project.source.module2), but this (predictably) caused my builds to fail on the Docker instances. I don't just want this to work on my local machine, but also on the Docker instances I'm using to build and test this software, and on any user's machine that subsequently installs it (i.e. by doing a pip install Project. What is the most robust way to make sure this dependency is satisfied? How can I make sure module1 can import module2 regardless of where module1 itself is imported from? Any python 3.x.x solution is welcome.
I figured out a way to do it (credit here) - it's a little inelegant, but extremely robust. Works on my local machine independent of whether using an import statement or running a script directly, as well as my build servers which use github actions and Travis CI.
Basically, I added a file in the source directory, called context.py with the following contents:
import os
import sys
fileLocation = os.path.dirname(os.path.abspath(__file__))
sourceLocation = os.path.abspath(os.path.join(fileLocation, '..', 'source/'))
sys.path.insert(0, sourceLocation)
This finds out the current file being executed from python, and then uses that to add to the python path. And then in my module1.py file, at the top I have:
import context
import module2
Now, whenever module1 is imported, it successfully imports module2. More elegant answers or comments on why this works and in which cases it might fail are appreciated.
Some background - I am working on a project, and have been writing tests with PyTest.
The structure of the folders is as following:
src/
...
tests/
helpers/
fixtures.py
functions.py
conftest.py
test1.py
test2.py
...
The helpers sub-folder holds functions and fixtures which should be accessible across all the tests.
In order to do so I added the following to conftest.py:
import os
import sys
sys.path.append(os.path.join(os.path.dirname(__file__), 'helpers'))
from fixtures import fixture1, fixture2, ....
This actually works, fixtures 1,2,.. are all available across the different tests.
And also:
import functions
works correctly from any file in the tests folder.
But the problem is that PyCharm can't recognize this change, and refer to the modules fixtures, functions as valid, which cancels the auto completion and gives pretty annoying false inspection comments.
If someone knows how to fix it (Notify PyCharm on the update), or maybe give another alternative for how things should be done (that's why I gave all of this context), I will be very grateful.
In the project view on the left, you should right-click on the helpers folder and click on: Mark Directory as->Sources Root, and this should resolve your problem.
After doing that, there will be no need for sys.path.append(os.path.join(os.path.dirname(__file__), 'helpers')) within PyCharm.
But since you already have a working system and trying to get rid of the annoying false error, only marking directory as source root should do the trick just fine.
I've been developing a project in python for some time using pycharm. I have no problem running it in pycharm, but I want to try and running it from the command line (I'm using windows).
When I try to run my main file using python <filename> from within the root directory, I get a module not found error. What is pycharm doing/how can i replicate it?
Also, pycharm creates pycache folders. If my understanding is correct its compiling the files, and that makes the runtime faster. Since my runtime is already long i'd like to do the same.
I'm using python 3.6
edit
File Structure
-Root\
----scheduler\
------main.py
------matrices
------models
------rhc
------Singleton.py
------utils.py
------__init__.py
------apis\
acedb.py
metrics.py
__init__.py
------matrices\
distancematrix.py
__init__.py
------models\
branch.py
constants.py
customer.py
dispatch.py
technician.py
workorder.py
__init__.py
------rhc\
pathfinder.py
rhc.py
schedule.py
sched_time.py
tech_schedule.py
__init__.py
Edit 2
Found the solution, i had to move my main files outside of the modules
Thanks!
If you have the below folder structure for instance. add an empty file named init.py and import from your app.py, in case you have a Module1Class , you can always import it like this
from modules.module1 import Module1Class
from modules.module2 import Module2Class
Folder structure
/app.py
/modules
/module1.py
/module2.py
/__init__.py
The first time you rum app.py from terminal like python app.py, the .pyc files will be created, leaving you with the following folder structure
/app.py
/modules
/module1.py
/module1.pyc
/module2.py
/module2.pyc
/__init__.py
Please refer to the python documentation as it's very well documented on how to create modules and importing them. I'm more used to python2.7 , so my answer might not be an exact fit to newer python versions.
https://docs.python.org/3/tutorial/modules.html
From there you can learn more about __ init __.py and module creation , exporting and importing
PS: I use only text editors to develop in python, as I find pycharm a bit on the heavy side, so I cannot explain how exactly pycharm works behind the curtains.
see ModuleNotFoundError: What does it mean __main__ is not a package? for a good example of ModuleNotFoundError description (was answered fast)
I bet that pycharm is configured to use a different python interpreter of virtual environment.
Suppose I'm writing a test. Obviously it is testing my app, so I need to import the app package somehow into the test script. The directory structure is this:
root/
app/
__init__.py
somemodule.py
tests/
my_test.py
And I run the tests like so:
cd tests
py.test # runs all the tests in the current directory
The question is: How should I import the application modules in my test modules?
In my_test.py, I tried doing from .. import app. This gives me an error Parent module '' not loaded, cannot perform relative import.
What is the standard way to accomplish this?
EDIT: Please note I edited the question to refer specifically to the py.test tool.
You should be able to make it run by properly configuring your py.test.
Add your module to your app/__init__.py the following line
from .somemodule import MyClass # or whatever ur class is called
Create a file called conftest.py in your main folder. You can leave it empty but it is used by py.test to find out the project path. Inside you can run some py.test initialization like adding fixtures.
In your my_test.py you will be able now to call
from app import MyClass
Now from your main folder you can finally:
py.test tests/test.py
This has worked for me. I think py.test has a way to include modules since you are probably not able to achieve the same without it. At least if I did not use py.test I would stick to modifying my PYTHONPATH to point to my application path.
EDIT:
Just to clarify py.test manipylates the sys.path for the testing session to include the root directory. Py.test identify the root path by using the conftest.py file. The root path is then added to the system path and used for testing.
You are indeed able to run:
py.test tests/test.py
and this would also work:
cd..
py.test rootTest/tests/test.py
I'm setting up some code for unittesting. My directory currently looks like this:
project/
src/
__init__.py
sources.py
test/
__init__.py
sources_test.py
In __init__.py for the test directory, I have these two lines:
import sys
sys.path.insert(0, '../')
In the test files, I have the line import src.sources.
When I use nose to run these tests from the project directory, everything works just fine. If I try to run the tests individually it gives me this error:
ImportError: No module named src.sources
I assume that this is because when I run the test from the command line it isn't using __init__.py. Is there a way I can make sure that it will use those lines even when I try to run the tests individually?
I could take the lines out of __init__.py and put them into my test files, but I'm trying to avoid doing that.
To run the tests individually I am running python sources_test.py
You're really trying to abuse packages here, and that isn't a good idea.
The simple solution is to not run the tests from within the tests directory. Just cd up a level, then do python tests/sources_test.py.
Of course that in itself isn't going to import test/__init__.py. For that, you really need to import the package. So python -m tests.sources_test is probably a better idea… except, of course, that if your package is made to be run as a script but not to be imported, that won't work.
Alternatively, you could (on POSIX platforms, at least) do PYTHONPATH=.. python sources_test.py from within tests. This is a bit hacky, but it should work.
Or, better, combine the above, and, from outside of tests, do PYTHONPATH=. python tests/sources_test.py.
A really hacky workaround is to explicitly import __init__. This should basically work for you simple use case, but everything ends up wrong—in particular, you end up with a module named __init__ instead of one named test, and of course your main module isn't named test.sources_test, and in fact there is no test package at all. Unless you accidentally re-import anything after modifying sys.path, in which case you may get duplicates of the modules.
If you write
import src.source
the python interpreter looks into the src directory for a __init__.py file. If it exists, you can use the directory as a package name. If your are not in your project directory, which is the case when you are in the src directory, then python looks into the directories in $PYTHONPATH environment variable (at least in linux, windows should also have some environment variable, maybe with another name), if it can find some directory src with a __init__.py file in it.
Did you set your $PYTHONPATH?