I have looked up solutions online for this issue, but none of them seem to address my specific situation.
I have a pycharm project with multiple directories, subdirs and files. When I invoke the main entry point method from the command line, I get
ModuleNotFoundError on all the imports across ~20 files. The solutions I found online recommend modifying the PYTHONPATH. This is not a viable solution for my use case because
I would have to add the sys.path.append call in all my files. That's the only way that it seems to work.
I cannot use any third party libs.
I will be sharing the project as a zip file. Note: I cannot use github to share the project nor is my intention to create a distributable. So when someone else unzips the project on their PC, they should be able to run it from the command line with out any issues. They should not have to modify their env variables to run it.
What are my options?
EDIT:
Project structure:
Project-
|
|
mod-
|
|
data-
|
|
ClassA.py
ClassB.py
|
config-
|
|
ClassC.py
ClassD.py
...
main.py
Import statements look like this:
from mod.data.ClassA import ClassA
Error:
ModuleNotFoundError: No module named 'ClassA'
This error shows up for every import statement in my python files.
When I add sys.path.append it works, but I have to do it for every import statement and its a hard coded abs path that will need to be updated by anyone with the zip.
If your modules are contained in the same directory as your executable, as you seem to indicate, it's this simple:
File structure:
f1.py
m1
__init__.py
C1.py
C1.py
class C1:
def test(self):
print("Hi!")
f1.py
#!/usr/bin/env python
from m1.C1 import C1
C1().test()
Execution:
> python f1.py
Hi!
You do have __init__.py files in your module directories, right? If not, then that's probably why you're having trouble with this.
Related
Forgive me for another "relative imports" post but I've read every stack overflow post about them and I'm still confused.
I have a file structure that looks something like this:
|---MyProject
| |---constants.py
| |---MyScripts
| | |---script.py
Inside of script.py I have from .. import constants.py
And of course when I run python3 script.py, I get the error ImportError: attempted relative import with no known parent package.
From what I understand, this doesn't work because when you run a python file directly you can't use relative imports. After hours of searching it seems that the most popular solution is to add "../" to sys.path and then do import constants.py but this feels a little hacky. I've got to believe that there is a better solution than this right? Importing constants has got to be a fairly common task.
I've also seen some people recommend adding __init__.py to make the project into a package but I don't really understand how this solves anything?
Are there other solutions out there that I've either missed or just don't understand?
Your library code should live in a package. For your folder to be a package, it needs to have __init__.py.
Your scripts can live in the top directory in development. Here is how you would structure this layout:
|---MyProject
| |---myproject
| | |---__init__.py
| | |---constants.py
| |---script.py
If you were to productionize this, you could place scripts into a special folder, but then they would rely on the package being installed on your system. The layout could look like this:
|---MyProject
| |---myproject
| | |---__init__.py
| | |---constants.py
| |---scripts
| | |---script.py
In both cases your imports would be normal absolute (non-relative) imports like
from myproject.constants import ...
OR
from myproject import constants
You are correct that attempting a relative import or a path modification in a standalone script is hacky. If you want to use the more flexible second layout, make a setup.py install script in the root folder, and run python setup.py develop.
|---MyProject
| |---__init__.py # File1
| |---constants.py
| |---MyScripts
| | |---__init__.py # File 2
| | |---script.py
And then the content of File1 __init__.py is:
from . import constants
from . import MyScripts
And then the content of File2 __init__.py is:
from . import script
By converting MyProject to a package, you would be able to import constants like:
from MyProject import constants
# Rest of your code
After that, you need to add the root of your MyProject to your python path. If you are using PyCharm, you do not need to do anything. By default the following line is added to your Python Console or Debugging sessions:
import sys
sys.path.extend(['path/to/MyProject'])
If you are not using PyCharm, one is is to add the above script before codes you run or for debugging, add that as a code to run before every session, or you define a setup.py for your MyProject package, install it in your environment and nothing else need to be changed.
The syntax from .. import x is only intended to be used inside packages (thus your error message). If you don't want to make a package, the other way to accomplish what you want is to manipulate the path.
In absolute terms, you can do:
import sys
sys.path.append(r'c:\here\is\MyProject')
import constants
where "MyProject" is the folder you described.
If you want to use relative paths, you need to do a little more work:
import inspect
import os
import sys
parent_dir = os.path.split(
os.path.dirname(inspect.getfile(inspect.currentframe())))[0]
sys.path.append(parent_dir)
import constants
To put it concisely, the error message is telling you exactly what's happening; t's just not explaining why. .. isn't a package. It resolves to MyProject but MyProject isn't a package. To make it a package, you must include an “__init__.py” file directly in the “MyProject” directory. It can even be an empty file if you're feeling lazy.
But given what you've shown us, it doesn't seem like your project is supposed to be a package. Try changing the import in script.py to import constants and running python3 MyScripts/script.py from your project directory.
I'm learning Python using a book that uses Window's based examples and they use these as examples (the comments are the filenames):
# dir1\__init__.py
print('dir1 init')
x = 1
# dir1\dir2\__init__.py
print('dir1 init')
y = 2
# dir1\dir2\mod.py
print('in mod.py')
z = 3
Now using IDLE, it attempts to import the files using:
import dir1.dir2.mod
Which will display:
dir1 init
dir2 init
in mod.py
When I attempt to do this on IDLE on my Mac it comes up with an error saying there is no module named dir1.
The point of the exercise is to package imports but I don't if I'm formatting the files incorrectly or importing incorrectly.
I think the files are not in the same directory from where the IDLE is running or looking at.
Try to see where the IDLE is on your Mac, complete path. Once find that out may be try putting the files in the same place so that python can see those files.
As an experiment, I'd suggest, writing your scripts in an editor like VS Code. cd into the directory where you have saved the script. Run the script by python yourfilename.py and see what it says then. Personally, I have never used the IDLE but I have come across python not being able to find the file.
I'm not sure what exactly you are asking, but your file structure should look like this in Finder:
which is:
dir1
|-- __init__.py
|-- dir2
|-- __init__.py
|-- mod.py
And you need to make sure that your working directory in IDLE is the directory where dir1 is located. I don't think you can change that in IDLE, but you could do this to make it work:
>>> import sys
>>> sys.path.insert(0,'/path/to/directory/where/dir1/is/stored')
I have the following folder structure for my Python project (in Pycharm):
Tool
|___Script
|___logs
|___ tests
| |___setA
| |___setB
| |___setC
| |___testSetC.py
|___ __init__.py
|______script.py
and I'm trying to import methods defined in script.py in testSetC.py.
I have from Script.script import * at the top of testSetC.pyand the script runs without any issues inside Pycharm. However, when I tried to run the same from the command line, it failed with the error message ModuleNotFoundError: No module named 'Script'.
I poked around and found that Pycharm's run configuration automatically adds the Tool directory to PYTHONPATH essentially masking the error.
What is the correct way to do this without manually amending sys.path in every test script? It looks hackish IMO.
I understand that this question has been asked several times but after reading them, and making the suggested fixes, I'm still stumped.
My project structure is as follows:
Project
|
src
|
root - has __init__.py
|
nested - has __init__.py
|
tests - has __init__.py
|
utilities - has __init__.py
|
services - has __init__.py
I've successfully run a unittest regression class from Eclipse without any issues.
As soon as I attempted to run the same class from the command-line (as other users who will be running the suite do not have access to Eclipse) I receive the error:
ImportError: No module named 'root'
As you can see from above, the module root has an __init__.py
All __init__.py modules are completely empty.
And assistance would be gratefully received.
Try adding a sys.path.append to the list of your imports.
import sys
sys.path.append("/Project/src/")
import root
import root.nested.tests
Just a note for anyone who arrives at this issue, using what Gus E showed in the accept answer and some further experience I've found the following to be very useful to ensure that I can run my programs from the command-line on my machine or on another colleague's should the need arise.
import sys
import os
sys.path.append(os.path.join(os.path.dirname(__file__), "..", ".."))
When I execute the 'main' method, which is located in the 'nested' directory, it ensures that the 'src' directory is added to the PYTHONPATH at the time of execution meaning all following imports will not throw an error.
Obviously, you need to adjust the number of ".." arguments to the os.path.join() method as determined by the location in your program of where your main method is executed from
Yet another way to solve this without the path goes like this: consider the following code where inside your folder name 'app' you have 3 files x.py, y.py and an empty init.py. So to run x.py you have an import from y such that:
x.py
from app.y import say_hi
print ("ok x is here")
say_hi()
And
y.py
print ("Im Y")
def say_hi():
print ("Y says hi")
so the folder structure would look like this:
testpy
app
__init__.py
x.py
y.py
Solution: in the folder BEFORE app do the following:
$ python -m app.x
Note: I did not use x.py (simply app.x)
Result:
Nespresso#adhg MINGW64 ~/Desktop/testpy
$ python -m app.x
Im Y
ok x is here
Y says hi
If anybody lands here:
I encountered this error as well. In my case, I used ~/my/path/ at the path.sys.append(...), and the fix was replacing ~ with the explicit path name (you can inquire it if you type pwd when you are on linux shell, or use os.path.expanduser(..) )
Another way to do this so python searches for modules in the current directory is to add it as an environment variable to your .bash_profile / .zshrc / etc. like so:
export PYTHONPATH="${PYTHONPATH}:."
I have a directory struture like that:
project
| __init__.py
| project.py
| src/
| __init__.py
| class_one.py
| class_two.py
| test/
| __init__.py
| test_class_one.py
Which project.py just instantiate ClassOne and run it.
My problem is in the tests, I don't know how to import src classes. I've tried importing these ways and I got nothing:
from project.src.class_one import ClassOne
and
from ..src.class_one import ClassOne
What am I doing wrong? Is there a better directory structure?
----- EDIT -----
I changed my dir structure, it's like this now:
Project/
| project.py
| project/
| __init__.py
| class_one.py
| class_two.py
| test/
| __init__.py
| test_class_one.py
And in the test_class_one.py file I'm trying to import this way:
from project.class_one import ClassOne
And it still doesn't work. I'm not using the executable project.py inside a bin dir exactly because I can't import a package from a higher level dir. :(
Thanks. =D
It all depends on your python path. The easiest way to achieve what you're wanting to do here is to set the PYTHONPATH environment variable to where the "project" directory resides. For example, if your source is living in:
/Users/crocidb/src/project/
I would:
export PYTHONPATH=/Users/crocidb/src
and then in the test_one.py I could:
import project.src.class_one
Actually I would probably do it this way:
export PYTHONPATH=/Users/crocidb/src/project
and then this in test_one.py:
import src.class_one
but that's just my preference and really depends on what the rest of your hierarchy is. Also note that if you already have something in PYTHONPATH you'll want to add to it:
export PYTHONPATH=/Users/crocidb/src/project:$PYTHONPATH
or in the other order if you want your project path to be searched last.
This all applies to windows, too, except you would need to use windows' syntax to set the environment variables.
From Jp Calderone's excellent blog post:
Do:
name the directory something related to your project. For example, if your
project is named "Twisted", name the
top-level directory for its source
files Twisted. When you do releases,
you should include a version number
suffix: Twisted-2.5.
create a directory Twisted/bin and put your executables there, if you
have any. Don't give them a .py
extension, even if they are Python
source files. Don't put any code in
them except an import of and call to a
main function defined somewhere else
in your projects. (Slight wrinkle:
since on Windows, the interpreter is
selected by the file extension, your
Windows users actually do want the .py
extension. So, when you package for
Windows, you may want to add it.
Unfortunately there's no easy
distutils trick that I know of to
automate this process. Considering
that on POSIX the .py extension is a
only a wart, whereas on Windows the
lack is an actual bug, if your
userbase includes Windows users, you
may want to opt to just have the .py
extension everywhere.)
If your project is expressable as a single Python source file, then put it
into the directory and name it
something related to your project. For
example, Twisted/twisted.py. If you
need multiple source files, create a
package instead (Twisted/twisted/,
with an empty
Twisted/twisted/__init__.py) and place
your source files in it. For example,
Twisted/twisted/internet.py.
put your unit tests in a sub-package of your package (note - this means
that the single Python source file
option above was a trick - you always
need at least one other file for your
unit tests). For example,
Twisted/twisted/test/. Of course, make
it a package with
Twisted/twisted/test/__init__.py.
Place tests in files like
Twisted/twisted/test/test_internet.py.
add Twisted/README and Twisted/setup.py to explain and
install your software, respectively,
if you're feeling nice.
Don't:
put your source in a directory called src or lib. This makes it hard
to run without installing.
put your tests outside of your Python package. This makes it hard to
run the tests against an installed
version.
create a package that only has a __init__.py and then put all your code into __init__.py. Just make a module
instead of a package, it's simpler.
try to come up with magical hacks to make Python able to import your module
or package without having the user add
the directory containing it to their
import path (either via PYTHONPATH or
some other mechanism). You will not
correctly handle all cases and users
will get angry at you when your
software doesn't work in their
environment.