We Keep Coding

How to write __init__.py (and perhaps __main__.py) file to avoid import ModuleNotFoundError

I thought I had understood the import system, however I'm struggling to understand an apparently trivial case: I have a very simple Python application with the following structure:
.
└── myapp
├── __init__.py
├── lib.py
└── myapp.py
The content of lib.py is a trivial function:
def funct():
print("hello from function in lib")
myapp.py is supposed to be the entrance point of the application:
import lib
def main():
lib.funct()
if __name__ == "__main__":
print("calling main")
main()
When I run the main script it works:
> python myapp/myapp.py
calling main
hello from function in lib
However, when I just import the package from IPython for instance, it fails:
In [2]: import myapp
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-2-cabddf3cb27d> in <module>
----> 1 import myapp
~/test/myapp/__init__.py in <module>
----> 1 import myapp.myapp
~/test/myapp/myapp.py in <module>
----> 1 import lib
2
3
4 def main():
5 lib.funct()
ModuleNotFoundError: No module named 'lib'
Why is this happening? Eventually, I'd like that my application is executable with python -m myapp, but also importable.

By default the import searches for module on the paths present in sys.path and one of the path present there is of current directory so
when you executed the main script:
python myapp/myapp.py
The import in myapp.py file searched for lib module in its current directory i.e "myapp" and as lib.py is in same directory, it executed perfectly
But when you imported myapp as a package in IPython,
The import in myapp.py file searches from IPython's path, where Lib.py is not present hence the classic no module found.
There are few things you can do here
use relevant path in myapp.py like
from . import lib
Note: This will generate error if you executed myapp.py directly as a script so handle accordingly
update the sys.path by appending lib.py's path (Not Recommended)
sys.path.append("....../lib.py")
watch this for clear understanding.
Also just to point out, __name__=="__main__" is only true when you execute the file directly. so the code inside if in myapp.py will not work when you'll use it as a module in package.
I hope this was helpful :)

Understanding how Python imports modules from a subdirectory while using VS-Code

Consider the following directory structure with the following contents:
my-project
├── main.py
├── lib
│ ├── __init__.py
│ ├── foo.py
│ └── bar.py
main.py
import sys
sys.path.append("./lib")
from foo import hello_world
if __name__ == "__main__":
hello_world()
foo.py
from lib.bar import name
def hello_world():
print(f"Hello {name}")
if __name__ == "__main__":
hello_world()
bar.py
name = "Grogu"
When I run main.py, I run into no errors and I get what I expect: Hello Grogu. However, when I run lib/foo.py, I get an error:
Traceback (most recent call last):
File "my-project/lib/foo.py", line 1, in <module>
from lib.bar import name
ModuleNotFoundError: No module named 'lib'
If I change lib.bar into bar, it works as expected, but I don't understand why VSCode doesn't seem to know what the lib folder is.
I'm using VSCode and I think there's something I'm doing wrong. My friend who uses pyCharm is able to run lib/foo.py just fine. It might have to be how VSCode and Pycharm interpret the source. Does anyone have any tips?
Basically I'm trying to import functions from a subdirectory and also preserve the ability to run individual files within the subdirectory, but I can't seem to find the proper way to do this in VScode.

When you execute a Python script, the specific directory where the script is located is added to the path, but not its parent's directory.
So, when you run python foo.py from inside lib the path will contain .../my-project/lib but not .../my-project, and therefore Python cannot find the module lib. However, it works with bar because the file does exist in .../my-project/lib.
To be able to run from both the root dir and lib, you can insert/append the parent directory (i.e. the root dir) to the path in the lib __init__.py:
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent))
Then, change main.py:
from lib.foo import hello_world
if __name__ == "__main__":
hello_world()
And foo.py:
from bar import name
def hello_world():
print(f"Hello {name}")
if __name__ == "__main__":
hello_world()

Just import bar directly:
foo and bar are both the modules that at the same level, there's no need to back to lib, then to search bar. We can directly run from bar import name.
More detailed information please refer to Python Docs - The import System

How to properly call functions defined in other directories in Python?

Sorry, this has been already answered for sure, but I cannot find the answer to my problem... I want to make two separate scripts callable. Let me explain in detail with an example.
I have a directory structure similar to this:
maindir
|- subdir
| |- script.py
| `- myfunc.py
`- main.py
with the following content:
In myfunc.py there is
def myverynicefunc():
print('Hello, I am your very nice func :)')
in script.py there is
import myfunc
def scriptfunc():
print('I am the script function :)')
myfunc.myverynicefunc()
and in main.py there is
from subdir.script import scriptfunc
scriptfunc()
If I go to the subdir directory and execute the script it works, I mean:
.../main_dir/subdir$ python3 script.py
Hello, I am your very nice func :)
However if I try to execute the main.py script it fails:
.../main_dir$ python3 main.py
Traceback (most recent call last):
File "main.py", line 1, in <module>
from subdir.script import scriptfunc
File "/home/alf/Escritorio/main_dir/subdir/script.py", line 1, in <module>
import myfunc
ModuleNotFoundError: No module named 'myfunc'
If I modify the content of script.py to
from . import myfunc
def scriptfunc():
print('I am the script function :)')
myfunc.myverynicefunc()
now the situation is the inverse, the main.py script works ok:
.../main_dir$ python3 main.py
Hello, I am your very nice func :)
I am the script function :)
but the script.py script fails:
.../main_dir/subdir$ python3 script.py
Traceback (most recent call last):
File "script.py", line 1, in <module>
from . import myfunc
ImportError: cannot import name 'myfunc'
Is there a way to make both calls to main.py and to script.py to work?

Try this in your script.py-
import sys
current_path = sys.path[0]
if current_path.split('/')[-1] != 'subdir':
sys.path.insert(0, current_path+'/subdir/')
import myfunc
By this, if the current directory for python is the parent directory of the file, that is the maindir, it would change the path to directory and then import it.
With this, it should work in both scenarios. Hope this helps. :)

with the second scenario, you can do
python3 -c "import subdir.script" in your main directory

Splitting my code into multiple files in Python 3

I wish to split my code into multiple files in Python 3.
I have the following files:
/hello
__init__.py
first.py
second.py
Where the contents of the above files are:
first.py
from hello.second import say_hello
say_hello()
second.py
def say_hello():
print("Hello World!")
But when I run:
python3 first.py
while in the hello directory I get the following error:
Traceback (most recent call last):
File "first.py", line 1, in <module>
from hello.second import say_hello
ImportError: No module named 'hello'

Swap out
from hello.second import say_hello
for
from second import say_hello
Your default Python path will include your current directory, so importing straight from second will work. You don't even need the __init__.py file for this. You do, however, need the __init__.py file if you wish to import from outside of the package:
$ python3
>>> from hello.second import say_hello
>>> # Works ok!

You shouldn't run python3 in the hello directory.
You should run it outside the hello directory and run
python3
>>> import hello.first
By the way, __init__.py is no longer needed in Python 3. See PEP 420.

Packages are not meant to be imported from the current directory.
It is possible to make it work using if/else tests or try/except handlers, but it's more work than it is worth.
Just cd .. so you aren't in the package's directory and it will work fine.

Can anyone explain python's relative imports? [duplicate]

This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed 2 years ago.
I can't for the life of me get python's relative imports to work. I have created a simple example of where it does not function:
The directory structure is:
__init__.py
start.py
parent.py
sub/
__init__.py
relative.py
/start.py contains just: import sub.relative
/sub/relative.py contains just from .. import parent
All other files are blank.
When executing the following on the command line:
$ cd /
$ python start.py
I get:
Traceback (most recent call last):
File "start.py", line 1, in <module>
import sub.relative
File "/home/cvondrick/sandbox/sub/relative.py", line 1, in <module>
from .. import parent
ValueError: Attempted relative import beyond toplevel package
I am using Python 2.6. Why is this the case? How do I make this sandbox example work?

You are importing from package "sub". start.py is not itself in a package even if there is a __init__.py present.
You would need to start your program from one directory over parent.py:
./start.py
./pkg/__init__.py
./pkg/parent.py
./pkg/sub/__init__.py
./pkg/sub/relative.py
With start.py:
import pkg.sub.relative
Now pkg is the top level package and your relative import should work.
If you want to stick with your current layout you can just use import parent. Because you use start.py to launch your interpreter, the directory where start.py is located is in your python path. parent.py lives there as a separate module.
You can also safely delete the top level __init__.py, if you don't import anything into a script further up the directory tree.

If you are going to call relative.py directly and i.e. if you really want to import from a top level module you have to explicitly add it to the sys.path list.
Here is how it should work:
# Add this line to the beginning of relative.py file
import sys
sys.path.append('..')
# Now you can do imports from one directory top cause it is in the sys.path
import parent
# And even like this:
from parent import Parent
If you think the above can cause some kind of inconsistency you can use this instead:
sys.path.append(sys.path[0] + "/..")
sys.path[0] refers to the path that the entry point was ran from.

Checking it out in python3:
python -V
Python 3.6.5
Example1:
.
├── parent.py
├── start.py
└── sub
└── relative.py
- start.py
import sub.relative
- parent.py
print('Hello from parent.py')
- sub/relative.py
from .. import parent
If we run it like this(just to make sure PYTHONPATH is empty):
PYTHONPATH='' python3 start.py
Output:
Traceback (most recent call last):
File "start.py", line 1, in <module>
import sub.relative
File "/python-import-examples/so-example-v1/sub/relative.py", line 1, in <module>
from .. import parent
ValueError: attempted relative import beyond top-level package
If we change import in sub/relative.py
- sub/relative.py
import parent
If we run it like this:
PYTHONPATH='' python3 start.py
Output:
Hello from parent.py
Example2:
.
├── parent.py
└── sub
├── relative.py
└── start.py
- parent.py
print('Hello from parent.py')
- sub/relative.py
print('Hello from relative.py')
- sub/start.py
import relative
from .. import parent
Run it like:
PYTHONPATH='' python3 sub/start.py
Output:
Hello from relative.py
Traceback (most recent call last):
File "sub/start.py", line 2, in <module>
from .. import parent
ValueError: attempted relative import beyond top-level package
If we change import in sub/start.py:
- sub/start.py
import relative
import parent
Run it like:
PYTHONPATH='' python3 sub/start.py
Output:
Hello from relative.py
Traceback (most recent call last):
File "sub/start.py", line 3, in <module>
import parent
ModuleNotFoundError: No module named 'parent'
Run it like:
PYTHONPATH='.' python3 sub/start.py
Output:
Hello from relative.py
Hello from parent.py
Also it's better to use import from root folder, i.e.:
- sub/start.py
import sub.relative
import parent
Run it like:
PYTHONPATH='.' python3 sub/start.py
Output:
Hello from relative.py
Hello from parent.py