What is the __main__.py file for, what sort of code should I put into it, and when should I have one?
Often, a Python program is run by naming a .py file on the command line:
$ python my_program.py
You can also create a directory or zipfile full of code, and include a __main__.py. Then you can simply name the directory or zipfile on the command line, and it executes the __main__.py automatically:
$ python my_program_dir
$ python my_program.zip
# Or, if the program is accessible as a module
$ python -m my_program
You'll have to decide for yourself whether your application could benefit from being executed like this.
Note that a __main__ module usually doesn't come from a __main__.py file. It can, but it usually doesn't. When you run a script like python my_program.py, the script will run as the __main__ module instead of the my_program module. This also happens for modules run as python -m my_module, or in several other ways.
If you saw the name __main__ in an error message, that doesn't necessarily mean you should be looking for a __main__.py file.
What is the __main__.py file for?
When creating a Python module, it is common to make the module execute some functionality (usually contained in a main function) when run as the entry point of the program. This is typically done with the following common idiom placed at the bottom of most Python files:
if __name__ == '__main__':
# execute only if run as the entry point into the program
main()
You can get the same semantics for a Python package with __main__.py, which might have the following structure:
.
└── demo
├── __init__.py
└── __main__.py
To see this, paste the below into a Python 3 shell:
from pathlib import Path
demo = Path.cwd() / 'demo'
demo.mkdir()
(demo / '__init__.py').write_text("""
print('demo/__init__.py executed')
def main():
print('main() executed')
""")
(demo / '__main__.py').write_text("""
print('demo/__main__.py executed')
from demo import main
main()
""")
We can treat demo as a package and actually import it, which executes the top-level code in the __init__.py (but not the main function):
>>> import demo
demo/__init__.py executed
When we use the package as the entry point to the program, we perform the code in the __main__.py, which imports the __init__.py first:
$ python -m demo
demo/__init__.py executed
demo/__main__.py executed
main() executed
You can derive this from the documentation. The documentation says:
__main__ — Top-level script environment
'__main__' is the name of the scope in which top-level code executes.
A module’s __name__ is set equal to '__main__' when read from standard
input, a script, or from an interactive prompt.
A module can discover whether or not it is running in the main scope
by checking its own __name__, which allows a common idiom for
conditionally executing code in a module when it is run as a script or
with python -m but not when it is imported:
if __name__ == '__main__':
# execute only if run as a script
main()
For a package, the same effect can be achieved by including a
__main__.py module, the contents of which will be executed when the module is run with -m.
Zipped
You can also zip up this directory, including the __main__.py, into a single file and run it from the command line like this - but note that zipped packages can't execute sub-packages or submodules as the entry point:
from pathlib import Path
demo = Path.cwd() / 'demo2'
demo.mkdir()
(demo / '__init__.py').write_text("""
print('demo2/__init__.py executed')
def main():
print('main() executed')
""")
(demo / '__main__.py').write_text("""
print('demo2/__main__.py executed')
from __init__ import main
main()
""")
Note the subtle change - we are importing main from __init__ instead of demo2 - this zipped directory is not being treated as a package, but as a directory of scripts. So it must be used without the -m flag.
Particularly relevant to the question - zipapp causes the zipped directory to execute the __main__.py by default - and it is executed first, before __init__.py:
$ python -m zipapp demo2 -o demo2zip
$ python demo2zip
demo2/__main__.py executed
demo2/__init__.py executed
main() executed
Note again, this zipped directory is not a package - you cannot import it either.
Some of the answers here imply that given a "package" directory (with or without an explicit __init__.py file), containing a __main__.py file, there is no difference between running that directory with the -m switch or without.
The big difference is that without the -m switch, the "package" directory is first added to the path (i.e. sys.path), and then the files are run normally, without package semantics.
Whereas with the -m switch, package semantics (including relative imports) are honoured, and the package directory itself is never added to the system path.
This is a very important distinction, both in terms of whether relative imports will work or not, but more importantly in terms of dictating what will be imported in the case of unintended shadowing of system modules.
Example:
Consider a directory called PkgTest with the following structure
:~/PkgTest$ tree
.
├── pkgname
│ ├── __main__.py
│ ├── secondtest.py
│ └── testmodule.py
└── testmodule.py
where the __main__.py file has the following contents:
:~/PkgTest$ cat pkgname/__main__.py
import os
print( "Hello from pkgname.__main__.py. I am the file", os.path.abspath( __file__ ) )
print( "I am being accessed from", os.path.abspath( os.curdir ) )
from testmodule import main as firstmain; firstmain()
from .secondtest import main as secondmain; secondmain()
(with the other files defined similarly with similar printouts).
If you run this without the -m switch, this is what you'll get. Note that the relative import fails, but more importantly note that the wrong testmodule has been chosen (i.e. relative to the working directory):
:~/PkgTest$ python3 pkgname
Hello from pkgname.__main__.py. I am the file ~/PkgTest/pkgname/__main__.py
I am being accessed from ~/PkgTest
Hello from testmodule.py. I am the file ~/PkgTest/pkgname/testmodule.py
I am being accessed from ~/PkgTest
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "pkgname/__main__.py", line 10, in <module>
from .secondtest import main as secondmain
ImportError: attempted relative import with no known parent package
Whereas with the -m switch, you get what you (hopefully) expected:
:~/PkgTest$ python3 -m pkgname
Hello from pkgname.__main__.py. I am the file ~/PkgTest/pkgname/__main__.py
I am being accessed from ~/PkgTest
Hello from testmodule.py. I am the file ~/PkgTest/testmodule.py
I am being accessed from ~/PkgTest
Hello from secondtest.py. I am the file ~/PkgTest/pkgname/secondtest.py
I am being accessed from ~/PkgTest
Note: In my honest opinion, running without -m should be avoided. In fact I would go further and say that I would create any executable packages in such a way that they would fail unless run via the -m switch.
In other words, I would only import from 'in-package' modules explicitly via 'relative imports', assuming that all other imports represent system modules. If someone attempts to run your package without the -m switch, the relative import statements will throw an error, instead of silently running the wrong module.
You create __main__.py in yourpackage to make it executable as:
$ python -m yourpackage
__main__.py is used for python programs in zip files. The __main__.py file will be executed when the zip file in run. For example, if the zip file was as such:
test.zip
__main__.py
and the contents of __main__.py was
import sys
print "hello %s" % sys.argv[1]
Then if we were to run python test.zip world we would get hello world out.
So the __main__.py file run when python is called on a zip file.
If your script is a directory or ZIP file rather than a single python file, __main__.py will be executed when the "script" is passed as an argument to the python interpreter.
Related
I created a python project, using Pycharm if it matters, that looks like this:
outer_dir/
outside_main.py
module_example/
module_main.py
tests/
test_1.py
…
level_1/
foo_1.py
level_2/
foo_2.py
main.py calls functions from foo_1.py
foo_1.py calls functions from foo_2.py
The general use case I'm trying to emulate is a python "package" or repo called module_example that has been downloaded and used by outside_main.py at the same level as the top-level package directory.
Here's how module_main.py looks:
# module_main.py
from level_1.foo_1 import foo_function_1
from level_1.level_2.foo_2 import foo_function_2
def main():
print(foo_1())
print(foo_2())
if __name__ == "__main__":
main()
And I can call this top level script from test_1.py just fine like this:
# test_1.py
from module_main import main
if __name__ == "__main__":
main()
However, when I try to use main() from outside the module_example directory, with code like this:
# outside_main.py
from module_example.module_main import main
if __name__ == "__main__":
main()
I get this error:
Traceback (most recent call last):
File "/path/to/outside_main.py", line 1, in <module>
from module_example.module_main import main
File "/path/to/module_example/module_main.py", line 1, in <module>
from level_1.foo_1 import foo_function_1
ModuleNotFoundError: No module named 'level_1'
What seems to be happening is that regardless of directory of the python script being imported, its own imports are being searched for relative to the directory from which the main script is executed.
I'm looking for a method of importing that works equally whether I'm calling outside_main.py or test_1.py, and regardless of what directory the terminal is currently pointing at.
I've read this post about how python imports work, and it basically suggests I'm out of luck, but that makes no sense to me, all I'm trying to do is emulate using someone's open source library from github, they usually have a tests folder within their repo, that works fine, but also I can just unzip the entire repo in my project and call import on their py files without issue. How do they do it?
Before entering the hell of sys.path-patching, I would suggest to wrap your code that currently lives in module_example into a package_example directory, add a setup.py and throw a number of __init__.py in.
Make package_example a git repo.
In every other project where you want to use module_example.[...], you create a venv and pip -e /path/to/package_example.
outer_dir/
outside_main.py
package_example/
.git/
...
setup.py
module_example/
__init__.py
module_main.py
tests/
...
where __init__.py (one in each subdirectory that contains modules that are to be imported) can just be empty files and setup.py has to be filled according to the distutils documentation.
Solution 1:
Use explicit relative imports (relative to the files inside the directory module_example). Note that the usage of implicit relative imports has already been removed in Python3
Implicit relative imports should never be used and have been removed in Python 3.
outer_dir/module_example/module_main.py
# Note the "." in the beginning signifying it is located in the current folder (use ".." to go to parent folder)
from .level_1.foo_1 import foo_function_1
from .level_1.level_2.foo_2 import foo_function_2
# All the rest the same
Solution 2:
Include the path to module_example in your PYTHONPATH environment variable (or you can also choose to update sys.path directly)
export PYTHONPATH=$(pwd):$(pwd)/module_example
Output:
Both solutions above was successful for me.
Before the fix:
$ python3 outside_main.py
Traceback (most recent call last):
File "outside_main.py", line 1, in <module>
from module_example.module_main import main
File "/home/nponcian/Documents/PearlPay/Program/2021_07Jul_31_StackOverflow/1/module_example/module_main.py", line 1, in <module>
from level_1.foo_1 import foo_function_1
ModuleNotFoundError: No module named 'level_1'
After the fix:
$ python3 outside_main.py
This is foo_function_1
This is foo_function_2
I have __init__.py in a folder called test_module. in the __init__.py i have below code. however when i try to execute from parent folder of test_module with following command python test_module i get following error can't find '__main__' module in 'test_module. is this not possible? or will i have to run python test_module/__init__.py?
def main():
print('test')
if __name__ == '__main__':
main()
The __init__.py module is executed when the package is imported. The purpose of __init__.py files per the documentation is as follows:
The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later.
In order for a Python package to be directly executed, it needs to have an entry point, designated by a module within the package named __main__.py. Thus the error can't find '__main__' module in 'test_module': You have attempted to directly execute the package, but Python cannot locate an entry point to begin executing top-level code.
Consider the following package structure:
test_module/
__init__.py
__main__.py
Where __init__.py contains the following:
print("Running: __init__.py")
Where __main__.py contains the following:
print("Running: __main__.py")
When we execute the test_module package with the command python test_module, we get the following output:
> python test_module
Running: __main__.py
However, if we enter the Python shell and import test_module, the output is as follows:
>>> import test_module
Running: __init__.py
Thus in order to get the behavior you want when attempting to directly execute test_module, simply create a new __main__.py file within test_module and transfer the code from __init__.py to the new __main__.py.
I have a simple python project fro learning with two files __init__.py and __main__.py
When I executed python -m pkg_name
it runs both __init__.py and __main__.py
When I execute python -m pkg_name.__init__.py
it invokes __init__.py twice.
I want to know why __init__.py is called twice when i call __init__.py
Is it like the static code in java where when we call the class all the data
in static code is automatically triggered.
What is the relevance of __init__.py in python and benefits of it getting executed when package is imported/loaded or called for processing.
Please help me understand the concepts better.
"""Run a sequence of programs, testing python code __main__ variable
Each program (except the first) receives standard output of the
previous program on its standard input, by default. There are several
alternate ways of passing data between programs.
"""
def _launch():
print('Pipeline Launched')
if __name__ == '__main__':
print('This module is running as the main module!')
_launch()
> __init__.py
"""This is the __init__.py file of pipleline package
Used for testing the import statements.
"""
print(__name__)
print('This is the __init__.py file of pipleline package')
print('Exiting __init__ of pipeline package after all initialization')
The following command is used to execute a Python module or package:
python -m module
Where module is the name of the module/package without .py extension.
if the name matches a script, it is byte-compiled and executed,
if the name matches a directory with a __init__.py file and a __main__.py file, the directory is considered as being a Python package and is loaded first. Then the __main__.py script is executed.
if the name contains dots, e.g.: "mylib.mypkg.mymodule", Python browse each package and sub-package matching the dotted name and load it. Then it execute the last module (or last package which must contain a __main__.py file).
A (short) description is done in the official documentation: 29.4. main — Top-level script environment.
Your problem
If you run this command:
python -m pkg_name
It loads (and run) the __init__.py and __main__.py: this is the normal behavior.
If you run this command:
python -m pkg_name.__init__.py
It should fail if you leave the ".py" extension.
If it runs, the command loads the pkg_name package first: it execute the __init__.py first. Then it runs it again.
It is used to define a folder as a package, which contains required modules and resources.
You can use is as an empty file or add docs about the package or setup initial conditions for the module.
Please checkout the python documentation.
Also, as mentioned by Natecat, __init__.py gets executed whenever you load a package. That's why when you explicitly call __init__.py, it loads the package (1st load) then executes __init__.py (2nd load).
If this is my directory tree
temp
├── __init__.py
└── __main__.py
0 directories, 2 files
And I have the following code in __init__.py and in __main__.py
__init__.py
"""Initializes the module"""
CONSTANT = 1
sys.exit("what is happening here")
__main__.py
# from . import CONSTANT
# from temp import CONSTANT
if __name__ == "__main__":
print "This should never run"
I am getting two problems here that I am trying to figure out
On running python . in the temp directory I get the output
This should never run, shouldn't the module be initialized first with the __init__.py file resulting in the abort?
Second how do I go about doing imports in python modules? Neither of the two options I have mentioned above works. I can neither do from . import CONSTANT nor from temp import CONSTANT in the code above. What is the right way to do relative imports?
I am running this on Python 2.7.5, apologies if this has already been asked before.
You should be running it from out of the temp directory. If someDir contains your temp directory, then:
someDir $ python -m temp #someDir/temp/__init__.py is your file.
On running python . in the temp directory I get the output This should never run, shouldn't the module be initialized first with the init.py file resulting in the abort?
If you run it from outside, __init__.py will be called. And sys.exit will be called too.
Second how do I go about doing imports in python modules? Neither of the two options I have mentioned above works. I can neither do from . import CONSTANT nor from temp import CONSTANT in the code above. What is the right way to do relative imports?
You are doing it just fine. Just import sys in your __init__.py file. And fix the spelling of CONSTANT.
Also why do I need the -m flag? Isn't it ok to just do python temp from the parent directory of temp?
You need the -m flag to tell that you are using packages. If you dont use it you wont be able to do relative imports.
When you tell Python to run a directory, Python does not treat the directory as a package. Instead, Python adds that directory to sys.path and runs its __main__.py. __init__.py is not executed, and relative imports will not view the directory as a package.
If you want to run a package's __main__.py and treat it as part of the package, with __init__.py executed and all, go to the directory containing the package and run
python -m packagename
You are running inside temp; this is not considered a package and __init__.py is not loaded. Only if the parent of the current directory is on the module loading path and you explicitly load temp as a module, is __init__.py loaded.
Because temp is not a package you can't use relative imports here. Instead, every Python file inside of the directory is considered a top-level module all by themselves.
You'd have move to the parent of the temp directory, then run:
python -m temp
for Python to import temp as a package and then run the __main__ module in that package.
I have written a library with the following structure:
libProject/
setup.py
libname/
__init__.py
script1.py
script2.py
...
I use it primarily by importing it
from libname.script1 import Fun1
...
I would now like to make a command line interface to it, but I'm confused where do I put my argpase code within this project so that I can use
$ python libname <somecommand>
I found a solution. Would still keep it open for better answers.
In the libname directory, touch a new file called __main__.py
libname.__main__ will be executed when libname directory is called as script.
Now put your command line code inside this file
#__main__.py
import sys
from .script1 import Fun1
print Fun1(sys.arg[-1])