I am trying to add 'project_root' into __init__.py and all modules can use it, but it doesn't work.
Environment: Python 3.7.0 MACOS MOJAVE
file structure
·
├── __init__.py
└── a.py
the codes in __init__.py file:
import sys
project_root = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
sys.path.append(project_root)
and in another file
print(project_root)
If I run python a.py in the same dir ,or run python a.py out of the dir, error are the same below:
Traceback (most recent call last):
File "a.py", line 1, in <module>
print(project_root)
NameError: name 'project_root' is not defined
My question is why it doesn't work and how to fix it.
Another question is what if you want to share some variables for other modules in a same package, how to do it?
Let us try to understand by example.
Code and directory explanation:
Suppose we have the following directory and file structure:
dir_1
├── __init__.py
└── a.py
b.py
__init__.py contains:
import sys,os
# Just to make things clear
print("Print statement from init")
project_root = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
sys.path.append(project_root)
a.py contains:
def func():
print("func from a.py")
Let us start importing things:
Suppose you start with having the following code in b.py:
from dir_1.a import func
func()
Executing the above will give the following output:
Print statement from init
func from a.py
So, from the above, we understand that the print statement from __init__.py is being executed. Now, let's add print(project_root) in b.py:
from dir_1.a import func
func()
print(project_root)
Executing the above will result in an error saying:
...
NameError: name 'project_root' is not defined
This happened because we did not have to import the print statement from __init__.py it just gets executed. But that is not the case for a variable.
Let's try to import the variable and see what happens:
from dir_1.a import func
from dir_1 import project_root
func()
print(project_root)
Executing the above file will give the following output:
Print statement from init
func from a.py
/home/user/some/directory/name/dir_1
Long story short, you need to import the variable defined in __init__.py
or anywhere else in order to use it.
Hope this helps : )
You have to import the variable:
from dir import project_root
Where dir is the directory you are in
Related
Yet another python imports question.
I have a directory structure
thing
|- __init__.py
|- run.py
|- mod
|- __init__.py
|- what
|- __init__.py
|- yo.py
The contents of yo.py are
class Yo:
def __init__(self):
print("initialized What")
And the contents of my run.py are
from mod.what import yo
y = yo.Yo
print(y)
y()
Everything works great.
<class 'mod.what.yo.Yo'>
initialized What
But I need to import like this:
from mod import what
y = what.yo.Yo
print(y)
y()
and then I get
Traceback (most recent call last):
File "/Users/pavelkomarov/Desktop/thing/run.py", line 4, in <module>
y = what.yo.Yo
AttributeError: module 'mod.what' has no attribute 'yo'
Why? I've got __init__.pys out the wazoo. Shouldn't python be able to find the class? This structure is important to me because I have a lot of classes below a certain module and need to be able to access them all, preferably without having to do more granular imports of each one, which takes a ton of code.
Thanks to Maurice and the people over at that other thread.
I've found putting
import importlib
import pkgutil
for mod_info in pkgutil.walk_packages(__path__, __name__ + '.'):
mod = importlib.import_module(mod_info.name)
# Emulate `from mod import *`
try:
names = mod.__dict__['__all__']
except KeyError:
names = [k for k in mod.__dict__ if not k.startswith('_')]
in mod's __init__.py does the trick. It also works if you put it in what's __init__.py.
I disagree that this is anything close to Zen, but it works.
I am running multiple tests in a tests package, and I want to print each module name in the package, without duplicating code.
So, I wanted to insert some code to __init__.py or conftest.py that will give me the executing module name.
Let's say my test modules are called: checker1, checker2, etc...
My directory structure is like this:
tests_dir/
├── __init__.py
├── conftest.py
├── checker1
├── checker2
└── checker3
So, inside __init__.py I tried inserting:
def module_name():
return os.path.splitext(__file__)[0]
But it still gives me __init__.py from each file when I call it.
I also tried using a fixture inside conftest.py, like:
#pytest.fixture(scope='module')
def module_name(request):
return request.node.name
But it seems as if I still need to define a function inside each module to get module_name as a parameter.
What is the best method of getting this to work?
Edit:
In the end, what I did is explained here:
conftest.py
#pytest.fixture(scope='module', autouse=True)
def module_name(request):
return request.node.name
example for a test file with a test function. The same needs to be added to each file and every function:
checker1.py
from conftest import *
def test_columns(expected_res, actual_res, module_name):
expected_cols = expected_res.columns
actual_cols = actual_res.columns
val = expected_cols.difference(actual_cols) # verify all expected cols are in actual_cols
if not val.empty:
log.error('[{}]: Expected columns are missing: {}'.format(module_name, val.values))
assert val.empty
Notice the module_name fixture I added to the function's parameters.
expected_res and actual_res are pandas Dataframes from excel file.
log is a Logger object from logging package
In each module (checker1, checker2, checker3, conftest.py), in the main function, execute
print(__name__)
When the __init__.py file imports those packages, it should print the module name along with it.
Based on your comment, you can perhaps modify the behaviour in the __init__.py file for local imports.
__init.py__
import sys, os
sys.path.append(os.path.split(__file__)[0])
def my_import(module):
print("Module name is {}".format(module))
exec("import {}".format(module))
testerfn.py
print(__name__)
print("Test")
Directory structure
tests_dir/
├── __init__.py
└── testerfn.py
Command to test
import tests_dir
tests_dir.my_import("testerfn")
I have a module - let's call it foo - and I want to make it usable via a python -m foo call. My program look like this:
my_project
├── foo
│ └── __init__.py
└── my_program.py
In __init__.py I have some code which I run when calling python -m foo:
def bar(name):
print(name)
# -- code used to 'run' the module
def main(name):
bar("fritz")
if __name__ == "__main__":
main()
Since I have a fair amount of execution code in __init__.py now (argparse stuff and some logic) I want to separate it into a __main__.py:
my_project
├── foo
│ ├── __init__.py
│ └── __main__.py
└── my_program.py
Despite that looks very simple to me I didn't manage to import stuff located in __init__.py from __main__.py yet.
I know - if foo is located in site-packages or accessible via PYTHONPATH I can just import foo..
But in case I want to execute __main__.py directly (e.g. from some IDE) with foo located anywhere (i.e. not a folder where Python looks for packages) - is there a way to import foo (__init__.py from the same directory)?
I tried import . and import foo - but both approaches fail (because they just mean something else of course)
What I can do - at least to explain my goal - is something like this:
sys.path.append(os.path.join(os.path.dirname(__file__), ".."))
import foo
Works, but is ugly and a bit dangerous since I don't even know if I really import foo from the same directory..
You can manually set the module import state as if __main__.py were executed with -m:
# foo/__main__.py
import os
import sys
if __package__ is None and __name__ == "__main__": # executed without -m
# set special attributes as if part of the package
__file__ = os.path.abspath(__file__)
__package__ = os.path.basename(os.path.dirname(__file__))
# replace import path for __main__ with path for package
main_path = os.path.dirname(__file__)
try:
index = sys.path.index(dir_path)
if index != 0 or index != 1:
raise ValueError('expected script directory after current directory or matching it')
except ValueError:
raise RuntimeError('sys.path does not include script directory as expected')
else:
sys.path[index] = main_path
# import regularly
from . import bar
This exploits that python3 path/to/foo/__main__.py executes __main__ as a standalone script: __package__ is None and the __name__ does not include the package either. The search path in this case is <current directory>, <__main__ directory>, ..., though it gets collapsed if the two are the same: the index is either 0 or 1.
As with all trickery on internals, there is some transient state where invariants are violated. Do not perform any imports before the module is patched!
I have some like this:
/package
__init__.py
file1.py
file2.py
and inside __init__.py I define some_variable=True
When I try to import some_variable inside my files
from . import some_variable
I get ImportError: cannot import name some_variable.
I get the same error if I do
from folder.package import some_variable
Why am I not able to import this variable?
The really confusing/frustrating part is that PyCharm's autocomplete recognises that the variable exists.
I am making a bot in python 3 and wish it to be easily expanded so I have a central file and then one for each command. I wish to know if there is a way to import a sub-directory full of modules without importing each separately. For example:
example
├── commands
│ ├── bar.py
│ └── foo.py
└── main.py
And the code in main.pywould be something like:
import /commands/*
Thanks :D
Solution:
Import each separately with:
from commands import foo, bar
from commands import * Does not work.
If you're using python3, the importlib module can be used to dynamically import modules. On python2.x, there is the __import__ function but I'm not very familiar with the semantics. As a quick example,
I have 2 files in the current directory
# a.py
name = "a"
and
# b.py
name = "b"
In the same directory, I have this
import glob
import importlib
for f in glob.iglob("*.py"):
if f.endswith("load.py"):
continue
mod_name = f.split(".")[0]
print ("importing {}".format(mod_name))
mod = importlib.import_module(mod_name, "")
print ("Imported {}. Name is {}".format(mod, mod.name))
This will print
importing b Imported <module 'b' from '/tmp/x/b.py'>.
Name is b
importing a Imported <module 'a' from '/tmp/x/a.py'>.
Name is a
Import each separately with:
from commands import bar and
from commands import foo
from commands import * Does not work.