I have the following Python package with 2 moludes:
-pack1
|-__init__
|-mod1.py
|-mod2.py
-import_test.py
with the code:
# in mod1.py
a = 1
and
# in mod2.py
from mod1 import a
b = 2
and the __init__ code:
# in __init__.py
__all__ = ['mod1', 'mod2']
Next, I am trying to import the package:
# in import_test.py
from pack1 import *
But I get an error:
ModuleNotFoundError: No module named 'mod1'
If I remove the dependency "from mod1 import a" in mod2.py, the import goes correctly. But that dependency makes the import incorrect with that exception "ModuleNotFoundError".
???
The issue here is that from mod2 perspective the first level in which it will search for a module is in the path from which you are importing it (here I am assuming that pack1 is not in your PYTHONPATH and that you are importing it from the same directory where pack1 is contained).
This means that if pack1 is in the directory /dir/to/pack1 and you do:
from mod1 import a
Python will look for mod1 in the same directory as pack1, i.e., /dir/to/pack1.
To solve your issue it is enough to do either:
from pack1.mod1 import a
or in Python 3.5+
from .mod1 import a
As a side note, unless this is a must for you, I do not recommend designing your package to be used as from pack import *, even if __all__ exists to give you better control of your public API.
Related
I have the following project structure:
- workflow/
file1.ipynb
file2.ipynb
...
- utils/
__init__.py
function_one.py
function_two.py
...
I am working on file1.ipynb, so far I have found a way to import the variables defined in init.py through the following code:
utils = importlib.machinery.SourceFileLoader('utils', '/home/utils/__init__.py').load_module()
Let's assume my __init__.py contains the following:
from .function_one import *
I can then use the variables defined inside the __init__.py file.
However, every time I want to call any of these variables I need to use the following syntax:
utils.function_one ...
I want to be able to write function_one without the utils at the beginning.
How can I import directly the variables defined inside the __init__.py ?
I don't know why you don't import your module with the normal import mechanism: from ..utils import * or depending on where your python interpreter was started just from utils import * but if you insist on using utils = importlib.machinery.SourceFileLoader('utils', '/home/utils/__init__.py').load_module() you can hack all values into your globals like this:
tmp = globals()
for attr_name in dir(utils):
if not attr_name.startswith("_"): # don't import private and dunder attributes
tmp[attr_name] = getattr(utils, attr_name)
del tmp
function_one(...) # should work now
Try this:
from ..utils import *
I am making a bot in python 3 and wish it to be easily expanded so I have a central file and then one for each command. I wish to know if there is a way to import a sub-directory full of modules without importing each separately. For example:
example
├── commands
│ ├── bar.py
│ └── foo.py
└── main.py
And the code in main.pywould be something like:
import /commands/*
Thanks :D
Solution:
Import each separately with:
from commands import foo, bar
from commands import * Does not work.
If you're using python3, the importlib module can be used to dynamically import modules. On python2.x, there is the __import__ function but I'm not very familiar with the semantics. As a quick example,
I have 2 files in the current directory
# a.py
name = "a"
and
# b.py
name = "b"
In the same directory, I have this
import glob
import importlib
for f in glob.iglob("*.py"):
if f.endswith("load.py"):
continue
mod_name = f.split(".")[0]
print ("importing {}".format(mod_name))
mod = importlib.import_module(mod_name, "")
print ("Imported {}. Name is {}".format(mod, mod.name))
This will print
importing b Imported <module 'b' from '/tmp/x/b.py'>.
Name is b
importing a Imported <module 'a' from '/tmp/x/a.py'>.
Name is a
Import each separately with:
from commands import bar and
from commands import foo
from commands import * Does not work.
I'm trying to create an auto_import function which is part of a library: the purpose of this to avoid listing from .x import y many times in __init__ files, only do something this import lib; lib.auto_import(__file__) <- this would search for python files in that folder where the __init__ is present and would import all stuff by exec statement (i.e. exec('from .x import abc')).
My problem is that, somehow the 'from' statement always tries to import .x from lib directory, even if I change the cwd to the directory where the actual __init__ file is placed... How should I solve this? How should I change the search dir for from . statement?
Structure:
$ ls -R
.:
app.py lib x
./lib:
__init__.py auto_import.py
./x:
__init__.py y
./x/y:
__init__.py y.py
e.g.: ./x/y/__init__.py contains import lib; lib.auto_import(__file__)
auto_import is checking for files in dir of __file__ and import them with exec('from .{} import *') (but this from . is always the lib folder and not the dir of __file__, and that is my question, how to change this to dir of __file__
Of course the whole stuff is imported in app.py like:
import x
print(x.y)
Thanks
EDIT1: final auto_import (globals() / gns cannot be avoided )
import os, sys, inspect
def auto_import(gns):
current_frame = inspect.currentframe()
caller_frame = inspect.getouterframes(current_frame)[1]
src_file = caller_frame[1]
for item in os.listdir(os.path.dirname(src_file)):
item = item.split('.py')[0]
if item in ['__init__', '__pycache__']:
continue
gns.update(__import__(item, gns, locals(), ['*'], 1).__dict__)
The problem of your approach is that auto_import is defined in lib/auto_import.py so the context for exec('from .x import *') is always lib/. Even though you manage to fix the path problem, lib.auto_import(__file__) will not import anything to the namespace of lib.x.y, because the function locates in another module.
Use the built-in function __import__
Here is the auto_import script:
myimporter.py
# myimporter.py
def __import_siblings__(gns, lns={}):
for name in find_sibling_names(gns['__file__']):
gns.update((k,v) for k,v in __import__(name, gns,lns).__dict__.items() if not k.startswith('_'))
import re,os
def find_sibling_names(filename):
pyfp = re.compile(r'([a-zA-Z]\w*)\.py$')
files = (pyfp.match(f) for f in os.listdir(os.path.dirname(filename)))
return set(f.group(1) for f in files if f)
Inside your lib/x/y/__init__.py
#lib/x/y/__init__.py
from myimporter import __import_siblings__
__import_siblings__(globals())
Let's say you have a dummy module that need to be imported to y:
#lib/x/y/dummy.py
def hello():
print 'hello'
Test it:
import x.y
x.y.hello()
Please be aware that from lib import * is usually a bad habit because of namespace pollution. Use it with caution.
Refs:
1
2
Use sys module
With this folder structure:
RootDir
|-- module.py
|--ChildDir
|-- main.py
Now in main.py you can do
import sys
sys.path.append('..')
import module
A believe there are other hacks, but this is the one I know of and works for my purpose. I am not sure, whether it is the best option to go for some kind of auto_import stuff though.
I have created my own module with filename mymodule.py. The file contains:
def testmod():
print "test module success"
I have placed this file within /Library/Python/2.7/site-packages/mymodule/mymodule.py
I have also added a __init__.py file, these have compiled to generate
__init__.pyc and mymodule.pyc
Then in the python console I import it
import mymodule
which works fine
when I try to use mymodule.testmod() I get the following error:
AttributeError: 'module' object has no attribute 'testmod'
So yeah it seems like it has no functions?
You have a package mymodule, containing a module mymodule. The function is part of the module, not the package.
Import the module:
import mymodule.mymodule
and reference the function on that:
mymodule.mymodule.testmod()
You can use from ... import and import ... as to influence what gets imported exactly:
from mymodule import mymodule
mymodule.testmod()
or
from mymodule import mymodule as nestedmodule
nestedmodule.testmod
or
from mymodule.mymodule import testmod
testmod()
etc.
For:
app/
__init__.py
abc.py
mod/
__init__.py
def.py
How would I import abc.py from def.py?
to import module 'abc.py' that is in the parent directory of your current module:
import os
parentdir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
os.sys.path.insert(0,parentdir)
import abc
in module def if you want to import say abc just do:
from ..abc import *
Note: as def is a python keyword, using that name for a module does not sound like a good idea.