I have following file structure
./AppFolder
__init__.py
main.py
./plugin
__init__.py
./simple_plugin_1
__init__.py
simple_plugin_1.py
./simple_plugin_2
__init__.py
simple_plugin_2.py
and i want to load all plugin modules recursively into main.py. So can I use __init__.py in AppFolder as following?
import os
import glob
__all__ = [ os.path.basename(f)[:-3] for f in glob.glob(os.path.dirname(__file__)+"/plugin/*/*.py")]
Or is there are other techniques?
I have this bit of code that I use for this purpose, just use this as your top level __init__.py:
import pkgutil
__all__ = []
for loader, module_name, is_pkg in pkgutil.walk_packages(__path__):
module = loader.find_module(module_name).load_module(module_name)
exec('%s = module' % module_name)
__all__.append(module_name)
Basically same idea as what you have, but just using pkgutil.walk_packages to find all the modules instead of a file glob.
Related
I am trying to import python module from one directory to another directory but not able to do it.
Here is my folder structure
Parent_Folder
---------Module1
----__init__.py (empty file)
----mymodule1.py
---------Module2
----__init__.py (empty file)
----mymodule2.py
I am trying to import
mymodule1.py
into
mymodule2.py
Here is the command that I have used to import the python module:
import Module1.mymodule1
but getting an error like
ModuleNotFoundError: No module named 'Module1'
I do see there are option like system-path at runtime, PYTHONPATH but I don't want to use those options.
Do you all have any other recommended solutions?
You can insert this at the top of module1.py, or in the init.py in Module1 if you're importing module1.py as well:
import sys
sys.path.append("../Module2") #assuming linuxOS
import module2
This adds the Module2 directory to the system path and allows you to import files as if they were local to your current directory.
Other answers can be found in this link: beyond top level package error in relative import
I'll give you an example of importing modules
I created the module1.py file
print("Module : 1")
def module01():#function
print("module : 1.2")
class Module02:# class
def module021(self):# function
print('MODULE 1')
to import you need to indicate the file name
module2.py
from module1 import module01 #import the function
from module1 import Module02 #importing the class from module2.py
module01()
mod1 = Module02() #
mod1.module021()# imported class function
I'm currently in the 'strategy.py' file and I'm trying to import 'utils.py' and 'BaseStrategy.py' ('utils.py' is a file with some functions in it, 'BaseStrategy.py' contains a class with the name 'BaseStrategy').
Folder structure:
program\
__init__.py
main.py
folder1\
__init__.py
utils.py
BaseStrategy.py
folder2\
__init__.py
strategy.py
In 'main.py' I'm able to import both files like this:
from folder1.BaseStrategy import BaseStrategy
from folder1 import utils
So how can I import 'utils.py' and 'BaseStrategy.py' in 'strategy.py'?
import sys
sys.path.insert(0, 'program/folder1')
from folder1 import utils
from folder1 import BaseStrategy
You can also change the:
sys.path.insert(0, 'program/folder1')
To:
sys.path.append('../')
But it will mess up with the import from your parent directory (main.py). However you can overcome that with :
import os,sys
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
Essentially, you need to put a blank file named __init__.py in the subdirectory. If you want to take a look at the documentation you can do so here (6.4).
find:
./main/
./main/folder2
./main/folder2/__init__.py
./main/folder1
./main/folder1/base.py
./main/folder1/__init__.py
./main/__init__.py
cat ./main/folder1/base.py
import sys
import os
sys.path.append(os.getcwd() + '/..')
import folder2
I have this structure:
root/
A.py
test.txt
module/
__init__.py
B.py
This is A.py
import B
This is B.py
with open('../test.txt'):
pass
If I run this from the root/
python A.py
I get
IOError: [Errno 2] No such file or directory: '../text.txt'
But changing the path in B.py to test.txt works.
Now, I think it makes sense to me why. Clearly once in the context of A.py as our __main__, referencing ../test/txt is relative to A.py, not B.py. How do I make it do what I need it to? That is, I want to reference the file relative to B.py and I want it to work no matter where I call it from.
PS: A.py is actually my Flask app.py
PPS: I don't want to just leave it as test.txt because I have a notebook
root/
dev/
notebook.ipynb
which imports B via
import sys
sys.path.insert(0, '..')
import B
Python by default adds some buildin values to the global namespace. __file__ is a string representing the path of the file.
Example from pep-3147/#file:
>>> import foo
>>> foo.__file__
'foo.py'
>>> # baz is a package
>>> import baz
>>> baz.__file__
'baz/__init__.py'
Using pathlib to navigate you can then get the module that holds B.py and specify a path relative to the module.
# B.py
from pathlib import Path
module = Path(__file__).parent
with open(module / "../test.txt"):
pass
file structure
root
├── A.py
├── documents
│ ├── file1.txt
│ ├── __init__.py
│
├── files.py
├── module
├── B.py
├── __init__.py
content of A.py
from module.B import func_read
from files import file1
import os
running_file_path, filename = os.path.split(os.path.abspath(__file__))
file_to_read = os.path.join(running_file_path, file1)
result = func_read(file_to_read)
print(result)
content of file1.txt
THIS IS a sample dcoument
content of B.py
import os
def func_read(file):
if file is None or not os.path.isfile(file):
return "invalid file path or no file found"
data = None
with open(file, 'r') as f:
data = f.readlines()
return data
content of file.py
import os
from documents import document_path
file1 = os.path.join(document_path, 'file1.txt')
content of documents/__init__.py
import os
document_path = 'documents'
you can save the documents name in the file.py and save all document in document folder
all you need to load the document by there name from the file.py in different python files.
i have added document folder path in init file so if it is refactored or moved else where you need to change the
documnet folder path there relative to the folder and rest thing can be work somth.
NOTE: THIS IS A SIMPLE APPROACH YOU CAN ORGANISE YOUR PROJECT LIKE YOU WANT
with __init__.py in the directory, I was able to import it by
from subdirectory.file import *
But I wish to import every file in that subdirectory; so I tried
from subdirectory.* import *
which did not work. Any suggestions?
Found this method after trying some different solutions (if you have a folder named 'folder' in adjacent dir):
for entry in os.scandir('folder'):
if entry.is_file():
string = f'from folder import {entry.name}'[:-3]
exec (string)
If you have the following structure:
$ tree subdirectory/
subdirectory/
├── file1.py
├── file2.py
└── file3.py
and you want a program to automatically pick up every module which is located in this subdirectory and process it in a certain way you could achieve it as follows:
import glob
# Get file paths of all modules.
modules = glob.glob('subdirectory/*.py')
# Dynamically load those modules here.
For how to dynamically load a module see this question.
In your subdirectory/__init__.py you can import all local modules via:
from . import file1
from . import file2
# And so on.
You can import the content of local modules via
from .file1 import *
# And so on.
Then you can import those modules (or the contents) via
from subdirectory import *
With the attribute __all__ in __init__.py you can control what exactly will be imported during a from ... import * statement. So if you don't want file2.py to be imported for example you could do:
__all__ = ['file1', 'file3', ...]
You can access those modules via
import subdirectory
from subdirectory import *
for name in subdirectory.__all__:
module = locals()[name]
Your __init__.py file should look like this:
from file1 import *
from file2 import *
And then you can do:
from subdirectory import *
I need to dynamically import modules into my project from another package.
The structure is like:
project_folder/
project/
__init__.py
__main__.py
plugins/
__init__.py
plugin1/
__init__.py
...
plugin2/
__init__.py
...
I made this function to load a module:
import os
from importlib.util import spec_from_file_location, module_from_spec
def load_module(path, name=""):
""" loads a module by path """
try:
name = name if name != "" else path.split(os.sep)[-1] # take the module name by default
spec = spec_from_file_location(name, os.path.join(path, "__init__.py"))
plugin_module = module_from_spec(spec)
spec.loader.exec_module(plugin_module)
return plugin_module
except Exception as e:
print("failed to load module", path, "-->", e)
It works, unless the module uses relative imports:
failed to load module /path/to/plugins/plugin1 --> Parent module 'plugin1' not loaded, cannot perform relative import
What am I doing wrong?
I managed to solve my own issue after a LOT of googling. Turns out I needed to import using relative paths:
>>> from importlib import import_module
>>> config = import_module("plugins.config")
>>> config
<module 'plugins.config' from '/path/to/plugins/config/__init__.py'>
>>>
I had a similar problem not long ago. I added the path of the project folder to the sys.path using the module's absolute path like this:
import sys
import os
sys.path.append(os.path.dirname(os.path.realpath(__file__))+'/..')
This adds the project_folder to the sys.path thus allowing the import statement to find the plugin modules.