I am using django and I have a file named models.admin.py and I want to do the following idea in models.py:
from "models.admin" import *
however, I get a syntax error for having double quotes. But if I just do
from models.admin import *
then I get "ImportError: No module named admin"
Is there any way to import from a python file that has a period in its name?
Actually, you can import a module with an invalid name. But you'll need to use imp for that, e.g. assuming file is named models.admin.py, you could do
import imp
with open('models.admin.py', 'rb') as fp:
models_admin = imp.load_module(
'models_admin', fp, 'models.admin.py',
('.py', 'rb', imp.PY_SOURCE)
)
But read the docs on imp.find_module and imp.load_module before you start using it.
If you really want to, you can import a module with an unusual filename (e.g., a filename containing a '.' before the '.py') using the imp module:
>>> import imp
>>> a_b = imp.load_source('a.b', 'a.b.py')
>>> a_b.x
"I was defined in a.b.py!"
However, that's generally a bad idea. It's more likely that you're trying to use packages, in which case you should create a directory named "a", containing a file named "b.py"; and then "import a.b" will load a/b.py.
The file is called models/admin.py. (Source)
That is, it should be called admin.py in a directory called models.
Then you can import using from models.admin import *, assuming that it is in your Python path.
Like below
Assume dir structure is like this:
C:.
│ script.py
│
└───Parent
└───Child
├───1.1
│ main.py
│
└───1.2
**assume you want to import main.py in script.py **
your main.py looks like below
def my_function():
print("Hello from a function")
your script.py looks like below
from os import path
import importlib
from os.path import dirname
import sys
import importlib.util
def getPath():
# your logic to get to the path
return path.join(dirname(__file__),'Parent','Child','1.1','main.py')
file_path = getPath()
module_name = 'main'
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
#call functions like this
module.my_function()
Check out this gist
No, you can't import a python file as a module if its name contains a period (or a question mark, or exclamation mark, etc). A python module's name (not including the .py) must be a valid python name (ie can be used as a variable name).
In my case, I am using grafanalib, and the filename has to be xx.dashboard.py based on the doc. However, I do want to import this file to simplify the uploading step.
I got warning when I use import imp:
the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses
Here is the simple demo using importlib and pathlib:
foo.bar.py and main.py are in the same foler.
# foo.bar.py
num = 42
# main.py
import importlib.machinery
import pathlib
module = importlib.machinery.SourceFileLoader(
"foo_bar",
pathlib.Path(__file__).parent.joinpath("foo.bar.py").resolve().as_posix(),
).load_module()
print(module.num) # 42
You are not referencing files in the import statement, you are referencing modules and packages.
Please read the docs, they are very clear on that matter.
Anyway, since you are using django, the usual approach won't work. If you want to keep models in separate files, rather than in models.py, you have to take extra steps, outlined, for example, here.
Edit:
Well, I don't really know what the questioneer means when he mentions admin and whether or not it is related to the admin interface of django. My points still stand.
Related
I have imports.py containing:
import os as exported_os
and foo.py containing:
from imports import exported_os
print(exported_os.path.devnull) # works
from imports.exported_os.path import devnull # doesn't
Is there a way to make the second import work? I tried adding __path__ to imports.py and fiddling with it but couldn't get anything.
Actual usecase: os is some_library_version_n and exported_os is some_library_version (I'm trying to avoid having many instances of some_library_version_n across different files).
One approach
Directory structure:
__init__.py
foo.py
imports/
├ __init__.py
└ exported_os/
├ __init__.py
└ path.py
imports/exported_os/__init__.py:
from . import path
from os import * # not necessary for the question
# but it makes `exported_os` more like `os`
# e.g., `exported_os.listdir` will be callable
imports/exported_os/path.py:
from os.path import *
In this way, you can use exported_os as if it is os with a submodule path. Different with import, from takes modules and classes.
Another approach
imports.py:
import os
import sys
ms = []
for m in sys.modules:
if m.startswith('os'):
ms.append(m)
for m in ms:
sys.modules['imports.exported_os' + m[2:]] = sys.modules[m]
Or, by explicitly extending sys.modules you can use exported_os as if os with its submodules.
Why you cannot simply change the name of os
If you open .../lib/python3.9/os.py you can find the following line:
sys.modules['os.path'] = path
So even if you copy .../lib/python3.9/os.py to .../lib/python3.9/exported_os.py, the following does not work:
from exported_os.path import devnull
But if you change the line sys.modules['os.path'] to sys.modules['exported_os.path'] it works.
The error you are getting would be something like:
ModuleNotFoundError: No module named 'imports.exported_os'; 'imports' is not a package
When you code from imports import exported_os, then imports can refer to a module implemented by file imports.py. But when the name imports is part of a hierarchy as in from imports.exported_os.path import devnull, then imports must be a package implemented as a directory in a directory structure such as the following:
__init__.py
imports
__init__.py
exported_os
__init__.py
path.py
where directory containing the top-most __init__.py must be in the sys.path search path.
So, unless you want to rearrange your directory structure to something like the above, the syntax (and selective importing) you want to use is really not available to you without getting into the internals of Python's module system.
Although this is not a solution to your wanting to be able to do an from ... import ... due to your unique versioning issue, let me suggest an alternate method of doing this versioning. In your situation you could do the following. Create a package, my_imports (give it any name you want):
my_imports
__init__.py
The contents of __init__.py is:
import some_library_version_n as some_library_version
Then in foo.py and in any other file that needs this module:
from my_imports import *
This is another method of putting the versioning dependency in one file. If you had other similar dependencies, you would, of course, add them to this file and you could import from my_imports just the names you are interested. You still have the issue that you are importing the entire module some_library_version.
However, we could take this one step further. Suppose the various versions of your library had components A, B and C that you might be interested in importing individually or all together. Then you could do the following. Let's instead name the package some_library_version, since it will only be dealing with this one versioning issue:
some_library_version/init.py
from some_library_version_n import A
from some_library_version_n import B
from some_library_version_n import C
foo.py
from some_library_version import A, C
Most of the answers added are accured but dont add context of why works in that way, GyuHyeon explains it well but it just resumes it into import is a fancy file include system that checks into the std libraries, then the installed ones and finaly into the context provided, context is added on where is called and the from given.
This example gives the various method of importing a specific function dirname(), the lib os is just a folder, if you imagine that os is in your working folder the import path whoud be the same or './os' and beause python, everything is a class, so import will search for the .os/__init__.py so if your library dont have one importing the subdirs it will have no efect.
from os.path import dirname as my_fucntion # (A_2)
from os import path as my_lib # (B_2)
from os.path import dirname # (C_1)
from os import path # (B_1)
import os # (A_1)
if __name__ == '__main__':
print(os.path.dirname(__file__)) # (A_1)
print(path.dirname(__file__)) # (B_1)
print(dirname(__file__)) # (C_1)
print(my_lib.dirname(__file__)) # (B_2)
print(my_fucntion(__file__)) # (A_2)
You could try to go with sys.path.append(...), e.g.:
import sys
sys.path.append(<your path to devnull goes here>)
Maybe not so nice, but you could use the pathlib library and path joins to construct the path (but some assumptions on file structure unfortunately have to be made if you the files are in separate folder structures):
from pathlib import Path
from os import path
sys.path.append(path.join(str(Path(__file__).parents[<integer that tells how many folders to go up>]), <path to devnull>))
Instead of pathlib you could also use the dirname function from os.path.
After appending to the system path, you could just use:
import devnull
This question already has answers here:
How can I import a module dynamically given its name as string?
(10 answers)
How can I import a module dynamically given the full path?
(35 answers)
Closed 3 months ago.
I have multiple files with a structure like a file example.py:
def initialize(context):
pass
def daj_omacku_teplu(context, data):
pass
def hmataj_pomaly(context, data):
pass
def chvatni_paku(context, data):
pass
def mikaj_laktom(context, data):
pass
and I need to be able to dynamically import methods from "example.py" in a different python file like:
for fn in os.listdir('.'):
if os.path.isfile(fn):
from fn import mikaj_laktom
mikaj_laktom(example_context, sample_data)
For multiple reasons, I can not change the structure of example.py so I need to make a mechanism to load methods and evaluate them. I tried to use importlib but it can only import a class, not file with only methods defined.
Thanks for the help.
Python import does not support importing using paths, so you will need to have the files accessible as modules, see (sys.path). Assuming for now that your sources are located in the same folder as the main script, I would use the following (or similar):
import sys
def load_module(module):
# module_path = "mypackage.%s" % module
module_path = module
if module_path in sys.modules:
return sys.modules[module_path]
return __import__(module_path, fromlist=[module])
# Main script here... Could be your for loop or anything else
# `m` is a reference to the imported module that contains the functions
m = load_module("example")
m.mikaj_laktom(None, [])
The source files can also be part of another package, in which case you will need an __init__.py in the same folder with the .py files (see packages) and you import with "mypackage.module" notation. (Note that the top level folder should be in your path, in the above example this is the folder containing "mypackage")
UDPATE:
As pointed out by #skyking there are lib that can help you do the same thing. See this post
My comment on __init__.py is outdate since things have changed in py3. See this post for some more detailed explanation
You were on the right track with importlib. It can be used to load modules by name, however I do not think you can load them into the global namespace in this way (as in from module import function). So you need to load them as module objects and call your required method:
import glob, importlib, os, pathlib, sys
# The directory containing your modules needs to be on the search path.
MODULE_DIR = '/path/to/modules'
sys.path.append(MODULE_DIR)
# Get the stem names (file name, without directory and '.py') of any
# python files in your directory, load each module by name and run
# the required function.
py_files = glob.glob(os.path.join(MODULE_DIR, '*.py'))
for py_file in py_files:
module_name = pathlib.Path(py_file).stem
module = importlib.import_module(module_name)
module.mikaj_laktom()
Also, be careful using '.' as your MODULE_DIR, as this will presumably try to load the current python file as well, which might cause some unexpected behaviour.
Edit: if using Python2, you won't have pathlib in the standard library, so use
module_name = os.path.splitext(os.path.split(py_file)[1])[0]
to get the equivalent of Path.stem.
What would be the best (read: cleanest) way to tell Python to import all modules from some folder?
I want to allow people to put their "mods" (modules) in a folder in my app which my code should check on each startup and import any module put there.
I also don't want an extra scope added to the imported stuff (not "myfolder.mymodule.something", but "something")
If transforming the folder itself in a module, through the use of a __init__.py file and using from <foldername> import * suits you, you can iterate over the folder contents
with "os.listdir" or "glob.glob", and import each file ending in ".py" with the __import__ built-in function:
import os
for name in os.listdir("plugins"):
if name.endswith(".py"):
#strip the extension
module = name[:-3]
# set the module name in the current global name space:
globals()[module] = __import__(os.path.join("plugins", name)
The benefit of this approach is: it allows you to dynamically pass the module names to __import__ - while the ìmport statement needs the module names to be hardcoded, and it allows you to check other things about the files - maybe size, or if they import certain required modules, before importing them.
Create a file named
__init__.py
inside the folder and import the folder name like this:
>>> from <folder_name> import * #Try to avoid importing everything when you can
>>> from <folder_name> import module1,module2,module3 #And so on
You might want to try that project: https://gitlab.com/aurelien-lourot/importdir
With this module, you only need to write two lines to import all plugins from your directory and you don't need an extra __init__.py (or any other other extra file):
import importdir
importdir.do("plugins/", globals())
I have a directory, let's call it Storage full of packages with unwieldy names like mypackage-xxyyzzww, and of course Storage is on my PYTHONPATH. Since packages have long unmemorable names, all of the packages are symlinked to friendlier names, such as mypackage.
Now, I don't want to rely on file system symbolic links to do this, instead I tried mucking around with sys.path and sys.modules. Currently I'm doing something like this:
import imp
imp.load_package('mypackage', 'Storage/mypackage-xxyyzzww')
How bad is it to do things this way, and is there a chance this will break in the future? One funny thing is that there's even no mention of imp.load_package function in the docs.
EDIT: besides not relying on symbolic links, I can't use PYTHONPATH variable anymore.
Instead of using imp, you can assign different names to imported modules.
import mypackage_xxyyzzww as mypackage
If you then create a __init__.py file inside of Storage, you can add several of the above lines to make importing easier.
Storage/__init__.py:
import mypackage_xxyyzzww as mypackage
import otherpackage_xxyyzzww as otherpackage
Interpreter:
>>> from Storage import mypackage, otherpackage
importlib may be more appropriate, as it uses/implements the PEP302 mechanism.
Follow the DictImporter example, but override find_module to find the real filename and store it in the dict, then override load_module to get the code from the found file.
You shouldn't need to use sys.path once you've created your Storage module
#from importlib import abc
import imp
import os
import sys
import logging
logging.basicConfig(level=logging.DEBUG)
dprint = logging.debug
class MyImporter(object):
def __init__(self,path):
self.path=path
self.names = {}
def find_module(self,fullname,path=None):
dprint("find_module({fullname},{path})".format(**locals()))
ml = imp.find_module(fullname,path)
dprint(repr(ml))
raise ImportError
def load_module(self,fullname):
dprint("load_module({fullname})".format(**locals()))
return imp.load_module(fullname)
raise ImportError
def load_storage( path, modname=None ):
if modname is None:
modname = os.path.basename(path)
mod = imp.new_module(modname)
sys.modules[modname] = mod
assert mod.__name__== modname
mod.__path__=[path]
#sys.meta_path.append(MyImporter(path))
mod.__loader__= MyImporter(path)
return mod
if __name__=="__main__":
load_storage("arbitrary-path-to-code/Storage")
from Storage import plain
from Storage import mypkg
Then when you import Storage.mypackage, python will immediately use your importer without bothering to look on sys.path
That doesn't work. The code above does work to import ordinary modules under Storage without requiring Storage to be on sys.path, but both 3.1 and 2.6 seem to ignore the loader attribute mentioned in PEP302.
If I uncomment the sys.meta_path line, 3.1 dies with StackOverflow, and 2.6 dies with ImportError. hmmm... I'm out of time now, but may look at it later.
Packages are just entries in the namespace. You should not name your path components with anything that is not a legal python variable name.
I have a file called foobar (without .py extension). In the same directory I have another python file that tries to import it:
import foobar
But this only works if I rename the file to foobar.py. Is it possible to import a python module that doesn't have the .py extension?
Update: the file has no extension because I also use it as a standalone script, and I don't want to type the .py extension to run it.
Update2: I will go for the symlink solution mentioned below.
You can use the imp.load_source function (from the imp module), to load a module dynamically from a given file-system path.
import imp
foobar = imp.load_source('foobar', '/path/to/foobar')
This SO discussion also shows some interesting options.
Here is a solution for Python 3.4+:
from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader
spec = spec_from_loader("foobar", SourceFileLoader("foobar", "/path/to/foobar"))
foobar = module_from_spec(spec)
spec.loader.exec_module(foobar)
Using spec_from_loader and explicitly specifying a SourceFileLoader will force the machinery to load the file as source, without trying to figure out the type of the file from the extension. This means that you can load the file even though it is not listed in importlib.machinery.SOURCE_SUFFIXES.
If you want to keep importing the file by name after the first load, add the module to sys.modules:
sys.modules['foobar'] = foobar
You can find an implementation of this function in a utility library I maintain called haggis. haggis.load.load_module has options for adding the module to sys.modules, setting a custom name, and injecting variables into the namespace for the code to use.
Like others have mentioned, you could use imp.load_source, but it will make your code more difficult to read. I would really only recommend it if you need to import modules whose names or paths aren't known until run-time.
What is your reason for not wanting to use the .py extension? The most common case for not wanting to use the .py extension, is because the python script is also run as an executable, but you still want other modules to be able to import it. If this is the case, it might be beneficial to move functionality into a .py file with a similar name, and then use foobar as a wrapper.
imp.load_source(module_name, path) should do or you can do the more verbose imp.load_module(module_name, file_handle, ...) route if you have a file handle instead
importlib helper function
Here is a convenient, ready-to-use helper to replace imp, with an example, based on what was mentioned at: https://stackoverflow.com/a/43602645/895245
main.py
#!/usr/bin/env python3
import os
import importlib
import sys
def import_path(path):
module_name = os.path.basename(path).replace('-', '_')
spec = importlib.util.spec_from_loader(
module_name,
importlib.machinery.SourceFileLoader(module_name, path)
)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
sys.modules[module_name] = module
return module
notmain = import_path('not-main')
print(notmain)
print(notmain.x)
not-main
x = 1
Run:
python3 main.py
Output:
<module 'not_main' from 'not-main'>
1
I replace - with _ because my importable Python executables without extension have hyphens. This is not mandatory, but produces better module names.
This pattern is also mentioned in the docs at: https://docs.python.org/3.7/library/importlib.html#importing-a-source-file-directly
I ended up moving to it because after updating to Python 3.7, import imp prints:
DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
and I don't know how to turn that off, this was asked at:
The imp module is deprecated
How to ignore deprecation warnings in Python
Tested in Python 3.7.3.
If you install the script with package manager (deb or alike) another option would be to use setuptools:
"...there’s no easy way to have a script’s filename match local conventions on both Windows and POSIX platforms. For another, you often have to create a separate file just for the “main” script, when your actual “main” is a function in a module somewhere... setuptools fixes all of these problems by automatically generating scripts for you with the correct extension, and on Windows it will even create an .exe file..."
https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation
import imp has been deprecated.
The following is clean and minimal for me:
import sys
import types
import pathlib
def importFileAs(
modAsName: str,
importedFilePath: typing.Union[str, pathlib.Path],
) -> types.ModuleType:
""" Import importedFilePath as modAsName, return imported module
by loading importedFilePath and registering modAsName in sys.modules.
importedFilePath can be any file and does not have to be a .py file. modAsName should be python valid.
Raises ImportError: If the file cannot be imported or any Exception: occuring during loading.
Refs:
Similar to: https://stackoverflow.com/questions/19009932/import-arbitrary-python-source-file-python-3-3
but allows for other than .py files as well through importlib.machinery.SourceFileLoader.
"""
import importlib.util
import importlib.machinery
# from_loader does not enforce .py but importlib.util.spec_from_file_location() does.
spec = importlib.util.spec_from_loader(
modAsName,
importlib.machinery.SourceFileLoader(modAsName, importedFilePath),
)
if spec is None:
raise ImportError(f"Could not load spec for module '{modAsName}' at: {importedFilePath}")
module = importlib.util.module_from_spec(spec)
try:
spec.loader.exec_module(module)
except FileNotFoundError as e:
raise ImportError(f"{e.strerror}: {importedFilePath}") from e
sys.modules[modAsName] = module
return module
And then I would use it as so:
aasMarmeeManage = importFileAs('aasMarmeeManage', '/bisos/bpip/bin/aasMarmeeManage.cs')
def g_extraParams(): aasMarmeeManage.g_extraParams()