I am trying to write a function which will take class name a argument and import that class and perform some task.
def search(**kwargs):
"""
:return:
"""
try:
model = __import__('girvi.models', globals(), locals(), kwargs['model_name'], -1)
# Do some task
return results
except Exception:
raise Exception('Model not found')
But thing is model has the class which is in kwargs['model_name'] imported successfully but how do i access it. Please can someone help me.
I would try the following:
import importlib
import sys
def import_class(class_name, module_name = "girvi.models"):
module = importlib.import_module(module_name)
class_object = getattr(module, class_name) # does not work for nested classes
return class_object
According to __import__ one should rather use importlib.import_module.
Related
I'm working on a project that dynamically collects classes from another module and creates new classes based off the ones it found. For example, if the source module has a "AdaptiveFilter" class, a new class called "AdaptiveFilterNode" is created that contains an "AdaptiveFilter" instance. I've been adding these generated classes to a dynamically created module (module_from_spec()) and created a MetaPathFinder and Loader to extend the import system. This works perfectly fine when importing the dynamic module, e.g. import nodes.auto_gen or from nodes.auto_gen import AdaptiveFilterNode, but fails when trying to directly import a class, e.g. import nodes.auto_gen.AdaptiveFilterNode. I need to be able to directly import a class because the generated classes need to be pickleable, which requires a direct import. I believe the problem is in the Finder that I made:
class MyNodeFinder(MetaPathFinder):
def find_spec(self, fullname, path=None, target=None):
if fullname == 'nodes.auto_gen':
auto_gen_spec = = ModuleSpec('nodes.auto_gen',
MyLoader())
return auto_gen_spec
return None
From my limited understanding on the import system, there's nothing in the finder or the module spec that tells the import system there's classes contained in the nodes.auto_gen module until the module is already loaded. How can I create a finder/loader that replicates the standard import system for a dynamic module with dynamic classes?
EDIT: Full example, module "dynamic_creator":
dynamic_creator/__init__.py
from importlib.machinery import ModuleSpec
from importlib.util import module_from_spec
import sys
from importlib.abc import MetaPathFinder
from importlib.abc import Loader
__version__ = '0.0.0'
class _MyWrapperClass():
wrapped_func = None
def __init__(self):
super().__init__()
def __call__(self, *args, **kwargs):
return self.wrapped_func(*args, **kwargs)
class MyFinder(MetaPathFinder):
def find_spec(self, fullname, path=None, target=None):
if fullname == 'dynamic_creator.auto_gen':
auto_node_spec = ModuleSpec('dynamic_creator.auto_gen',
MyLoader())
return auto_node_spec
return None
class MyLoader(Loader):
def create_module(self, spec):
return module_from_spec(ModuleSpec('dynamic_creator.auto_gen', None))
def exec_module(self, module):
def create_type(func):
t = type(func.__name__, (_MyWrapperClass, ), {})
t.wrapped_func = func
# Wait, you can do this?
t.__module__ = 'dynamic_creator.auto_gen'
return t
funcs_to_wrap = [
sum,
abs,
pow
]
for func in funcs_to_wrap:
setattr(module, func.__name__, create_type(func))
my_finder = MyFinder()
sys.meta_path.append(my_finder)
Test script:
import dynamic_creator.auto_gen # Works
from dynamic_creator.auto_gen import abs # Works
import dynamic_creator.auto_gen.pow as my_pow # Fails
EDIT: The reported error is as follows:
No module named 'dynamic_creator.auto_gen.pow'; 'dynamic_creator.auto_gen' is not a package
I have class called 'my_class' placed in 'my_module'. And I need to import this class. I tried to do it like this:
import importlib
result = importlib.import_module('my_module.my_class')
but it says:
ImportError: No module named 'my_module.my_class'; 'my_module' is not a package
So. As I can see it works only for modules, but can't handle classes. How can I import a class from a module?
It is expecting my_module to be a package containing a module named 'my_class'. If you need to import a class, or an attribute in general, dynamically, just use getattr after you import the module:
cls = getattr(import_module('my_module'), 'my_class')
Also, yes, it does only work with modules. Remember importlib.import_module is a wrapper of the internal importlib.__import__ function. It doesn't offer the same amount of functionality as the full import statement which, coupled with from, performs an attribute look-up on the imported module.
import importlib
import logging
logger = logging.getLogger(__name__)
def factory(module_class_string, super_cls: type = None, **kwargs):
"""
:param module_class_string: full name of the class to create an object of
:param super_cls: expected super class for validity, None if bypass
:param kwargs: parameters to pass
:return:
"""
module_name, class_name = module_class_string.rsplit(".", 1)
module = importlib.import_module(module_name)
assert hasattr(module, class_name), "class {} is not in {}".format(class_name, module_name)
logger.debug('reading class {} from module {}'.format(class_name, module_name))
cls = getattr(module, class_name)
if super_cls is not None:
assert issubclass(cls, super_cls), "class {} should inherit from {}".format(class_name, super_cls.__name__)
logger.debug('initialising {} with params {}'.format(class_name, kwargs))
obj = cls(**kwargs)
return obj
I followed advice here on stackoverflow on how to import a class
from a path outside of the root folder:
How to dynamically load a Python class
Does python have an equivalent to Java Class.forName()?
Unfortunately this raises the error:
ValueError: Empty module name
It is my understanding that import should not return an empty module, like the load_source method from imp. So I do not understand this error nor how to approach it.
What am I implementing wrong here?
Could you tilt me into the right direction here?
Thanks!
Code:
klass = __import__('..folder.module.Class_A')
some_object = klass()
class class_B(some_object):
def run(self):
print ('Test OK!')
Imported class (content of module):
class Class_A():
def __init__(self, arg1, *args):
def run(self):
pass
First of all you are not using import correctly.
Option 1:
According to https://docs.python.org/2/library/functions.html#import you should be doing:
klass = __import__('folder.module', globals(), locals(), ['Class_A'], -1)
Now if you want Class_A itself you should do:
some_object = klass.Class_A
Then you can inherit from it using:
class class_B(some_object):
def run(self):
print ('Test OK!')
Option 2:
from folder.module import Class_A
Then you can inherit from it using:
class class_B(Class_A):
def run(self):
print ('Test OK!')
Note: In folder.module folder should be a python package and module should be a python module
I have the following interface :
class Interface(object):
__metaclass__ = abc.ABCMeta
#abc.abstractmethod
def run(self):
"""Run the process."""
return
I have a collections of modules that are all in the same directory. Each module contains a single class that implements my interface.
For example Launch.py :
class Launch(Interface):
def run(self):
pass
Let's say I have 20 modules, that implements 20 classes. I would like to be able to launch a module that would check if some of the classes do not implement the Interface.
I know I have to use :
issubclass(Launch, ProcessInterface) to know if a certain class implements my process interface.
introspection to get the class that is in my module.
import modules at runtime
I am just not sure how to do that.
I can manage to use issubclass inside a module.
But I cannot use issubclass if I am outside the module.
I need to :
get the list of all modules in the directory
get the class in each module
do issubclass on each class
I would need a draf of a function that could do that.
You're probably looking for something like this:
from os import listdir
from sys import path
modpath = "/path/to/modules"
for modname in listdir(modpath):
if modname.endswith(".py"):
# look only in the modpath directory when importing
oldpath, path[:] = path[:], [modpath]
try:
module = __import__(modname[:-3])
except ImportError:
print "Couldn't import", modname
continue
finally: # always restore the real path
path[:] = oldpath
for attr in dir(module):
cls = getattr(module, attr)
if isinstance(cls, type) and not issubclass(cls, ProcessInterface):
# do whatever
Interesting usecase today: I need to migrate a module in our codebase following code changes. The old mynamespace.Document will disappear and I want to ensure smooth migration by replacing this package by a code object that will dynamically import the correct path and migrate the corresponding objects.
In short:
# instanciate a dynamic package, but do not load
# statically submodules
mynamespace.Document = SomeObject()
assert 'submodule' not in mynamespace.Document.__dict__
# and later on, when importing it, the submodule
# is built if not already available in __dict__
from namespace.Document.submodule import klass
c = klass()
A few things to note:
I am not talking only of migrating code. A simple huge sed would in a sense be enough to change the code in order to migrate some imports, and I would not need a dynamic module. I am talking of objects. A website, holding some live/stored objects will need migration. Those objects will be loaded assuming that mynamespace.Document.submodule.klass exists, and that's the reason for the dynamic module. I need to provide the site with something to load.
We cannot, or do not want to change the way objects are unpickled/loaded. For simplicity, let's just say that we want to make sure that the idiom from mynamespace.Document.submodule import klass has to work. I cannot use instead from mynamespace import Document as container; klass = getattr(getattr(container, 'submodule'), 'klass')
What I tried:
import sys
from types import ModuleType
class VerboseModule(ModuleType):
def __init__(self, name, doc=None):
super(VerboseModule, self).__init__(name, doc)
sys.modules[name] = self
def __repr__(self):
return "<%s %s>" % (self.__class__.__name__, self.__name__)
def __getattribute__(self, name):
if name not in ('__name__', '__repr__', '__class__'):
print "fetching attribute %s for %s" % (name, self)
return super(VerboseModule, self).__getattribute__(name)
class DynamicModule(VerboseModule):
"""
This module generates a dummy class when asked for a component
"""
def __getattr__(self, name):
class Dummy(object):
pass
Dummy.__name__ = name
Dummy.__module__ = self
setattr(self, name, Dummy)
return Dummy
class DynamicPackage(VerboseModule):
"""
This package should generate dummy modules
"""
def __getattr__(self, name):
mod = DynamicModule("%s.%s" % (self.__name__, name))
setattr(self, name, mod)
return mod
DynamicModule("foobar")
# (the import prints:)
# fetching attribute __path__ for <DynamicModule foobar>
# fetching attribute DynamicModuleWorks for <DynamicModule foobar>
# fetching attribute DynamicModuleWorks for <DynamicModule foobar>
from foobar import DynamicModuleWorks
print DynamicModuleWorks
DynamicPackage('document')
# fetching attribute __path__ for <DynamicPackage document>
from document.submodule import ButDynamicPackageDoesNotWork
# Traceback (most recent call last):
# File "dynamicmodule.py", line 40, in <module>
# from document.submodule import ButDynamicPackageDoesNotWork
#ImportError: No module named submodule
As you can see the Dynamic Package does not work. I do not understand what is happening because document is not even asked for a ButDynamicPackageDoesNotWork attribute.
Can anyone clarify what is happening; and if/how I can fix this?
The problem is that python will bypass the entry in for document in sys.modules and load the file for submodule directly. Of course this doesn't exist.
demonstration:
>>> import multiprocessing
>>> multiprocessing.heap = None
>>> import multiprocessing.heap
>>> multiprocessing.heap
<module 'multiprocessing.heap' from '/usr/lib/python2.6/multiprocessing/heap.pyc'>
We would expect that heap is still None because python can just pull it out of sys.modules but That doesn't happen. The dotted notation essentially maps directly to {something on python path}/document/submodule.py and an attempt is made to load that directly.
Update
The trick is to override pythons importing system. The following code requires your DynamicModule class.
import sys
class DynamicImporter(object):
"""this class works as both a finder and a loader."""
def __init__(self, lazy_packages):
self.packages = lazy_packages
def load_module(self, fullname):
"""this makes the class a loader. It is given name of a module and expected
to return the module object"""
print "loading {0}".format(fullname)
components = fullname.split('.')
components = ['.'.join(components[:i+1])
for i in range(len(components))]
for component in components:
if component not in sys.modules:
DynamicModule(component)
print "{0} created".format(component)
return sys.modules[fullname]
def find_module(self, fullname, path=None):
"""This makes the class a finder. It is given the name of a module as well as
the package that contains it (if applicable). It is expected to return a
loader for that module if it knows of one or None in which case other methods
will be tried"""
if fullname.split('.')[0] in self.packages:
print "found {0}".format(fullname)
return self
else:
return None
# This is a list of finder objects which is empty by defaule
# It is tried before anything else when a request to import a module is encountered.
sys.meta_path=[DynamicImporter('foo')]
from foo.bar import ThisShouldWork