My goal is to load dynamically my different subclasses and execute them. To call my script, I am using this:
python Plugin.py Nmap execute google.com
Or
python Plugin.py Dig execute google.com
Here it's the code
Parent Class: Plugin.py
class Plugin(object)
def __init__(self):
self.sName = ''
def execPlugin(self):
return 'something'
def main():
# get the name of the plugin
sPlugin = sys.argv[1]
# create the object
mMod = __import__(sPlugin, fromlist=[sPlugin])
mInstance = getattr(mMod, sPlugin)
oPlugin = mInstance()
print oPlugin
print type(oPlugin)
if (sys.argv[2] == 'execute'):
# then execute it
return oPlugin.execPlugin(sys.argv[3])
if __name__ == '__main__':
main()
Sub Class located in Nmap/Nmap.py
class Nmap(Plugin):
def __init__(self):
self.sName = 'Nmap'
def execPlugin(self):
return 'something else'
Sub Class located in Dig/Dig.py
class Dig(Plugin):
def __init__(self):
self.sName = 'Dig'
def execPlugin(self):
return 'yeahhh'
My problem is located in
oPlugin = mInstance()
With the following error
TypeError: 'module' object is not callable
I tried so many things but nothing worked. How can I solve my problem?
You have a structure like:
Plugin.py
/Nmap
__init__.py
Nmap.py
# class Nmap(Plugin): ...
In Plugin.py, when you do mMod = __import__(sPlugin, fromlist=[sPlugin]) where sPlugin == 'Nmap', this makes mMod refer to the directory /Nmap, not the file Nmap.py (note that both files and directories can be modules in Python). Hence mInstance = getattr(mMod, sPlugin) makes mInstance the file Nmap.py rather than the class Nmap.
There are two ways to fix this, either:
Use the __init__.py in /Nmap to bring the class "up" one level, i.e. include from Nmap import Nmap; or
Add an extra level of getattr into Plugin.py.
Additionally, you should follow the style guide's naming conventions, which might have helped you track the issue down faster.
Related
I have two python files main.py and conftest.py. I want to access a variable of a method of the class Test declared in main.py from a function declared in conftest.py.
I have tried a bit, but I know it's wrong as I get a syntax error in the first place. Is there any way to do this?
main.py
class Test():
def test_setup(self):
#make new directory for downloads
new_dir = r"D:\Selenium\Insights\timestamp}".format(timestamp=datetime.now().strftime('%Y-%m-%d_%H-%M-%S'))
# print(new_dir)
if not os.path.exists(new_dir):
os.makedirs(new_dir)
saved_dir=new_dir
conftest.py
from main import Test
def newfunc():
dir=Test.test_setup()
print(dir.saved_dir)
There are some errors in your code, but essentially, to access to the variable saved_dir you have to define it as an attribute of the class Test, and after that instantiate an object of that class.
In your code saved_dir is a local variable of the method test_setup so is not visible outside of that context.
I show you the 2 possible correct files:
File main.py
from datetime import datetime
import os
class Test():
def __init__(self):
self.new_dir = ""
self.saved_dir = ""
def test_setup(self):
#make new directory for downloads
#new_dir = r"D:\Selenium\Insights\timestamp}".format(timestamp=datetime.now().strftime('%Y-%m-%d_%H-%M-%S'))
timestamp=datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
self.new_dir = "/home/frank/Selenium/Insights/timestamp/" + timestamp
# print(new_dir)
if not os.path.exists(self.new_dir):
os.makedirs(self.new_dir)
self.saved_dir = self.new_dir
def get_saved_dir(self):
return self.saved_dir
Pay attention: don't use directly the previous code because in main.py I have adjusted the value of new_dir according to my environment (see /home/frank/Selenium/Insights/timestamp/ instead of your D:\Selenium\Insights\timestamp).
File conftest.py:
from main import Test
def newfunc():
test_class = Test()
test_class.test_setup()
print(test_class.get_saved_dir())
newfunc()
If you want to access to the attribute saved_dir directly without the use of method get_saved_dir() (not very object oriented) the file conftest.py becomes:
from main import Test
def newfunc():
test_class = Test()
test_class.test_setup()
# access directly to attribute saved_dir (not properly Object Oriented)
print(test_class.saved_dir)
newfunc()
Variable must be declared as belonging to the class
class Test():
def __init__(self):
self.new_dir = ""
self.saved_dir = ""
def test_setup(self):
#make new directory for downloads
self.new_dir = r"D:\Selenium\Insights\timestamp}".format(timestamp=datetime.now().strftime('%Y-%m-%d_%H-%M-%S'))
# print(self.new_dir)
if not os.path.exists(self.new_dir):
os.makedirs(self.new_dir)
self.saved_dir=self.new_dir
Then calling it
def newfunc():
dir=Test().test_setup()
print(dir.saved_dir)
I have the following bizarre set up. Consider 3 scripts in different directories:
root1/folderA/scriptA.py
root2/folderB/scriptB.py
root2/folderC/scriptC.py
The first file and it's location are fully modifiable. The second and third are completely fixed.
scriptA.py contains a parent class:
class A:
def get_path(self):
# ...
# code to determine "my_path"
# ...
return my_path
scriptB.py contains a child class:
from root1.folderA.scriptA import A
class B(A):
pass
scriptC.py contains the code to execute:
from root2.folderB.scriptB import B
if __name__ == "__main__":
b = B()
print(b.get_path()) # want print root/folderB/scriptB.py
In scriptC.py what I want is the behaviour to get the path of the child class's declaration file (without any hard-coding). Is there any way to program the A.get_path() to have this behavoir?
I've tried looking into the inspect, os.path and pathlib modules but I haven't had any luck.
It looks like the trick is to use __init_subclass__ which runs whenever a class is sub-classed, in conjunction with a class's __module__ attribute to retrieve its containing module, and __file__ to retrieve the absolute path to the python script or module.
For example, in script_a.py:
import sys
from pathlib import Path
class PathInfo:
__slots__ = ()
path: Path
def __init_subclass__(cls, **kwargs):
mod = sys.modules[cls.__module__]
# `__file__` is a string, so we want to convert to a `Path` so it's easier to work with
cls.path = Path(mod.__file__)
class A(PathInfo):
__slots__ = ()
script_b.py:
from script_a import A
class B(A):
pass
# prints: /Users/<name>/PycharmProjects/my-project/script_a.py
print(A().path)
# prints: /Users/<name>/PycharmProjects/my-project/script_b.py
print(B.path)
I have a factory as shown in the following code:
class ClassFactory:
registry = {}
#classmethod
def register(cls, name):
def inner_wrapper(wrapped_class):
if name in cls.registry:
print(f'Class {name} already exists. Will replace it')
cls.registry[name] = wrapped_class
return wrapped_class
return inner_wrapper
#classmethod
def create_type(cls, name):
exec_class = cls.registry[name]
type = exec_class()
return type
#ClassFactory.register('Class 1')
class M1():
def __init__(self):
print ("Starting Class 1")
#ClassFactory.register('Class 2')
class M2():
def __init__(self):
print("Starting Class 2")
This works fine and when I do
if __name__ == '__main__':
print(ClassFactory.registry.keys())
foo = ClassFactory.create_type("Class 2")
I get the expected result of dict_keys(['Class 1', 'Class 2']) Starting Class 2
Now the problem is that I want to isolate classes M1 and M2 to their own files m1.py and m2.py, and in the future add other classes using their own files in a plugin manner.
However, simply placing it in their own file
m2.py
from test_ import ClassFactory
#MethodFactory.register('Class 2')
class M2():
def __init__(self):
print("Starting Class 2")
gives the result dict_keys(['Class 1']) since it never gets to register the class.
So my question is: How can I ensure that the class is registered when placed in a file different from the factory, without making changes to the factory file whenever I want to add a new class? How to self register in this way? Also, is this decorator way a good way to do this kind of thing, or are there better practices?
Thanks
How can I ensure that the class is registered when placed in a file different from the factory, without making changes to the factory file whenever I want to add a new class?
I'm playing around with a similar problem, and I've found a possible solution. It seems too much of a 'hack' though, so set your critical thinking levels to 'high' when reading my suggestion below :)
As you've mentioned in one of your comments above, the trick is to force the loading of the individual *.py files that contain individual class definitions.
Applying this to your example, this would involve:
Keeping all class implementations in a specific folders, e.g., structuring the files as follows:
.
└- factory.py # file with the ClassFactory class
└─ classes/
└- __init__.py
└- m1.py # file with M1 class
└- m2.py # file with M2 class
Adding the following statement to the end of your factory.py file, which will take care of loading and registering each individual class:
from classes import *
Add a piece of code like the snippet below to your __init__.py within the classes/ foder, so that to dynamically load all classes [1]:
from inspect import isclass
from pkgutil import iter_modules
from pathlib import Path
from importlib import import_module
# iterate through the modules in the current package
package_dir = Path(__file__).resolve().parent
for (_, module_name, _) in iter_modules([package_dir]):
# import the module and iterate through its attributes
module = import_module(f"{__name__}.{module_name}")
for attribute_name in dir(module):
attribute = getattr(module, attribute_name)
if isclass(attribute):
# Add the class to this package's variables
globals()[attribute_name] = attribute
If I then run your test code, I get the desired result:
# test.py
from factory import ClassFactory
if __name__ == "__main__":
print(ClassFactory.registry.keys())
foo = ClassFactory.create_type("Class 2")
$ python test.py
dict_keys(['Class 1', 'Class 2'])
Starting Class 2
Also, is this decorator way a good way to do this kind of thing, or are there better practices?
Unfortunately, I'm not experienced enough to answer this question. However, when searching for answers to this problem, I've came across the following sources that may be helpful to you:
[2] : this presents a method for registering class existence based on Python Metaclasses. As far as I understand, it relies on the registering of subclasses, so I don't know how well it applies to your case. I did not follow this approach, as I've noticed that the new edition of the book suggests the use of another technique (see bullet below).
[3], item 49 : this is the 'current' suggestion for subclass registering, which relies on the definition of the __init_subclass__() function in a base class.
If I had to apply the __init_subclass__() approach to your case, I'd do the following:
Add a Registrable base class to your factory.py (and slightly re-factor ClassFactory), like this:
class Registrable:
def __init_subclass__(cls, name:str):
ClassFactory.register(name, cls)
class ClassFactory:
registry = {}
#classmethod
def register(cls, name:str, sub_class:Registrable):
if name in cls.registry:
print(f'Class {name} already exists. Will replace it')
cls.registry[name] = sub_class
#classmethod
def create_type(cls, name):
exec_class = cls.registry[name]
type = exec_class()
return type
from classes import *
Slightly modify your concrete classes to inherit from the Registrable base class, e.g.:
from factory import Registrable
class M2(Registrable, name='Class 2'):
def __init__(self):
print ("Starting Class 2")
I'm supplid with script.py file with Python contents of which this is psuedo code:
import pandas as pd
import numpy as np
params = {...}
class SomeClass:
def __init__(self, ab):
self.ab = ab
self.de = None
def main(self):
self.foo()
#staticmethod
def DoSomethingUsefull(abc, params=params):
abc['x'] = abc['Red Fox'].copy()
return some_list(abc);
def foo(self):
self.de = SomeClass.DoSomethingUsefull(self.ab)
self.de['de'] = "de"
I cannot change the contents of the file, just use it as is. Eventually I need to execute the script from external (C#) code, currently trying to run from cmd (Python is installed). My question is how to execute from command line? If it's not possible to execute from command line and a wrapper script is needed, what will it look like?
Using WinPython 3.9.10
as #DeepSpace said, this file just contains a definition of a class, it won't do anything by itself.
You can create a python file that instantiates the class and calls some of it's methods
something like
import SomeClass
ab = {'Red Fox': 42}
myClass = SomeClass(ab) # instantiate the class
myClass.main() # call the main method.
Bear also in mind that you will need to modify the File at least to correct the typos like self.de - None otherwise you won't be able to import the class.
If you are going to modify the file anyway to create the typos (Yeah, I know you said you can't modify it, but you must), you could just make the file "executable" by putting the code outside the class like this:
if __name__ == "__main__":
ab = {'Red Fox': 42}
myClass = SomeClass(ab) # instantiate the class
myClass.main() # call the main method.
This way you don't need an extra file.
I'm currently writing some kind of tiny api to support extending module classes. Users should be able to just write their class name in a config and it gets used in our program. The contract is, that the class' module has a function called create(**kwargs) to return an instance of our base module class, and is placed in a special folder. But the isinstance check Fails as soon as the import is made dynamically.
modules are placed in lib/services/name
module base class (in lib/services/service)
class Service:
def __init__(self, **kwargs):
#some initialization
example module class (in lib/services/ping)
class PingService(Service):
def __init__(self, **kwargs):
Service.__init__(self,**kwargs)
# uninteresting init
def create(kwargs):
return PingService(**kwargs)
importing function
import sys
from lib.services.service import Service
def doimport( clazz, modPart, kw, class_check):
path = "lib/" + modPart
sys.path.append(path)
mod = __import__(clazz)
item = mod.create(kw)
if class_check(item):
print "im happy"
return item
calling code
class_check = lambda service: isinstance(service, Service)
s = doimport("ping", "services", {},class_check)
print s
from lib.services.ping import create
pingService = create({})
if isinstance(pingService, Service):
print "why this?"
what the hell am I doing wrong
here is a small example zipped up, just extract and run test.py without arguments
zip example
The problem was in your ping.py file. I don't know exactly why, but when dinamically importing it was not accepting the line from service import Service, so you just have to change it to the relative path: from lib.services.service import Service. Adding lib/services to the sys.path could not make it work the inheritance, which I found strange...
Also, I am using imp.load_source which seems more robust:
import os, imp
def doimport( clazz, modPart, kw, class_check):
path = os.path.join('lib', modPart, clazz + '.py')
mod = imp.load_source( clazz, path )
item = mod.create(kw)
if class_check(item):
print "im happy"
return item