I want to get low level access to webcam properties using DirectShow's IAMVideoProcAmp.
There are several Python modules )pywin32, pywintypes, comtypes, win32com, pythoncom) that are used in this context and they seem to be related somehow. But I have no clue where to start.
I found some examples (here, here, here) but I could not figure out how to get a IID / CLSID to use like
import win32com.client
clsid='{9BA05972-F6A8-11CF-A442-00A0C90A8F39}'
ShellWindows=win32com.client.Dispatch(clsid)
or with a clear name like
import win32com.client
xl = win32com.client.Dispatch("Excel.Application")
or
from comtypes import client, GUID
graph = client.CreateObject(some_CLSID)
graph.QueryInterface(...)
Can someone help me with this?
I found another example (dshow.py), but it has some dependencies that I could not find (interfaces, uuids).
This page from Microsoft lists the procedures as
Call QueryInterface on the capture filter for the IAMVideoProcAmp interface.
or
Query the capture filter for the IAMCameraControl.
and states some C++ code for this:
// Query the capture filter for the IAMVideoProcAmp interface.
IAMVideoProcAmp *pProcAmp = 0;
hr = pCap->QueryInterface(IID_IAMVideoProcAmp, (void**)&pProcAmp);
hr = m_pProcAmp->GetRange(VideoProcAmp_Brightness, &Min, &Max, &Step,
&Default, &Flags);
Edit:
I finally found some code that looks good so far:
jaraco
It seem to do exactly what I am trying to write and uses some elements from
DirectShow (see here):
from comtypes.gen.DirectShowLib import (FilterGraph, CaptureGraphBuilder2, ...)
jaraco.video claims to be "a port of the VideoCapture module in pure Python using ctypes and comtypes."
It is using a DirectShow.tlb file (whatever that is) to get the definitions
into comtypes
A type library (.tlb) is a binary file that stores information about a
COM or DCOM object's properties and methods in a form that is
accessible to other applications at runtime.
Identifying required values to replicate the code
At a second glance at the code excerpt at the end of your post, I realize that you only need the IID and not the CLSID for IAMVideoProcAmp in order to acquire an instance of it.
Looking at line 8733 of this source of strmif.h, noted as the required header for the interface, I found that IID_IAMVideoProcAmp is C6E13360-30AC-11d0-A18C-00A0C9118956.
Above this section of strmif.h, you can identify what integers correspond to which properties in the tagVideoProcAmpProperty enum, such as 0 for VideoProcAmp_Brightness. Below this section of strmif.h, you can identify what integers correspond to which functions in the IAMVideoProcAmpVtbl VTable, such as 3 for GetRange. I am unfamiliar with how to interact with COM objects in Python, but in Java you would need to determine these property and function indices in order to replicate the C++ code excerpts that demonstrate how to acquire an instance of IAmVideoProcAmp.
Acquiring an instance of IAMVideoProcAmp
As you may have noticed, the C++ code excerpt invokes QueryInterface on something called pCap and notes that you need to "Query the capture filter for the IAMVideoProcAmp interface." This sibling of the article you linked explains how to do this:
To create a DirectShow capture filter for the device, call the IMoniker::BindToObject method to get an IBaseFilter pointer. Then call IFilterGraph::AddFilter to add the filter to the filter graph:
IBaseFilter *pCap = NULL;
hr = pMoniker->BindToObject(0, 0, IID_IBaseFilter, (void**)&pCap);
if (SUCCEEDED(hr))
{
hr = m_pGraph->AddFilter(pCap, L"Capture Filter");
}
Now that you know how to acquire pCap, you notice you need something called pMoniker, which is defined earlier in the same article. The code is fairly long so I omit it here.
Doing all of this in Python
As I noted earlier, I have never used any Python COM library, so I cannot easily whip up an example, but your goal should be to replicate in Python the function invocations made in the C++ examples to acquire an instance of IAMVideoProcAmp and modifying them to suit your needs.
I finally found some example library that is working:
jaraco
It does exactly what I am trying to achive and uses some elements from
DirectShow (see here):
from comtypes.gen.DirectShowLib import (FilterGraph, CaptureGraphBuilder2, ...)
jaraco.video claims to be "a port of the VideoCapture module in pure Python using ctypes and comtypes."
It is using a DirectShow.tlb file (whatever that is) to get the definitions
into comtypes
A type library (.tlb) is a binary file that stores information about a
COM or DCOM object's properties and methods in a form that is
accessible to other applications at runtime.
The imports are auto-generated in __init__.py and can be used easily:
from api.objects import ..., IMediaControl, IAMVideoProcAmp, IAMCameraControl, ...
and can be used
def _get_camera_control(self):
return self._get_graph_builder_interface(IAMCameraControl)
def get_camera_control_property(self, i):
video_properties = self._get_camera_control()
return video_properties.Get(i)
Then you can use the functions in combination with the enum stated in the docs, e.g.
# CameraControl_Exposure = 4
print(d.get_camera_control_property(4))
Using python's wmi module to create a vss snapshot, I've found that the parameters don't work unless I reverse them:
import wmi
def vss_create():
shadow_copy_service = wmi.WMI(moniker='winmgmts:\\\\.\\root\\cimv2:Win32_ShadowCopy')
res = shadow_copy_service.Create('ClientAccessible', 'C:\\')
In the msdn docs, the function is instead supposed to be used this way:
Win32_ShadowCopy.Create("C:\\", "ClientAccessible");
Why is this the case, and is there a way to use the intended order?
Summary
It looks like the parameter ordering for wmi object's methods is reversed from normal by the PyWin32 layer, and this behaviour has been present for at least five years. The relevant wmi spec says that a wmi client can pass the parameters in any order, so PyWin32 is not 'wrong' to do this, although I can't tell if it's deliberate or accident. I speculate that it's unlikely to change, for backwards compatibility reasons, but you can work around this and put the parameters in the order you want by specifying them as keyword parameters: Create(Volume=, Context=).
Details
NB. In the below details, I'm trying to go down in layers from the Python WMI module code to WMI objects accessed by COM in PyWin32 code, to WMI objects as documented and used in other languages, to WMI object specification by MOF files, to specification documents. There's several layers and I write "WMI" a lot, meaning different things at different layers.
When you say "Python's wmi module" do you mean Tim Golden's Python WMI module (link to source code) that builds on PyWin32?
When you get a Python WMI object from the wmi module, the initialization steps it goes through are inside the class _wmi_object, and include querying the underlying wmi object for its available methods:
for m in ole_object.Methods_:
self.methods[m.Name] = None
I'm going to skip beneath Python's wmi module, and use PyWin32 directly to look at what you get when querying a WMI COM object for its available methods:
>>> from win32com.client import GetObject
>>> vss = GetObject('winmgmts:\\\\.\\root\\cimv2:Win32_ShadowCopy')
>>> [method.Name for method in list(vss.Methods_)]
[u'Create', u'Revert']
And we see the Win32_ShadowCopy object has the methods Create and Revert. So that's where the Python wmi wrapper first learns about the Create method you are using.
From there, the Python WMI wrapper class does some setup work which I haven't traced through fully, but it seems to initialize class _wmi_method once for each available method of the COM object. This class includes these initialization steps:
self.method = ole_object.Methods_ (method_name)
self.in_parameter_names = [(i.Name, i.IsArray) for i in self.in_parameters.Properties_]
A list comprehension to get the available parameters for each method. Going back to my testing to explore that without the Python WMI layer, it gives output like this:
>>> CreateMethod = vss.Methods_('Create')
>>> [n.Name for n in list(CreateMethod.InParameters.Properties_)]
[u'Context', u'Volume']
This example test shows the PyWin32 later, the COM object for Win32_ShadowCopy, the Create method - lists its available parameters in the order you are seeing - the "wrong" order. The Python WMI layer is picking up that ordering.
When you call the Win32_ShadowCopy object's Create() method through Python WMI's wrapper, the _wmi_method does this:
def __call__ (self, *args, **kwargs):
for n_arg in range (len (args)):
arg = args[n_arg]
parameter = parameters.Properties_[n_arg]
parameter.Value = arg
In other words; it pairs up the parameters you pass in (*args) with the stored parameter list, one by one, taking the parameters in the order you pass them, and pairing them with the method parameters in the order WMI returned them - i.e. it's not intelligent, it just links the first parameter you enter with 'Context' and the second with 'Volume' and gets them backwards, and your code crashes.
The call method also includes Python's **kwargs parameter which takes all given keywords, suggesting you can do
Create(Volume='C:\\', Context="ClientAccessible")
and put them in the order you want by using them as keyword arguments. (I haven't tried).
I have tried tracing the .Properties_ lookup through PyWin32com to try and identify where the ordering comes from at the lower layers, and it goes through a long chain of dynamic and cached lookups. I can't see what happens and I don't understand enough COM or PyWin32 to know what kinds of things to be looking for, so that's a dead end for me.
Taking a different approach and trying to find out from the WMI object setup files where the ordering comes from: run mofcomp.exe which ships with Windows and processes Managed Object Format (MOF) files... click Connect, Create Class "Win32_ShadowCopy"; Click the "Create" method in the methods list, then click the "Edit Method" button; then click "Edit Input Arguments" then click "Show MOF", and get this result:
[abstract]
class __PARAMETERS
{
[in, ID(0): DisableOverride ToInstance] string Volume;
[in, ID(1): DisableOverride ToInstance] string Context = "ClientAccessible";
};
That's the "correct" order of the parameters coming out of the Windows MOF files, with numeric IDs for the parameters - implying they have a correct ordering 0, 1, etc.
c:\windows\system32\wbem\vss.mof, the MOF file which appears to cover the Volume Shadow Copy objects contains this:
[static,implemented,constructor] uint32 Create([in] string Volume,[in] string Context = "ClientAccessible",[out] string ShadowID);
and the PowerShell example in the comments at this MSDN link includes $class.create("C:\", "ClientAccessible").
So those three things all tie up with the same ordering and implies there is a correct or standard ordering.
That leaves me thinking of these possibilities:
There is ordering information coming out of PythonCOM and the wmi module should look at it, but doesn't. - I have looked around quickly, and can't find ID / ordering data with the parameters list, so that seems unlikely.
There is ordering information somewhere unknown to me which the PyWin32 COM layer should be looking at but doesn't. - Not sure here.
There is no official ordering. Trying to confirm this point, we get a fun chain:
What is WMI? Microsoft's implementation of standard management frameworks WBEM and CIM, specified by the DTMF. (DTMF = Distributed Management Task Force, WBEM is Web Based Enterprise Management and CIM is the Common Information Model).
MOF is the Managed Object Format, a text representation of CIM
This document: http://www.dmtf.org/sites/default/files/standards/documents/DSP0221_3.0.0.pdf appears to be the MOF specification. Check section 7.3.3 Class Declaration, from page 18:
line 570:
"A method can have zero or more parameters".
lines 626 through 628:
Method parameters are identified by name and not by position and
clients invoking a method can pass the corresponding arguments in
any order. Therefore parameters with default values can be added to
the method signature at any position.
I don't know for sure if that's an authoritative and current specification, nor have I read all of it looking for exceptions, but it sounds like you should use named parameters.
The WMI objects and methods have a MOF definition, and the MOF specification says you shouldn't rely on the parameter ordering; however, accessing the WMI objects via COM via PyWin32 is showing a different ordering to (MSDN docs, the MOF file and the PowerShell examples). I still don't know why.
And Googling that leads me to this mailing list post by Tim Golden, author of the Python wmi module, saying basically the same thing as I've just found, except five years ago:
the method definition picks up the parameters in the order in which WMI returns them [..]
I've got no idea if there is any guarantee about the order of parameters [..]
Glancing at a few other method definitions, it does seem as though WMI is consistently returning params in the reverse order of their definition in the MOF.
At this point, it looks like PyWin32 is returning a reversed list to the typical Windows parameter order, but is that a bug if the CIM managed object method parameter list specification document explicitly says don't rely on the parameter ordering?
I have an application, written in Python, which is used by a fairly technical audience (scientists).
I'm looking for a good way to make the application extensible by the users, i.e. a scripting/plugin architecture.
I am looking for something extremely lightweight. Most scripts, or plugins, are not going to be developed and distributed by a third-party and installed, but are going to be something whipped up by a user in a few minutes to automate a repeating task, add support for a file format, etc. So plugins should have the absolute minimum boilerplate code, and require no 'installation' other than copying to a folder (so something like setuptools entry points, or the Zope plugin architecture seems like too much.)
Are there any systems like this already out there, or any projects that implement a similar scheme that I should look at for ideas / inspiration?
Mine is, basically, a directory called "plugins" which the main app can poll and then use imp.load_module to pick up files, look for a well-known entry point possibly with module-level config params, and go from there. I use file-monitoring stuff for a certain amount of dynamism in which plugins are active, but that's a nice-to-have.
Of course, any requirement that comes along saying "I don't need [big, complicated thing] X; I just want something lightweight" runs the risk of re-implementing X one discovered requirement at a time. But that's not to say you can't have some fun doing it anyway :)
module_example.py:
def plugin_main(*args, **kwargs):
print args, kwargs
loader.py:
def load_plugin(name):
mod = __import__("module_%s" % name)
return mod
def call_plugin(name, *args, **kwargs):
plugin = load_plugin(name)
plugin.plugin_main(*args, **kwargs)
call_plugin("example", 1234)
It's certainly "minimal", it has absolutely no error checking, probably countless security problems, it's not very flexible - but it should show you how simple a plugin system in Python can be..
You probably want to look into the imp module too, although you can do a lot with just __import__, os.listdir and some string manipulation.
Have a look at at this overview over existing plugin frameworks / libraries, it is a good starting point. I quite like yapsy, but it depends on your use-case.
While that question is really interesting, I think it's fairly hard to answer, without more details. What sort of application is this? Does it have a GUI? Is it a command-line tool? A set of scripts? A program with an unique entry point, etc...
Given the little information I have, I will answer in a very generic manner.
What means do you have to add plugins?
You will probably have to add a configuration file, which will list the paths/directories to load.
Another way would be to say "any files in that plugin/ directory will be loaded", but it has the inconvenient to require your users to move around files.
A last, intermediate option would be to require all plugins to be in the same plugin/ folder, and then to active/deactivate them using relative paths in a config file.
On a pure code/design practice, you'll have to determine clearly what behavior/specific actions you want your users to extend. Identify the common entry point/a set of functionalities that will always be overridden, and determine groups within these actions. Once this is done, it should be easy to extend your application,
Example using hooks, inspired from MediaWiki (PHP, but does language really matters?):
import hooks
# In your core code, on key points, you allow user to run actions:
def compute(...):
try:
hooks.runHook(hooks.registered.beforeCompute)
except hooks.hookException:
print('Error while executing plugin')
# [compute main code] ...
try:
hooks.runHook(hooks.registered.afterCompute)
except hooks.hookException:
print('Error while executing plugin')
# The idea is to insert possibilities for users to extend the behavior
# where it matters.
# If you need to, pass context parameters to runHook. Remember that
# runHook can be defined as a runHook(*args, **kwargs) function, not
# requiring you to define a common interface for *all* hooks. Quite flexible :)
# --------------------
# And in the plugin code:
# [...] plugin magic
def doStuff():
# ....
# and register the functionalities in hooks
# doStuff will be called at the end of each core.compute() call
hooks.registered.afterCompute.append(doStuff)
Another example, inspired from mercurial. Here, extensions only add commands to the hg commandline executable, extending the behavior.
def doStuff(ui, repo, *args, **kwargs):
# when called, a extension function always receives:
# * an ui object (user interface, prints, warnings, etc)
# * a repository object (main object from which most operations are doable)
# * command-line arguments that were not used by the core program
doMoreMagicStuff()
obj = maybeCreateSomeObjects()
# each extension defines a commands dictionary in the main extension file
commands = { 'newcommand': doStuff }
For both approaches, you might need common initialize and finalize for your extension.
You can either use a common interface that all your extension will have to implement (fits better with second approach; mercurial uses a reposetup(ui, repo) that is called for all extension), or use a hook-kind of approach, with a hooks.setup hook.
But again, if you want more useful answers, you'll have to narrow down your question ;)
Marty Allchin's simple plugin framework is the base I use for my own needs. I really recommand to take a look at it, I think it is really a good start if you want something simple and easily hackable. You can find it also as a Django Snippets.
I am a retired biologist who dealt with digital micrograqphs and found himself having to write an image processing and analysis package (not technically a library) to run on an SGi machine. I wrote the code in C and used Tcl for the scripting language. The GUI, such as it was, was done using Tk. The commands that appeared in Tcl were of the form "extensionName commandName arg0 arg1 ... param0 param1 ...", that is, simple space-separated words and numbers. When Tcl saw the "extensionName" substring, control was passed to the C package. That in turn ran the command through a lexer/parser (done in lex/yacc) and then called C routines as necessary.
The commands to operate the package could be run one by one via a window in the GUI, but batch jobs were done by editing text files which were valid Tcl scripts; you'd pick the template that did the kind of file-level operation you wanted to do and then edit a copy to contain the actual directory and file names plus the package commands. It worked like a charm. Until ...
1) The world turned to PCs and 2) the scripts got longer than about 500 lines, when Tcl's iffy organizational capabilities started to become a real inconvenience. Time passed ...
I retired, Python got invented, and it looked like the perfect successor to Tcl. Now, I have never done the port, because I have never faced up to the challenges of compiling (pretty big) C programs on a PC, extending Python with a C package, and doing GUIs in Python/Gt?/Tk?/??. However, the old idea of having editable template scripts seems still workable. Also, it should not be too great a burden to enter package commands in a native Python form, e.g.:
packageName.command( arg0, arg1, ..., param0, param1, ...)
A few extra dots, parens, and commas, but those aren't showstoppers.
I remember seeing that someone has done versions of lex and yacc in Python (try: http://www.dabeaz.com/ply/), so if those are still needed, they're around.
The point of this rambling is that it has seemed to me that Python itself IS the desired "lightweight" front end usable by scientists. I'm curious to know why you think that it is not, and I mean that seriously.
added later: The application gedit anticipates plugins being added and their site has about the clearest explanation of a simple plugin procedure I've found in a few minutes of looking around. Try:
https://wiki.gnome.org/Apps/Gedit/PythonPluginHowToOld
I'd still like to understand your question better. I am unclear whether you 1) want scientists to be able to use your (Python) application quite simply in various ways or 2) want to allow the scientists to add new capabilities to your application. Choice #1 is the situation we faced with the images and that led us to use generic scripts which we modified to suit the need of the moment. Is it Choice #2 which leads you to the idea of plugins, or is it some aspect of your application that makes issuing commands to it impracticable?
When i searching for Python Decorators, found a simple but useful code snippet. It may not fit in your needs but very inspiring.
Scipy Advanced Python#Plugin Registration System
class TextProcessor(object):
PLUGINS = []
def process(self, text, plugins=()):
if plugins is ():
for plugin in self.PLUGINS:
text = plugin().process(text)
else:
for plugin in plugins:
text = plugin().process(text)
return text
#classmethod
def plugin(cls, plugin):
cls.PLUGINS.append(plugin)
return plugin
#TextProcessor.plugin
class CleanMarkdownBolds(object):
def process(self, text):
return text.replace('**', '')
Usage:
processor = TextProcessor()
processed = processor.process(text="**foo bar**", plugins=(CleanMarkdownBolds, ))
processed = processor.process(text="**foo bar**")
I enjoyed the nice discussion on different plugin architectures given by Dr Andre Roberge at Pycon 2009. He gives a good overview of different ways of implementing plugins, starting from something really simple.
Its available as a podcast (second part following an explanation of monkey-patching) accompanied by a series of six blog entries.
I recommend giving it a quick listen before you make a decision.
I arrived here looking for a minimal plugin architecture, and found a lot of things that all seemed like overkill to me. So, I've implemented Super Simple Python Plugins. To use it, you create one or more directories and drop a special __init__.py file in each one. Importing those directories will cause all other Python files to be loaded as submodules, and their name(s) will be placed in the __all__ list. Then it's up to you to validate/initialize/register those modules. There's an example in the README file.
Actually setuptools works with a "plugins directory", as the following example taken from the project's documentation:
http://peak.telecommunity.com/DevCenter/PkgResources#locating-plugins
Example usage:
plugin_dirs = ['foo/plugins'] + sys.path
env = Environment(plugin_dirs)
distributions, errors = working_set.find_plugins(env)
map(working_set.add, distributions) # add plugins+libs to sys.path
print("Couldn't load plugins due to: %s" % errors)
In the long run, setuptools is a much safer choice since it can load plugins without conflicts or missing requirements.
Another benefit is that the plugins themselves can be extended using the same mechanism, without the original applications having to care about it.
Expanding on the #edomaur's answer may I suggest taking a look at simple_plugins (shameless plug), which is a simple plugin framework inspired by the work of Marty Alchin.
A short usage example based on the project's README:
# All plugin info
>>> BaseHttpResponse.plugins.keys()
['valid_ids', 'instances_sorted_by_id', 'id_to_class', 'instances',
'classes', 'class_to_id', 'id_to_instance']
# Plugin info can be accessed using either dict...
>>> BaseHttpResponse.plugins['valid_ids']
set([304, 400, 404, 200, 301])
# ... or object notation
>>> BaseHttpResponse.plugins.valid_ids
set([304, 400, 404, 200, 301])
>>> BaseHttpResponse.plugins.classes
set([<class '__main__.NotFound'>, <class '__main__.OK'>,
<class '__main__.NotModified'>, <class '__main__.BadRequest'>,
<class '__main__.MovedPermanently'>])
>>> BaseHttpResponse.plugins.id_to_class[200]
<class '__main__.OK'>
>>> BaseHttpResponse.plugins.id_to_instance[200]
<OK: 200>
>>> BaseHttpResponse.plugins.instances_sorted_by_id
[<OK: 200>, <MovedPermanently: 301>, <NotModified: 304>, <BadRequest: 400>, <NotFound: 404>]
# Coerce the passed value into the right instance
>>> BaseHttpResponse.coerce(200)
<OK: 200>
As one another approach to plugin system, You may check Extend Me project.
For example, let's define simple class and its extension
# Define base class for extensions (mount point)
class MyCoolClass(Extensible):
my_attr_1 = 25
def my_method1(self, arg1):
print('Hello, %s' % arg1)
# Define extension, which implements some aditional logic
# or modifies existing logic of base class (MyCoolClass)
# Also any extension class maby be placed in any module You like,
# It just needs to be imported at start of app
class MyCoolClassExtension1(MyCoolClass):
def my_method1(self, arg1):
super(MyCoolClassExtension1, self).my_method1(arg1.upper())
def my_method2(self, arg1):
print("Good by, %s" % arg1)
And try to use it:
>>> my_cool_obj = MyCoolClass()
>>> print(my_cool_obj.my_attr_1)
25
>>> my_cool_obj.my_method1('World')
Hello, WORLD
>>> my_cool_obj.my_method2('World')
Good by, World
And show what is hidden behind the scene:
>>> my_cool_obj.__class__.__bases__
[MyCoolClassExtension1, MyCoolClass]
extend_me library manipulates class creation process via metaclasses, thus in example above, when creating new instance of MyCoolClass we got instance of new class that is subclass of both MyCoolClassExtension and MyCoolClass having functionality of both of them, thanks to Python's multiple inheritance
For better control over class creation there are few metaclasses defined in this lib:
ExtensibleType - allows simple extensibility by subclassing
ExtensibleByHashType - similar to ExtensibleType, but haveing ability
to build specialized versions of class, allowing global extension
of base class and extension of specialized versions of class
This lib is used in OpenERP Proxy Project, and seems to be working good enough!
For real example of usage, look in OpenERP Proxy 'field_datetime' extension:
from ..orm.record import Record
import datetime
class RecordDateTime(Record):
""" Provides auto conversion of datetime fields from
string got from server to comparable datetime objects
"""
def _get_field(self, ftype, name):
res = super(RecordDateTime, self)._get_field(ftype, name)
if res and ftype == 'date':
return datetime.datetime.strptime(res, '%Y-%m-%d').date()
elif res and ftype == 'datetime':
return datetime.datetime.strptime(res, '%Y-%m-%d %H:%M:%S')
return res
Record here is extesible object. RecordDateTime is extension.
To enable extension, just import module that contains extension class, and (in case above) all Record objects created after it will have extension class in base classes, thus having all its functionality.
The main advantage of this library is that, code that operates extensible objects, does not need to know about extension and extensions could change everything in extensible objects.
setuptools has an EntryPoint:
Entry points are a simple way for distributions to “advertise” Python
objects (such as functions or classes) for use by other distributions.
Extensible applications and frameworks can search for entry points
with a particular name or group, either from a specific distribution
or from all active distributions on sys.path, and then inspect or load
the advertised objects at will.
AFAIK this package is always available if you use pip or virtualenv.
You can use pluginlib.
Plugins are easy to create and can be loaded from other packages, file paths, or entry points.
Create a plugin parent class, defining any required methods:
import pluginlib
#pluginlib.Parent('parser')
class Parser(object):
#pluginlib.abstractmethod
def parse(self, string):
pass
Create a plugin by inheriting a parent class:
import json
class JSON(Parser):
_alias_ = 'json'
def parse(self, string):
return json.loads(string)
Load the plugins:
loader = pluginlib.PluginLoader(modules=['sample_plugins'])
plugins = loader.plugins
parser = plugins.parser.json()
print(parser.parse('{"json": "test"}'))
I have spent time reading this thread while I was searching for a plugin framework in Python now and then. I have used some but there were shortcomings with them. Here is what I come up with for your scrutiny in 2017, a interface free, loosely coupled plugin management system: Load me later. Here are tutorials on how to use it.
I've spent a lot of time trying to find small plugin system for Python, which would fit my needs. But then I just thought, if there is already an inheritance, which is natural and flexible, why not use it.
The only problem with using inheritance for plugins is that you dont know what are the most specific(the lowest on inheritance tree) plugin classes are.
But this could be solved with metaclass, which keeps track of inheritance of base class, and possibly could build class, which inherits from most specific plugins ('Root extended' on the figure below)
So I came with a solution by coding such a metaclass:
class PluginBaseMeta(type):
def __new__(mcls, name, bases, namespace):
cls = super(PluginBaseMeta, mcls).__new__(mcls, name, bases, namespace)
if not hasattr(cls, '__pluginextensions__'): # parent class
cls.__pluginextensions__ = {cls} # set reflects lowest plugins
cls.__pluginroot__ = cls
cls.__pluginiscachevalid__ = False
else: # subclass
assert not set(namespace) & {'__pluginextensions__',
'__pluginroot__'} # only in parent
exts = cls.__pluginextensions__
exts.difference_update(set(bases)) # remove parents
exts.add(cls) # and add current
cls.__pluginroot__.__pluginiscachevalid__ = False
return cls
#property
def PluginExtended(cls):
# After PluginExtended creation we'll have only 1 item in set
# so this is used for caching, mainly not to create same PluginExtended
if cls.__pluginroot__.__pluginiscachevalid__:
return next(iter(cls.__pluginextensions__)) # only 1 item in set
else:
name = cls.__pluginroot__.__name__ + 'PluginExtended'
extended = type(name, tuple(cls.__pluginextensions__), {})
cls.__pluginroot__.__pluginiscachevalid__ = True
return extended
So when you have Root base, made with metaclass, and have tree of plugins which inherit from it, you could automatically get class, which inherits from the most specific plugins by just subclassing:
class RootExtended(RootBase.PluginExtended):
... your code here ...
Code base is pretty small (~30 lines of pure code) and as flexible as inheritance allows.
If you're interested, get involved # https://github.com/thodnev/pluginlib
You may also have a look at Groundwork.
The idea is to build applications around reusable components, called patterns and plugins. Plugins are classes that derive from GwBasePattern.
Here's a basic example:
from groundwork import App
from groundwork.patterns import GwBasePattern
class MyPlugin(GwBasePattern):
def __init__(self, app, **kwargs):
self.name = "My Plugin"
super().__init__(app, **kwargs)
def activate(self):
pass
def deactivate(self):
pass
my_app = App(plugins=[MyPlugin]) # register plugin
my_app.plugins.activate(["My Plugin"]) # activate it
There are also more advanced patterns to handle e.g. command line interfaces, signaling or shared objects.
Groundwork finds its plugins either by programmatically binding them to an app as shown above or automatically via setuptools. Python packages containing plugins must declare these using a special entry point groundwork.plugin.
Here are the docs.
Disclaimer: I'm one of the authors of Groundwork.
In our current healthcare product we have a plugin architecture implemented with interface class. Our tech stack are Django on top of Python for API and Nuxtjs on top of nodejs for frontend.
We have a plugin manager app written for our product which is basically pip and npm package in adherence with Django and Nuxtjs.
For new plugin development(pip and npm) we made plugin manager as dependency.
In Pip package:
With the help of setup.py you can add entrypoint of the plugin to do something with plugin manager(registry, initiations, ...etc.)
https://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation
In npm package:
Similar to pip there are hooks in npm scripts to handle the installation.
https://docs.npmjs.com/misc/scripts
Our usecase:
plugin development team is separate from core devopment team now. The scope of plugin development is for integrating with 3rd party apps which are defined in any of the categories of the product. The plugin interfaces are categorised for eg:- Fax, phone, email ...etc plugin manager can be enhanced to new categories.
In your case: Maybe you can have one plugin written and reuse the same for doing stuffs.
If plugin developers has need to use reuse core objects that object can be used by doing a level of abstraction within plugin manager so that any plugins can inherit those methods.
Just sharing how we implemented in our product hope it will give a little idea.