What I want to specifically know is if there is a way to customize a piece of code already running by certain software.
I'll explain quickly the process of this software. When opening, of course there's all those pieces of codes being loaded, mostly divised in two entities, UI and Core, mostly because it will eventually be able to work without an interface (which is not the case right now, but that's not my point here).
So the soft is structure mostly like this:
software/
core/
some_functions.py
ui/
function_a.py
class processUIBase(inheritanceUI):
def __init__(blabla):
code = foo
def method(self):
print "hello world"
def createWidget(self):
widget = QLabel('Widget setting:')
layout.addWidget(widget)
return widget
function_b.py
[...]
Then comes in the plugins. There's a folder the software checks to bring the plugins, which works in a similar fashion (core/Ui).
/home/.software/
Startup/ (What is loaded as plugins with the core)
__init__.py
plugins_that_works_well.py
StartupUI/ (What is loaded as plugins with the UI)
__init__.py
menu.py
Basically, all this works right now.
But what I'd like to know is if there's a way for me to make something in the plugins folder (so without touching the software's base scripts) to override, for example, the function 'createWidget', to add some new lines (eg: widget_b = QCheckbox, layout,addWidget(widget_b).
How can I achieve that?
Thanks!
It's difficult to change the behavior of code you can't directly modify. One solution that's kind of risky, but which is possible to do, is to swap out the original version of the function/class in the module with your own version, such that your own version acts very similar.
Here, you might be able to make a subclass and reassign it:
from software.ui import function_a
# maintain a reference to the original version, so it doesn't get lost
_ProcessUIBase = function_a.ProcessUIBase
# make a new subclass that does what you want
class ProcessUICustom(_ProcessUIBase):
# override the superclass's method
def createWidget(self):
# add the functionality you want
self.widget_b = QCheckbox, layout,addWidget(widget_b)
# then do the original functionality
return super().createWidget()
# replace the original class with our new version
function_a.ProcessUIBase = ProcessUICustom
This change will reflect across the rest of the program (as soon as someone imports the module in which you do this) for anything else that tries to use function_a.ProcessUIBase. It won't affect any other programs that import the same library, and the change will revert as soon as the program finishes executing. Also, there are ways around it by using importlib to reload the original function_a, which will wipe away the change. But it might be what you're looking for in this case.
Note that, if something else in the main part of the program has already imported ProcessUIBase directly (i.e. not just the module it's in, but the actual class):
from software.ui.function_a import ProcessUIBase
then this won't affect the version that's already been imported.
In essence, this is like changing someone's address in a directory. You can misdirect new people who look for that person, but if someone else already knows where to find that person, they won't be fooled.
Related
I am trying to code up a module which has two classes. First class is called as TextProcessing
class TextProcessing(object):
""" To carry out text processing
"""
def __init__(self,):
pass
It has various methods in there for pre-processing text.
Similary other class is for other data wrangling on pre-processed data.
I am saving these two classes in a python file to make it a module.
Now lets say a user downloads this python module and would now want to run the various methods of each class.
I wanted to provide some sort of documentation about the module, methods of each class to a user when she imports the module so that she is aware of which function to call and what parameters to pass.
Think of how a scikit learn documentation is on their documentation page.
http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html
Even the documentation we get to see when we do
help(some_python_module)
is fine too.
Issue is I don't have a documentation page like sklearn to show documentation. And I wanted a user to know documentation of various methods she can use once she imports the module in python console.
Is there a way I can print that documentation info to the console when a user imports the module?
It can show the doc string of each Class and Method.
This is a very weird thing to do, but it's definitely possible.
The easiest thing to do is just to call help. While it's intended to be called from the interactive prompt, there's nothing stopping you from calling it from your own code.
Of course you could instead extract the docstrings (they're stored as __doc__ on every module, class, and function), textwrap them yourself, and print them out, but if you're trying to reproduce the same thing help does, that's a lot of work for no real benefit.
The only tricky bit is that the thing you want to invoke the help system on is "this current module". How do you refer to that? It's a bit clunky, but you have this current module's name as __name__, so you can look it up in sys.modules.
So:
"""Helpful module"""
import sys
class Spam:
"""Classy class"""
def eggs(self):
"Functional function"
return 2
help(sys.modules[__name__])
Now, when you import helpful for the first time in a session, it will print out the help.
Of course that will be pretty odd if someone's trying to run a script that does an import helpful, rather than doing it from an interactive session. So you may want to only do this in interactive sessions, by checking sys.flags:
if sys.flags.interactive:
help(sys.modules[__name__])
What if someone does an import otherthing, and that otherthing does an import helpful? You'll get the same help, which may be confusing.
If that's a problem, the only real option I can think of is to check whether the calling frame comes from the top-level script (and that the flags are interactive). That's pretty hacky, and something you shouldn't even consider unless you really need to, so I'll just direct you to the inspect module and hope you don't need it.
I have a syntax/understanding problem in an application using Python 3, PyQt5 and Qt5. I am unsure which of these is causing the problem.
I am charged with porting a GUI application which worked under Windows to Linux, with newer versions of libraries. I do not have access to the original running under Windows.
In several places I see:
menu = QMenu(self)
action = menu.addAction("Some string")
action.triggered[()].connect(self.handler)
I presume this used to work. I am porting to Python 3.5, PyQt 5.7 & Qt 5.7. I have reason to believe the code was written for earlier versions of each of these.
Executing it now generates a 'there is no matching overloaded signal' error, on the action.triggered[()] segment.
By guesswork and looking at a couple of examples I found somewhere, I have changed the last line to:
action.triggered.connect(self.handler)
and it seems to work now.
Could someone explain what the original triggered[()] syntax meant/worked, and which "product" the evident change was in --- it would be nice to be able to read about where this got changed? Is my replacement by simply triggered correct/same behaviour?
The relevant change happened in PyQt-5.3. It is not entirely safe to simply remove the old syntax, as this could easily create a subtle bug in both current and future code.
The issue is that certain signals like triggered will always send a default boolean value unless you take steps to eliminate it. Qt defines these signals like this:
triggered(bool checked = false)
PyQt previously implemented this as two overloads:
triggered(bool checked)
triggered()
and if you wanted to explicitly select the latter overload, you had to use the following slightly awkward sytax:
triggered[()].connect(slot)
But this is now no longer an option, as only the first overload is implemented. So to get exactly the same behaviour, it is necessary to either wrap the handler like this:
triggered.connect(lambda checked: slot())
or decorate it like this:
#QtCore.pyqtSlot()
def handleTriggered(self):
pass
Otherwise, it is quite easy to fall into a little trap. Imagine you have a method defined like this:
def frobnicate(self, foobar=True):
if foobar:
# do something nice
else:
# empty my home directory without prompting
Later on, you decide to hook this up to a button, like this:
self.button.clicked.connect(self.frobnicate)
Which all seems perfectly fine, until you find out that the clicked signal works just like triggered, and always sends False by default...
Of course, if you're absolutely certain that you will never connect slots which take arguments to signals like triggered and clicked, you could get away with connecting them all in the simplistic fashion shown above. But, really, that is just an accident waiting to happen...
I'm using Sphinx to document a project that depends on wxPython, using the autodocs extension so that it will automatically generate pages from our docstrings. The autodocs extension automatically operates on every module you import, which is fine for our packages but is a problem when we import a large external library like wxPython. Thus, instead of letting it generate everything from wxPython I'm using the unittest.mock library module (previously the external package Mock). The most basic setup works fine for most parts of wxPython, but I've run into a situation I can't see an easy way around (likely because of my relative unfamiliarity with mock until this week).
Currently, the end of my conf.py file has the following:
MOCK_MODULES = ['wx.lib.newevent'] # I've skipped irrelevant entries...
for module_name in MOCK_MODULES:
sys.modules[module_name] = mock.Mock()
For all the wxPython modules but wx.lib.newevent, this works perfectly. However, here I'm using the newevent.NewCommandEvent() function[1] to create an event for a particular scenario. In this case, I get a warning on the NewCommandEvent() call with the note TypeError: 'Mock' object is not iterable.
While I can see how one would use patching to handle this for building out unit tests (which I will be doing in the next month!), I'm having a hard time seeing how to integrate that at a simple level in my Sphinx configuration.
Edit: I've just tried using MagicMock() as well; this still produces an error at the same point, though it now produces ValueError: need more than 0 values to unpack. That seems like a step in the right direction, but I'm still not sure how to handle this short of explicitly setting it up for this one module. Maybe that's the best solution, though?
Footnotes
Yes, that's a function, naming convention making it look like a class notwithstanding; wxPython follows the C++ naming conventions which are used throughout the wxWidgets toolkit.
From the error, it looks like it is actually executing newevent.NewCommandEvent(), so I assume that somewhere in your code you have a top-level line something like this:
import wx.lib.newevent
...
event, binder = wx.lib.newevent.NewCommandEvent()
When autodoc imports the module, it tries to run this line of code, but since NewCommandEvent is actually a Mock object, Python can't bind its output to the (event, binder) tuple. There are two possible solutions. The first is to change your code to that this is not executed on import, maybe by wrapping it inside if __name__ == '__main__'. I would recommend this solution because creating objects like this on import can often have preblematic side effects.
The second solution is to tell the Mock object to return appropriate values thus:
wx.lib.newevent.NewCommandEvent = mock.Mock(return_value=(Mock(), Mock()))
However, if you are doing anything in your code with the returned values you might run into the same problem further down the line.
Obviously, registering classes in Python is a major use-case for metaclasses. In this case, I've got a serialization module that currently uses dynamic imports to create classes and I'd prefer to replace that with a factory pattern.
So basically, it does this:
data = #(Generic class based on serial data)
moduleName = data.getModule()
className = data.getClass()
aModule = __import__(moduleName)
aClass = getattr(aModule, className)
But I want it to do this:
data = #(Generic class based on serial data)
classKey = data.getFactoryKey()
aClass = factory.getClass(classKey)
However, there's a hitch: If I make the factory rely on metaclasses, the Factory only learns about the existence of classes after their modules are imported (e.g., they're registered at module import time). So to populate the factory, I'd have to either:
manually import all related modules (which would really defeat the purpose of having metaclasses automatically register things...) or
automatically import everything in the whole project (which strikes me as incredibly clunky and ham-fisted).
Out of these options, just registering the classes directly into a factory seems like the best option. Has anyone found a better solution that I'm just not seeing? One option might be to automatically generate the imports required in the factory module by traversing the project files, but unless you do that with a commit-hook, you run the risk of your factory getting out of date.
Update:
I have posted a self-answer, to close this off. If anyone knows a good way to traverse all Python modules across nested subpackages in a way that will never hit a cycle, I will gladly accept that answer rather than this one. The main problem I see happening is:
\A.py (import Sub.S2)
\Sub\S1.py (import A)
\Sub\S2.py
\Sub\S3.py (import Sub.S2)
When you try to import S3, it first needs to import Main (otherwise it won't know what a Sub is). At that point, it tries to import A. While there, the __init__.py is called, and tries to register A. At this point, A tries to import S1. Since the __init__.py in Sub is hit, it tries to import S1, S2, and S3. However, S1 wants to import A (which does not yet exist, as it is in the process of being imported)! So that import fails. You can switch how the traversal occurs (i.e., depth first rather than breadth first), but you hit the same issues. Any insight on a good traversal approach for this would be very helpful. A two-stage approach can probably solve it (i.e., traverse to get all module references, then import as a flat batch). However, I am not quite sure of the best way to handle the final stage (i.e., to know when you are done traversing and then import everything). My big restriction is that I do not want to have a super-package to deal with (i.e., an extra directory under Sub and A). If I had that, it could kick off traversal, but everything would need to import relative to that for no good reason (i.e., all imports longer by an extra directory). Thusfar, adding a special function call to sitecustomize.py seems like my only option (I set the root directory for the package development in that file anyway).
The solution I found to this was to do all imports on the package based off of a particular base directory and have special __init__.py functions for all of the ones that might have modules with classes that I'd want to have registered. So basically, if you import any module, it first has to import the base directory and proceeds to walk every package (i.e., folder) with a similar __init__.py file.
The downside of this approach is that the same modules are sometimes imported multiple times, which is annoying if anyone leaves code with side effects in a module import. However, that's bad either way. Unfortunately, some major packages (cough, cough: Flask) have serious complaints with IDLE if you do this (IDLE just restarts, rather than doing anything). The other downside is that because modules import each other, sometimes it attempts to import a module that it is already in the process of importing (an easily caught error, but one I'm still trying to stamp out). It's not ideal, but it does get the job done. Additional details on the more specific issue are attached, and if anyone can offer a better answer, I will gladly accept it.
I have a library that interfaces with an external tool and exposes some basic keywords to use from robotframework; This library is implemented as a python package, and I would like to implement extended functionality that implements complex logic, and exposes more keywords, within modules of this package. The package is given test case scope, but I'm not entirely sure how this works. If I suggest a few ways I have thought of, could someone with a bit more knowledge let me know where I'm on the right track, and where I'm barking up the wrong tree...
Use an instance variable - if the scope is such that the python interpreter will see the package as imported by the current test case (i.e this is treated as a separate package in different test cases rather than a separate instance of the same package), then on initialisation I could set a global variable INSTANCE to self and then from another module within the package, import INSTANCE and use it.
Use an instance dictionary - if the scope is such that all imports see the package as the same, I could use robot.running.context to set a dictionary key such that there is an item in the instance dictionary for each context where the package has been imported - this would then mean that I could use the same context variable as a lookup key in the modules that are based on this. (The disadvantage of this one is that it will prevent garbage collection until the package itself is out of scope, and relies on it being in scope persistently.)
A context variable that I am as of yet unaware of that will give me the instance that is in scope. The docs are fairly difficult to search, so it's fully possible that there is something that I'm missing that will make this trivial. Also just as good would be something that allowed me to call the keywords that are in scope.
Some excellent possibility I haven't considered....
So can anyone help?
Credit for this goes to Kevin O. from the robotframework user group, but essentially the magic lives in robot.libraries.BuiltIn.BuiltIn().get_library_instance(library_name) which can be used like this:
from robot.libraries.BuiltIn import BuiltIn
class SeleniumTestLibrary(object):
def element_should_be_really_visible(self):
s2l = BuiltIn().get_library_instance('Selenium2Library')
element = s2l._element_find(locator, True, False)
It sounds like you are talking about monkeypatching the imported code, so that other modules which import that package will also see your runtime modifications. (Correct me if I'm wrong; there are a couple of bits in your question that I'm not quite following)
For simple package imports, this should work:
import my_package
def method_override():
return "Foo"
my_package.some_method = method_override
my_package, in this case, refers to the imported module, and is not just a local name, so other modules will see the overridden method.
This won't work in cases where other code has already done
from my_package import some_method
Since in that case, some_method is a local name in the place it is imported. If you replace the method elsewhere, that change won't be seen.
If this is happening, then you either need to change the source to import the entire module, or patch a little bit deeper, by replacing method internals:
import my_package
def method_override():
return "Foo"
my_package.some_method.func_code = method_override.func_code
At that point, it doesn't matter how the method was imported in any other module; the code object associated with the method has been replaced, and your new code will run rather than the original.
The only thing to worry about in that case is that the module is imported from the same path in every case. The Python interpreter will try to reuse existing modules, rather than re-import and re-initialize them, whenever they are imported from the same path.
However, if your python path is set up to contain two directories, say: '/foo' and '/foo/bar', then these two imports
from foo.bar import baz
and
from bar import baz
would end up loading the module twice, and defining two versions of any objects (methods, classes, etc) in the module. If that happens, then patching one will not affect the other.
If you need to guard against that case, then you may have to traverse sys.modules, looking for the imported package, and patching each version that you find. This, of course, will only work if all of the other imports have already happened, you can't do that pre-emptively (without writing an import hook, but that's another level deeper again :) )
Are you sure you can't just fork the original package and extend it directly? That would be much easier :)