I have a memoizer decorator class in a library, as such:
class memoizer(object):
def __init__(self, f):
"some code here"
def __call__(self, *args, **kwargs):
"some code here"
When I use it for functions in the library, I use #memoizer. However, I'd like to have the client (ie the programmer using the library) initialize the memoization class from outside the library with some arguments so that it holds for all uses of the decorator used in the client program. Particularly, this particular memoization class saves the results to a file, and I want the client to be able to specify how much memory the files can take. Is this possible?
You can achieve this using decorator factory:
class DecoratorFactory(object):
def __init__(self, value):
self._value = value
def decorator(self, function):
def wrapper(*args, **kwargs):
print(self._value)
return function(*args, **kwargs)
return wrapper
factory = DecoratorFactory("shared between all decorators")
#factory.decorator
def dummy1():
print("dummy1")
#factory.decorator
def dummy2():
print("dummy2")
# prints:
# shared between all decorators
# dummy1
dummy1()
# prints:
# shared between all decorators
# dummy2
dummy2()
If you don't like factories you can create global variables within some module and set them before usage of our decorators (not nice solution, IMO factory is more clean).
Related
I'm implementing a RESTful web service in python and would like to add some QOS logging functionality by intercepting function calls and logging their execution time and so on.
Basically i thought of a class from which all other services can inherit, that automatically overrides the default method implementations and wraps them in a logger function. What's the best way to achieve this?
Something like this? This implictly adds a decorator to your method (you can also make an explicit decorator based on this if you prefer that):
class Foo(object):
def __getattribute__(self,name):
attr = object.__getattribute__(self, name)
if hasattr(attr, '__call__'):
def newfunc(*args, **kwargs):
print('before calling %s' %attr.__name__)
result = attr(*args, **kwargs)
print('done calling %s' %attr.__name__)
return result
return newfunc
else:
return attr
when you now try something like:
class Bar(Foo):
def myFunc(self, data):
print("myFunc: %s"% data)
bar = Bar()
bar.myFunc(5)
You'll get:
before calling myFunc
myFunc: 5
done calling myFunc
What if you write a decorator on each functions ? Here is an example on python's wiki.
Do you use any web framework for doing your webservice ? Or are you doing everything by hand ?
Given the code below, I am trying to identify the arguments sent to decorators that are used by a class.
Specifically, I am trying to identify the value 100 that is passed into the make_hyper decorator that is used in the Dog class.
Optimally, I need to be able to get this value without running the method directly since the actual method code I am working with takes a long time to run.
import inspect
def make_hyper(new_volume):
def decorator(decorated_method):
def wrapped_method(self, *args, **kwargs):
self.volume = new_volume
return decorated_method(self, *args, **kwargs)
return wrapped_method
return decorator
class Dog(object):
def __init__(self, name):
self.name = name
self.volume = 1
#make_hyper(new_volume=100)
def bark(self):
if self.volume >= 10:
print('[{}]: BARK!!'.format(self.name))
elif self.volume >= 5:
print('[{}]: Bark!'.format(self.name))
else:
print('[{}]: Bark'.format(self.name))
I have tried to use inspect.getargspec and a handful of other things and have (done my best to) scour Stackoverflow and the Internet at large but I cannot find a solution.
Any help would be greatly appreciated! Thanks for your time!
UPDATE:
Someone asked me to clarify what I was trying to do. Apologies for not being more clear.
The actual code that the above toy represents is a test automation framework that includes decorators specifying test tags:
class TestTags(object):
WIFI = 'wifi'
BLE = 'ble'
NIGHTLY = 'nightly'
REGRESSION = 'regression'
class TestBase(unittest.TestCase):
<define common test stuff and test decorators>
class ThingTester(TestBase):
#TestBase.tags(TestTags.WIFI, TestTags.BLE, TestTags.REGRESSION)
def test_all_the_things(self):
<test all the things>
# what I'm trying to get
test_tags = ???
print(test_tags) # prints out ('wifi', 'ble', 'regression')
I wrote a utility that goes through all the test modules, classes and individual tests and creates an HTML+JavaScript page that defines a Test Plan. One piece of data that I want to add to the page is what tags are associated with each test.
I found that I can make the tags decorator save an attribute to the class (i.e. self.tags = tags) but this requires the test to be run before the value is saved to the TestBase object and I need to be able to generate the test plan independently of running the tests.
I'm not sure if I'm understanding entirely what you want, but if you intend to inspect the existing test to create a plan or a report, then you probably want to store the decorator arguments in the test function, instead of the class instance, right? For example:
def TestTags(*tags):
def decorator(f):
def wrapper(*args, **kwargs):
# ...
return f(*args, **kwargs)
wrapper.tags = tags
return wrapper
return decorator
#TestTags(TAG1, TAG2)
def myTest(...):
...
print(myTest.tags)
>>> (TAG1, TAG2)
EDIT: Just realized that doesn't work out of the box for instance methods. You can still use it like this though:
def TestTags(*tags):
def decorator(f):
def wrapper(*args, **kwargs):
# ...
return f(*args, **kwargs)
wrapper.tags = tags
return wrapper
return decorator
class TestSuite(...):
#TestTags(TAG1, TAG2)
def myTest(...):
...
print(TestSuite.myTest.tags) # From class function
>>> (TAG1, TAG2)
myTestSuite = TestSuite()
print(myTestSuite.myTest.__func__.tags) # From instance method
>>> (TAG1, TAG2)
What I am trying to do is write a wrapper around another module so that I can transform the parameters that are being passed to the methods of the other module. That was fairly confusing, so here is an example:
import somemodule
class Wrapper:
def __init__(self):
self.transforms = {}
self.transforms["t"] = "test"
# This next function is the one I want to exist
# Please understand the lines below will not compile and are not real code
def __intercept__(self, item, *args, **kwargs):
if "t" in args:
args[args.index("t")] = self.transforms["t"]
return somemodule.item(*args, **kwargs)
The goal is to allow users of the wrapper class to make simplified calls to the underlying module without having to rewrite all of the functions in the module. So in this case if somemodule had a function called print_uppercase then the user could do
w = Wrapper()
w.print_uppercase("t")
and get the output
TEST
I believe the answer lies in __getattr__ but I'm not totally sure how to use it for this application.
__getattr__ combined with defining a function on the fly should work:
# somemodule
def print_uppercase(x):
print(x.upper())
Now:
from functools import wraps
import somemodule
class Wrapper:
def __init__(self):
self.transforms = {}
self.transforms["t"] = "test"
def __getattr__(self, attr):
func = getattr(somemodule, attr)
#wraps(func)
def _wrapped(*args, **kwargs):
if "t" in args:
args = list(args)
args[args.index("t")] = self.transforms["t"]
return func(*args, **kwargs)
return _wrapped
w = Wrapper()
w.print_uppercase('Hello')
w.print_uppercase('t')
Output:
HELLO
TEST
I would approach this by calling the intercept method, and entering the desired method to execute, as a parameter for intercept. Then, in the intercept method, you can search for a method with that name and execute it.
Since your Wrapper object doesn't have any mutable state, it'd be easier to implement without a class. Example wrapper.py:
def func1(*args, **kwargs):
# do your transformations
return somemodule.func1(*args, **kwargs)
Then call it like:
import wrapper as w
print w.func1('somearg')
I am using nose test generators feature to run the same test with different contexts. Since it requires the following boiler plate for each test:
class TestSample(TestBase):
def test_sample(self):
for context in contexts:
yield self.check_sample, context
def check_sample(self, context):
"""The real test logic is implemented here"""
pass
I decided to write the following decorator:
def with_contexts(contexts=None):
if contexts is None:
contexts = ['twitter', 'linkedin', 'facebook']
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield f, self, context # The line which causes the error
return wrapper
return decorator
The decorator is used in the following manner:
class TestSample(TestBase):
#with_contexts()
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
When the tests executed an error is thrown specifying that the attribute which is being accessed is not available. However If I change the line which calls the method to the following it works fine:
yield getattr(self, f.__name__), service
I understand that the above snippet creates a bound method where as in the first one self is passed manually to the function. However as far as my understanding goes the first snippet should work fine too. I would appreciate if anyone could clarify the issue.
The title of the question is related to calling instance methods in decorators in general but I have kept the description specific to my context.
You can use functools.partial to tie the wrapped function to self, just like a method would be:
from functools import partial
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield partial(f, self), context
return wrapper
Now you are yielding partials instead, which, when called as yieldedvalue(context), will call f(self, context).
As far as I can tell, some things don't fit together. First, your decorator goes like
def with_contexts(contexts=None):
if contexts is None:
contexts = ['twitter', 'linkedin', 'facebook']
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield f, self, context # The line which causes the error
return wrapper
return decorator
but you use it like
#with_contexts
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
This is wrong: this calls with_context(test_sample), but you need with_context()(test_sample). So do
#with_contexts()
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
even if you don't provide the contexts argument.
Second, you decorate the wrong function: your usage shows that the test function yields the check function for each context. The function you want to wrap does the job of the check function, but you have to name it after the test function.
Applying self to a method can be done with partial as Martijn writes, but it can as well be done the way Python does it under the hood: with
method.__get__(self, None)
or maybe better
method.__get__(self, type(self))
you can achieve the same. (Maybe your original version works as well, with yielding the function to be called and the arguments to use. It was not clear to me that this is the way it works.)
I'm trying to write a library that will register an arbitrary list of service calls from multiple service endpoints to a container. I intend to implement the service calls in classes written one per service. Is there a way to maintain the boundedness of the methods from the service classes when registering them to the container (so they will still have access to the instance data of their owning object instance), or must I register the whole object then write some sort of pass through in the container class with __getattr__ or some such to access the methods within instance context?
container:
class ServiceCalls(object):
def __init__(self):
self._service_calls = {}
def register_call(self, name, call):
if name not in self._service_calls:
self._service_calls[name] = call
def __getattr__(self, name):
if name in self._service_calls:
return self._service_calls[name]
services:
class FooSvc(object):
def __init__(self, endpoint):
self.endpoint = endpoint
def fooize(self, *args, **kwargs):
#call fooize service call with args/kwargs utilizing self.endpoint
def fooify(self, *args, **kwargs):
#call fooify service call with args/kwargs utilizing self.endpoint
class BarSvc(object):
def __init__(self, endpoint):
self.endpoint = endpoint
def barize(self, *args, **kwargs):
#call barize service call with args/kwargs utilizing self.endpoint
def barify(self, *args, **kwargs):
#call barify service call with args/kwargs utilizing self.endpoint
implementation code:
foosvc = FooSvc('fooendpoint')
barsvc = BarSvc('barendpoint')
calls = ServiceCalls()
calls.register('fooize', foosvc.fooize)
calls.register('fooify', foosvc.fooify)
calls.register('barize', barsvc.barize)
calls.register('barify', barsvc.barify)
calls.fooize(args)
I think this answers your question:
In [2]: f = 1 .__add__
In [3]: f(3)
Out[3]: 4
You won't need the staticmethod function when adding these functions to classes, because they are effectively already "staticed".
What you are trying to do will work fine, as you can see by running your own code. :)
The object foosvc.fooize is called a "bound method" in Python, and it contains both, a reference to foosvc and to the function FooSvc.fooize. If you call the bound method, the reference to self will be passed implicitly as the first paramater.
On a side note, __getattr__() shouldn't silently return None for invalid attribute names. Better use this:
def __getattr__(self, name):
try:
return self._service_calls[name]
except KeyError:
raise AttributeError
I don't understand the use case for this -- it seems to me that the easy, simple, idiomatic way to accomplish this is to just pass in an object.
But: program to the interface, not the implementation. Only assume that the object has the method you need -- don't touch the internals or any other methods.