Suppose I have this decorator:
def decorator(f):
def f_wrap(*args):
for item in args:
print(args)
return f(*args)
return f_wrap
When used as "permanent" decorators with the # syntax, args retrieves the arguments of the wrapped function. For example, when used with the class below, I receive the instance of MyObject.
Class MyObject(object):
def __init__(self):
pass
#decorator
def function(self):
return
How can I achieve the same result using a "fluid" decorator. Or a decorator that is not permanently bound to the function it is decorating? For example:
def decorator(f):
def f_wrap(*args):
if (not args):
print("Nothing in args")
return f(*args)
return f_wrap
class MyClass(object):
def __init__(self):
pass
def function(self):
return
if __name__ == "__main__":
myobj = MyClass()
myobj.function = decorator(myobj.function)
myobj.function()
In this case, the args tuple always returns empty (I always get "Nothing in args"), even though I anticipated that it would return the instance variable myobj.
EDIT:
In case it was not clear from #AChampion's post the solution is to simply call the fluid-decoratored method as an "unbound" method. E.g.,
from types import MethodType
def decorator(f):
def f_wrap(*args):
# I replaced this with an iteration through
# args. It's a bit more demonstrative.
for item in args:
print(item)
return f(*args)
return f_wrap
class MyClass(object):
def __init__(self):
pass
def function(self):
return
if __name__ == "__main__":
myobj = MyClass()
myobj.function = MethodType(decorator(MyClass.function), myobj)
myobj.function()
The reason for the difference is that you are wrapping different things, a unbound method vs a bound method:
class MyObject(object):
#decorator
def function(self):
pass
Is equivalent to:
import types
class MyClass(object):
def function(self):
pass
m = MyClass(object)
m.function = types.MethodType(decorator(MyClass.function), m)
Not:
m.function = decorator(m.function)
The first being an unbound method, the second being a bound method.
You aren't using all properly. all returns a bool on whether all conditions are met inside what you are checking for in all. In your case, you aren't really doing anything. You will always evaluate to True with how you are using all.
I believe what you are looking for is simply this:
if not args:
Now, ultimately what this checks is if the method you are executing has *args. For the case of the function you have, you aren't passing any arguments, therefore, with the if not args check, you will actually get:
"Nothing in args"
However, if you add an argument to your method as such:
def function(self, x):
return
Then call: myobj.function(1)
You will not get "Nothing in args".
To answer your last question about not getting your instance. If you print out f using this method of calling your decorator:
myobj.function = decorator(myobj.function)
myobj.function()
You will get a bound method:
<bound method MyClass.function of <__main__.MyClass object at 0x102002390>>
Now, set up your decorator as such:
#decorator
def function(self):
return
You will see you get a function attached to your class object:
<function MyClass.function at 0x102001620>
Hence showing that they aren't doing the exact same thing you would expect. Hope this helps clarify a bit.
Related
I understand what I am asking here is probably not the best code design, but the reason for me asking is strictly academic. I am trying to understand how to make this concept work.
Typically, I will return self from a class method so that the following methods can be chained together. My understanding is by returning self, I am simply returning an instance of the class, for the following methods to work on.
But in this case, I am trying to figure out how to return both self and another value from the method. The idea is if I do not want to chain, or I do not call any class attributes, I want to retrieve the data from the method being called.
Consider this example:
class Test(object):
def __init__(self):
self.hold = None
def methoda(self):
self.hold = 'lol'
return self, 'lol'
def newmethod(self):
self.hold = self.hold * 2
return self, 2
t = Test()
t.methoda().newmethod()
print(t.hold)
In this case, I will get an AttributeError: 'tuple' object has no attribute 'newmethod' which is to be expected because the methoda method is returning a tuple which does not have any methods or attributes called newmethod.
My question is not about unpacking multiple returns, but more about how can I continue to chain methods when the preceding methods are returning multiple values. I also understand that I can control the methods return with an argument to it, but that is not what I am trying to do.
As mentioned previously, I do realize this is probably a bad question, and I am happy to delete the post if the question doesnt make any sense.
Following the suggestion by #JohnColeman, you can return a special tuple with attribute lookup delegated to your object if it is not a normal tuple attribute. That way it acts like a normal tuple except when you are chaining methods.
You can implement this as follows:
class ChainResult(tuple):
def __new__(cls, *args):
return super(ChainResult, cls).__new__(cls, args)
def __getattribute__(self, name):
try:
return getattr(super(), name)
except AttributeError:
return getattr(super().__getitem__(0), name)
class Test(object):
def __init__(self):
self.hold = None
def methoda(self):
self.hold = 'lol'
return ChainResult(self, 'lol')
def newmethod(self):
self.hold = self.hold * 2
return ChainResult(self, 2)
Testing:
>>> t = Test()
>>> t.methoda().newmethod()
>>> print(t.hold)
lollol
The returned result does indeed act as a tuple:
>>> t, res = t.methoda().newmethod()
>>> print(res)
2
>>> print(isinstance(t.methoda().newmethod(), tuple))
True
You could imagine all sorts of semantics with this, such as forwarding the returned values to the next method in the chain using closure:
class ChainResult(tuple):
def __new__(cls, *args):
return super(ChainResult, cls).__new__(cls, args)
def __getattribute__(self, name):
try:
return getattr(super(), name)
except AttributeError:
attr = getattr(super().__getitem__(0), name)
if callable(attr):
chain_results = super().__getitem__(slice(1, None))
return lambda *args, **kw: attr(*(chain_results+args), **kw)
else:
return attr
For example,
class Test:
...
def methodb(self, *args):
print(*args)
would produce
>>> t = Test()
>>> t.methoda().methodb('catz')
lol catz
It would be nice if you could make ChainResults invisible. You can almost do it by initializing the tuple base class with the normal results and saving your object in a separate attribute used only for chaining. Then use a class decorator that wraps every method with ChainResults(self, self.method(*args, **kw)). It will work okay for methods that return a tuple but a single value return will act like a length 1 tuple, so you will need something like obj.method()[0] or result, = obj.method() to work with it. I played a bit with delegating to tuple for a multiple return or to the value itself for a single return; maybe it could be made to work but it introduces so many ambiguities that I doubt it could work well.
I'm trying to create a method inside my class that counts the complete run of a specific function. I want to use a simple decorator. I found this reference and rewrite this simple script:
class myclass:
def __init__(self):
self.cnt = 0
def counter(function):
"""
this method counts the runtime of a function
"""
def wrapper(self, **args):
function(**args)
self.counter += 1
return wrapper
#myclass.counter
def somefunc():
print("hello from somefunc")
if __name__ == "__main__":
obj = myclass()
# or if comment #myclass.counter
# somefunc = myclass.counter(somefunc)
somefunc()
And of course, I get:
TypeError: wrapper() missing 1 required positional argument: 'self'
I tried to rewrite the counter as a class method:
class myclass:
def __init__(self):
self.cnt = 0
def counter(self, function):
"""
this method counts the runtime of a function
"""
def wrapper(**args):
function(**args)
self.cnt += 1
return wrapper
def somefunc():
print("hello from somefunc")
if __name__ == "__main__":
obj = myclass()
somefunc = obj.counter(somefunc)
for i in range(10):
somefunc()
print(obj.cnt)
Which works fine but I think it is not a valid decorator definition. Is there any way to define the decorator inside the class method and pass the self-argument to its function? or defining a decorator inside a class is useless?
EDIT:------
First I can't define the decoration outside of the class method. Second I'm trying to make a scheduled class that runs a specific function (as input) for a fixed interval and a specific amount of time so I need to count it.
So I was able to draft up something for you, below is the code:
def count(func):
def wrapper(self):
TestClass.call_count += 1
func(self)
return wrapper
class TestClass(object):
call_count = 0
#count
def hello(self):
return 'hello'
if __name__ == '__main__':
x = TestClass()
for i in range(10):
x.hello()
print(TestClass.call_count)
Why would it cause problems to have the decorator in a class:
It's not straight forward to have a decorator function inside the class. The reasons are below:
Reason 1
Every class method must take an argument self which is the instance of the class through which the function is being called. Now if you make the decorator function take a self argument, the decorator call #count would fail as it get converted to count() which doesn't pass the self argument and hence the error:
TypeError: wrapper() missing 1 required positional argument: 'self'
Reason 2
Now to avoid that you can make your decorator as static by changing the declaration like below:
#staticmethod
def count(func):
pass
But then you have another error:
TypeError: 'staticmethod' object is not callable
Which means you can't have a static method as well. If you can't have a static method in a class, you have to pass the self instance to the method but if you pass the self instance to it, the #count decorator call wouldn't pass the self instance and hence it won't work.
So here is a blog that explains it quite well, the issues associated with it and what are the alternatives.
I personally prefer the option to have a helper class to hold all my decorators that can be used instead of the only class in which it's defined. This would give you the flexibility to reuse the decorators instead of redefining them which would follow the ideology
code once, reuse over and over again.
Your second code example is functionally equivalent to a standard decorator. The standard decorator syntax is just a shorthand for the same thing, namely, reassigning a function value equal to a closure (a function pointer with arguments predefined), where the closure is your decorator wrapper holding the original as its predefined argument.
Here's an equivalent with standard syntax. Notice you need to create the counter class instance in advance. The decorator syntax refers to that instance, because it must indicate the specific object which holds your counter, rather than just the class of the object:
class myclass:
def __init__(self):
self.cnt = 0
def counter(self,function):
"""
this method counts the number of runtime of a function
"""
def wrapper(**args):
function(self,**args)
self.cnt += 1
return wrapper
global counter_object
counter_object = myclass()
#counter_object.counter
def somefunc(self):
print("hello from somefunc")
if __name__ == "__main__":
for i in range(10):
somefunc()
print(counter_object.cnt)
I have a class which maintains a list of functions. These functions are just objects sitting in a queue and every so often the class pops one off and executes it. However, there are times when I would like to print out this list, and I'm imagining code as follows:
for function in self.control_queue:
print function.summarize()
if function.ready():
function()
In other words, I would like to call methods called summarize() and ready(), that I want to define somewhere, on these function objects. Also, I would like to be able to toss anonymous functions on this queue - i.e., generate everything dynamically.
you can make it a class and define __call__
class MyClass():
def summarize(self):
#summarize stuff
pass
def ready(self):
#ready stuff
pass
def _call__(self):
#put the code here, for when you call myClass()
pass
How you run it:
function = MyClass()
print function.summarize()
if function.ready():
function()
You have a couple possible approaches.
You could add the definitions to functions.
def foo():
pass
# later..
foo.summarize = lambda: "To pair with bar"
foo.ready = lambda: True
You could create class objects to wrap the function operation.
class Func():
def summarize(self):
return "Function!"
def ready(self):
return self.ready
def __call__(self):
# Act as a function
Or you can have a function which checks the function label for these capabilities.
def summarize_func(func):
return func.__name__ # Or branch here on specific names/attributes
def ready_func(func):
return True # Or branch on names/attributes
Finally to accommodate anonymous functions you can check for prescience of these attributes and return optimistically if the attributes are absent. Then you can combine above approaches with something that will work on any function.
def summarize_func(func):
if hasattr(func, summarize):
return func.summarize()
else:
# Note this will just be '<lambda>' for anonymous funcs
return func.__name__
def ready_func(func):
if hasattr(func, ready):
return func.ready()
else:
return True
One option is to implement function as a class instance:
class Function(object):
def summarize(self): pass # some relevant code here
def __call__(self): pass # and there
and use it later with
function = Function()
With __call__ magic method implemented, this function becomes a callable object.
For sure, you can assign attributes to functions, but it is rather obscure and conterintuitive:
>>> def summ(a): return sum(a)
...
>>> def function(a): return a
...
>>> function.sum=summ
>>> function.sum([1,2,3])
6
I have been testing out this caching method/code:
http://code.activestate.com/recipes/498245-lru-and-lfu-cache-decorators/?c=15348
and in some cases, I get this (or similar) error:
"AttributeError: 'str' object has no attribute 'module'"
and here are code examples, these work fine:
if __name__ == '__main__':
#lru_cacheItem(maxsize=20)
def f(x, y):
return 3*x+y
domain = range(5)
from random import choice
for i in range(1000):
r = f(choice(domain), choice(domain))
print('Hits:{0}'.format(f.hits), 'Misses:{0}'.format(f.misses))
#lfu_cacheItem(maxsize=20)
def f(x, y):
return 3*x+y
domain = range(5)
from random import choice
for i in range(1000):
r = f(choice(domain), choice(domain))
print('Hits:{0}'.format(f.hits), 'Misses:{0}'.format(f.misses))
#lru_cacheItem(maxsize=20)
def myString(a, b):
return '{0} and {1}'.format(a, b)
a = 'crap'
b = 'shit'
for i in range(1000):
r = myString(a, b)
print('Hits:{0}'.format(myString.hits), 'Misses:{0}'.format(myString.misses))
and this does not:
if __name__ == '__main__':
class P4client(object):
def __init__(self):
pass
def checkFileStats(self, filePath):
results = 'The filepath: {0}'.format(filePath)
print results
return results
p4client = P4client()
filePath = (r"C:\depot\tester.tga")
#lfu_cacheItem
def p4checkFileStats(filePath):
'''Will cache the fstats return'''
p4checkResults = p4client.checkFileStats(filePath)
return p4checkResults
p4checkFileStats(filePath)
I am not sure how to fix this ... it appears to be an issue in functools, I assume somehow do to that fact I am calling a class/method within the function I am wrapping?
#lfu_cacheItem
def p4checkFileStats(filePath):
You are missing parenthesis here:
#lfu_cacheItem()
def p4checkFileStats(filePath):
All decorators which expect "options", i.e. that you can use as:
#decorator(a=Something, b=Other, ...)
def the_function(...):
Must always be called when decorating, even if you do not provide arguments:
#decorator()
def the_function(...):
Why you wonder? Well, first of all remember that decorators are normal functions that accept a function as argument:
In [1]: def hooray(func):
...: print("I'm decorating function: {.__name__}".format(func))
...: return func
In [2]: #hooray
...: def my_function(): pass
I'm decorating function: my_function
As you can see hooray was called. In fact this is what really happens when using a decorator:
In [3]: def my_function(): pass
...: my_function = hooray(my_function)
...:
I'm decorating function: my_function
Now, if you want to pass options to the decorator you can create a function that returns a decorator. This is exactly what happens with lfu_cache from the recipe you link:
def lfu_cache(maxsize=100):
# ...
def decorating_function(user_function):
# ...
return decorating_function
Now here you can see that lfu_cache is really a function. this function creates a decorator, called decorating_function and returns it. This means that when calling:
#lfu_cache(maxsize=20)
def my_function(): pass
This is what happens:
def my_function(): pass
decorator = lfu_cache(maxsize=20)
my_function = decorator(my_function)
As you can see first lfu_cache is called, and returns a decorator. Afterwards the decorator is called to decorate the function.
What happens if you forget the parenthesis? What does this:
#lfu_cache
def my_function(): pass
do?
Pretty simple, it uses lfu_cache as a simple decorator:
def my_function(): pass
my_function = lfu_cache(my_function)
But this is bad! You passed a function as maxsize parameter and the value returned by lfu_cache is the decorating_function of before!
to learn more about decorators read this So answer.
I am programming in Python, and I am wondering if i can test if a function has been called in my code
def example():
pass
example()
#Pseudocode:
if example.has_been_called:
print("foo bar")
How would I do this?
If it's OK for the function to know its own name, you can use a function attribute:
def example():
example.has_been_called = True
pass
example.has_been_called = False
example()
#Actual Code!:
if example.has_been_called:
print("foo bar")
You could also use a decorator to set the attribute:
import functools
def trackcalls(func):
#functools.wraps(func)
def wrapper(*args, **kwargs):
wrapper.has_been_called = True
return func(*args, **kwargs)
wrapper.has_been_called = False
return wrapper
#trackcalls
def example():
pass
example()
#Actual Code!:
if example.has_been_called:
print("foo bar")
A minimal example using unittest.mock.Mock from the standard library:
from unittest.mock import Mock
def example():
pass
example_mock = Mock(side_effect=example)
example_mock()
#Pseudocode:
if example_mock.called:
print("foo bar")
Console output after running the script:
foo bar
This approach is nice because it doesn't require you to modify the example function itself, which is useful if you want to perform this check in some unit-testing code, without modifying the source code itself (EG to store a has_been_called attribute, or wrap the function in a decorator).
Explanation
As described in the documentation for the unittest.mock.Mock class, the side_effect argument to the Mock() constructor specifies "a function to be called whenever the Mock is called".
The Mock.called attribute specifies "a boolean representing whether or not the mock object has been called".
The Mock class has other attributes you may find useful, EG:
call_count: An integer telling you how many times the mock object has been called
call_args: This is either None (if the mock hasn’t been called), or the arguments that the mock was last called with
call_args_list: This is a list of all the calls made to the mock object in sequence (so the length of the list is the number of times it has been called). Before any calls have been made it is an empty list
The Mock class also has convenient methods for making assert statements based on how many times a Mock object was called, and what arguments it was called with, EG:
assert_called_once_with(*args, **kwargs): Assert that the mock was called exactly once and that that call was with the specified arguments
We can use mock.Mock
from unittest import mock
def check_called(func):
return mock.Mock(side_effect=func)
#check_called
def summator(a, b):
print(a + b)
summator(1, 3)
summator.assert_called()
assert summator.called == True
assert summator.call_count > 0
summator.assert_called_with(1, 3)
summator.assert_called_with(1, 5) # error
# AssertionError: Expected call: mock(1, 5)
# Actual call: mock(1, 3)
Memoization functions have been around since the 1960s. In python you can use them as decorators on your example() function.
The standard memoization function looks something like this:
def memoize(func):
memo = {}
def wrapper(*args):
if not args in memo:
memo[args] = func(*args)
return memo[args]
return wrapper
and you decorate your function like this:
#memoize
def example():
pass
In python3.2, you can use the functools.lru_cache instead of the memoziation function.
import functools
#functools.lru_cache(maxsize=None)
def example():
pass
Here's a decorator that will watch all your functiona, using colorama, and return a nice output.
try:
import colorama
except ImportError:
class StdClass: pass
def passer(*args, **kwargs): pass
colorama = StdClass()
colorama.init = passer
colorama.Fore = StdClass()
colorama.Fore.RED = colorama.Fore.GREEN = ''
def check_for_use(show=False):
if show:
try:
check_for_use.functions
except AttributeError:
return
no_error = True
for function in check_for_use.functions.keys():
if check_for_use.functions[function][0] is False:
print(colorama.Fore.RED + 'The function {!r} hasn\'t been called. Defined in "{}" '.format(function, check_for_use.functions[function][1].__code__.co_filename))
no_error = False
if no_error:
print(colorama.Fore.GREEN + 'Great! All your checked function are being called!')
return check_for_use.functions
try:
check_for_use.functions
except AttributeError:
check_for_use.functions = {}
if colorama:
colorama.init(autoreset=True)
def add(function):
check_for_use.functions[function.__name__] = [False, function]
def func(*args, **kwargs):
check_for_use.functions[function.__name__] = [True, function]
function(*args, **kwargs)
return func
return add
#check_for_use()
def hello():
print('Hello world!')
#check_for_use()
def bonjour(nb):
print('Bonjour tout le monde!')
# hello(); bonjour(0)
hello()
check_for_use(True) # outputs the following
Output:
Hello world!
The function 'bonjour' hasn't been called. Defined in "path_to_file.py"
You can also create a variable and increment it in the function. Later you can check if it's 1 or >= 0.