We Keep Coding

Is there a pythonic way to skip decoration on a subclass' method?

I have an class which decorates some methods using a decorator from another library. Specifically, the class subclasses flask-restful resources, decorates the http methods with httpauth.HTTPBasicAuth().login_required(), and does some sensible defaults on a model service.
On most subclasses I want the decorator applied; therefore I'd rather remove it than add it in the subclasses.
My thought is to have a private method which does the operations and a public method which is decorated. The effects of decoration can be avoided by overriding the public method to call the private one and not decorating this override. Mocked example below.
I am curious to know if there's a better way to do this. Is there a shortcut for 'cancelling decorators' in python that gives this effect?
Or can you recommend a better approach?
Some other questions have suitable answers for this, e.g. Is there a way to get the function a decorator has wrapped?. But my question is about broader design - i am interested in any pythonic way to run the operations in decorated methods without the effects of decoration. E.g. my example is one such way but there may be others.
def auth_required(fn):
def new_fn(*args, **kwargs):
print('Auth required for this resource...')
fn(*args, **kwargs)
return new_fn
class Resource:
name = None
#auth_required
def get(self):
self._get()
def _get(self):
print('Getting %s' %self.name)
class Eggs(Resource):
name = 'Eggs'
class Spam(Resource):
name = 'Spam'
def get(self):
self._get()
# super(Spam, self)._get()
eggs = Eggs()
spam = Spam()
eggs.get()
# Auth required for this resource...
# Getting Eggs
spam.get()
# Getting Spam

Flask-HTTPAuth uses functools.wraps in the login_required decorator:
def login_required(self, f):
#wraps(f)
def decorated(*args, **kwargs):
...
From Python 3.2, as this calls update_wrapper, you can access the original function via __wrapped__:
To allow access to the original function for introspection and other
purposes (e.g. bypassing a caching decorator such as lru_cache()),
this function automatically adds a __wrapped__ attribute to the
wrapper that refers to the function being wrapped.
If you're writing your own decorators, as in your example, you can also use #wraps to get the same functionality (as well as keeping the docstrings, etc.).
See also Is there a way to get the function a decorator has wrapped?

Another common option is to have the decorated function keep a copy of the original function that can be accessed:
def auth_required(fn):
def new_fn(*args, **kwargs):
print('Auth required for this resource...')
fn(*args, **kwargs)
new_fn.original_fn = fn
return new_fn
Now, for any function that has been decorated, you can access its original_fn attribute to get a handle to the original, un-decorated function.
In that case, you could define some type of dispatcher that either makes plain function calls (when you are happy with the decorator behavior) or makes calls to thing.original_fn when you prefer to avoid the decorator behavior.
Your proposed method is also a valid way to structure it, and whether my suggestion is "better" depends on the rest of the code you're dealing with, who needs to read it, and other kinds of trade-offs.

I am curious to know if there's a better way to do this. Is there a
shortcut for 'cancelling decorators' in python that gives this effect?
Use the undecorated library. It digs through all the decorators and returns just the original function. The docs should be self-explanatory, basically you just call: undecorated(your_decorated_function)

Allow help() to work on partial function object

I'm trying to make sure running help() at the Python 2.7 REPL displays the __doc__ for a function that was wrapped with functools.partial. Currently running help() on a functools.partial 'function' displays the __doc__ of the functools.partial class, not my wrapped function's __doc__. Is there a way to achieve this?
Consider the following callables:
def foo(a):
"""My function"""
pass
partial_foo = functools.partial(foo, 2)
Running help(foo) will result in showing foo.__doc__. However, running help(partial_foo) results in the __doc__ of a Partial object.
My first approach was to use functools.update_wrapper which correctly replaces the partial object's __doc__ with foo.__doc__. However, this doesn't fix the 'problem' because of how pydoc.
I've investigated the pydoc code, and the issue seems to be that partial_foo is actually a Partial object not a typical function/callable, see this question for more information on that detail.
By default, pydoc will display the __doc__ of the object type, not instance if the object it was passed is determined to be a class by inspect.isclass. See the render_doc function for more information about the code itself.
So, in my scenario above pydoc is displaying the help of the type, functools.partial NOT the __doc__ of my functools.partial instance.
Is there anyway to make alter my call to help() or functools.partial instance that's passed to help() so that it will display the __doc__ of the instance, not type?

I found a pretty hacky way to do this. I wrote the following function to override the __builtins__.help function:
def partialhelper(object=None):
if isinstance(object, functools.partial):
return pydoc.help(object.func)
else:
# Preserve the ability to go into interactive help if user calls
# help() with no arguments.
if object is None:
return pydoc.help()
else:
return pydoc.help(object)
Then just replace it in the REPL with:
__builtins__.help = partialhelper
This works and doesn't seem to have any major downsides, yet. However, there isn't a way with the above naive implementation to support still showing the __doc__ of some functools.partial objects. It's all or nothing, but could probably attach an attribute to the wrapped (original) function to indicate whether or not the original __doc__ should be shown. However, in my scenario I never want to do this.
Note the above does NOT work when using IPython and the embed functionality. This is because IPython directly sets the shell's namespace with references to the 'real' __builtin__, see the code and old mailing list for information on why this is.
So, after some investigation there's another way to hack this into IPython. We must override the site._Helper class, which is used by IPython to explicitly setup the help system. The following code will do just that when called BEFORE IPython.embed:
import site
site._Helper.__call__ = lambda self, *args, **kwargs: partialhelper(*args, **kwargs)
Are there any other downsides I'm missing here?

how bout implementing your own?
def partial_foo(*args):
""" some doc string """
return foo(*((2)+args))
not a perfect answer but if you really want this i suspect this is the only way to do it

You identified the issue - partial functions aren't typical functions, and the dunder variables don't carry over. This applies not just to __doc__, but also __name__, __module__, and more. Not sure if this solution existed when the question was asked, but you can achieve this more elegantly ("elegantly" up to interpretation) by re-writing partial() as a decorator factory. Since decorators (& factories) do not automatically copy over dunder variables, you need to also use #wraps(func):
def wrapped_partial(*args, **kwargs):
def foo(func):
#wraps(func)
def bar(*fargs,**fkwargs):
return func(*args, *fargs, **kwargs, **fkwargs)
return bar
return foo
Usage example:
#wrapped_partial(3)
def multiply_triple(x, y=1, z=0):
"""Multiplies three numbers"""
return x * y * z
# Without decorator syntax: multiply_triple = wrapped_partial(3)(multiply_triple)
With output:
>>>print(multiply_triple())
0
>>>print(multiply_triple(3,z=3))
9
>>>help(multiply_triple)
help(multiply_triple)
Help on function multiply_triple in module __main__:
multiply_triple(x: int, y: int = 1, z: int = 0)
Multiplies three numbers
Thing that didn't work, but informative when using multiple decorators
You might think, as I first did, that based upon the stacking syntax of decorators in PEP-318, you could put the wrapping and the partial function definition in separate decorators, e.g.
def partial_func(*args, **kwargs):
def foo(func):
def bar(*fargs,**fkwargs):
return func(*args, *fargs, **kwargs, **fkwargs)
return bar
return foo
def wrapped(f):
#wraps(f)
def wrapper(*args, **kwargs):
return f(*args, **kwargs)
return wrapper
#wrapped
#partial_func(z=3)
def multiply_triple(x, y=1, z=0):
"""Multiplies three numbers"""
return x * y * z
In these cases (and in reverse order), the decorators are applied one at a time, and the #partial_func interrupts wrapping. This means that if you are trying to use any decorator that you want to wrap, you need to rewrite the decorator in a factory where the decorator's return function is itself decorated by #wraps(func). If you are using multiple decorators, they all have to be turned into wrapped factories.
Alternate method to have decorators "wrap"
Since decorators are just functions, you can write a copy_dunder_vars(obj1, obj2) function that retruns obj2 but with all the dunder variables from obj1. Call as:
def foo()
pass
foo = copy_dunder_vars(decorator(foo), foo)
This goes against the preferred syntax, but practicality beats purity. I think "not forcing you to rewrite decorators that you're borrowing from elsewhere and leaving largely unchanged" fits into that category. After all that wrapping, don't forget ribbon and a bow ;)

Decorator that can interact with the wrapped function's arguments

I am trying to build a decorator that can inspect the wrapped function's arguments, the context for the need is to allow easier validation of certain arguments before they hit the function, avoiding the boiler plate of checking in every place I need said check/validation.
This is not a difficult decorator to write though, and this gets the job done (to make sure a value is not None for example):
def check_arg(func):
def wrapped(value):
assert value is not None
func(value)
return wrapped
The above example is not production code, it is merely an example of something that will work, but the problem becomes apparent when the function to decorate can have other decorators being used.
In that case, the above decorator example will not work, because value can now be another decorator. I could add another nested function here but it looks like there should be a better way of dealing with it programmatically, plus, it would break whenever the number of decorators change.
How could I get to the arguments of the wrapped function regardless of the number of decorators used by it?
EDIT: I should've noted than I am not simply stacking decorators, I am using one decorator that decorates re-usable decorators. In the example above I would re-use that in this context:
#check_arg
def some_other_decorator(func):
def wrapped(*args, **kw):
... # some interaction here
return wrapped
As I was writing this edit, I realized that the check_arg decorator was in the wrong place, it should've been in the wrapped function call within the some_other_decorator.

Are you in control of the other decorators? If so, make sure all of them play nice by properly
wrapping the calling function:
import functools
def check_arg(func):
#functools.wraps(func)
def wrapped(value):
assert value is not None
func(value)
return wrapped

How can I see if a method is a decorator?

Is it possible to inspect a function/method to see whether it can be used as a decorator? In that it follows the usual way decorators wrap other functions and return a callable? Specifically, I'm looking to validate 3rd party code.

By applying a suspected decorator, catching exceptions, and then testing whether the result contains a __call__ method, you could produce a guess as to whether a given callable is a decorator or not. But it will be only a guess, not a guarantee.
Beyond that, I do not believe what you want will be possible in general, due to the dynamically typed nature of the Python language and to the special treatment of built-in functions in the CPython interpreter. It is not possible to programmatically tell whether a callable will accept another callable as an argument, or what type its return value will have. Also, in CPython, for functions implemented in C, you cannot even inspect a callable to see how many arguments it accepts.
The word "decorator" can be taken to mean different things. One way to define it is, a decorator is any callable that accepts a single (callable) argument and returns a callable.
Note that I have not even used the word "function" in this definition; it would actually be incorrect to do so. Indeed, some commonly used decorators have strange properties:
The built-in classmethod and staticmethod decorators return descriptor objects, not functions.
Since language version 2.6 you can decorate classes, not just functions and methods.
Any class containing an __init__(self, somecallable) method and a __call__(self, *args, **kwargs) method can be used as a decorator.

Since there is no standardized decorator in Python, there's no real way of telling if a function is a decorator unless you know something about the decorator you're looking for.
If the decorator is under your control, you can add a mark to indicate it's a decorated function. Otherwise there is no real unified way of doing this. Take this example for instance:
def decorator(func):
return g
#decorator
def f()
pass
def g():
pass
In the above example, in run-time, f and g will be identical, and there is no way of telling the two apart.

Any callable with the right number of arguments can be used as a decorator. Remember that
#foo
def bar(...):
is exactly the same as
def bar(...):
...
bar = foo(bar)
Naturally, since foo could return anything, you have no way of checking whether a function has been decorated or not. Although foo could be nice and leave a mark, it has no obligation to do so.
If you are given some Python code and you want to find all the things that are decorators, you can do so by parsing the code into an abstract syntax tree then walking the tree looking for decorated functions. Here's an example, storing the .ids of the decorators. Obviously, you could store the astobjects if you wanted to.
>>> class DecoratorFinder(ast.NodeVisitor):
... def __init__(self, *args, **kwargs):
... super(DecoratorFinder, self).__init__(*args, **kwargs)
... self.decorators = set()
...
... def visit_FunctionDef(self, node):
... self.decorators.update(dec.id for dec in node.decorator_list)
... self.generic_visit(node)
...
>>> finder = DecoratorFinder()
>>> x = ast.parse("""
... #dec
... def foo():
... pass
... """)
>>> finder.visit(x)
>>> finder.decorators
set(['dec'])

No this is not possible. May be instead of checking if f is a decorator, you should think why you need to check that?
If you are expecting some specific decorator, you can directly check that, if you want some specific behavior/methods/attributes you can check that
If you want to check if some callable f can be used as decorator, you can test the decorator behavior by passing some dummy function, but in general it may not work or have different behavior for different inputs.
Here is a such naive check:
def decorator1(func):
def _wrapper(*args, **kwargs):
print "before"
func(*args, **kwargs)
print "after"
return _wrapper
def dummy_func(): pass
out_func = decorator1(dummy_func)
if callable(out_func) and dummy_func != out_func:
print "aha decorated!"

I've never done anything like this, but in general python relies on "duck-typing" in situations like this. So you could just try to decorate a dummy function and see if a callable is returned.

Is it bad practice to use self in decorators?

While I'm aware that you can't reference self directly in a decorator, I was wondering if it's bad practice to work around that by pulling it from args[0]. My hunch is that it is, but I want to be sure.
To be more specific, I'm working on an API to a web service. About half the commands require a token to be passed that can be later used to undo it. What I would like is to make that token an optional parameter and if none is supplied, to generate one. Generating a token requires making an authenticated call to the server, which needs data from the object.
While I know I could do it:
def some_command(self, ..., undo_token = None):
if undo_token = None:
undo_token = self.get_undo_token()
...
return fnord
I feel like there could be a better way than to have the same code in a dozen or so methods. My thought was to write a decorator:
#decorator
def undoable(fn, *args, **kwargs):
if 'undo_token' not in kwargs:
kwargs['undo_token'] = args[0].get_undo_token()
return (fn(*args, **kwargs), kwargs['undo_token'])
So I can more cleanly write
#undoable
def some_command(self, ...):
...
return foo
#undoable
def some_other_command(self, ...):
...
return bar
Am I setting myself up for trouble down the line?

I don't understand what you're coding for undoable -- that's not how decorators are normally coded and I don't know where that #decorator is coming from (is there a from youforgottotelluswhence import decorator or something even more evil? see why I can't stand the use of from to build "artificial barenames" instead of using nice decorated names?-).
With normal decorator coding, e.g....:
import functools
def undoable(f):
#functools.wraps(f)
def wrapper(self, *a, **k):
tok = k.get('undo_token')
if tok is None:
tok = k['undo_token'] = self.get_undo_token()
return f(self, *a, **k), tok
return wrapper
there's absolutely no problem naming the wrapper's first, mandatory positional argument self, and much gain of clarity in using this rather than the less-readable args[0].

Decorators extend the functionality of the function it decorates in a generic way. If decorators to do not make any assumption about the function or it's args or kwargs, it is in most generic form and can be easily used with many functions.
How ever, if you want to do something with what is being passed onto the function, it should be fine but their applicability is limited and can break, if the underlying details, which you have used in your decorator changes.
In the above decorator, if the object removes the method get_undo_token(), you will need to revisit the decorator too. It is fine to do that but document the constraints and also add that documentation to the method doc it self.
Do it only if absolutely necessary. It serves to create more generic decorators.