We Keep Coding

How to detect missing arguments in decorator?

I would like to define a decorator with a parameter that raises an error if the parameter is missing.
Here's a naive attempt on a simplified example:
def decorator_with_arg(a=None):
if a is None :
raise ValueError("Missing argument in decorator")
def decorator(func):
def wrapped_func(x):
return func(x+ a)
return wrapped_func
return decorator
But when I use this decorator without a parameter it's not raising any errors:
#decorator_with_arg
def simple_func(x):
return 2*x
simple_func(1)
How can I raise an exception?

You are not using your decorator correctly, in your code simple_func(1) will just return wrapped_func, because #decorator_with_arg will simply do:
simple_func = decorator_with_arg(simple_func)
# ^ this is passing a=simple_func
# now simple_func is the decorator function defined inside decorator_with_arg
You need to call your decorator_with_arg in order for it to return decorator which will then be used to decorate the function:
#decorator_with_arg(100)
def simple_func(x):
return 2*x
print(simple_func(1)) # 202
In any case, if you want to make an argument mandatory, simply declare it without a default value:
def decorator_with_arg(a):
# ...
And remove the if a is None check.
If you want to avoid mistakes in using #decorator_with_arg instead of #decorator_with_arg(), you can add a check to ensure a is not a function:
def decorator_with_arg(a):
if callable(a):
raise TypeError("Incorrect use of decorator")
def decorator(func):
def wrapped_func(x):
return func(x + a)
return wrapped_func
return decorator
#decorator_with_arg
def func():
return 1
# TypeError: Incorrect use of decorator
#decorator_with_arg(123)
def func():
return 1
# All fine

Comparing a wrapped function whether it is instance of a decorator

I have decorator1 and decorator2 functions. And I am using these to decorate a function.
#decorator1("some", "args")
#decorator2(1,2)
#decorator1()
def my_func(): print("my func")
When I call wrapped functions iteratively over and over again, output becomes like this:
for my_func.__wrapped__:
decorator1
decorator2
decorator1
my func
for my_func.__wrapped__.__wrapped__():
decorator2
decorator1
my func
Problem is every wrapped function's name is my_func. I want to check if a function in this chain is instance of lets say decorator1. I want to know this because I will use arguments of decorators. (I already know them by using __closure__ cells.)
Clarification
I decided to give and example to show my purpose.
#route("/index")
def index(): pass
#route("/settings")
#need_permission("admin")
def settings: pass
#route("/blog")
#need_permission("admin", "user")
def blog(): pass
I can get all these route functions somewhere else, and I want to extract which one needs which permissions.
Here my findings:
>>> blog()
route blog
permissions admin user
>>> blog.__closure__[0].cell_contents
('/blog',)
>>> blog.__closure__[1].cell_contents()
permissions admin user
>>> blog.__closure__[0].cell_contents.__closure__[0].cell_contents
('admin', 'user')
>>> blog.__closure__[0].cell_contents.__closure__[1].cell_contents()
>>>
I just want to extract the tuples that hold permissions. I can apply my decorators in some specific order and extract easily or I need to implement my DecoratorApplier function as #Poolka pointed out. If there is no way of knowing like first option, i will follow second.

instance of a decorator - decorators implemented with functions and regular functions are all just functions of type function.
Not sure what exactly you wanna get. The code below is how I would approach the problem. Basically I add attribute primal_type to all the functions involved into decoration carnival to store the name of the functions/decorators. I do this with another decorator named DecoratorApplier. The code seems to perform something related to the issue in the question.
EDIT
Added clarification didn't make everything clear. I guess it's not good practrice to mix function and decorator logics in that way. Maybe there is another option to get desired info inside the function? Anyway two modified versions of my original approach below (oda stands for optional decorator arguments).
(1) - with DecoratorApplier
import functools
def decorator_1(*d1_args, **d1_kwargs):
def decorator(func):
#functools.wraps(func)
def wrapped(*args, **kwargs):
print('inside decorator_1', d1_args, d1_kwargs)
return func(*args, **kwargs)
return wrapped
return decorator
def decorator_2(*d2_args, **d2_kwargs):
def decorator(func):
#functools.wraps(func)
def wrapped(*args, **kwargs):
print('inside decorator_2', d2_args, d2_kwargs)
return func(*args, **kwargs)
return wrapped
return decorator
class DecoratorApplier:
def __init__(self, *decorators):
self.decorators = decorators
def __call__(self, func):
func.oda = dict()
for decorator in self.decorators:
func = decorator[0](*decorator[1], **decorator[2])(func)
(
func
.oda
.setdefault(decorator[0].__name__, list())
.extend([decorator[1], decorator[2]])
)
return func
#DecoratorApplier(
(decorator_1, (1, 2), {'x': 10, 'y': 20}),
(decorator_2, tuple(), dict()))
def f_1():
print('inside f_1')
print(f_1.oda)
return
if __name__ == '__main__':
f_1()
(2) - with modifying original decorators
import functools
def decorator_1(*d1_args, **d1_kwargs):
def decorator(func):
#functools.wraps(func)
def wrapped(*args, **kwargs):
print('inside decorator_1', d1_args, d1_kwargs)
(
kwargs
.setdefault('oda', dict())
.setdefault('decorator_1', list())
.extend([d1_args, d1_kwargs])
)
return func(*args, **kwargs)
return wrapped
return decorator
def decorator_2(*d2_args, **d2_kwargs):
def decorator(func):
#functools.wraps(func)
def wrapped(*args, **kwargs):
print('inside decorator_2', d2_args, d2_kwargs)
(
kwargs
.setdefault('oda', dict())
.setdefault('decorator_2', list())
.extend([d2_args, d2_kwargs])
)
return func(*args, **kwargs)
return wrapped
return decorator
#decorator_1(1, 2, x=10, y=20)
#decorator_2()
def f_1(oda=None):
print('inside f_1')
print(' oda', oda)
return
if __name__ == '__main__':
f_1()

How do I modify parameters stored by functools.wraps?

I have a decorator that validates some parameters and passes an validated key to various functions:
from functools import wraps
ref validate(f):
#wraps(f) # This is to ensure docstrings are passed through the decorated function
def redirect_if_invalid(request, *args, **kwargs):
if request.valid == False:
return HttpResponseRedirect('/login')
else:
newkwargs = { 'key': request.key }
return f(request, *args, **newkwargs)
return redirect_if_invalid
This is used by some other functions:
#validate
def genericHandler(request, key)
pass
I'd call the function like this:
genericHandler(request)
And the decorator generates the 'key' kwarg. However, I'd like to optionally pass in the key at some other point, ie call:
genericHandler(request, 'keydata')
Currently this gives me an error:
TypeError: genericHandler() got multiple values for keyword argument 'key'
How can I get around this? To reiterate, the main goal is to be able to call genericHandler() with or without the optional parameter, and have the decorator generate the parameter only if it's missing.
So far inside the decorator, I can't figure out how to determine whether the 'key' parameter was passed in or not because functools.wraps() seems to hide it.

There's not any reasonable way to do this if you want your wrapper's signature to still be (request, *args, **kwargs). On the other hand, it looks like your decorator already assumes that the wrapped function takes a key parameter, so why not rewrite the wrapper to take one as well? In that case it becomes trivial to check if it's been passed or not.
def validate(f):
#wraps(f)
def redirect_if_invalid(request, key=None):
# do the validation
if key is None:
key = request.key
return f(request, key)
return redirect_if_invalid
You can add the *args and **kwargs parameters back if you like, of course.

So the best way for me to do this was to explicitly pass kwargs as kwargs. So decorated functions should actually be:
#validate
def genericHandler(request, **kwargs)
key = kwargs.get('key')
pass
This way, I can call the function either with or without args:
genericHandler(request)
or
genericHandler(request, **{ 'key' : key })
And the actual decorated looks like:
def validate(f):
#wraps(f) # This is to ensure docstrings are passed through the decorated function
def redirect_if_invalid(request, *args, **kwargs):
key = kwargs.get('key')
if not key:
kwargs.set('key', request.key)
return f(request, *args, **kwargs)
return redirect_if_invalid

Replacing macro-style class method with a decorator?

I'm having a lot of trouble getting a good grasp on decorators despite having read many an article on the subject (including [this][1] very popular one on SO). I'm suspecting I must be stupid, but with all the stubbornness that comes with being stupid, I've decided to try to figure this out.
That, and I suspect I have a good use case...
Below is some code from a project of mine that extracts text from PDF files. Processing involves three steps:
Set up PDFMiner objects needed for processing of PDF file (boilerplate initializations).
Apply a processing function to the PDF file.
No matter what happens, close the file.
I recently learned about context managers and the with statement, and this seemed like a good use case for them. As such, I started by defining the PDFMinerWrapper class:
class PDFMinerWrapper(object):
'''
Usage:
with PDFWrapper('/path/to/file.pdf') as doc:
doc.dosomething()
'''
def __init__(self, pdf_doc, pdf_pwd=''):
self.pdf_doc = pdf_doc
self.pdf_pwd = pdf_pwd
def __enter__(self):
self.pdf = open(self.pdf_doc, 'rb')
parser = PDFParser(self.pdf) # create a parser object associated with the file object
doc = PDFDocument() # create a PDFDocument object that stores the document structure
parser.set_document(doc) # connect the parser and document objects
doc.set_parser(parser)
doc.initialize(self.pdf_pwd) # pass '' if no password required
return doc
def __exit__(self, type, value, traceback):
self.pdf.close()
# if we have an error, catch it, log it, and return the info
if isinstance(value, Exception):
self.logError()
print traceback
return value
Now I can easily work with a PDF file and be sure that it will handle errors gracefully. In theory, all I need to do is something like this:
with PDFMinerWrapper('/path/to/pdf') as doc:
foo(doc)
This is great, except that I need to check that the PDF document is extractable before applying a function to the object returned by PDFMinerWrapper. My current solution involves an intermediate step.
I'm working with a class I call Pamplemousse which serves as an interface to work with the PDFs. It, in turn, uses PDFMinerWrapper each time an operation must be performed on the file to which the object has been linked.
Here is some (abridged) code that demonstrates its use:
class Pamplemousse(object):
def __init__(self, inputfile, passwd='', enc='utf-8'):
self.pdf_doc = inputfile
self.passwd = passwd
self.enc = enc
def with_pdf(self, fn, *args):
result = None
with PDFMinerWrapper(self.pdf_doc, self.passwd) as doc:
if doc.is_extractable: # This is the test I need to perform
# apply function and return result
result = fn(doc, *args)
return result
def _parse_toc(self, doc):
toc = []
try:
toc = [(level, title) for level, title, dest, a, se in doc.get_outlines()]
except PDFNoOutlines:
pass
return toc
def get_toc(self):
return self.with_pdf(self._parse_toc)
Any time I wish to perform an operation on the PDF file, I pass the relevant function to the with_pdf method along with its arguments. The with_pdf method, in turn, uses the with statement to exploit the context manager of PDFMinerWrapper (thus ensuring graceful handling of exceptions) and executes the check before actually applying the function it has been passed.
My question is as follows:
I would like to simplify this code such that I do not have to explicitly call Pamplemousse.with_pdf. My understanding is that decorators could be of help here, so:
How would I implement a decorator whose job would be to call the with statement and execute the extractability check?
Is it possible for a decorator to be a class method, or must my decorator be a free-form function or class?

The way I interpreted you goal, was to be able to define multiple methods on your Pamplemousse class, and not constantly have to wrap them in that call. Here is a really simplified version of what it might be:
def if_extractable(fn):
# this expects to be wrapping a Pamplemousse object
def wrapped(self, *args):
print "wrapper(): Calling %s with" % fn, args
result = None
with PDFMinerWrapper(self.pdf_doc) as doc:
if doc.is_extractable:
result = fn(self, doc, *args)
return result
return wrapped
class Pamplemousse(object):
def __init__(self, inputfile):
self.pdf_doc = inputfile
# get_toc will only get called if the wrapper check
# passes the extractable test
#if_extractable
def get_toc(self, doc, *args):
print "get_toc():", self, doc, args
The decorator if_extractable is defined is just a function, but it expects to be used on instance methods of your class.
The decorated get_toc, which used to delegate to a private method, simply will expect to receive a doc object and the args, if it passed the check. Otherwise it doesn't get called and the wrapper returns None.
With this, you can keep defining your operation functions to expect a doc
You could even add some type checking to make sure its wrapping the expected class:
def if_extractable(fn):
def wrapped(self, *args):
if not hasattr(self, 'pdf_doc'):
raise TypeError('if_extractable() is wrapping '\
'a non-Pamplemousse object')
...

A decorator is just a function that takes a function and returns another. You can do anything you like:
def my_func():
return 'banana'
def my_decorator(f): # see it takes a function as an argument
def wrapped():
res = None
with PDFMineWrapper(pdf_doc, passwd) as doc:
res = f()
return res
return wrapper # see, I return a function that also calls f
Now if you apply the decorator:
#my_decorator
def my_func():
return 'banana'
The wrapped function will replace my_func, so the extra code will be called.

You might want to try along the lines of this:
def with_pdf(self, fn, *args):
def wrappedfunc(*args):
result = None
with PDFMinerWrapper(self.pdf_doc, self.passwd) as doc:
if doc.is_extractable: # This is the test I need to perform
# apply function and return result
result = fn(doc, *args)
return result
return wrappedfunc
and when you need to wrap the function, just do this:
#pamplemousseinstance.with_pdf
def foo(doc, *args):
print 'I am doing stuff with', doc
print 'I also got some good args. Take a look!', args

Here is some demonstration code:
#! /usr/bin/python
class Doc(object):
"""Dummy PDFParser Object"""
is_extractable = True
text = ''
class PDFMinerWrapper(object):
'''
Usage:
with PDFWrapper('/path/to/file.pdf') as doc:
doc.dosomething()
'''
def __init__(self, pdf_doc, pdf_pwd=''):
self.pdf_doc = pdf_doc
self.pdf_pwd = pdf_pwd
def __enter__(self):
return self.pdf_doc
def __exit__(self, type, value, traceback):
pass
def safe_with_pdf(fn):
"""
This is the decorator, it gets passed the fn we want
to decorate.
However as it is also a class method it also get passed
the class. This appears as the first argument and the
function as the second argument.
"""
print "---- Decorator ----"
print "safe_with_pdf: First arg (fn):", fn
def wrapper(self, *args, **kargs):
"""
This will get passed the functions arguments and kargs,
which means that we can intercept them here.
"""
print "--- We are now in the wrapper ---"
print "wrapper: First arg (self):", self
print "wrapper: Other args (*args):", args
print "wrapper: Other kargs (**kargs):", kargs
# This function is accessible because this function is
# a closure, thus still has access to the decorators
# ivars.
print "wrapper: The function we run (fn):", fn
# This wrapper is now pretending to be the original function
# Perform all the checks and stuff
with PDFMinerWrapper(self.pdf, self.passwd) as doc:
if doc.is_extractable:
# Now call the orininal function with its
# argument and pass it the doc
result = fn(doc, *args, **kargs)
else:
result = None
print "--- End of the Wrapper ---"
return result
# Decorators are expected to return a function, this
# function is then run instead of the decorated function.
# So instead of returning the original function we return the
# wrapper. The wrapper will be run with the original functions
# argument.
# Now by using closures we can still access the original
# functions by looking up fn (the argument that was passed
# to this function) inside of the wrapper.
print "--- Decorator ---"
return wrapper
class SomeKlass(object):
#safe_with_pdf
def pdf_thing(doc, some_argument):
print ''
print "-- The Function --"
# This function is now passed the doc from the wrapper.
print 'The contents of the pdf:', doc.text
print 'some_argument', some_argument
print "-- End of the Function --"
print ''
doc = Doc()
doc.text = 'PDF contents'
klass = SomeKlass()
klass.pdf = doc
klass.passwd = ''
klass.pdf_thing('arg')
I recommend running that code to see how it works. Some of the interesting points to look out for tho:
First you will notice that we only pass a single argument to pdf_thing() but if you look at the method it takes two arguments:
#safe_with_pdf
def pdf_thing(doc, some_argument):
print ''
print "-- The Function --"
This is because if you look at the wrapper where we all the function:
with PDFMinerWrapper(self.pdf, self.passwd) as doc:
if doc.is_extractable:
# Now call the orininal function with its
# argument and pass it the doc
result = fn(doc, *args, **kargs)
We generate the doc argument and pass it in, along with the original arguments (*args, **kargs). This means that every method or function that is wrapped with this decorator receives an addition doc argument in addition to the arguments listed in its declaration (def pdf_thing(doc, some_argument):).
Another thing to note is that the wrapper:
def wrapper(self, *args, **kargs):
"""
This will get passed the functions arguments and kargs,
which means that we can intercept them here.
"""
Also captures the self argument and does not pass it to the method being called. You could change this behaviour my modifying the function call from:
result = fn(doc, *args, **kargs)
else:
result = None
To:
result = fn(self, doc, *args, **kargs)
else:
result = None
and then changing the method itself to:
def pdf_thing(self, doc, some_argument):
Hope that helps, feel free to ask for more clarification.
EDIT:
To answer the second part of your question.
Yes is can be a class method. Just place safe_with_pdf inside of SomeKlass above and calls to it e.g. The first method in the class.
Also here is a reduced version of the above code, with the decorator in the class.
class SomeKlass(object):
def safe_with_pdf(fn):
"""The decorator which will wrap the method"""
def wrapper(self, *args, **kargs):
"""The wrapper which will call the method is a doc"""
with PDFMinerWrapper(self.pdf, self.passwd) as doc:
if doc.is_extractable:
result = fn(doc, *args, **kargs)
else:
result = None
return result
return wrapper
#safe_with_pdf
def pdf_thing(doc, some_argument):
"""The method to decorate"""
print 'The contents of the pdf:', doc.text
print 'some_argument', some_argument
return '%s - Result' % doc.text
print klass.pdf_thing('arg')

How to build a decorator with optional parameters? [duplicate]

This question already has answers here:
How to create a decorator that can be used either with or without parameters?
(16 answers)
Making decorators with optional arguments [duplicate]
(14 answers)
Closed 9 years ago.
I would like to make a decorator which could be used with or without a parameter :
Something like this :
class d(object):
def __init__(self,msg='my default message'):
self.msg = msg
def __call__(self,fn):
def newfn():
print self.msg
return fn()
return newfn
#d('This is working')
def hello():
print 'hello world !'
#d
def too_bad():
print 'does not work'
In my code, only the use of decorator with parameter is working: How to proceed to have both working (with and without parameter)?

I found an example, you can use #trace or #trace('msg1','msg2'): nice!
def trace(*args):
def _trace(func):
def wrapper(*args, **kwargs):
print enter_string
func(*args, **kwargs)
print exit_string
return wrapper
if len(args) == 1 and callable(args[0]):
# No arguments, this is the decorator
# Set default values for the arguments
enter_string = 'entering'
exit_string = 'exiting'
return _trace(args[0])
else:
# This is just returning the decorator
enter_string, exit_string = args
return _trace

If you want to take parameters to your decorator, you need to always call it as a function:
#d()
def func():
pass
Otherwise, you need to try to detect the difference in parameters--in other words, you need to magically guess what the caller means. Don't create an API that needs to guess; consistently say what you mean to begin with.
In other words, a function should either be a decorator, or a decorator factory; it shouldn't be both.
Note that if all you want to do is store a value, you don't need to write a class.
def d(msg='my default message'):
def decorator(func):
def newfn():
print msg
return func()
return newfn
return decorator
#d('This is working')
def hello():
print 'hello world !'
#d()
def hello2():
print 'also hello world'

If you don't mind relying on using named arguments, I made something similar to what you need:
def cached_property(method=None, get_attribute=lambda a: '_%s_cached' % (a,)):
"""
Caches an object's attribute.
Can be used in the following forms:
#cached_property
#cached_property()
#cached_property(get_attribute=lambda x: 'bla')
#param method: the method to memoizes
#param get_attribute: a callable that should return the cached attribute
#return a cached method
"""
def decorator(method):
def wrap(self):
private_attribute = get_attribute(method.__name__)
try:
return getattr(self, private_attribute)
except AttributeError:
setattr(self, private_attribute, method(self))
return getattr(self, private_attribute)
return property(wrap)
if method:
# This was an actual decorator call, ex: #cached_property
return decorator(method)
else:
# This is a factory call, ex: #cached_property()
return decorator
This works because only one non keyword argument, the function decorated is passed to the decorator.
Notice that I also used the arguments passed to the decorated function, in this case 'self'.

This would work.
def d(arg):
if callable(arg): # Assumes optional argument isn't.
def newfn():
print('my default message')
return arg()
return newfn
else:
def d2(fn):
def newfn():
print(arg)
return fn()
return newfn
return d2
#d('This is working')
def hello():
print('hello world !')
#d # No explicit arguments will result in default message.
def hello2():
print('hello2 world !')
#d('Applying it twice')
#d('Would also work')
def hello3():
print('hello3 world !')
hello()
hello2()
hello3()
Output:
This is working
hello world !
my default message
hello2 world !
Applying it twice
Would also work
hello3 world !
If a decorator function #invocation isn't passed any explicit arguments, it is called with the function defined in the following def. If it is passed arguments, then it is first called with them and then the result of that preliminary call (which must itself also be a callable) is called with the function being defined. Either way, the return value of the last or only call is bound to the defined function name.

You have to detect if the argument to the decorator is a function, and use a simple decorator in that case. And then you need to hope that you never need to pass only a function to the parametrized decorator.