Let's say we have multiple functions which all accept an URL as their first argument and this URL needs to be validated. This can be nicely solved with a decorator
def validate_url(f):
def validated(url, *args, **kwargs):
assert len(url.split('.')) == 3 # trivial example
return f(url, *args, **kwargs)
return validated
#validate_url
def some_func(url, some_other_arg, *some_args, **some_kwargs):
pass
This approach will work and allow me to factor the validation behavior out of many instances of similar functions. But now I would want to write a class method which also takes a validated URL. However, the naive approach will not work
class SomeClass:
#validate_url
def some_method(self, url, some_other_args):
pass
because we will end up attempting to validate self and not url. My question is how to write a single decorator which will work for both functions and methods with the minimum amount of boilerplate.
Note 1: I am aware why this happens, it's just that I don't know how to work around this in the most elegant manner.
Note 2: The URL validation problem is just an example, so checking if isinstance(args[0], str) is not a good solution.
One solution would be to somehow detect whether the decorated function is a class method or not — which seems to be difficult if not impossible (as far as I can tell anyway) to do so cleanly. The inspect module's ismethod() and isfunction() don't work inside a decorator used inside a class definition.
Given that, here's a somewhat hacky way of doing it which checks to see if the decorated callable's first argument has been given the name "self", which is the coding convention for it in class methods (although it is not a requirement, so caveat emptor and use at your own risk).
The following code seems to work in both Python 2 and 3. However in Python 3 it may raise DeprecationWarnings depending on exactly what sub-version is being used—so they have been suppressed in a section of the code below.
from functools import wraps
import inspect
import warnings
def validate_url(f):
#wraps(f)
def validated(*args, **kwargs):
with warnings.catch_warnings():
# Suppress DeprecationWarnings in this section.
warnings.simplefilter('ignore', category=DeprecationWarning)
# If "f"'s first argument is named "self",
# assume it's a method.
if inspect.getargspec(f).args[0] == 'self':
url = args[1]
else: # Otherwise assume "f" is a ordinary function.
url = args[0]
print('testing url: {!r}'.format(url))
assert len(url.split('.')) == 3 # Trivial "validation".
return f(*args, **kwargs)
return validated
#validate_url
def some_func(url, some_other_arg, *some_args, **some_kwargs):
print('some_func() called')
class SomeClass:
#validate_url
def some_method(self, url, some_other_args):
print('some_method() called')
if __name__ == '__main__':
print('** Testing decorated function **')
some_func('xxx.yyy.zzz', 'another arg')
print(' URL OK')
try:
some_func('https://bogus_url.com', 'another thing')
except AssertionError:
print(' INVALID URL!')
print('\n** Testing decorated method **')
instance = SomeClass()
instance.some_method('aaa.bbb.ccc', 'something else') # -> AssertionError
print(' URL OK')
try:
instance.some_method('foo.bar', 'arg 2') # -> AssertionError
except AssertionError:
print(' INVALID URL!')
Output:
** Testing decorated function **
testing url: 'xxx.yyy.zzz'
some_func() called
URL OK
testing url: 'https://bogus_url.com'
INVALID URL!
** Testing decorated method **
testing url: 'aaa.bbb.ccc'
some_method() called
URL OK
testing url: 'foo.bar'
INVALID URL!
Related
I am string with decorators and the first use I have is to wrap a HTTP call to account for failed connections. The working code is as follows:
import requests
class Gotify:
def __init__(self):
self.url = "https://postman-echo.com/status/200"
def ensure_safe_call(caller):
def wrapper(*args, **kwargs):
try:
r = caller(*args, **kwargs)
r.raise_for_status()
except Exception as e:
try:
print(f"cannot reach gotify: {e}: {r.text}")
except NameError:
print(f"cannot reach gotify: {e} (the response r does not exist)")
else:
print("OK notified gotify of result change")
return wrapper
#ensure_safe_call
def send(self, title, message):
return requests.get(self.url)
Gotify().send("hello", "world")
This correct displays OK notified gotify of result change.
When editing this in PyCharm, I get two warning which I do not understand:
and
What do they mean in the context of my decorators (there are none when I do not use decorators)
class Gotify:
def __init__(self):
self.url = "https://postman-echo.com/status/200"
def ensure_safe_call(caller):
Because ensure_safe_call is a class method, the first argument (in your case caller) is actually the self argument, the instance of the clas object--Gotify.
Hence the warning message about the Gotify object not being callable (it's not callable because you have not overridden the __call__ class method in your Gotify class)
Function ensure_safe_call lacks a positional argument -- this is because ensure_safe_call only takes in the self argument, and doesn't specify any actual input arguments (recall that caller == self given the way you have it defined). Thus, your decorator ensure_safe_call cannot wrap anything, because it's accepting no position arguments.
You need to define a positional argument
def ensure_safe_call(self, caller):
...
I have a class that handles the API calls to a server. Certain methods within the class require the user to be logged in. Since it is possible for the session to run out, I need some functionality that re-logins the user once the session timed out. My idea was to use a decorator. If I try it like this
class Outer_Class():
class login_required():
def __init__(self, decorated_func):
self.decorated_func = decorated_func
def __call__(self, *args, **kwargs):
try:
response = self.decorated_func(*args, **kwargs)
except:
print('Session probably timed out. Logging in again ...')
args[0]._login()
response = self.decorated_func(*args, **kwargs)
return response
def __init__(self):
self.logged_in = False
self.url = 'something'
self._login()
def _login(self):
print(f'Logging in on {self.url}!')
self.logged_in = True
#this method requires the user to be logged in
#login_required
def do_something(self, param_1):
print('Doing something important with param_1')
if (): #..this fails
raise Exception()
I get an error. AttributeError: 'str' object has no attribute '_login'
Why do I not get a reference to the Outer_Class-instance handed over via *args? Is there another way to get a reference to the instance?
Found this answer How to get instance given a method of the instance? , but the decorated_function doesn't seem to have a reference to it's own instance.
It works fine, when Im using a decorator function outside of the class. This solves the problem, but I like to know, if it is possible to solve the this way.
The problem is that the magic of passing the object as the first hidden parameter only works for a non static method. As your decorator returns a custom callable object which is not a function, it never receives the calling object which is just lost in the call. So when you try to call the decorated function, you only pass it param_1 in the position of self. You get a first exception do_something() missing 1 required positional argument: 'param_1', fall into the except block and get your error.
You can still tie the decorator to the class, but it must be a function to have self magic work:
class Outer_Class():
def login_required(decorated_func):
def inner(self, *args, **kwargs):
print("decorated called")
try:
response = decorated_func(self, *args, **kwargs)
except:
print('Session probably timed out. Logging in again ...')
self._login()
response = decorated_func(self, *args, **kwargs)
return response
return inner
...
#this method requires the user to be logged in
#login_required
def do_something(self, param_1):
print('Doing something important with param_1', param_1)
if (False): #..this fails
raise Exception()
You can then successfully do:
>>> a = Outer_Class()
Logging in on something!
>>> a.do_something("foo")
decorated called
Doing something important with param_1
You have the command of
args[0]._login()
in the except. Since args[0] is a string and it doesn't have a _login method, you get the error message mentioned in the question.
Suppose I have the following TestCase
class TestSomething(unittest.TestCase):
def test_a(self):
# Do some testing
def test_b(self):
# Do some other testing
Is it possible for me to run TestSomething.test_a if I have a reference to that test? What I am looking to do is something like:
def run_test(test):
# Somehow runs the test
# HERE IS THE PART I AM REALLY STUCK ON
run_test(TestSomething.test_a)
I know that it's an awkward thing to do for normal unit testing. What I am trying to do is provide a test to be run as an argument to a function decorator. Essentially:
#corresponding_test(TestSomething.test_a)
def my_function_a():
# Something here
And then in the decorator basically check if the test for that function passes before running the function.
OP clearly stated that the real world use case is more involved, but this still needs saying:
Disclaimer: This is not a good, standard way to run unit tests. If you use this code to run unit tests, you're [probably] doing it wrong.
That said, your question intrigued me, so I went ahead and wrote a working demo for you:
"""
The `only_if_test_passes` decorator can be used to run a function if and
only if the argument test (unbound `TestCase` method) passes.
"""
import inspect
from unittest import TestCase, TestResult
class TestError(Exception):
pass
class MyTests(TestCase):
def test_pass(self):
# This passes because nothing went wrong
pass
def test_fail(self):
self.fail('This test will always fail')
def only_if_test_passes(test_method):
# Comments are computed values when passed MyTests.test_pass
test_case_class = inspect._findclass(test_method) # MyTests
test_case_name = test_case_class.__name__ # 'MyTests'
test_name = test_method.__name__ # 'test_pass'
# Introspection for Python 2:
# test_case_class = test_method.im_class
# test_case_name = test_case_class.__name__ # Same as for Python 3
# test_name = test_method.if_func.func_name
def decorator(fn):
def decorated(*args, **kwargs):
test_result = TestResult()
case = test_case_class(test_name) # MyTests('test_pass')
case(test_result)
if test_result.wasSuccessful():
return fn(*args, **kwargs)
else:
raise TestError('Unit test failed: {}.{}'.format(
test_case_name, test_name))
return decorated
return decorator
#only_if_test_passes(MyTests.test_pass)
def this_will_run():
print('This should output')
#only_if_test_passes(MyTests.test_fail)
def this_wont_ever_run():
print("Don't bother; you'll never see this.")
if __name__ == "__main__":
this_will_run()
this_wont_ever_run()
gist
The introspection will be a little different in Python 2.
See also: unittest.TestCase docs
I have some blocks of code which need to be wrapped by function.
try:
if config.DEVELOPMENT == True:
# do_some_stuff
except:
logger.info("Config is not set for development")
Then I'll do again:
try:
if config.DEVELOPMENT == True:
# do_some_another_stuff
except:
logger.info("Config is not set for development")
So, how can I wrap this "do_some_stuff" and "do_some_another_stuff"?
I'm trying to write function with contextmanager:
#contextmanager
def try_dev_config(name):
try:
if name is not None:
yield
except Exception as e:
print "not dev config"
with try_dev_config("config.DEVELOPMENT"):
# do_some_stuff
And I got an error:
RuntimeError: generator didn't yield
You could pass in a function.
boolean = True
def pass_this_in():
print("I just did some stuff")
def the_try_except_bit(function):
try:
if boolean:
function()
except:
print("Excepted")
# Calling the above code
the_try_except_bit(pass_this_in)
If you want to reduce the "pass_this_in" definition bit, then you can use lambda function definitions:
pass_this_in = lambda : print("I just did some stuff")
I am not sure that a context manager is the good method to achieve what you want. The context manager goal is to provide a mecanism to open/instantiate a resource, give access to it (or not) and close/clean it automatically when you no more need it.
IMHO, what you need is a decorator.
A decorator aims at executing code around a function call. It would force you to put each block of code in a function but I don't think it is so difficult. You can implement it like this:
class Config(object):
"""for demonstration purpose only: used to have a config.DEVELOPMENT value"""
DEVELOPMENT = True
class Logger(object):
"""for demonstration purpose only: used to have a logger.info method"""
#staticmethod
def info(msg):
print("Logged: {}".format(msg))
def check_dev_config(config, logger):
def dev_config_checker(func):
def wrapper(*args, **kwargs):
try:
if config.DEVELOPMENT:
func(*args, **kwargs)
except Exception as err:
logger.info(
"Config is not set for developpement: {}".format(err))
return wrapper
return dev_config_checker
#check_dev_config(Config, Logger)
def do_stuff_1():
print("stuff 1 done")
#check_dev_config(Config, Logger)
def do_stuff_2():
raise Exception("stuff 2 failed")
do_stuff_1()
do_stuff_2()
This code prints
stuff 1 done
Logged: Config is not set for developpement: stuff 2 failed
Explanations:
The check_dev_config function is actually a decorator generator which accepts the config and the logger as arguments.
It returns the dev_config_checker function which is an actual (and parameterised) decorator, and which accepts a function to decorate as argument.
This decorator returns a wrapper function which will actually run code around the decorated function call. In this function, the decorated function is called inside a try/except structure and only if the config.DEVELOPMENT is evaluated to True. In case of exception, the logger is used to log an information.
Each block of code to decorate is put into a function (do_stuff_1, do_stuff_2 and decorated with the check_dev_config decorator generator, giving it the config and the logger.
When decorated functions are called, they are called via their decorator and not directly. As you can see, the do_stuff_2 exception has been catched and the a message has been logged.
I am trying to use mock python library, and decided to use a decorator to hide a few repetitive operations for setting mock side_effect vars with corresponding replacement methods.
So far I have the following code, which does not enter the tests, it basically does nothing. Any ideas on how to fix this? Thanks!
import unittest
from mock import patch
# Replacement method for requests.get call
def mock_requests_get(url=None, **kwargs):
if url == URL1:
return {'url':url, 'status_code':200, 'status':'success'}
if url == URL2: A
return {'url':url, 'status_code':403}
# Replacement method for requests.post call
def mock_requests_post(url, data=None, **kwargs):
if url == URL1 and data['data']=='go':
return {'url':url, 'status_code':200, 'status':'success'}
if url == URL2 and data['data']!='gogo':
return {'url':url, 'status_code':403}
# Decorator which sets mock replacement methods
def mock_type(method=''):
def _mock(func):
def _decorator(mock_get, mock_post, *args, **kwargs):
print method
if method == 'GET':
mock_get.side_effect = mock_requests_get
if method == 'POST':
mock_post.side_effect = mock_requests_post
func(mock_get, mock_post, *args, **kwargs)
return _decorator
return _mock
#patch('requests.post')
#patch('requests.get')
class TestKeywordsApi(unittest.TestCase):
def setUp(self):
self.ka = KeywordsApi()
#mock_type('GET')
def test_get(self, mock_get, mock_post):
# Replace this code in mock_type decorator:
#mock_get.side_effect=mock_requests_get
print self.ka.get(URL1)
print self.ka.get(URL2)
# Do asserts
#mock_type('POST')
def test_post(self, mock_get, mock_post):
# Replace this code in mock_type decorator:
#mock_post.side_effect=mock_requests_post
print self.ka.post(URL1, {'data':'go'})
print self.ka.post(URL2, {'data':'go'})
# Do asserts
Make sure to replace the functions in the execution namespace. Depending on the imports, your current code may or may not be replacing what you think it is replacing. The following does what I believe you were trying to do.
import kwmodule
import unittest, mock
def mock_requests_get(url=None, **kwargs):
"Replacement method for requests.get call"
pass
def mock_requests_post(url, data=None, **kwargs):
"Replacement method for requests.post call"
pass
#mock.patch('kwmodule.requests.post', mock_requests_post)
#mock.patch('kwmodule.requests.get', mock_requests_get)
class TestKeywordsApi(unittest.TestCase):
def setUp(self):
self.ka = kwmodule.KeywordsApi()
However, you may be more successful with replacing get & post with autospecced MagicMock objects. If that doesn't work for you, then use your own custom replacement.
For instance, the following patch decorator would work if you know the order of calls to "get."
return_values = [{'url':url, 'status_code':200, 'status':'success'},
{'url':url, 'status_code':403},
AssertionError('Unexpected call.')]
#mock.patch('kwmodule.requests.get', autospec=True, side_effect=return_values)
This will return a "success" condition the first time it is called, an error the second time, and it will raise an AssertionError if it is called a third time. Nine times out of ten, I use side_effect instead of writing my own return generating function.