I'm building a rate-limiting decorator in flask using redis stores that will recognize different limits on different endpoints. (I realize there are a number of rate-limiting decorators out there, but my use case is different enough that it made sense to roll my own.)
Basically the issue I'm having is ensuring that the keys I store in redis are class-specific. I'm using the blueprint pattern in flask, which basically works like this:
class SomeEndpoint(MethodView):
def get(self):
# Respond to get request
def post(self):
# Respond to post request
The issue here is that I want to be able to rate limit the post method of these classes without adding any additional naming conventions. In my mind the best way to do this would be something like this:
class SomeEndpoint(MethodView):
#RateLimit # Access SomeEndpoint class name
def post(self):
# Some response
but within the decorator, only the post function is in scope. How would I get back to the SomeEndpoint class given the post function? This is the basic layout of the decorator. That might be confusing, so here's a more concrete example of the decorator.
class RateLimit(object):
"""
The base decorator for app-specific rate-limiting.
"""
def __call__(self, f):
def endpoint(*args, **kwargs):
print class_backtrack(f) # Should print SomeEnpoint
return f(*args, **kwargs)
return endpoint
basically looking for what that class_backtrack function looks like. I've looked through the inspect module, but I haven't found anything that seems to accomplish this.
You can decorate the entire class instead of just the methods:
def wrap(Class, method):
def wrapper(self, *args, **kwargs):
print Class
return method(self, *args, **kwargs)
return method.__class__(wrapper, None, Class)
def rate_limit(*methods):
def decorator(Class):
for method_name in methods:
method = getattr(Class, method_name)
setattr(Class, method_name, wrap(Class, method))
return Class
return decorator
#rate_limit('post')
class SomeEndpoint(object):
def post(self):
pass
class Subclass(SomeEndpoint):
pass
a = Subclass()
a.post()
# prints <class 'SomeEndpoint'>
Related
I have written a python Flask application, which has a class and methods as below.
class PythonSample:
def method1():
pass # does something
def method2():
pass # does something
Now I have written another class which has decorator functions as below.
class PythonAuth:
def oauthAuth():
pass
Now I'm wiring oauthAuth decorator for all the methods of PythonSample class as below
import oauthAuth from PythonAuth
class PythonSample
#oauthAuth
def method1():
pass # does something
#oauthAuth
def method2():
pass # does something
Applying decorator at each method works fine.
Question: Instead of applying oauthAuth decorator to each of the methods. Is there a way to configure in python, as apply oauthAuth decorator to all the methods in a class and exclude certain methods.
Something like include auth for certain URLs and exclude authentication for certain urls
Please ignore the syntax of the python code here.
You can use a class decorator plus some magic.
Decorating Functions
Assume you have a decorator that just logs a string before calling the function.
def log(func):
def logged_func(*args, **kwargs):
print('logged')
func(*args, **kwargs)
return logged_func
Decorating classes
You can use the same trick, but with a class. log_all is a class decorator, cls is a class type. We use vars to walk the class dictionary, and look for methods by using callable(v). Decorate the method with log(v) and use setattr to change the cls definition to the new decorated method. Just like function decorators, return the class in the end.
def log_all(cls):
for k, v in vars(cls).items():
if callable(v):
setattr(cls, k, log(v))
return cls
I am ignoring k essentially, but k is the method name, you could leverage it to achieve your usage scenario.
Full code
Here is a full example, that should make some sense now.
def log(func):
def logged_func(*args, **kwargs):
print('logged')
func(*args, **kwargs)
return logged_func
def log_all(cls):
for k, v in vars(cls).items():
if callable(v):
setattr(cls, k, log(v))
return cls
#log_all
class A:
def method(self):
pass
Every method in class A should be decorated with the log decorator.
>>> a = A()
>>> a.method()
logged
I am implementing a content-aware caching system for a Django REST API. I would like to develop a component which can be added to existing views that would modify the behavior of the base class by checking the cache and falling back to the base class behavior on a miss.
basically, I have something like this:
class Base:
def get(self, request, *args, **kwargs):
....
return Response
class AnotherBase:
def get(self, request, *args, **kwargs):
....
return Response
class Derived(Base):
pass
class OtherDerived(AnotherBase):
pass
and my initial thought was to do something along the lines of
class Cacheable:
def get(self, request, *args, **kwargs):
cache_key = self.get_cache_key(request)
base_get = #.... and this is the problem
return cache.get(cache_key, base_get(request, *args, **kwargs))
def get_cache_key(self, request):
# .... do stuff
class Derived(Cacheable, Base):
pass
class AnotherDerived(Cacheable, AnotherBase):
pass
So clearly this doesn't work, as I don't know how, or if it's possible, or if it's advisable to access the sibling superclass(es) from a mixin.
My goal is an implementation that allows me to add caching behavior to existing views without touching the internals of the existing classes.
Given a view class, C, s.t. C.get(request, *args, **kwargs) -> Response, is there a function, F, s.t. F(C).get(... does the cache check before falling back to C.get? And in this quasi-formal notation, we'll say that adding a mixin to the leftmost parent class in the class definition counts as a function.
Is it more appropriate to use method decorators? or how would a class decorator work?
And then I've seen references to __metaclass__ in researching this, but I'm not clear on what that approach looks like.
This is Python 3.6
Simple example:
def Base:
def _get_data(self):
# get the data eg from database
return self._get_data_native()
def get(self, request, *args, **kwargs):
return Response(self._get_data())
def Cacheable(Base):
def _get_data(self):
# get from cache ...
result = ...
if result is None:
# or from base ...
result = ...
return result
def Derived(Cacheable):
def _get_data_native(self):
# get the data eg from database
...
By inheriting from Cacheable, you include the caching here, because _get_data is overwritten there.
For this problem, you don't need metaclasses or decorators, if you want to just add caching at one place.
Of course, a decorator could be used for including caching in an even more generic way.
See for example this answer: Is there a decorator to simply cache function return values?
The answer was a decorator and some Django-specific libraries.
from django.utils.decorators import method_decorator
from django.core.cache import cache
def cached_get(cache_key_func=None):
"""
Decorator to be applied via django.utils.decorators.method_decorator
Implements content-aware cache fetching by decorating the "get" method
on a django View
:param cache_key_func: a function of fn(request, *args, **kwargs) --> String
which determines the cache key for the request
"""
def decorator(func):
def cached_func(request, *args, **kwargs):
assert cache_key_func is not None, "cache_key_function is required"
key = cache_key_func(request, *args, **kwargs)
result = cache.get(key)
if result is None:
return func(request, *args, **kwargs)
return Response(result)
return cached_func
return decorator
#method_decorator(cached_get(cache_key_func=get_cache_key), name="get")
class SomeView(BaseView):
...
def get_cache_key(request):
# do arbitrary processing on request, the following is the naïve melody
key = urllib.urlencode(request.query_params)
return key
So the solution is to use Django's built-in method_decorator which applies its first argument, a decorator, to the decorated class's method, named by the second argument, name, to method_decorator. I define a higher-order function, cached_get, which takes another function as its argument, and returns a curried function (closure, so called). By calling this, with the function get_cache_key (and not, mind you, invoking that function) I have a decorator that will be applied to the 'get' method on SomeView.
The decorator itself is a straightforward Python decorator -- in this application, it is cached_func and the original, undecorated get method is func. Thus, cached_func replaces SomeView.get, so when SomeView.get is called, it first checks the cache, but falls back to the undecorated method on a miss.
I'm hopeful this approach provides a balance of generic applicability with content-aware key derivation.
My two cents:
You're walking into obscure territory here. Get familiar with all the related concepts, try a few, then decide.
Here is a good tutorial about metaclasses.
Here there's one about decorators.
I'm in no way affiliated to that site.
I wanted to make it easier to register callbacks using decorators when designing a library, but the problem is that they both use the same instance of the Consumer.
I am trying to allow both these examples to co-exist in the same project.
class SimpleConsumer(Consumer):
#Consumer.register_callback
def callback(self, body)
print body
class AdvancedConsumer(Consumer):
#Consumer.register_callback
def callback(self, body)
print body
a = AdvancedConsumer()
s = SimpleConsumer()
What happens here is that the callback implementation of AdvancedConsumer will override the one of the SimpleConsumer, as it is defined last.
The implementation of the decorator class is pretty simple.
class Consumer(object):
def start_consumer(self):
self.consuming_messages(callback=self._callback)
#classmethod
def register_callback(cls, function):
def callback_function(cls, body):
function(cls, body)
cls._callback = callback_function
return callback_function
I am very happy with the implementation, but as there is a possibility that someone will register a second callback I would like to ensure that it won't be a problem in the future. So, does anyone have a suggestion on how to implement this in a way that is not static?
The implementation shown here is obviously simplified, and as a precaution I have something like this in the code.
if cls._callback:
raise RuntimeError('_callback method already defined')
You can do it with a class decorator:
def register_callback(name):
def decorator(cls):
cls._callback = getattr(cls, name)
return cls
return decorator
#register_callback('my_func')
class SimpleConsumer(Consumer):
def my_func(self, body):
print body
If you want to decorate a method, you will get only a function in it so you cannot access any information about the class that the method is contained in.
But if only one callback should be available per class why just not call it _callback?
class SimpleConsumer(Consumer):
def _callback(self, body):
print body
Or do something like:
class SimpleConsumer(Consumer):
def my_func(self, body):
print body
_callback = my_func
?
I'd like to do something like this:
class Basehandler(webapp.RequestHandler):
def __init__(self):
if checkforspecialcase: #check something that always needs to be handled
return SpecialCaseHandler.get()
class NormalHandler(Basehandler):
def get(self):
print 'hello world'
return
class SpecialCaseHandler(Basehandler):
def get(self):
print 'hello special world'
return
The idea is that no matter what handler is initially called, if a certain case is met, we basically switch to another handler.
I'm pretty new to python, so I'm not sure whether what I'm trying to do is possible. Or whether this is the best approach. What I'm really trying to do is make sure to show someone the complete-your-profile page if they've started the registration process but haven't completed it... no matter what request they're making. So the "checkforspecialcase" looks at their sessions and checks for incomplete info.
To keep things DRY, use the Template Method pattern
class BaseHandler(webapp.RequestHandler):
def DoGet(self, *args):
''' defined in derived classes, actual per-handler get() logic'''
pass
def get(self, *args):
# don't get caught in endless redirects!
if specialCase and not self.request.path.startswith('/special'):
self.redirect('/special')
else:
self.DoGet(*args)
class NormalHandler(BaseHandler):
def DoGet(self, *args):
# normal stuff
class SpecialHandler(BaseHandler):
def DoGet(self, *args):
# SPECIAL stuff
WSGIApplication routes incoming requests based on the URL. For example,
application = webapp.WSGIApplication(
[('/special-case', SpecialCaseHandler)])
When checkforspecialcase passes, you can use self.redirect('/special-case').
Your Basehandler could just implement a get() that checks for the special case and either redirects or calls self.view(), and each handler could implement view() (or whatever you'd like to call it) rather than get().
I'm not really into writing a class for each of my handlers, or using inheritance so conspicuously, so I'd recommend rolling decorators like these:
routes = []
def get (route):
def makeHandler (handle, *args, **kwargs):
class Handler (webapp.RequestHandler):
def get (self, *args, **kwargs):
shouldRedirectToCompleteProfile = # do your test
if shouldRedirectToCompleteProfile:
self.redirect('/special-case')
else:
handle(self, *args, **kwargs)
routes.append((route, Handler))
return Handler
return makeHandler
def post (route):
def makeHandler (handle, *args, **kwargs):
class Handler (webapp.RequestHandler):
def post (self, *args, **kwargs):
handle(self, *args, **kwargs)
routes.append((route, Handler))
return Handler
return makeHandler
#get('/')
def home (ctx):
# <...>
#get('/whatever/(.*)/(.*)')
def whatever (ctx, whatever0, whatever1):
# <...>
#post('/submit')
def submit (ctx):
# <...>
application = webapp.WSGIApplication(routes)
I am trying to define a base request handling class so that the webapp pages may inherit some basic methods and variable which otherwise would be required to be repeatedly be defined for each page of the application. A sort of similar functionality like django preprocessors. This is my base class from which other pages inherit:
class BasePage(webapp.RequestHandler):
def __init__(self):
self.user = users.get_current_user()
self.template_values = {
'user': self.user,
'environ': self, #I don't like the idea of passing the whole environ object to a template
##The below three functions cannot be executed during _init_ because of absence of self.request
#'openid_providers': self.openid_providers(),
#'logout_url': self.get_logout_url(),
#'request': self.get_request(),
}
##A sort of similar functionality like render_to_response in django
def render_template(self, template_name, values = None, *args, **kwargs):
#PATH is the directory containing the templates
if values:
for value in values: self.template_values[value] = values[value]
self.response.out.write(template.render(PATH+template_name, self.template_values, *args, **kwargs))
##Returns request as the name suggests
def logout_url(self):
return users.create_logout_url(self.request.url)
##Returns request as the name suggests
def request(self):
return request
##Returns openid login urls
def openid_providers(self):
#OPENID_POVIDERS is a list of dictionary
for p in OPENID_PROVIDERS:
p['login_url'] = users.create_login_url(self.request.get('next', '/') , p['name'], p['url'])
return OPENID_PROVIDERS
Everything is working fine except that I cannot pass some variables during initialization as self.request is not available. So for a workaround what I did is pass on whole self variable as a template variable.
Is there some other way to provide the template variables (request, logout_url etc) to the templates?
A much simpler solution than bgporter's is to do the common setup in the initialize method of webapp.RequestHandler. Here's an example from work, where we wanted to add a Django-like is_ajax method to the request object:
class BaseHandler(webapp.RequestHandler):
def initialize(self, request, response):
super(BaseHandler, self).initialize(request, response)
# Add a Django-like is_ajax() method to the request object
request.is_ajax = lambda: \
request.environ.get('HTTP_X_REQUESTED_WITH') == 'XMLHttpRequest'
This method is called to, uh, initialize each request handler with the current request and response objects, before the appropriate get or post (or whatever) methods are called.
I've solved that problem in my AppEngine code by using the Template Method Pattern
Basically, the base class looks like:
class MyBasePage(webapp.RequestHandler):
def __init__(self):
# common setup/init stuff here,
# omitted for this discussion
def Setup(self):
# request handling setup code needed in both GET/POST methods, like
# checking for user login, getting session cookies, etc.
# omitted for this discussion
def get(self, *args):
self.Setup()
# call the derived class' 'DoGet' method that actually has
# the logic inside it
self.DoGet(*args)
def post(self, *args):
self.Setup()
# call the derived class' 'DoPost' method
self.DoPost(*args)
def DoGet(self, *args):
''' derived classes override this method and
put all of their GET logic inside. Base class does nothing.'''
pass
def DoPost(self, *args):
''' derived classes override this method and
put all of their POST logic inside. Base class does nothing.'''
pass
...your derived classes then mostly just need to worry about the guts of those DoGet() and DoPost() methods.