We Keep Coding

aiohttp middleware and updating the url for a request

I am looking for help with an aiohttp middleware I'm working on that will automatically add trailing slashes to a uri if it's missing. Aiohttp requires that when you define a url, you define two routes for each path in your list of routes, one with a trailing slash and one without. They have an example middleware that addresses this by finding if the uri does not end with a / and adding it if not, but then they use an http 302 redirect to tell the client to go to the new uri. Otherwise a relative uri like /endpoint will 404 while /endpoint/ will work.
This is their middleware:
https://github.com/aio-libs/aiohttp/blob/master/aiohttp/web_middlewares.py
Using their middleware as a basis, I'm trying to accomplish the same thing but without a redirect on the client side. I want the handling to only be on the server. My current attempt at this is shown below and seems to update the request in flight, but I still get a 404:
from aiohttp.web_urldispatcher import SystemRoute
def middleware(f):
f.__middleware_version__ = 1
return f
def trailing_slashes():
#middleware
async def impl(request, handler):
if isinstance(request.match_info.route, SystemRoute):
rel_url = str(request.rel_url)
if '?' in rel_url:
path, query = rel_url.split('?', 1)
query = f'?{query}'
else:
path = rel_url
query = ''
if not path.endswith('/'):
rel_url = f'{path}/{query}'
request = request.clone(rel_url=rel_url)
return await handler(request)
return impl
This is a class that implements the middleware and illustrates the problem.
from aiohttp import web
import slashes_mw
class ClassName(object):
def __init__(self):
self.app = web.Application(middlewares=[slashes_mw.trailing_slashes()])
self.app.add_routes(self.get_routes())
web.run_app(self.app, port=80, host='0.0.0.0')
def get_routes(self):
return [
web.get('/', self.handler),
web.get('/{name}/', self.handler)
]
def handler(self, request):
return web.Response(text='hello')
ClassName()
Any ideas? Thanks for your help.

I think the redirect approach that they propose is quite correct, as the routes are simply not the same. But you might try to just register the second "version" of the route as well:
def get_routes(self):
return [
web.get('/', self.handler),
web.get('/{name}/', self.handler),
web.get('/{name}', self.handler)
]
This should handle your problem server-side (NOT TESTED).
As I understand the framework, your approach does not work because the specific handler is selected BEFORE the different middlewares are called. If you call the handler function in the middleware, there is no "routing" lookup done anymore. Or at least that is how I understand their documentation: https://docs.aiohttp.org/en/stable/web_advanced.html#middlewares
Might be interesting as well: https://docs.aiohttp.org/en/stable/web_quickstart.html#variable-resources

Tornado routing to a "base" handler

I use tornado 4.5.2 with routing implementation.
My server have two versions of API, let them call base and fancy. So a client is able to use both of them:
GET /base/foo
GET /base/baz
GET /fancy/foo
GET /fancy/baz
However, some fancy handlers may not be implemented; In this case a base one should be used.
In example:
application = web.Application([
(r"/base/foo", handlers.BaseFooHandler, {"some": "settings"}),
(r"/base/baz", handlers.BaseBazHandler, {"some": "settings"}),
(r"/fancy/foo", handlers.FancyFooHandler, {"some": "settings"}),
])
when cilent requests GET /fancy/baz the BaseBazHandler should do the job.
How can I achieve that with tornado routing?

Since you're registering your routes using a decorator, you can create a custom router that will respond to all the unmatched/unregistered /fancy/.* routes. For this to work correctly, you'll have to register your router at the end.
That way your custom router will be matched only if there isn't already a /fancy/... route registered. So, that means the custom router class will need to do these things:
Check if a fallback BaseBazHandler exists or not.
If exists, forward the request to it.
Else, return a 404 error.
Before proceeding any further, you'll have to create a custom class to handle 404 requests. This is necessary because if not handler is found, then this is the easiest way to return a 404 error.
class Handle404(RequestHandler):
def get(self):
self.set_status(404)
self.write('404 Not Found')
Okay, now let's write the custom router:
from tornado.routing import Router
class MyRouter(Router):
def __init__(self, app):
self.app = app
def find_handler(self, request, **kwargs):
endpoint = request.path.split('/')[2] # last part of the path
fallback_handler = 'Base%sHandler' % endpoint.title()
# fallback_handler will look like this - 'BaseBazHandler'
# now check if the handler exists in the current file
try:
handler = globals()[fallback_handler]
except KeyError:
handler = Handle404
return self.app.get_handler_delegate(request, handler)
Finally, after you've added all other routes, you can register your custom router:
from tornado.routing import PathMatches
application.add_handlers(r'.*', # listen for all hosts
[
(PathMatches(r"/fancy/.*"), MyRouter(application)),
]
)
I should point out that MyRouter.find_handler, only check checks for handlers in the current module (file). Modify the code to search for handlers in different modules, if you want.

Routing with python Falcon

I am new to Falcon framework of python. I have a question regarding the usage of middleware class of Falcon. Is it wise to use custom routers and authentication of requests in the middleware or should this be handled only on the routing
**main.py**
import falcon
import falcon_jsonify
import root
from waitress import serve
if __name__ == "__main__":
app = falcon.API(
middleware=[falcon_jsonify.Middleware(help_messages=True),
root.customRequestParser()]
)
serve(app, host="0.0.0.0", port=5555)
root.py where I am planning to write the custom routes
import json
import falcon
class Home(object):
#classmethod
def getResponse(self):
return {"someValue": "someOtherValue"}
def process_request_path(path):
path = path.lstrip("/").split("/")
return path
class customRequestParser(object):
def process_request(self, req, resp):
print process_request_path(req.path)
I also saw examples using app = falcon.API(router=CustomRouter()). I saw a documentation on the falcon official documentation page - http://falcon.readthedocs.io/en/stable/api/routing.html
Please let me know if there are any references that I can look through.

To quote the Falcon Community FAQ
How do I authenticate requests?
Hooks and middleware components can be used together to authenticate and authorize requests. For example, a middleware component could be used to parse incoming credentials and place the results in req.context. Downstream components or hooks could then use this information to authorize the request, taking into account the user’s role and the requested resource.
Falcon's Hooks are decorators used on the either a particular request function (i.e. on_get) or on an entire class. They're great for validating incoming requests, so as the FAQ says, authentication could be done at this point.
Here's an (untested) example I knocked up:
def AuthParsingMiddleware(object):
def process_request(self, req, resp):
req.context['GodMode'] = req.headers.get('Auth-Token') == 'GodToken':
# Might need process_resource & process_response
def validate_god_mode(req, resp, resource, params):
if not req.context['GodMode']:
raise falcon.HTTPBadRequest('Not authorized', 'You are not god')
def GodLikeResource(object):
#falcon.before(validate_god_mode):
def on_get(self, req, resp):
resp.body = 'You have god mode; I prostrate myself'
app = falcon.API(
middleware=[falcon_jsonify.Middleware(help_messages=True),
AuthParsingMiddleware()]
)
app.add_route('/godlikeresource', GodLikeResource())
Or better...
There is a falcon-auth package.

Can a WSGI middleware modify a request body and then pass it along?

I'm trying to write a middleware that rewrites POST requests to a different method when it finds a "_method" parameter in the body. Someone on the Internet wrote this piece of code:
from werkzeug import Request
class MethodRewriteMiddleware(object):
def __init__(self, app, input_name='_method'):
self.app = app
self.input_name = input_name
def __call__(self, environ, start_response):
request = Request(environ)
if self.input_name in request.form:
method = request.form[self.input_name].upper()
if method in ['GET', 'POST', 'PUT', 'DELETE']:
environ['REQUEST_METHOD'] = method
return self.app(environ, start_response)
As I understand the code, it parses the form, retrieves a possible "_method" parameter and if it's found and whitelisted it will overwrite the current method. It works fine for DELETE requests and it rewrites the method with no problems. However, when I try to send a regular, non-rewritten POST this middleware makes the whole app hang. My best guess is that since I accessed the body in the middleware, the body is no longer available for the application so it hangs forever. However, this doesn't seem to impact rewritten requests, so the code deepest code path (checking the whitelist) works correctly but the other code path is somehow destroying/blocking the request.
I don't think it's relevant, but I'm mounting this middleware on top of a Flask app.
EDIT: I think trying to access the request from a handler in Flask is blocking. Does Flask use mutexes or something like that internally?
I'm not even sure how to debug this.

environ is a dictionary with a wsgi.input key that is a stream object. The manual way to do what you're looking for is to read the data, make any changes and then create a new string to replace the old stream in the environ dictionary. The example below (without any error handling or other important things) allows you to work with the request body:
def __call__(self, environ, start_response):
body = environ['wsgi.input'].read()
... do stuff to body ...
new_stream = io.ByteIO(modified_body)
environ['wsgi.input'] = new_stream
return self.app(environ, start_response)

Calling flask restful API resource methods

I'm creating an API with Flask that is being used for a mobile platform, but I also want the application itself to digest the API in order to render web content. I'm wondering what the best way is to access API resource methods inside of Flask? For instance if I have the following class added as a resource:
class FooAPI(Resource):
def __init__(self):
# Do some things
super(FooAPI, self).__init__()
def post(self, id):
#return something
def get(self):
#return something
api = Api(app)
api.add_resource(FooAPI, '/api/foo', endpoint = 'foo')
Then in a controller I want:
#app.route("/bar")
def bar():
#Get return value from post() in FooAPI
How do I get the return value of post() from FooAPI? Can I do it somehow through the api variable? Or do I have to create an instance of FooAPI in the controller? It seems like there has to be an easy way to do this that I'm just not understanding...

The obvious way for your application to consume the API is to invoke it like any other client. The fact that the application would be acting as a server and a client at the same time does not matter, the client portion can place requests into localhost and the server part will get them in the same way it gets external requests. To generate HTTP requests you can use requests, or urllib2 from the standard library.
But while the above method will work just fine it seems overkill to me. In my opinion a better approach is to expose the common functionality of your application in a way that both the regular application and the API can invoke. For example, you could have a package called FooLib that implements all the shared logic, then FooAPI becomes a thin wrapper around FooLib, and both FooAPI and FooApp call FooLib to get things done.

Another approach is to have both the app and API in the same Flask(-RESTful) instance. Then, you can have the app call the API methods/functions internally (without HTTP). Let's consider a simple app that manages files on a server:
# API. Returns filename/filesize-pairs of all files in 'path'
#app.route('/api/files/',methods=['GET'])
def get_files():
files=[{'name':x,'size':sys.getsizeof(os.path.join(path,x))} for x in os.listdir(path)]
return jsonify(files)
# app. Gets all files from the API, uses the API data to render a template for the user
#app.route('/app/files/',methods=['GET'])
def app_get_files():
response=get_files() # you may verify the status code here before continuing
return render_template('files.html',files=response.get_json())
You can push all your requests around (from the API to the app and back) without including them in your function calls since Flask's request object is global. For example, for an app resource that handles a file upload, you can simply call:
#app.route('/app/files/post',methods=['POST'])
def app_post_file():
response=post_file()
flash('Your file was uploaded succesfully') # if status_code==200
return render_template('home.html')
The associated API resource being:
#app.route('/api/files/',methods=['POST'])
def post_file():
file=request.files['file']
....
....
return jsonify({'some info about the file upload'})
For large volumes of application data, though, the overhead of wrapping/unwrapping JSON makes Miguel's second solution preferrable.
In your case, you would want to call this in your contoller:
response=FooAPI().post(id)

I managed to achieve this, sometimes API's get ugly, in my case, I need to recursively call the function as the application has a extremely recursive nature (a tree). Recursive functions itself are quite expensive, recursive HTTP requests would be a world of memory and cpu waste.
So here's the snippet, check the third for loop:
class IntentAPI(Resource):
def get(self, id):
patterns = [pattern.dict() for pattern in Pattern.query.filter(Pattern.intent_id == id)]
responses = [response.dict() for response in Response.query.filter(Response.intent_id == id)]
return jsonify ( { 'patterns' : patterns, 'responses' : responses } )
def delete(self, id):
for pattern in Pattern.query.filter(Pattern.intent_id == id):
db.session.delete(pattern)
for response in Response.query.filter(Response.intent_id == id):
db.session.delete(response)
for intent in Intent.query.filter(Intent.context == Intent.query.get(id).set_context):
self.delete(intent.id) #or IntentAPI.delete(self, intent.id)
db.session.delete(Intent.query.get(id))
db.session.commit()
return jsonify( { 'result': True } )