Ok so I have my apps, that takes requests from root / Almost everything is using traversal.
But i'd like to make on top of that site a rest api.
So I'm off with two choices. I either separate the that in two different apps and put that rest application to : rest.site.com, Or I can move it to site.com/rest/*traversal
If I'm doing "/rest/*traversal", I guess I'll have to add a route called rest_traversal where the traversal path will be *traversal with the route /rest/*traversal. I did that once for an admin page.
I was wondering if there was a cleanest way to do that. I tried to use virtual_root, but as I understand virtual_root is actually getting added to the path for traversal.
like having virtual_root = /cms and requesting /fun will create the following path /cms/fun
I on the other hand wish to have /cms/fun turned into /fun
I know this has been answered already, but in case someone arrives here looking for another possible way to make "subapps" and using them in pyramid, I wanted to point out that some interesting things can be done with pyramid.wsgi
"""
example of wsgiapp decorator usage
http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch/api/wsgi.html
"""
from pyramid.wsgi import wsgiapp2, wsgiapp
from pyramid.config import Configurator
from webob import Request, Response
import pprint
# define some apps
def wsgi_echo(environ, start_response):
"""pretty print out the environ"""
response = Response(body=pprint.pformat({k: v for k, v in environ.items()
if k not in ["wsgi.errors",
"wsgi.input",
"SCRIPT_NAME"]}))
return response(environ, start_response)
print Request.blank("/someurl").send(wsgi_echo).body
# convert wsgi app to a pyramid view callable
pyramid_echo = wsgiapp(wsgi_echo)
pyramid_echo_2 = wsgiapp2(wsgi_echo)
# wire up a pyramid application
config = Configurator()
config.add_view(pyramid_echo, name="foo") # /foo
config.add_view(pyramid_echo, name="bar") # /bar
config.add_view(pyramid_echo_2, name="foo_2") # /foo
config.add_view(pyramid_echo_2, name="bar_2") # /bar
pyramid_app = config.make_wsgi_app()
#call some urls
foo_body = Request.blank("/foo").send(pyramid_app).body
bar_body = Request.blank("/bar").send(pyramid_app).body
foo_body_2 = Request.blank("/foo_2").send(pyramid_app).body
bar_body_2 = Request.blank("/bar_2").send(pyramid_app).body
# both should be different because we arrived at 2 different urls
assert foo_body != bar_body, "bodies should not be equal"
# should be equal because wsgiapp2 fixes stuff before calling
# application in fact there's an additional SCRIPT_NAME in the
# environment that we are filtering out
assert foo_body_2 == bar_body_2, "bodies should be equal"
# so how to pass the path along? like /foo/fuuuu should come back
# /fuuuu does it
foo_body = Request.blank("/foo_2/fuuuu").send(pyramid_app).body
assert "'/fuuuu'," in foo_body, "path didn't get passed along"
# tldr: a wsgi app that is decorated with wsgiapp2 will recieve data
# as if it was mounted at "/", any url generation it has to do should
# take into account the SCRIPT_NAME variable that may arrive in the
# environ when it is called
If you're using traversal already, why not just use it to return your "rest API root" object when Pyramid traverses to /rest/? From there, everything will work naturally.
class ApplicationRoot(object):
def __getitem__(self, name):
if name == "rest":
return RestAPIRoot(parent=self, name=name)
...
If your "application tree" and "API tree" have the same children and you want to have different views registered for them depending on which branch of the tree the child is located in, you can use containment view predicate to register your API views, so they will only match when the child is inside the "API branch":
containment
This value should be a reference to a Python class or interface that a
parent object in the context resource’s lineage must provide in order
for this view to be found and called. The resources in your resource
tree must be “location-aware” to use this feature.
If containment is not supplied, the interfaces and classes in the
lineage are not considered when deciding whether or not to invoke the
view callable.
Another approach would be not to build a separate "API tree" but to use your "main" application's "URI-space" as RESTful API. The only problem with this is that GET and possibly POST request methods are already "taken" on your resources and mapped to your "normal" views which return HTML or consume HTTP form POSTs. There are numerous ways to work around this:
register the API views with a separate name, so, say GET /users/123 would return HTML and GET /users/123/json would return a JSON object. Similarly, POST /users/123 would expect HTTP form to be posted and POST /users/123/json would expect JSON. A nice thing about this approach is that you can easily add, say, an XML serializer at GET /users/123/xml.
use custom view predicates so GET /users/123 and GET /users/123?format=json are routed to different views. Actually, there's a built-in request_param predicate for that since Pyramid 1.2
use xhr predicate to differentiate requests based on HTTP_X_REQUESTED_WITH header or accept predicate to differentiate on HTTP_ACCEPT header
Related
Using URL Dispatch in Pyramid and pattern matching inside the route definition, I would like to ensure that URLs generated by route_path are valid and match the given route - what's the best way to do that?
This is my route definition - I want /numbers followed by a numeric string that's equivalent to 1 or greater...
config.add_route('numbers', 'numbers/{n:[1-9]\d*}/')
Then when I generate a route in a view for example, this is fine:
request.route_url('numbers', n=123)
It returns /numbers/123/ which is fine...
However, if I do...
request.route_url('numbers', n='chicken')
Pyramid returns /numbers/chicken/ which clearly doesn't match the pattern and will give a 404 when passed back to the app.
Is there a way that I can force Pyramid to check that the n matches the given pattern when it's generating the route? This would mean that I could be certain that route_url and route_path would give valid URLs that would work in the app.
You can use a pregenerator to add custom logic when generating urls. The regex patterns in the urls have only ever been used for matching purposes.
def check_n(request, elements, kw):
if not is_valid(kw['n']):
raise ValueError('invalid url parameter')
return elements, kw
config.add_route('numbers', 'numbers/{n:[1-9]\d*}/', pregenerator=check_n)
If you want to write something that validates the generated urls, it's not easy. The pregenerator doesn't know what route is being used and the route object matches paths, not kwargs. Your best bet is to wrap the route generation and validate the resulting url, instead of validating the kwargs themselves.
def validated_route_path(request, *args, **kwargs):
route_name = args[0]
path = request.route_path(*args, **kwargs)
route_intr = request.registry.introspector.get('routes', route_name)
route = route_intr['object']
if route.match(path) is None:
raise ValueError('invalid url generated')
return path
config.add_request_method(validated_route_path)
request.validated_route_path('numbers', n='chicken') # -> ValueError
Is there a way to detect unused templates in a Django project?
Before Django 1.3, that would have been possible with a simple string-matching function like this one. But since 1.3, there are generic class based views that automatically generate a template_name, if you don't override it (e.g. DetailView).
Also, if you override 3rd party module templates, those templates aren't used anywhere directly in your views.
Maybe it could be done by crawling all URL definitions, loading the corresponding views and getting the template_name from them?
I was curious if you could do this by monkey patching/decorating get_template instead. I think you can, though you have to find all the template loading
functions (I have two in my example below).
I used wrapt when I noticed it went beyond just loader.get_template, but it seems to the trick just fine. Of course, keep this 50000 km away from prod, but...
Now, the thing to follow as well is that I am driving this with unittests and nosetests so, if you have full branch coverage of your template-using Python code, you should be able to get most templates (assuming I didn't miss any get_template-type functions).
in settings.py
This is the "brains" to patch get_template & co.
import wrapt
import django.template.loader
import django.template.engine
def wrapper(wrapped, instance, args, kwargs):
#concatenate the args vector into a string.
# print "\n\n\n\n%s\nI am a wrapper \nusage:%s\n%s\n\n\n\n\n" % ("*"*80, usage, "*"*80)
try:
return wrapped(*args, **kwargs)
finally:
usage = ",".join([unicode(arg) for arg in args if arg])
track_usage(usage)
#you have to wrap whatever is loading templates...
#imported django module + class/method/function path of what needs to be
#wrapped within that module. comment those 2 lines out and you are back to
#normal
wrapt.wrap_function_wrapper(django.template.loader, 'get_template', wrapper)
wrapt.wrap_function_wrapper(django.template.engine, 'Engine.find_template', wrapper)
See safely-applying-monkey-patches-in-python for more details on wrapt. Actually easier to use than to understand the docs, decorators make my brain hurt.
Also, to track which django functions were doing the actual loads I mispelled some template names on purpose in the code and in templates, ran unit tests on it and looked at the stacktraces for missing template exceptions.
This is my rather badly-written function which adds to a set and puts it into
a json output....
def track_usage(usage):
fnp_usage = "./usage.json"
try:
with open(fnp_usage, "r") as fi:
data = fi.read()
#read the set of used templates from the json file
j_data = json.loads(data)
s_used_file = set(j_data.get("li_used"))
except (IOError,),e:
s_used_file = set()
j_data = dict()
s_used_file.add(usage)
#convert the set back to a list for json compatibility
j_data["li_used"] = list(s_used_file)
with open(fnp_usage, "w") as fo:
json.dump(j_data, fo)
and the ouput (with a script to format it):
import sys
import json
fnp_usage = sys.argv[1]
with open(fnp_usage, "r") as fi:
data = fi.read()
#read the set of used templates from the json file
j_data = json.loads(data)
li_used_file = j_data.get("li_used")
li_used_file.sort()
print "\n\nused templates:"
for t in li_used_file:
print(t)
From wrapping the 2 functions above, it seems to have caught extends, %includes and straight get_templates, as well as list-type templates that were being used by class-based views. It even caught my dynamically generated templates which aren't even on the file system but get loaded with a custom loader.
used templates:
bootstrap/display_form.html
bootstrap/errors.html
bootstrap/field.html
bootstrap/layout/baseinput.html
bootstrap/layout/checkboxselectmultiple.html
bootstrap/layout/field_errors.html
bootstrap/layout/field_errors_block.html
bootstrap/layout/help_text.html
bootstrap/layout/help_text_and_errors.html
bootstrap/layout/radioselect.html
bootstrap/whole_uni_form.html
django_tables2/table.html
dynamic_template:db:testdb:name:pssecurity/directive.PrimaryDetails.json
uni_form/layout/div.html
uni_form/layout/fieldset.html
websec/__base.html
websec/__full12.html
websec/__l_right_sidebar.html
websec/bootstrapped_home.html
websec/changedb.html
websec/login.html
websec/requirejs_config.html
websec/topnav.html
websec/user_msg.html
It's not possible to detect unused templates for certain, even in the absence of generic views, because you can always write code like this:
get_template(any_code_you_like()).render(context)
So even prior to Django 1.3 the django-unused-templates application you linked to could only have worked for projects that respected some kind of discipline about the use of templates. (For example, always having a string literal as the template argument to functions like get_template and render_to_response.)
Loading all the views wouldn't be sufficient either: a view may use different templates under different circumstances:
def my_view(request):
if request.user.is_authenticated():
return render(request, 'template1.html')
else:
return render(request, 'template2.html')
And of course templates may not be used by views at all, but by other parts of the system (for example, e-mail messages).
In Tornado, how do you tell apart the different request types? Also, what is the proper way to split out the requests? In the end if I go to /item/1.xml, I want xml, /item/1.html to be a proper html view etc.
Something like:
def getXML():
return self.render('somexmlresult.xml')
def getHTML():
return self.rendeR('htmlresult.html')
or
def get():
if request == 'xml':
return self.render('somexmlresult.xml')
elif request == 'html':
return self.render('htmlresult.html')
~ edit ~ I was shooting for something along the lines of rails' implementation seen here
I would prefer a self describing url like a RESTful application. An url part need not be required to represent the format of the resource. http://www.enterprise.com/customer/abc/order/123 must represent the resource irrespective of whether it is xml/html/json. A way to send the requested format is to send it as one of the request parameters.
http://www.enterprise.com/customer/abc/order/123?mimetype=application/xml
http://www.enterprise.com/customer/abc/order/123?mimetype=application/json
http://www.enterprise.com/customer/abc/order/123?mimetype=text/html
Use the request parameter to serialize to the appropriate format.
mimetype is the correct way to do this, however I can see where an end user would want a more simplistic way of accessing the data in the format they wish.
In order to maintain compatibility with standards compliant libraries etc you should ultimately determine the response type based on the mimetype requested and respond with the appropriate mimetype in the headers.
A way to achieve this while not breaking anything would be to add a parser that checks the URI that was requested for a suffix that matches a tuple of defined suffixes that the route can respond to, if it does and the mimetype is not already specified change the mimetype passed in to be the correct type for the suffix.
Make sure that the final decision is based on the supplied mimetype not the suffix.
This way others can interact with your RESTful service in the way their used to and you can still maintain ease of use for humans etc.
~ edit ~
Heres an example regexp that checks to see if it ends in .js | .html | .xml | .json. This is assuming your given the full URI.
(?:([^:/?#]+):)?(?://([^/?#]*))?([^?#]*\.(?:js|html|xml|json))(?:\?([^#]*))?(?:#(.*))?
Here's an example that's easier to interpret but less robust
^https?://(?:[a-z\-]+\.)+[a-z]{2,6}(?:/[^/#?]+)+\.(?:js|html|xml|json)$
These regex's are taken from rfc2396
First, set up the handlers to count on a restful style URI. We use 2 chunks of regex looking for an ID and a potential request format (ie html, xml, json etc)
class TaskServer(tornado.web.Application):
def __init__(self, newHandlers = [], debug = None):
request_format = "(\.[a-zA-Z]+$)?"
baseHandlers = [
(r"/jobs" + request_format, JobsHandler),
(r"/jobs/", JobsHandler),
(r"/jobs/new" + request_format, NewJobsHandler),
(r"/jobs/([0-9]+)/edit" + request_format, EditJobsHandler)
]
for handler in newHandlers:
baseHandlers.append(handler)
tornado.web.Application.__init__(self, baseHandlers, debug = debug)
Now, in the handler define a reusable function parseRestArgs (I put mine in a BaseHandler but pasted it here for ease of understanding/to save space) that splits out ID's and request formats. Since you should be expecting id's in a particular order, I stick them in a list.
The get function can be abstracted more but it shows the basic idea of splitting out your logic into different request formats...
class JobsHandler(BaseHandler):
def parseRestArgs(self, args):
idList = []
extension = None
if len(args) and not args[0] is None:
for arg in range(len(args)):
match = re.match("[0-9]+", args[arg])
if match:
slave_id = int(match.groups()[0])
match = re.match("(\.[a-zA-Z]+$)", args[-1])
if match:
extension = match.groups()[0][1:]
return idList, extension
def get(self, *args):
### Read
job_id, extension = self.parseRestArgs(args)
if len(job_id):
if extension == None or "html":
#self.render(html) # Show with some ID voodoo
pass
elif extension == 'json':
#self.render(json) # Show with some ID voodoo
pass
else:
raise tornado.web.HTTPError(404) #We don't do that sort of thing here...
else:
if extension == None or "html":
pass
# self.render(html) # Index- No ID given, show an index
elif extension == "json":
pass
# self.render(json) # Index- No ID given, show an index
else:
raise tornado.web.HTTPError(404) #We don't do that sort of thing here...
if i have this URL in from python code on appengine
http://localhost:8080/blog/view/2f1cab5844fb432b8426ae666c4ac493
how can i get the value of the key : 2f1cab5844fb432b8426ae666c4ac493
#Herms answer will work, but you may prefer this instead:
In the code that creates your webapp instance, capture the key part of the URL with a regex, like:
def main():
application = webapp.WSGIApplication( [
(r'/blog/view/(\w+)', MyBlogViewHandler),
## others listed here...
])
...then code your handler class like this - the key you captured will be passed to your get() method as an argument:
class MyBlogViewHandler(webapp.RequestHandler):
def get(self, key):
# do something useful with the 'key' argument
You can access the requested URL via self.request, assuming you're extending the standard webapp.RequestHandler class. That will give you access to the path and query, and you should be able to extract the values you want from the path.
Here's some documentation on the Request object:
http://code.google.com/appengine/docs/python/tools/webapp/requestclass.html
I am trying to have a custom URL which looks like this:
example.com/site/yahoo.com
which would hit this script like this=
example.com/details?domain=yahoo.com
can this be done using app.yaml?
the basic idea is to call "details" with the input "yahoo.com"
You can't really rewrite the URLs per se, but you can use regular expression groups to perform a similar kind of thing.
In your app.yaml file, try something like:
handlers:
- url: /site/(.+)
script: site.py
And in your site.py:
SiteHandler(webapp.RequestHandler):
def get(self, site):
# the site parameter will be what was passed in the URL!
pass
def main():
application = webapp.WSGIApplication([('/site/(.+)', SiteHandler)], debug=True)
util.run_wsgi_app(application)
What happens is, whatever you have after /site/ in the request URL will be passed to SiteHandler's get() method in the site parameter. From there you can do whatever it is you wanted to do at /details?domain=yahoo.com, or simply redirect to that URL.