I'm writing a admin website which control several websites with same program and database schema but different content. The URL I designed like this:
http://example.com/site A list of all sites which under control
http://example.com/site/{id} A brief overview of select site with ID id
http://example.com/site/{id}/user User list of target site
http://example.com/site/{id}/item A list of items sold on target site
http://example.com/site/{id}/item/{iid} Item detailed information
# ...... something similar
As you can see, nearly all URL are need the site_id. And in almost all views, I have to do some common jobs like query Site model against database with the site_id. Also, I have to pass site_id whenever I invoke request.route_path.
So... is there anyway for me to make my life easier?
It might be useful for you to use a hybrid approach to get the site loaded.
def groupfinder(userid, request):
user = request.db.query(User).filter_by(id=userid).first()
if user is not None:
# somehow get the list of sites they are members
sites = user.allowed_sites
return ['site:%d' % s.id for s in sites]
class SiteFactory(object):
def __init__(self, request):
self.request = request
def __getitem__(self, key):
site = self.request.db.query(Site).filter_by(id=key).first()
if site is None:
raise KeyError
site.__parent__ = self
site.__name__ = key
site.__acl__ = [
(Allow, 'site:%d' % site.id, 'view'),
]
return site
We'll use the groupfinder to map users to principals. We've chosen here to only map them to the sites which they have a membership within. Our simple traversal only requires a root object. It updates the loaded site with an __acl__ that uses the same principals the groupfinder is setup to create.
You'll need to setup the request.db given patterns in the Pyramid Cookbook.
def site_pregenerator(request, elements, kw):
# request.route_url(route_name, *elements, **kw)
from pyramid.traversal import find_interface
# we use find_interface in case we improve our hybrid traversal process
# to take us deeper into the hierarchy, where Site might be context.__parent__
site = find_interface(request.context, Site)
if site is not None:
kw['site_id'] = site.id
return elements, kw
Pregenerator can find the site_id and add it to URLs for you automatically.
def add_site_route(config, name, pattern, **kw):
kw['traverse'] = '/{site_id}'
kw['factory'] = SiteFactory
kw['pregenerator'] = site_pregenerator
if pattern.startswith('/'):
pattern = pattern[1:]
config.add_route(name, '/site/{site_id}/' + pattern, **kw)
def main(global_conf, **settings):
config = Configurator(settings=settings)
authn_policy = AuthTktAuthenticationPolicy('seekrit', callback=groupfinder)
config.set_authentication_policy(authn_policy)
config.set_authorization_policy(ACLAuthorizationPolicy())
config.add_directive(add_site_route, 'add_site_route')
config.include(site_routes)
config.scan()
return config.make_wsgi_app()
def site_routes(config):
config.add_site_route('site_users', '/user')
config.add_site_route('site_items', '/items')
We setup our application here. We also moved the routes into an includable function which can allow us to more easily test the routes.
#view_config(route_name='site_users', permission='view')
def users_view(request):
site = request.context
Our views are then simplified. They are only invoked if the user has permission to access the site, and the site object is already loaded for us.
Hybrid Traversal
A custom directive add_site_route is added to enhance your config object with a wrapper around add_route which will automatically add traversal support to the route. When that route is matched, it will take the {site_id} placeholder from the route pattern and use that as your traversal path (/{site_id} is path we define based on how our traversal tree is structured).
Traversal happens on the path /{site_id} where the first step is finding the root of the tree (/). The route is setup to perform traversal using the SiteFactory as the root of the traversal path. This class is instantiated as the root, and the __getitem__ is invoked with the key which is the next segment in the path ({site_id}). We then find a site object matching that key and load it if possible. The site object is then updated with a __parent__ and __name__ to allow the find_interface to work. It is also enhanced with an __acl__ providing permissions mentioned later.
Pregenerator
Each route is updated with a pregenerator that attempts to find the instance of Site in the traversal hierarchy for a request. This could fail if the current request did not resolve to a site-based URL. The pregenerator then updates the keywords sent to route_url with the site id.
Authentication
The example shows how you can have an authentication policy which maps a user into principals indicating that this user is in the "site:" group. The site (request.context) is then updated to have an ACL saying that if site.id == 1 someone in the "site:1" group should have the "view" permission. The users_view is then updated to require the "view" permission. This will raise an HTTPForbidden exception if the user is denied access to the view. You can write an exception view to conditionally translate this into a 404 if you want.
The purpose of my answer is just to show how a hybrid approach can make your views a little nicer by handling common parts of a URL in the background. HTH.
For the views, you could use a class so that common jobs can be carried out in the __init__ method (docs):
from pyramid.view import view_config
class SiteView(object):
def __init__(self, request):
self.request = request
self.id = self.request.matchdict['id']
# Do any common jobs here
#view_config(route_name='site_overview')
def site_overview(self):
# ...
#view_config(route_name='site_users')
def site_users(self):
# ...
def route_site_url(self, name, **kw):
return self.request.route_url(name, id=self.id, **kw)
And you could use a route prefix to handle the URLs (docs). Not sure whether this would be helpful for your situation or not.
from pyramid.config import Configurator
def site_include(config):
config.add_route('site_overview', '')
config.add_route('site_users', '/user')
config.add_route('site_items', '/item')
# ...
def main(global_config, **settings):
config = Configurator()
config.include(site_include, route_prefix='/site/{id}')
Related
I have a catchall route that is catching my request before my call to add_route within an action created with a custom directive. How can I prevent that from happening? Ie. manually place the catchall route at the end of the route matching process or put my action's add_route at the front of the route matching process?
I tried swapping the order of the includes between admin/routes.py and routes.py but that didn't seem to have an affect. A quick solution (and probably good idea thinking about it now) is to filter admin from the pattern in the catchall. But I feel like this is going to come up again where I cannot do that so I'm asking this question.
__init__.py
def main(global_config, **settings):
#...
config.include('.directives')
#...
config.include(".routes")
#...
for k in (".customer", ".admin"):
config.scan(k) # this picks up the admin/admin_routes.py
return config.make_wsgi_app()
admin/routes.py
def includeme(config):
config.add_dispatched_route(
"admin-plants-edit",
"/admin/plants/{id}",
'plant',
)
routes.py
def includeme(config):
#...
config.add_route("page-view", "/*url")
directives.py
from pyramid.config import PHASE0_CONFIG
from pyramid.httpexceptions import HTTPNotFound
def includeme(config):
def add_dispatched_route(config, route_name, route_pattern, dispatch_with):
def route_factory(request):
api = request.find_service(name=dispatch_with)
obj = api.by_id(request.matchdict["id"])
if not obj:
raise HTTPNotFound()
return obj
def route_pregenerator(request, elements, kw):
api = request.find_service(name=dispatch_with)
try:
obj = kw.pop(api.get_pregenerator_kw())
except KeyError:
pass
else:
kw["id"] = obj.id
return elements, kw
def register():
config.add_route(route_name, route_pattern, factory=route_factory, pregenerator=route_pregenerator)
config.action(('dispatched_route', route_name), register, order=PHASE0_CONFIG)
config.add_directive('add_dispatched_route', add_dispatched_route)
The way the phases work in Pyramid doesn't allow you to re-order actions within a phase. Basically imagine that every call to config.add_route gets appended to a list, and then later that list is iterated in order to add the routes. You cannot inject a call to config.add_route before another call to it.
However if you wrap every call to add_route in your own action, then you have an opportunity to defer everything, order things, and then call config.add_route with each route in the order you wish.
Let's say I've created a (hopefully) reusable app, fooapp:
urls.py
urls('^(?P<userid>\d+)/$', views.show_foo),
and fooapp's views.py:
def show_foo(request, userid):
usr = shortcuts.get_object_or_404(User, pk=userid)
... display a users' foo ...
return render_to_response(...)
Since it's a reusable app, it doesn't specify any access control (e.g. #login_required).
In the site/project urls.py, the app is included:
urls('^foo/', include('fooapp.urls')),
How/where can I specify that in this site only staff members should be granted access to see a user's foo?
How about if, in addition to staff members, users should be able to view their own foo (login_required + request.user.id == userid)?
I didn't find any obvious parameters to include..
Note: this has to do with access control, not permissions, i.e. require_staff checks User.is_staff, login_required checks if request.user is logged in, and user-viewing-their-own-page is described above. This question is in regards to how a site can specify access control for a reusable app.
Well, I've found a way that works by iterating over the patterns returned by Django's include:
from django.core.urlresolvers import RegexURLPattern, RegexURLResolver
def urlpatterns_iterator(patterns):
"""Recursively iterate through `pattern`s.
"""
_patterns = patterns[:] # create a copy
while _patterns:
cur = _patterns.pop()
if isinstance(cur, RegexURLPattern):
yield cur
elif isinstance(cur, RegexURLResolver):
_patterns += cur.url_patterns
else:
raise ValueError("I don't know how to handle %r." % cur)
def decorate(fn, (urlconf_module, app_name, namespace)):
"""Iterate through all the urls reachable from the call to include and
wrap the views in `fn` (which should most likely be a decorator).
(the second argument is the return value of Django's `include`).
"""
# if the include has a list of patterns, ie.: url(<regex>, include([ url(..), url(..) ]))
# then urlconf_module doesn't have 'urlpatterns' (since it's already a list).
patterns = getattr(urlconf_module, 'urlpatterns', urlconf_module)
for pattern in urlpatterns_iterator(patterns):
# the .callback property will set ._callback potentially from a string representing the path to the view.
if pattern.callback:
# the .callback property doesn't have a setter, so access ._callback directly
pattern._callback = fn(pattern._callback)
return urlconf_module, app_name, namespace
this is then used in the site/project's urls.py, so instead of:
urls('^foo/', include('fooapp.urls')),
one would do:
from django.contrib.admin.views.decorators import staff_member_required as _staff_reqd
def staff_member_required(patterns): # make an include decorator with a familiar name
return decorate(_staff_reqd, patterns)
...
urls('^foo/', staff_member_required(include('fooapp.urls'))),
I want to work with multiple databases with Python Pyramid Framework and SQL Alchemy.
I have 1 database with user information, and multiple databases (with the same structure) where the application information is stored. Each user at login time selects a database and is only shown information from that database (not others).
How should I structure my code?
I was thinking on saving in the session the dbname and on every request check user permission on the selected database and generate a new db session. So my view would look like (PSEUDO CODE):
#view_config(route_name='home', renderer='json')
def my_view_ajax(request):
try:
database = int(request.GET['database'])
# check user permissions from user database
engine = create_engine('postgresql://XXX:XXX#localhost/'+database)
DBSession.configure(bind=engine)
items = DBSession.query('table').all()
except DBAPIError:
return 'error'
return items
Should I generate a new db session with the user information on each request? Or is there a better way?
Thanks
This is quite easy to do in Pyramid+SQLAlchemy, but you'll likely want to
switch to a heavier boilerplate, more manual session management style, and you'll want to be up on the session management docs for SQLA 'cause you can easily trip up when working with multiple concurrent sessions. Also, things like connection management should stay out of views, and be in components that live in the server start up lifecycle and are shared across request threads. If you're doing it right in Pyramid, your views should be pretty small and you should have lots of components that work together through the ZCA (the registry).
In my apps, I have a db factory objects that get sessions when asked for them, and I instantiate these objects in the server start up code (the stuff in __ init __.py) and save them on the registry. Then you can attach sessions for each db to your request object with the reify decorator, and also attach a house keeping end of request cleanup method to close them. This can be done either with custom request factories or with the methods for attaching to the request right from init, I personally wind up using the custom factories as I find it easier to read and I usually end up adding more there.
# our DBFactory component, from some model package
class DBFactory(object):
def __init__(self, db_url, **kwargs):
db_echo = kwargs.get('db_echo', False)
self.engine = create_engine(db_url, echo=db_echo)
self.DBSession = sessionmaker(autoflush=False)
self.DBSession.configure(bind=self.engine)
self.metadata = Base.metadata
self.metadata.bind = self.engine
def get_session(self):
session = self.DBSession()
return session
# we instantiate them in the __init__.py file, and save on registry
def main(global_config, **settings):
"""runs on server start, returns a Pyramid WSGI application """
config = Configurator(
settings=settings,
# ask for a custom request factory
request_factory = MyRequest,
)
config.registry.db1_factory = DBFactory( db_url=settings['db_1_url'] )
config.registry.db2_factory = DBFactory( db_url=settings['db_2_url'] )
# and our custom request class, probably in another file
class MyRequest(Request):
"override the pyramid request object to add explicit db session handling"
#reify
def db1_session(self):
"returns the db_session at start of request lifecycle"
# register callback to close the session automatically after
# everything else in request lifecycle is done
self.add_finished_callback( self.close_dbs_1 )
return self.registry.db1_factory.get_session()
#reify
def db2_session(self):
self.add_finished_callback( self.close_dbs_2 )
return self.registry.db2_factory.get_session()
def close_dbs_1(self, request):
request.db1_session.close()
def close_dbs_2(self, request):
request.db2_session.close()
# now view code can be very simple
def my_view(request):
# get from db 1
stuff = request.db1_session.query(Stuff).all()
other_stuff = request.db2_session.query(OtherStuff).all()
# the above sessions will be closed at end of request when
# pyramid calls your close methods on the Request Factory
return Response("all done, no need to manually close sessions here!")
How do you get the model object of a tastypie modelresource from it's uri?
for example:
if you were given the uri as a string in python, how do you get the model object of that string?
Tastypie's Resource class (which is the guy ModelResource is subclassing ) provides a method get_via_uri(uri, request). Be aware that his calls through to apply_authorization_limits(request, object_list) so if you don't receive the desired result make sure to edit your request in such a way that it passes your authorisation.
A bad alternative would be using a regex to extract the id from your url and then use it to filter through the list of all objects. That was my dirty hack until I got get_via_uri working and I do NOT recommend using this. ;)
id_regex = re.compile("/(\d+)/$")
object_id = id_regex.findall(your_url)[0]
your_object = filter(lambda x: x.id == int(object_id),YourResource().get_object_list(request))[0]
You can use get_via_uri, but as #Zakum mentions, that will apply authorization, which you probably don't want. So digging into the source for that method we see that we can resolve the URI like this:
from django.core.urlresolvers import resolve, get_script_prefix
def get_pk_from_uri(uri):
prefix = get_script_prefix()
chomped_uri = uri
if prefix and chomped_uri.startswith(prefix):
chomped_uri = chomped_uri[len(prefix)-1:]
try:
view, args, kwargs = resolve(chomped_uri)
except Resolver404:
raise NotFound("The URL provided '%s' was not a link to a valid resource." % uri)
return kwargs['pk']
If your Django application is located at the root of the webserver (i.e. get_script_prefix() == '/') then you can simplify this down to:
view, args, kwargs = resolve(uri)
pk = kwargs['pk']
Are you looking for the flowchart? It really depends on when you want the object.
Within the dehydration cycle you simple can access it via bundle, e.g.
class MyResource(Resource):
# fields etc.
def dehydrate(self, bundle):
# Include the request IP in the bundle if the object has an attribute value
if bundle.obj.user:
bundle.data['request_ip'] = bundle.request.META.get('REMOTE_ADDR')
return bundle
If you want to manually retrieve an object by an api url, given a pattern you could simply traverse the slug or primary key (or whatever it is) via the default orm scheme?
In a Django application, I have more than a handful of views which return JSON, with an invocation similar to:
return HttpResponse(json.dumps(content), mimetype="application/json")
I want to start creating views that return either HTML or JSON depending on the Accept headers from the request. Possibly other types, too, but those are the main ones. I also want to get multiple URLs routed to this view; the file extensions ".html" and ".json" help tell clients which types they should Accept when making their request, and I want to avoid the "?format=json" antipattern.
What's the correct, blessed way to do this in Django with a minimum of boilerplate or repeated code?
(Edit: Rephrase in order to better follow SO's community guidelines.)
I think a class-based view mixin (django 1.3+) is the easiest way to do this. All your views would inherit from a base class that contains logic to respond with the appropriate content.
I think I may not be seeing your big picture here but this is what I would do:
Have a html template that you render when html is requested and keep your json.dumps(content) for when json is requested. Seems to be obvious but I thought i should mention it anyway.
Set your URLs to send you "json" or 'html'. :
(r'^some/path/(?P<url_path>.*)\.(?P<extension>html|json)$', 'some.redirect.view'),
(r'^/(?P<extension>html|json)/AppName', include(MyApp)),
# etc etc
and your view:
def myRedirectView(request, url_path, extension):
view, args, kwargs = resolve("/" + extension + "/" + urlPath)
kwargs['request'] = request
return view(*args, **kwargs)
I know this is a bit vague because I haven't fully thought it through but its where I would start.
I have addressed this by creating a generic view class, based on Django's own generic.View class, that defines a decorator 'accept_types'. This modifies the view to which it is applied so that it returns None if the indicated content-type is not in the Accept header. Then, the get() method (which is called by the generic.View dispatcher) looks like this:
def get(self, request):
self.request = request # For clarity: generic.View does this anyway
resultdata = { 'result': data, etc. }
return (
self.render_uri_list(resultdata) or
self.render_html(resultdata) or
self.error(self.error406values())
)
The actual view renderers are decorated thus:
#ContentNegotiationView.accept_types(["text/uri-list"])
def render_uri_list(self, resultdata):
resp = HttpResponse(status=200, content_type="text/uri-list")
# use resp.write(...) to assemble rendered response body
return resp
#ContentNegotiationView.accept_types(["text/html", "application/html", "default_type"])
def render_html(self, resultdata):
template = loader.get_template('rovserver_home.html')
context = RequestContext(self.request, resultdata)
return HttpResponse(template.render(context))
The (one-off) generic view class that declares the decorator looks like this:
class ContentNegotiationView(generic.View):
"""
Generic view class with content negotiation decorators and generic error value methods
Note: generic.View dispatcher assigns HTTPRequest object to self.request.
"""
#staticmethod
def accept_types(types):
"""
Decorator to use associated function to render the indicated content types
"""
def decorator(func):
def guard(self, values):
accept_header = self.request.META.get('HTTP_ACCEPT',"default_type")
accept_types = [ a.split(';')[0].strip().lower()
for a in accept_header.split(',') ]
for t in types:
if t in accept_types:
return func(self, values)
return None
return guard
return decorator
(The parameter handling in the decorator should be generalized - this code works, but is still in development as I write this. The actual code is in GitHub at https://github.com/wf4ever/ro-manager/tree/develop/src/roverlay/rovweb/rovserver, but in due course should be separated to a separate package. HTH.)