Let's say I've created a (hopefully) reusable app, fooapp:
urls.py
urls('^(?P<userid>\d+)/$', views.show_foo),
and fooapp's views.py:
def show_foo(request, userid):
usr = shortcuts.get_object_or_404(User, pk=userid)
... display a users' foo ...
return render_to_response(...)
Since it's a reusable app, it doesn't specify any access control (e.g. #login_required).
In the site/project urls.py, the app is included:
urls('^foo/', include('fooapp.urls')),
How/where can I specify that in this site only staff members should be granted access to see a user's foo?
How about if, in addition to staff members, users should be able to view their own foo (login_required + request.user.id == userid)?
I didn't find any obvious parameters to include..
Note: this has to do with access control, not permissions, i.e. require_staff checks User.is_staff, login_required checks if request.user is logged in, and user-viewing-their-own-page is described above. This question is in regards to how a site can specify access control for a reusable app.
Well, I've found a way that works by iterating over the patterns returned by Django's include:
from django.core.urlresolvers import RegexURLPattern, RegexURLResolver
def urlpatterns_iterator(patterns):
"""Recursively iterate through `pattern`s.
"""
_patterns = patterns[:] # create a copy
while _patterns:
cur = _patterns.pop()
if isinstance(cur, RegexURLPattern):
yield cur
elif isinstance(cur, RegexURLResolver):
_patterns += cur.url_patterns
else:
raise ValueError("I don't know how to handle %r." % cur)
def decorate(fn, (urlconf_module, app_name, namespace)):
"""Iterate through all the urls reachable from the call to include and
wrap the views in `fn` (which should most likely be a decorator).
(the second argument is the return value of Django's `include`).
"""
# if the include has a list of patterns, ie.: url(<regex>, include([ url(..), url(..) ]))
# then urlconf_module doesn't have 'urlpatterns' (since it's already a list).
patterns = getattr(urlconf_module, 'urlpatterns', urlconf_module)
for pattern in urlpatterns_iterator(patterns):
# the .callback property will set ._callback potentially from a string representing the path to the view.
if pattern.callback:
# the .callback property doesn't have a setter, so access ._callback directly
pattern._callback = fn(pattern._callback)
return urlconf_module, app_name, namespace
this is then used in the site/project's urls.py, so instead of:
urls('^foo/', include('fooapp.urls')),
one would do:
from django.contrib.admin.views.decorators import staff_member_required as _staff_reqd
def staff_member_required(patterns): # make an include decorator with a familiar name
return decorate(_staff_reqd, patterns)
...
urls('^foo/', staff_member_required(include('fooapp.urls'))),
Related
I have a catchall route that is catching my request before my call to add_route within an action created with a custom directive. How can I prevent that from happening? Ie. manually place the catchall route at the end of the route matching process or put my action's add_route at the front of the route matching process?
I tried swapping the order of the includes between admin/routes.py and routes.py but that didn't seem to have an affect. A quick solution (and probably good idea thinking about it now) is to filter admin from the pattern in the catchall. But I feel like this is going to come up again where I cannot do that so I'm asking this question.
__init__.py
def main(global_config, **settings):
#...
config.include('.directives')
#...
config.include(".routes")
#...
for k in (".customer", ".admin"):
config.scan(k) # this picks up the admin/admin_routes.py
return config.make_wsgi_app()
admin/routes.py
def includeme(config):
config.add_dispatched_route(
"admin-plants-edit",
"/admin/plants/{id}",
'plant',
)
routes.py
def includeme(config):
#...
config.add_route("page-view", "/*url")
directives.py
from pyramid.config import PHASE0_CONFIG
from pyramid.httpexceptions import HTTPNotFound
def includeme(config):
def add_dispatched_route(config, route_name, route_pattern, dispatch_with):
def route_factory(request):
api = request.find_service(name=dispatch_with)
obj = api.by_id(request.matchdict["id"])
if not obj:
raise HTTPNotFound()
return obj
def route_pregenerator(request, elements, kw):
api = request.find_service(name=dispatch_with)
try:
obj = kw.pop(api.get_pregenerator_kw())
except KeyError:
pass
else:
kw["id"] = obj.id
return elements, kw
def register():
config.add_route(route_name, route_pattern, factory=route_factory, pregenerator=route_pregenerator)
config.action(('dispatched_route', route_name), register, order=PHASE0_CONFIG)
config.add_directive('add_dispatched_route', add_dispatched_route)
The way the phases work in Pyramid doesn't allow you to re-order actions within a phase. Basically imagine that every call to config.add_route gets appended to a list, and then later that list is iterated in order to add the routes. You cannot inject a call to config.add_route before another call to it.
However if you wrap every call to add_route in your own action, then you have an opportunity to defer everything, order things, and then call config.add_route with each route in the order you wish.
I have a multi-tenant django project, where there is a single django instance serving multiple tenants from separate databases. It is fully operational in production.
I have template caching enabled, which was fine, until I implemented a per-tenant template loader, which essentially allows tenantXYZ to define a customised copy of /some_app/templates/some_template.html in a different directory. So if the url starts with tenantXYZ, the page loads the template from that custom directory instead of the base version of /some_app/templates/some_template.html, allowing very powerful per-tenant customisation.
The problem is template caching (which we only enable on production) returns a cached copy of /some_app/templates/some_template.html, therefore overriding the loader's functionality.
My plan is to subclass Django's django.template.loaders.cached.Loader to make it use a separate cache per tenant (which is what multi-tenant tools do with the normal cache anyway, see django-db-multitenant section on cache KEY_FUNCTION)
https://github.com/django/django/blob/master/django/template/loaders/cached.py
But I am struggling to understand exactly how the cached.Loader works. It seems that its 'cache' is just a dictionary (in which case I can change it to use a dictionary per tenant to keep the caches separate) and infers that there is only one instance of the loader running.
Are those assumptions correct, or does it actually use the CACHE settings (e.g. in a base class or elsewhere I may not be aware of)
Is there anything else I am missing which I need to take into account?
Here is Django's cached.Loader source code:
"""
Wrapper class that takes a list of template loaders as an argument and attempts
to load templates from them in order, caching the result.
"""
import hashlib
from django.template import TemplateDoesNotExist
from django.template.backends.django import copy_exception
from .base import Loader as BaseLoader
class Loader(BaseLoader):
def __init__(self, engine, loaders):
self.template_cache = {}
self.get_template_cache = {}
self.loaders = engine.get_template_loaders(loaders)
super().__init__(engine)
def get_contents(self, origin):
return origin.loader.get_contents(origin)
def get_template(self, template_name, skip=None):
"""
Perform the caching that gives this loader its name. Often many of the
templates attempted will be missing, so memory use is of concern here.
To keep it in check, caching behavior is a little complicated when a
template is not found. See ticket #26306 for more details.
With template debugging disabled, cache the TemplateDoesNotExist class
for every missing template and raise a new instance of it after
fetching it from the cache.
With template debugging enabled, a unique TemplateDoesNotExist object
is cached for each missing template to preserve debug data. When
raising an exception, Python sets __traceback__, __context__, and
__cause__ attributes on it. Those attributes can contain references to
all sorts of objects up the call chain and caching them creates a
memory leak. Thus, unraised copies of the exceptions are cached and
copies of those copies are raised after they're fetched from the cache.
"""
key = self.cache_key(template_name, skip)
cached = self.get_template_cache.get(key)
if cached:
if isinstance(cached, type) and issubclass(cached, TemplateDoesNotExist):
raise cached(template_name)
elif isinstance(cached, TemplateDoesNotExist):
raise copy_exception(cached)
return cached
try:
template = super().get_template(template_name, skip)
except TemplateDoesNotExist as e:
self.get_template_cache[key] = copy_exception(e) if self.engine.debug else TemplateDoesNotExist
raise
else:
self.get_template_cache[key] = template
return template
def get_template_sources(self, template_name):
for loader in self.loaders:
yield from loader.get_template_sources(template_name)
def cache_key(self, template_name, skip=None):
"""
Generate a cache key for the template name, dirs, and skip.
If skip is provided, only origins that match template_name are included
in the cache key. This ensures each template is only parsed and cached
once if contained in different extend chains like:
x -> a -> a
y -> a -> a
z -> a -> a
"""
dirs_prefix = ''
skip_prefix = ''
if skip:
matching = [origin.name for origin in skip if origin.template_name == template_name]
if matching:
skip_prefix = self.generate_hash(matching)
return '-'.join(s for s in (str(template_name), skip_prefix, dirs_prefix) if s)
def generate_hash(self, values):
return hashlib.sha1('|'.join(values).encode()).hexdigest()
def reset(self):
"Empty the template cache."
self.template_cache.clear()
self.get_template_cache.clear()
Here is what I want to do:
Retrieve a list of all objects in all my django apps that inherit from forma.Form
Find all CharField defined on the form
Verify that each of these fields has a max_length kwarg specified
My goal is the write a unit test to fail if a form exists within our system that does not have a max length specified.
how would I do this?
I found a way to use reflection style code to retrieve all of the objects inheriting from the models.Form class, see test below:
class TestAllFormsMeetBasicBoundaryTests(TestCase):
def all_subclasses(self, cls):
return cls.__subclasses__() + [g for s in cls.__subclasses__()
for g in self.all_subclasses(s)]
# this is teh awesome-sause! dynamically finding objects in the project FTW!
def test_all_char_fields_contain_max_length_value(self):
from django.apps import apps
import django.forms
import importlib
log.info('loading all forms modules from apps')
at_least_one_module_found = False
for app_name in settings.PROJECT_APPS: #this is a list of in project apps
app = apps.get_app_config(app_name)
try:
importlib.import_module('{0}.forms'.format(app.label))
log.info('forms loaded from {0}'.format(app.label))
at_least_one_module_found = True
except ImportError as e:
pass
self.assertTrue(at_least_one_module_found, 'No forms modules were found in any of the apps. this should never be true!')
all_forms = self.all_subclasses(django.forms.Form)
messages = []
for form in all_forms:
for key in form.declared_fields.keys():
field = form.declared_fields[key]
if isinstance(field, django.forms.CharField) and form.__module__.partition('.')[0] in settings.PROJECT_APPS and field.max_length is None:
messages.append('{0}.{1}.{2} is missing a max_length value.'.format(form.__module__, form.__name__, key))
pass
self.assertEquals(0, len(messages),
'CharFields must always have a max_length attribute specified. please correct the following:\n--{0}'
.format('\n--'.join(messages)))
I'm writing a admin website which control several websites with same program and database schema but different content. The URL I designed like this:
http://example.com/site A list of all sites which under control
http://example.com/site/{id} A brief overview of select site with ID id
http://example.com/site/{id}/user User list of target site
http://example.com/site/{id}/item A list of items sold on target site
http://example.com/site/{id}/item/{iid} Item detailed information
# ...... something similar
As you can see, nearly all URL are need the site_id. And in almost all views, I have to do some common jobs like query Site model against database with the site_id. Also, I have to pass site_id whenever I invoke request.route_path.
So... is there anyway for me to make my life easier?
It might be useful for you to use a hybrid approach to get the site loaded.
def groupfinder(userid, request):
user = request.db.query(User).filter_by(id=userid).first()
if user is not None:
# somehow get the list of sites they are members
sites = user.allowed_sites
return ['site:%d' % s.id for s in sites]
class SiteFactory(object):
def __init__(self, request):
self.request = request
def __getitem__(self, key):
site = self.request.db.query(Site).filter_by(id=key).first()
if site is None:
raise KeyError
site.__parent__ = self
site.__name__ = key
site.__acl__ = [
(Allow, 'site:%d' % site.id, 'view'),
]
return site
We'll use the groupfinder to map users to principals. We've chosen here to only map them to the sites which they have a membership within. Our simple traversal only requires a root object. It updates the loaded site with an __acl__ that uses the same principals the groupfinder is setup to create.
You'll need to setup the request.db given patterns in the Pyramid Cookbook.
def site_pregenerator(request, elements, kw):
# request.route_url(route_name, *elements, **kw)
from pyramid.traversal import find_interface
# we use find_interface in case we improve our hybrid traversal process
# to take us deeper into the hierarchy, where Site might be context.__parent__
site = find_interface(request.context, Site)
if site is not None:
kw['site_id'] = site.id
return elements, kw
Pregenerator can find the site_id and add it to URLs for you automatically.
def add_site_route(config, name, pattern, **kw):
kw['traverse'] = '/{site_id}'
kw['factory'] = SiteFactory
kw['pregenerator'] = site_pregenerator
if pattern.startswith('/'):
pattern = pattern[1:]
config.add_route(name, '/site/{site_id}/' + pattern, **kw)
def main(global_conf, **settings):
config = Configurator(settings=settings)
authn_policy = AuthTktAuthenticationPolicy('seekrit', callback=groupfinder)
config.set_authentication_policy(authn_policy)
config.set_authorization_policy(ACLAuthorizationPolicy())
config.add_directive(add_site_route, 'add_site_route')
config.include(site_routes)
config.scan()
return config.make_wsgi_app()
def site_routes(config):
config.add_site_route('site_users', '/user')
config.add_site_route('site_items', '/items')
We setup our application here. We also moved the routes into an includable function which can allow us to more easily test the routes.
#view_config(route_name='site_users', permission='view')
def users_view(request):
site = request.context
Our views are then simplified. They are only invoked if the user has permission to access the site, and the site object is already loaded for us.
Hybrid Traversal
A custom directive add_site_route is added to enhance your config object with a wrapper around add_route which will automatically add traversal support to the route. When that route is matched, it will take the {site_id} placeholder from the route pattern and use that as your traversal path (/{site_id} is path we define based on how our traversal tree is structured).
Traversal happens on the path /{site_id} where the first step is finding the root of the tree (/). The route is setup to perform traversal using the SiteFactory as the root of the traversal path. This class is instantiated as the root, and the __getitem__ is invoked with the key which is the next segment in the path ({site_id}). We then find a site object matching that key and load it if possible. The site object is then updated with a __parent__ and __name__ to allow the find_interface to work. It is also enhanced with an __acl__ providing permissions mentioned later.
Pregenerator
Each route is updated with a pregenerator that attempts to find the instance of Site in the traversal hierarchy for a request. This could fail if the current request did not resolve to a site-based URL. The pregenerator then updates the keywords sent to route_url with the site id.
Authentication
The example shows how you can have an authentication policy which maps a user into principals indicating that this user is in the "site:" group. The site (request.context) is then updated to have an ACL saying that if site.id == 1 someone in the "site:1" group should have the "view" permission. The users_view is then updated to require the "view" permission. This will raise an HTTPForbidden exception if the user is denied access to the view. You can write an exception view to conditionally translate this into a 404 if you want.
The purpose of my answer is just to show how a hybrid approach can make your views a little nicer by handling common parts of a URL in the background. HTH.
For the views, you could use a class so that common jobs can be carried out in the __init__ method (docs):
from pyramid.view import view_config
class SiteView(object):
def __init__(self, request):
self.request = request
self.id = self.request.matchdict['id']
# Do any common jobs here
#view_config(route_name='site_overview')
def site_overview(self):
# ...
#view_config(route_name='site_users')
def site_users(self):
# ...
def route_site_url(self, name, **kw):
return self.request.route_url(name, id=self.id, **kw)
And you could use a route prefix to handle the URLs (docs). Not sure whether this would be helpful for your situation or not.
from pyramid.config import Configurator
def site_include(config):
config.add_route('site_overview', '')
config.add_route('site_users', '/user')
config.add_route('site_items', '/item')
# ...
def main(global_config, **settings):
config = Configurator()
config.include(site_include, route_prefix='/site/{id}')
Pyramid documentation shows us how to use i18n inside views (and templates as well). But how to does one use it outside of views and templates where we have no access to current request (for example, in forms and models)?
#Michael said to pass request to models and forms. But is it right? I mean if form fields defines before __init__() method calls, the same with models. They don't see any parameters from views...
In Pylons we could simply use get_lang() and set_lang() and define preferable language in parent controller and then use ugettext() and ungettext() in any place we want without calling it from request directly every possible time (in views).
How to do that in Pyramid? Note that the language must be set from user's settings (session, cookies, db, etc).
My solution is to create the form class when it's needed with localizer as parameter. For example
forms.py
class FormFactory(object):
def __init__(self, localizer):
self.localizer = localizer
_ = self.localizer
self.required_msg = _(u'This field is required.')
self.invalid_email_msg = _(u'Invalid email address.')
self.password_not_match_msg = _(u'Password must match')
def make_contact_form(self):
_ = self.localizer
class ContactForm(Form):
email = TextField(_(u'Email address'), [
validators.Required(self.required_msg),
validators.Email(self.invalid_email_msg)
])
content = TextAreaField(_(u'Content'), [
validators.Required(self.required_msg)
])
return ContactForm
When you need to use the form
#view_config(route_name='front_pages.contact_us',
renderer='myweb:templates/front_pages/contact_us.genshi')
def contact_us(request):
"""Display contact us form or send mail
"""
_ = get_localizer(request)
factory = FormFactory(_)
ContactForm = factory.make_contact_form()
form = ContactForm(request.params)
return dict(form=form)
As you can see, we get the localizer in the view, and pass it to the FormFactory, then create a form with that factory. By doing that, all messages in the form was replaced with current locale language.
Likewise, you can do the same with model.
Have you found pyramid.18n.get_localizer yet?
http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch/narr/i18n.html#using-a-localizer
Actually I had this very same problem. What I ended up doing was to see how the default locale negotiator works - it looks for a LOCALE property on the given request object. So just use a dummy to create the localizer. You may cache this value too, if you want
def my_get_localizer(locale=None):
request = Request({})
request._LOCALE_ = locale
return get_localizer(request)
Alternatively, join the irc channel #pyramid # freenode and pester the guys enough there to split the functionality of get_localizer in 2 separate documented functions (get_localizer and get_localizer_for_locale_name) for us to enjoy ;)
Also, notice that Pyramid TranslationStrings are lazy, so you can translate them as late as you want, e.g.
class MyModel(Base):
description = TranslationString("My model number ${number}")
...
def view(request):
m = MyModel()
localizer = get_localizer(request)
description = localizer.translate(m.description, mapping={'number': 1})
Sidenote: pylons' i18n was the worst can of worms I had opened in ages. The set_lang, get_lang hack was really awful, and pain in the ass as we needed to send emails to users in their native languages and then tried to restore the language back... also, it was IMPOSSIBLE to translate anything outside of a request in a pylons program, as a translator or the registry did not exist then.
You can make a localizer, and then translate a template accordingly.
When making the localizer, you can pass the lang you want (whether you have it from db or else). Hope it can help.
For the sake of clarity, I will set it as 'fr' below
from pyramid.i18n import make_localizer, TranslationStringFactory
from mako.template import Template
from mako.lookup import TemplateLookup
import os
absolute_path = os.path.dirname(os.path.realpath(__file__))
tsf = TranslationStringFactory('your_domain')
mako_lookup = TemplateLookup(directories=['/'])
template = Template(filename=template_path, lookup=mako_lookup)
localizer = make_localizer("fr", [absolute_path + '/../locale/'])
def auto_translate(*args, **kwargs):
return localizer.translate(tsf(*args, **kwargs))
# Pass _ pointer (translate function) to the context
_ = auto_translate
context.update({
"_": _
})
html = template.render(**context)
EDIT
You can also put this logic into a small function
def get_translator(lang):
"""
Useful when need to translate outside of queries (no pointer to request)
:param lang:
:return:
"""
localizer = make_localizer(lang, [absolute_path + '/../locale/'])
def auto_translate(*args, **kwargs):
return localizer.translate(tsf(*args, **kwargs))
_ = auto_translate
return _