We Keep Coding

Django domain + regex parameter not working on production machine

I currently have a django view with a fairly simple search function (takes user input, returns a list of objects). For usability, I'd like the option of passing search paramters via url like so:
www.example.com/search/mysearchstring
Where mysearchstring is the input to the search function. I'm using regex to validate any alphanumeric or underscore characters.
The problem I'm having is that while this works perfectly in my development environment, it breaks on the live machine.
Currently, I am using this exact same method (with different regex patterns) in other django views without any issues. This leads me to believe that either.
1) My regex is truly bad (more likely)
2) There is a difference in regex validators between environments (less likely)
The machine running this is using django 1.6 and python 2.7, which are slightly behind my development machine, but not significantly.
urls.py
SEARCH_REGEX = '(?P<pdom>\w*)?'
urlpatterns = patterns('',
....
url(r'^polls/search/' + SEARCH_REGEX, 'polls.views.search'),
...)
Which are passed to the view like this
views. py
def search(request, pdom):
...
When loading up the page, I get the following error:
ImproperlyConfigured: "^polls/search/(?P<pdom>\w*)?" is not a valid regular expression: nothing to repeat
I've been scratching my head over this one for a while. I've attempted to use a few different methods of encapsulation around the expression with no change in results. Would appreciate any insight!

I would change it to this:
SEARCH_REGEX = r'(?P<pdom>.+)$'
It's usually a good idea to use raw strings r'' for regular expressions in python.
The group will match the entire content of the search part of your url. I would handle query string validation in the view, instead of in the url regex. If someone tries to search polls/search/two+words, you should not return a 404, but instead a 400 status and a error message explaining that the search string was malformed.
Finally, you might want to follow the common convention for search urls. Which is to use a query parameter called q. So your url-pattern would be ^polls/search/$, and then you just handle the q in the view using something like this:
def search_page_view(request):
query_string = request.GET.get('q', '')

Flask-Babel localized strings within js

I'm pretty new to both Python and Flask (with Jinja2 as template engine) and I am not sure I am doing it the right way. I am using Flask-Babel extension to add i18n support to my web application. I want to get localized strings from my js code, for instance:
var helloWorld = gettext('Hello, world');
console.log(helloWorld); //should log a localized hello world message
For this, I configured babel (babel.cfg):
[python: **/**.py]
[jinja2: **/**.html]
extensions=jinja2.ext.autoescape,jinja2.ext.with_
[javascript: **/**.js]
encoding = utf-8
And its initialization is (imports omitted for simplicity):
#main Flask app
app = Flask(__name__)
#localization
babel = Babel(app)
LANGUAGES = {
'ca': 'Catalan',
'en': 'English',
'de': 'Deutsch',
'es': 'Español',
'fi': 'Finnish',
'it': 'Italian'
}
#babel.localeselector
def get_locale():
return request.accept_languages.best_match(LANGUAGES.keys())
#some more stuff...
Babel identifies that string when building the POT/PO language files, but it seems I can't access these localized strings from js code since gettext function is not defined. It seems like Jinja2 is ignoring this part.
Any hints?

I have finally found a solution, although I am not sure it is the way to go. The idea is to wrap the javascript code within an html template, which is interpretated by Jinja2 before it is rendered and apply a custom Jinja2 filter to get rid of some minor issues. I tried to keep js files separately but it did not work.
It seems that gettext function can be used like so:
var helloWorld = {{gettext('Hello, world')}};
But then, no quotes are inserted, and hence, js interpreter throws an error:
var helloWorld = Hello, world;
That's why I have finally applied a custom filter. A working example would be as follows.
hello_world.html:
<script type="text/javascript">
var x = {{gettext('Hello, world')|generate_string|safe}};
console.log(x); //logs the localized hello world message
</script>
app.py:
#Jinja2 filters
from jinja2 import evalcontextfilter, Markup
#Mind the hack! Babel does not work well within js code
#app.template_filter()
#evalcontextfilter
def generate_string(eval_ctx, localized_value):
if localized_value is None:
return ""
else:
return Markup("\"" + localized_value + "\"").unescape()
Hope this helps!

Providing translations in rendered JavaScript is a bit fragile. Also, I usually do not generate JavaScript using Jinja because it uses the same type of brackets and easily turns into mess when abused (it's always possible to have dynamic data and static JavaScript).
Alternative lightweight approach is to do the same JSON trick, but using data-attributes:
<div id="view-i18n" data-i18n='{{ view_i18n|tojson }}'> ... </div>
(NB: single quotes!)
But it's also good for a limited number of translations.
Perhaps, the most solid approach is to have the same translations in JavaScript as there are in the Flask app.
With a help of a utility called pojson it is possible to convert po-files to json (see https://github.com/ZeWaren/flask-i18n-example for an example) as part of the build process (for instance, right after making mo-files). Translations can be easily added to some unique enough global namespace variable by prepending the output of pojson with var some_unique_name = to have access to it. Or put the file under static into locale-specific file (eg static/view-es.json , static/view-fr.json, etc) and get it with ajax call.
Some things to consider though. You may need to break your translation domain into separate Python and Javascript by controlling babel extraction options if you really want to make JSON smaller. Also, having all translation strings in Javascript has security aspects. Maybe, you do not want to expose certain phrases only admins see to be open to other category of users. But then more translation domains are needed for different levels of access. Also header information may need to be removed to prevent leaking translator's emails and whatnot. This, of course, complicates the build process somewhat, but the more translation JavaScript side needs with time, the more automation pays itself off.

Add custom tokens in Jinja2 (e.g. %% somevar %%)

I'm making a Flask app for local development (on a Mac) of HTML templates that will eventually be served through ASP.NET.
For the purposes of local development, I want a way to replace the contents of .NET-style tokens with some data, meaning that Jinja2 would need to be able to recognize %% ... %% tokens in addition to the standard ones: {{ ... }}, <% ... %>, etc.
Everything I've found online pertains to the inclusion of some new functionality within the existing tags (e.g. {{ my_custom_function | arg1 arg2 }})
But what about defining a new pattern for tags altogether? Has anyone done this successfully? And will it require modification to the Jinja2 core?

As far as I know, you can use one set for block_start_string and block_end_string, as well as one set for variable_start_string and variable_end_string.
From jinja2/environment.py
`block_start_string`
The string marking the begin of a block. Defaults to ``'{%'``.
`block_end_string`
The string marking the end of a block. Defaults to ``'%}'``.
`variable_start_string`
The string marking the begin of a print statement.
Defaults to ``'{{'``.
`variable_end_string`
The string marking the end of a print statement. Defaults to
``'}}'``.
You can override these with environment variables. Though, I don't think there is a way to have multiple types recognized. For instance, you can't have {{ and <% both work, but with a little hackery you certainly could.

In addition to the correct answer maybe an example implementation to change the variable marker:
Add following header to the first line in the jinja2 template file.
#jinja2: variable_start_string: "#{" , variable_end_string: "}#"

i18n translation with a different domain in .py files

I'm a little confused on i18n translations in py files. I have a string where what needs to be translated is already in the plone domain in plone.pot so I want to specify the domain to be plone for that translation only. When I do the following, I get an error. And, this does get rendered in a page template so there's no need to call the translate function, is this correct?
raise ValueError(_(u'Some string', domain='plone'))
TypeError: __call__() got an unexpected keyword argument 'domain'
How should this be done? Thanks a lot!

You need to have two different message factories, i.e. where you do
from zope.i18nmessageid import MessageFactory
_ = MessageFactory('my.package')
you should be also instantiating the Plone one.
_p = MessageFactory('plone')
Then _p('Contributors') would give you the translated string.
Have a look here too.

Python: Need to replace a series of different substrings in HTML template with additional HTML or database results

Situation:
I am writing a basic templating system in Python/mod_python that reads in a main HTML template and replaces instances of ":value:" throughout the document with additional HTML or db results and then returns it as a view to the user.
I am not trying to replace all instances of 1 substring. Values can vary. There is a finite list of what's acceptable. It is not unlimited. The syntax for the values is [colon]value[colon]. Examples might be ":gallery: , :related: , :comments:". The replacement may be additional static HTML or a call to a function. The functions may vary as well.
Question:
What's the most efficient way to read in the main HTML file and replace the unknown combination of values with their appropriate replacement?
Thanks in advance for any thoughts/solutions,
c

There are dozens of templating options that already exist. Consider genshi, mako, jinja2, django templates, or more.
You'll find that you're reinventing the wheel with little/no benefit.

If you can't use an existing templating system for whatever reason, your problem seems best tackled with regular expressions:
import re
valre = re.compile(r':\w+:')
def dosub(correspvals, correspfuns, lastditch):
def f(value):
v = value.group()[1:-1]
if v in correspvals:
return correspvals[v]
if v in correspfuns:
return correspfuns[v]() # or whatever args you need
# what if a value has neither a corresponding value to
# substitute, NOR a function to call? Whatever...:
return lastditch(v)
return f
replacer = dosub(adict, another, somefun)
thehtml = valre.sub(replacer, thehtml)
Basically you'll need two dictionaries (one mapping values to corresponding values, another mapping values to corresponding functions to be called) and a function to be called as a last-ditch attempt for values that can't be found in either dictionary; the code above shows you how to put these things together (I'm using a closure, a class would of course do just as well) and how to apply them for the required replacement task.

This is probably a job for a templating engine and for Python there are a number of choices. In this stackoveflow question people have listed their favourites and some helpfully explain why: What is your single favorite Python templating engine?