We Keep Coding

Django domain + regex parameter not working on production machine

I currently have a django view with a fairly simple search function (takes user input, returns a list of objects). For usability, I'd like the option of passing search paramters via url like so:
www.example.com/search/mysearchstring
Where mysearchstring is the input to the search function. I'm using regex to validate any alphanumeric or underscore characters.
The problem I'm having is that while this works perfectly in my development environment, it breaks on the live machine.
Currently, I am using this exact same method (with different regex patterns) in other django views without any issues. This leads me to believe that either.
1) My regex is truly bad (more likely)
2) There is a difference in regex validators between environments (less likely)
The machine running this is using django 1.6 and python 2.7, which are slightly behind my development machine, but not significantly.
urls.py
SEARCH_REGEX = '(?P<pdom>\w*)?'
urlpatterns = patterns('',
....
url(r'^polls/search/' + SEARCH_REGEX, 'polls.views.search'),
...)
Which are passed to the view like this
views. py
def search(request, pdom):
...
When loading up the page, I get the following error:
ImproperlyConfigured: "^polls/search/(?P<pdom>\w*)?" is not a valid regular expression: nothing to repeat
I've been scratching my head over this one for a while. I've attempted to use a few different methods of encapsulation around the expression with no change in results. Would appreciate any insight!

I would change it to this:
SEARCH_REGEX = r'(?P<pdom>.+)$'
It's usually a good idea to use raw strings r'' for regular expressions in python.
The group will match the entire content of the search part of your url. I would handle query string validation in the view, instead of in the url regex. If someone tries to search polls/search/two+words, you should not return a 404, but instead a 400 status and a error message explaining that the search string was malformed.
Finally, you might want to follow the common convention for search urls. Which is to use a query parameter called q. So your url-pattern would be ^polls/search/$, and then you just handle the q in the view using something like this:
def search_page_view(request):
query_string = request.GET.get('q', '')

Flask-Babel localized strings within js

I'm pretty new to both Python and Flask (with Jinja2 as template engine) and I am not sure I am doing it the right way. I am using Flask-Babel extension to add i18n support to my web application. I want to get localized strings from my js code, for instance:
var helloWorld = gettext('Hello, world');
console.log(helloWorld); //should log a localized hello world message
For this, I configured babel (babel.cfg):
[python: **/**.py]
[jinja2: **/**.html]
extensions=jinja2.ext.autoescape,jinja2.ext.with_
[javascript: **/**.js]
encoding = utf-8
And its initialization is (imports omitted for simplicity):
#main Flask app
app = Flask(__name__)
#localization
babel = Babel(app)
LANGUAGES = {
'ca': 'Catalan',
'en': 'English',
'de': 'Deutsch',
'es': 'Español',
'fi': 'Finnish',
'it': 'Italian'
}
#babel.localeselector
def get_locale():
return request.accept_languages.best_match(LANGUAGES.keys())
#some more stuff...
Babel identifies that string when building the POT/PO language files, but it seems I can't access these localized strings from js code since gettext function is not defined. It seems like Jinja2 is ignoring this part.
Any hints?

I have finally found a solution, although I am not sure it is the way to go. The idea is to wrap the javascript code within an html template, which is interpretated by Jinja2 before it is rendered and apply a custom Jinja2 filter to get rid of some minor issues. I tried to keep js files separately but it did not work.
It seems that gettext function can be used like so:
var helloWorld = {{gettext('Hello, world')}};
But then, no quotes are inserted, and hence, js interpreter throws an error:
var helloWorld = Hello, world;
That's why I have finally applied a custom filter. A working example would be as follows.
hello_world.html:
<script type="text/javascript">
var x = {{gettext('Hello, world')|generate_string|safe}};
console.log(x); //logs the localized hello world message
</script>
app.py:
#Jinja2 filters
from jinja2 import evalcontextfilter, Markup
#Mind the hack! Babel does not work well within js code
#app.template_filter()
#evalcontextfilter
def generate_string(eval_ctx, localized_value):
if localized_value is None:
return ""
else:
return Markup("\"" + localized_value + "\"").unescape()
Hope this helps!

Providing translations in rendered JavaScript is a bit fragile. Also, I usually do not generate JavaScript using Jinja because it uses the same type of brackets and easily turns into mess when abused (it's always possible to have dynamic data and static JavaScript).
Alternative lightweight approach is to do the same JSON trick, but using data-attributes:
<div id="view-i18n" data-i18n='{{ view_i18n|tojson }}'> ... </div>
(NB: single quotes!)
But it's also good for a limited number of translations.
Perhaps, the most solid approach is to have the same translations in JavaScript as there are in the Flask app.
With a help of a utility called pojson it is possible to convert po-files to json (see https://github.com/ZeWaren/flask-i18n-example for an example) as part of the build process (for instance, right after making mo-files). Translations can be easily added to some unique enough global namespace variable by prepending the output of pojson with var some_unique_name = to have access to it. Or put the file under static into locale-specific file (eg static/view-es.json , static/view-fr.json, etc) and get it with ajax call.
Some things to consider though. You may need to break your translation domain into separate Python and Javascript by controlling babel extraction options if you really want to make JSON smaller. Also, having all translation strings in Javascript has security aspects. Maybe, you do not want to expose certain phrases only admins see to be open to other category of users. But then more translation domains are needed for different levels of access. Also header information may need to be removed to prevent leaking translator's emails and whatnot. This, of course, complicates the build process somewhat, but the more translation JavaScript side needs with time, the more automation pays itself off.

How can I get a list of all the strings that Babel knows about?

I have an application in which the main strings are in English and then various translations are made in various .po/.mo files, as usual (using Flask and Flask-Babel). Is it possible to get a list of all the English strings somewhere within my Python code? Specifically, I'd like to have an admin interface on the website which lets someone log in and choose an arbitrary phrase to be used in a certain place without having to poke at actual Python code or .po/.mo files. This phrase might change over time but needs to be translated, so it needs to be something Babel knows about.
I do have access to the actual .pot file, so I could just parse that, but I was hoping for a cleaner method if possible.

You can use polib for this.
This section of the documentation shows examples of how to iterate over the contents of a .po file. Here is one taken from that page:
import polib
po = polib.pofile('path/to/catalog.po')
for entry in po:
print entry.msgid, entry.msgstr

If you alredy use babel you can get all items from po file:
from babel.messages.pofile import read_po
catalog = read_po(open(full_file_name))
for message in catalog:
print message.id, message.string
See http://babel.edgewall.org/browser/trunk/babel/messages/pofile.py.
You alredy can try get items from mo file:
from babel.messages.mofile import read_mo
catalog = read_po(open(full_file_name))
for message in catalog:
print message.id, message.string
But when I try use it last time it's not was availible. See http://babel.edgewall.org/browser/trunk/babel/messages/mofile.py.
You can use polib as #Miguel wrote.

Template tags successfully translated but ugettext_lazy() in utils not translating

I successfully managed to get trans and blocktrans to translate text.
However...
I have a function defined in utils.py that returns a dictionary containing strings of which some of them I need converted to the current language.
EDIT: *This is a utils.py that I've created in my project directory that are called by views to perform certain auxillary functions on a dict and then return the updated dict
I have done something like this:
try:
path = default_storage.save(customercode + '/' + file.name, ContentFile(file.read()))
results['status'] = 'success'
results['message'] = _(u'Your file has been successfully uploaded')
except Exception as e:
results['status'] = 'error'
results['message'] = _(u'There was an error uploading your file: ') + str(e)
return results
I have also added from django.utils.translation import ugettext_lazy as _ to the top of this utils.py file..
And this "results" dictionary is used in one of my views wherein the whole dictionary after some further processing is passed as a context variable to the template.
I have correctly set the translation in the .po file. All other template tags translate perfectly. Only the above mentioned strings do not translate.
Any help would be greatly appreciated.
UPDATE: I tried the same process in a forms filed label and it translated just fine. It's only the aforementioned areas where it won't translate!
PS: This is my first question on stackoverflow. I apologize in advance if I've made mistakes asking this question.

Just as I expected,
I was making a very silly mistake. I accessed the incorrect variable in the template while trying to print the translated value! >_<
But I guess one thing I've learnt is staying away from a problem and then coming back to it after a long time helps. You've got to learn the problem again and that sometimes helps you find silly bugs like these!

Django i18n: Common causes for translations not appearing

I am making a multilingual Django website. I created a messages file, populated and compiled it. I checked the site (the admin in this case,) in my wanted language (Hebrew) and most phrases appear in Hebrew like they should, but some don't. I checked the source and these still appear as _('Whatever') like they should, also they are translated on the messages file, and yes, I remembered to do compilemessages.
What are some common causes for translations not to appear like that?

Maybe the translated strings are marked as fuzzy?

Just got hit by one. I had the locale/ directory in the root of my project, but by default Django looks for translations in the INSTALLED_APPS directories, and in the default translations. So it didn't find the translations I added. But some of my strings were in the default translations that come with Django (eg "Search") so a few strings were translated, which confused me.
To add the directory where my translations were to the list of places that Django will look for translations, I had to set the LOCALE_PATHS setting. So in my case, where the locale/ directory and settings.py were both in the root of the django project I could put the following in settings.py:
from os import path
LOCALE_PATHS = (
path.join(path.abspath(path.dirname(__file__)), 'locale'),
)

I'm trying to provide a complete check list:
In settings.py, are USE_I18N, USE_L10N, LANGUAGE_CODE, and LOCALE_PATHS set properly?
See this list for all allowed values of language identifiers. Note that Simplified Chinese is specified by zh-hans, not zh-cn.
In settings.py, is django.middleware.locale.LocaleMiddleware included in MIDDLEWARE in correct order?
Have you (re)run django-admin makemessages -l <locale-name> with the correct local name, from the correct place?
You can see an incomplete list of allowed locale name by running ls your/path/to/python/site-packages/django/conf/locale/ on your machine, or by taking a look at the source code
Note that you have to use _ rather than - here. For example, to specify simplified Chinese, execute django-admin makemessages -l zh_Hans, not zh_CN or zh_hans or zh-Hans or anything else.
Have you removed all fuzzy tags in your PO file(s)?
Have you (re)compiled the OP file(s) with django-admin compilemessages?
Have you restarted the web server?
Additional notes:
If some of your translation is overridden by Django's default translation, use contextual markers to bypass it. For example,
models.py
first_name = models.CharField(
pgettext_lazy('override default', 'first name'),
max_length=30
)
last_name = models.CharField(
pgettext_lazy('override default', 'last name'),
max_length=150
)
django.po
#: models.py:51
msgctxt "override default"
msgid "first name"
msgstr "姓"
#: models.py:55
msgctxt "override default"
msgid "last name"
msgstr "名"
and you'll see 姓, 名 instead of the default 姓氏, 名字.

I was having this problem with my project right now. I had the variable LANGUAGES on settings.py set this way:
LANGUAGES = (
('en', _('English')),
('pt-br', _('Brazilian Portuguese')),
)
And a folder structure with a locale folder, and a subfolder pt-br inside. Turns out that my translations weren't loading. The LANGUAGES variable follows the pt-br pattern, and the folders must be on pt_BR pattern. At least that's the only way it's working here!

Hi just attach some fixes I had to do in the past:
Restart the webserver!
In the setting file
- USE_I18N = True is needed
django.middleware.locale.LocaleMiddleware among middleware modules (but this is not your case for sure as long as Django will not care about your local at all)
django.core.context_processors.i18n among TEMPLATE_CONTEXT_PROCESSORS
for sure I had other issues related to translations but I don't remember them all... hope this can help!

A possible cause is Lazy Translation.
In example, in views.py you should use ugettext:
from django.utils.translation import ugettext as _
But in models.py, you should use ugettext_lazy:
from django.utils.translation import ugettext_lazy as _

Another cause can be a wrong directory structure.
Read well the manage command's error message about which directory to create before running the makemassages command for the app translation. (It must be locale for an app, not conf/locale.) Note that the management commands work fine even with the wrong directory structure.

I noticed that when I had % in my text, the translated text was not being used. There may be other characters that can cause this problem. I fixed the problem by escaping the % as %%.

My answer here, all my translations were working except for 2 DateFields that I was using in a ModelForm.
Turns out that I had a widget in my forms.py that was not working well with the translations.
I just removed it for now so I can enjoy Xmas =D