I have an i18nized Python Django application. It currently uses two languages; German (DE) and French (FR).
I have all my keys (.po-/.mo-files) translated and ready in German, however for French, some are missing.
In the Django settings I specified 'de' as the LANGUAGE_CODE.
I can switch from one language to the other just fine without issues. The routing works fine and every other feature I need is handled by the Django Middleware.
However, in the current scenario when I switch from German to French, all the keys which are missing in French, just fallback to the German values. But I would like them to just default to their keys.
E.g.
Current Scenario
Sortiment (available in French) -> Assortiment
Gratis Lieferung (not available in French) -> Gratis Lieferung
Expected Scenario
Sortiment (available in French) -> Assortiment
Gratis Lieferung (not available in French) -> free.shipping.info
What would be a clean solution to solve this? I couldn't find anything in the Django documentation. I'd like to solve this without using additional plugins.
And one solution I could come up with, would be to just add all the missing keys in the french translations and have their values also be their keys but this doesn't feel right.
E.g. in django.po
msgid "searchsuggest.placeholder"
msgstr "searchsuggest.placeholder"
Another possible solution is to not set the LANGUAGE_CODE in the settings.py which works as I would want it for french, e.g. I go to mypage.com/fr/ and all my translated keys are shown the correct corresponding value while untranslated keys are just shown as keys (See 'Expected Scenario'). But when I do this, the German version only shows the keys, no values. E.g. I go to mypage.com/ (German should be implicit) and this is what I see:
assortment.menu.title
free.shipping.info
More information
My urls.py
urlpatterns = i18n_patterns(
# app endpoints
url(r'^$', home, name='home'),
url(r'^cart', include('app.cart.urls')),
url(r'^', include('app.infra.url.urls')),
prefix_default_language=False,
)
My settings.py
TIME_ZONE = 'UTC'
LANGUAGE_CODE = 'de'
LANGUAGES = [
('de', _('German')),
('fr', _('French')),
]
LOCALE_PATHS = [
../a/dir
]
USE_I18N = True
USE_L10N = True
USE_TZ = True
# And somewhere I use this
'django.middleware.locale.LocaleMiddleware',
My jinja template global translation function:
from django.utils.translation import ugettext as _
from jinja2.ext import Extension
class ViewExtension(Extension):
def __init__(self, environment):
super(ViewExtension, self).__init__(environment)
environment.globals['trans'] = trans
# defaults back to german if not found
def trans(translation_key, **kwargs):
translation = _(translation_key) % kwargs
if translation == translation_key:
# this only happens if my LANGUAGE_CODE is not set
translation_logger.warning(f'Missing translation key "{translation_key}".')
return translation
I'm still happy if anyone has a "proper" solution for this but I ended up solving it with a shell script:
#!/bin/bash
# A simple script to make sure that all translation files contain all the msgids.
# If a msgid previously didn't exist in a file, it will be created with the msgstr set to the same as the msgid.
SCRIPTPATH=`dirname $0`
MSGIDS=`find $SCRIPTPATH -name "*.po" -type f -print0 | xargs grep -h msgid | sort | uniq | awk '{print $2}'`
find $SCRIPTPATH -name "*.po" -type f | while read FILE; do
current_msgids=`grep -h msgid $FILE | awk '{print $2}'`
for msg in $MSGIDS; do
[[ $current_msgids =~ (^|[[:space:]])"$msg"($|[[:space:]]) ]] || printf "\nmsgid $msg\nmsgstr $msg\n" >> $FILE
done
done
I just included this script before running compilemessages in our Makefile.
Related
I'm thinking of creating a multilingual web page with fastapi-babel.
I have configured according to the documentation.
The translation from English to French was successful.
However, I created a .po file for another language, translated it, compiled it, but the translated text does not apply.
from fastapi_babel import _
from fastapi_babel.middleware import InternationalizationMiddleware as I18nMiddleware
from fastapi_babel import Babel
from fastapi_babel import BabelConfigs
configs = BabelConfigs(
ROOT_DIR=__file__,
BABEL_DEFAULT_LOCALE="en",
BABEL_TRANSLATION_DIRECTORY="lang",
)
logger.info(f"configs: {configs.__dict__}")
babel = babel(configs)
babel.install_jinja(templates)
app.add_middleware(I18nMiddleware, babel=babel)
#app.get("/items/{id}", response_class=HTMLResponse)
async def read_item(request: Request, id: str):
babel.locale = "en"
logger.info(_("Hello World"))
babel. locale = "fa"
logger.info(_("Hello World"))
babel.locale = "ja"
logger.info(_("Hello World"))
return templates.TemplateResponse('item.html', {'request': request, 'id': id})
Above, the result will be:
INFO: Hello World
INFO: Bonjour le monde
INFO: Hello World
How can the translation be applied to languages other than French?
I was using the old version 0.0.3.
When I changed the version to the latest 0.0.8, the translation was reflected in languages other than French.
pip install fastapi-babel==0.0.8
Note
You need restart FastAPI server, after pybabel compile -d lang
If BABEL_DEFAULT_LOCALE and babel.locale is same, it doesn't translate.
babel = Babel(
configs=BabelConfigs(
ROOT_DIR=__file__,
BABEL_DEFAULT_LOCALE="en",
BABEL_TRANSLATION_DIRECTORY="lang",
)
)
babel.locale = "en"
When you update translation files.
Run this 2 commands.
pybabel extract -F babel.cfg -o messages.pot .
pybabel compile -d lang
Please don't run this command after you create .po file.
pybabel init -i messages.pot -d lang -l fa
If you run, your po file will be reset. (Delete all your translations.)
I have a Django webapp, and I'd like to check if it's running on the Heroku stack (for conditional enabling of debugging, etc.) Is there any simple way to do this? An environment variable, perhaps?
I know I can probably also do it the other way around - that is, have it detect if it's running on a developer machine, but that just doesn't "sound right".
An ENV var seems to the most obvious way of doing this. Either look for an ENV var that you know exists, or set your own:
on_heroku = False
if 'YOUR_ENV_VAR' in os.environ:
on_heroku = True
more at: http://devcenter.heroku.com/articles/config-vars
Similar to what Neil suggested, I would do the following:
debug = True
if 'SOME_ENV_VAR' in os.environ:
debug = False
I've seen some people use if 'PORT' in os.environ: But the unfortunate thing is that the PORT variable is present when you run foreman start locally, so there is no way to distinguish between local testing with foreman and deployment on Heroku.
I'd also recommend using one of the env vars that:
Heroku has out of the box (rather than setting and checking for your own)
is unlikely to be found in your local environment
At the date of posting, Heroku has the following environ variables:
['PATH', 'PS1', 'COLUMNS', 'TERM', 'PORT', 'LINES', 'LANG', 'SHLVL', 'LIBRARY_PATH', 'PWD', 'LD_LIBRARY_PATH', 'PYTHONPATH', 'DYNO', 'PYTHONHASHSEED', 'PYTHONUNBUFFERED', 'PYTHONHOME', 'HOME', '_']
I generally go with if 'DYNO' in os.environ:, because it seems to be the most Heroku specific (who else would use the term dyno, right?).
And I also prefer to format it like an if-else statement because it's more explicit:
if 'DYNO' in os.environ:
debug = False
else:
debug = True
First set the environment variable ON_HEROKU on heroku:
$ heroku config:set ON_HEROKU=1
Then in settings.py
import os
# define if on heroku environment
ON_HEROKU = 'ON_HEROKU' in os.environ
Read more about it here: https://devcenter.heroku.com/articles/config-vars
My solution:
$ heroku config:set HEROKU=1
These environment variables are persistent – they will remain in place across deploys and app restarts – so unless you need to change values, you only need to set them once.
Then you can test its presence in your app.:
>>> 'HEROKU' in os.environ
True
The most reliable way would be to set an environment variable as above.
If that's not possible, there are a few signs you can look for in the filesystem, but they may not be / are not foolproof
Heroku instances all have the path /app - the files and scripts that are running will be under this too, so you can check for the presence of the directory and/or that the scripts are being run from under it.
There is an empty directory /etc/heroku
/etc/hosts may have some heroku related domains added
~ $ cat /etc/hosts
<snip>.dyno.rt.heroku.com
Any of these can and may change at any moment.
Your milage may vary
DATABASE_URL environment variable
in_heroku = False
if 'DATABASE_URL' in os.environ:
in_heroku = True
I think you need to enable the database for your app with:
heroku addons:create heroku-postgresql:hobby-dev
but it is free and likely what you are going to do anyways.
Heroku makes this environment variable available when running its apps, in particular for usage as:
import dj_database_url
if in_heroku:
DATABASES = {'default': dj_database_url.config()}
else:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
Not foolproof as that variable might be defined locally, but convenient for simple cases.
heroku run env
might also show other possible variables like:
DYNO_RAM
WEB_CONCURRENCY
but I'm not sure if those are documented like DATABASE_URL.
Short version: check that the time zone is UTC/GMT:
if not 'ORIGINAL_TIMEZONE' in os.environ:
f = os.popen('date +%Z')
tz = f.read().upper()
os.environ['ORIGINAL_TIMEZONE']=tz
tz = os.environ['ORIGINAL_TIMEZONE']
if tz != '' and (not 'utc' in tz.lower()) and (not 'gmt' in tz.lower()):
print 'Definitely not running on Heroku (or in production in general)'
else:
print 'Assume that we are running on Heroku (or in production in general)'
This is more conservative than if tz=='UTC\n': if in doubt, assume that we are in production. Note that we are saving the timezone to an environment variable because settings.py may be executed more than once. In fact, the development server executes it twice, and the second time the system timezone is already 'UTC' (or whatever is in settings.TIMEZONE).
Long version:
making absolutely sure that we never run on Heroku with DEBUG=True, and that we never run the development server on Heroku even with DEBUG=False. From settings.py:
RUNNING_DEV_SERVER = (len(sys.argv) > 1) and (sys.argv[1] == 'runserver')
DEBUG = RUNNING_DEV_SERVER
TEMPLATE_DEBUG = DEBUG
# Detect the timezone
if not 'ORIGINAL_TIMEZONE' in os.environ:
f = os.popen('date +%Z')
tz = f.read().upper()
os.environ['ORIGINAL_TIMEZONE']=tz
print ('DEBUG: %d, RUNNING_DEV_SERVER: %d, system timezone: %s ' % (DEBUG, RUNNING_DEV_SERVER, tz))
if not (DEBUG or RUNNING_DEV_SERVER):
SECRET_KEY = os.environ['SECRET_KEY']
else:
print 'Running in DEBUG MODE! Hope this is not in production!'
SECRET_KEY = 'DEBUG_INSECURE_SECRET_KEY_ae$kh(7b%$+a fcw_bdnzl#)$t88x7h2-p%eg_ei5m=w&2p-)1+'
# But what if we are idiots and are still somehow running with DEBUG=True in production?!
# 1. Make sure SECRET_KEY is not set
assert not SECRET_KEY in os.environ
# 2. Make sure the timezone is not UTC or GMT (indicating production)
tz = os.environ['ORIGINAL_TIMEZONE']
assert tz != '' and (not 'UTC' in tz) and (not 'GMT' in tz)
# 3. Look for environment variables suggesting we are in PROD
for key in os.environ:
for red_flag in ['heroku', 'amazon', 'aws', 'prod', 'gondor']:
assert not red_flag in key.lower()
assert not red_flag in os.environ[key].lower()
If you really want to run the development server on Heroku, I suggest you add an environment variable specifying the date when you can do that. Then only proceed if this date is today. This way you'll have to change this variable before you begin development work, but if you forget to unset it, next day you will still be protected against accidentally running it in production. Of course, if you want to be super-conservative, you can also specify, say, a 1-hour window when exceptions apply.
Lastly, if you decided to adopt the approach suggested above, while you are at it, also install django-security, add djangosecurity to INSTALLED_APPS, and add to the end of your settings.py:
if not (DEBUG or RUNNING_DEV_SERVER):
### Security
SECURE_SSL_REDIRECT = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_HSTS_SECONDS = 86400000
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_BROWSER_XSS_FILTER = True
SESSION_COOKIE_SECURE = True
SESSION_COOKIE_HTTPONLY = True
CSRF_COOKIE_HTTPONLY = True # May have problems with Ajax
CSRF_COOKIE_SECURE = True
Following the instructions on https://github.com/linuxlewis/djorm-ext-pgfulltext I added a search_index to one of my existing PostgreSQL models, as follows:
search_index = VectorField()
objects = SearchManager(
fields = ('materials','origin','style'),
config = 'pg_catalog.english',
search_field = 'search_index',
auto_update_search_field = True
)
Using my virtual-env and having pgfulltext installed, with this version djorm-ext-pgfulltext==0.9.3 according to pip freeze, I cd to the directory where manage.pyis located. And tried to run:
./manage.py update_search_field app_name
Wich resulted in the following error:
Unknown command: 'update_search_field' Type 'manage.py help' for
usage.
Am I missing something on how this command is used? Should I add the index directly to my db instance with psql? And if so, how could I achieve this?
Any help will be much appreciated.
"Unknown command"
...this usually means you forgot to add the app to your INSTALLED_APPS in settings.py
That's not surprising since it seems they forgot to mention that on the readme for the app, but you need something like:
INSTALLED_APPS = (
# ... (your existing apps)
'djorm_pgfulltext',
)
Django needs to be told about the app in this way so that it knows to look for the management/commands directory in the app.
when deploying a Django solution in production, I ran into a problem with compressed and compiled static files. I have Django running under a Nginx reverse proxy that also takes care of serving static files.
Some of the files (notably js and css files) are not found, and I notice that this happens because of the compression enabled by the following settings in settings.py
COMPRESS_ENABLED = True
COMPRESS_PRECOMPILERS = (
('text/coffeescript', 'coffee --compile --stdio'),
('text/less', 'lessc {infile} {outfile}'),
)
COMPRESS_JS_FILTERS = [
'compressor.filters.jsmin.JSMinFilter'
]
INSTALLED_APPS = (
# [..]
'compressor',
# [..]
)
So, what is the most correct way to freeze the precompilations and compressions in order to deploy them in production? Thanks
additional to the above comment, maybe its worth to look at this
https://github.com/jezdez/django_compressor
Hope this helps.
I guess you are looking for a way to compress files on your dev machine, before deploying:
./manage.py compress --force
You should also set COMPRESS_OFFLINE in settings:
COMPRESS_OFFLINE = True
if 'compress' in sys.argv:
# compressor related settings when called as management command
TEMPLATE_DEBUG = False
COMPRESS_ENABLED = True
I have a Django webapp, and I'd like to check if it's running on the Heroku stack (for conditional enabling of debugging, etc.) Is there any simple way to do this? An environment variable, perhaps?
I know I can probably also do it the other way around - that is, have it detect if it's running on a developer machine, but that just doesn't "sound right".
An ENV var seems to the most obvious way of doing this. Either look for an ENV var that you know exists, or set your own:
on_heroku = False
if 'YOUR_ENV_VAR' in os.environ:
on_heroku = True
more at: http://devcenter.heroku.com/articles/config-vars
Similar to what Neil suggested, I would do the following:
debug = True
if 'SOME_ENV_VAR' in os.environ:
debug = False
I've seen some people use if 'PORT' in os.environ: But the unfortunate thing is that the PORT variable is present when you run foreman start locally, so there is no way to distinguish between local testing with foreman and deployment on Heroku.
I'd also recommend using one of the env vars that:
Heroku has out of the box (rather than setting and checking for your own)
is unlikely to be found in your local environment
At the date of posting, Heroku has the following environ variables:
['PATH', 'PS1', 'COLUMNS', 'TERM', 'PORT', 'LINES', 'LANG', 'SHLVL', 'LIBRARY_PATH', 'PWD', 'LD_LIBRARY_PATH', 'PYTHONPATH', 'DYNO', 'PYTHONHASHSEED', 'PYTHONUNBUFFERED', 'PYTHONHOME', 'HOME', '_']
I generally go with if 'DYNO' in os.environ:, because it seems to be the most Heroku specific (who else would use the term dyno, right?).
And I also prefer to format it like an if-else statement because it's more explicit:
if 'DYNO' in os.environ:
debug = False
else:
debug = True
First set the environment variable ON_HEROKU on heroku:
$ heroku config:set ON_HEROKU=1
Then in settings.py
import os
# define if on heroku environment
ON_HEROKU = 'ON_HEROKU' in os.environ
Read more about it here: https://devcenter.heroku.com/articles/config-vars
My solution:
$ heroku config:set HEROKU=1
These environment variables are persistent – they will remain in place across deploys and app restarts – so unless you need to change values, you only need to set them once.
Then you can test its presence in your app.:
>>> 'HEROKU' in os.environ
True
The most reliable way would be to set an environment variable as above.
If that's not possible, there are a few signs you can look for in the filesystem, but they may not be / are not foolproof
Heroku instances all have the path /app - the files and scripts that are running will be under this too, so you can check for the presence of the directory and/or that the scripts are being run from under it.
There is an empty directory /etc/heroku
/etc/hosts may have some heroku related domains added
~ $ cat /etc/hosts
<snip>.dyno.rt.heroku.com
Any of these can and may change at any moment.
Your milage may vary
DATABASE_URL environment variable
in_heroku = False
if 'DATABASE_URL' in os.environ:
in_heroku = True
I think you need to enable the database for your app with:
heroku addons:create heroku-postgresql:hobby-dev
but it is free and likely what you are going to do anyways.
Heroku makes this environment variable available when running its apps, in particular for usage as:
import dj_database_url
if in_heroku:
DATABASES = {'default': dj_database_url.config()}
else:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
Not foolproof as that variable might be defined locally, but convenient for simple cases.
heroku run env
might also show other possible variables like:
DYNO_RAM
WEB_CONCURRENCY
but I'm not sure if those are documented like DATABASE_URL.
Short version: check that the time zone is UTC/GMT:
if not 'ORIGINAL_TIMEZONE' in os.environ:
f = os.popen('date +%Z')
tz = f.read().upper()
os.environ['ORIGINAL_TIMEZONE']=tz
tz = os.environ['ORIGINAL_TIMEZONE']
if tz != '' and (not 'utc' in tz.lower()) and (not 'gmt' in tz.lower()):
print 'Definitely not running on Heroku (or in production in general)'
else:
print 'Assume that we are running on Heroku (or in production in general)'
This is more conservative than if tz=='UTC\n': if in doubt, assume that we are in production. Note that we are saving the timezone to an environment variable because settings.py may be executed more than once. In fact, the development server executes it twice, and the second time the system timezone is already 'UTC' (or whatever is in settings.TIMEZONE).
Long version:
making absolutely sure that we never run on Heroku with DEBUG=True, and that we never run the development server on Heroku even with DEBUG=False. From settings.py:
RUNNING_DEV_SERVER = (len(sys.argv) > 1) and (sys.argv[1] == 'runserver')
DEBUG = RUNNING_DEV_SERVER
TEMPLATE_DEBUG = DEBUG
# Detect the timezone
if not 'ORIGINAL_TIMEZONE' in os.environ:
f = os.popen('date +%Z')
tz = f.read().upper()
os.environ['ORIGINAL_TIMEZONE']=tz
print ('DEBUG: %d, RUNNING_DEV_SERVER: %d, system timezone: %s ' % (DEBUG, RUNNING_DEV_SERVER, tz))
if not (DEBUG or RUNNING_DEV_SERVER):
SECRET_KEY = os.environ['SECRET_KEY']
else:
print 'Running in DEBUG MODE! Hope this is not in production!'
SECRET_KEY = 'DEBUG_INSECURE_SECRET_KEY_ae$kh(7b%$+a fcw_bdnzl#)$t88x7h2-p%eg_ei5m=w&2p-)1+'
# But what if we are idiots and are still somehow running with DEBUG=True in production?!
# 1. Make sure SECRET_KEY is not set
assert not SECRET_KEY in os.environ
# 2. Make sure the timezone is not UTC or GMT (indicating production)
tz = os.environ['ORIGINAL_TIMEZONE']
assert tz != '' and (not 'UTC' in tz) and (not 'GMT' in tz)
# 3. Look for environment variables suggesting we are in PROD
for key in os.environ:
for red_flag in ['heroku', 'amazon', 'aws', 'prod', 'gondor']:
assert not red_flag in key.lower()
assert not red_flag in os.environ[key].lower()
If you really want to run the development server on Heroku, I suggest you add an environment variable specifying the date when you can do that. Then only proceed if this date is today. This way you'll have to change this variable before you begin development work, but if you forget to unset it, next day you will still be protected against accidentally running it in production. Of course, if you want to be super-conservative, you can also specify, say, a 1-hour window when exceptions apply.
Lastly, if you decided to adopt the approach suggested above, while you are at it, also install django-security, add djangosecurity to INSTALLED_APPS, and add to the end of your settings.py:
if not (DEBUG or RUNNING_DEV_SERVER):
### Security
SECURE_SSL_REDIRECT = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_HSTS_SECONDS = 86400000
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_BROWSER_XSS_FILTER = True
SESSION_COOKIE_SECURE = True
SESSION_COOKIE_HTTPONLY = True
CSRF_COOKIE_HTTPONLY = True # May have problems with Ajax
CSRF_COOKIE_SECURE = True