In short, can I use in python many .mo files for the same language in the same time?
In my python application, I need to use gettext for I18N. This app uses kind of a plug-in system. This means you can download a plug-in and put it in the appropriate directory, and it runs like any other python package. The main application stores the .mo file it uses in let's say ./locale/en/LC_MESSAGES/main.mo. And the plug-in nr 1 has its own .mo file called plugin1.mo in that same directory.
I would use this to load the main.mo I18N messages:
gettext.install('main', './locale', unicode=False)
How can I install the others too, so that all the plug-ins are translated the way they should be?
The solutions I thought of:
Should I gettext.install() in each package's namespace? But this would override the _() defined previously and mess the future translations of the main application.
Is there a way to combine two .mo files in one (when a new plug-in is installed for example)?
At runtime can I combine them into one GNUTranslation object? Or override the default _() method that is added to the global namespace? Then, how would I go with that option?
Instead of _('Hello World'), I would use _('plugin1', 'Hello World in plug-in 1')
Note: The application is not supposed to be aware of all the plug-ins to be installed, so it cannot already have all the messages translated in its main.mo file.
gettext.install() installs the inalterable one and only _ into the builtins (module __builtin__ or builtins in py3) - app-global. This way there is no flexibility.
Note: Python name resolution order is: locals > module-globals > builtins .
So anyway gettext.translation() (Class-based API) or even gettext.GNUTranslations() (e.g. for custom .mo path schemes) would be used explicitely for having multiple translations separately or mixed-style at the same time.
Some options:
Via t = gettext.translation(...); _ = t.ugettext you could simply put separate translations as _ into each module-global namespace - probably more automated way in real world. Just perhaps the main translation could still go into the builtins too (via main_t.install()).
When a mix-all of all/many translations is ok or is what you want, you could chain several translations globally via t.install(); t.add_fallback(t_plugin1); t.add_fallback(t_plugin1);...; - and preserve an app-global approach otherwise.
gettext keywords other than _ could be used - and could be feed via the xgettext -k other_keyword option. But I'd dislike lengthy and module-unique names.
( Yet personally I prefer the keyword I generally over _, and I also enable an operator scheme like I % "some text" instead of _("some text") or I("some text"). Via I = t; t.__call__ = t.__mod__ = t.ugettext effectively; plus a small patch of pygettext.py. This pattern is more pleasant for typing, looks more readable and Pythonic to me, and avoids the crucial/ugly name collision of _ in Python with the interactive-last-result-anaphor (see sys.displayhook) when using gettext'ed modules on interactive Python prompts. _ is also "reserved" as my preferred (local) placeholder for unused values in expressions like _, _, x, y, z, _ = some_tuple.
After all gettext is a rather simple module & mechanism, and things are easily customizable in Python.)
You should use different domains for each plugin. The domain can be package name to prevent conflicts.
I do not understand why you need to translate something outside the plugin by using plugin's domain, but if you really need to, then you should disambiguate the domain each time.
Each plugin can provide it's own "undescore", readily bound to the plugin's domain:
from my.plugin import MessageFactory as _my_plugin
Please, note, that underscore is only a convention so the extraction tools can find i18n-enabled messages in the program. Plugins' messages should be marked with underscore in their respective packages (you do put them into separate packages, right?). In all other places, you are free to call these factories by some other name and leave underscore for the main program translation domain.
I am less sure about .mo-files, but you can surely compile all your .po files into one .mo file. However, if plugins are written by independent uncooperative authors, there could be msgid conflicts.
UPDATE:
If plugins are in the same package with the main app, then there is no sense in using individual translation domains for them (this is not your case). If plugins are in the separate packages, then extraction should be done independently for those packages. In both cases you have no problem with variable _. If for some reason main app wants plugins' translations in its code, use some other name for _, as in the answer. Of course, extraction tools will not identify anything but underscore.
In other words, plugins should care about their translations on their own. The main app could use plugin-specific translation function as part of plug-in API. Extraction or manual addition of strings into po/mo-files are also not of concern for the main app: its up to plugin author to provide translations.
Related
How can I jump to a function definition using Vim? For example with Visual Assist, I can type Alt+g under a function and it opens a context menu listing the files with definitions.
How can I do something like this in vim?
Use ctags. Generate a tags file, and tell vim where it is using the :tags command. Then you can just jump to the function definition using Ctrl-]
There are more tags tricks and tips in this question.
If everything is contained in one file, there's the command gd (as in 'goto definition'), which will take you to the first occurrence in the file of the word under the cursor, which is often the definition.
g* does a decent job without ctags being set up.
That is, type g,* (or just * - see below) to search for the word under the cursor (in this case, the function name). Then press n to go to the next (or Shift-n for previous) occurrence.
It doesn't jump directly to the definition, given that this command just searches for the word under the cursor, but if you don't want to deal with setting up ctags at the moment, you can at least save yourself from having to re-type the function name to search for its definition.
--Edit--
Although I've been using g* for a long time, I've recently discovered two shortcuts for these shortcuts!
(a) * will jump to the next occurrence of the word under the cursor. (No need to type the g, the 'goto' command in vi).
(b) # goes to the previous occurrence, in similar fashion.
N and n still work, but '#' is often very useful to start the search initially in the reverse direction, for example, when looking for the declaration of a variable under the cursor.
Use gd or gD while placing the cursor on any variable in your program.
gd will take you to the local declaration.
gD will take you to the global declaration.
more navigation options can be found in here.
Use cscope for cross referencing large project such as the linux kernel.
TL;DR:
You can do this using internal VIM functionality but a modern (and much easier) way is to use COC for intellisense-like completion and one or more language servers (LS) for jump-to-definition (and way way more). For even more functionality (but it's not needed for jump-to-definition) you can install one or more debuggers and get a full blown IDE experience.
Best second is to use native VIM's functionality called define-search but it was invented for C preprocessor's #define directive and for most other languages requires extra configuration, for some isn't possible at all (also you miss on other IDE features). Finally, a fallback to that is ctags.
Quick-start:
install vim-plug to manage your VIM plug-ins
add COC and (optionally) Vimspector at the top of ~/.vimrc:
call plug#begin()
Plug 'neoclide/coc.nvim', {'branch': 'release'}
Plug 'puremourning/vimspector'
call plug#end()
" key mappings example
nmap <silent> gd <Plug>(coc-definition)
nmap <silent> gD <Plug>(coc-implementation)
nmap <silent> gr <Plug>(coc-references)
" there's way more, see `:help coc-key-mappings#en'
call :source $MYVIMRC | PlugInstall to reload VIM config and download plug-ins
restart vim and call :CocInstall coc-marketplace to get easy access to COC extensions
call :CocList marketplace and search for language servers, e.g.:
type python to find coc-jedi,
type php to find coc-phpls, etc.
(optionally) see :h VimspectorInstall to install additional debuggers, e.g.:
:VimspectorInstall debugpy,
:VimspectorInstall vscode-php-debug, etc.
Full story:
Language server (LS) is a separate standalone application (one for each programming language) that runs in the background and analyses your whole project in real time exposing extra capabilities to your editor (any editor, not only vim). You get things like:
namespace aware tag completion
jump to definition
jump to next / previous error
find all references to an object
find all interface implementations
rename across a whole project
documentation on hover
snippets, code actions, formatting, linting and more...
Communication with language servers takes place via Language Server Protocol (LSP). Both nvim and vim8 (or higher) support LSP through plug-ins, the most popular being Conquer of Completion (COC).
List of actively developed language servers and their capabilities is available on Lang Server website. Not all of those are provided by COC extensions. If you want to use one of those you can either write a COC extension yourself or install LS manually and use the combo of following VIM plug-ins as alternative to COC:
LanguageClient - handles LSP
deoplete - triggers completion as you type
Communication with debuggers takes place via Debug Adapter Protocol (DAP). The most popular DAP plug-in for VIM is Vimspector.
Language Server Protocol (LSP) was created by Microsoft for Visual Studio Code and released as an open source project with a permissive MIT license (standardized by collaboration with Red Hat and Codenvy). Later on Microsoft released Debug Adapter Protocol (DAP) as well. Any language supported by VSCode is supported in VIM.
I personally recommend using COC + language servers provided by COC extensions + ALE for extra linting (but with LSP support disabled to avoid conflicts with COC) + Vimspector + debuggers provided by Vimspector (called "gadgets") + following VIM plug-ins:
call plug#begin()
Plug 'neoclide/coc.nvim'
Plug 'dense-analysis/ale'
Plug 'puremourning/vimspector'
Plug 'scrooloose/nerdtree'
Plug 'scrooloose/nerdcommenter'
Plug 'sheerun/vim-polyglot'
Plug 'yggdroot/indentline'
Plug 'tpope/vim-surround'
Plug 'kana/vim-textobj-user'
\| Plug 'glts/vim-textobj-comment'
Plug 'janko/vim-test'
Plug 'vim-scripts/vcscommand.vim'
Plug 'mhinz/vim-signify'
call plug#end()
You can google each to see what they do.
Native VIM jump to definition:
If you really don't want to use Language Server and still want a somewhat decent jump to definition with native VIM you should get familiar with :ij and :dj which stand for include-jump and definition-jump. These VIM commands let you jump to any file that's included by your project or jump to any defined symbol that's in any of the included files. For that to work, however, VIM has to know how lines that include files or define symbols look like in any given language. You can set it up per language in ~/.vim/ftplugin/$file_type.vim with set include=$regex and set define=$regex patterns as described in :h include-search, although, coming up with those patterns is a bit of an art and sometimes not possible at all, e.g. for languages where symbol definition or file import can span over multiple lines (e.g. Golang). If that's your case the usual fallback is ctags as described in other answers.
As Paul Tomblin mentioned you have to use ctags.
You could also consider using plugins to select appropriate one or to preview the definition of the function under cursor.
Without plugins you will have a headache trying to select one of the hundreds overloaded 'doAction' methods as built in ctags support doesn't take in account the context - just a name.
Also you can use cscope and its 'find global symbol' function. But your vim have to be compiled with +cscope support which isn't default one option of build.
If you know that the function is defined in the current file, you can use 'gD' keystrokes in a normal mode to jump to definition of the symbol under cursor.
Here is the most downloaded plugin for navigation
http://www.vim.org/scripts/script.php?script_id=273
Here is one I've written to select context while jump to tag
http://www.vim.org/scripts/script.php?script_id=2507
Another common technique is to place the function name in the first column. This allows the definition to be found with a simple search.
int
main(int argc, char *argv[])
{
...
}
The above function could then be found with /^main inside the file or with :grep -r '^main' *.c in a directory. As long as code is properly indented the only time the identifier will occur at the beginning of a line is at the function definition.
Of course, if you aren't using ctags from this point on you should be ashamed of yourself! However, I find this coding standard a helpful addition as well.
1- install exuberant ctags. If you're using osx, this article shows a little trick:
http://www.runtime-era.com/2012/05/exuberant-ctags-in-osx-107.html
2- If you only wish to include the ctags for the files in your directory only, run this command in your directory:
ctags -R
This will create a "tags" file for you.
3- If you're using Ruby and wish to include the ctags for your gems (this has been really helpful for me with RubyMotion and local gems that I have developed), do the following:
ctags --exclude=.git --exclude='*.log' -R * `bundle show --paths`
credit: https://coderwall.com/p/lv1qww
(Note that I omitted the -e option which generates tags for emacs instead of vim)
4- Add the following line to your ~/.vimrc
set autochdir
set tags+=./tags;
(Why the semi colon: http://vim.wikia.com/wiki/Single_tags_file_for_a_source_tree )
5- Go to the word you'd like to follow and hit ctrl + ] ; if you'd like to go back, use ctrl+o (source: https://stackoverflow.com/a/53929/226255)
To second Paul's response: yes, ctags (especially exuberant-ctags (http://ctags.sourceforge.net/)) is great. I have also added this to my vimrc, so I can use one tags file for an entire project:
set tags=tags;/
Install cscope. It works very much like ctags but more powerful. To go to definition, instead of Ctrl + ], do Ctrl + \ + g. Of course you may use both concurrently. But with a big project (say Linux kernel), cscope is miles ahead.
After generating ctags, you can also use the following in vim:
:tag <f_name>
Above will take you to function definition.
I have been told that doing this would be a not-very-good practice (it is present in the main answer of Python pattern for sharing configuration throughout application though):
configfile.py
SOUNDENABLED = 1
FILEPATH = 'D:\\TEMP\\hello.txt'
main.py
import configfile
if configfile.SOUNDENABLED == 1:
....
f = open(configfile.FILEPATH, 'a')
...
This is confirmed by the fact that many people use INI files for local configuration with ConfigParser module or iniparse or other similar modules.
Question: Why would using an INI file for local configuration + an INI parser Python module be better than just importing a configfile.py file containing the right config values as constants?
The only concern here is that a .py can have arbitrary Python code, so it has a potential to break your program in arbitrary ways.
If you can trust your users to use it responsibly, there's nothing wrong with this setup. If fact, at one of my previous occupations, we were doing just that, without any problems that I'm aware of. Just on the contrary: it allowed users to eliminate duplication by autogenerating repetitive parts and importing other config files.
Another concern is if you have many files, the configuration ones are better be separated from regular code ones, so users know which files they are supposed to be able to edit (the above link addresses this, too).
Importing a module executes any code that it contains. Nothing restricts your configfile.py to containing only definitions. Down the line, this is a recipe for security concerns and obscure errors. Also, you are bound to Python's module search path for finding the configuration file. What if you want to place the configuration file somewhere else, or if there is a name conflict?
This is a perfectly acceptable practice. Some examples of well-known projects using this method are Django and gunicorn.
It could be better for some reasons
The only extension that config file could have is py.
You cannot distribute your program with configs in separate directory unless you put an __init__.py into this directory
Nasty users of your program can put any python script in config and do bad things.
For example, the YouCompleteMe autocompletion engine stores config in python module, .ycm_extra_conf.py. By default, each time config is imported, it asks you, whether you sure that the file is safe to be executed.
How would you change configuration without restarting your app?
Generally, allowing execution of code that came from somewhere outside is a vulnerability, that could lead to very serious consequences.
However, if you don't care about these, for example, you are developing web application that executes only on your server, this is an acceptable practice to put configuration into python module. Django does so.
A "settings file" would be a file where things like "background color", "speed of execution", "number of x's" are defined. Currently, I implemented it as a single setting.py file, which I import in the beginning. Someone told me I should make it a settings.ini file instead, but I don't see why! Care to clarify, what is the optimal option?
There is no optimal solution; it is a matter of preference.*
Normally, settings do not need to be expressed in a Turing-complete language: they're often just a bunch of flags and options, sometimes strings and numbers, etc. An argument for having a settings.py file (though very unorthodox) would be if the end-user was expected to write code to generate very esoteric configurations (e.g. maps for a game). This would then be fairly similar to shell script .bashrc-style files.
But again, in 99.9% of programs, the settings are often just a bunch of flags and options, sometimes strings and numbers, etc. It's fine to store them as JSON or XML. It also makes it easy to perform reflection on your settings: for example, automatically listing them in a tree manner, or automatically creating a GUI out of the descriptions.
(Also it may be a (unlikely?) security issue if you allow people to inject code by modifying the settings file.)
*edit: no pun intended...
There are a few reasons why separating out config files from main codebase is a good idea. Of course it depends on your use case and you should evaluate against your usecase.
Configuration can be managed by end user, who do not understand programming languages. It makes more sense to factor out configuration and use a simple ini file which uses simple key-value pairs for config parameters.
Configuration varies based on the installation environment. Your code runs on multiple environment and they all use different configuration. It is very easy to maintain such cases by having separate config files and same source code installed on those environments.
There are package managers that knows what is a config file and what is a source file. They are intelligent to not override any changed config on version upgrade etc. So you do not have to worry about resetting config parameters after version upgrade of package. For example you ship your product with a default config file. User fine tuned few parameters. You shipped another version of the package. User should not expect a config reset after version upgrade.
One problem with having a settings file being a Python module is that it can contain code that will be executed when you import it. This may allow malicious code to be inserted into your program.
For Python use stock libraries:
YAML style configuration files:
http://www.yaml.org/start.html
http://pypi.python.org/pypi/PyYAML/
(Used e.g. Google App Engine)
INI: http://docs.python.org/library/configparser.html
Don't use XML for hand-edited config files.
I was wondering how I can provide an API for my Python program to enable others to extend it with plugins.
I thought about something like from myProgram.plugins import aClassToExtendByOthers, registerThatClass.
But I have no idea how to provide this.
I could use an exec statement within my loadPlugins function for every plugin in the plugins-folder, but this would not enable importing stuff I would like to
provide for people to write those plugins.
For a system that I use in several of my programs, I define a directory of plugins, and provide a base plugin class for all plugins to subclass. I then import all modules in the directory, and selectively initialize (by checking to see if they subclass my base plugin class), and store instances of plugins in a dictionary (or list). I have found that the command dispatch pattern has worked effectively for me as a way to structure the plugins and pass events. The base plugin (or another optional interface class) can provide the methods that the plugin needs to interact with the application. I hope this helps. It may not be the best way to do it, but it has worked for me.
In addition, you could do additional filtering, such as requiring plugin files to have a prefix (e.g. __plug_file.py__ and __plug_other.py__).
you can use imp module (see docs.python.org)
sys.path.insert(0, pluginsDir)
lst = map(lambda x: os.path.splitext(os.path.basename(x))[0], glob.glob(os.path.join(pluginsDir, "*.py")))
for module in lst:
try:
f, fn, d = imp.find_module(module,[pluginsDir])
loaded = imp.load_module(module, f, fn, d)
for fully functional example see the loader of ojuba control center
http://git.ojuba.org/cgit/occ/tree/OjubaControlCenter/loader.py
I'm internationalizing a Python application, with two goals in mind:
The application loads classes from multiple packages, each with its own i18n domain. So modules in package A are using domain A, modules in package B are using domain B, etc.
The locale can be changed while the application is running.
Python's gettext module makes internationalizing a single-domain single-language application very easy; you just set the locale, then call gettext.install(), which finds the right Translation and installs its ugettext method in the global namespace as _. But obviously a global value only works for a single domain, and since it loads a single Translation, it only works for that locale.
I could instead load the Translation at the top of each module, and bind its ugettext method as _. Then each module would be using the _ for its own domain. But this still breaks when the locale is changed, because _ is for the wrong locale.
So I guess I could load the correct ugettext at the top of every function, but that seems awfully cumbersome. An alternative would be to still do it per-module, but instead of binding ugettext, I would substitute my own function, which would check the current locale, get its Translation, and forward the string to it. Does anyone have a cleaner solution?
how about you bind _ to a function roughly like this (for each module):
def _(message):
return my_gettext(__name__, message)
This allows you to use gettext while at the same time perform any lookup on a per-module-per-call-base that allows you to switch locale as well.