I was looking for a quick way to autoformat/pretty-print JSON in Vim the other day and found this great little command on Stack Overflow: :%!python -m json.tool
That sent me on a search for a list of other Python tools to pretty-print common web files, but I couldn't find much. Is there a good resource/list of Python tools that they find particularly useful for cleaning up poorly formatted web stuff inside Vim (e.g. HTML, XML, JavaScript, etc.)?
Python
Are you just looking for a resource for Python one-liners? You could browse through the Python standard library documentation to find more inspiration.
Or simply google "python one-liners json.tool" to find additional resources. For example, this Reddit post: Suggestion for a Python blogger: figure out what what all the stdlib main functionality is, and document it
Command line
Vim supports more than just Python (e.g. HTML Tidy as Keith suggested). Any tool that can accept pipe/standard input will integrate well with Vim.
The % command just picks a range that contains the entire file, and ! filters that range through an external program.
See :help :% and :help :!
For XHTML and XML files you can use tidy.
:%!tidy -i -asxhtml -utf8
:`<,`>!tidy -i -xml -utf8
The last one works on visual selections.
Vim has a command to do it, = (equal), like in ggvG= will reindent the whole file. Try :help = for more info about how to use functions and external programs with =. The default configuration uses internal indenting rules which works for most file types.
There are loads of good tools that are can convert text between the two formats:
par: for hard line wrapping.
pandoc: for HTML, LaTeX, rst, and Markdown
autopep8: for parsing Python code into an AST and spitting it out as pep8 compliant.
...
Vim is designed to make use of such utilities by the powerful formatprg settings. That by default is mapped to the gq operator. It works well with Vim motions, Vim text objects, selections, etc.
For instance, I use the setting below for on my Python files
au FileType python setlocal formatprg=autopep8\ --indent-size\ 0\ -
John MacFarlne has a good article about creating a specialised script using pandoc which you could stick in your vimrc.
!autopep8 -i % seems to work fine in vim . The -i switch is to over-write the existing file in place. Use more # autopep8 --help.
There is a vim plugin for this if you really are thinking of being a power user. Outside of vim you can test it with autopep8 -diff {filename} or autopep8 {filename}.
Related
I observed that Atom editor has one very good functionality, which is when I type while working with python
from XXX import
it shows list of items that can be imported from XXX
Atom shows a list of things that can be imported. Can VIM also be configured for the same ?
Is there any plugin ?
Vim has four features that can be used to complete imports in Python:
:help 'include' tells Vim how to recognize an "include" in your code. The default value for Python (^\s*\(from\|import\)) is reasonable.
:help 'define' can be used to tell Vim how a definition is supposed to look. There's no Python-specific default value but the following would be a good start:
:set define=^\\s*def
:help compl-define lets you complete from the definitions gathered in included files with <C-x><C-d>. Or you could customize :help 'complete' to include definitions and simply use <C-p> or <C-n>:
:set complete+=d
:help 'path' tells Vim where to look for files when you do :find or gf or include/define completion. For it to be of any use, though, 'path' must be set carefully.
Everything is here and relatively easy to set up, but there certainly are Python plugins that do all that for you in a smarter way.
It turns out what I was looking for is CTRL+Space
In Atom this part is automated as in you don't even have to press CTRL+Space
There are a lot. But these are the current best:
Deoplete + deoplete-jedi
https://github.com/zchee/deoplete-jedi
https://github.com/Shougo/deoplete.nvim
Using LSP
https://github.com/neovim/python-client
https://github.com/autozimu/LanguageClient-neovim
A localized command line application looks strange when some part of the messages are in the user language and some other parts, in English.
I don't know if I messed up anything when I installed Python 3 from source, it seems there are no *.mo files, so argparse (among the whole) is not localization aware.
The API does not seems to offer a way to localize, neither. Or did I missed it?
I could patch argparse.py, but I won't, as I want it to be portable, and I'm not OK with suggesting users to patch their Python installation.
Question in fewer words: how to localize argpase without patching Python standard library?
Related question: is this possible?
By default argparse uses the gettext module to translate the messages so you can easily translate them if you want to.
To generate the *.pot files (which you can convert to *.mo files after translation) you can use the pygettext program which is available in the built-in gettext module in python.
Usage:
python pygettext.py argparse.py
This will generate a messages.pot which you can translate, after that just generate the .mo (many ways to do this, just google).
For more info about pygettext see the Python manual about the subject: Internationalizing your programs and modules.
I work on my Django project with emacs. In my virtualenv "postactivate" script I have the following simple command:
find -L . -type f -name "*.py" | xargs etags -e > /dev/null 2>&1 &
The TAGS file generates just fine but the system seems rather dumb. When the cursor is a model filter call, e.g.
MyModel.objects.filter(...)
and I hit M-., sometimes emacs takes me place where MyModel is imported at the time of the file (the actual import statement). I only ever want to visit class, method, and function definitions.
Is there a way to make etags smarter?
Thanks,
Ryan Kaskel
Getting correct module analysis with a language like python is very hard, due to his dynamic nature the best way to get correct information is doing static analysis or heuristics.
Currently the best I've found is exploring methods with the ropemacs extension that has great features like code assist (quite smart) and calltips.
Unfortunately it's not easy to get it right with ropemacs, you should install first pymacs and then configure install various rope libraries. (I'm working on a packaged version of it)
Another package that would statically analyze your python code and produce "smarter tags" would be something like pysmell, but I haven't used it extensively
I have used just Windows for programming so far. Now, I have an internship starting in two weeks and I will be using just Linux environment with Python programming language. I've installed Ubuntu on my system but have no exposure to shell scripting.
I need some advice on how I can quickly learn to use the Linux terminal quickly. Any books or web resources that you can suggest?
Also, is there a particular IDE that is generally preferred for Python programming on Linux, or is Vim preferred? How can I best prepare myself for the internship ahead?
Thanks for taking the time.
As an intern you'll want to use the tools your mentor is most comfortable with. If you get stuck you'll be able to ask for advice quickly.
Learning your way around either vi, vim, or emacs to start with will help. The basic concepts used in one will transfer to the other. You'll need to be able to open and read files, search through files, edit and save files, and learn how to apply any python formatting helpers correctly.
You should also familiarize yourself with version control if you haven't already. Again any one will do, you need to focus on concepts and etiquette rather than the specific tool.
The goal of the internship (and really your entire time at university) should be used to learn concepts rather than specific tools. If you learn the concepts you'll be well placed to apply those concepts using any tool. You will also "learn how to learn" a new tool, which is really valuable.
Your lack of shell scripting knowledge shouldn't matter in this case, although it won't be hard to learn. I read over some shell tutorials and put them into practice. Try doing everything from the command line, including find (grep), find/replace all (sed), finding files (find), automating things using python scripts etc. Basically, don't cheat. You'll pick up a lot this way. You'll also probably end up wondering how you ever managed with Windows.
What I use depends on the project. I really like Eclipse+PyDev but that's my personal preference, I also use Vim depending on where I am/what I'm doing. Remember you can just type python from the command line and it drops you into the python environment.
I recommend Eclipse + PyDev too. You can get started quickly with this develop environment. I also recommend the website Dive Into Python. It provides you a online free version of Dive Into Python book, which is very easy to read, easy to understand, and very suitable for Python beginners. If you really want a paper book at hand, Learning Python, a.k.a. The Animal Guide, is simply the best.
Learn to understand man(ual) pages.
For almost any old linux command/program there is a man page which usually explains the command in good detail.
So basics for filesystem navigation:
Show directory contents (list)
ls
Show hidden files
ls -a
Show details
ls -l
Change directory
cd /full/path/name
Print current directory
pwd
Delete a file
rm file
Delete a directory (recursive)
rm -r directoryName
Make a directory
mkdir directoryName
Move (or rename) a file
mv /path/to/file /new/path/to/file
Show the man page for mv
man mv
Learning vim might be necessary, depending on your intern environment. I do my Python (and everything that isn't simple text editing) in Eclipse. You should in any case learn enough to open a file, makes some changes and save the changes in Vim.
Keep in mind, Ubuntu is very easy. To make things harder on yourself, use the command line for every conceivable thing. Open programs by typing their names into a terminal. Browse your files with the terminal. Do simple editing with vim. That should provide good practice for the day you need to SSH into a computer in Neverland and download and install a local copy of your favorite interpreter from source in order to set up a cron job to run a script to play a clock noise.
In addition to the great advice already written, I'd suggest you install IPython (Open a terminal with Applications>Accessories>Terminal and type):
sudo apt-get install ipython
Also at the terminal, you can then type ipython to start the Python interpreter.
Unlike the built in python interpreter, ipython gives you tab completion.
For example, if you type the name of an object followed by a period and TAB (e.g. sys.[TAB]), ipython will show you (almost) all of object's attributes.
Type a question mark after an object name (e.g. sys?), and you get documentation on that object.
This is a great way to explore Python.
have no exposure to shell scripting
Good! You've got Python so hopefully there should be no need to resort to writing actual scripts with the shell. It may be more powerful than DOS batch files, but it's just as ugly.
I need some advice on how I can quickly learn to use the Linux terminal quickly.
Something like this?
As well as learning the commands, you'll want to get used to using tab-completion and arrow key command recall (if you don't already do that with the Windows Command Prompt), scrolling with shift-arrows, and so on. Also useful to know the & (perform in background) command suffix, ctrl-C-to-stop, ctrl-Z-to-pause, jobs, and screen.
Incidentally if you will be spending any amount of time in the interactive Python interpreter it is well worth adding tab completion there, too. (This is just as much the case on Windows, but on Win you tend not to get pyreadline by default.)
is there a particular IDE that is generally preferred for Python programming on Linux
Just like on Windows, there are IDEs available if you want them but many people just use a normal text editor. vim is fine if that's what you like. nano is another in-terminal text editor you usually get that's relatively simple. Ubuntu's default desktop-based editor gedit is also fine. It's a matter of personal taste.
(If you are interning at a particular company they might have their own development environment they'd prefer you to use.)
For a Python IDE, I recommend using either IDLE or Eclipse with PyDev.
Keep in mind you can also just use python on the linux command-line. It supports loading code from files, and if you use two command windows then one of them will be your "REPL" where you will be running python and dynamically loading code - and the other window can run your editor.
Regarding linux command line, I cannot recommend any great resources. However, you will be off to a great start if you immerse yourself in this environment and only use linux for the next 2 weeks. Just keep learning, and when you do not know how to do something, read a manpage or google it to find the answer.
for a very beginner intro to the command line, check out: http://en.flossmanuals.net/CommandLineIntro/GettingStarted
As far as a Python editor goes, I personally prefer to use SciTE. It's just a programmer's text editor with syntax highlighting for various languages. I prefer a lightweight editor over a more complicated environment, but if you want a full-fledged IDE you can always try out NetBeans, IDLE, or Komodo (all of which are available in both Windows and Linux).
as for terminall and quick way to understand it's and learn it there are a nice cheat sheets on net like this:
http://fosswire.com/post/2007/8/unixlinux-command-cheat-sheet/
I'm working on a Python program that makes heavy use of eggs (Plone). That means there are 198 directories full of Python code I might want to search through while debugging. Is there a good way to search only the .py files in only those directories, avoiding unrelated code and large binary files?
find DIRECTORY -name "*.py" | xargs grep PATTERN
By the way, since writing this, I have discovered ack, which is a much better solution.
(And since that edit, I have discovered ag).
grep -r -n "PATTERN" --include="*.py" DIRECTORY
I would strongly recommend ack, a grep substitute, "aimed at programmers with large trees of heterogeneous source code" (from the website)
I also use ack a lot these days. I did tweak it a bit to find all the relevant file types:
# Add zcml to the xml type:
--type-add
xml=.zcml
# Add more files the plone type:
--type-add
plone=.dtml,.zpt,.kss,.vpy,.props
# buildout config files
--type-set
buildout=.cfg
# Include our page templates to the html type so we can limit our search:
--type-add
html=.pt,.zpt
# Create txt file type:
--type-set
txt=.txt,.rst
# Define i18n file types:
--type-set
i18n=.pot,.po
# More options
--follow
--ignore-case
--nogroup
Important to remember is that ack won't find files if the extension isn't in its configuration. See "ack --help-types" for all the available types.
I also assume you are using omelette so you can grep/ack/find all the related files?
This problem was the motivation for the creation of collective.recipe.omelette. It is a buildout recipe which can symlink all the eggs from your working set into one directory structure, which you can point your favorite search utility at.
find <directory> -name '*.py' -exec grep <pattern> {} \;
There's also GNU idutils if you want to grep for identifiers in a large source tree very very quickly. It requires building a search database in advance, by running mkid (and tweaking its config file to not ignore .py files). z3c.recipe.tag takes care of that, if you use buildout.
Just in case you want a non-commandline OSS solution...
I use pycharm. It has built in support for buildout. You point it at a buildout generated bin/instance and it sets the projects external dependencies to all the eggs used by the instance. Then all the IDE's introspection and code navigation work nicely. Goto definition, goto instances, refactoring support and of course search.
I recomend grin to search, omelette when working with plone and the pydev-feature 'Globals browser' (with eclipse or aptana studio).
And simply because there are not enough answers...
If you're developing routinely, it's well worth the effort to install Eclipse with Pydev (or even easier, Aptana Studio - which is a modified Eclipse), in which case the find tools are right there.
OpenGrok is an excellent choice for source searching and navigation. Runs on Java, though.
I really wish there was something like https://oracle.github.io/opengrok/
My grepping life is way more satisfying since discovering Emacs' rgrep command.
Say I want to find 'IPortletDataProvider' in Plone's source. I do:
M-x rgrep
Emacs prompts for the search string (IPortletDataProvider)
... then which files to search (*.py)
... then which directory (~/Plone/buildout-cache/eggs). If I'm already editing a file, this defaults to that file's directory, which is usually exactly what I want.
The results appear in a new buffer. At the top is the find | xargs grep command Emacs ran. All matches are highlighted. I can search the buffer using the standard text search commands. Best of all, I can hit Enter (or click) on a match to open that file.
It's a pretty nice way to work. I like that I don't have to remember find | xargs grep argument sequences, but that all that power is there if I need it.