Emacs Python.el, Syntax Highlighting Quirks - python

I'm using python.el version 0.23.1 for Emacs right now. The syntax highlighting seems to be a bit off -- any variable name containing an underscore followed by a keyword will result in the keyword being highlighted. Example, "foo_list" will result in "list" being highlighted.
More for my own understanding of Emacs-Lisp than anything (it's not a big deal) how do I go about fixing that. Here is where I think the relevant code is in "python.el", lines 312-318. I suspect the problem is with the definition of "symbol-start" but I have no idea what that name refers to.
(defvar python-font-lock-keywords
;; Keywords
`(,(rx symbol-start
(or "and" "del" "from" "not" "while" "as" "elif" "global" "or" "with"
"assert" "else" "if" "pass" "yield" "break" "except" "import"
"print" "class" "exec" "in" "raise" "continue" "finally" "is"
"return" "def" "for" "lambda" "try" "self")
symbol-end)
One thing I've struggled with in Emacs-Lisp so far is that I am finding it difficult in situations like these to follow names back to their definitions.
Thanks in advance!

When you say you're using python-mode 0.23.1, do you mean the one that comes bundled with Emacs or this one: http://launchpad.net/python-mode ? (which seems to be on version 6.something)
The reason I'm asking is that I can't reproduce what you're seeing. In an empty python buffer, I inserted
def x ():
a_list =3
and "list" is only highlighted when I delete the "a_". I'm using the version bundled with Emacs, with a snapshot version of Emacs so this might be the difference?
Incidentally, the font lock rule that you quote looks right to me: maybe the problem is that in your version "_" isn't set to have symbol syntax? You can check by typing
M-: (string (char-syntax ?_))
when in a python buffer. You should get "_" (which means symbol). If you get "." (punctuation) or something else weird, that probably explains what's gone wrong.

Related

PEP8 - Contradiction between E129 and E127/E128

According to the PEP standards, indents should come before binary operators. Furthermore, multiline conditions should be enclosed within parentheses to avoid using backslashes before newlines. These two conventions lead to the following situation
if (long_condition_1
or long_condition_2):
do_some_function()
This code in turn breaks E129 visually indented line with same indent as next logical line in PEP8. However, the second line must be indented exactly four spaces, as otherwise it breaks E128 or E127 for under-indented or over-indented lines.
How should one format the above so that it confirms to PEP8 standards?
This should work properly
if (long_condition_1 or
long_condition_2):
do_some_function()
The answer to this question has changed over time. Due to a change in stance from PEP8, W503 is now widely regarded to go against PEP8.
PEP8 now says it's fine to break before OR after, but to keep it consistent locally.
For newer code, Knuth-style is preferred (which I think refers to breaking before the operator).
if (
long_condition_1
or long_condition_2
or (
long_condition_3
and long_condition4
)
):
do_some_function()
if any((long_condition_1,
long_condition_2)):
do_some_function()
it's better to read when both conditions aligned too ...

What does a semicolon do?

I got a function online to help me with my current project and it had semicolons on some of the lines. I was wondering why? Is it to break the function?
def containsAny(self, strings=[]):
alphabet = 'abcdefghijklmnopqrstuvwxyz0123456789'
for string in strings:
for char in string:
if char in alphabet: return 1;
return 0;
The function I got online with little modification:
for string in strings:
for char in string:
if char in alphabet: return 1;
Is the above saying the following?
if char in alphabet:
return 1
break
The semicolon does nothing in the code you show.
I suspect this is someone who programs in another language (C, Java, ...) that requires semicolons at the end of statements and it's just a habit (happens to me sometimes too).
If you want to put several Python statements on the same line, you can use a semi-colon to separate them, see this Python Doc:
A suite is a group of statements controlled by a clause. A suite can
be one or more semicolon-separated simple statements on the same line
as the header, following the header’s colon, or it can be one or more
indented statements on subsequent lines
The semicolon here does not do anything. People who come from C/C++/Java/(many other language) backgrounds tend to use the semicolon out of habit.
In general the semicolon does nothing. But if you are using the Jupyter Notebook (depending on your version), you might get a figure plotted twice. The semicolon at the end of your plot command prevents this:
df.plot();
Programmers of C, C++, and Java are habituated of using a semicolon to tell the compiler that this is the end of a statement, but for Python this is not the case.
The reason is that in Python, newlines are an unambiguous way of separating code lines; this is by design, and the way this works has been thoroughly thought through. As a result, Python code is perfectly readable and unambiguous without any special end-of-statement markers (apart from the newline).
As other answers point out, the semicolon does nothing there. It's a separator (e.g. print 1;print 2). But it does not work like this: def func():print 1;print 2;;print'Defined!' (;; is a syntax error). Out of habit, people tend to use it (as it is required in languages such as C/Java...).

Problems with Nested Functions

THIS TURNED OUT TO BE A SYNTAX ERROR ON MY PART A LINE EARLIER IN THE CODE.
Hello, I'm having some trouble with a nested function I wrote in python. Here is the relevant code.
device = "/dev/sr0"
def burn():
global device
burnaudiotrack(device)
createiso(device)
burntrack2(device)
I'm confused, because every time I try to run the script, python returns this:
File "./install.py", line 72
burnaudiotrack(device)
^
SyntaxError: invalid syntax
I've nested functions before, and done so in a similar manner. I feel like I'm missing something fairly obvious here, but I can't pinpoint it. Thank you for your help/suggestions!.
EDIT:
Full code: (I tried to just post relevant info in the original)
http://dpaste.com/hold/291347/
It's a tad messy, and there may be other errors, but this one is vexing me at the moment.
You are missing a close parenthesis on line 61.
Looks like the quote and paren at the end of the line are swapped.
speed = raw_input("Recomended(4);Default(8))"
should be
speed = raw_input("Recomended(4);Default(8)")
The code you have pasted into your question appears to have tabs as well as spaces. You should (according to PEP-8) always use spaces for indenting in Python. Check your text editor settings.
What's probably happened is you have some mix of tabs and spaces that looks correct in your editor, but is being interpreted differently by the Python compiler. The Python compiler sees a different inconsistent indenting, and throws a SyntaxError.
Update: As another answer points out, you are missing a closing parenthesis on a line of code you didn't show in your original question. Nevertheless, my comments about tabs in your source still hold.

How to fold long docstrings in python source code in VIM?

Does anybody know of a method, or perhaps a plugin, that will
automatically fold long docstrings in Python? I have docstrings in my
code that span several pages, so it is troublesome to keep paging
through them. The other tricky part is that there is embedded python
testing code in the docstrings, so that might make parsing them
difficult. Note that I only need to automatically fold the entire
docstring, regardless of what is in it.
This is a bit of a dirty hack, but you can go through the python syntax file (:sp $VIMRUNTIME/syntax/python.vim) and find all the syntax regions for triple-quoted strings (search for ''' and """) and add the fold keyword to the end of those statements. Then just set foldmethod=syntax for python files and the comments should be folded.
I'm not sure about a plugin or automation, but if you type zf/ you can then search for something and it will fold up to the next instance of it. So in a document like the following (where [] is the cursor):
def foo():
"""[]
Some long docstring
that takes up many
lines
"""
pass
Look at edit2 first for the updated search string!
If you use the command zf/"""[ENTER], it should fold everything from the current line (the beginning of the docstring) to the next occurrence of """ which should be the end of the docstring.
I know this isn't automation, but perhaps it will help in the interim, or lead you down the right path to automating it. See edit2 for a better search function, although I still don't know how to automate.
Hope this helps.
Edit: in a corollary, you can search for any docstring with /"""\_.\{-}""", although this will also return the code within the docstring. To search for a function definition followed by a docstring, you can use /def\_.\{-}"""\_.\{-}""", although this breaks on a def inside the docstring.
Edit2: Actually, some more playing with regexs led me to this: /def.\{-}):\_s*"""\_.\{-}""" which should find any function followed by a docstring. It searches for def followed by any characters, then ): followed by a newline and/or whitespace followed by """ followed by any number of lines than the next """, but always ensures the 2nd triple quote is the one immediately following the first.
In your .vimrc add:
" folding
set foldmethod=indent
This will auto-fold at every indentation, which, in python, translates to docstrings. It works VERY VERY nice. Give it a try. The above answer is correct, but requires a bunch of keystrokes (blah!)
I think I've found a slight improvement to the answer of #too_much_php by trying to figure out how to do the same thing without root access. Copying the syntax region definitions from $VIMRUNTIME/syntax/python.vim to ~/.vim/after/syntax/python.vim and editing them solved the problem for me. Here's my code for ~/.vim/after/syntax/python.vim:
syn region docString1
\ start=+[uU]\=\z('''\|"""\)+ end="\z1" keepend transparent fold
syn region docString2
\ start=+[uU]\=[rR]\z('''\|"""\)+ end="\z1" keepend transparent fold
Now I can run :set foldmethod=syntax to fold docstrings.
PS: Credits to #too_much_php for the initial idea
You can do this with :set foldmethod=marker foldmarker=""",""", I think. I haven't tested it, but that should do the trick. The arguments to foldmarker are start and end markers.
I wrote a vimscript plugin many years ago to do exactly this, but never got around to publishing it. I've put it up as a gist, but have also pasted the source below. I've also replicated its functionality for Sublime Text here.
"
" Fold multi-line Python comments into one line.
"
" Also maps the "-" key to toggle expansion and <C-f> to toggle all folding.
"
setlocal foldmethod=syntax
setlocal foldtext=FoldText()
setlocal fillchars=
map <buffer> - za
map <buffer> <C-f> :call ToggleFold()<CR>
let b:folded = 1
hi Folded gui=bold cterm=bold guifg=cyan ctermfg=cyan guibg=NONE ctermbg=NONE
function! ToggleFold()
if b:folded == 0
exec "normal! zM"
let b:folded = 1
else
exec "normal! zR"
let b:folded = 0
endif
endfunction
function! s:Strip(string)
return substitute(a:string, '^[[:space:][:return:][:cntrl:]]\+\|[[:space:][:return:][:cntrl:]]\+$', '', '')
endfunction
" Extract the first line of a multi-line comment to use as the fold snippet
function! FoldText()
let l:snippet = getline(v:foldstart)
if len(s:Strip(l:snippet)) == 3
let l:snippet = strpart(l:snippet, 1) . s:Strip(getline(v:foldstart + 1))
endif
return '+' . l:snippet . ' ...'
endfunction
My solution involves using SimpylFold. After installing it using Vundle, I put this in my vimrc file:
autocmd FileType python setlocal foldlevel=2
Which makes all docstrings folded by default when you open a python file, which is super awesome.

Is there anything that cannot appear inside parentheses?

I was intrigued by this answer to my question about getting vim to highlight unmatched brackets in python code. Specifically, I'm talking about the second part of his answer where he mentions that the C syntax highlighting is actually flagging as an error any instance of curly braces inside parens. It is an unobtrusive cue that you have unclosed parens when all of your downstream curly braces light up in red.
That trick works because C syntax doesn't allow curly braces inside parentheses. To satisfy my (morbid?) curiosity, can I do something similar with python code? Is there anything in python syntax that isn't legal inside parentheses?
Note: I'm not trolling for a better answer to my other question (there are plenty of good answers there already). I'm merely curious if this trick is even possible with python code.
Any Python statement (import, if, for, while, def, class etc.) cannot be in the parentheses:
In [1]: (import sys)
------------------------------------------------------------
File "<ipython console>", line 1
(import sys)
^
<type 'exceptions.SyntaxError'>: invalid syntax
Here's an exact answer:
http://docs.python.org/reference/expressions.html#grammar-token-expression_list
http://docs.python.org/reference/compound_stmts.html#function
I'm not sure what are you trying to do, but how about "def" or "class"?
this snippet is valid when it's not inside parenthesis
class dummy: pass

Categories