We Keep Coding

Embed Python code in Verilog file in Emacs

I am working on a project whereby I need to embed Python within a Verilog file. The Python isn't really intended for execution in the normal sense as it will be read by a secondary tool. The Python will be written blocks that have some fixed demarcation (such as #+BEGIN_SRC, in org-babel).
module name ();
#+BEGIN_SRC python
def my_function ():
...
#+END_SRC
always #(posedge clk)
...
endmodule
Within Emacs this causes havoc, although Python-mode and Verilog-mode work fine, when combining both in the same file things quickly break-down as one would expect. Indentation is hopelessly broken as is syntax-highlighting. I understand this is a very weird thing to do, and I understand that there will almost certainly never been any real need to do this under normal circumstances, however for this particular case it is necessary.
My question: is there anyway within Emacs to specify multiple major modes within the same file. For example, is there some way that I can write a file using Verilog-mode as my major mode, but use Python-mode within the predefined blocks that are then ignored in the reset of the file.

There are a number of possibilities listed here:
http://emacswiki.org/emacs/MultipleModes
I've used multi-mode with latex and haskell, and it works OK.

introspective code completion with VIM? ... or other lightweight editor with this feature?

I've been all over the web trying to find a way to get VIM to have code completion similar to PyDev. It doesn't seem like it is possible!
-I have tried to use the omnicompletion suggested at this link: http://blog.dispatched.ch/2009/05/24/vim-as-python-ide/ .
-I have tried several addons to alleviate the problem, none work.
The "omnicomplete" functionality is NOT what I am looking for. It just takes all the words in the file you are working on and uses those to try and complete what I am doing. For example if I wrote:
import numpy
a_single_array = range(100)
np.a#[then I hit cntrl+n to code complete]
It would spit out "a_single_array" as a possible completion -- but that is absurd! That is not a valid completion for "numpy.a ..."
What is the issue here? All the addon would have to do is run a dir(work you want to find) from the folder you are in and then filter the output! This cannot be that difficult! (I suppose you would also have to read the file you are currently editing and filter that as well to take note of name changes... but that's pretty much it!)
Speaking of how easy it would be... if there isn't anything already made, I was thinking of writing the script myself! Any guides on how to do THAT?

No, the omni completion functionality is EXACTLY what you are looking for.
You are using <C-n> instead of <C-x><C-o>:
type <C-n> & <C-p> to complete with words from the buffer (after and before the cursor respectively)
type <C-x><C-o> to complete method/properties names
It's specifically explained in the article you linked:
In V7, VIM introduced omni completion – given it is configured to recognize Python (if not, this feature is only a plugin away) Ctrl+x Ctrl+o opens a drop down dialog like any other IDE – even the whole Pydoc gets to be displayed in a split window.

Ctrln is insert-completion.
Ctrlx Ctrlo is omni-completion.
I remap omnicompletion to CtrlSpace:
inoremap <C-Space> <C-x><C-o>
You could also try SuperTab.

I have no idea about the various completion options for Python in Vim. But if you want to roll your own you'd be well advised to study and modify one of the existing ones, like this:
http://www.vim.org/scripts/script.php?script_id=1542
Also, if all your omnicompletion is doing is listing words in current file then you don't have it set up properly for Python-specific completion. . . . Not sure how good the specialized Python completion systems get, but they certainly does compete based on Python units external to your current file. . . .

How do you avoid leaving debugger print/pdb statements in python open source apps when you're debugging them and just finish?

Sometimes when developing using open source software, you need to read it's source code (specially zope/plone). A lot of times I need to write print statements, or debug calls (import pdb), or comment try/except clauses, you name it.
Sometimes I have a lot of files opened when trying to find an issue, and sometimes I forget to remove these print/debug alterations.
So, my question is: how do you keep yourself organized when doing it? Do you write "TODOs" along the modifications and search for them later, do you keep everything opened in your editor and when you find what you were looking for you just revert the files (This approach isn't useful when you're searching for a really big problem that needs days, you need to turn off your computer and return the other day)? Or you just don't do nothing since print statements in development environment is nothing to worry about?
I'm using Vim. I'm just interested to know how other programmers treat this issue.

I used to run into that problem a lot. Now, as part of my check-in process, I run a find/grep script combo that looks for my debugging statements. The only caveat is that I must keep my added debugging statements consistent so grep can find them all.
something like this:
## pre-checkin_scan.bin
find . -name "*.py" -exec grep -H --file=/homes/js/bin/pre-checkin_scan_regexp_list.grep {} \;
## pre-checkin_scan_regexp_list.grep
## (The first pattern is to ignore Doxygen comments)
^##[^#]
pdb
^ *print *( *" *Dbg
^ *print *( *" *Debug
^ *debug

In case of my own projects, the source code is always in version control. Before committing, I always check the graphical diff so that I can see what has changed, what the commit message should be and whether I can split up into smaller commits. That way, I almost always recognize temporary garbage like print statements. If not, I usually notice it shortly afterwards and can do an uncommit if I haven't yet pushed (works for DVCS like git and bzr, not with subversion).
Concerning problems that take multiple days, it's just the same thing. I don't commit until the problem is solved and then look at the diff again.
A text editor that allows editing within the graphical diff view would be really helpful in these cases, but I'm mostly using Eclipse, which doesn't support that.

Well +1 for starting this discussion. Yes sometime this happen to me. I left those pdb and commit the code to the central code base, git. I use 'emacs'. So, Before commit the code I usually search for pdb in the file. But it is hectic checking each file.So, Before committing the code I usually check the diff very carefully. I am also finding the better way to resolve this issue.

I also develop Python with Vim. I have never had to substantially modify the source code for debugging. I do sometimes put debugging print statements, and I have the habit of putting "# XXX" after every one. Then when I want to remove them (before a commit), and just search for the XXX and delete those lines.
For exceptions, I have arranged to run my code in the Vim buffer with an external interpreter that is set up to automatically enter the debugger on any uncaught exception. Then I'm placed automatically in the code at the point the exception occured. I use a modified debugger that can also signal Vim (actually GTK Gvim) to open that source at that line.
Caught exceptions should report meaningful errors, anyway. It is considered by many to be bad practice to do things like:
try:
... some code
except:
handle everything
Since you probably aren't actually handling every possible error case. By not doing that you also enable the automatic debugging.

I can give you three suggestions:
Do not remove debugger statements. By this, I mean leave them in, but make them conditional on being in debug mode:
# Set this to True to enable Debug code
XYZ_Debug = False
if XYZ_Debug:
do_debugging()
Oh, and if the debugging code is just to print things out, you should get familiar with logging (PyMOTW). If you are using logging, you could:
import logging
# Set this to True to enable debug
XYZ_Debug = False
log = logging.getLogger("XYZ")
log.setLevel(logging.DEBUG if XYZ_Debug else logging.INFO)
log.debug("debug output")
Put the same unique tag (in a comment) after each line, or near each block:
do_debug_code() # XYZZY
I then use Emacs' Ibuffer feature, mark all Python buffers then search for occurrences of this tag. Using some combination of find/grep/sed as in other answers would work as well.
If you are using Mercurial and know Mercurial Queues (or might want to learn them), maintain the debug code as a patch in your queue. When you are ready for "production"; or push of the current changes; pop the patch containing the debug code and go. You could achieve something like this outside of version control with diff and patch.

Using cscope to browse Python code with VIM?

Has anyone managed successfully using cscope with Python code? I have VIM 7.2 and the latest version of cscope installed, however it doesn't get my code's tags correctly (always off by a couple of lines). I tried the pycscope script but its output isn't supported by the modern version of cscope.
Any ideas? Or an alternative for browsing Python code with VIM? (I'm specifically interested in the extra features cscope offers beyond the simple tags of ctags)

EDIT: I'm going to run through the process step by step:
Preparing the sources:
exhuberant ctags, has an option: -x
Alternatively, ctags can generate a cross reference file which lists,
in human readable form, information about the various source objects
found in a set of language files.
This is the key to the problem:
ctags -x $(ls **/*.py); # replace with find if no zsh
will give you your database of source objects in a known, format, described under
man ctags; # make sure you use exuberant ctags!
Gnu Global is not limited to only the "out of the box" type of files. Any regular file format will serve.
Also, you can use gtags-cscope, which comes with global as mentioned in section 3.7 of the manual, for a possible shortcut using gtags. You'll end up with an input of a ctags tabular file which Global/gtags can parse to get your objects, or you can use the source for pycscope together with your ctags file of known format to get an input for the vim cscope commands in
if_cscope.txt.
Either way it's quite doable.
Perhaps you'd prefer idutils?
Definintely possible since
z3c.recipe.tags
on pypi makes use of both ctags and idutils to create tag files for a buildout, which is a method I shall investigate in short while.
Of course, you could always use the greputils script below, it has support for idutils , we know idutils works with python, and if that fails, there is also something called vimentry from this year that also uses python, idutils and vim.
Reference links (not complete list):
gtags vimscript, uses Gnu global. updated 2008
greputils vimscript, contains support for the *id idutils, 2005
lid vimscript, Ancient, but this guy is pretty good, his tag and buffer howtos are amazing 2002
An updated version of pyscope, 2010
Hopefully this helps you with your problem, I certainly helped me. I would have been quite sad tonight with a maggoty pycscope.

This seems to work for me:
Change to the top directory of your python code. Create a file called cscope.files:
find . -name '*.py' > cscope.files
cscope -R
You may need to perform a cscope -b first if the cross references don't get built properly.

From a correspondence with the maintainer of cscope, this tool isn't designed to work with Python, and there are no plans to implement that compatibility. Whatever works now, apparently works by mistake, and there is no promise whatsoever that it will keep working.
It appears I've been using an out-of-date version of pycscope. The latest version 0.3 is supported by the cscope DB. The author of pycscope told me that he figured out the output format for the cscope DB from reading the source code of cscope. That format isn't documented, on purpose, but nevertheless it currently works with pycsope 0.3, which is the solution I'll be using.
I'm going to accept this answer since unfortunately no other answer provided help even after bounty was declared. No answers are upvoted, so I honestly have no idea where the bounty will go.

There is a wonderful Python-mode-klen plugin. If you have it and rope (python refactoring library) installed, then going to the definition of a particular term is as simple as <C-c>g or <C-c>rag (first is filetype mapping, second is a global one). There are much more useful features, some useless for me. All of them are disableable. Features from list of questions found at cscope-intro:
Where is this symbol used? <C-c>f. Rather confusing though, as results in quickfix list do show - instead of the actual lines (though they point to the correct location). Maybe it will be fixed.
Where is it defined?, What is this global symbol's definition?, Where is this function in the source files? <C-c>g
What is <...> global symbol's definition? <C-c>raj
Not very much, but I am not too experienced user of ropevim.

I got the same question you got, after browsing the internet, I found a way to fix this:
create a python script: cscope_scan.py
import os
codeRootDir = os.getcwd()
__revision__ = '0.1'
__author__ = 'lxd'
FILE_TYPE_LIST= ['py']
if __name__ == '__main__':
import os
f = open('cscope.files','w')
for root,dirs,files in os.walk(codeRootDir):
for file in files:
for file_type in FILE_TYPE_LIST:
if file.split('.')[-1] == file_type:
f.write('%s\n' %os.path.join(root,file))
f.close()
cmd = 'cscope -bk'
os.system(cmd)
excute this script under you code's root folder, this will generate the cscope.files and then excute cscope -b I don't know what happens to my computer, the last two lines aren't working well, but I think manually type a cscope -bk is acceptable:)

This hack also seems to force cscope to go through Python files:
cscope -Rb -s *
If you accept that cscope is apparently not designed to work with Python.
Superset any language any tool question: How to find all occurrences of a variable in Vim?

Switching from python-mode.el to python.el

I recently tried switching from using python-mode.el to python.el for editing python files in emacs, found the experience a little alien and unproductive, and scurried back. I've been using python-mode.el for something like ten years, so perhaps I'm a little set in my ways. I'd be interested in hearing from anyone who's carefully evaluated the two modes, in particular of the pros and cons they perceive of each and how their work generally interacts with the features specific to python.el.
The two major issues for me with python.el were
Each buffer visiting a python file gets its own inferior interactive python shell. I am used to doing development in one interactive shell and sharing data between python files. (Might seem like bad practice from a software-engineering perspective, but I'm usually working with huge datasets which take a while to load into memory.)
The skeleton-mode support in python.el, which seemed absolutely gratuitous (python's syntax makes such automation unnecessary) and badly designed (for instance, it has no knowledge of "for" loop generator expressions or "<expr 1> if <cond> else <expr 2>" expressions, so you have to go back and remove the colons it helpfully inserts after insisting that you enter the expression clauses in the minibuffer.) I couldn't figure out how to turn it off. There was a python.el variable which claimed to control this, but it didn't seem to work. It could be that the version of python.el I was using was broken (it came from the debian emacs-snapshot package) so if anyone knows of an up-to-date version of it, I'd like to hear about it. (I had the same problem with the version in CVS emacs as of approximately two weeks ago.)

For what it's worth, I do not see the behavior you are seeing in issue #1, "Each buffer visiting a python file gets its own inferior interactive python shell."
This is what I did using python.el from Emacs 22.2.
C-x C-f foo.py
[insert: print "foo"]
C-x C-f bar.py
[insert: print "bar"]
C-c C-z [*Python* buffer appears]
C-x o
C-c C-l RET ["bar" is printed in *Python*]
C-x b foo.py RET
C-c C-l RET ["foo" is printed in the same *Python* buffer]
Therefore the two files are sharing the same inferior python shell. Perhaps there is some unforeseen interaction between your personal customizations of python-mode and the default behaviors of python.el. Have you tried using python.el without your .emacs customizations and checking if it behaves the same way?
The major feature addition of python.el over python-mode is the symbol completion function python-complete-symbol. You can add something like this
(define-key inferior-python-mode-map "\C-c\t" 'python-complete-symbol)
Then typing
>>> import os
>>> os.f[C-c TAB]
you'll get a *Completions* buffer containing
Click <mouse-2> on a completion to select it.
In this buffer, type RET to select the completion near point.
Possible completions are:
os.fchdir os.fdatasync
os.fdopen os.fork
os.forkpty os.fpathconf
os.fstat os.fstatvfs
os.fsync os.ftruncate
It'll work in .py file buffers too.

I can't reproduce this behavior on Emacs v23.1, this must have been changed since then.
Forget about any mode's skeleton support and use the hyper-advanced and extensible yasnippet instead, it's really worth a try!

Note nearly everything said here is obsolete meanwhile as things changed.
python-mode.el commands are prefixed "py-" basically, you should be able to use commands from both, nonewithstanding which one was loaded first.
python-mode.el does not unload python.el; beside of python-mode-map, which is re-defined.
The diff is in the menu displayed and keysetting however, the last one loaded will determine.

python-mode.el is written by the Python community. python.el is written by the emacs community. I've used python-mode.el for as long as I can remember and python.el doesn't even come close to the standards of python-mode.el. I trust the Python community better than the Emacs community to come up with a decent mode file. Just stick with python-mode.el, is there really a reason not to?

python-mode.el has no support triple-quoted strings, so if your program contains long docstrings, all the syntax coloring (and associated syntaxic features) tends to break down.
my .02

Debian has deleted the python-mode package, alas, so I felt compelled to try python.el. I loaded it and ran "describe-bindings". It appeared to be designed for elisp coders who think c-X ; is the intuitive binding for commenting a line of Python code. (Wow.) Also, I found no way at all to comment a region of code, or, rather, no binding with the strings "region" and "comment" in it.
Good old python-mode can still be cloned via git clone https://gitlab.com/python-mode-devs/python-mode.git. It was last edited a week ago at this writing, so it's safe to assume it's not abandoned.