I am working on a project whereby I need to embed Python within a Verilog file. The Python isn't really intended for execution in the normal sense as it will be read by a secondary tool. The Python will be written blocks that have some fixed demarcation (such as #+BEGIN_SRC, in org-babel).
module name ();
#+BEGIN_SRC python
def my_function ():
...
#+END_SRC
always #(posedge clk)
...
endmodule
Within Emacs this causes havoc, although Python-mode and Verilog-mode work fine, when combining both in the same file things quickly break-down as one would expect. Indentation is hopelessly broken as is syntax-highlighting. I understand this is a very weird thing to do, and I understand that there will almost certainly never been any real need to do this under normal circumstances, however for this particular case it is necessary.
My question: is there anyway within Emacs to specify multiple major modes within the same file. For example, is there some way that I can write a file using Verilog-mode as my major mode, but use Python-mode within the predefined blocks that are then ignored in the reset of the file.
There are a number of possibilities listed here:
http://emacswiki.org/emacs/MultipleModes
I've used multi-mode with latex and haskell, and it works OK.
Related
Being in the process of switching from Matlab to Python/IPython/Spyder I ran into the following issue.
In a Matlab script (and I actually did not notice this until I switched to Python) there is no warning if you are using a variable which you did not define in that script.
What I kept doing all the times in Matlab was to have, say, 2 (or more) scripts which I would then execute one after the other. The second script would typically use variables defined in the first, but would not give me any warning.
In Spyder I noticed that the situation is different. In my hypothetical second script, all the variables which are not defined in the second script itself give me a warning (undefined name '...'), which is not nice too see...
Another typical example, would be the following: I have a main script but there is something in this script I wanna look at better. So, without touching the script, I would create another file where I would copy paste a few lines from the script to play a little with them.
In this new file I would be using variables which are in the console already but that are techically unknown to the file itself, so would give me a warning.
So I guess I am simply asking if there a way to suppress this kind of warnings.
Or there might be a deeper question of whether there s a more pythonic way of working, but there I am really not sure about...
For various reasons, in one project I generate executable code by means of generating AST from various source files the compiling that to bytecode (though the question could also work for cases where the bytecode is generated directly I guess).
From some experimentation, it looks like the debugger more or less just uses the lineno information embedded in the AST alongside the filename passed to compile in order to provide a representation for the debugger's purposes, however this assumes the code being executed comes from a single on-disk file.
That is not necessarily the case for my project, the executable code can be pieced together from multiple sources, and some or all of these sources may have been fetched over the network, or been retrieved from non-disk storage (e.g. database).
And so my Y questions, which may be the wrong ones (hence the background):
is it possible to provide a memory buffer of some sort, or is it necessary to generate a singular on-disk representation of the "virtual source"?
how well would the debugger deal with jumping around between the different bits and pieces if the virtual source can't or should not be linearised[0]
and just in case, is the assumption of Python only supporting a single contiguous source file correct or can it actually be fed multiple sources somehow?
[0] for instance a web-style literate program would be debugged in its original form, jumping between the code sections, not in the so-called "tangled" form
Some of this can be handled by the trepan3k debugger. For other things various hooks are in place.
First of all it can debug based on bytecode alone. But of course stepping instructions won't be possible if the line number table doesn't exist. And for that reason if for no other, I would add a "line number" for each logical stopping point, such as at the beginning of statements. The numbers don't have to be line numbers, they could just count from 1 or be indexes into some other table. This is more or less how go's Pos type position works.
The debugger will let you set a breakpoint on a function, but that function has to exist and when you start any python program most of the functions you define don't exist. So the typically way to do this is to modify the source to call the debugger at some point. In trepan3k the lingo for this is:
from trepan.api import debug; debug()
Do that in a place where the other functions you want to break on and that have been defined.
And the functions can be specified as methods on existing variables, e.g. self.my_function()
One of the advanced features of this debugger is that will decompile the bytecode to produce source code. There is a command called deparse which will show you the context around where you are currently stopped.
Deparsing bytecode though is a bit difficult so depending on which kind of bytecode you get the results may vary.
As for the virtual source problem, well that situation is somewhat tolerated in the debugger, since that kind of thing has to go on when there is no source. And to facilitate this and remote debugging (where the file locations locally and remotely can be different), we allow for filename remapping.
Another library pyficache is used to for this remapping; it has the ability I believe remap contiguous lines of one file into lines in another file. And I think you could use this over and over again. However so far there hasn't been need for this. And that code is pretty old. So someone would have to beef up trepan3k here.
Lastly, related to trepan3k is a trepan-xpy which is a CPython bytecode debugger which can step bytecode instructions even when the line number table is empty.
I'm working on a plug-in for Vim, and I'd like to test that it behaves correctly, under start-up, when users edit files e.t.c.
To do this, I'd like to start a terminal, and feed keys in to it.
I'm thinking of doing it all from a python script. Is there a way to do this?
In pseudo-python it might look something like this:
#start a terminal. Here konsole
konsole = os.system('konsole --width=200 --height=150')
#start vim in that terminal
konsole.feed_keys("vim\n")
#run the vim function to be tested
konsole.feed_keys(":let my_list = MyVimFunction()\n")
#save the return value to the file system
konsole.feed_keys(":writefile(my_list, '/tmp/result')\n")
#load result into python
with open('/tmp/result', 'r') as myfile:
data = myfile.read()
#validate the result
assertEqual('expect result', data)
I think you should verify the core functionality of your plugin inside Vim, using unit tests. There's a wide variety of Vim plugins, but most provide some additional mappings or commands, to be invoked by the user, and they usually leave behind some side effects in the buffer, or output, or opened windows. That can be verified from inside Vim. There are a various approaches for that, mine is the runVimTests test framework; the plugin page has links to several alternatives.
With the core functionality thus covered, there's little left to test "interactively". (I mean stuff like forgotten debug output, too long execution times, display mess-ups.) Since you're usually a heavy user of Vim and your plugin yourself, that mostly covers it.
Of course, if your plugin embeds itself tightly into Vim (like an "IDE for XXX"; though this is usually frowned upon), you may consider some external test driver. Maybe others will contribute pointers to some general-purpose, terminal-driven test frameworks. I'm almost sure such exist.
While I'm maintaining a plugin that permits to run unit tests on VimL functions and feed the quickfix window with the results, I use another couple of tools to check the state of the buffer after some actions, and even run the thing from travis -> vimrunner+rspec, and VimFlavour for installing the dependencies. (I vaguely remember a Python alternative inspired by vimrunner)
It mostly works well. Alas it uses the client-server feature and :redir (instead of the more recent execute() function). Even with the use of :silent, :redir catches noise which it returns to the client. Thus sometimes I fight tests that fail for very odd reasons. I also find myself inserting some pseudo-pauses to be sure that Vim has finished to interpret what I've feed it.
You'll find example of use in some of my plugins. See for instance lh-brackets or lh-cpp tests (.travis.yml file + .rspec/ directory + Rakefile + Gemfile + some helpers from vim-UT)
Is there a debugger that can debug the Python virtual machine while it is running Python code, similar to the way that GDB works with C/C++? I have searched online and have come across pdb, but this steps through the code executed by the Python interpreter, not the Python interpreter as its running the program.
The reference implementation of Python, CPython, is written in C. You can use GDB to debug it as you would debug any other program written in C.
That said, Python does have a few little helpers for use in GDB buried under Misc/gdbinit. It's got comments to describe what each command does, but I'll repeat them here for convenience:
pyo: Dump a PyObject *.
pyg: Dump a PyGC_Head *.
pylocals: Print the local variables of the current Python stack frame.
lineno: Get the current Python line number.
pyframe: Print the source file name, line, and function.
pyframev: pyframe + pylocals
printframe: pyframe if within PyEval_EvalFrameEx; built-in frame otherwise
pystack: Print the Python stack trace.
pystackv: Print the Python stack trace with local variables.
pu: Print a Unicode string.
It looks like the Fedora project has also assembled their own collection of commands to assist with debugging which you may want to look at, too.
If you're looking to debug Python at the bytecode level, that's exactly what pdb does.
If you're looking to debug the CPython reference interpreter… as icktoofay's answer says, it's just a C program like any other, so you can debug it the same way as any other C program. (And you can get the source, compile it with extra debugging info, etc. if you want, too.)
You almost certainly want to look at EasierPythonDebugging, which shows how to set up a bunch of GDB helpers (which are Python scripts, of course) to make your life easier. Most importantly: The Python stack is tightly bound to the C stack, but it's a big mess to try to map things manually. With the right helpers, you can get stack traces, frame dumps, etc. in Python terms instead of or in parallel with the C terms with no effort. Another big benefit is the py-print command, which can look up a Python name (in nearly the same way a live interpreter would), call its __repr__, and print out the result (with proper error handling and everything so you don't end up crashing your gdb session trying to walk the PyObject* stuff manually).
If you're looking for some level in between… well, there is no level in between. (Conceptually, there are multiple layers to the interpreter, but it's all just C code, and it all looks alike to gdb.)
If you're looking to debug any Python interpreter, rather than specifically CPython, you might want to look at PyPy. It's written in a Python-like language called RPython, and there are various ways to use pdb to debug the (R)Python interpreter code, although it's not as easy as it could be (unless you use a flat-translated PyPy, which will probably run about 100x too slow to be tolerable). There are also GDB debug hooks and scripts for PyPy just like the ones for CPython, but they're not as complete.
I'm writing a script to extract some useful data about a series of chemical simulations I've been running.
To get this data I need (1) a C-program that calculates the density from a file type called *.pdb. I already have (1). And (2) I need to use a program called vmd to get that pdb. In order to accomplish (2) from the command line, I can submit a tcl script, as vmd has a build in tcl interpreter.
These functions -- calling the vmd to run the tcl script, then running the compiled c-program -- will be the key activities of my wrapper data extraction script.
I would like to eliminate the superfluous TCL script, reducing my count from 2 scripts (wrapper script + tcl script for vmd) down to 1. But I'm not sure quite how to do this. One potentially solution seems to be to embed my TCL script within my wrapper script, if there's a way to make such an embedded script callable from external programs.
Most of my data collection scripts so far have been in BASH, so ideally I would like to stick to a BASH script as I'm very familiar with bash scripting versus having only beginning knowledge of Python/Perl.
Here are my questions:
1. Can you embed a TCL script inside a Bash script?
2. Can you make this script callable by an external program?
e.g. in pseudocode:
#!/bin/bash
....
tclembed extract {
#tcl script
...
}
...
vmd -dispdev text -e extract.tcl >& extract_results.log #where vmd is
#an external program
3. If the answer to #2 is no, can you do this in Python, perhaps with the Minotaur library? I would consider the switch to python, if so...
http://markmail.org/message/6kogjphzqtn4ilch
4. If not, how would you suggest trying to merge these two scripts (a tcl routine and a bash script that calls it) into a single file?
5. If anybody HAS gotten external calls of this nature to work using Minotaur, can you post some explanatory code?
I've thought of one non-embedding solution which to #4, which would be to write a function in my Bash script that writes a file with the entire tcl script. That way I would have a single script, but could dump the subscript for use with external programs, later deleting it. I have a feeling this solution is kinda kludgy though I know for sure that it works, vs. embedded solutions.
There have been several Tcl-Python alloys. As Rafe Kettler's comment above sketches, the place to start is with a standard Python installation. This includes Tkinter, which builds in a full Tcl interpreter, accessible as described in the Wiki page mentioned. So, yes, it is feasible to "do this in Python".
I really don't get what this has to do with vmd, though. vmd builds in a Tcl interpreter already. While I entirely support the aim of "reduction of moving parts", so that you have, for example, one script, rather than two, coding something in Python, when vmd already exposes Tcl, doesn't seem like a step in the direction Jason R. Mick wants to go.
SOMEWHAT LATER: after an exchange of comments with Jason R. Mick, it occurred to me he might find
#!/bin/bash
echo "Here's a bit of bash-iness."
MYSCRIPT='
puts "Here I am, inside Tcl."
puts "See? I can do calculations: [expr 3 + 5]."
exit 0
'
tclsh << HERE
$MYSCRIPT
HERE
suggestive. Its output, of course, is
Here's a bit of bash-iness.
Here I am, inside Tcl.
See? I can do calculations: 8.
I wrote this in terms of tclsh, but, if I'm keeping up, Jason R. Mick will actually want to use vmd. The appropriate homologue for *vmd is something like
...
vmd -dispdev text -eofexit << HERE > output.log
$MYSCRIPT
HERE
While I can think of several other ways to meld bash and Tcl, I believe this one is most in the spirit of the original question.
I want to note, too, that, from the little I know of vmd, it would be entirely appropriate to do the same with Python in place of Tcl: vmd is equally adept with either.
Finally, both Python and Tcl are general-purpose languages, with approximately the same power as bash, so yet another direction to take this project would be to write it entirely in Tcl (or Python), rather than bash. Embedding scripts in the way illustrated above is at least as easy in Tcl (or Python) as in bash.
1. Can you embed a TCL script inside a Bash script?
Not easily. The best way is to write the script to a temporary file and pass the name of that file to tclsh (or wish if it is a Tcl/Tk program). That should be a "simple matter of programming", i.e., some awkward coding but not fundamentally hard.
2. Can you make this script callable by an external program?
I don't quite understand what you want to do here. You can put a #! line at the start of a Tcl script and mark the file executable. That works well. The best way of all to do that is this:
#!/usr/bin/env tclsh8.5
your tcl script here...
3. If the answer to #2 is no, can you do this in Python?
This wiki page mentions something called Typcl, which is reported to allow doing Tcl from inside Python. I have never tried it.
(I think questions 4 and 5 are largely irrelevant based on my answers above.)