I'm fairly new to Emacs and finally figured out how to set up python environment. I'm using elpy and iPython as python shell interpreter. My question is as follows:
After I press C-c C-c, the buffer on the right shows the executed result. The red box is the code being executed and the blue box is the executed result returned from python. My questions are:
Is there a way only showing the result?
And every time after the script being executed, how to set the buffer on the right only showing the current result, ie, clear previous executed results? Thanks.
Is there a way only showing the result?
elpy already has a custom variable for this:
(setq elpy-shell-echo-input nil)
And every time after the script being executed, how to set the buffer
on the right only showing the current result, ie, clear previous
executed results? Thanks.
elpy doesn't have this functionality. But you can create a wrapper function and bind it to the same keys:
(define-key elpy-mode-map (kbd "C-c C-c")
(lambda (p)
(interactive "P")
(ignore-errors
(with-current-buffer (process-buffer (python-shell-get-process-or-error))
(let ((comint-buffer-maximum-size 0))
(comint-clear-buffer))))
(elpy-shell-send-region-or-buffer p)))
My .emacs file (emacs 23.4.1) contains python and latex related code. For both there is eval-after-load (code that I want to be executed just once when emacs initiates) and hooks. A relevant part of it is:
(setq py-install-directory "~/.emacs.d/python-mode.el-6.1.3")
(add-to-list 'load-path py-install-directory)
(require 'python-mode)
(defun my-eval-after-load-python()
(setq initial-frame-alist '((top . 48) (left . 45) (width . 142) (height . 57)))
(split-window-horizontally (floor (* 0.49 (window-width)))))
(eval-after-load "python-mode" '(my-eval-after-load-python))
All hooks work fine, but my-eval-after-load-python doesn't, which causes the frame to be split into two windows everytime emacs initiates for every extension (for example: emacs file.py, emacs file.tex, emacs file). I tried to change it to:
(eval-after-load "python-mode"
'(progn
(setq initial-frame-alist '((top . 48) (left . 45) (width . 142) (height . 57)))
(split-window-horizontally (floor (* 0.49 (window-width))))
, but it still doesn't work. There's probably a beginner mistake going on here, but I'm unable to find it. How would I split the window just the first time a python script is opened (emacs file.py) and not every time I open a new buffer file2.py?
It sounds like something is causing (load "python-mode") to happen "everytime emacs initiates for every extension" (I'm not sure what you actually mean by that).
Your code is also strange in that you are forcibly loading python-mode with require, and then subsequently evaluating eval-after-load for that same library, even though you know that it's definitely already loaded. This will still work, but it's odd. One tends to use eval-after-load to avoid loading something up front (letting autoloading deal with it on demand, but still having the custom code running at that time).
Edit: Oh, do you just mean that when you start Emacs it evaluates your eval-after-load code? That's because you've told it to -- you loaded python mode, and then told Emacs that if/when python mode is loaded, split the screen in two.
Maybe introduce some boolean
(defvar my-action-done nil)
Put them non-nil following action and
(unless my-action-done
do ...
You do (require 'python-mode) right at the beginning, so python-mode is always loaded even before you get to the (eval-after-load "python-mode" ...) part. I thinkg your my-eval-after-load-python is not meant to be loaded when python-mode is loaded but when python-mode is entered. So you want to use
(add-hook 'python-mode-hook #'my-eval-after-load-python)
From what I have read, there are two ways to debug code in Python:
With a traditional debugger such as pdb or ipdb. This supports commands such as c for continue, n for step-over, s for step-into etc.), but you don't have direct access to an IPython shell which can be extremely useful for object inspection.
Using IPython by embedding an IPython shell in your code. You can do from IPython import embed, and then use embed() in your code. When your program/script hits an embed() statement, you are dropped into an IPython shell. This allows the full inspection of objects and testing of Python code using all the IPython goodies. However, when using embed() you can't step-by-step through the code anymore with handy keyboard shortcuts.
Is there any way to combine the best of both worlds? I.e.
Be able to step-by-step through your code with handy pdb/ipdb keyboard shortcuts.
At any such step (e.g. on a given statement), have access to a full-fledged IPython shell.
IPython debugging as in MATLAB:
An example of this type of "enhanced debugging" can be found in MATLAB, where the user always has full access to the MATLAB engine/shell, and she can still step-by-step through her code, define conditional breakpoints, etc. From what I have discussed with other users, this is the debugging feature that people miss the most when moving from MATLAB to IPython.
IPython debugging in Emacs and other editors:
I don't want to make the question too specific, but I work mostly in Emacs, so I wonder if there is any way to bring this functionality into it. Ideally, Emacs (or the editor) would allow the programmer to set breakpoints anywhere on the code and communicate with the interpreter or debugger to have it stop in the location of your choice, and bring to a full IPython interpreter on that location.
What about ipdb.set_trace() ? In your code :
import ipdb; ipdb.set_trace()
update: now in Python 3.7, we can write breakpoint(). It works the same, but it also obeys to the PYTHONBREAKPOINT environment variable. This feature comes from this PEP.
This allows for full inspection of your code, and you have access to commands such as c (continue), n (execute next line), s (step into the method at point) and so on.
See the ipdb repo and a list of commands. IPython is now called (edit: part of) Jupyter.
ps: note that an ipdb command takes precedence over python code. So in order to write list(foo) you'd need print(list(foo)), or !list(foo) .
Also, if you like the ipython prompt (its emacs and vim modes, history, completions,…) it's easy to get the same for your project since it's based on the python prompt toolkit.
You can use IPython's %pdb magic. Just call %pdb in IPython and when an error occurs, you're automatically dropped to ipdb. While you don't have the stepping immediately, you're in ipdb afterwards.
This makes debugging individual functions easy, as you can just load a file with %load and then run a function. You could force an error with an assert at the right position.
%pdb is a line magic. Call it as %pdb on, %pdb 1, %pdb off or %pdb 0. If called without argument it works as a toggle.
(Update on May 28, 2016) Using RealGUD in Emacs
For anyone in Emacs, this thread shows how to accomplish everything described in the OP (and more) using
a new important debugger in Emacs called RealGUD which can operate with any debugger (including ipdb).
The Emacs package isend-mode.
The combination of these two packages is extremely powerful and allows one to recreate exactly the behavior described in the OP and do even more.
More info on the wiki article of RealGUD for ipdb.
Original answer:
After having tried many different methods for debugging Python, including everything mentioned in this thread, one of my preferred ways of debugging Python with IPython is with embedded shells.
Defining a custom embedded IPython shell:
Add the following on a script to your PYTHONPATH, so that the method ipsh() becomes available.
import inspect
# First import the embed function
from IPython.terminal.embed import InteractiveShellEmbed
from IPython.config.loader import Config
# Configure the prompt so that I know I am in a nested (embedded) shell
cfg = Config()
prompt_config = cfg.PromptManager
prompt_config.in_template = 'N.In <\\#>: '
prompt_config.in2_template = ' .\\D.: '
prompt_config.out_template = 'N.Out<\\#>: '
# Messages displayed when I drop into and exit the shell.
banner_msg = ("\n**Nested Interpreter:\n"
"Hit Ctrl-D to exit interpreter and continue program.\n"
"Note that if you use %kill_embedded, you can fully deactivate\n"
"This embedded instance so it will never turn on again")
exit_msg = '**Leaving Nested interpreter'
# Wrap it in a function that gives me more context:
def ipsh():
ipshell = InteractiveShellEmbed(config=cfg, banner1=banner_msg, exit_msg=exit_msg)
frame = inspect.currentframe().f_back
msg = 'Stopped at {0.f_code.co_filename} at line {0.f_lineno}'.format(frame)
# Go back one level!
# This is needed because the call to ipshell is inside the function ipsh()
ipshell(msg,stack_depth=2)
Then, whenever I want to debug something in my code, I place ipsh() right at the location where I need to do object inspection, etc. For example, say I want to debug my_function below
Using it:
def my_function(b):
a = b
ipsh() # <- This will embed a full-fledged IPython interpreter
a = 4
and then I invoke my_function(2) in one of the following ways:
Either by running a Python program that invokes this function from a Unix shell
Or by invoking it directly from IPython
Regardless of how I invoke it, the interpreter stops at the line that says ipsh(). Once you are done, you can do Ctrl-D and Python will resume execution (with any variable updates that you made). Note that, if you run the code from a regular IPython the IPython shell (case 2 above), the new IPython shell will be nested inside the one from which you invoked it, which is perfectly fine, but it's good to be aware of. Eitherway, once the interpreter stops on the location of ipsh, I can inspect the value of a (which be 2), see what functions and objects are defined, etc.
The problem:
The solution above can be used to have Python stop anywhere you want in your code, and then drop you into a fully-fledged IPython interpreter. Unfortunately it does not let you add or remove breakpoints once you invoke the script, which is highly frustrating. In my opinion, this is the only thing that is preventing IPython from becoming a great debugging tool for Python.
The best you can do for now:
A workaround is to place ipsh() a priori at the different locations where you want the Python interpreter to launch an IPython shell (i.e. a breakpoint). You can then "jump" between different pre-defined, hard-coded "breakpoints" with Ctrl-D, which would exit the current embedded IPython shell and stop again whenever the interpreter hits the next call to ipsh().
If you go this route, one way to exit "debugging mode" and ignore all subsequent breakpoints, is to use ipshell.dummy_mode = True which will make Python ignore any subsequent instantiations of the ipshell object that we created above.
You can start IPython session from pudb and go back to the debugging session as you like.
BTW, ipdb is using IPython behind the scenes and you can actually use IPython functionality such as TAB completion and magic commands (the one starts with %). If you are OK with ipdb you can start it from IPython using commands such as %run and %debug. ipdb session is actually better than plain IPython one in the sense you can go up and down in the stack trace etc. What is missing in ipdb for "object inspection"?
Also, python.el bundled with Emacs >= 24.3 has nice ipdb support.
Looks like the approach in #gaborous's answer is deprecated.
The new approach seems to be:
from IPython.core import debugger
debug = debugger.Pdb().set_trace
def buggy_method():
debug()
Prefixing an "!" symbol to commands you type in pdb seems to have the same effect as doing something in an IPython shell. This works for accessing help for a certain function, or even variable names. Maybe this will help you to some extent. For example,
ipdb> help(numpy.transpose)
*** No help on (numpy.transpose)
But !help(numpy.transpose) will give you the expected help page on numpy.transpose. Similarly for variable names, say you have a variable l, typing "l" in pdb lists the code, but !l prints the value of l.
You can start IPython from within ipdb.
Induce the ipdb debugger1:
import idpb; ipdb.set_trace()
Enter IPython from within in the ipdb> console2:
from IPython import embed; embed()
Return to the ipdb> console from within IPython:
exit
If you're lucky enough to be using Emacs, things can be made even more convenient.
This requires using M-x shell. Using yasnippet and bm, define the following snippet. This will replace the text ipdb in the editor with the set-trace line. After inserting the snippet, the line will be highlighted so that it is easily noticeable and navigable. Use M-x bm-next to navigate.
# -*- mode: snippet -*-
# name: ipdb
# key: ipdb
# expand-env: ((yas-after-exit-snippet-hook #'bm-toggle))
# --
import ipdb; ipdb.set_trace()
1 All on one line for easy deletion. Since imports only happen once, this form ensures ipdb will be imported when you need it with no extra overhead.
2 You can save yourself some typing by importing IPython within your .pdbrc file:
try:
from IPython import embed
except:
pass
This allows you to simply call embed() from within ipdb (of course, only when IPython is installed).
Did you try this tip?
Or better still, use ipython, and call:
from IPython.Debugger import Tracer; debug_here = Tracer()
then you can just use
debug_here()
whenever you want to set a breakpoint
the right, easy, cool, exact answer for the question is to use %run macro with -d flag.
In [4]: run -d myscript.py
NOTE: Enter 'c' at the ipdb> prompt to continue execution.
> /cygdrive/c/Users/mycodefolder/myscript.py(4)<module>()
2
3
----> 4 a=1
5 b=2
One option is to use an IDE like Spyder which should allow you to interact with your code while debugging (using an IPython console, in fact). In fact, Spyder is very MATLAB-like, which I presume was intentional. That includes variable inspectors, variable editing, built-in access to documentation, etc.
If you type exit() in embed() console the code continue and go to the next embed() line.
The Pyzo IDE has similar capabilities as the OP asked for. You don't have to start in debug mode. Similarly to MATLAB, the commands are executed in the shell. When you set up a break-point in some source code line, the IDE stops the execution there and you can debug and issue regular IPython commands as well.
It does seem however that step-into doesn't (yet?) work well (i.e. stopping in one line and then stepping into another function) unless you set up another break-point.
Still, coming from MATLAB, this seems the best solution I've found.
From python 3.2, you have the interact command, which gives you access to the full python/ipython command space.
Running from inside Emacs' IPython-shell and breakpoint set via pdb.set_trace() should work.
Checked with python-mode.el, M-x ipython RET etc.
Developing New Code
Debugging inside IPython
Use Jupyter/IPython cell execution to speed up experiment iterations
Use %%debug for step through
Cell Example:
%%debug
...: for n in range(4):
...: n>2
Debugging Existing Code
IPython inside debugging
Debugging a broken unit test: pytest ... --pdbcls=IPython.terminal.debugger:TerminalPdb --pdb
Debugging outside of test case: breakpoint(), python -m ipdb, etc.
IPython.embed() for full IPython functionality where needed while in the debugger
Thoughts on Python
I agree with the OP that many things MATLAB does nicely Python still does not have and really should since just about everything in the language favors development speed over production speed. Maybe someday I will contribute more than trivial bug fixes to CPython.
https://github.com/ipython/ipython/commit/f042f3fea7560afcb518a1940daa46a72fbcfa68
See also Is it possible to run commands in IPython with debugging?
If put import ipdb; ipdb.set_trace() at cell outside function, it will occur error.
Using %pdb or %debug, you can only see the filnal error result. You cannot see the code doing step by step.
I use following skill:
%%writefile temp.py
.....cell code.....
save the code of cell to file temp.py.
and then
%run -i -d temp.py, it will run the cell code by pdb .
-i: run the file in IPython’s namespace instead of an empty one.
-d: run your program under the control of pdb, the Python debugger.
When I am debugging code in Emacs, I like to have two buffers open, the first one with the source code I am debugging, and the second one with the debugger (pdb for Python).
I have the following keyboard shortcuts defined on my .emacs file:
(require 'gud)
(define-key gud-mode-map '[C-f10] 'gud-next)
(define-key gud-mode-map '[C-f11] 'gud-step)
(define-key gud-mode-map '[C-f5] 'gud-cont)
(define-key gud-mode-map '[C-f12] 'gud-break)
With the above, I can trigger the GUD shortcuts for gud-next, gud-step, etc. from the buffer where pdb is running, but I can't trigger them from the buffer that has the python code.
I would like to use keyboard shortcuts on the buffer with the source code to trigger GUD commands for the debugger. Is there any way to do this?
I am using the most recent version of python-mode (6.0.4) and Emacs 23.3.1.
Try using global-set-key instead:
(global-set-key [C-f10] 'gud-next)
(global-set-key [C-f11] 'gud-step)
(global-set-key [C-f5] 'gud-cont)
(global-set-key [C-f12] 'gud-break)
IIRC, this worked for me.
I am a fairly proficient vim user, but friends of mine told me so much good stuff about emacs that I decided to give it a try -- especially after finding about the aptly-named evil mode...
Anyways, I am currently working on a python script that requires user input (a subclass of cmd.Cmd). In vim, if I wanted to try it, I could simply do :!python % and then could interact with my script, until it quits. In emacs, I tried M-! python script.py, which would indeed run the script in a separate buffer, but then RETURNs seems not to be sent back to the script, but are caught by the emacs buffer instead. I also tried to have a look at python-mode's C-c C-c, but this runs the script in some temporary directory, whereas I just want to run it in (pwd).
So, is there any canonical way of doing that?
I don't know about canonical, but if I needed to interact with a script I'd do M-xshellRET and run the script from there.
There's also M-xterminal-emulator for more serious terminal emulation, not just shell stuff.
I like to use the Emacs "compile" command to test/run my python scripts. M-XcompileRET will pull up the default "make -k" but if you delete that and put in the command line for your script (including options), subsequent "compiles" will provide the new "compile" command automatically. All the output from your script will appear in the compile buffer. (As opposed to the shell, this provides a nice clean buffer each time it is invoked. Good for searching and such. If you forget to save your script before your run, compile will ask you if you would like to save the file.)
You will lose your the command line when you restart Emacs. But you can get Emacs to set the compile-command for the buffer holding your script by putting at the bottom of the python script this sort of code (actually a python comment):
# Trigger emacs to run this script using the "compile" command
# ;;; Local Variables: ***
# ;;; compile-command: "my_cool_script.py --complicated_option some_filename.txt" ***
# ;;; end: ***
This is handy for scripts with complicated invocations.
Note: The python comment character '#' protects this from the python interpreter while Emacs knows to set these variables because it looks at the bottom of every file when it opens them.
I'd love to be able to jump to 'compile errors' in my python script the way the compile command does when you use it for compiling C code but I'm too lazy to create the Emacs regular expression to make this work. Perhaps that would make another great question for stack overflow!
I currently use these hook to define my compilation commands:
(defun convert-filename-to-executable (file)
(if (eq system-type 'windows-nt)
(concat (file-name-sans-extension file) ".exe")
;; linux
(concat "./" (file-name-sans-extension file))))
(add-hook 'c++-mode-hook
(lambda ()
(unless (file-exists-p "Makefile")
(set (make-local-variable 'compile-command)
(let* ((file (file-name-nondirectory buffer- file-name))
(executable (convert-filename-to-executable file)))
(concat "g++ -g -Wall -o "
(file-name-sans-extension file)
" "
file
" && "
executable))))))
(add-hook 'c-mode-hook
(lambda ()
(unless (file-exists-p "Makefile")
(set (make-local-variable 'compile-command)
(let* ((file (file-name-nondirectory buffer-file-name))
(executable (convert-filename-to-executable file)))
(concat "gcc -g -ansi -Wall -Wpedantic -Wextra -Wc++-compat -Wconversion -o "
(file-name-sans-extension file)
" "
file
" && "
executable))))))
(add-hook 'python-mode-hook
(lambda ()
(set (make-local-variable 'compile-command)
(concat "python " buffer-file-name))))
(add-hook 'perl-mode-hook
(lambda ()
(set (make-local-variable 'compile-command)
(concat "python " buffer-file-name))))
Together with this lambda set to call the compile function interactively:
(global-set-key (kbd "<f4>") (lambda () (interactive) (setq current-prefix-arg '(4)) (call-interactively 'compile)))
One button to rule them all!
If you press F4 (in my case, you can set the key yourself in the lambda for global-set-key), then a file opened in C++ or C mode will be compiled and a file in python or perl mode will be run (interactively)
The I think ansi-term has the most faithful emulation of a terminal. But I don't see a way to pass arguments to the process. You can of course just launch it from a shell inside the ansi-term buffer.
But I think the best thing to do is to not use python-send-buffer, but instead to use a new function which does it "right", that is by sending the path to the current file instead of making a temp file. There are some slight differences of course in that you have to save the current file first, but the following should at least get you on the right track.
(defun python-send-file ()
(interactive)
(save-buffer)
(python-send-string (concat "execfile('" (buffer-file-name) "')")))
;; This overwrites the `python-send-buffer' binding so you may want to pick another key
(eval-after-load "python"
(define-key python-mode-map "\C-c\C-c" 'python-send-file))
I checked and this allows you to interact. To get tabs you have a few options.
C-qTAB will always give you a literal tab
You can rebind tab to be a literal tab in inferior-python-mode-map:
(define-key inferior-python-mode-map "\C-i" 'self-insert-command)
I'm sure there are others that I can't think of
If you use C-c C-c a buffer is created (look for inferior-python). Try changing to that buffer*, every time you hit C-c C-c the result is shown there, you need to see that buffer to get the results. Use C-x 2 so you can see both buffers at the same time.
Also try C-c C-z (switch to shell).
*I use Ibuffer to manage buffers, is very good.
(btw, this http://tuhdo.github.io/index.html is an excelent place to learn some emacs)
EDIT: Have you tried C-c C-p ?
Yet another option:
Using C-c C-c works nicely with fgallina's python.el -- pwd will be the location of the buffer's file.