nosetests --pdb let's me halt upon error or failure, but this is too late for my needs. Stepping through code during execution helps me debug where the problem is.
However, nosetests are helpful as they allow tests that rely on relative imports (i.e. tests in a package).
How can I set breakpoints before the tests are executed?
Currently I'm using:
python -m pdb /path/to/my/nosetests testfile.py
This solution isn't adequate. Nosetests interfere with pdb output, and my keyboard controls (e.g. arrow keys) are broken.
Using import pdb; pdb.set_trace() would seem like a good idea, however nosetests is blocking my access to the pdb console.
Even better than remembering to use -s is to use the set_trace variant that comes with Nose. Add
from nose.tools import set_trace; set_trace()
wherever you'd like to break in to the debugger. The stdin/out redirection will be taken care of for you. The only strange side effect I've run into is the inability to restart your code from within pdb (using run) while debugging during a nose run.
You can add
import pdb; pdb.set_trace()
anywhere in your source that you want to stop in the debugger.
Make sure you pass -s to nose so that it does not capture stdout.
If you have ipython, for unlimited awesomeness use:
import ipdb; ipdb.set_trace()
*unlimited awesomeness: just like ipython - auto-completion, coloring etc.
If you are using pytest, you can use
import pytest; pytest.set_trace()
See documentation.
Related
I saw this question: Can I debug with python debugger when using py.test somehow? but it doesn't really help, because I need to debug hooks, some of them not written by me, where modifying the code of the hook is really cumbersome.
Also, pytest runs through pipenv run. It's already difficult to make them both work together. I couldn't so far find a combination of pdb, pipenv and pytest that would launch each other.
Another way I could do it is by calling pytest.main() from my code, however, this means that other people who want to run my tests will have to use this "trampoline" to run other tests. I can live with this, but it still feels like it shouldn't be necessary.
I guess this is what you need, invoke pdb as early as possible:
`pipenv --py` -c 'import pdb, pytest; pdb.set_trace(); pytest.main()'
When writing python code (mostly numpy + matplotlib), I usually just type the code in vim and run the program to test it:
python2 foo.py
Occasionally, when this is not sufficient and I need to inspect the problem more thoroughly, I just launch the program in ipython:
ipython -pylab foo.py, and then inspect the variables, test some commands and so on. I like ipython, because of the tab completion and the availability of bash commands.
This worked well enough for me, but now my programs grew bigger and include many subroutines (in multiple files). The ipython approach doesn't work any more, because it always runs the complete code till the end of foo.py (when it drops into the pylab shell). What I'd like to do instead is, stop execution at a given line in a subroutine (could be in another file) and inspect variables there. I.e. set a break point at which the pylab shell kicks in.
Is there an easy way to adapt my ipython way of working? E.g. stop at a line in bar.py
ipython -pylab --stop-at bar.py:423 foo.py
or, stop at a subroutine name in bar.py
ipython -pylab --stop-at bar.py:subroutine-name foo.py
You can import the pdb module into the code and then add a pdb.set_trace() call where you would like to have the code stop. Ipython will drop into the interactive debugger, and you are free to step though your code as you wish.
You can drop into a IPython debug session by inserting the following code at the desired point:
import sys, IPython
IPython.Shell.IPShell(argv=[])
IPython.Debugger.Pdb(IPython.ipapi.get().options.colors).set_trace(sys._getframe())
Not exactly what you seem to be looking for, but it works quite nicely for me. It also makes it really easy to have complex conditional breakpoints. There are several other methods of starting an IPython debug session from within your source file floating around on the web, but this - in my experience anyway - is the most reliable in terms of loading the correct colours, tab completion working properly etc.
Once the debug session has started you can set further breakpoints using the break command:
ipdb> break test.py:11
Breakpoint 1 at /tmp/test.py:11
ipdb> b my_function
Breakpoint 2 at /tmp/test.py:5
To make it easy to insert, you can set a macro/key combination in your editor. I'm also a Vim user and I have the following keymap in my vimrc:
nmap <C-P><C-D> oimport sys, IPython<CR>IPython.Shell.IPShell(argv=[])<CR>IPython.Debugger.Pdb(IPython.ipapi.get().options.colors).set_trace(sys._getframe())<ESC>:w<CR>
From normal mode, pressing Ctrl-P then Ctrl-D inserts the debug code after the current line with the correct indentation and then saves the file.
I use emacs for all my code edit needs. Typically, I will use M-x compile to run my test runner which I would say gets me about 70% of what I need to do to keep the code on track however lately I've been wondering how it might be possible to use M-x pdb on occasions where it would be nice to hit a breakpoint and inspect things.
In my googling I've found some things that suggest that this is useful/possible. However I have not managed to get it working in a way that I fully understand.
I don't know if it's the combination of buildout + appengine that might be making it more difficult but when I try to do something like
M-x pdb
Run pdb (like this): /Users/twillis/projects/hydrant/bin/python /Users/twillis/bin/pdb /Users/twillis/projects/hydrant/bin/devappserver /Users/twillis/projects/hydrant/parts/hydrant-app/
Where .../bin/python is the interpreter buildout makes with the path set for all the eggs.
~/bin/pdb is a simple script to call into pdb.main using the current python interpreter
HellooKitty:hydrant twillis$ cat ~/bin/pdb
#! /usr/bin/env python
if __name__ == "__main__":
import sys
sys.version_info
import pdb
pdb.main()
HellooKitty:hydrant twillis$
.../bin/devappserver is the dev_appserver script that the buildout recipe makes for gae project and .../parts/hydrant-app is the path to the app.yaml
I am first presented with a prompt
Current directory is /Users/twillis/bin/
C-c C-f
Nothing happens but
HellooKitty:hydrant twillis$ ps aux | grep pdb
twillis 469 100.0 1.6 168488 67188 s002 Rs+ 1:03PM 0:52.19 /usr/local/bin/python2.5 /Users/twillis/projects/hydrant/bin/python /Users/twillis/bin/pdb /Users/twillis/projects/hydrant/bin/devappserver /Users/twillis/projects/hydrant/parts/hydrant-app/
twillis 477 0.0 0.0 2435120 420 s000 R+ 1:05PM 0:00.00 grep pdb
HellooKitty:hydrant twillis$
something is happening
C-x [space]
will report that a breakpoint has been set. But I can't manage to get get things going.
It feels like I am missing something obvious here. Am I?
So, is interactive debugging in emacs worthwhile? is interactive debugging a google appengine app possible? Any suggestions on how I might get this working?
Hmm. You're doing this a little differently than I do. I haven't experimented with your method. I use the pdb library module directly, with no wrapper script, just using the "-m" python command-line option to tell python to run the module as a script.
To be excessively thorough, here's my sequence of operations:
I hit Alt-X in EMACS, type "pdb", then return.
EMACS prompts me with "Run pdb (like this):" and I type "python -m pdb myprogram.py".
EMACS creates a debugger mode window for pdb, where I can give the debugger commands, and tracks the execution of the program in the source code.
I suppose it's possible there's some reason this doesn't work well with the appengine. I recommend getting it working first with a trivial python program and once you know that's working, try stepping up to the full app.
In practice, I don't do much python debugging with pdb. Most of my debugging is essentially "printf debugging", done inserting print statements into my unit tests and (occasionally) into the actual code.
I'm working in a project that recently switched to the pytest unittest framework. I was used to calling my tests from Eclipse, so that I can use the debugger (e.g. placing breakpoints to analyze how a test failure develops). Now this is no longer possible, since the only way to run the tests is via the command line blackbox.
Is there some way to use pytest from within Python, so that one is not forced to drop out of the IDE? The tests should of course not be run in a separate process.
I think I can now answer my own question, it's pretty simple:
import pytest
pytest.main(args)
which is documented in the Section "Calling pytest from Python code".
Then I can run this module and/or start it with the integrated debugger.
args is the list of command-line arguments, so for example to run only particular tests I can use something like:
args_str = "-k test_myfavorite"
args = args_str.split(" ")
pytest.main(args)
It seems that now (py.test version 2.0+) someone can also do this :
import pytest
pytest.main('-x {0}'.format(argument))
# Or
# pytest.main(['-x', 'argument'])
Ref
This is now supported by pytest and described nicely in the documentation.
You can invoke pytest from Python code directly:
import pytest
pytest.main()
this acts as if you would call “pytest” from the command line. It will not raise SystemExit but return the exitcode instead. You can pass in options and arguments:
pytest.main(["-x", "mytestdir"])
For me it was this:
pytest.main(["-x", "path to test file", "args"])
For example:
import pytest
pytest.main(["-x", "/api/test", "-vv"])
Maybe you could give a try to pycharm it has direct integration with py.test (I use it at work) and debugger runs perfectly.
I have not tried with eclipse, but as was suggested in a related question, it is possible to use the --pdb command line option with py.test. Maybe it is possible to configure eclipse that way.
However, calling the standard import pdb;pdb.set_trace() will not directly call the debugger. First it will issue an error which in turn will activate the debugger. This might or might not make things work differently.
You can just run py.test --pdb if you just want to a debugger and don't need the IDE
Are there any ways to debug python scripts not leaving vim in *nix systems (executing the script, setting up breakpoints, showing variables in watch-list, etc)?
Use pdb:
import pdb
def main():
list = [1,2,3]
pdb.set_trace()
list = [2,3,4]
if __name__ == '__main__':
main()
Now run using :!python % and you'll hit your breakpoint and be able to debug interactively like in gdb.
As of Python 3.7, you can use breakpoint() builtin without importing anything.
Built-in breakpoint() calls sys.breakpointhook(). By default, the latter imports pdb and then calls pdb.set_trace()
Inheriting code from Pierre-Antoine's answer, the code would look like this:
def main():
list = [1,2,3]
breakpoint()
list = [2,3,4]
if __name__ == '__main__':
main()
Source: https://docs.python.org/3/whatsnew/3.7.html#pep-553-built-in-breakpoint
Try pyclewn. It allows to use vim as front end for pdb. You can create/delete break points, control flow of debugging process, look at values of your variables. All from vim!
Also try https://pypi.python.org/pypi/pudb - its like pdb but more advanced. Contains code highlighting, stack, showing avaliable values, etc. Not only-vim solution but for me works perfectly.
Three Steps:
Install:
pip install pudb
Paste set_trace in code
from pudb import set_trace; set_trace()
Run your code
As 2020 the Debugger Adapter Protocol is taken care by vimspector.
Supporting Cpp, Python, Java, Js, Go ...
See my other answer
The vimpdb plugin integrates the Python debugger pdb into the VIM editor.
I do recommend it.
Hope it helps.
Vim and pdb-clone is the combination I use. I use Home - pyclewn which provides a replacement for pdb called pdb-clone that is quite faster than vanilla pdb. It integrates well with vim via a plugin, and the thing I appreciate most is that it takes care of breakpoints outside the code, not setting traces within, thus not messing up my line numbers. It does not have a watch window for python yet. You might have a look at vim-debug too, which I could not get to work with my existing highlighting setup.
See the "Debugging" section in this blog post. It shows how to setup F7 to set breakpoints and Shift+F7 to remove breakpoints. It also uses pdb, as mentioned before. With a little modification, you can replace the use of pdb with ipdb (pdb using ipython), which is a lot nicer to use.
It sounds like you want to use VIM as a Python IDE.
A quick Google search found this and this example, with many more.
EDIT: Well, Ok, it seems likely you've searched more than I have.
I hope someone else has some ideas.
From what I know, there is one more option: You could use Eclipse + PyDev for project managing and Vim as an editor for Eclipse. That way You could use the best of both worlds.
Also, I haven't tried it, but You could try this script.