The title says it all. When you are working R and using RStudio, its really easy and simple to debug something by dropping a browser() call anywhere in your code and seeing what goes wrong. Is there a way to do that with Python? I'm slowly getting very sick of print statement debugging.
It looks like you are looking for ipdb
The basic usage is to set:
import ipdb
ipdb.set_trace()
in your code to explore; this will take you right to that part of code, so you can explore all the variables at that point.
For your specific use case: "Would it be a setting in my Console so that it Opens pdb right before something crashes" (a comment to another answer), you can use context manager: launch_ipdb_on_exception
For example:
from ipdb import launch_ipdb_on_exception
def silly():
my_list = [1,2,3]
for i in xrange(4):
print my_list[i]
if __name__ == "__main__":
with launch_ipdb_on_exception():
silly()
Will take you to ipdb session:
5 for i in xrange(4):
----> 6 print my_list[i]
7
ipdb> i
3
you can use python's debugger
import pdb
pdb.set_trace()
this will pause the script in debug mode
Example:
my_file=open('running_config','r')
word_count={}
special_character_count={}
import pdb
pdb.set_trace() <== The code will pause here
for config_lines in my_file.readlines():
l=config_lines.strip()
lines=l.upper()
Console:
> /home/samwilliams/workspace/parse_running_config/file_operations.py(6)<module>()
-> for config_lines in my_file.readlines():
(Pdb) print special_character_count
{}
(Pdb)
Related
Imagine I am debugging the following script:
import ipdb
def slow_function(something):
# I'm a very slow function
return something_else
def fast_function(something_else):
# There's a bug here
return final_output
something = 1
something_else = slow_function(something)
ipdb.set_trace()
final_output = fast_function(something_else)
ipdb.set_trace()
When the ipdb.set_trace() line is met the debugger shell is prompted and I can now execute the final_output = fast_function(something_else) statement to check whether fast_function is behaving as intended. I see there's a bug so I go into the source code and I fix it. Now I want to see whether the fix is correct but I don't want to run the script a second time (because it's slow), nor I want to save something_else on disk (because, maybe, it's very large).
Is there a way to update fast_function() in the debugger shell so that the new source code is used?
You can execute a single-line python statement inside pdb by preceding it with an exclamation sign. The output of help exec as produced by pdb follows:
(!) statement
Execute the (one-line) statement in the context of the current
stack frame. The exclamation point can be omitted unless the
first word of the statement resembles a debugger command. To
assign to a global variable you must always prefix the command
with a 'global' command, e.g.:
(Pdb) global list_options; list_options = ['-l']
(Pdb)
Using this facility, you can save the source code of the function in a file and update that function inside pdb as follows:
!exec(open("fast_function.py", "r").read())
Demonstration:
$ cat test.py
import pdb;
def foo():
print('Foo');
foo()
pdb.set_trace()
foo()
$ python3 test.py
Foo
> test.py(8)<module>()
-> foo()
(Pdb) cont
Foo
$ cat foo.py
def foo():
print('Modified Foo');
$ python3 test.py
Foo
> test.py(8)<module>()
-> foo()
(Pdb) !exec(open("foo.py", "r").read())
(Pdb) cont
Modified Foo
If it's a short function you can just overwrite the existing function in one pdb line. Note that the exclamation sign is optional there.
(pdb)...
(pdb)!def fast_function(something_else): print("Hello world");return True
If the function is a bit larger in code length then you can leverage the normal interactive shell(see this)
(Pdb) !import code; code.interact(local=vars())
(InteractiveConsole)
In : def fast_function(something_else):
...: print 'hello in pdb'
...:
In : # use ctrl+d here to return to pdb shell...
(Pdb) !fast_function(arg)
hello in pdb
If the code of the function is not easily manageable with the interactive shell either, Leons' recommendation will be better I guess.
Just keep in mind that, every pdb line can work like executing a line of normal python code and you can achieve pretty much anything on the fly! pdb is more powerful than some other graphical debugging tools in this sense.
P.S. seems like PyCharm does support Evaluating expression feature according to ADR
edit 19/01/2018
You can write result in a file on memory. for example /dev/shm is a tmpfs partition. size could be optimized with protocol dump's keyargs.
# save result
with open('/dev/shm/data.pk', 'w' ) as data:
pickle.dump(something_else, data, protocole=3)
You can use pickle to store result in file the first time and reload it to debug second function
import pickle
def slow_function(something):
# I'm a very slow function
return something + 42
def fast_function(something_else):
# There's a bug here
return something_else + 42
something = 1
something_else = slow_function(something)
# save result
with open('data.pk', 'w' ) as data:
pickle.dump(something_else, data)
second launch
import ipdb
# load result from disk
with open('data.pk', 'r' ) as data:
something_else = pickle.load(data)
ipdb.set_trace()
fast_function(something_else)
Sounds like I'm missing something extremely simple, I'm trying to set a breakpoint in my python code using:
if(some condition):
pdb.set_trace()
My error in the code comes after a large number of iterations..difficult to debug using print etc.
I am able to print stuff when the condition hits but I would like to set brk-pt.
--EDIT--
Actual code:
import pdb
if (node_num == 16):
print node_num
pdb.set_trace()
I see you found your solution Sanjay. But for those who arrived here looking for a means to set a conditional breakpoint with pdb read on:
Instead of hard coding conditions such as if node_num == 16:, run pdb in interactive mode. Sample code:
import pdb
for node_num in range(50):
do_something(node_num)
...
In the shell start the script in debug mode using -m pdb:
[rick#rolled ~]$ python -m pdb abc.py
> /home/dcadm/abc.py(1)<module>()
-> import pdb
(Pdb) l
1 -> import pdb
2
3 for node_num in range(50) :
4 foo = 2**node_num
[EOF]
(Pdb) b 4, node_num > 4
Breakpoint 1 at /home/dcadm/abc.py:4
(Pdb) c
> /home/dcadm/abc.py(4)<module>()
-> foo = 2**node_num
(Pdb) node_num
5
(Pdb)
The pdb shell command b 4, node_num > 4 breaks at line 4 when node_num is greater than 4.
To actually use conditional breakpoints in pdb, you can do the following:
b(reak) [([filename:]lineno | function) [, condition]]
https://docs.python.org/3/library/pdb.html#pdbcommand-break
Eg I'm running some test code that iterates over django views. I want to break only when the particular view I'm interested in is reached:
b C:\Users\powlo\project\tests\TestCase.py:350, view.view_name
== 'app.views.export'
I am not sure as to why your code isn't working, but what you can do is on your local machine, create a new file for your minimum example to see if you can do what you want to do
import pdb
for node_num in range(50):
if node_num == 16:
print(node_num)
pdb.set_trace()
Now running it:
16
> /tmp/tmp.py(3)<module>()
-> for node_num in range(50):
(Pdb) p node_num
16
As you can see this worked as intended with this trivial example, it's up to you to figure out how to adapt this to your code, and/or figure out what else did you do to your code/environment that prevented that prompt from showing up.
Alternatively, if you have a function that is dying in an exception and you want to know the exact line that caused it, you should use post_mortem instead. Wrap the problematic section of code with this
try:
problem_function()
except Exception: # or the specific exception type thrown
pdb.post_mortem()
raise
What post_mortem would do is dump a break point right at the point where the exception happened (specifically in that stack frame), and so this allows all values to be inspected and then let you continue execution. However I also put a raise at the end to allow the exception to continue as normal, and this is intended as execution doesn't normally from where it die but just pause at that exception handling block due to the post_mortem call. Might as well just give up after inspecting what went wrong.
I have a simple python script (say, simple.py) , like
a = 5
b = a + a
print(b)
But I want the following: output the result as if these commands were executed in the interpreter, like
>>> a = 5
>>> b = a + a
>>> print(b)
10
And I want this in stdout :)
Is that possible in some simple way?
Also the same question for IPython.
This question can be helpful for writing a small "how-to" for working with Python/IPython interpreters
You can use the exec statement to execute a line of code (the following is Python 2 code, change the print statements to print() if you want it to be Python 3):
import sys
def run(filename):
with open(filename) as f:
for line in f:
print ">>> ", line
exec line
if __name__ == "__main__":
try:
run(sys.argv[1])
except KeyError:
print "You should pass one filename as parameter."
I should note that it does not appear to handle multi-line statements very well yet (as it's evaluating each line separately).
I modified from #Simeon 's code.
Save the code below as theprinter.py
import sys
if __name__ == '__main__':
c = {}
for line in sys.stdin:
sys.stdout.write('>>> %s' % line)
exec(line, c)
Usage: cat yourscript.py | python theprinter.py
Depending on which tools you are using to create your how-to, you may also want to use Sphinx, which is a tool to produce documentation in Python. It is able to output many formats, notably HTML and PDF. It has a very nice doctest module which allows to put Python instructions in your documentation. They will be run when you compile your documentation and the instructions as well as their results will be displayed.
This should be a good start for embedding and using a Python shell interactively (ActiveState code recipe).
For IPython, you could probably use IPShellEmbed.
I often have the case that I'll be writing a script, and I'm up to a part of the script where I want to play around with some of the variables interactively. Getting to that part requires running a large part of the script I've already written.
In this case it isn't trivial to run this program from inside the shell. I would have to recreate the conditions of that function somehow.
What I want to do is call a function, like runshell(), which will run the python shell at that point in the program, keeping all variables in scope, allowing me to poke around in it.
How would I go about doing that?
import code
code.interact(local=locals())
But using the Python debugger is probably more what you want:
import pdb
pdb.set_trace()
By far the most convenient method that I have found is:
import IPython
IPython.embed()
You get all your global and local variables and all the creature comforts of IPython: tab completion, auto indenting, etc.
You have to install the IPython module to use it of course:
pip install ipython
For practicality I'd like to add that you can put the debugger trace in a one liner:
import pdb; pdb.set_trace()
Which is a nice line to add to an editor that supports snippets, like TextMate or Vim+SnipMate. I have it set up to expand "break" into the above one liner.
You can use the python debugger (pdb) set_trace function.
For example, if you invoke a script like this:
def whatever():
x = 3
import pdb
pdb.set_trace()
if __name__ == '__main__':
whatever()
You get the scope at the point when set_trace is called:
$ python ~/test/test.py
--Return--
> /home/jterrace/test/test.py(52)whatever()->None
-> pdb.set_trace()
(Pdb) x
3
(Pdb)
Not exactly a perfect source but I've written a few manhole's before, here is one I wrote for an abandoned pet project http://code.google.com/p/devdave/source/browse/pymethius/trunk/webmud/handlers/konsole.py
And here is one from the Twisted Library http://twistedmatrix.com/trac/browser/tags/releases/twisted-8.1.0/twisted/manhole/telnet.py the console logic is in Shell.doCommand
I have a MOTD-type message which prints on invocation of the interpreter. Currently this is printed up in sitecustomize. I'd like to suppress the message if the interpreter is not in interactive mode; unfortunately all of the checks in
Tell if Python is in interactive mode do not work in sitecustomize. (sys.argv, sys.ps1, __main__.__file__ are not populated.) Are there checks which work in sitecustomize?
JAB got me looking at the code and I ultimately came up with this:
import ctypes
import getopt
ctypes.pythonapi.Py_GetArgcArgv.restype = None
ctypes.pythonapi.Py_GetArgcArgv.argtypes = [
ctypes.POINTER(ctypes.c_int),
ctypes.POINTER(ctypes.POINTER(ctypes.c_char_p))]
count = ctypes.c_int()
args = ctypes.pointer(ctypes.c_char_p())
ctypes.pythonapi.Py_GetArgcArgv(ctypes.byref(count), ctypes.byref(args))
argc = count.value
argv = [args[i] for i in range(count.value)]
if argc > 1:
interactive = False
opts, args = getopt.getopt(argv[1:], 'i')
for o, a in opts:
if o == '-i':
interactive = True
else:
interactive = True
Kinda ugly (and for Py3k the c_char_p need to be c_wchar_p) but does the job.
Perhaps this idea for checking interpreter interactivity that utilizes the inspect module and checks stack frames might be of some use to you:
http://mail.python.org/pipermail/pythonmac-sig/2002-February/005054.html
You could also try looking directly at the source of pydoc.help(), which the above-linked code snippets were inspired by.
Just realized that you could simply utilize a file containing your interactive prompt with the PYTHONSTARTUP environment variable. The commands in the file pointed to by PYTHONSTARTUP will only be executed when the interpreter is run interactively.
http://docs.python.org/tutorial/interpreter.html#the-interactive-startup-file
If you don't want to set the environment variable outside of Python, you might be able to set the variable to the desired file in sitecustomize.py, but when I tried looking into it to find the loading order it took me right back to the link from the first part of my answer.
Checking the sys.flags is a cleaner way.
>>> import sys
>>> sys.flags.interactive
1
Note, the IDLE is also interactive in its nature, but the flag is not set. I would do below:
>>> if sys.flags.interactive or sys.modules.has_key('idlelib'):
>>> pass # do stuff specific to interactive.