Sounds like I'm missing something extremely simple, I'm trying to set a breakpoint in my python code using:
if(some condition):
pdb.set_trace()
My error in the code comes after a large number of iterations..difficult to debug using print etc.
I am able to print stuff when the condition hits but I would like to set brk-pt.
--EDIT--
Actual code:
import pdb
if (node_num == 16):
print node_num
pdb.set_trace()
I see you found your solution Sanjay. But for those who arrived here looking for a means to set a conditional breakpoint with pdb read on:
Instead of hard coding conditions such as if node_num == 16:, run pdb in interactive mode. Sample code:
import pdb
for node_num in range(50):
do_something(node_num)
...
In the shell start the script in debug mode using -m pdb:
[rick#rolled ~]$ python -m pdb abc.py
> /home/dcadm/abc.py(1)<module>()
-> import pdb
(Pdb) l
1 -> import pdb
2
3 for node_num in range(50) :
4 foo = 2**node_num
[EOF]
(Pdb) b 4, node_num > 4
Breakpoint 1 at /home/dcadm/abc.py:4
(Pdb) c
> /home/dcadm/abc.py(4)<module>()
-> foo = 2**node_num
(Pdb) node_num
5
(Pdb)
The pdb shell command b 4, node_num > 4 breaks at line 4 when node_num is greater than 4.
To actually use conditional breakpoints in pdb, you can do the following:
b(reak) [([filename:]lineno | function) [, condition]]
https://docs.python.org/3/library/pdb.html#pdbcommand-break
Eg I'm running some test code that iterates over django views. I want to break only when the particular view I'm interested in is reached:
b C:\Users\powlo\project\tests\TestCase.py:350, view.view_name
== 'app.views.export'
I am not sure as to why your code isn't working, but what you can do is on your local machine, create a new file for your minimum example to see if you can do what you want to do
import pdb
for node_num in range(50):
if node_num == 16:
print(node_num)
pdb.set_trace()
Now running it:
16
> /tmp/tmp.py(3)<module>()
-> for node_num in range(50):
(Pdb) p node_num
16
As you can see this worked as intended with this trivial example, it's up to you to figure out how to adapt this to your code, and/or figure out what else did you do to your code/environment that prevented that prompt from showing up.
Alternatively, if you have a function that is dying in an exception and you want to know the exact line that caused it, you should use post_mortem instead. Wrap the problematic section of code with this
try:
problem_function()
except Exception: # or the specific exception type thrown
pdb.post_mortem()
raise
What post_mortem would do is dump a break point right at the point where the exception happened (specifically in that stack frame), and so this allows all values to be inspected and then let you continue execution. However I also put a raise at the end to allow the exception to continue as normal, and this is intended as execution doesn't normally from where it die but just pause at that exception handling block due to the post_mortem call. Might as well just give up after inspecting what went wrong.
Related
My code is raising a custom exception which is putting my debugger in the wrong place. For example in the following code, I would like to print the value of my important_local_variable from the debugger, however it seems to be inaccessable as the debugger is starting inside the CustomException:
test.py
class CustomException(Exception):
pass
def my_func():
important_local_variable = 0
badness = 1/important_local_variable
try:
my_func()
except:
raise CustomException
python3 -m ipdb test.py
bash-3.2$ python3 -m pdb test.py
> /Users/max/test.py(1)<module>()
-> class CustomException(Exception):
(Pdb)
You may just need to adjust which stack frame you're in with u and d in pdb. In your toy example, once the ZeroDivisionError hits, you can go down into the function and pull out the local variable just fine:
C:\Users\Randy\test>python -m pdb test.py
> c:\users\randy\test\test.py(1)<module>()
-> class CustomException(Exception):
(Pdb) n
> c:\users\randy\test\test.py(4)<module>()
-> def my_func():
(Pdb) n
> c:\users\randy\test\test.py(8)<module>()
-> try:
(Pdb) n
> c:\users\randy\test\test.py(9)<module>()
-> my_func()
(Pdb) n
ZeroDivisionError: division by zero
> c:\users\randy\test\test.py(9)<module>()
-> my_func()
(Pdb) d
> c:\users\randy\test\test.py(6)my_func()
-> badness = 1/important_local_variable
(Pdb) important_local_variable
0
If it's the initial
> c:\users\randy\test\test.py(1)<module>()
-> class CustomException(Exception):
that was concerning you, that's just the interpreter running the code from the start, not the exception being triggered. This happens since we started pdb from the beginning of the program. You just need to step through the code with n until the errors start actually being thrown, as I did in the example above.
Yes it is trying to define the exception class. You should let the debugger continue.
As per comment, it is the first line of the script.
Also even if you had a main function, the python control will start from first statement of module.
Imagine I am debugging the following script:
import ipdb
def slow_function(something):
# I'm a very slow function
return something_else
def fast_function(something_else):
# There's a bug here
return final_output
something = 1
something_else = slow_function(something)
ipdb.set_trace()
final_output = fast_function(something_else)
ipdb.set_trace()
When the ipdb.set_trace() line is met the debugger shell is prompted and I can now execute the final_output = fast_function(something_else) statement to check whether fast_function is behaving as intended. I see there's a bug so I go into the source code and I fix it. Now I want to see whether the fix is correct but I don't want to run the script a second time (because it's slow), nor I want to save something_else on disk (because, maybe, it's very large).
Is there a way to update fast_function() in the debugger shell so that the new source code is used?
You can execute a single-line python statement inside pdb by preceding it with an exclamation sign. The output of help exec as produced by pdb follows:
(!) statement
Execute the (one-line) statement in the context of the current
stack frame. The exclamation point can be omitted unless the
first word of the statement resembles a debugger command. To
assign to a global variable you must always prefix the command
with a 'global' command, e.g.:
(Pdb) global list_options; list_options = ['-l']
(Pdb)
Using this facility, you can save the source code of the function in a file and update that function inside pdb as follows:
!exec(open("fast_function.py", "r").read())
Demonstration:
$ cat test.py
import pdb;
def foo():
print('Foo');
foo()
pdb.set_trace()
foo()
$ python3 test.py
Foo
> test.py(8)<module>()
-> foo()
(Pdb) cont
Foo
$ cat foo.py
def foo():
print('Modified Foo');
$ python3 test.py
Foo
> test.py(8)<module>()
-> foo()
(Pdb) !exec(open("foo.py", "r").read())
(Pdb) cont
Modified Foo
If it's a short function you can just overwrite the existing function in one pdb line. Note that the exclamation sign is optional there.
(pdb)...
(pdb)!def fast_function(something_else): print("Hello world");return True
If the function is a bit larger in code length then you can leverage the normal interactive shell(see this)
(Pdb) !import code; code.interact(local=vars())
(InteractiveConsole)
In : def fast_function(something_else):
...: print 'hello in pdb'
...:
In : # use ctrl+d here to return to pdb shell...
(Pdb) !fast_function(arg)
hello in pdb
If the code of the function is not easily manageable with the interactive shell either, Leons' recommendation will be better I guess.
Just keep in mind that, every pdb line can work like executing a line of normal python code and you can achieve pretty much anything on the fly! pdb is more powerful than some other graphical debugging tools in this sense.
P.S. seems like PyCharm does support Evaluating expression feature according to ADR
edit 19/01/2018
You can write result in a file on memory. for example /dev/shm is a tmpfs partition. size could be optimized with protocol dump's keyargs.
# save result
with open('/dev/shm/data.pk', 'w' ) as data:
pickle.dump(something_else, data, protocole=3)
You can use pickle to store result in file the first time and reload it to debug second function
import pickle
def slow_function(something):
# I'm a very slow function
return something + 42
def fast_function(something_else):
# There's a bug here
return something_else + 42
something = 1
something_else = slow_function(something)
# save result
with open('data.pk', 'w' ) as data:
pickle.dump(something_else, data)
second launch
import ipdb
# load result from disk
with open('data.pk', 'r' ) as data:
something_else = pickle.load(data)
ipdb.set_trace()
fast_function(something_else)
The title says it all. When you are working R and using RStudio, its really easy and simple to debug something by dropping a browser() call anywhere in your code and seeing what goes wrong. Is there a way to do that with Python? I'm slowly getting very sick of print statement debugging.
It looks like you are looking for ipdb
The basic usage is to set:
import ipdb
ipdb.set_trace()
in your code to explore; this will take you right to that part of code, so you can explore all the variables at that point.
For your specific use case: "Would it be a setting in my Console so that it Opens pdb right before something crashes" (a comment to another answer), you can use context manager: launch_ipdb_on_exception
For example:
from ipdb import launch_ipdb_on_exception
def silly():
my_list = [1,2,3]
for i in xrange(4):
print my_list[i]
if __name__ == "__main__":
with launch_ipdb_on_exception():
silly()
Will take you to ipdb session:
5 for i in xrange(4):
----> 6 print my_list[i]
7
ipdb> i
3
you can use python's debugger
import pdb
pdb.set_trace()
this will pause the script in debug mode
Example:
my_file=open('running_config','r')
word_count={}
special_character_count={}
import pdb
pdb.set_trace() <== The code will pause here
for config_lines in my_file.readlines():
l=config_lines.strip()
lines=l.upper()
Console:
> /home/samwilliams/workspace/parse_running_config/file_operations.py(6)<module>()
-> for config_lines in my_file.readlines():
(Pdb) print special_character_count
{}
(Pdb)
In pdb (or ipdb) we can execute statements and evaluate expressions with the ! or p commands:
p expression
Evaluate the expression in the current context and print its value.
[!]statement
Execute the (one-line) statement in the context of the current stack frame. The exclamation point can be omitted unless the first word of the statement resembles a debugger command. To set a global variable, you can prefix the assignment command with a global command on the same line
So, for example, I can type p reddit.get_subreddits() while debugging in ipdb and the code will be executed in the current context and I will see the return value.
Is there a way I can debug the execution of such "manually typed" expressions?
Basically I would like to do is s reddit.get_subreddits(), but that just executes the step command and ignores the expression.
EDIT: A trivial example
Take this simple function:
import random
def get_value_for_weekday(weekday_index=None):
values = [10, 20, 20, 10, 30, 30, 30]
if not weekday_index:
# If no weekday provided, return the average of all weekdays
return sum(values) / 7
return averages[weekday_index]
if __name__ == '__main__':
while True:
import ipdb; ipdb.set_trace() # enter ipbd for debug
get_value_for_weekday(random.randint(0, 7))
Which is bugged because of the if not weekday_index (it should check weekday_index is not None.)
Let's assume I notice I get 10 half the number of times I was expecting. So I added a import ipdb; ipdb.set_trace() before the call to the function to try and debug the code.
So I'm in the ipdb console and I suddenly get the idea that maybe the problem is when I pass 0 as weekday_index.
I can test my hypothesis directly in ipdb:
ipdb> p get_value_for_weekday(0)
22
Ok, so I realize there's something wrong when weekday_index=0.
What I would like to do now is debug step by step the call to get_value_for_weekday(0), so that I could see that I erranously enter the if block.
Obviously I could exit ipdb, stop the script, change the code to always pass 0, relaunch the script and when I enter ipdb, debug the call with the ipdb step (s) command.
But wouldn't it be easier if I could just do s get_value_for_weekday(0) much the same way I was able to do p get_value_for_weekday(0)?
Is there a way do something like this?
I think you're looking for the (d)ebug command which, for some reason, is not specified in the Debugger Commands. Just for future reference, pdb has a nice set of commands specified (which you can see by typing help in the interactive prompt). On to the debug command:
(Pdb) help debug
debug code
Enter a recursive debugger that steps through the code
argument (which is an arbitrary expression or statement to be
executed in the current environment).
Which seems to do what you're after. Using your sample script from the terminal:
python -m pdb pdbscript.py
After issuing two n commands in order for the function to get parsed (I believe this is how pdb works). You can issue a debug get_value_for_weekday(0) command to recursively step in the function:
(Pdb) debug get_value_for_weekday(0)
ENTERING RECURSIVE DEBUGGER
> <string>(1)<module>()
((Pdb)) s
--Call--
> /home/jim/Desktop/pdbscript.py(3)get_value_for_weekday()
-> def get_value_for_weekday(weekday_index=None):
((Pdb)) n
> /home/jim/Desktop/pdbscript.py(4)get_value_for_weekday()
-> values = [10, 20, 20, 10, 30, 30, 30]
((Pdb)) n
> /home/jim/Desktop/pdbscript.py(5)get_value_for_weekday()
-> if not weekday_index:
((Pdb)) p weekday_index
0
((Pdb)) n
> /home/jim/Desktop/pdbscript.py(7)get_value_for_weekday()
-> return sum(values) / 7
Do note, I feel really sketchy about this form of meta-debugging but it seems to be what you're after.
With regards to your example,
you don't need to exit pdb and change the code.
You can step into the function (with 's') and set weekday_index=0 inside.
One solution to your original problem is to use the debugger's jump command as follows:
jump before the function call using 'j #line-number'
step in the function with 's'
set the input params, and continue debugging.
This worked when I tried it, but the debugger complained when I tried to do step 3 before step 2 for some reason.
My code is, for better or worse, rife with single letter variables (it's physics stuff, so those letters are meaningful), as well as NumPy's, which I'm often interacting with.
When using the Python debugger, occasionally I'll want to look at the value of, say, n. However, when I hit n<enter>, that's the PDB command for (n)ext, which has a higher priority. print n works around looking at it, but how can I set it?
Use an exclamation mark ! before a statement to have it run :
python -m pdb test.py
> /home/user/test.py(1)<module>()
-> print('foo')
(Pdb) !n = 77
(Pdb) !n
77
(Pdb) n
foo
> /home/user/test.py(2)<module>()
-> print('bar')
(Pdb)
The docs say:
! statement
Execute the (one-line) statement in the context of the current stack frame. The exclamation point can be omitted unless the first word of the statement resembles a debugger command. [...]
You can use semicolons, so just put something else in front of it:
ipdb> print n
2
ipdb> n
> 145 <some code here>
146
147
ipdb> 1; n=4
1
ipdb> print n
4
That is not the direct answer to your question, but it may help you: PuDB is a console-based visual interface for PDB which separates commands from variable manipulation by design.
Use brackets (variable_name).
For example, if you have one variable named q, and you want to check out its value.
If you directly input q in the prompt, then the ipdb debugging process will quit and break up.
>>> q
Instead, you should input (q) to check this variable:
>>> (q)
Then its value will be printed.
Eric IDE, Wing IDE & Spyder to mention just a few all have visual debuggers that are worth a go as they separate the display of values from the commands.