I know how to use PyCharm's debugger but that has only deepened my curiosity of how it accomplishes the task of being so tightly coupled to the Python interpreter.
Does cPython have some sort of intrepreter hooks buried in itself or does PyCharm somehow copy the source code, instrument the code, and then exec it?
Thanks to #unholySheep I was able to go from the github src on PyDev.Debugger back to sys.settrace which lead to a post on Python Module of the week on settrace.
Once the tracing script has the stack frame, it is likely a non-trivial task of inspecting the frame's stack content and or using code/exec/eval to run "watch" statements in context. As for break points, that would be trivial as it is just a task of matching the frame's line number and filepath.
Related
I am looking for the following (IMHO, very important) feature:
Suppose I have two functions fa() and fb(), both of them has a breakpoint.
I am now stopped in the breakpoint in fa function.
In the interactive debugger console I am calling fb().
I want to stop in fb breakpoint, but, unfortunately pb() runs but ignores the breakpoint.
someone in another SO thread called it "nested breakpoints".
I am a developer that come from Matlab, in Matlab no matter how a function is called, from console, from debugger. if it has a breakpoint it stops.
I read past threads about this subject and did not find any solution.
I also tried latest pycharm community and latest pydev and no luck.
I also read that visual studio can not make it.
Is this inherent in Python and technically can not be done?
Is there a technique / another IDE that supports it?
I followed this question and saw nobody is answering, I thought about this feature too so I started digging through google and found your old question :)
TL;DR - You can't do that
(I tried with PyCharm, Visual-Studio and Eric Python IDE).
My guess that not working because it adds more complexity to debugging - what happened when you step to the next line? and if you have many threads/processes? what happens to mutable types?
My way to do it
If you have 2 functions and you want to debug both of them:
def parent():
dummy_debuggable_var = 1
print('Running child() function')
out = child(dummy_debuggable_var) #BP1 is here
print(out)
def child(x):
print('Calculating ...')
return x+2
And your goal is to debug the dummy_debuggable_var and also the child function:
Put the break point where is comment is
Run the script
When the break point stops, inspect your dummy_debuggable_var
step-into (on PyCharm - F7)
Inspect your child function
While you're inside the child function, you can look on your PyCharm's frames stack tab on your debugger window, this will let you jump back to the parent's frame and inspect it.
This is more sequential process, and not parallel, but it's simple and it works.
Take a look at Eric Python IDE and VSC(Visual Studio Code)
I have started using VS with Python and I was expecting to have similar features as in R.
What I need is to be able to edit and execute line by line ("Sent to Interactive" command) as well as see current values of the defined variables?
This last item is missing and I am not able to find any way to display values of
all current variables in Local, Watch or any other window while doing interactive Edit Execute?
(There are similar question at StackOverflow but couldn't find answer to this particular one)
There isn't any obvious way to inspect variables in the current context in Interactive Window in PTVS, unfortunately. We are aware of this deficiency, and would like to fix it in future versions.
In the meantime, there are some workarounds that may be "good enough". The most obvious one is to use dir() to dump the names of the locals (and you can turn it into a one-liner dict comprehension to dump the values alongside with the names).
Alternatively, you can attach PTVS debugger to the Python process backing the Interactive by using the $attach magic command. This works the same as a regular debugging session, and so you won't see the locals until you somehow pause the process. The easiest way to do so is to cause an exception to be raised in the REPL, e.g. simply by typing raise Exception - then you'll be paused in the right frame, and can see all your variables in Locals and edit them as usual; of course, you'll have to unpause the process before you can use the REPL again.
I'm working on a python project that spans across multiple files and directories. Here's my workflow:
Run main python script
Main script calls some functions in other files
Functions in other files/directories execute
In the middle of execution, there is a bug in one of the functions, but I find the bug only after the main script finishes. Sometimes, there may not be a bug, but rather some parameter that needs tweaking.
I go back and fix the bug/make the necessary tweaks and re-run the main program and this time it executes fine.
Obviously, this workflow is terribly inefficient as considerable amount of code (that runs prior to the buggy function) gets re-executed. What would be ideal is to run the program in ipython and after discovering the issue and making the necessary changes, restart from the place where the buggy function executions starts and not from the beginning. I'm not sure how to achieve this and any help would be much appreciated.
I know how to rerun lines from ipython history (%rerun) and how to ensure autoreload of changed files in ipython, but in this case, I can't really type out the lines of code into ipython. Writing unit tests may not always be feasible, so I need an alternate solution. My use case is something similar to setting a "breakpoint" and then re-executing code past the breakpoint multiple times so as to avoid re-executing the code prior to the breakpoint more than once, while ensuring that all the necessary variables (until that stage) are correctly populated. One final condition is that I may not be able to use an IDE and vim is the only editor available across all the environments I work with.
You could start writing test cases for every function and try to debug each function separately instead of the whole program/script.
There is a python unittest-module: https://docs.python.org/3.4/library/unittest.html and a lot of tutorials like (just an example): http://docs.python-guide.org/en/latest/writing/tests/
It seems annoying writing tests but thinking about test cases gives deeper understanding of "How should the function behave if...".
You could use the code module to include "breakpoints" in your code
import code
# ... later in your program add
code.interact(local=locals()) # enter python interpreter at this point (ctrl+D to continue execution)
Is there a debugger that can debug the Python virtual machine while it is running Python code, similar to the way that GDB works with C/C++? I have searched online and have come across pdb, but this steps through the code executed by the Python interpreter, not the Python interpreter as its running the program.
The reference implementation of Python, CPython, is written in C. You can use GDB to debug it as you would debug any other program written in C.
That said, Python does have a few little helpers for use in GDB buried under Misc/gdbinit. It's got comments to describe what each command does, but I'll repeat them here for convenience:
pyo: Dump a PyObject *.
pyg: Dump a PyGC_Head *.
pylocals: Print the local variables of the current Python stack frame.
lineno: Get the current Python line number.
pyframe: Print the source file name, line, and function.
pyframev: pyframe + pylocals
printframe: pyframe if within PyEval_EvalFrameEx; built-in frame otherwise
pystack: Print the Python stack trace.
pystackv: Print the Python stack trace with local variables.
pu: Print a Unicode string.
It looks like the Fedora project has also assembled their own collection of commands to assist with debugging which you may want to look at, too.
If you're looking to debug Python at the bytecode level, that's exactly what pdb does.
If you're looking to debug the CPython reference interpreter… as icktoofay's answer says, it's just a C program like any other, so you can debug it the same way as any other C program. (And you can get the source, compile it with extra debugging info, etc. if you want, too.)
You almost certainly want to look at EasierPythonDebugging, which shows how to set up a bunch of GDB helpers (which are Python scripts, of course) to make your life easier. Most importantly: The Python stack is tightly bound to the C stack, but it's a big mess to try to map things manually. With the right helpers, you can get stack traces, frame dumps, etc. in Python terms instead of or in parallel with the C terms with no effort. Another big benefit is the py-print command, which can look up a Python name (in nearly the same way a live interpreter would), call its __repr__, and print out the result (with proper error handling and everything so you don't end up crashing your gdb session trying to walk the PyObject* stuff manually).
If you're looking for some level in between… well, there is no level in between. (Conceptually, there are multiple layers to the interpreter, but it's all just C code, and it all looks alike to gdb.)
If you're looking to debug any Python interpreter, rather than specifically CPython, you might want to look at PyPy. It's written in a Python-like language called RPython, and there are various ways to use pdb to debug the (R)Python interpreter code, although it's not as easy as it could be (unless you use a flat-translated PyPy, which will probably run about 100x too slow to be tolerable). There are also GDB debug hooks and scripts for PyPy just like the ones for CPython, but they're not as complete.
Think about this scenario:
I debug my Django project and I step through the code (in and out). The debugger sometimes enters Django libraries or other external libraries.
Does anyone know how to prevent the debugger from entering external code? Or at least a 'big' step out to get the debugger back to the project code?
Does anyone know how to prevent the debugger from entering external code?
Yes, Dmitry Trofimov knows;
(...) add modules you don't want to trace to the dict DONT_TRACE in <pycharm-distr>/helpers/pydev/pydevd.py
That is a hacky solution (...)
If you want this feature to be less hacky you can vote on it by visiting issue
PY-9101 Implement "Do not step into the classes" option for Python debugger
Those using pdb might be interested to know there is such a feature in pdb;
Starting with Python 3.1, Pdb class has a new argument called skip -
class pdb.Pdb(completekey='tab', stdin=None, stdout=None, skip=None, nosigint=False)
The skip argument, if given, must be an iterable of glob-style module
name patterns. The debugger will not step into frames that originate
in a module that matches one of these patterns. 1
1 Whether a frame is considered to originate in a certain module is
determined by the __name__ in the frame globals.
The example given in the docs shows how to skip Django's packages -
import pdb; pdb.Pdb(skip=['django.*']).set_trace()
Everything looks the same to the debugger, it can't distinguish between your code or Django's code – it's all Python. So it will run everything, however if you want to stop it from drilling down so low you'll have to start “stepping over” lines of code instead of “stepping into” them.
According to the PyCharm docs you'll want to use F8 when ever you see a line of code that looks like it could be a gateway into Django's internals. If you accidently find yourself inside Django's source code you can hit Shift+F8 until you're out of it.