I'm doing experience with my first small applications, essentially data viewers based on Pandas and matplotlib, using PyQt for the GUI part.
What I find now difficult is to understand what goes wrong in my code, because the error does not get propagated to the iPython console I launch my script from.
It simply won't do what's expected, but there is no information as to 'why'.
To fix ideas, let's say I have a button that should plot a certain curve to the canvas. If there's a fail in the indexing operation of the data, therefore nothing can be plotted, then nothing will appear on the canvas, but I'll get no traceback that actually index so-and-so wasn't to be found.
Using the debugger proves quite cumbersome, too.
I had a situation where, while running my main(), I could interact with the IPython shell and do things like:
main.my_plot_function()
from which I would get a standard output, and see what is wrong. Although sub-optimal, this did the trick.
I had to reset Spyder this morning (wasn't launching on Windows), and since then, when I launch my script, the console is unresponsive. So I can't do `main.my_plot_function()' anymore.
Generally speaking, is there a way to instruct Spyder or the console that I want to see what's going on in the background? Some "verbose" switch?
I am not sure what you mean by wanting to know what's going on in the background. I assume that you wish to know at many points in the code, what the variable types and values are and/or where the current point of execution is.
There are two options:
1) Use print statements wherever you need to know what's going on. For example, if you have a plot function, simply put some print statements inside the function to print out the sizes of the lists/arrays being plotted etc. You can also look for useful functions in this regard, i.e., type() to print out the type of a variable to make sure it is what you think it is, print(locals()) to print names and values of all local variables etc.
2) Use pdb to introduce break points and run your main script from the command line. This will stop the script execution where you want and from the pdb console, you can inspect the data-structures. There are of course other debuggers you can use, such as pudb (with a basic GUI and some extra features than pdb).
There is no general "verbose" mode in spyder or any other Python IDE I know of.
Related
Being in the process of switching from Matlab to Python/IPython/Spyder I ran into the following issue.
In a Matlab script (and I actually did not notice this until I switched to Python) there is no warning if you are using a variable which you did not define in that script.
What I kept doing all the times in Matlab was to have, say, 2 (or more) scripts which I would then execute one after the other. The second script would typically use variables defined in the first, but would not give me any warning.
In Spyder I noticed that the situation is different. In my hypothetical second script, all the variables which are not defined in the second script itself give me a warning (undefined name '...'), which is not nice too see...
Another typical example, would be the following: I have a main script but there is something in this script I wanna look at better. So, without touching the script, I would create another file where I would copy paste a few lines from the script to play a little with them.
In this new file I would be using variables which are in the console already but that are techically unknown to the file itself, so would give me a warning.
So I guess I am simply asking if there a way to suppress this kind of warnings.
Or there might be a deeper question of whether there s a more pythonic way of working, but there I am really not sure about...
I am trying to find a way that I can have a program step through Python code line by line and do something with the results of each line. In effect a debugger that could be controlled programmatically rather than manually. pdb would be exactly what I am looking for if it returned its output after each step as a string and I could then call pdb again to pickup where I left off. However, instead it outputs to stdout and I have to manually input "step" via the keyboard.
Things I have tried:
I am able to redirect pdb's stdout. I could redirect it to a second
Python program which would then process it. However, I cannot
figure out how to have the second Python program tell pdb to
step.
Related to the previous one, if I could get pdb to step all the way
through to the end (perhaps I could figure out something to spoof a
keyboard repeatedly entering "step"?) and redirect the output to a
file, I could then write another program that acted like it was
stepping through the program when it was actually just reading the
file line by line.
I could use exec to manually run lines of Python code. However,
since I would be looking at one line at a time, I would need to
manually detect and handle things like conditionals, loops, and
function calls which quickly gets very complicated.
I read some posts that say that pdb is implemented using
sys.settrace. If nothing else works I should be able to recreate
the behavior I need using this.
Is there any established/straight forward way to implement the behavior that I am looking for?
sys.settrace() is the fundamental building block for stepping through Python code. pdb is implemented entirely in Python, so you can just look at the module to see how it does things. It also has various public functions/methods for stepping under program control, read the library reference for your version of Python for details.
I read some posts that say that pdb is implemented using sys.settrace.
If nothing else works I should be able to recreate the behavior I need
using this.
Don't view this as a last resort. I think it's the best approach for what you want to accomplish.
I am new to Python, have some experience in MatLab and r. My question is: Is it possible to run part of the code in .py block by block (line by line)?
In r or Matlab, I can first have some data and variables loaded in the memory first. Then experimentally I can run a line or two to try out the syntax... this is particularly useful for new learners I believe. I know there is something called the iPython which can execute Python code line by line however this is not what I am after. Thanks.
Since ipython has already been discounted, I'm not sure this answer will be better. But I will tell you the two things that I do.
I drop into the debugger at the point where I want to "try out" something, so the code will run up to that point, and then drop me into the debugger. You do this simply by inserting this code at that point:
import pdb; pdb.set_trace()
Once you've done what needs to be done, you can either press q to quit, or c to continue running the process.
I use the -i option to python. This enters interactive mode at the end of your python code. This is useful if you want to set up a bunch of data structures, and try out some code on it, instead of typing all of it into a python shell first. (that might be why you rejected ipython?)
I think what you need is a debugger.
You can use the pydev plugin for Eclipse which has a debugger.
Another option is pdb as already suggested but it's not very easy to use.
I'm trying hard to like IPython Notebook, but maybe because I'm so used to writing code in vi and executing it at the command line I'm finding some of its defaults challenging. Can anything be done (perhaps in a configuration file somewhere) about the following?
I'd like %hist to output line numbers by default without having to remember the -n and without having to set up an alias every time.
How do I set %automagic to "off" by default to stop IPython polluting my namespace with its un-percented magics and shell commands? I know I can use the --no-import-all option with --pylab option: is there an equivalent --no-automagic option?
It drives me mad that I'm never quite sure what is the status of the objects bound to my variable names: changing and running a cell beneath the one I'm using can alter an object I'm referring to in the current cell. To avoid this, I've got into the habit of using Run All or Run All Above, but that sometimes repeats lengthy calculations and reimports stuff I'd rather not bother with: can I flag some cells to be not-rerun by Run All?
Can I get vi-style key-bindings for editing cells?
Why does IPython notebook hang my browser if the kernel is using lots of memory? I thought they were separate processes with the kernel just reporting back its results.
(Please try to ask one question per question - SO is designed to work that way. However, I'm not feeling so mean that I'd refuse to answer)
I don't think the behaviour of %hist is configurable, sorry.
Set c.InteractiveShell.automagic = False in your config files.
There has been some discussion of a %%cache cell magic that could avoid re-running long running cells by storing values computed in that cell. It has not been implemented yet, though.
Yes: https://github.com/ivanov/ipython-vimception
It shouldn't hang just because of kernel memory use - if your code is producing a lot of output, though, that can hang the browser because adding lots of things to the DOM gums it up.
I try Eclipse+PyDev pair for some of my work. (Eclipse v3.5.0 + PyDev v1.5.6) I couldn't find a way to expose all of my variables to the PyDev console (Through PyDev console -> Console for current active editor option) I use a simple code to describe the issue. When I step-by-step go through the code I can't access my "x" variable from the console. It is viewed on Variables tab, but that's not really what I want.
Any help is appreciate.
See my screenshot for better description:
EDIT:
Assume adding a simple func like:
def myfunc(x):
return x**x
When I debug with the function added in the code I can access myfunc from the console easily. (Type myfunc and it will be available after this automatic execution:
>>> from part2.test import myfunc
>>> myfunc
Then when I do myfunc(5) it acts just like in the Python interpreter. It would be so useful to access variables in the similar fashion for debugging my code. I have big arrays and I do various tests and operations during debug process. Like:
Get my x and do x.sum(), later do x[::10], or transpose operate with other arrays observe results, experiment etc...
Hope there will be a better solution.
Update:
In the latest PyDev versions, it's possible to right-click a frame in the stack and select PyDev > Debug console to have the interactive console with more functions associated to a context during a debug session.
Unfortunately, the actual interactive console, which would be the preferred way of playing with code (with code-completion, etc -- http://pydev.org/manual_adv_interactive_console.html) has no connection to a debug session right now (this is planned but still not implemented).
Still, with the 'simpler' console available, you are still able to interactively inspect and play with the variables available in a breakpoint scope: http://pydev.org/manual_adv_debug_console.html (which is the same as you'd have with pdb -- it's just a matter of typing code in the available console after a breakpoint is hit).
Cheers,
Fabio
For this sort of exploratory debugging I like to use pdb, the batteries-included debugger. I haven't used it inside PyDev so I don't know how it would all fit together. My guess is it will do what you expect it to. An example of its usage:
import pdb
def myfunc(x):
pdb.set_trace()
return x**x
This will break right before executing the return statement, and it allows you to use full Pythonic statements to figure out what's going on. I use it like an interactive print statement: setting the place where I want to dive in, examining values and figuring results, and stepping through to watch it happen. Perhaps this is a lazy way of debugging, but sometimes you need more information before you can make less-lazy decisions :-)
The page I usually reference is at Python Conquers The Universe which also links a few other sources of information.