I have created a Custom Extension Command in Python. I installed it, but as expected I am getting errors (quote from SPSS log output - the only way I know for debugging Python programs in SPSS):
Extension command TEST_EXTENSION could not be loaded. The module or a module that it requires may be missing, or there may be syntax errors in it.
The error is probably from the .xmlor from the Run(args) function. The CustomFunction() I am implementing was tested thoroughly.
What would be a good practice for debugging this, and the other potential errors ? The official IBM-SPSS-Statistics-Extension-Command says to
set the
SPSS_EXTENSIONS_RAISE
environment variable to "true"
but I don't know how to do that, nor of this will work regardless of the source of the error.
#horace
You set the environment variable on Windows via the Control Panel > System > Advanced system settings > Environment Variables. The exact wording varies with different Windows versions. I usually choose System variables, although either will usually work. You need to restart Statistics after that. Once you have set this variable, errors in the Python code will produce a traceback. The traceback is ordinarily suppressed as it is of no use to users, but it is very helpful for developers.
The traceback only appears for errors in the Python code. The "could not be loaded" error you reported happens before Python gets control, so no traceback would be produced. There are two common causes for this error. The first is that the xml file defining the extension command or the corresponding Python module was not found by Statistics. The extension command definitions are loaded at Statistics startup or by running the EXTENSION command. Execute SHOW EXT. from the Syntax Editor to see the places where Statistics looks for extension files.
The second cause is a syntax error in the Python code. Run
begin program.
import yourmodule
end program.
to see if any errors are reported.
More generally, there are two useful strategies for debugging. The first is to run the code in external mode, where you run the code from Python. That way you can step through the code using your IDE or the plain Python debugger. See the programmability documentation for details. There are some limitations on what can be done in external mode, but it is often a good solution.
The second is to use an IDE that supports remote debugging. I use Wing IDE, but there are other IDEs that can do this. That lets me jump into the debugger from within Statistics, step through the Python code, and do all the other things you want in a debugger.
HTh
Related
My latest python debugging workflow appears extremely slow to me, and little satifying. How can I improve?
Setting: I work with some third-party python packages from github.
Workflow:
run into error after entering some command to the terminal (Ubuntu WSL, python 3.7)
read terminal error message output, most likely the first or last one is helpful
from the last message i take the code reference (ctrl+left mouse in vscode) and look at the code
i find some function call in the third party module that looks very unrelated to the problem
i add import pdb to the module, and a pdb.set_trace() before that function call
i run the program again, and it stops at the breakpoint
using n,r,u,d i try to navigate closer to the source of the error
i eventually find some error raise condition in some other module, where some property of a certain variable is checked. the variable itself is defined some levels up in the stack
re-running the program and stopping at the same breakpoint as before, i try to navigate to the point where the variable is set. I don't know on which level of the stack it is set, so i miss it sometimes. I set intermediate breakpoints to save me some work when re-running
i finally find the actual cause of the error. I can check out the workspace and eventually fix the error.
i go through all the modules and remove the import pdb and the pdb.set_trace
Thanks for any suggestions
are you using an IDE, not fully clear in your question?
they tend to have graphic ways of setting breakpoints and stepping,
and it saves the hassle of changing the source.
not going into ide opinions, but examples of ide's with debuggers are spyder, thonny and others.
you can also run the debugger via commandline to avoid changing source, but I don't think that's the way to go if you are looking to simplify the cognotive load.
Yes these things you have to do and in extra you can do include logging everywhere as applicable to get exact point where it got occurred.
Is there a simple way, preferably in PyCharm (2017.1) but via command-line python (3.5) if necessary, to detect all code places where a statement is referring to an unresolved reference, e.g. because an import statement is missing?
I am new to Python/PyCharm. More generally, any syntax errors or anything in a similar vein would be a bonus. All I am looking for is the kind of errors I would get if I were "compiling" and "linking" in another language.
I have looked at Can PyCharm list all of Python errors in a project? and PyCharm's "Inspect Code". It is way more complex than I had in mind (and takes ages to run). I see that Python Rope: How to Find all missing imports and errors in all sub modules refactoring recommends pylint, but I wasn't looking for lint-like. I just want darn-obvious errors!
I am tasked with porting a fair-size (32K lines) application, which (apparently) runs under Windows, to Linux. The first thing I want to do is get rid of some of the imports all over the place. If my application executes a line which then has an unresolved reference I get a runtime error, but I want to pick them all up at edit-time. And there will be paths of code which are Windows-only, but I still want to know of any errors like this.
To answer my own question:
From Can PyCharm list all of Python errors in a project?, you can indeed use Code|Inspect Code to get all these errors/warnings in PyCharm as a list where you can click to get to the code. It does take a long time, but at least it's built into PyCharm, and the errors reported correspond to what you see in the editor window.
Pycharm will list all errors and 'warnings' for each source file at the right-hand side of the editor window.
They are represented as short lines or small blocks, depending on the size of the error or 'warning'. Errors are shown in red.
You click on them to take you to the place of the problem in the source.
Warnings are mostly Python style-guide violations (PEP 8).
Another option is to use pylint. This is lint for Python. This will detect missing imports.
You can integrate it into PyCharm by following the instructions in https://stackoverflow.com/a/46409649/4459346
I have had trouble setting up the pycharm ide on my macosx10.7 with python3..
I have scoured every resource available and tried hundreds of approaches, at this point I must accept my incompetence and seek help via this channel.
In my research, I notice a lack of ground-up explanations on python integration into macosx and how to configure pycharm to import modules, run code within the editor, etc. If i ever solve this I will make a very detailed tutorial.
I have imported python3 successfully, it looks like it is linked appropriately from /sys/lib/frameworks to /usr/lib ...etc -- version control is working just fine.
I think my issue is either in setting environmental variables (tried the program to fix this and tried macports) and in the script needed to execute. it will catch errors throughout but final product does not run in python and returns printout of :
/Library/Frameworks/Python.framework/Versions/3.2/bin/python3.2 /Users/anon/Desktop/pythonpractice/Py_Ex/classes.py
Process finished with exit code 0
i really need to get this configuration sound for my python programming class. please help (I've been through every line of pycharm website) .. preferably is there a way to map it via terminal? thanks for anyone who took the time to read this.
Summary of the discussion above:
Python 3.2.2 installation was broken on this Mac, installing ActiveState Python 3.2.2 from scratch and configuring it in PyCharm has fixed the problem.
Python path to be used in PyCharm settings: /Library/Frameworks/Python.framework/Versions/3.2/bin/python3
Incompatible third-party plug-ins may break PyCharm, uninstall/disable them in Preferences | Plugins.
Ensure the latest PyCharm version is installed.
User's code depends on the graphics.py module which was not in the project or in the PYTHONPATH. Putting it into the project has solved the problem.
Most likely the wrong Run/Debug configuration was used in PyCharm, the easiest way to run or debug such scripts is by using the editor context menu Run and Debug actions. PyCharm creates the configuration automatically and debugging works fine as shown of the screenshot:
If one wants to configure and debug it, he can use the code.zip file to get started.
Sorry for the comments mess above, but it was not possible to move it into chat as user had only 1 reputation point, hence not able to use the chat feature of StackOverflow.
I use Notepad++ for writing and running Python scripts. It is a great text editor, except for debugging. Is there a way to step through the code, use break points, view variable values etc. in Notepad++ like you can in Visual Studio?
Does such a plug-in exist? Not that I know of. I agree completely with qor72 on that note.
Is it possible to create such a plugin / functionality? Possibly.
After doing some quick digging, I did find a plugin that looks promising, Python Script. In short it allows you to run python scripts that can access the NPP modules (file menus etc...) as well as the Scintilla Methods which appear to give access to things like the markers on the pages.
To accomplish such a feat I could see the task being broken into a few large blocks (I feel a new open-source project coming on...)
Using Python Script, integrate the python debugger(PDB) as mentioned by Shashi.
Using the Scintilla Methods, add trace back calls where a NPP marker is placed
Redirect PDB outputs and process them to show where the file is stopped (again using the Scintilla methods).
While at the newly created breakpoint and using PDB determine all of the variables in the current namespace. Take this info and dump it to a CMD window, or if you want to get fancy some GUI created with Tk / wxPython
Closing Thoughts
While I think it's possible to create such a plug in, it would be quite an undertaking. Along that line, you might be better off trying to find a different editor that has this built into it already and just create macros (or whatever the IDE calls them) to add in the things you like most about NPP.
Please note that I am a daily user of NPP and have been for many years so I definitely understand why you'd like to have the functionally added to NPP. One of my favorite things about NPP is the speed in which it opens and searches files... moving to a bloated IDE, IMO, would not be worth it to me.
My current work flow is to do all of my editing in NPP and just double click to run the modules. If it fails or goes off in the weeds, I launch IDLE to debug it.
I really hope someone tells me I'm wrong (I'd love to have that feature in Notepad++) but, Notepad++ is designed as a programmers editor, not an IDE. While it has a lot of cool functionality, that level of debugging isn't part of the core tool.
Not seeing anything in the npp-plugins either.
I think python debugger
is the best option if editor is not providing facility :)
Quick guide:
from pdb import set_trace as bp
code
code
bp()
code
code
At the (Pdb) prompt, enter s to step, p foo to print foo, and c to continue executing the code until hitting another breakpoint.
Have you thought of using Komodo.
It's open source and has ports for Windows, Linux and MAC (I think).
This may be an alternative, and if you want some advice from notepad++ users, have a look at the following post on this very site:
Komodo Edit and Notepad++ ::: Pros & Cons ::: Python dev
Some npp users here seemed to have made the switch for python editing running etc...
personally don't know much about debugging on Komodo but as it's an IDE so would be surprised if you couldn't do it easily
I don't really see why Shashi's answer hasn't been upvoted. For the link that he has given supplies a way to step through python scripts as the OP has requested.
So for all who don't know about the pdb module, upon importing it the pdb.set_trace() function allows one to step through the area of code after it. And it is very much similar to the visual studios method of debugging. While you're stepping through the code you are able to input a variety of commands.
One of them is p <expression> and that allows the user to print the current state of variables within the local and global scope.
I know it's 11 years on, and I'm a bit late to the game, and I know it's not Notepad++ but please do consider Visual Studio Code.
It's free, easy to install (both the editor itself plus any python interpreters it uses) and it's widely used and nowhere near as bloated as it's Visual Studio counterpart. It also appears to be the IDE of choice for a lot of Cisco-related course material.
Write your code, click to the left of code pane to insert your breakpoints click the Debugger icon (highlighted), and you're away:
My application crashed recently on a client's computer. I suspect it's because of PyQt's own memory management, which can lead to a invalid memory accesses when not properly handled.
When Python crashes like this, no backtrace is printed, only a data dump is written to the disk.
Is there a possibility to find out where in the Python code the crash occured?
Here's the dump: http://pastie.org/768550
Is this a linux core dump? If so you can examine it with gdb. You will need to run in on a system with an identical OS and version of Python, including 3rd party libraries. Run gdb -c /path/to/core/file. Once gdb has loaded then the command bt will list the stack trace for the main thread, and thread apply all bt will list the stack trace for all threads.
How useful this will be depends on whether the version of Python includes the full symbol table (i.e. is a debug build of Python) - if it is not, then you will only see addresses as offsets to the main C entry points. However this can still be of some use in diagnosing what went wrong.
If it is some other OS that does not support gdb then you are on your own - presumably the OS will have its own debugging tools.
Edit:
There is a page on the Python wiki describing how to get a python stack trace with gdb.
However a quick look at the link in the question shows that the OS is Windows, so gdb is of no use. The information in the Windows dump is minimal, so I think you are out of luck.
My only suggestions are:
try to reproduce the crash in-house.
get the user to reproduce the bug while running a tool that will catch the crash and do a proper memory dump. It is about a decade since I have done serious windows debugging so I don't know what tools are available now - there used to be one called Dr.Watson, but it may be obsolete.
If the user can't reproduce the crash then you are out of luck, on the other hand if it never happens again it is not really that big a problem. ;-)
Update:
Google tells me that Dr Watson is still the default crash handler on Windows XP (and presumably other versions of Windows) - the stack dump that was linked in the question probably came from it. However the default data saved by Dr Watson is fairly minimal, but you can configure it to save more - see this article. In short, if you run drwtsn32 -i it will pop up a dialog to let you set the options.
There's a file named gdbinit in the Python source tree (in Misc/gdbinit) which provides a set of macros for gdb so as to display the current interpreter context. Just type source gdbinit in gdb and then you can execute macros such as pystack. The list of available macros can be obtained simply by reading the file's source code.
(you can find it directly here: http://svn.python.org/view/python/trunk/Misc/gdbinit?view=log).
Of course, if the crash is so severe that it has corrupted the interpreter's internal structures, the macros may fail or crash. Also, it is better to compile the interpreter in debug mode, otherwise gdb may fail to locate the required symbols and variables.
Not sure if it helps, but if you can catch the exception, you could use http://github.com/gooli/pydump to store a dump and load it later in a Python debugger.
Does your application produce a log? If so, you can have logging produce an in-memory log which you might be able to find within the core dump. Also, you can have them send you the log file itself instead of the core dump.