Is there a simple way, preferably in PyCharm (2017.1) but via command-line python (3.5) if necessary, to detect all code places where a statement is referring to an unresolved reference, e.g. because an import statement is missing?
I am new to Python/PyCharm. More generally, any syntax errors or anything in a similar vein would be a bonus. All I am looking for is the kind of errors I would get if I were "compiling" and "linking" in another language.
I have looked at Can PyCharm list all of Python errors in a project? and PyCharm's "Inspect Code". It is way more complex than I had in mind (and takes ages to run). I see that Python Rope: How to Find all missing imports and errors in all sub modules refactoring recommends pylint, but I wasn't looking for lint-like. I just want darn-obvious errors!
I am tasked with porting a fair-size (32K lines) application, which (apparently) runs under Windows, to Linux. The first thing I want to do is get rid of some of the imports all over the place. If my application executes a line which then has an unresolved reference I get a runtime error, but I want to pick them all up at edit-time. And there will be paths of code which are Windows-only, but I still want to know of any errors like this.
To answer my own question:
From Can PyCharm list all of Python errors in a project?, you can indeed use Code|Inspect Code to get all these errors/warnings in PyCharm as a list where you can click to get to the code. It does take a long time, but at least it's built into PyCharm, and the errors reported correspond to what you see in the editor window.
Pycharm will list all errors and 'warnings' for each source file at the right-hand side of the editor window.
They are represented as short lines or small blocks, depending on the size of the error or 'warning'. Errors are shown in red.
You click on them to take you to the place of the problem in the source.
Warnings are mostly Python style-guide violations (PEP 8).
Another option is to use pylint. This is lint for Python. This will detect missing imports.
You can integrate it into PyCharm by following the instructions in https://stackoverflow.com/a/46409649/4459346
Related
Pylance doesn't recognize the "socket" module. There's no Intellisense for it and when I write code, I only get squiggly lines on everything I call, even though it's a member of the core python library, isn't it?
Pylance throws errors for simple comparisons like this one, even though I'm only comparing two integers:
Now I know I could ignore all these errors, but is this a known problem? I did some quick searches, but I wasn't able to find anything other then "this gets fixed in the next version", even though I have a much higher version than that.
I did a restart of VS Code and reinstalled all the extensions multiple times already. Do you know any other possible fixes for this?
The version of the Pylance extension used is v2022.8.40.
VS Code 1.70.2, Python extension v2022.13.12381007, IntelliCode extension v1.2.23
My latest python debugging workflow appears extremely slow to me, and little satifying. How can I improve?
Setting: I work with some third-party python packages from github.
Workflow:
run into error after entering some command to the terminal (Ubuntu WSL, python 3.7)
read terminal error message output, most likely the first or last one is helpful
from the last message i take the code reference (ctrl+left mouse in vscode) and look at the code
i find some function call in the third party module that looks very unrelated to the problem
i add import pdb to the module, and a pdb.set_trace() before that function call
i run the program again, and it stops at the breakpoint
using n,r,u,d i try to navigate closer to the source of the error
i eventually find some error raise condition in some other module, where some property of a certain variable is checked. the variable itself is defined some levels up in the stack
re-running the program and stopping at the same breakpoint as before, i try to navigate to the point where the variable is set. I don't know on which level of the stack it is set, so i miss it sometimes. I set intermediate breakpoints to save me some work when re-running
i finally find the actual cause of the error. I can check out the workspace and eventually fix the error.
i go through all the modules and remove the import pdb and the pdb.set_trace
Thanks for any suggestions
are you using an IDE, not fully clear in your question?
they tend to have graphic ways of setting breakpoints and stepping,
and it saves the hassle of changing the source.
not going into ide opinions, but examples of ide's with debuggers are spyder, thonny and others.
you can also run the debugger via commandline to avoid changing source, but I don't think that's the way to go if you are looking to simplify the cognotive load.
Yes these things you have to do and in extra you can do include logging everywhere as applicable to get exact point where it got occurred.
Running on Mac Sierra, the autocompletion in Spyder (from Anaconda distribution), seems quite erratic. When used from the Ipython console, works as expected. However, when used from the editor (which is my main way of writing), is erratic. The autocompletion works (i.e. when pressing TAB a little box appears showing options) for some modules, such as pandas or matplotlib. So writing 'pd.' and hitting TAB, gets the box with options as expected. However, this does not happen with many other objects: for example, after defining a dataframe named 'df', typing 'df.' TAB shows nothing. In the Ipython console, 'df.' TAB would show the available procedures for that dataframe, such as groupby, and also its columns, etc..
So the question is threefold. First, is there any particular configuration that should be enabled to get this to work? I don't think so, given some time spent googling, but just wanna make sure. Second, could someone state what is the official word on what works and what doesn't in terms of autocompletion (e.g. what particular modules do work from the editor, and which ones doesn't?). Finally, what are the technical aspects of the differences between the editor and the Ipython console in the performance of the autocompletion with Spyder? I read something about Jedi vs. PsychoPy modules, so got curious (however, please keep in mind that although I have scientific experience, I am relatively new to computation, so please keep it reasonably simple for an educated but not expert person).
UPDATE: As a side question, it would be great to know why is the autocompletion better in Rodeo (another IDE). It is more new, has way fewer overall options than Spyder, but the autocompletion works perfectly in the editor.
(Spyder developer here)
My answers:
is there any particular configuration that should be enabled to get this to work?
In Spyder 3.1 we added the numpydoc library to improve completions of some objects (like Matplotlib figures and NumPy arrays). If Dataframe completions are not working for you (they are for me), please open an issue in our issue tracker on Github to track and solve this problem.
could someone state what is the official word on what works and what doesn't in terms of autocompletion (e.g. what particular modules do work from the editor, and which ones doesn't?)
The most difficult part is getting completions of definitions when an object is generated by functions or methods developed in C/C++/Fortran and not in Python. I mean, things like
import numpy as np
a = np.array([])
a.<TAB>
As I said, this should be working now for arrays, figures and dataframes, but it doesn't work for all libraries (and most scientific Python libraries are created in C/C++/Fortran and wrapped in Python for speed).
The problem is that the completion libraries we use (Rope and Jedi) can't deal with this case very well because array (for example) can't be introspected in a static way (i.e. without running code involving it). So we have to resort to tricks like analyzing array's docstring to see its return type and introspect that instead.
what are the technical aspects of the differences between the editor and the Ipython console in the performance of the autocompletion with Spyder?
The most important difference is that in the IPython console you have to run your code before getting completions about it. For example, please run this in a fresh IPython console
In [1]: import pandas as pd
...: df = pd.Da<Tab>
and you will see that it won't return you any completions for Da (when it obviously should return Dataframe).
But, after evaluation, it is quite simple to get completions. You can simply run
dir(pd)
to get them (that's what IPython essentially does internally).
On the other hand, Spyder's Editor doesn't have a console to run code into, so it has to get completions by running static analysis tools in your code (like Jedi and Rope). As I said, they introspect your code without running it. While they work very well for pure Python code, they have the problems I described above for compiled libraries.
And trying to evaluate the code you have in the Editor to get completions is usually not a good idea because:
It is not necessarily valid Python code all the time. For example, suppose you left an unclosed parenthesis somewhere, but you want to get completions at some other point. That should work without problems, right?
It could involve a very costly computation (e.g. loading a huge CSV in a Dataframe), so evaluating it every time to get completions (and that's a must because your code is different every time you ask for completions) could consume all your RAM in a blink.
it would be great to know why is the autocompletion better in Rodeo (another IDE)
Last time I checked (a couple of years ago), Rodeo evaluated your code to get completions. However, we'll take a look at what they are doing now to see if we can improve our completion machinery.
Autocompletion works correctly if there are NO white spaces in the project working directory path.
Autocomplete was not working for me at all.
So, I tried Tools -> Reset Sypder to factory defaults and it worked.
I have created a Custom Extension Command in Python. I installed it, but as expected I am getting errors (quote from SPSS log output - the only way I know for debugging Python programs in SPSS):
Extension command TEST_EXTENSION could not be loaded. The module or a module that it requires may be missing, or there may be syntax errors in it.
The error is probably from the .xmlor from the Run(args) function. The CustomFunction() I am implementing was tested thoroughly.
What would be a good practice for debugging this, and the other potential errors ? The official IBM-SPSS-Statistics-Extension-Command says to
set the
SPSS_EXTENSIONS_RAISE
environment variable to "true"
but I don't know how to do that, nor of this will work regardless of the source of the error.
#horace
You set the environment variable on Windows via the Control Panel > System > Advanced system settings > Environment Variables. The exact wording varies with different Windows versions. I usually choose System variables, although either will usually work. You need to restart Statistics after that. Once you have set this variable, errors in the Python code will produce a traceback. The traceback is ordinarily suppressed as it is of no use to users, but it is very helpful for developers.
The traceback only appears for errors in the Python code. The "could not be loaded" error you reported happens before Python gets control, so no traceback would be produced. There are two common causes for this error. The first is that the xml file defining the extension command or the corresponding Python module was not found by Statistics. The extension command definitions are loaded at Statistics startup or by running the EXTENSION command. Execute SHOW EXT. from the Syntax Editor to see the places where Statistics looks for extension files.
The second cause is a syntax error in the Python code. Run
begin program.
import yourmodule
end program.
to see if any errors are reported.
More generally, there are two useful strategies for debugging. The first is to run the code in external mode, where you run the code from Python. That way you can step through the code using your IDE or the plain Python debugger. See the programmability documentation for details. There are some limitations on what can be done in external mode, but it is often a good solution.
The second is to use an IDE that supports remote debugging. I use Wing IDE, but there are other IDEs that can do this. That lets me jump into the debugger from within Statistics, step through the Python code, and do all the other things you want in a debugger.
HTh
I'm trying to get back into coding for some math/physics experimentations and found VTK as a powerful tool using python. So I installed Python(x,y) and Pycharm Community edition. But I cannot get the Code Completion for VTK to work. I know this question has been posted quite a lot of times, but I couldn't find any concrete answer.
Here's what I know so far:
In order for Code Completion to work Pycharm constructs Skeletons. (Basically Python files with empty Classes/Methods that match the C++ API and can then be used like any other Python file for code completion.)
If I locate these files they don't appear to be complete and look something like this:
If this is indeed the skeleton (the file is called vtkRenderingPython.py) then shouldn't there be empty function declarations?
The result is that I get code completion for the classnames, but not the functions. For a library this huge that's rather annoying. Is there an easy way to get this working, or is this just a limitation I have to live with? Is there maybe a way to get complete Skeletons and replace the ones I have here? Am I missing the point entirely?
After another couple of hours I tried my luck with the PyDev extension for Eclipse. I didn't think that would work, but to my surprise it did! No settings necessary, it just worked out of the box.
The only drawback is that inherited methods are not shown in code completion. The base class is shown in the documentation window though so you can get the available functions by creating a temporary object of the base class and scrolling through the code completion there.