I'm using PyCharm with a remote interpreter. My code is showing a scatter figure using matplotlib. I can see the figure normally and interact with (zoom and rotate), but the interaction is too slow. I think this is because it's done through the SSH X-server.
My question is how to manually make the figure interactions graphically (using mouse, like zooming, panning, rotation, ...) execute faster? I think this should be via allowing the figure interactions to happen on local machine (machine running PyCharm), not the remote server.
One workaround is to save a .fig file, then copy it to my local machine, then load it. But I don't prefer this workaround.
Interacting with visualisations on separate windows is slow, not just on PyCharm but any IDE that you specify visualisations to be loaded separately. This is because they are loaded on either tkinter or any of the available modules like Qt5 and if you're on a mac, you can also use the inbuilt OSX rendering. Especially if your dataset is large then interaction becomes slower. It just seems that visualisations in Python are not exactly that optimised as you witness on Tableau or even Orange/Glue.
That's why I personally prefer visualising data inline through the IPython console (for that I use Spyder). You cannot interact with plots though, but the purpose suffices.
Related
Okay, so I started a school project where I need to make multiple plots pop up during runtime and each requires an event handler for user input. My main machine is a Mac, and I've got it working without any issue. The problem is that my research lab also has a few linux and windows machines, and I would like to get my program working on them as well. It seems that the underlying culprit is that each operating system uses a different backend for how it plots figures in matplotlib. I'm not a CS student, and thus feel a little unsure how to fix this. Is there a standard backend that windows 10, ubuntu, Mac, etc. that I could hard code into my program to make sure my code runs correctly on all of them?
The main idea of having backends is that you wouldn't need to care about how the plotting is performed and let the backend do everything. That said, you probably wouldn't want to hardcode any specific backend and let matplotlib decide which one to use on each machine.
You also may want to read matplotlib backends - do I care?
There might be some special issues, such that certain things don't work with certain backends, but since there is no information about that given in the question, the general tendency would be not to set any backend at all.
Running on Mac Sierra, the autocompletion in Spyder (from Anaconda distribution), seems quite erratic. When used from the Ipython console, works as expected. However, when used from the editor (which is my main way of writing), is erratic. The autocompletion works (i.e. when pressing TAB a little box appears showing options) for some modules, such as pandas or matplotlib. So writing 'pd.' and hitting TAB, gets the box with options as expected. However, this does not happen with many other objects: for example, after defining a dataframe named 'df', typing 'df.' TAB shows nothing. In the Ipython console, 'df.' TAB would show the available procedures for that dataframe, such as groupby, and also its columns, etc..
So the question is threefold. First, is there any particular configuration that should be enabled to get this to work? I don't think so, given some time spent googling, but just wanna make sure. Second, could someone state what is the official word on what works and what doesn't in terms of autocompletion (e.g. what particular modules do work from the editor, and which ones doesn't?). Finally, what are the technical aspects of the differences between the editor and the Ipython console in the performance of the autocompletion with Spyder? I read something about Jedi vs. PsychoPy modules, so got curious (however, please keep in mind that although I have scientific experience, I am relatively new to computation, so please keep it reasonably simple for an educated but not expert person).
UPDATE: As a side question, it would be great to know why is the autocompletion better in Rodeo (another IDE). It is more new, has way fewer overall options than Spyder, but the autocompletion works perfectly in the editor.
(Spyder developer here)
My answers:
is there any particular configuration that should be enabled to get this to work?
In Spyder 3.1 we added the numpydoc library to improve completions of some objects (like Matplotlib figures and NumPy arrays). If Dataframe completions are not working for you (they are for me), please open an issue in our issue tracker on Github to track and solve this problem.
could someone state what is the official word on what works and what doesn't in terms of autocompletion (e.g. what particular modules do work from the editor, and which ones doesn't?)
The most difficult part is getting completions of definitions when an object is generated by functions or methods developed in C/C++/Fortran and not in Python. I mean, things like
import numpy as np
a = np.array([])
a.<TAB>
As I said, this should be working now for arrays, figures and dataframes, but it doesn't work for all libraries (and most scientific Python libraries are created in C/C++/Fortran and wrapped in Python for speed).
The problem is that the completion libraries we use (Rope and Jedi) can't deal with this case very well because array (for example) can't be introspected in a static way (i.e. without running code involving it). So we have to resort to tricks like analyzing array's docstring to see its return type and introspect that instead.
what are the technical aspects of the differences between the editor and the Ipython console in the performance of the autocompletion with Spyder?
The most important difference is that in the IPython console you have to run your code before getting completions about it. For example, please run this in a fresh IPython console
In [1]: import pandas as pd
...: df = pd.Da<Tab>
and you will see that it won't return you any completions for Da (when it obviously should return Dataframe).
But, after evaluation, it is quite simple to get completions. You can simply run
dir(pd)
to get them (that's what IPython essentially does internally).
On the other hand, Spyder's Editor doesn't have a console to run code into, so it has to get completions by running static analysis tools in your code (like Jedi and Rope). As I said, they introspect your code without running it. While they work very well for pure Python code, they have the problems I described above for compiled libraries.
And trying to evaluate the code you have in the Editor to get completions is usually not a good idea because:
It is not necessarily valid Python code all the time. For example, suppose you left an unclosed parenthesis somewhere, but you want to get completions at some other point. That should work without problems, right?
It could involve a very costly computation (e.g. loading a huge CSV in a Dataframe), so evaluating it every time to get completions (and that's a must because your code is different every time you ask for completions) could consume all your RAM in a blink.
it would be great to know why is the autocompletion better in Rodeo (another IDE)
Last time I checked (a couple of years ago), Rodeo evaluated your code to get completions. However, we'll take a look at what they are doing now to see if we can improve our completion machinery.
Autocompletion works correctly if there are NO white spaces in the project working directory path.
Autocomplete was not working for me at all.
So, I tried Tools -> Reset Sypder to factory defaults and it worked.
I have been working with pyplot.imshow() all week.
Everytime I debug and I hit pyplot.imshow(), an image displays. Now, all of a sudden, no image displays until I hit pyplot.show().
What is going on here? What has changed?
Here is my setup:
I was using Pycharm.
I loaded matplotlib, sklearn, numpy, pandas.
I created a couple of classes in a separate file, and was running the classes to produce images in the file in question.
During the production of the images, I had to look up "imshow" (my experience level with python is only a few months). After implementing "imshow", the images appeared when I passed the "imshow" line in a debugging session. That is just the way it behaved.
I used this repeatedly for days. Then, all of a sudden (and I may have closed and reopened Pycharm, I can't remember), the "imshow()" function started intermittently displaying an image (sometimes it did, sometimes it didn't).
So I started adding "show()" beneath the "imshow()" calls...and that worked (sort of). Very soon, however that started working only intermittently. I would pass "show()" during debugging sessions and nothing would show up.
This mysterious behavior just began. I changed zero settings. No one with computer knowledge had any access to the PC (just my parents during the superbowl...).
Whether "imshow()" is supposed to display something or not, "show()" should always work, right? How is it that both of these functions just stopped working as expected, with only a possible IDE restart that I'm reasonably sure didn't happen...(no computer restart)...and no python package installs.
(the only thing I installed was notepadqq...but the functions stopped working properly several hours after that)
I think you may have changed the way you run your script (e.g. from ipython with the --pylab or --matplotlib switches to the standard python interpreter) or have changed version or default mode of ipython.
In any case, if you use pyplot.ion() before calling pyplot.imshow(...), the image should display automatically.
I have a Python script that generates an interactive matplotlib plot with several sliders and radio buttons using the matplotlib.widgets submodule. I'd like to let others play with the resulting plot without having to install python, scipy, numpy, and matplotlib.
I first tried converting my python script to a stand-alone executable that I could distribute. This turned out to be a nightmare - every package I tried (pyinstaller, py2exe, cx_Freeze) failed for one reason or another. The main issue I had was with integrating various scipy and matplotlib libraries, and I'm now very pessimistic about successfully "freezing" my interactive plot.
My next idea was to see if I could get some interactivity through a web browser. Using an IPython notebook with the third-party package JSAnimation initially seemed very promising (http://nbviewer.ipython.org/github/jakevdp/JSAnimation/blob/master/animation_example.ipynb), but it seems that the package only supports the matplotlib "animate" function. This doesn't quite fit the bill for what I'd like to do, as I have multiple sliders which 1) doesn't seem to be supported by this package, and 2) even if supported, would likely result in too many static figures to pre-render effectively, since any such arrangement would require snapshots for every possible combination of the three variables.
Any ideas for how to get this interactive matplotlib plot to others without requiring them to install Python?
I have a massive file I need to parse and render on a remote machine, I already have scripts written using mayavi to do exactly this. What I'd like to do is save the image as a png and then copy the image over and view it locally. This is pretty simple to do with matplotlib by setting the backend to 'Agg', however I'm having some serious problems doing this with mayavi. I've followed the guide here
http://docs.enthought.com/mayavi/mayavi/tips.html
but the problem is that importing mlab alone requires access to the xdisplay, so I can't even turn on the virtual window as suggested.
To reproduce this, access a machine through ssh and run a simple python script like:
#!/usr/bin/python
from mayavi import mlab
and it will error out with the standard;
Unable to access the X Display, is $DISPLAY set properly?
If anyone has a fix for this, or an alternative route to rendering a 3D image remotely I'd appreciate it. The 3D rendering provided by matplotlib is insufficient for what I need so any suggestions for working alternatives to mayavi would be appreciated as well.
If you are on a *nix platform running an X server, use the solution in the documentation under rendering using the virtual framebuffer as dpinte commented above. I have used this method successfully before to run mayavi scripts headlessly.
The basic idea is to start a virtual X server such as Xvfb, and then invoke mayavi in the display context of the virtual X server (and yes this explanation is not quite right, but it will suffice).