Okay, so I started a school project where I need to make multiple plots pop up during runtime and each requires an event handler for user input. My main machine is a Mac, and I've got it working without any issue. The problem is that my research lab also has a few linux and windows machines, and I would like to get my program working on them as well. It seems that the underlying culprit is that each operating system uses a different backend for how it plots figures in matplotlib. I'm not a CS student, and thus feel a little unsure how to fix this. Is there a standard backend that windows 10, ubuntu, Mac, etc. that I could hard code into my program to make sure my code runs correctly on all of them?
The main idea of having backends is that you wouldn't need to care about how the plotting is performed and let the backend do everything. That said, you probably wouldn't want to hardcode any specific backend and let matplotlib decide which one to use on each machine.
You also may want to read matplotlib backends - do I care?
There might be some special issues, such that certain things don't work with certain backends, but since there is no information about that given in the question, the general tendency would be not to set any backend at all.
Related
I'm using PyCharm with a remote interpreter. My code is showing a scatter figure using matplotlib. I can see the figure normally and interact with (zoom and rotate), but the interaction is too slow. I think this is because it's done through the SSH X-server.
My question is how to manually make the figure interactions graphically (using mouse, like zooming, panning, rotation, ...) execute faster? I think this should be via allowing the figure interactions to happen on local machine (machine running PyCharm), not the remote server.
One workaround is to save a .fig file, then copy it to my local machine, then load it. But I don't prefer this workaround.
Interacting with visualisations on separate windows is slow, not just on PyCharm but any IDE that you specify visualisations to be loaded separately. This is because they are loaded on either tkinter or any of the available modules like Qt5 and if you're on a mac, you can also use the inbuilt OSX rendering. Especially if your dataset is large then interaction becomes slower. It just seems that visualisations in Python are not exactly that optimised as you witness on Tableau or even Orange/Glue.
That's why I personally prefer visualising data inline through the IPython console (for that I use Spyder). You cannot interact with plots though, but the purpose suffices.
Running on Mac Sierra, the autocompletion in Spyder (from Anaconda distribution), seems quite erratic. When used from the Ipython console, works as expected. However, when used from the editor (which is my main way of writing), is erratic. The autocompletion works (i.e. when pressing TAB a little box appears showing options) for some modules, such as pandas or matplotlib. So writing 'pd.' and hitting TAB, gets the box with options as expected. However, this does not happen with many other objects: for example, after defining a dataframe named 'df', typing 'df.' TAB shows nothing. In the Ipython console, 'df.' TAB would show the available procedures for that dataframe, such as groupby, and also its columns, etc..
So the question is threefold. First, is there any particular configuration that should be enabled to get this to work? I don't think so, given some time spent googling, but just wanna make sure. Second, could someone state what is the official word on what works and what doesn't in terms of autocompletion (e.g. what particular modules do work from the editor, and which ones doesn't?). Finally, what are the technical aspects of the differences between the editor and the Ipython console in the performance of the autocompletion with Spyder? I read something about Jedi vs. PsychoPy modules, so got curious (however, please keep in mind that although I have scientific experience, I am relatively new to computation, so please keep it reasonably simple for an educated but not expert person).
UPDATE: As a side question, it would be great to know why is the autocompletion better in Rodeo (another IDE). It is more new, has way fewer overall options than Spyder, but the autocompletion works perfectly in the editor.
(Spyder developer here)
My answers:
is there any particular configuration that should be enabled to get this to work?
In Spyder 3.1 we added the numpydoc library to improve completions of some objects (like Matplotlib figures and NumPy arrays). If Dataframe completions are not working for you (they are for me), please open an issue in our issue tracker on Github to track and solve this problem.
could someone state what is the official word on what works and what doesn't in terms of autocompletion (e.g. what particular modules do work from the editor, and which ones doesn't?)
The most difficult part is getting completions of definitions when an object is generated by functions or methods developed in C/C++/Fortran and not in Python. I mean, things like
import numpy as np
a = np.array([])
a.<TAB>
As I said, this should be working now for arrays, figures and dataframes, but it doesn't work for all libraries (and most scientific Python libraries are created in C/C++/Fortran and wrapped in Python for speed).
The problem is that the completion libraries we use (Rope and Jedi) can't deal with this case very well because array (for example) can't be introspected in a static way (i.e. without running code involving it). So we have to resort to tricks like analyzing array's docstring to see its return type and introspect that instead.
what are the technical aspects of the differences between the editor and the Ipython console in the performance of the autocompletion with Spyder?
The most important difference is that in the IPython console you have to run your code before getting completions about it. For example, please run this in a fresh IPython console
In [1]: import pandas as pd
...: df = pd.Da<Tab>
and you will see that it won't return you any completions for Da (when it obviously should return Dataframe).
But, after evaluation, it is quite simple to get completions. You can simply run
dir(pd)
to get them (that's what IPython essentially does internally).
On the other hand, Spyder's Editor doesn't have a console to run code into, so it has to get completions by running static analysis tools in your code (like Jedi and Rope). As I said, they introspect your code without running it. While they work very well for pure Python code, they have the problems I described above for compiled libraries.
And trying to evaluate the code you have in the Editor to get completions is usually not a good idea because:
It is not necessarily valid Python code all the time. For example, suppose you left an unclosed parenthesis somewhere, but you want to get completions at some other point. That should work without problems, right?
It could involve a very costly computation (e.g. loading a huge CSV in a Dataframe), so evaluating it every time to get completions (and that's a must because your code is different every time you ask for completions) could consume all your RAM in a blink.
it would be great to know why is the autocompletion better in Rodeo (another IDE)
Last time I checked (a couple of years ago), Rodeo evaluated your code to get completions. However, we'll take a look at what they are doing now to see if we can improve our completion machinery.
Autocompletion works correctly if there are NO white spaces in the project working directory path.
Autocomplete was not working for me at all.
So, I tried Tools -> Reset Sypder to factory defaults and it worked.
I have been working with pyplot.imshow() all week.
Everytime I debug and I hit pyplot.imshow(), an image displays. Now, all of a sudden, no image displays until I hit pyplot.show().
What is going on here? What has changed?
Here is my setup:
I was using Pycharm.
I loaded matplotlib, sklearn, numpy, pandas.
I created a couple of classes in a separate file, and was running the classes to produce images in the file in question.
During the production of the images, I had to look up "imshow" (my experience level with python is only a few months). After implementing "imshow", the images appeared when I passed the "imshow" line in a debugging session. That is just the way it behaved.
I used this repeatedly for days. Then, all of a sudden (and I may have closed and reopened Pycharm, I can't remember), the "imshow()" function started intermittently displaying an image (sometimes it did, sometimes it didn't).
So I started adding "show()" beneath the "imshow()" calls...and that worked (sort of). Very soon, however that started working only intermittently. I would pass "show()" during debugging sessions and nothing would show up.
This mysterious behavior just began. I changed zero settings. No one with computer knowledge had any access to the PC (just my parents during the superbowl...).
Whether "imshow()" is supposed to display something or not, "show()" should always work, right? How is it that both of these functions just stopped working as expected, with only a possible IDE restart that I'm reasonably sure didn't happen...(no computer restart)...and no python package installs.
(the only thing I installed was notepadqq...but the functions stopped working properly several hours after that)
I think you may have changed the way you run your script (e.g. from ipython with the --pylab or --matplotlib switches to the standard python interpreter) or have changed version or default mode of ipython.
In any case, if you use pyplot.ion() before calling pyplot.imshow(...), the image should display automatically.
I just posted a question here why python imports take as long as they do. Are there environments that don't require reinitializing modules? If so, what are they?
Details: I'm trying to learn basic python syntax while using extended libraries (matplotlib, mayavi), and each time I test my code I wait (several!!) seconds for the modules to load. There must be a faster way to do this, but I don't know what environments are well suited. Suggestions?
Take a look at ipython and pandas they might be closer to what you want. Python does have a reload for modules but I'm not sure how well it works so anything that keeps a single python instance running and doesn't spawn python child processes is likely to fit the bill (sorry not sure what's available in that area).
http://ipython.org/
http://pandas.pydata.org/
Any environment with client/server architecture (short-lived cli/gui/web-clients, long-lived computational kernels) such as https://jupyter.org/ will do.
I have been developing a fairly extensive library of python modules that automate the more time consuming parts of "3D character development" for games/film/tv.
All of my code up until a few months ago has been run within Maya's dedicated python interpreter, however, my GUIs are built in PySide/PyQt, and so, run just fine in mac/windows/linux or a few other Graphics programs such as Nuke, XSI, Max.
What I would really like to figure out is a "simple" way to distribute my code to various different people ---> using various different operating Systems ---> potentially using various applications (Nuke, XSI, Max), which, in turn, have their own dedicated python interpreters.
The obvious option would be pip and easy_install.. These modules are clearly the "right" way to go, but its not really clear how a user would install/run them under the dedicated python installs that ship with Maya/Nuke/ etc...Though, it does seem possible (as explained here). Still Its going to be a pretty big barrier for a less-technical user.
Any help or points in the right direction would be immensely appreciated..
I would not say that pip/easy_install are the 'right' way for this problem. They are pretty good (not quite 'great') tools for motivated, technically inclined users -- but even in that context they have issues (such as unintended upgrades or deletions). Most importantly, they are opt-in methods: nobody can make you pip unless you want to. This means users can accidentally or deliberately get themselves into very different positions from each other, which makes support and maintenance a nightmare.
I've had very good luck in Maya distributing a zipped file containing a complete environment - all the modules etc. userSetup.py adds that zip to the path and the Python's native zipimport functionality handles the rest. This makes sure that there is only one file to maintain and distribute. It also fixes the common problem of leftover .pyc files creating havok after .py files get moved or renamed. Since this is all standard python, I'd assume this will work for any app-specific python that uses a 2.6+ version of python, though I've never tried it in Nuke or Max.
The main wrinkle will be modules with .pyd or other binary components, typically these don't work inside the zip files. I include a bootstrap routine which unpacks those to a (disposable) location on the user's disk and adds that to the path.
There's a detailed discussion of the method here and some background here