I write a number of plots to a pdf with a loop like the following. It works but there are two very annoying issues,
1) When the loop runs, i see a lot of windows ('Figure 1') popped up. I think the command plt.close(fig) does not work as intended. This' really annoying because I might be doing something else when it runs and those pop-ups block my view to the other tasks.
2) Probably related to 1), memory usage goes up dramatically. In my real script, plotting something like 50 pages of graphs eats up > 32 Gb of ram. How could that be?!
with PdfPages('Manyplots.pdf') as pdf:
for j in xrange(100):
fig = plt.figure(1, figsize=(5,5))
for fr in xrange(9):
pp = fig.add_subplot(3,3,fr+1)
pp.imshow(x, cmap=plt.cm.gray)
pdf.savefig()
plt.close(fig)
My questions are
1) any way to close a figure after the plot is done?
2) better still, how to suppress blank Figure pop-up since it should really be writing to an external file in the background,
3) any better way to save a series of plots to multiple pages of PDF?
Found the cause of the problem. My main script includes an import of someone's utility script, which imports pyplot and has an extra line,
import matplotlib.pyplot as plt
plt.ion()
When plt.ion is commented out, the popups are gone.
Related
After many hours searching I am looking for a straightforward answer to the question... "Is there ANY way to run a Python3 script from the command line, which generates a plot, and have the plot remain on-screen after the script ends"?
Ideally I would love to leave the plot running in the background, and have it remain interactive enough to allow for zooming, panning and resizing (don't care cabout updating data ... yet), but I'll settle for something as low-tech as just leaving the plot there, so I can rerun the same script with different data, so I can compare plots (ya, I know I can run the entire py script in the background if necessary, which is less elegant, but may be necessary).
Some possibilities that have some to mind that may or may not be possible: have the main script spawn a background/detached process that does the plotting; use threading; keep the script running while I want to zoom/pan/resize, then leave the plot in a static state (like a picture) when the script ends.
Tried maybe a dozen or so methods posted, but none work so far.
If it can't be done, please someone just give a short and simple answer that says so, so I can move on and kludge something together like writing an image file, and spawning a background shell that that displays a picture. Or possibly, going back to something low-tech like Bash, which will allow me to use GNUplot (yup, that works amazingly ok compared to matpltlib, so far).
Thanks to anyone who can save my sanity.
-G
Here is some of what I tried from other posts:
plt.show(block=False) will not even show a plot unless preceeded by a plt.pause(). even plt.draw() does not produce a plot. only plt.pause() before the plt.show(block=False) gives me a plot, and then the plot closes when the script ends
plt.pause(0.01) allows for zooming/resizing/etc while in a loop (which I can live with), but no way to leave (even a static plot) after the script ends. This is usable, if I can leave the plot on-screen after the script ends.
plt.draw(), plt.ion() gives anomalous results, including blank plots or no plots at all
You could just open the saved image figure, for example in this post
import sys
import subprocess
def openImage(path):
imageViewerFromCommandLine = {'linux':'xdg-open',
'win32':'explorer',
'darwin':'open'}[sys.platform]
subprocess.Popen([imageViewerFromCommandLine, path])
import matplotlib.pyplot as plt
import numpy as np
# Data for plotting
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2 * np.pi * t)
fig, ax = plt.subplots()
ax.plot(t, s)
ax.set(xlabel='time (s)', ylabel='voltage (mV)',
title='About as simple as it gets, folks')
ax.grid()
fig.savefig("test.png")
openImage('test.png')
Just use background execution.
So I've been testing PyPy a while (PyPy3.9 with matplotlib 3.6.3 using conda), and it seems satisfactory so far. Now, I tried using it along with matplotlib on jupyter, because that's what I do every day, and it seems that PyPy handles the matplotlib, or jupyter, far worse than CPython (CPython3.9 with matplotlib 3.6.3). I wrote a short notebook with following three cells,
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(-5,5, num=1024)
for i in range(20):
ybase = (np.random.randn(1024)+1j*np.random.randn(1024))*(1/(1+(np.arange(1024)/10)**(4)))
y = np.real(np.fft.fft(ybase))
ydots = [y+0.5*np.random.randn(1024), y*0+0.5]
fig, ax = plt.subplots()
ax.plot(x,y)
ax.errorbar(x, ydots[0], yerr=ydots[1], color='k', ls='', marker='o',
elinewidth=0.3, markersize=3)
ax.set_title(f"{i}")
fig.tight_layout()
This should display 20 instances of matplotlib figures with a line and a scatter. First of all, in both CPython and PyPy, they run without errors within reasonable time, and created 20 instances of empty matplotlib figure. Over time, although it is slow, the jupyter eventually renders the figures I draw when I was using CPython kernel.
However, if I use PyPy kernel, it draws up to ~6 figures yet other 14 figures still blank, and after that, although the loop has been technically finished, kernel is still busy and fails to run any subsequent commands.
The results of this depends on a lot of factors, for instance, when I refreshed the browser tab that displays the notebook may affect how many figures would be drawn in PyPy, but in CPython, although time it takes may vary, it eventually draws all figures regardless of these conditions. This gave me the impression that the PyPy, or the matplotlib on PyPy, does not handle the memory very well. Also, while sometimes it helps if I put gc.collect() at the end of each loop, it does not always work. And even if it succeeds, the controls of matplotlib figures are all dead, so this is not desirable.
So, why does this happen? Can it be resolved by updating a couple of packages?
I have a script that runs repeatedly and in the process it saves a figure into a folder. After a while I start getting warnings about too many open figures in memory.
I checked other questions on the topic, for example, this one and added plt.close('all') to my code so now it looks like this:
fig, ax = plt.subplots(figsize=(17,8))
plt.hist(results_df['Diff'], bins=100, density=True, histtype='step')
plt.savefig(f'backtester_results/figures/rf_model_{n_days}_data_and_{lag}_lag.png',
format='png')
plt.close('all')
And yet I keep getting figures piled up in memory and warnings after a while. Where did I go wrong?
Here's the warning:
RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory.
When reading in the official documentation I would assume that plt.close('all') only closes the windows without removing the figures (https://matplotlib.org/1.3.0/api/pyplot_api.html#matplotlib.pyplot.close).
As I understand you would need to clear the figure as follows:
fig.clf()
plt.close()
Source: (How can I release memory after creating matplotlib figures)
when I use inline plots in iPython (QtConsole), the first plot looks (more or less) fine, but then it gets weirder and weirder. When I plot something several times (so plot, see it displayed, plot again, see output etc.), it looks like it is being overlaid with the skewed previous picture. So after plotting a diagonal line (x=y) 4 times in a row I get something like this
If i right click and export it as svg everything looks good
(Exported PNG picture remains wrecked as the first one).
I guess the problem is similar to https://github.com/ipython/ipython/issues/1866, but I didn't got the upshot of the discussion (it got too technical and complicated for me to follow).
Is there any solution or work around for this issue?
I'm using
python 2.7
matplotlib 1.4.1
IPython 2.1.0
Here is a working example:
%matplotlib inline
% config InlineBackend.figure_format = 'svg'
import matplotlib.pyplot as plt
a=range(10)
fig,ax=plt.subplots()
ax.plot(a,a)
ax.axis('off')
if you remove plt.axis('off') line, weird things happen only outside of the axis box.
P.S. Originally I encountered this problem in connection with drawing graphs with networkx. If I use draw from networkx this problem does not occur. If I use draw_networkx, same as described above happens. That might point to the core of the problem... I'm trying to figure out what line of code makes one work better than the other...
After tinkering around with the draw and draw_networkx functions from networkx module, I found the workaround which makes the difference between draw and draw_networkx in this case.
Adding fig.set_facecolor('w') overlays whatever is in the background, so the new plots are started with a white sheet (but not a blank one, I guess).
So new working example is:
%matplotlib inline
% config InlineBackend.figure_format = 'svg'
import matplotlib.pyplot as plt
a=range(10)
fig,ax=plt.subplots()
fig.set_facecolor('w')
ax.plot(a,a)
ax.axis('off')
I am using an iterative loop to plot soame data using Matplotlib. When the code has saved around 768 plots, it throws the following exception.
RuntimeError: Could not allocate memory for image
My computer has around 3.5 GB RAM.
Is there any method to free the memory in parallel so that the memory does not get exhausted?
Are you remembering to close your figures when you are done with them? e.g.:
import matplotlib.pyplot as plt
#generate figure here
#...
plt.close(fig) #release resources associated with fig
As a slightly different answer, remember that you can re-use figures. Something like:
fig = plt.figure()
ax = plt.gca()
im = ax.imshow(data_list[0],...)
for new_data in data_list:
im.set_cdata(new_data)
fig.savefig(..)
Which will make your code run much faster as it will not need to set up and tear down the figure 700+ times.