I have a script that runs repeatedly and in the process it saves a figure into a folder. After a while I start getting warnings about too many open figures in memory.
I checked other questions on the topic, for example, this one and added plt.close('all') to my code so now it looks like this:
fig, ax = plt.subplots(figsize=(17,8))
plt.hist(results_df['Diff'], bins=100, density=True, histtype='step')
plt.savefig(f'backtester_results/figures/rf_model_{n_days}_data_and_{lag}_lag.png',
format='png')
plt.close('all')
And yet I keep getting figures piled up in memory and warnings after a while. Where did I go wrong?
Here's the warning:
RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory.
When reading in the official documentation I would assume that plt.close('all') only closes the windows without removing the figures (https://matplotlib.org/1.3.0/api/pyplot_api.html#matplotlib.pyplot.close).
As I understand you would need to clear the figure as follows:
fig.clf()
plt.close()
Source: (How can I release memory after creating matplotlib figures)
Related
After many hours searching I am looking for a straightforward answer to the question... "Is there ANY way to run a Python3 script from the command line, which generates a plot, and have the plot remain on-screen after the script ends"?
Ideally I would love to leave the plot running in the background, and have it remain interactive enough to allow for zooming, panning and resizing (don't care cabout updating data ... yet), but I'll settle for something as low-tech as just leaving the plot there, so I can rerun the same script with different data, so I can compare plots (ya, I know I can run the entire py script in the background if necessary, which is less elegant, but may be necessary).
Some possibilities that have some to mind that may or may not be possible: have the main script spawn a background/detached process that does the plotting; use threading; keep the script running while I want to zoom/pan/resize, then leave the plot in a static state (like a picture) when the script ends.
Tried maybe a dozen or so methods posted, but none work so far.
If it can't be done, please someone just give a short and simple answer that says so, so I can move on and kludge something together like writing an image file, and spawning a background shell that that displays a picture. Or possibly, going back to something low-tech like Bash, which will allow me to use GNUplot (yup, that works amazingly ok compared to matpltlib, so far).
Thanks to anyone who can save my sanity.
-G
Here is some of what I tried from other posts:
plt.show(block=False) will not even show a plot unless preceeded by a plt.pause(). even plt.draw() does not produce a plot. only plt.pause() before the plt.show(block=False) gives me a plot, and then the plot closes when the script ends
plt.pause(0.01) allows for zooming/resizing/etc while in a loop (which I can live with), but no way to leave (even a static plot) after the script ends. This is usable, if I can leave the plot on-screen after the script ends.
plt.draw(), plt.ion() gives anomalous results, including blank plots or no plots at all
You could just open the saved image figure, for example in this post
import sys
import subprocess
def openImage(path):
imageViewerFromCommandLine = {'linux':'xdg-open',
'win32':'explorer',
'darwin':'open'}[sys.platform]
subprocess.Popen([imageViewerFromCommandLine, path])
import matplotlib.pyplot as plt
import numpy as np
# Data for plotting
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2 * np.pi * t)
fig, ax = plt.subplots()
ax.plot(t, s)
ax.set(xlabel='time (s)', ylabel='voltage (mV)',
title='About as simple as it gets, folks')
ax.grid()
fig.savefig("test.png")
openImage('test.png')
Just use background execution.
I'm using mplfinance module to plot candlesticks. The problem is mplfinance uses too much memory when it generates plots. I have tried the instructions mentioned in free up the memory used by matplotlib but nothing changed and my code is still fulling up my computer memory.Here is my code:
fig, axlist = mpf.plot(hloc,hlines=hlines,
ylabel='Price(USDT)',type='candle',
style='binance',title=my_title,closefig=True,returnfig=True)
any suggestion is highly appreciated.
It would be helpful to see the rest of your code, to see how you are displaying plots and how many. That said, given the above code, when you are done with each plot you might try:
for ax in axlist:
del ax
del fig
This will save memory, but at the expense of some time (which will anyway not be noticeable unless your are making thousands of plots).
If you are saving your plots to image files (instead of displaying to the screen) then matplotlib.use("Agg") may help as well.
I had a look at other questions on that regard, but in all other cases people had
%matplotlib inline
which caused plots to show without being prompted. I don't have it.
My plotting code looks like this:
fig, ax = plt.subplots(figsize=(17,8))
ax = plt.hist(results_df['Diff'], bins=100, density=True, histtype='step')
plt.savefig('backtester_results/figures/rf_model_lag_{n_days}_data_{lag}_lag.png',
format='png')
It's an output from testing machine learning model I'm working on and I want to put it to run for 40 hours and there will be few hundred of these plots generated in the process. So I really want them not to just get saved in a folder and not take up memory.
Do you know how to suppress this behaviour or what could be causing it?
I'm using Python 3.6 and Spyder editor. In Spyder settings I have Graphics backend set up as 'Automatic'.
I am plotting histograms with quite large number of bins. I am using Spyder, Python 3.6.3. The problem I found is that the figure I am seeing in iPython console is NOT the same as the saved plots. I have seen this thread which asks a similar question, however, my problem is worse, as it's not just the fonts and sizes that vary, I am actually getting different counts!
E.g. the same script:
plt.clf()
fig, ax=plt.subplots()
fig.dpi=100
plt.hist(df['POS'], bins=nbins, range=(0,dict_l[c]))
plt.savefig('current_chr17.jpg', dpi=100)
will produce this plot as a saved figure:
and show this in iPython console (interactive mode on, I'm not even asking for plt.show):
Does anyone have any explanation as to what is going on?
I write a number of plots to a pdf with a loop like the following. It works but there are two very annoying issues,
1) When the loop runs, i see a lot of windows ('Figure 1') popped up. I think the command plt.close(fig) does not work as intended. This' really annoying because I might be doing something else when it runs and those pop-ups block my view to the other tasks.
2) Probably related to 1), memory usage goes up dramatically. In my real script, plotting something like 50 pages of graphs eats up > 32 Gb of ram. How could that be?!
with PdfPages('Manyplots.pdf') as pdf:
for j in xrange(100):
fig = plt.figure(1, figsize=(5,5))
for fr in xrange(9):
pp = fig.add_subplot(3,3,fr+1)
pp.imshow(x, cmap=plt.cm.gray)
pdf.savefig()
plt.close(fig)
My questions are
1) any way to close a figure after the plot is done?
2) better still, how to suppress blank Figure pop-up since it should really be writing to an external file in the background,
3) any better way to save a series of plots to multiple pages of PDF?
Found the cause of the problem. My main script includes an import of someone's utility script, which imports pyplot and has an extra line,
import matplotlib.pyplot as plt
plt.ion()
When plt.ion is commented out, the popups are gone.