I am relatively new to python / matplotlib and have to plot huge numpy arrays (more than 6 mio entries). The problem is that the 6 plots I have take more than 3 GB of RAM and take very long to load.
I researched a bit and found out that I can speed matplotlib up by not loading the axes and title every time.
So now the code looks like this but it's still quite slow.
Should I use another module instead of matplotlib?
How could I speed up the process?
Thanks a lot in advance
for key, typus in self.sensorObjects.items():
fig, ax = plt.subplots()
ax.set_title(key)
ax.set_xlabel('t (ms)')
ax.set_ylabel(self.sensorObjects[key][0].unit)
for sensor in typus:
data = sensor.physical_data
ax.plot(data)
fig.canvas.update()
Related
I'm using mplfinance module to plot candlesticks. The problem is mplfinance uses too much memory when it generates plots. I have tried the instructions mentioned in free up the memory used by matplotlib but nothing changed and my code is still fulling up my computer memory.Here is my code:
fig, axlist = mpf.plot(hloc,hlines=hlines,
ylabel='Price(USDT)',type='candle',
style='binance',title=my_title,closefig=True,returnfig=True)
any suggestion is highly appreciated.
It would be helpful to see the rest of your code, to see how you are displaying plots and how many. That said, given the above code, when you are done with each plot you might try:
for ax in axlist:
del ax
del fig
This will save memory, but at the expense of some time (which will anyway not be noticeable unless your are making thousands of plots).
If you are saving your plots to image files (instead of displaying to the screen) then matplotlib.use("Agg") may help as well.
I have a large df to plot (couple of millions of rows, 8 columns, obtained by concatenation of several files).
I want to plot several graphs using facet, in order to have complete view on data:
'''
rp = sns.relplot(data=df,
x='zscore',
y='%',
col='Nr',
row ="Support",
style="Metal",
kind='line')
'''
I tried both in Seaborn and Plotly Express but time to build this graphs is just too important, more than one hour on my laptop.
What can I improve, optimize, i order to speed graph creation?
Thank you!
PS. I do am a newbie in Python and programming ;)
I'm training a large DQN in Jupyter notebook. I'm having some trouble finding a way to update this plot in real-time without causing a memory leak. I currently have a dirty implementation that uses ~ 1GB of RAM per episode (14,000 steps). By the time I've gotten through 7 episodes like the screenshot below, I'm about halfway out of memory on my system.
From what I've read in other posts, attempting to plot in the same thread will cause a memory leak regardless of gc.collect() or del fig, fig.clear(), etc. How can I update this plot within a loop without causing a memory leak?
I found a similar question here, but couldn't quite figure out how to apply it in my case with multiple figures and data that is updated dynamically.
clear_output(wait=True)
plt.close()
plt.ion()
fig, axs = plt.subplots(2, figsize=(10,7))
fig.tight_layout()
color = [int((item + 1) * 255 / 2) for item in p_reward_history]
axs[0].scatter(tindex, p_reward_history[-plot_len:], c=color[-plot_len:], cmap='RdYlGn', linewidth=3)
axs[0].set_title('P&L Individual Transactions')
axs[0].plot(zero_line, color="black", linewidth=3)
axs[0].set_facecolor('#2c303c')
axs[1].set_title('P&L Running Total')
axs[1].set_facecolor('#2c303c')
axs[1].plot(running_rewards_history, color="#94c273", linewidth=3)
The variables that are dynamic are running_reward_history and p_reward_history. These are both lists that get new values appended to each loop.
Current implementation looks like this:
I prefer to work in Jupyter notebook, but if I need to train in a regular shell in order to update asynchronously, that is okay with me.
I am using matplotlib to plot more than one hundred of graphs. This is currently too slow and I'd like to optimize the code a bit.
Each figure contains up to 20 lines that am drawing this way (simplified):
f, ax = plt.subplots(1)
for i, y in enumerate(data):
ax.plot(tasks, res, marker=markers[i], label=labels[i])
I suppose that the method plot is actually drawing too much stuff (such as the axis). I tried using line.set_ydata but this replaced the previous line.
Is there a way to do something similar but faster?
I am using an iterative loop to plot soame data using Matplotlib. When the code has saved around 768 plots, it throws the following exception.
RuntimeError: Could not allocate memory for image
My computer has around 3.5 GB RAM.
Is there any method to free the memory in parallel so that the memory does not get exhausted?
Are you remembering to close your figures when you are done with them? e.g.:
import matplotlib.pyplot as plt
#generate figure here
#...
plt.close(fig) #release resources associated with fig
As a slightly different answer, remember that you can re-use figures. Something like:
fig = plt.figure()
ax = plt.gca()
im = ax.imshow(data_list[0],...)
for new_data in data_list:
im.set_cdata(new_data)
fig.savefig(..)
Which will make your code run much faster as it will not need to set up and tear down the figure 700+ times.