We Keep Coding

Writing png files to a zip archive with Python 3.5 zipfile, getting The parameter is incorrect with winzip

In a process almost perfectly described by the question in this post, I am
writing png files of matplotlib plots to a BytesIO instance. I am then writing each of those to another BytesIO instance with an instance of ZipFile, calling zipfile.writestr method.
Making the plots
import pandas as pd
import matplotlib.pyplot as plt
def write_plot(data):
plot_buff = BytesIO()
fig, ax = plt.subplots()
dataframe = pd.DataFrame(data)
dataframe.plot(x="length", y="left", ax=ax, color="b")
dataframe.plot(x="length", y="right", ax=ax, color="r")
plt.savefig(plot_buff)
return plot_buff
Archiving the plots
zip_buff = BytesIO()
with ZipFile(zip_buff, "w") as zipfile:
for number, data_set in enumerate(data_sets):
plot = write_plot(data_set)
zipfile.writestr("{}.png".format(number), plot.getvalue())
with open(file_path, "wb") as write_buff:
write_buff.write(zip_buff.getvalue())
But the zip archive I get back gives me the error :
Error 0x80070057: The parameter is incorrect
It opens fine in 7-zip, but I can't expect my users to know or try that.
Edit: Sorry, the missing "wb" param was a typo in the question, it is part of my actual code.

Apologies, this was a bad example. The issue I was having could not be determined from it and was pretty simple. In the course of building a fully verifiable example, I discovered that the example code does produce png images which can be extracted with WinZip. This gave me a reference point between my actual code which was, slightly different and the example. It turned out that I had put ":" into the file names.

Convert SVG/PDF to EMF

I am looking for a way to save a matplotlib figure as an EMF file. Matplotlib allows me to save as either a PDF or SVG vector file but not as EMF.
After a long search I still cannot seem to find a way to do this with python. Hopefully anyone has an idea.
My workaround is to call inkscape using subprocess but this is far from ideal as I would like to avoid the use of external programs.
I'm running python 2.7.5 and matplotlib 1.3.0 using the wx backend.

For anyone who still needs this, I wrote a basic function that can let you save a file as an emf from matplotlib, as long as you have inkscape installed.
I know the op didn't want inkscape, but people who find this post later just want to make it work.
import matplotlib.pyplot as plt
import subprocess
import os
inkscapePath = r"path\to\inkscape.exe"
savePath= r"path\to\images\folder"
def exportEmf(savePath, plotName, fig=None, keepSVG=False):
"""Save a figure as an emf file
Parameters
----------
savePath : str, the path to the directory you want the image saved in
plotName : str, the name of the image
fig : matplotlib figure, (optional, default uses gca)
keepSVG : bool, whether to keep the interim svg file
"""
figFolder = savePath + r"\{}.{}"
svgFile = figFolder.format(plotName,"svg")
emfFile = figFolder.format(plotName,"emf")
if fig:
use=fig
else:
use=plt
use.savefig(svgFile)
subprocess.run([inkscapePath, svgFile, '-M', emfFile])
if not keepSVG:
os.system('del "{}"'.format(svgFile))
#Example Usage
import numpy as np
tt = np.linspace(0, 2*3.14159)
plt.plot(tt, np.sin(tt))
exportEmf(r"C:\Users\userName", 'FileName')

I think the function is cool but inkscape syntax seems not working in my case. I search in other post and find it as:
inkscape filename.svg --export-filename filename.emf
So if I replace -M by --export-filename within the subprocess argument, everything works fine.

Save an "interactive figure" with matplotlib [duplicate]

Is there a way to save a Matplotlib figure such that it can be re-opened and have typical interaction restored? (Like the .fig format in MATLAB?)
I find myself running the same scripts many times to generate these interactive figures. Or I'm sending my colleagues multiple static PNG files to show different aspects of a plot. I'd rather send the figure object and have them interact with it themselves.

I just found out how to do this. The "experimental pickle support" mentioned by #pelson works quite well.
Try this:
# Plot something
import matplotlib.pyplot as plt
fig,ax = plt.subplots()
ax.plot([1,2,3],[10,-10,30])
After your interactive tweaking, save the figure object as a binary file:
import pickle
pickle.dump(fig, open('FigureObject.fig.pickle', 'wb')) # This is for Python 3 - py2 may need `file` instead of `open`
Later, open the figure and the tweaks should be saved and GUI interactivity should be present:
import pickle
figx = pickle.load(open('FigureObject.fig.pickle', 'rb'))
figx.show() # Show the figure, edit it, etc.!
You can even extract the data from the plots:
data = figx.axes[0].lines[0].get_data()
(It works for lines, pcolor & imshow - pcolormesh works with some tricks to reconstruct the flattened data.)
I got the excellent tip from Saving Matplotlib Figures Using Pickle.

As of Matplotlib 1.2, we now have experimental pickle support. Give that a go and see if it works well for your case. If you have any issues, please let us know on the Matplotlib mailing list or by opening an issue on github.com/matplotlib/matplotlib.

This would be a great feature, but AFAIK it isn't implemented in Matplotlib and likely would be difficult to implement yourself due to the way figures are stored.
I'd suggest either (a) separate processing the data from generating the figure (which saves data with a unique name) and write a figure generating script (loading a specified file of the saved data) and editing as you see fit or (b) save as PDF/SVG/PostScript format and edit in some fancy figure editor like Adobe Illustrator (or Inkscape).
EDIT post Fall 2012: As others pointed out below (though mentioning here as this is the accepted answer), Matplotlib since version 1.2 allowed you to pickle figures. As the release notes state, it is an experimental feature and does not support saving a figure in one matplotlib version and opening in another. It's also generally unsecure to restore a pickle from an untrusted source.
For sharing/later editing plots (that require significant data processing first and may need to be tweaked months later say during peer review for a scientific publication), I still recommend the workflow of (1) have a data processing script that before generating a plot saves the processed data (that goes into your plot) into a file, and (2) have a separate plot generation script (that you adjust as necessary) to recreate the plot. This way for each plot you can quickly run a script and re-generate it (and quickly copy over your plot settings with new data). That said, pickling a figure could be convenient for short term/interactive/exploratory data analysis.

Why not just send the Python script? MATLAB's .fig files require the recipient to have MATLAB to display them, so that's about equivalent to sending a Python script that requires Matplotlib to display.
Alternatively (disclaimer: I haven't tried this yet), you could try pickling the figure:
import pickle
output = open('interactive figure.pickle', 'wb')
pickle.dump(gcf(), output)
output.close()

Good question. Here is the doc text from pylab.save:
pylab no longer provides a save function, though the old pylab
function is still available as matplotlib.mlab.save (you can still
refer to it in pylab as "mlab.save"). However, for plain text
files, we recommend numpy.savetxt. For saving numpy arrays,
we recommend numpy.save, and its analog numpy.load, which are
available in pylab as np.save and np.load.

I figured out a relatively simple way (yet slightly unconventional) to save my matplotlib figures. It works like this:
import libscript
import matplotlib.pyplot as plt
import numpy as np
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2*np.pi*t)
#<plot>
plt.plot(t, s)
plt.xlabel('time (s)')
plt.ylabel('voltage (mV)')
plt.title('About as simple as it gets, folks')
plt.grid(True)
plt.show()
#</plot>
save_plot(fileName='plot_01.py',obj=sys.argv[0],sel='plot',ctx=libscript.get_ctx(ctx_global=globals(),ctx_local=locals()))
with function save_plot defined like this (simple version to understand the logic):
def save_plot(fileName='',obj=None,sel='',ctx={}):
"""
Save of matplolib plot to a stand alone python script containing all the data and configuration instructions to regenerate the interactive matplotlib figure.
Parameters
----------
fileName : [string] Path of the python script file to be created.
obj : [object] Function or python object containing the lines of code to create and configure the plot to be saved.
sel : [string] Name of the tag enclosing the lines of code to create and configure the plot to be saved.
ctx : [dict] Dictionary containing the execution context. Values for variables not defined in the lines of code for the plot will be fetched from the context.
Returns
-------
Return ``'done'`` once the plot has been saved to a python script file. This file contains all the input data and configuration to re-create the original interactive matplotlib figure.
"""
import os
import libscript
N_indent=4
src=libscript.get_src(obj=obj,sel=sel)
src=libscript.prepend_ctx(src=src,ctx=ctx,debug=False)
src='\n'.join([' '*N_indent+line for line in src.split('\n')])
if(os.path.isfile(fileName)): os.remove(fileName)
with open(fileName,'w') as f:
f.write('import sys\n')
f.write('sys.dont_write_bytecode=True\n')
f.write('def main():\n')
f.write(src+'\n')
f.write('if(__name__=="__main__"):\n')
f.write(' '*N_indent+'main()\n')
return 'done'
or defining function save_plot like this (better version using zip compression to produce lighter figure files):
def save_plot(fileName='',obj=None,sel='',ctx={}):
import os
import json
import zlib
import base64
import libscript
N_indent=4
level=9#0 to 9, default: 6
src=libscript.get_src(obj=obj,sel=sel)
obj=libscript.load_obj(src=src,ctx=ctx,debug=False)
bin=base64.b64encode(zlib.compress(json.dumps(obj),level))
if(os.path.isfile(fileName)): os.remove(fileName)
with open(fileName,'w') as f:
f.write('import sys\n')
f.write('sys.dont_write_bytecode=True\n')
f.write('def main():\n')
f.write(' '*N_indent+'import base64\n')
f.write(' '*N_indent+'import zlib\n')
f.write(' '*N_indent+'import json\n')
f.write(' '*N_indent+'import libscript\n')
f.write(' '*N_indent+'bin="'+str(bin)+'"\n')
f.write(' '*N_indent+'obj=json.loads(zlib.decompress(base64.b64decode(bin)))\n')
f.write(' '*N_indent+'libscript.exec_obj(obj=obj,tempfile=False)\n')
f.write('if(__name__=="__main__"):\n')
f.write(' '*N_indent+'main()\n')
return 'done'
This makes use a module libscript of my own, which mostly relies on modules inspect and ast. I can try to share it on Github if interest is expressed (it would first require some cleanup and me to get started with Github).
The idea behind this save_plot function and libscript module is to fetch the python instructions that create the figure (using module inspect), analyze them (using module ast) to extract all variables, functions and modules import it relies on, extract these from the execution context and serialize them as python instructions (code for variables will be like t=[0.0,2.0,0.01] ... and code for modules will be like import matplotlib.pyplot as plt ...) prepended to the figure instructions. The resulting python instructions are saved as a python script whose execution will re-build the original matplotlib figure.
As you can imagine, this works well for most (if not all) matplotlib figures.

If you are looking to save python plots as an interactive figure to modify and share with others like MATLAB .fig file then you can try to use the following code. Here z_data.values is just a numpy ndarray and so you can use the same code to plot and save your own data. No need of using pandas then.
The file generated here can be opened and interactively modified by anyone with or without python just by clicking on it and opening in browsers like Chrome/Firefox/Edge etc.
import plotly.graph_objects as go
import pandas as pd
z_data=pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/api_docs/mt_bruno_elevation.csv')
fig = go.Figure(data=[go.Surface(z=z_data.values)])
fig.update_layout(title='Mt Bruno Elevation', autosize=False,
width=500, height=500,
margin=dict(l=65, r=50, b=65, t=90))
fig.show()
fig.write_html("testfile.html")

Python SAC plot w/ grid

I'm required to use the information from a .sac file and plot it against a grid. I know that using various ObsPy functions one is able to plot the Seismograms using st.plot() but I can't seem to get it against a grid. I've also tried following the example given here "How do I draw a grid onto a plot in Python?" but have trouble when trying to configure my x axis to use UTCDatetime. I'm new to python and programming of this sort so any advice / help would be greatly appreciated.
Various resources used:
"http://docs.obspy.org/tutorial/code_snippets/reading_seismograms.html"
"http://docs.obspy.org/packages/autogen/obspy.core.stream.Stream.plot.html#obspy.core.stream.Stream.plot"

The Stream's plot() method actually automatically generates a grid, e.g. if you take the default example and plot it via:
from obspy.core import read
st = read() # without filename an example file is loaded
tr = st[0] # we will use only the first channel
tr.plot()
You may want to play with the number_of_ticks, tick_format and tick_rotationparameters as pointed out in http://docs.obspy.org/packages/autogen/obspy.core.stream.Stream.plot.html.
However if you want more control you can pass a matplotlib figure as input parameter to the plot() method:
from obspy.core import read
import matplotlib.pyplot as plt
fig = plt.figure()
st = read('/path/to/file.sac')
st.plot(fig=fig)
# at this point do whatever you want with your figure, e.g.
fig.gca().set_axis_off()
# finally display your figure
fig.show()
Hope it helps.

Is matplotlib savefig threadsafe?

I have an in-house distributed computing library that we use all the time for parallel computing jobs. After the processes are partitioned, they run their data loading and computation steps and then finish with a "save" step. Usually this involved writing data to database tables.
But for a specific task, I need the output of each process to be a .png file with some data plots. There are 95 processes in total, so 95 .pngs.
Inside of my "save" step (executed on each process), I have some very simple code that makes a boxplot with matplotlib's boxplot function and some code that uses savefig to write it to a .png file that has a unique name based on the specific data used in that process.
However, I occasionally see output where it appears that two or more sets of data were written into the same output file, despite the unique names.
Does matplotlib use temporary file saves when making boxplots or saving figures? If so, does it always use the same temp file names (thus leading to over-write conflicts)? I have run my process using strace and cannot see anything that obviously looks like temp file writing from matplotlib.
How can I ensure that this will be threadsafe? I definitely want to conduct the file saving in parallel, as I am looking to expand the number of output .pngs considerably, so the option of first storing all the data and then just serially executing the plot/save portion is very undesirable.
It's impossible for me to reproduce the full parallel infrastructure we are using, but below is the function that gets called to create the plot handle, and then the function that gets called to save the plot. You should assume for the sake of the question that the thread safety has nothing to do with our distributed library. We know it's not coming from our code, which has been used for years for our multiprocessing jobs without threading issues like this (especially not for something we don't directly control, like any temp files from matplotlib).
import pandas
import numpy as np
import matplotlib.pyplot as plt
def plot_category_data(betas, category_name):
"""
Function to organize beta data by date into vectors and pass to box plot
code for producing a single chart of multi-period box plots.
"""
beta_vector_list = []
yms = np.sort(betas.yearmonth.unique())
for ym in yms:
beta_vector_list.append(betas[betas.yearmonth==ym].Beta.values.flatten().tolist())
###
plot_output = plt.boxplot(beta_vector_list)
axs = plt.gcf().gca()
axs.set_xticklabels(betas.FactorDate.unique(), rotation=40, horizontalalignment='right')
axs.set_xlabel("Date")
axs.set_ylabel("Beta")
axs.set_title("%s Beta to BMI Global"%(category_name))
axs.set_ylim((-1.0, 3.0))
return plot_output
### End plot_category_data
def save(self):
"""
Make calls to store the plot to the desired output file.
"""
out_file = self.output_path + "%s.png"%(self.category_name)
fig = plt.gcf()
fig.set_figheight(6.5)
fig.set_figwidth(10)
fig.savefig(out_file, bbox_inches='tight', dpi=150)
print "Finished and stored output file %s"%(out_file)
return None
### End save

In your two functions, you're calling plt.gcf(). I would try generating a new figure every time you plot with plt.figure() and referencing that one explicitly so you skirt the whole issue entirely.