We Keep Coding

Preserving the WCS information of a FITS file when rebinned

Aim : Rebin an existing image (FITS file) and write the new entries into a new rebinned image (also a FITS file).
Issue : Rebinned FITS file and the original FITS file seem to have mismatched co-ordinates (figure shown later in the question).
Process : I will briefly describe my process to shed more light. The first step is to read the existing fits file and define numpy arrays
from math import *
import numpy as np
import matplotlib.pyplot as plt
from astropy.visualization import astropy_mpl_style
from astropy.io import fits
import matplotlib.pyplot as plt
%matplotlib notebook
import aplpy
from aplpy import FITSFigure
file = 'F0621_HA_POL_0700471_HAWDHWPD_PMP_070-199Augy20.fits'
hawc = fits.open(file)
stokes_i = np.array(hawc[0].data)
stokes_i_rebinned = congrid(stokes_i,newdim,method="neighbour", centre=False, minusone=False)
Here "congrid" is a function I have used for near-neigbhour rebinning that rebins the original array to a new dimension given by "newdim". Now the goal is to write this rebinned array back into the FITS file format as a new file. I have several more such arrays but for brevity, I just include one array as an example. To keep the header information same, I read the header information of that array from the existing FITS file and use that to write the new array into a new FITS file. After writing, the rebinned file can be read just like the original :-
header_0= hawc[0].header
fits.writeto("CasA_HAWC+_rebinned_congrid.fits", rebinned_stokes_i, header_0, overwrite=True)
rebinned_file = 'CasA_HAWC+_rebinned_congrid.fits'
hawc_rebinned= fits.open(rebinned_file)
To check how the rebinned image looks now I plot them
cmap = 'rainbow'
stokes_i = hawc[0]
stokes_i_rebinned = hawc_rebinned[0]
axi = FITSFigure(stokes_i, subplot=(1,2,1)) # generate FITSFigure as subplot to have two axes together
axi.show_colorscale(cmap=cmap) # show I
axi_rebinned = FITSFigure(stokes_i_rebinned, subplot=(1,2,2),figure=plt.gcf())
axi_rebinned.show_colorscale(cmap=cmap) # show I rebinned
# FORMATTING
axi.set_title('Stokes I (146 x 146)')
axi_rebinned.set_title('Rebinned Stokes I (50 x 50)')
axi_rebinned.axis_labels.set_yposition('right')
axi_rebinned.tick_labels.set_yposition('right')
axi.tick_labels.set_font(size='small')
axi.axis_labels.set_font(size='small')
axi_rebinned.tick_labels.set_font(size='small')
axi_rebinned.axis_labels.set_font(size='small')
As you see for the original and rebinned image, the X,Y co-ordinates seem mismatched and my best guess was that WCS (world co-ordinate system) for the original FITS file wasn't properly copied for the new FITS file, thus causing any mismatch. So how do I align these co-ordinates ?
Any help will be deeply appreciated ! Thanks

I'm posting my answer in an astropy slack channel here should this be useful for others.
congrid will not work because it doesn't include information about the WCS. For example, your CD matrix is tied to the original image, not the re-binned set. There are a number of way to re-bin data with proper WCS. You might consider reproject although this often requires another WCS header to re-bin to.
Montage (though not a Python tool but has Python wrappers) is potentially another way.

As #astrochun already said, your re-binning function does not adjust the WCS of the re-binned image. In addition to reproject and Montage, astropy.wcs.WCSobject has slice() method. You could try using it to "re-bin" the WCS like this:
from astropy.wcs import WCS
import numpy as np
wcs = WCS(hawc[0].header, hawc)
wcs_rebinned = wcs.slice((np.s_[::2], np.s_[::2]))
wcs_hdr = wcs_rebinned.to_header()
header_0.update(wcs_hdr) # but watch out for CD->PC conversion
You should also make a "real" copy of hawc[0].header in header_0= hawc[0].header, for example as header_0= hawc[0].header.copy() or else header_0.update(wcs_hdr) will modify hawc[0].header as well.

Python convert KML to image file

Looking for some guidance on how to convert a KML file to an image file showing simple polygons of the GPS data held in the file? I've been looking at ways to do this via python using mapnik and simplekml but I'm unsure if this is the correct usage of the tools.
Ideally, I just want a simple way to produce polygons from a KML file
Any advice very welcome

Manage to get a crud script working using geopandas
import geopandas as gpd
import matplotlib.pyplot as plt
gpd.io.file.fiona.drvsupport.supported_drivers['KML'] = 'rw'
# Filepath to KML file
fp = "history.kml"
polys = gpd.read_file(fp, driver='KML')
print(polys)
polys.plot()
plt.savefig('test.jpg')

Why geotiff could not be opened by gdal?

I'm newbie in python and in geoprocessing. I'm writing some program to calculate ndwi. To make this, I try to open geotiff dataset with gdal, but dataset can't be opened. I tried to open different tiff files (Landsat8 multiple data, Landsat7 composite, etc), but dataset is always None.
What reason to this could be? Or how can i find it out?
Here's a part of code:
import sys, os, struct
import gdal, gdalconst
from gdalconst import *
import numpy as np
from numpy import *
class GDALCalcNDWI ():
def calcNDWI(self, outFilePath):
gdal.AllRegister()
# this allows GDAL to throw Python Exceptions
gdal.UseExceptions()
filePath = "C:\\Users\\Daria\\Desktop.TIF\\170028-2007-05-21.tif"
# Open
dataset = gdal.Open(filePath, gdal.GA_ReadOnly)
# Check
if dataset is None:
print ("can't open tiff file")
sys.exit(-1)
Thanks

Whenever you have a well-known file reader that is returning None, make sure the path to your file is correct. I doubt you have a directory called Desktop.TIF, I'm assuming you just made a typo in your source code. You probably want C:\\Users\\Dara\\Desktop\\TIF\\170028-2007-05-21.tif as the path (note that Desktop.TIF ==> Desktop\\TIF).
The safest thing to do is right click on the file, go to properties, and copy/paste that path into your python source code.

Convert SVG/PDF to EMF

I am looking for a way to save a matplotlib figure as an EMF file. Matplotlib allows me to save as either a PDF or SVG vector file but not as EMF.
After a long search I still cannot seem to find a way to do this with python. Hopefully anyone has an idea.
My workaround is to call inkscape using subprocess but this is far from ideal as I would like to avoid the use of external programs.
I'm running python 2.7.5 and matplotlib 1.3.0 using the wx backend.

For anyone who still needs this, I wrote a basic function that can let you save a file as an emf from matplotlib, as long as you have inkscape installed.
I know the op didn't want inkscape, but people who find this post later just want to make it work.
import matplotlib.pyplot as plt
import subprocess
import os
inkscapePath = r"path\to\inkscape.exe"
savePath= r"path\to\images\folder"
def exportEmf(savePath, plotName, fig=None, keepSVG=False):
"""Save a figure as an emf file
Parameters
----------
savePath : str, the path to the directory you want the image saved in
plotName : str, the name of the image
fig : matplotlib figure, (optional, default uses gca)
keepSVG : bool, whether to keep the interim svg file
"""
figFolder = savePath + r"\{}.{}"
svgFile = figFolder.format(plotName,"svg")
emfFile = figFolder.format(plotName,"emf")
if fig:
use=fig
else:
use=plt
use.savefig(svgFile)
subprocess.run([inkscapePath, svgFile, '-M', emfFile])
if not keepSVG:
os.system('del "{}"'.format(svgFile))
#Example Usage
import numpy as np
tt = np.linspace(0, 2*3.14159)
plt.plot(tt, np.sin(tt))
exportEmf(r"C:\Users\userName", 'FileName')

I think the function is cool but inkscape syntax seems not working in my case. I search in other post and find it as:
inkscape filename.svg --export-filename filename.emf
So if I replace -M by --export-filename within the subprocess argument, everything works fine.

Save an "interactive figure" with matplotlib [duplicate]

Is there a way to save a Matplotlib figure such that it can be re-opened and have typical interaction restored? (Like the .fig format in MATLAB?)
I find myself running the same scripts many times to generate these interactive figures. Or I'm sending my colleagues multiple static PNG files to show different aspects of a plot. I'd rather send the figure object and have them interact with it themselves.

I just found out how to do this. The "experimental pickle support" mentioned by #pelson works quite well.
Try this:
# Plot something
import matplotlib.pyplot as plt
fig,ax = plt.subplots()
ax.plot([1,2,3],[10,-10,30])
After your interactive tweaking, save the figure object as a binary file:
import pickle
pickle.dump(fig, open('FigureObject.fig.pickle', 'wb')) # This is for Python 3 - py2 may need `file` instead of `open`
Later, open the figure and the tweaks should be saved and GUI interactivity should be present:
import pickle
figx = pickle.load(open('FigureObject.fig.pickle', 'rb'))
figx.show() # Show the figure, edit it, etc.!
You can even extract the data from the plots:
data = figx.axes[0].lines[0].get_data()
(It works for lines, pcolor & imshow - pcolormesh works with some tricks to reconstruct the flattened data.)
I got the excellent tip from Saving Matplotlib Figures Using Pickle.

As of Matplotlib 1.2, we now have experimental pickle support. Give that a go and see if it works well for your case. If you have any issues, please let us know on the Matplotlib mailing list or by opening an issue on github.com/matplotlib/matplotlib.

This would be a great feature, but AFAIK it isn't implemented in Matplotlib and likely would be difficult to implement yourself due to the way figures are stored.
I'd suggest either (a) separate processing the data from generating the figure (which saves data with a unique name) and write a figure generating script (loading a specified file of the saved data) and editing as you see fit or (b) save as PDF/SVG/PostScript format and edit in some fancy figure editor like Adobe Illustrator (or Inkscape).
EDIT post Fall 2012: As others pointed out below (though mentioning here as this is the accepted answer), Matplotlib since version 1.2 allowed you to pickle figures. As the release notes state, it is an experimental feature and does not support saving a figure in one matplotlib version and opening in another. It's also generally unsecure to restore a pickle from an untrusted source.
For sharing/later editing plots (that require significant data processing first and may need to be tweaked months later say during peer review for a scientific publication), I still recommend the workflow of (1) have a data processing script that before generating a plot saves the processed data (that goes into your plot) into a file, and (2) have a separate plot generation script (that you adjust as necessary) to recreate the plot. This way for each plot you can quickly run a script and re-generate it (and quickly copy over your plot settings with new data). That said, pickling a figure could be convenient for short term/interactive/exploratory data analysis.

Why not just send the Python script? MATLAB's .fig files require the recipient to have MATLAB to display them, so that's about equivalent to sending a Python script that requires Matplotlib to display.
Alternatively (disclaimer: I haven't tried this yet), you could try pickling the figure:
import pickle
output = open('interactive figure.pickle', 'wb')
pickle.dump(gcf(), output)
output.close()

Good question. Here is the doc text from pylab.save:
pylab no longer provides a save function, though the old pylab
function is still available as matplotlib.mlab.save (you can still
refer to it in pylab as "mlab.save"). However, for plain text
files, we recommend numpy.savetxt. For saving numpy arrays,
we recommend numpy.save, and its analog numpy.load, which are
available in pylab as np.save and np.load.

I figured out a relatively simple way (yet slightly unconventional) to save my matplotlib figures. It works like this:
import libscript
import matplotlib.pyplot as plt
import numpy as np
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2*np.pi*t)
#<plot>
plt.plot(t, s)
plt.xlabel('time (s)')
plt.ylabel('voltage (mV)')
plt.title('About as simple as it gets, folks')
plt.grid(True)
plt.show()
#</plot>
save_plot(fileName='plot_01.py',obj=sys.argv[0],sel='plot',ctx=libscript.get_ctx(ctx_global=globals(),ctx_local=locals()))
with function save_plot defined like this (simple version to understand the logic):
def save_plot(fileName='',obj=None,sel='',ctx={}):
"""
Save of matplolib plot to a stand alone python script containing all the data and configuration instructions to regenerate the interactive matplotlib figure.
Parameters
----------
fileName : [string] Path of the python script file to be created.
obj : [object] Function or python object containing the lines of code to create and configure the plot to be saved.
sel : [string] Name of the tag enclosing the lines of code to create and configure the plot to be saved.
ctx : [dict] Dictionary containing the execution context. Values for variables not defined in the lines of code for the plot will be fetched from the context.
Returns
-------
Return ``'done'`` once the plot has been saved to a python script file. This file contains all the input data and configuration to re-create the original interactive matplotlib figure.
"""
import os
import libscript
N_indent=4
src=libscript.get_src(obj=obj,sel=sel)
src=libscript.prepend_ctx(src=src,ctx=ctx,debug=False)
src='\n'.join([' '*N_indent+line for line in src.split('\n')])
if(os.path.isfile(fileName)): os.remove(fileName)
with open(fileName,'w') as f:
f.write('import sys\n')
f.write('sys.dont_write_bytecode=True\n')
f.write('def main():\n')
f.write(src+'\n')
f.write('if(__name__=="__main__"):\n')
f.write(' '*N_indent+'main()\n')
return 'done'
or defining function save_plot like this (better version using zip compression to produce lighter figure files):
def save_plot(fileName='',obj=None,sel='',ctx={}):
import os
import json
import zlib
import base64
import libscript
N_indent=4
level=9#0 to 9, default: 6
src=libscript.get_src(obj=obj,sel=sel)
obj=libscript.load_obj(src=src,ctx=ctx,debug=False)
bin=base64.b64encode(zlib.compress(json.dumps(obj),level))
if(os.path.isfile(fileName)): os.remove(fileName)
with open(fileName,'w') as f:
f.write('import sys\n')
f.write('sys.dont_write_bytecode=True\n')
f.write('def main():\n')
f.write(' '*N_indent+'import base64\n')
f.write(' '*N_indent+'import zlib\n')
f.write(' '*N_indent+'import json\n')
f.write(' '*N_indent+'import libscript\n')
f.write(' '*N_indent+'bin="'+str(bin)+'"\n')
f.write(' '*N_indent+'obj=json.loads(zlib.decompress(base64.b64decode(bin)))\n')
f.write(' '*N_indent+'libscript.exec_obj(obj=obj,tempfile=False)\n')
f.write('if(__name__=="__main__"):\n')
f.write(' '*N_indent+'main()\n')
return 'done'
This makes use a module libscript of my own, which mostly relies on modules inspect and ast. I can try to share it on Github if interest is expressed (it would first require some cleanup and me to get started with Github).
The idea behind this save_plot function and libscript module is to fetch the python instructions that create the figure (using module inspect), analyze them (using module ast) to extract all variables, functions and modules import it relies on, extract these from the execution context and serialize them as python instructions (code for variables will be like t=[0.0,2.0,0.01] ... and code for modules will be like import matplotlib.pyplot as plt ...) prepended to the figure instructions. The resulting python instructions are saved as a python script whose execution will re-build the original matplotlib figure.
As you can imagine, this works well for most (if not all) matplotlib figures.

If you are looking to save python plots as an interactive figure to modify and share with others like MATLAB .fig file then you can try to use the following code. Here z_data.values is just a numpy ndarray and so you can use the same code to plot and save your own data. No need of using pandas then.
The file generated here can be opened and interactively modified by anyone with or without python just by clicking on it and opening in browsers like Chrome/Firefox/Edge etc.
import plotly.graph_objects as go
import pandas as pd
z_data=pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/api_docs/mt_bruno_elevation.csv')
fig = go.Figure(data=[go.Surface(z=z_data.values)])
fig.update_layout(title='Mt Bruno Elevation', autosize=False,
width=500, height=500,
margin=dict(l=65, r=50, b=65, t=90))
fig.show()
fig.write_html("testfile.html")