Is there a way to write existing xy axes to a FITS file along with the data itself in Python?
For example here is some simple code saving a matrix to a FITS file named TestFITS:
import numpy as np
from astropy.io import fits
test_matrix = np.random.uniform(0,1,[5,3])
x = np.arange(5,5+len(test_matrix[:,0]))
y = np.arange(5,5+len(test_matrix[0,:]))
hdu = fits.PrimaryHDU(test_matrix)
hdu.writeto('TestFITS')
But if I wished to save x and y to the file as well could that be done?
You could save them as one-dimensional ImageHDUs in two extensions, next to the PrimaryHDU:
import numpy as np
from astropy.io import fits
test_matrix = np.random.uniform(0,1,[5,3])
x = np.arange(5,5+len(test_matrix[:,0]))
y = np.arange(5,5+len(test_matrix[0,:]))
fits.HDUList([
fits.PrimaryHDU(test_matrix),
fits.ImageHDU(x, name='X'),
fits.ImageHDU(y, name='Y'),
]).writeto('testxy.fits')
(The name parameter is not necessary, but can be a nice convenience.)
I´m trying to interpolate data from a 2D array in python using scipy.interpolate.interp2d. However, I don´t think I fully understand what it is doing ,so any help is appreciated. I am trying to do something similar to what is suggested here Link.
My code is below:
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
from scipy import interpolate
from scipy.interpolate import griddata
p = np.array([[1,2,3,4,5,6,7,8,9,1],[1,2,3,4,5,6,7,8,9,1],
[1,6,3,4,8,6,7,8,9,1],[1,2,3,4,5,6,7,89,9,1],
[1,56,3,4,5,6,7,67,6,1],[1,2,3,4,7,6,7,8,9,1],
[1,2,3,5,5,6,7,6,9,1],[1,2,3,6,5,6,7,45,9,1],
[9,2,3,4,21,6,7,8,9,1],[1,8,3,3,5,6,7,5,9,1]])
print(pf)
mymin, mymax = 0, 9
X = np.linspace(mymin,mymax,10)
Y = np.linspace(mymin,mymax,10)
Xnew = np.linspace(mymin,mymax,20)
Ynew = np.linspace(mymin,mymax,20)
f = interpolate.interp2d(x,y,p,kind='cubic')
Po = f(Xnew,Ynew)
print(Po)
When I run it, the following error message is displayed:
ValueError: Invalid length for input z for non rectangular grid
My data file is shared in the following link.
We can plot this data using the following script.
import matplotlib as mpl
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.cbook as cbook
def read_datafile(file_name):
data = np.loadtxt(file_name, delimiter=',')
return data
data = read_datafile('mah_data.csv')
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.set_title("Data")
ax1.set_xlabel('t')
ax1.set_ylabel('s')
ax1.plot(x,y, c='r', label='My data')
leg = ax1.legend()
plt.show()
How can we detect peaks in python? I can't find a suitable peak detection algorithm in Python.
You can use the argrelextrema function in scipy.signal to return the indices of the local maxima or local minima of an array. This works for multi-dimensional arrays as well by specifying the axis.
from scipy.signal import argrelextrema
ind_max = argrelextrema(z, np.greater) # indices of the local maxima
ind_min = argrelextrema(z, np.less) # indices of the local minima
maxvals = z[ind_max]
minvals = z[ind_min]
More specifically, one can use the argrelmax or argrelmin to find the local maximas or local minimas. This also works for multi dimensional arrays using the axis argument.
from scipy.signal import argrelmax, argrelmin
ind_max = argrelmax(z, np.greater) # indices of the local maxima
ind_min = argrelmin(z, np.less) # indices of the local minima
maxvals = z[ind_max]
minvals = z[ind_min]
For more details, one can refer to this link: https://docs.scipy.org/doc/scipy/reference/signal.html#peak-finding
Try using peakutil (http://pythonhosted.org/PeakUtils/). Here is my solution to your question using peakutil.
import pandas as pd
import peakutils
data = pd.read_csv("mah_data.csv", header=None)
ts = data[0:10000][1] # Get the second column in the csv file
print(ts[0:10]) # Print the first 10 rows, for quick testing
# check peakutils for all the parameters.
# indices are the index of the points where peaks appear
indices = peakutils.indexes(ts, thres=0.4, min_dist=1000)
print(indices)
You should also checkout peak finding in scipy (https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.find_peaks_cwt.html)
Try the findpeaks library.
pip install findpeaks
I can not find the data attached but suppose the data is a vector and stored in data:
import pandas as pd
data = pd.read_csv("mah_data.csv", header=None).values
# Import library
from findpeaks import findpeaks
# If the resolution of your data is low, I would recommend the ``lookahead`` parameter, and if your data is "bumpy", also the ``smooth`` parameter.
fp = findpeaks(lookahead=1, interpolate=10)
# Find peaks
results = fp.fit(data)
# Make plot
fp.plot()
# Results with respect to original input data.
results['df']
# Results based on interpolated smoothed data.
results['df_interp']
I have MODIS atmospheric product. I used the code below to read the data.
%matplotlib inline
import numpy as np
from pyhdf import SD
import matplotlib.pyplot as plt
files = ['file1.hdf','file2.hdf','file3.hdf']
for n in files:
hdf=SD.SD(n)
lat = (hdf.select('Latitude'))[:]
lon = (hdf.select('Longitude'))[:]
sds=hdf.select('Deep_Blue_Aerosol_Optical_Depth_550_Land')
data=sds.get()
attributes = sds.attributes()
scale_factor = attributes['scale_factor']
data= data*scale_factor
plt.contourf(lon,lat,data)
The problem is, in some days, there are 3 data sets (as in this case, some days have four datasets) so I can not use hstack or vstack to merge these datasets.
My intention is to get the single array from three different data arrays.
I have also attached datafiles along with this link:https://drive.google.com/open?id=0B2rkXkOkG7ExYW9RNERaZU5lam8
your help will be highly appreciated.
How do I import tif using gdal?
I'm trying to get my tif file in a usable format in Python, so I can analyze the data. However, every time I import it, I just get an empty list. Here's my code:
xValues = [447520.0, 432524.0, 451503.0]
yValues = [4631976.0, 4608827.0, 4648114.0]
gdal.AllRegister()
dataset = gdal.Open('final_snow.tif', GA_ReadOnly)
if dataset is None:
print 'Could not open image'
sys.exit(1)
data = np.array([gdal.Open(name, gdalconst.GA_ReadOnly).ReadAsArray() for name, descr in dataset.GetSubDatasets()])
print 'this is data ', data`
It always prints an empty list, but it doesn't throw an error. I checked out other questions, such as [this] (Create shapefile from tif file using GDAL) What might be the problem?
For osgeo.gdal, it should look like this:
from osgeo import gdal
gdal.UseExceptions() # not required, but a good idea
dataset = gdal.Open('final_snow.tif', gdal.GA_ReadOnly)
data = dataset.ReadAsArray()
Where data is either a 2D array for 1-banded rasters, or a 3D array for multiband.
An alternative with rasterio looks like:
import rasterio
with rasterio.open('final_snow.tif', 'r') as r:
data = r.read()
Where data is always a 3D array, with the first dimension as band index.