I want to apply Hough Transform on stock prices (array of numbers).
I read OpenCV and scikit-image docs and examples ,but got nothing how to apply the transformation to the arrays of numbers instead of images.
I created 2D array from data. First dimension is X(simply index of data) and second dimension is close prices.
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import pywt as wt
from skimage.transform import (hough_line, hough_line_peaks,probabilistic_hough_line)
from matplotlib import cm
path = "22-31May-100Tick.csv"
df = pd.read_csv(path)
y = df.Close.values
x = np.arange(0,len(y),1)
data = []
for i in x:
a = [i,y[i]]
data.append(a)
data = np.array(data)
How is it possible to apply the transformation with OpenCV or sickit-image?
Thank you
I have MODIS atmospheric product. I used the code below to read the data.
%matplotlib inline
import numpy as np
from pyhdf import SD
import matplotlib.pyplot as plt
files = ['file1.hdf','file2.hdf','file3.hdf']
for n in files:
hdf=SD.SD(n)
lat = (hdf.select('Latitude'))[:]
lon = (hdf.select('Longitude'))[:]
sds=hdf.select('Deep_Blue_Aerosol_Optical_Depth_550_Land')
data=sds.get()
attributes = sds.attributes()
scale_factor = attributes['scale_factor']
data= data*scale_factor
plt.contourf(lon,lat,data)
The problem is, in some days, there are 3 data sets (as in this case, some days have four datasets) so I can not use hstack or vstack to merge these datasets.
My intention is to get the single array from three different data arrays.
I have also attached datafiles along with this link:https://drive.google.com/open?id=0B2rkXkOkG7ExYW9RNERaZU5lam8
your help will be highly appreciated.
I have written this code which will generate a number of contour plots, each of which corresponds to a single text file. I have multiple text files. Currently, I am able to generate all of the images separately in png format without any issues.
When I try to save the images as a pdf file, it is saving only the last image generated in a loop.I tried using the PdfPages package. This question is similar to the one that I posted before but with a different question. Similar
Issue: I want to able to generate all of the images into a single pdf file automatically from python. So for eg. if I have 100 text files, then I want to save all of the 100 images onto a single pdf file.Also ideally I want to save 2 images in a single page in the pdf file. There are some questions in SO about this, but I couldn't find an appropriate solution for my issue. Since I have many case for which I have to generate the images, I want to save them as a single pdf file as it is more easier to analyze them. I would appreciate any suggestions/advice to help me with this.
This is link for the sample text file Sample Text
ges
from __future__ import print_function
import numpy as np
from matplotlib import pyplot as plt
from scipy.interpolate import griddata
from matplotlib.backends.backend_pdf import PdfPages
path = 'location of the text files'
FT_init = 5.4311
delt = 0.15
TS_init = 140
dj_length = 2.4384
def streamfunction2d(y,x,Si_f,q):
with PdfPages('location of the generated pdf') as pdf:
Stf= plt.contour(x,y,Si_f,20)
Stf1 = plt.colorbar(Stf)
plt.clabel(Stf,fmt='%.0f',inline=True)
plt.figtext(0.37,0.02,'Flowtime(s)',style= 'normal',alpha=1.0)
plt.figtext(0.5,0.02,str(q[p]),style= 'normal',alpha=1.0)
plt.title('Streamfunction_test1')
plt.hold(True)
plt.tight_layout()
pdf.savefig()
path1 = 'location where the image is saved'
image = path1+'test_'+'Stream1_'+str((timestep[p]))+'.png'
plt.savefig(image)
plt.close()
timestep = np.linspace(500,600,2)
flowtime = np.zeros(len(timestep))
timestep = np.array(np.round(timestep),dtype = 'int')
###############################################################################
for p in range(len(timestep)):
if timestep[p]<TS_init:
flowtime[p] = 1.1111e-01
else:
flowtime[p] = (timestep[p]-TS_init)*delt+FT_init
q = np.array(flowtime)
timestepstring=str(timestep[p]).zfill(4)
fname = path+"ddn150AE-"+timestepstring+".txt"
f = open(fname,'r')
data = np.loadtxt(f,skiprows=1)
data = data[data[:, 1].argsort()]
data = data[np.logical_not(data[:,11]== 0)]
Y = data[:,2] # Assigning Y to column 2 from the text file
limit = np.nonzero(Y==dj_length)[0][0]
Y = Y[limit:]
Vf = data[:,11]
Vf = Vf[limit:]
Tr = data[:,9]
Tr = Tr[limit:]
X = data[:,1]
X = X[limit:]
Y = data[:,2]
Y = Y[limit:]
U = data[:,3]
U = U[limit:]
V = data[:,4]
V = V[limit:]
St = data[:,5]
St = St[limit:]
###########################################################################
## Using griddata for interpolation from Unstructured to Structured data
# resample onto a 300x300 grid
nx, ny = 300,300
# (N, 2) arrays of input x,y coords and dependent values
pts = np.vstack((X,Y )).T
vals = np.vstack((Tr))
vals1 = np.vstack((St))
# The new x and y coordinates for the grid
x = np.linspace(X.min(), X.max(), nx)
y = np.linspace(Y.min(), Y.max(), ny)
r = np.meshgrid(y,x)[::-1]
# An (nx * ny, 2) array of x,y coordinates to interpolate at
ipts = np.vstack(a.ravel() for a in r).T
Si = griddata(pts, vals1, ipts, method='linear')
print(Ti.shape,"Ti_Shape")
Si_f = np.reshape(Si,(len(y),len(x)))
print(Si_f.shape,"Streamfunction Shape")
Si_f = np.transpose(Si_f)
streamfunction2d(y,x,Si_f,q)
Edit : As you mentioned matplotlib is probably able to handle everything by itself using PdfPages function. See this related answer. My original answer is a hack.
I think the error in your code is that you are creating another PdfPage object each time you go through the loop. My advice would be to add the PdfPage object as an argument to your streamfunction2d function and create the PdfPage object once and for all before the loop (using a with statement as in the documentation seems a good idea).
Example:
def streamfunction2d(y,x,Si_f,q,pdf):
# (...)
pdf.savefig(plt.gcf())
with PdfPages('output.pdf') as pdf:
for p in range(len(timestep)):
# (...)
streamfunction2d(y,x,Si_f,q,pdf)
Original answer:
Here is a quick and dirty solution using the pdfunite software.
from matplotlib import pyplot as plt
import numpy as np
import subprocess
import os
X = np.linspace(0,1,100)
for i in range(10):
# random plot
plt.plot(X,np.cos(i*X))
# Save each figure as a pdf file.
plt.savefig("page_{:0}.pdf".format(i))
plt.clf()
# Calling pdfunite to merge all the pages
subprocess.call("pdfunite page_*.pdf united.pdf",shell=True)
# Removing temporary files
for i in range(10):
os.remove("page_{:0}.pdf".format(i))
It uses two things:
You can save your figures as pdf using matplotlib's savefig command.
You can call other programs using the subprocess library. I used pdfunite to merge all the pages. Be sure it is available on your machine !
If you want to have several graph by page, you can use subplots.
Alternatively, you could use another python library (such as pyPDF) to merge the pages, but it would require slightly more code. Here is an (untested) example:
from matplotlib import pyplot as plt
import numpy as np
from pyPdf import PdfFileWriter, PdfFileReader
# create an empty pdf file
output = PdfFileWriter()
X = np.linspace(0,1,100)
for i in range(10):
# random plot
plt.plot(X,np.cos(i*X))
# Save each figure as a pdf file.
fi = "page_{:0}.pdf".format(i)
plt.savefig(fi)
plt.clf()
# add it to the end of the output
input = PdfFileReader(file(fi, "rb"))
output.addPage(input.getPage(0))
# Save the resulting pdf file.
outputStream = file("document-output.pdf", "wb")
output.write(outputStream)
I'm attempting to read the data from a FITS file using the astropy module fits and then standard numpy array handling. However, for some reason I am receiving the following error:
IndexError: too many indices
This is the code that I am using:
from astropy.io import fits
import matplotlib.pyplot as plt
hdulist = fits.open('/Users/iMacHome/Downloads/spec-1959-53440-0605.fits')
hdu = hdulist[1]
data = hdu.data
flux = data[:, 1]
^ Error Traceback to the flux = data[:, 1] line.
loglam = data[:, 2]
This may be a question that perhaps astronomers could answer (or, specifically, astronomers familiar with .fits files from the SDSS), but I welcome the input from numpy and python users!
I have just had the following answer from the SDSS help desk:
Replace:
flux = data[:,0]
loglam = data[:,1]
with
flux = data['flux']
loglam = data['loglam']
This is the correct way to access fields in a Numpy record array.
I am using Mayavi2 in a Python script to calculate 3d iso-surfaces. As a result I get a vtkPoints object. Now I want to convert this vtkPoints object ('vtkout' in the code sample below) to a simple numpy array with 3 lines containing all x, y and z values.
I get vtkout using a code like this:
import numpy
from enthought.mayavi import mlab
import array
randVol = numpy.random.rand(50,50,50) # fill volume with some random potential
X, Y, Z = numpy.mgrid[0:50, 0:50, 0:50] # grid
surf = mlab.contour3d(X, Y, Z, randVol, contours=[0.5]) # calc contour
vtkout = surf.contour.contour_filter.output.points # get the vtkPoints object
At the moment I use the following code to extract the points into an array:
pointsArray = numpy.zeros((3, vtkout.number_of_points))
for n in range(vtkout.number_of_points):
pointsArray[0,n] = vtkout[n][0]
pointsArray[1,n] = vtkout[n][1]
pointsArray[2,n] = vtkout[n][2]
I wonder if there is no general routine doing such conversions for me in a convenient, fast and safe way?
vtk_points.to_array() did not work for me (to_array() doesn't seem to exist in plain vtk).
What has actually worked in my case is using the numpy_support module:
from vtk.util import numpy_support
as_numpy = numpy_support.vtk_to_numpy(vtk_points.GetData())
As confirmed from comments on the original post, you might try:
vtkout.to_array().T
This is a direct method that does not require looping.