I want to rewrite a code from matlab to python. In matlab I have the following:
interp1(TMP.time_hor, TMP.lane_hor, TMP.travel_time, 'next') % matlab
Which interpolation is meant by 'next'? Usually by default is linear. Is there a numpy equivalent? For example:
numpy.interp(x, xp, fp, left=None, right=None, period=None) # python
which is 'linear' interpolated...
Which interpolation is meant by 'next'? Usually by default is linear. Is there a numpy equivalent?
The interpolation method 'next' interpolates to the next data point in the data set (see: https://www.mathworks.com/help/matlab/ref/interp1.html).
Looking at NumPy's documentation (https://numpy.org/doc/stable/reference/generated/numpy.interp.html), it appears as though they use linear interpolation, so if you want the same output, you just need to specify this in your MATLAB command, like this:
interp1(TMP.time_hor, TMP.lane_hor, TMP.travel_time, 'linear')
That being said, 'linear' is the default interpolation method for interp1, so you can also simply leave that argument out and use the command:
interp1(TMP.time_hor, TMP.lane_hor, TMP.travel_time)
I hope this helps!
Edit: I just realized what you were asking was backwards you want to interpolate using the 'next' method in Python instead. Here's how I'd do it:
import numpy as np
import scipy as sp
# Generate data
x = np.linspace(0, 1, 10)
y = np.exp(x)
# Create interpolation function
f = sp.interpolate.interp1d(x, y, 'next')
# Create resampled range
x_resampled = np.linspace(x[0], x[-1], 100)
# Here's your interpolated data
y_interpolated = f(x_resampled)
I have a series of N images that are recorded at different times. I have stacked the images into a 3-D dask array and rechunked them along the time axis. I would now like to perform a linear fit at each pixel position across the image, but I am running into the following error when using da.map_blocks as I try to scale up: TypeError: expected 1D or 2D array for y
I found one other post, applying-a-function-along-an-axis-of-a-dask-array, related to this but it didn't address an issue with specifically setting the chunk size. When using da.apply_along_axis I found an issue similar to the one reported in dask-performance-apply-along-axis wherein only one CPU seems to be utilized during the computation (even for chunked data).
MWE: Works properly
import dask.array as da
import numpy as np
import matplotlib.pyplot as plt
plt.style.use('ggplot')
def f(y, args, axis=None):
return np.polyfit(args[0], y.squeeze(), args[1])[:, None, None]
deg = 1
nsamp=20*10*10
shape=(20,10,10)
chunk_size=(20,1,1)
a = da.linspace(1, nsamp, nsamp).reshape(shape)
chunked = a.rechunk(chunk_size)
times = da.linspace(1, shape[0], shape[0])
results = chunked.map_blocks(f, chunks=(20,1,1), args=[times, deg], dtype='float').compute()
m_fit = results[0]
b_fit = results[1]
# Plot a few fits to visually examine them
fig, ax = plt.subplots(nrows=1, ncols=1)
for (x,y) in zip([1,9], [1,9]):
ax.scatter(times, chunked[:,x,y])
ax.plot(times, np.polyval([m_fit[x, y], b_fit[x,y]], times))
The array, chunked, looks like this:
The resulting plot looks like this,
Which is exactly what I would expect and so all is well! However, the issue arises whenever I try to use a chunksize larger than one.
MWE: Raises TypeError
nsamp=20*10*10
shape=(20,10,10)
chunk_size=(20,5,5) # Chunking the data now
a = da.linspace(1,nsamp, nsamp).reshape(shape)
chunked = a.rechunk(chunk_size)
times = da.linspace(1, shape[0], shape[0])
results = chunked.map_blocks(f, chunks=(20,1,1), args=[times, 1], dtype='float') # error
Does anyone have any ideas as to what is happening here?
It looks like maybe your function expects single-dimensional inputs. I wonder if there is a way that you can write a Python function that wraps your function and handles the unpacking and then repacking of one-dimensional inputs. If you can get that function to work on a single numpy array of shape (20, 2, 2) for example then you can probably use Dask to then apply that function across many similarly sized chunks
I have some data, y vs x, which I would like to interpolate at a finer resolution xx using a cubic spline.
Here is my dataset:
import numpy as np
print np.version.version
import scipy
print scipy.version.version
1.9.2
0.15.1
x = np.array([0.5372973, 0.5382103, 0.5392305, 0.5402197, 0.5412042, 0.54221, 0.543209,
0.5442277, 0.5442277, 0.5452125, 0.546217, 0.5472153, 0.5482086,
0.5492241, 0.5502117, 0.5512249, 0.5522136, 0.5532056, 0.5532056,
0.5542281, 0.5552039, 0.5562125, 0.5567836])
y = np.array([0.01, 0.03108, 0.08981, 0.18362, 0.32167, 0.50941, 0.72415, 0.90698,
0.9071, 0.97955, 0.99802, 1., 0.97863, 0.9323, 0.85344, 0.72936,
0.56413, 0.36997, 0.36957, 0.17623, 0.05922, 0.0163, 0.01, ])
xx = np.array([0.5372981, 0.5374106, 0.5375231, 0.5376356, 0.5377481, 0.5378606,
0.5379731, 0.5380856, 0.5381981, 0.5383106, 0.5384231, 0.5385356,
0.5386481, 0.5387606, 0.5388731, 0.5389856, 0.5390981, 0.5392106,
0.5393231, 0.5394356, 0.5395481, 0.5396606, 0.5397731, 0.5398856,
0.5399981, 0.5401106, 0.5402231, 0.5403356, 0.5404481, 0.5405606,
0.5406731, 0.5407856, 0.5408981, 0.5410106, 0.5411231, 0.5412356,
0.5413481, 0.5414606, 0.5415731, 0.5416856, 0.5417981, 0.5419106,
0.5420231, 0.5421356, 0.5422481, 0.5423606, 0.5424731, 0.5425856,
0.5426981, 0.5428106, 0.5429231, 0.5430356, 0.5431481, 0.5432606,
0.5433731, 0.5434856, 0.5435981, 0.5437106, 0.5438231, 0.5439356,
0.5440481, 0.5441606, 0.5442731, 0.5443856, 0.5444981, 0.5446106,
0.5447231, 0.5448356, 0.5449481, 0.5450606, 0.5451731, 0.5452856,
0.5453981, 0.5455106, 0.5456231, 0.5457356, 0.5458481, 0.5459606,
0.5460731, 0.5461856, 0.5462981, 0.5464106, 0.5465231, 0.5466356,
0.5467481, 0.5468606, 0.5469731, 0.5470856, 0.5471981, 0.5473106,
0.5474231, 0.5475356, 0.5476481, 0.5477606, 0.5478731, 0.5479856,
0.5480981, 0.5482106, 0.5483231, 0.5484356, 0.5485481, 0.5486606,
0.5487731, 0.5488856, 0.5489981, 0.5491106, 0.5492231, 0.5493356,
0.5494481, 0.5495606, 0.5496731, 0.5497856, 0.5498981, 0.5500106,
0.5501231, 0.5502356, 0.5503481, 0.5504606, 0.5505731, 0.5506856,
0.5507981, 0.5509106, 0.5510231, 0.5511356, 0.5512481, 0.5513606,
0.5514731, 0.5515856, 0.5516981, 0.5518106, 0.5519231, 0.5520356,
0.5521481, 0.5522606, 0.5523731, 0.5524856, 0.5525981, 0.5527106,
0.5528231, 0.5529356, 0.5530481, 0.5531606, 0.5532731, 0.5533856,
0.5534981, 0.5536106, 0.5537231, 0.5538356, 0.5539481, 0.5540606,
0.5541731, 0.5542856, 0.5543981, 0.5545106, 0.5546231, 0.5547356,
0.5548481, 0.5549606, 0.5550731, 0.5551856, 0.5552981, 0.5554106,
0.5555231, 0.5556356, 0.5557481, 0.5558606, 0.5559731, 0.5560856,
0.5561981, 0.5563106, 0.5564231, 0.5565356, 0.5566481, 0.5567606])
I am trying to fit using the scipy InterpolatedUnivariateSpline method, interpolated with a 3rd order spline k=3, and extrapolated as zeros ext='zeros':
import scipy.interpolate as interp
yspline = interp.InterpolatedUnivariateSpline(x,y, k=3, ext='zeros')
yvals = yspline(xx)
import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(x, y, 'ko', label='Values')
ax.plot(xx, yvals, 'b-.', lw=2, label='Spline')
plt.xlim([min(x), max(x)])
However, as you can see in this image, my Spline returns NaN values :(
Is there a reason? I am pretty sure my x values are all increasing, so I am stumped as to why this is happening. I have many other datasets I am fitting using this method, and it only fails on this specific set of data.
Any help is greatly appreciated.
Thank you for reading.
EDIT!
The solution was that I have duplicate x values, with differing y values!
For this interpolation, you should rather use scipy.interpolate.interp1d with the argument kind='cubic' (see a related SO question )
I have yet to find a use case where InterpolatedUnivariateSpline can be used in practice (or maybe I just don't understand its purpose). With your code I get,
So the interpolation works but shows extremely strong oscillations, making it unusable, which is typically the result I was getting with this interpolation method in the past. With a lower order spline (e.g. k=1) that works better, but then you lose the advantage of cubic interpolation.
I've also encountered the problem with InterpolatedUnivariateSpline returning NaN values. But in my case the reason was not in having duplicates in x array but because values in x were decreasing when docs states that values "must be increasing".
So, in such a case, instead of original x and y one must supply them reversed: x[::-1] and y[::-1].
I'm trying to make a 3D plot that consists of a series of 2D planes through an RGB stack, like this:
I know that it's possible to do this using mpl_toolkits.mplot3d by passing the x, y, z coordinates and the RGB(A) colours of each pixel to plot_surface:
import numpy as np
from matplotlib import pyplot as pp
from mpl_toolkits.mplot3d.axes3d import Axes3D
def plot_stack_slices(rgbstack, scale=(1., 1., 1.), z_interval=10.):
fig, ax = pp.subplots(1,1,subplot_kw={'projection':'3d'})
ax.invert_zaxis()
ax.hold(True)
sx, sy, sz = scale
nz, ny, nx, nc = rgbstack.shape
stack_xyz = np.mgrid[:nx*sx:nx*1j, :ny*sy:ny*1j, :nz*sz:nz*1j]
slices = rgbstack[::-z_interval]
slice_xyz = np.rollaxis(stack_xyz, 3, 0)[::-z_interval]
surflist = []
for (img,xyz) in zip(slices, slice_xyz):
x, y, z = xyz
s = ax.plot_surface(x, y, z, facecolors=img**0.75,
rstride=50, cstride=50)
surflist.append(s)
return fig, ax, surflist
Unfortunately this becomes extremely slow if I set rstride=1, cstride=1 in order to display the textures at full resolution.
I'm also aware that Mayavi can easily handle displaying multiple 2D textures at full resolution:
from mayavi import mlab
def plot_stack_slices2(stack, scale=(1., 1., 20.), z_interval=10.):
mfig = mlab.figure(bgcolor=(1,)*3)
sx, sy, sz = scale
nz, ny, nx = stack.shape
slices = stack[::-z_interval]
slice_z = np.linspace(0,nz*sz,nz)[::z_interval]
surflist = []
for (img,z) in zip(slices, slice_z):
im = mlab.imshow(img.T, colormap='gray', figure=mfig)
im.actor.scale = [sx,sy,sz]
im.actor.position = [0, 0, z]
surflist.append(z)
return fig, surflist
However, the problem now is that there does not seem to be any way of displaying true-colour RGB textures using Mayavi - according to the docs I can only specify either a single (R, G, B) tuple, or a pre-defined colourmap.
Does anyone know of a better way to display true-colour 2D RGB textures in a 3D plot?
Given enough time I could probably figure out how do do this in Vtk or even pure OpenGL if necessary, but I'm really hoping that there are existing libraries that will do the job.
Big thanks to aestrivex for providing working solutions using Mayavi/VTK - it's useful info that I may need for doing more complicated things in the future.
In the end I actually chose to go with cgohlke's suggestion of using visvis, which turned out to be a lot simpler to implement:
import visvis as vv
vv.use('wx')
import numpy as np
from matplotlib.image import imread
from matplotlib.cbook import get_sample_data
imgdata = imread(get_sample_data('lena.png'))
nr, nc = imgdata.shape[:2]
x,y = np.mgrid[:nr, :nc]
z = np.ones((nr, nc))
for ii in xrange(5):
vv.functions.surf(x, y, z*ii*100, imgdata, aa=3)
I don't know about other libraries -- volshow looks neat but I havent tested it -- but you can do this in vtk.
I have been working on doing this generally in mayavi (see How to directly set RGB/RGBA colors in mayavi) but for certain image sources mayavi structures the vtk pipeline in a way that was not designed to deal with this at all. My efforts to convert a 2D vtk.ImageData to true color starting with mlab.imshow were met with resistance at every step, but I managed it.
First, here is how I have managed to do it in mayavi using mlab. This is far too hacky and "magic"-reliant even for my standards:
from mayavi import mlab
import numpy as np
from tvtk.api import tvtk
k=mlab.imshow(np.random.random((10,10)),colormap='bone')
colors=tvtk.UnsignedCharArray()
colors.from_array(np.random.randint(256,size=(100,3)))
k.mlab_source.dataset.point_data.scalars=colors
k.actor.input.point_data.scalars=colors
#the latter set of scalars is what is actually used in the VTK pipeline in this
#case, but if they don't play nice with the mayavi source then tvtk will
#complain because we are circumventing the structure it expects
k.actor.input.scalar_type='unsigned_char'
k.actor.input.number_of_scalar_components=3
k.image_map_to_color.lookup_table=None
k.actor.input.modified()
mlab.draw()
#this draw fails. As it fails, there is an interaction here, somewhere deep in
#tvtk, causing the ImageData to partially reset.. I have not been able to track
#it down yet. ignore the error output
k.actor.input.scalar_type='unsigned_char'
k.actor.input.number_of_scalar_components=3
#now after we reset these back to what they should be, it works
mlab.draw()
mlab.show()
But in pure tvtk it's not nearly so bad:
import numpy as np
from tvtk.api import tvtk
colors=np.random.randint(256,size=(100,3))
an_image=tvtk.ImageData()
an_image.number_of_scalar_components=3
an_image.scalar_type='unsigned_char'
an_image.point_data.scalars=tvtk.UnsignedCharArray()
an_image.point_data.scalars.from_array(colors)
an_image.dimensions=np.array((10,10,1))
an_actor=tvtk.ImageActor()
an_actor.input=an_image
an_actor.interpolate=False
ren=tvtk.Renderer()
renWin=tvtk.RenderWindow()
renWin.add_renderer(ren)
ren.add_actor2d(an_actor)
iren=tvtk.RenderWindowInteractor()
iren.render_window=renWin
iren.interactor_style=tvtk.InteractorStyleTrackballCamera()
renWin.render()
iren.start()
Of course, doing it in vtk is more work. You might even be able to wrap this nicely so that it's pretty reasonable.
I want to fix mayavi to handle this properly, but as you can see from my snippet it is not straightforward and could take a while.
I am working in image processing right now in python using numpy and scipy all the time. I have one piece of code that can enlarge an image, but not sure how this works.
So please some expert in scipy/numpy in python can explain to me line by line. I am always eager to learn.
import numpy as N
import os.path
import scipy.signal
import scipy.interpolate
import matplotlib.pyplot as plt
import matplotlib.cm as cm
def enlarge(img, rowscale, colscale, method='linear'):
x, y = N.meshgrid(N.arange(img.shape[1]), N.arange(img.shape[0]))
pts = N.column_stack((x.ravel(), y.ravel()))
xx, yy = N.mgrid[0.:float(img.shape[1]):1/float(colscale),
0.:float(img.shape[0]):1/float(rowscale)]
large = scipy.interpolate.griddata(pts, img.flatten(), (xx, yy), method).T
large[-1,:] = large[-2,:]
large[:,-1] = large[:,-2]
return large
Thanks a lot.
First, a grid of empty points is created with point per pixel.
x, y = N.meshgrid(N.arange(img.shape[1]), N.arange(img.shape[0]))
The actual image pixels are placed into the variable pts which will be needed later.
pts = N.column_stack((x.ravel(), y.ravel()))
After that, it creates a mesh grid with one point per pixel for the enlarged image; if the original image was 200x400, the colscale set to 4 and rowscale set to 2, the mesh grid would have (200*4)x(400*2) or 800x800 points.
xx, yy = N.mgrid[0.:float(img.shape[1]):1/float(colscale),
0.:float(img.shape[0]):1/float(rowscale)]
Using scipy, the points in pts variable are interpolated into the larger grid. Interpolation is the manner in which missing points are filled or estimated usually when going from a smaller set of points to a larger set of points.
large = scipy.interpolate.griddata(pts, img.flatten(), (xx, yy), method).T
I am not 100% certain what the last two lines do without going back and looking at what the griddata method returns. It appears to be throwing out some additional data that isn't needed for the image or performing a translation.
large[-1,:] = large[-2,:]
large[:,-1] = large[:,-2]
return large