Interpolation of a huge 2D array in python - python

I've just plotted the following colormap from a 35x800 numpy array:
As you can see, the map appears crenelated: this is because cells contain probability = 0 (artefacts produced by model simulation method). I need to interpolate the data for (i) make a smooth and elegant colormap and (ii) obtain the full matrix for follow-up computations. However, I don't know how to proceed and the interpolation method I should use. Any idea?

Related

Python - How to resample a 2D shape?

I am writing a python script for some geometrical data manipulation (calculating motion trajectories for a multi-drive industrial machine). Generally, the idea is that there is a given shape (let's say - an ellipse, but it general case it can be any convex shape, defined with a series of 2D points), which is rotated and it's uppermost tangent point must be followed. I don't have a problem with the latter part but I need a little hint with the 2D shape preparation.
Let's say that the ellipse was defined with too little points, for example - 25. (As I said, ultimately this can be any shape, for example a rounded hexagon). To maintain necessary precision I need far more points (let's say - 1000), preferably equally distributed over whole shape or with higher density of points near corners, sharp curves, etc.
I have a few things ringing in my head, I guess that DFT (FFT) would be a good starting point for this resampling, analyzing the scipy.signal.resample() I have found out that there are far more functions in the scipy.signal package which sound promising to me...
What I'm asking for is a suggestion which way I should follow, what tool I should try for this job, which may be the most suitable. Maybe there is a tool meant exactly for what I'm looking for or maybe I'm overthinking this and one of the implementations of FFT like resample() will work just fine (of course, after some adjustments at the starting and ending point of the shape to make sure it's closing without issues)?
Scipy.signal sounds promising, however, as far as I understand, it is meant to work with time series data, not geometrical data - I guess this may cause some problems as my data isn't a function (in a mathematical understanding).
Thanks and best regards!
As far as I understood, what you want is to get an interpolated version of your original data.
The DFT (or FFT) will not achieve this purpose, since it will perform an Fourier Transform (which is not what you want).
Talking theoretically, what you need to interpolate your data is to define a function to calculate the result in the new-data-points.
So, let's say your data contains 5 points, in which one you have a 1D (to simplify) number stored, representing your data, and you want a new array with 10 points, filled with the linear-interpolation of your original data.
Using numpy.interp:
import numpy as np
original_data = [2, 0, 3, 5, 1] # define your data in 1D
new_data_resolution = 0.5 # define new sampling distance (i.e, your x-axis resolution)
interp_data = np.interp(
x = np.arange(0, 5-1+new_data_resolution , new_data_resolution), # new sampling points (new axis)
xp = range(original_data),
fp = original_data
)
# now interp_data contains (5-1) / 0.5 + 1 = 9 points
After this, you will have a (5-1) / new_resolution (which is greater than 5, since new_resolution < 1)-length data, which values will be (in this case) a linear interpolation of your original data.
After you have achieved/understood this example, you can dive in the scipy.interpolate module to get a better understanding in the interpolation functions (my example uses a linear function to get the data in the missing points).
Applying this to n-D dimensional arrays is straight-forward, iterating over each dimension of your data.

Convert 2D Array to a 3D Space

I am trying to develop a 3D cube with values from a flat 2D Plane. I am having a lot of difficulty trying to pseudo code it out so I was hoping to get some input from you guys.
I will try my best to express myself through pictures as I am able to visualize what I am trying to achieve.
I have a 2D output based on the black line in this figure:
I have an array with data of amplitude as each index's value i.e (0; 1) -> the 0 is the x coordinate (sample) and 1 as the y coordinate (amplitude) or as another example (~1900; ~0.25).
How do I take this 1 dimensional sequence and extrude it into a 3D picture like below:
Is there perhaps a library that does such? Or am I going about it the wrong way? The data is from a matched filter output of a sonar signal and I wish to visualize the concentration of the intensity versus where it is located in a sample on a 3D plane. The data has peaks that have inclining and declining gradient slopes before a peak.
I cannot seem to wrap my mind around such a task. Is there a library or a term used to associate what I wish to accomplish?
EDIT: I found this https://www.tutorialspoint.com/matplotlib/matplotlib_3d_surface_plot.htm
But it requires all x, y and z points. Whereas I only have x and y. Additionally I need to be able to access every coordinate (x, y, z) to be able to do range and angle estimation from sample (0, 1) (Transmitted sound where power is highest). I would only like to basically see the top of this though on another 2D axis...
EDIT 2: Following up on a comment below, I would like to convert Figure 1 above into the below image using a library if there exists.
Thanks so much in advanced!

Bilinear interpolation from a (distorted) rectangular 2D grid to arbitrary points, in Python

The task at hand is seemingly simple:
I have a 2D grid of data. The data is available in 2D arrays for X and Y coordinates, as well as the input variable which I want to interpolate. This means I can plot the data using rectangular cells, which means it is possible to use bilinear interpolation. Unfortunately, the data is not precisely aligned with the coordinates, and also not precisely spaced. There were some numerics involved in creating the data, which means that all sampling locations are a little off the mark, and the cell spacing is smooth but not uniform.
I would like to interpolate from this input grid to a set of predefined sample coordinates (as opposed to simply refining the mesh).
In short, an example for my type of input is:
# a nice, regular grid
Xs, Ys = np.meshgrid(np.linspace(0, 1, num=3), np.linspace(0, 1, num=5))
# ...perturbed by some systematic and some random noise ...
X_in = Xs + np.random.normal(scale=0.03, size=(5, 3))
# ...and some systematic deviation
Y_in = (Ys + np.random.normal(scale=0.03, size=(5, 3)))* (1 + Xs**1.5)
# and some variable at each node to interpolate
Z_in = np.random.normal(scale=1, size=(5, 3))
So (X_in, Y_in) are arrays of shape (n, m) which define a mesh with quadrilateral cells, and Z_in another array of ther same shape which provides a value at each node in that mesh. I am looking for some Python library that performs bilinear interpolation of Z_in across those cells.
However, all methods I have found so far either ignore the rectangular structure (and triangulate the data, or fit some 2D spline through arbitrary point clouds), or require a perfectly rectangular and equally-spaced grid as input (which mine is not).
Examples of answers/methods that seem not to be appliccable:
This answer recommends using scipy.ndimage.map_coordinates -- but that effectively uses the indices of the 2D input data array as coordinates, which won't work for me.
scipy.interpolate.interp2d requires either a regular grid (node locations provided by 1D X and Y arrays), or an irregular one, which is flattened, which means that the algorithm cannot know which nodes form a cell. This means it either fits some spline through unstructured data, or triangulates it. And it only interpolates onto regular grids or individual points.
scipy.interpolate.RectBivariateSpline is recommended for interpolation from gridded data but only accepts input points which are perfectly aligned with the coordinate system.
There's also a Matplotlib toolkit for interpolation, which I had thought should be able to do this sort of thing, as it also does interpolated contour plots of rectangular meshes, but as it turns out, even though mpl_toolkits.basemap.interp accepts arbitrary quadrilateral meshes as target for interpolation, it cannot use them as inputs ...
Upon closer inspection, it turns out that even matplotlib.plt.contour() does not seem to perform bilinear interpolation when plotting the input data:
plt.contour(X_in, Y_in, Z_in, levels=np.linspace(Z_in.min(), Z_in.max(), 50))
plt.plot(X_in, Y_in, 'k-')
plt.plot(X_in.T, Y_in.T, 'k-')
As you can see, the contour lines within the cells are straight, but with bilinear interpolation, they should not be, and there should not be those empty quadrilateral areas in the mittle of some cells. I suspect that Matplotlib only finds the contour values on the cell edges and simply draws straight lines between them.
I have found two explanations of the maths of bilinear interpolation from grids which are not perfectly aligned, but I was hoping to come across a ready-made implementation somewhere because I'm sure that this kind of task is not so rare, and a numpy or scipy implementation (if it exists) is probably way faster than whatever I'd implement myself.

How to visualize a matrix of categories as an RGB image?

I am using neural network to do semantic segmentation(human parsing), something like taking a photo of people as input and the neural network tells that every pixel is most likely to be head, leg, background or some other parts of human. The algorithm runs smoothly and giving a numpy.ndarray as output . The shape of the array is (1,23,600,400), where 600*400 is the resolution of the input image and 23 is the number of categories. The 3d matrix looks like a 23-layer stacked 2d matrices, where each layer using a matrix of float to tell the possibility that each pixel is of that category.
To visualize the matrix like the following figure, I used numpy.argmax to squash the 3d matrix into a 2d matrix that holds the index of the most possible category. But I don't have any idea how to proceed to get the visualization I want.
EDIT
Actually, I can do it in a trivial way. That is, use a for loop to traverse through every pixel and assign a color to it to get a image. However, this is not a vectorized coding, since numpy has built-in way to speed up matrix manipulation. And I need to save CPU cycles for real time segmentation.
It's fairly easy. All you need to have is a lookup table mapping the 23 labels into unique colors. The easiest way is to have a 23-by-3 numpy array with each row storing the RGB values for the corresponding label:
import numpy as np
import matplotlib.pyplot as plt
lut = np.random.rand(23, 3) # using random mapping - but you can do better
lb = np.argmax(prediction, axis=1) # converting probabilities to discrete labels
rgb = lut[lb[0, ...], :] # this is all it takes to do the mapping.
plt.imshow(rgb)
plt.show()
Alternatively, if you are only interested in the colormap for display purposes, you can use cmap argument of plt.imshow, but this will requires you to transform lut into a "colormap":
from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list('new_map', lut, N=23)
plt.imshow(lb[0, ...], cmap=cmap)
plt.show()

Fast way using healpy to get convolution of a healpy map and a 2D Gaussian

I have a healpix map with nside=512 (= huge numpy.ndarray, about 3.1E6 elements). I need to convolute it with a 2D Gaussian.
I tried creating a healpix map of the Gausian, multiplying the two numpy.ndarray and getting the sum; this is too slow.
second thing I tried is defining a Gaussian function (not a map) and then a 2D function which for given x,y returns the value of the huge_map*gausian at the position x,y. then I used scipy.integrate.nquad to get the integral. Faster than the first method, but ...
Is there a way how to use the expansion to spherical harmonics to get the integral of the huge_map*Gaussian even faster? Something like expand the huge_map in spherical harm., expand the Gaussian in spherical harm., multiply the coefficients, transform back, get the sum.
And possible modifications like: a) work instead with log of the huge_map and log of the Gaussian, then the operation to by done to the coefficients is clearly sum. b) get somehow the integral directly from the coefficients, without the back transformation? c) limit the expansion to some number and speed up the whole thing?
how to implement it correctly using healpy ?
you can try to use healpy.smoothing, for example:
wmap_map_I_smoothed = hp.smoothing(wmap_map_I, fwhm=60, arcmin=True)
hp.mollview(wmap_map_I_smoothed, min=-1, max=1, title='Map smoothed 1 deg')
Documentation:
http://healpy.readthedocs.org/en/latest/generated/healpy.sphtfunc.smoothing.html

Categories