I am trying to develop a 3D cube with values from a flat 2D Plane. I am having a lot of difficulty trying to pseudo code it out so I was hoping to get some input from you guys.
I will try my best to express myself through pictures as I am able to visualize what I am trying to achieve.
I have a 2D output based on the black line in this figure:
I have an array with data of amplitude as each index's value i.e (0; 1) -> the 0 is the x coordinate (sample) and 1 as the y coordinate (amplitude) or as another example (~1900; ~0.25).
How do I take this 1 dimensional sequence and extrude it into a 3D picture like below:
Is there perhaps a library that does such? Or am I going about it the wrong way? The data is from a matched filter output of a sonar signal and I wish to visualize the concentration of the intensity versus where it is located in a sample on a 3D plane. The data has peaks that have inclining and declining gradient slopes before a peak.
I cannot seem to wrap my mind around such a task. Is there a library or a term used to associate what I wish to accomplish?
EDIT: I found this https://www.tutorialspoint.com/matplotlib/matplotlib_3d_surface_plot.htm
But it requires all x, y and z points. Whereas I only have x and y. Additionally I need to be able to access every coordinate (x, y, z) to be able to do range and angle estimation from sample (0, 1) (Transmitted sound where power is highest). I would only like to basically see the top of this though on another 2D axis...
EDIT 2: Following up on a comment below, I would like to convert Figure 1 above into the below image using a library if there exists.
Thanks so much in advanced!
So, I have three numpy arrays which store latitude, longitude, and some property value on a grid -- that is, I have LAT(y,x), LON(y,x), and, say temperature T(y,x), for some limits of x and y. The grid isn't necessarily regular -- in fact, it's tripolar.
I then want to interpolate these property (temperature) values onto a bunch of different lat/lon points (stored as lat1(t), lon1(t), for about 10,000 t...) which do not fall on the actual grid points. I've tried matplotlib.mlab.griddata, but that takes far too long (it's not really designed for what I'm doing, after all). I've also tried scipy.interpolate.interp2d, but I get a MemoryError (my grids are about 400x400).
Is there any sort of slick, preferably fast way of doing this? I can't help but think the answer is something obvious... Thanks!!
Try the combination of inverse-distance weighting and
scipy.spatial.KDTree
described in SO
inverse-distance-weighted-idw-interpolation-with-python.
Kd-trees
work nicely in 2d 3d ..., inverse-distance weighting is smooth and local,
and the k= number of nearest neighbours can be varied to tradeoff speed / accuracy.
There is a nice inverse distance example by Roger Veciana i Rovira along with some code using GDAL to write to geotiff if you're into that.
This is of coarse to a regular grid, but assuming you project the data first to a pixel grid with pyproj or something, all the while being careful what projection is used for your data.
A copy of his algorithm and example script:
from math import pow
from math import sqrt
import numpy as np
import matplotlib.pyplot as plt
def pointValue(x,y,power,smoothing,xv,yv,values):
nominator=0
denominator=0
for i in range(0,len(values)):
dist = sqrt((x-xv[i])*(x-xv[i])+(y-yv[i])*(y-yv[i])+smoothing*smoothing);
#If the point is really close to one of the data points, return the data point value to avoid singularities
if(dist<0.0000000001):
return values[i]
nominator=nominator+(values[i]/pow(dist,power))
denominator=denominator+(1/pow(dist,power))
#Return NODATA if the denominator is zero
if denominator > 0:
value = nominator/denominator
else:
value = -9999
return value
def invDist(xv,yv,values,xsize=100,ysize=100,power=2,smoothing=0):
valuesGrid = np.zeros((ysize,xsize))
for x in range(0,xsize):
for y in range(0,ysize):
valuesGrid[y][x] = pointValue(x,y,power,smoothing,xv,yv,values)
return valuesGrid
if __name__ == "__main__":
power=1
smoothing=20
#Creating some data, with each coodinate and the values stored in separated lists
xv = [10,60,40,70,10,50,20,70,30,60]
yv = [10,20,30,30,40,50,60,70,80,90]
values = [1,2,2,3,4,6,7,7,8,10]
#Creating the output grid (100x100, in the example)
ti = np.linspace(0, 100, 100)
XI, YI = np.meshgrid(ti, ti)
#Creating the interpolation function and populating the output matrix value
ZI = invDist(xv,yv,values,100,100,power,smoothing)
# Plotting the result
n = plt.normalize(0.0, 100.0)
plt.subplot(1, 1, 1)
plt.pcolor(XI, YI, ZI)
plt.scatter(xv, yv, 100, values)
plt.title('Inv dist interpolation - power: ' + str(power) + ' smoothing: ' + str(smoothing))
plt.xlim(0, 100)
plt.ylim(0, 100)
plt.colorbar()
plt.show()
There's a bunch of options here, which one is best will depend on your data...
However I don't know of an out-of-the-box solution for you
You say your input data is from tripolar data. There are three main cases for how this data could be structured.
Sampled from a 3d grid in tripolar space, projected back to 2d LAT, LON data.
Sampled from a 2d grid in tripolar space, projected into 2d LAT LON data.
Unstructured data in tripolar space projected into 2d LAT LON data
The easiest of these is 2. Instead of interpolating in LAT LON space, "just" transform your point back into the source space and interpolate there.
Another option that works for 1 and 2 is to search for the cells that maps from tripolar space to cover your sample point. (You can use a BSP or grid type structure to speed up this search) Pick one of the cells, and interpolate inside it.
Finally there's a heap of unstructured interpolation options .. but they tend to be slow.
A personal favourite of mine is to use a linear interpolation of the nearest N points, finding those N points can again be done with gridding or a BSP. Another good option is to Delauney triangulate the unstructured points and interpolate on the resulting triangular mesh.
Personally if my mesh was case 1, I'd use an unstructured strategy as I'd be worried about having to handle searching through cells with overlapping projections. Choosing the "right" cell would be difficult.
I suggest you taking a look at GRASS (an open source GIS package) interpolation features (http://grass.ibiblio.org/gdp/html_grass62/v.surf.bspline.html). It's not in python but you can reimplement it or interface with C code.
Am I right in thinking your data grids look something like this (red is the old data, blue is the new interpolated data)?
alt text http://www.geekops.co.uk/photos/0000-00-02%20%28Forum%20images%29/DataSeparation.png
This might be a slightly brute-force-ish approach, but what about rendering your existing data as a bitmap (opengl will do simple interpolation of colours for you with the right options configured and you could render the data as triangles which should be fairly fast). You could then sample pixels at the locations of the new points.
Alternatively, you could sort your first set of points spatially and then find the closest old points surrounding your new point and interpolate based on the distances to those points.
There is a FORTRAN library called BIVAR, which is very suitable for this problem. With a few modifications you can make it usable in python using f2py.
From the description:
BIVAR is a FORTRAN90 library which interpolates scattered bivariate data, by Hiroshi Akima.
BIVAR accepts a set of (X,Y) data points scattered in 2D, with associated Z data values, and is able to construct a smooth interpolation function Z(X,Y), which agrees with the given data, and can be evaluated at other points in the plane.
I am just starting to use healpy in python. In shortly, I have a data of position let
x = np.array([7.47098722, 7.47805867, 7.47098779....12.57804461, 12.5809734])
y = np.array([58.32516929, 58.33223959, 58.33516828, ..., 63.41415155, 63.41707922])
where phi(x, y) is the potential of the matter at these location. Now, I want to calculate the power spectrum for this. But I don't understand how to set my data for healpy to make a map by using mollview and calculating the power spectrum by using anafast.
You just need to convert the thetas and phis to pixel indices using one of
healpy's pixel functions, healpy.pixelfunc.ang2pix. You can read more about it from https://healpy.readthedocs.io/en/latest/generated/healpy.pixelfunc.ang2pix.html
After you have indices, you also have potential values for those indices, in other words, you have the corresponding HEALPix map(use mollview to get a
mollview projection of this map). Finally, use healpy's anafast, https://healpy.readthedocs.io/en/latest/healpy_spht.htm and you have you power spectrum.
For more information about healpy, kindly refer-
https://buildmedia.readthedocs.org/media/pdf/healpy/1.8.6/healpy.pdf
https://healpix.sourceforge.io/
also, visit https://healpix.jpl.nasa.gov
I have a HEALPix map that I have read in using healpy, however it is in galactic coordinates and I need it in celestial/equatorial coordinates. Does anybody know of a simple way to convert the map?
I have tried using healpy.Rotator to convert from (l,b) to (phi,theta) and then using healpy.ang2pix to reorder the pixels, but the map still looks strange.
It would be great if there was a function similar to Rotator that you could call like: map = AnotherRotator(map,coord=['G','C']). Anybody know of any such function??
Thanks,
Alex
I realise that this was asked ages ago, but I was having the same problem myself this week and found your post. I've found a couple of potential solutions, so I'll share incase someone else comes upon this and finds it useful.
Solution 1: This sort of depends on the format your data is coming in. Mine came in a (theta, phi) grid.
import numpy as np
import healpy as H
map = <your original map>
nside = <your map resolution, mine=256>
npix = H.nside2npix(nside)
pix = N.arange(npix)
t,p = H.pix2ang(nside,pix) #theta, phi
r = H.Rotator(deg=True, rot=[<THETA ROTATION>, <PHI ROTATION>])
map_rot = np.zeros(npix)
for i in pix:
trot, prot = r(t[i],p[i])
tpix = int(trot*180./np.pi) #my data came in a theta, phi grid -- this finds its location there
ppix = int(prot*180./np.pi)
map_rot[i] = map[ppix,tpix] #this being the rright way round may need double-checking
Solution 2: Haven't quite finished testing this, but just came across it AFTER doing the annoying work above...
map_rot = H.mollview(map,deg=True,rot=[<THETA>,<PHI>], return_projected_map=True)
which gives a 2D numpy array. I'm interested to know how to convert this back into a healpix map...
I found another potential solution, after searching off and on for a few months. I haven't tested it very much yet, so please be careful!
Saul's Solution 2, above, is the key (great suggestion!)
Basically, you combine the functionality of healpy.mollview (gnomview, cartview, and orthview work as well) with that of the reproject_to_healpix function in the reproject package (http://reproject.readthedocs.org/en/stable/).
The resulting map is suitable for my angular scales, but I can't say how accurate the transformation is compared to other methods.
-----Basic Outline----------
Step 1: Read-in the map and make the rectangular array via cartview. As Saul indicated above, this is also one way to do the rotation. If you're just doing a standard rotation/coordinate transformation, then all you need is the coord keyword. From Celestial to Galactic coordinates, set coord = ['C','G']
map_Gal = hp.cartview(map_Cel, coord=['C','G'], return_projected_map=True, xsize=desired_xsize, norm='hist',nest=False)
Step 2: Write a template all-sky FITS header (as in the example below). I wrote mine to have the same average pixel-scale as my desired HEALPix map.
Step 3: Use reproject.transform_to_healpix
reproject includes a function for mapping a "normal" array (or FITS file) into the HEALPix projection. Combine that with the ability to return the array created by healpy.mollview/cartview/orthview/gnomview, and you can rotate a HEALPix map of one coordinate system (Celestial) into another coordinate system (Galactic).
map_Gal_HP, footprint_Gal_HP = rp.reproject_to_healpix((map_Gal, target_header), coord_system_out= 'GALACTIC', nside=nside, nested=False)
It comes down, essentially, to those two commands. However you'll have to make a template header, giving the pixel scale and size corresponding to the intermediary all-sky map you want to make.
-----Full Working Example (iPython notebook format + FITS sample data)------
https://github.com/aaroncnb/healpix_coordtrans_example/tree/master
The code there should run very quickly, but that's because the maps are heavily degraded. I did the same for my NSIDE 1024 and 2048 maps, and it took
about an hour.
------Before and After Images------
This function seems to do the trick (reasonably slow, but should be better than the for loop):
def rotate_map(hmap, rot_theta, rot_phi):
"""
Take hmap (a healpix map array) and return another healpix map array
which is ordered such that it has been rotated in (theta, phi) by the
amounts given.
"""
nside = hp.npix2nside(len(hmap))
# Get theta, phi for non-rotated map
t,p = hp.pix2ang(nside, np.arange(hp.nside2npix(nside))) #theta, phi
# Define a rotator
r = hp.Rotator(deg=False, rot=[rot_phi,rot_theta])
# Get theta, phi under rotated co-ordinates
trot, prot = r(t,p)
# Interpolate map onto these co-ordinates
rot_map = hp.get_interp_val(hmap, trot, prot)
return rot_map
Using this on data from PyGSM gives the following:
hp.mollview(np.log(rotate_map(gsm.generated_map_data, 0,0)))
Upon rotation of phi:
hp.mollview(np.log(rotate_map(gsm.generated_map_data, 0,np.pi)))
Or rotating theta:
hp.mollview(np.log(rotate_map(gsm.generated_map_data, np.pi/4,0)))
What is the best way to resample a 11000 x 13000 cells raster image? I would want to recalculate every cell using the values of all cells within a circle of x meters. It is similar to a convolution, but the calculation will differ each run.
I could well work this out by exporting the map to an ascii format such as asciiGrid, but I do hope ArcGis provides a better way of realizing it.
The desired result would look something like this:
for y in lyrYrange:
for x in lyrXrange:
window = lyr.getWindow(x, y, sizeX, sizeY)
newlyr[x, y] = DoMyProcessing(window)
The unknown part here is what function to call instead of the fictitious "getWindow()".