I'm working with some instrument data that has records the temperature at a specific latitude, longitude, and pressure (height) coordinate. I need to create a 3d grid from this instrument data that I can then use to take a vertical cross sections of the interpolated gridded data. I've looked at pretty much every interpolation function/library I can find and I'm still having trouble just wrapping my head around how to do this.
I'd prefer not to use Mayavi, since it seems to bug out on my school's server and I'd rather not try to deal with fixing it right now.
The data is currently in 4 separate 1d arrays and I used those to mock up some scatter plots of what I'm trying to get.
Here is the structure of my instrument data points:
And here is what I'm trying to create:
Ultimately, I'd like to create some kind of 3d contour from these points that I can take slices of. Each of the plotted points has a corresponding temperature attached to it, which is really what I think is throwing me off in terms of dimensions and whatnot.
There are a few options to go from the unstructured data which you have to a structured dataset.
The simplest option might be to use the scipy interpolate.griddata method which can interpolate unstructured points using, linear or cubic interpolation.
Another option is to define your grid and then average all of the unstructured points which fall into each grid cell, giving you some gridded representation of the data. You could use a tool such as CIS to do this easily (full disclosure, I wrote this package to do exactly this kind of thing).
Or, there are more complicated methods of interpolating the data by trying to determine the most likely value of the grid points based on the unstructured data, for example using kriging with the pyKriging package, though I've never used this.
Related
I want to detect lines within Scatter Plots, using python.
Specifically, my data set is of the Cartesian coordinates of points. (This data was gathered using four ultrasound sensors on servos).
Here are some data sets ,
And here are the lines I'd like to detect.
the problem is to write a python program that returns the start and endpoints of high certainty lines, given a list of points on the scatter plot.
The difficulty is that piecewise linear regression can't be applied directly since data is vertically stacked.
Is there a well known solution to this problem? Or maybe an ingenuous application of piecewise linear regression could work?
I'd really appreciate some functioning python code!
I have some data of soil's moisture content (Theta) in the form of 3D-domain points (CSV file of the columns x, y, z, Theta). I want to take cross sections from the 3D domain in some specific positions (section ABCD in the figure). I want to calculate the value of Theta in a 5*5 grid in the cross-section, but the points around each node of the grid are not coplanar with the unknown point. I did this before for 2D domains in python, but the 3D domains seem more complicated for me. I found that plotly can make something like that in its virtual environment but I want this to output a numpy array or pandas DataFrame to draw it as a contour in the jupyter notebook.
I know that finding the grid involves finding the value of each point like P0 in the figure by interpolation or gridding from its neighbors, then to draw the cross section using matplotlib, but I don' know how to do it.
Related question, Is slicing 2D grids from 3D grids available in matplotlib or similar libraries?
Thanks for all help.
The underlying problem is 3D interpolation. There are numerous packages which can do this type of thing, or you can write your own (using, e.g. KDE, which is basically just a type of smoothing/binning). There is a lot of material on the topic, like
This answer https://stackoverflow.com/a/15753011/230468
The scipy docs
This extensive set of option on scicomp.stack
And this blog post (with some good examples)
Have you tried playing with pyugrid? It's a library specifically for manipulating unstructured grids, so it sounds like it might be of some use to you. Check out these example notebooks.
I have satellite sweep data that I am attempting to plot on a basemap using pcolormesh.
The data is organized as a 2d matrix, (bTemp), with two corresponding 2D arrays lat and lon, that give the corresponding latitude and longitude at each point. So my plotting code looks like
m = Basemap()
m.drawcoastlines()
m.pcolormesh(lon, lat, bTemp)
However, this doesn't quite give me the result I am looking for. A large stripe runs across the map.
I think the cause of this is that my longitude increases non-monotonically along a given ray, at the wrap-around point.
Here is a plot of on ray in my longitude array
plot(lon[100,:])
What would the best way to fix this be, so that the pcolormesh plot just jumps to the other side of the map without connecting the two points with a filled in area?
You could try using numpy.unwrap to unwrap your data. It was designed exactly for this: to unwrap angular jumps (in radians) that are artifacts due to the way how we use single-valued functions.
You need to convert your longitudes back and forth between degrees and radians for this to work, but the gist is this:
import numpy as np
lon_unwrapped = np.rad2deg(np.unwrap(np.deg2rad(lon)))
This will find those jumps in your data that are larger than 180 degrees, and take their 360-degree complement. Example:
np.rad2deg(np.unwrap(np.deg2rad([150,-150])))
# array([ 150., 210.])
Additionally, unwrap accepts ndarray data, and you can tell it along which axes to perform the unravelling. By default this is the last dimension, which seems to be fine for your case, where you want to unwrap lon[i,:].
I'm stuck at this things for last couple of days. I have two sets of unstructured data. First set of data has only 9583 points and second one has 60000 points. I need to interpolate a particular field of first set data(x1[len = 9583], y1[len = 9583], temperature1[len = 9583]) to second set of data (x2[len = 60000], y2[len = 60000], temperature2[len = 60000]). I have tried with python "SmoothBivariateSpline" and looked for other options. But I did not find any reasonable solutions. I'm open for either python or c++ . It would be highly appreciable if anybody can help me figuring out the solution. I attached the figure for your convenience
Figure shows two sets of unstructured
There's a bunch on SO on unstructured interpolation, including:
Python/Scipy 2D Interpolation (Non-uniform Data)
What to do if I want 3D spline/smooth interpolation of random unstructured data?
3D interpolation of 2 lists in python
And of course, there's always the SciPy Interpolation Documentation
Contrary to the title of your question, your data looks more like what Numpy calls "1D data" (that is XY data) rather than "2D data" (XYZ data). It also looks like your data is mostly sampling & noise, rather than fit to some mathematical function. So, I'd suggest starting at the top of the list from the SciPy Docs, skipping over the univariate and multivariate stuff and trying something in the 1-D Splines section, perhaps Scipy.interpolate.BSpline.
I am trying to create a 2D Contour Map in Python that looks like this:
In this case, it is a map of chemical concentration for a number of points on the map. But for the sake of simplicity, we could just say it's elevation.
I am given the map, in this case 562 by 404px. I am given a number of X & Y coordinates with the given value at that point. I am not given enough points to smoothly connect the line, and sometimes very few data points to draw from. It's my understanding that Spline plots should be used to smoothly connect the points.
I see that there are a number of libraries out there for Python which assist in creation of the contour maps similar to this.
Matplotlib's Pyplot Contour looks promising.
Numpy also looks to have some potential
But to me, I don't see a clear winner. I'm not really sure where to start, being new to this programming graphical data such as this.
So my question really is, what's the best library to use? Simpler would be preferred. Any insight you could provide that would help get me started the proper way would be fantastic.
Thank you.
In the numpy example that you show, the author is actually using Matplotlib. While there are several plotting libraries, Matplotlib is the most popular for simple 2D plots like this. I'd probably use that unless there is a compelling reason not to.
A general strategy would be to try to find something that looks like what you want in the Matplotlib example gallery and then modify the source code. Another good source of high quality Matplotlib examples that I like is:
http://astroml.github.com/book_figures/
Numpy is actually a N-dimensional array object, not a plotting package.
You don't need every pixel with data. Simply mask your data array. Matplotlib will automatically plot the area that it can and leave other area blank.
I was having this same question. I found that matplotlib has interpolation which can be used to smoothly connect discrete X-Y points.
See the following docs for what helped me through:
Matplotlib's matplotlib.tri.LinearTriInterpolator docs.
Matplotlib's Contour Plot of Irregularly Spaced Data example
How I used the above resources loading x, y, z points in from a CSV to make a topomap end-to-end