I am basically trying to achieve the opposite of this question.
I have a set of latitude and longitude coordinates (with values) in the WGS84 coordinate system, that I would like to write to a geotiff (or just add to a gdal dataset) via the gdal python bindings.
For example, my starting data might be:
lat = np.array([45.345,56.267,23.425])
lon = np.array([134.689,128.774,111.956])
value = np.array([3.0,6.2,2.5])
How might one do this? Thanks!
Although it is not in your question, it appears you need to project the lat/long data from the WGS84 datum to a UTM projection. This can be using the ogr2ogr command line from GDAL using the two options -a_srs 4326 -t_srs ???? (the target SRID). It can also be done internally with Python using the OGR module of GDAL. Here is an example of use.
There are two independent ways to get a raster from point data. The first is to interpolate the values in data, so that the values flood the region (or sometimes only the convex hull). There are many methods and tools to interpolate values in 2D. With GDAL, a comand-line tool gdal_grid is useful for this purpose, although I don't think it is possible to use from Python. Probably the simplest would be to use scipy.interpolate. Once you have a 2D NumPy array, it is simple to create a raster file with GDAL/Python.
The second method of converting the points to a raster is to burn the point locations to pixels on a raster. Unlike the first method, only the locations where the points are have values, while the values are not interpolated anywhere else in the raster. Rasterising or burning vectors into a raster can be done from a GDAL command line tool gdal_rasterize. It can also be done internally with GDAL/Python, here is an example.
It is possible to use gdal_grid from Python. I am using it.
All you need to do is construct the command as if you were using it from the command line and put it inside a subprocess.call(com, shell=True). You need to import subprocess module first.
This is actually how I am using it:
pcall= "gdal_grid --config 'NUM_THREADS=ALL_CPUS GDAL_CACHEMAX=2000'\
-overwrite -a invdist:power=2.0:smoothing=2.0:radius1=360.0:radius2=360.0\
-ot UInt16 -of GTiff -outsize %d %d -l %s -zfield 'Z' %s %s "%(npx, npy,\
lname,ptshapefile,interprasterfile)
subprocess.call(pcall, shell= True)
NUM_THREADS option is available from gdal 1.10+
Related
I would like to convert my dataset of SVY21 coordinates, into WGS84 coordinates.
I am currently using this script from this repo I found but this script this yields inaccurate results with a discrepancy of up to 0.04, so the coordinates that I convert end up being on an entirely different geographical location in the same country.
Was wondering if there is anyone who can assist to help me with a script for converting a large dataset from SVY21 to WGS84?
E.G I want to convert
38816.0396118, 34379.9602051
but instead I get
1.36728713070321, 103.890645888016
when I should be getting
1.327235496598071, 103.93042021823591
I would do it on those online converters but my file sizes are pretty big(can go up to few GB) so it's better to run a script on my local computer instead using either Python or C++ or any other alternatives that will work. Also most online converters have a limit on file size as well.
Here's an accurate converter link: https://dominoc925-pages.appspot.com/webapp/calc_svy21/default.html but it doesnt accept my file size.
Appreciate the help :D Thanks~!
You probably have your coordinates the wrong way around. Consider the following:
import pyproj
xfm = pyproj.Transformer.from_crs('EPSG:3414', 'EPSG:4326')
x, y = 38816.0396118, 34379.9602051
print(xfm.transform(x, y))
# prints: (1.3673123058118237, 103.89064694097199)
print(xfm.transform(y, x))
# prints: (1.3271927478890677, 103.93050656742128)
Still about a ten-thousandth of a degree off, but I don't know how good pyproj's coordinate space definitions are in this case.
How to extract a profile of values from a raster along a given shapefile line in Python?
I am struggling finding a method to extract a profile of values (e.g. topographic profile) from a raster (geotiff). The library Rasterio has a method to clip/extract value from a raster based on a polygon, but I cannot find an equivalent method for a line shapefile.
There is a basic method with scipy, but it does not inherently conserve geographic information like a method based on higher level toolbox like rasterio could provide.
In other words, I am looking for an equivalent in Python of what the tool Terrain Profile in QGIS offers.
Thanks
This is a bit different than extracting for a polygon, as you want to sample every pixel touched by the line, in the order they are touched (the polygon approaches don't care about pixel order).
It looks like it would be possible to adapt this approach to use rasterio instead. Given a line read from a shapefile using geopandas or fiona as a shapely object, you use the endpoints to derive a new equidistant projection that you use as dst_crs in a WarpedVRT and read pixel values from that. It looks like you would need to calculate the length of your line in terms of the number of pixels you want sampled, this is the width parameter of the WarpedVRT.
This approach may need to be adapted further if your line is not an approximately straight line between the endpoints.
If you want to just get the raw pixel values under the line, you should be able to use a mask in rasterio or rasterize directly, for each line. You may want to use the all_touched=True in the case of lines.
I had a similar problem and found a solution which works for me. The solution uses shapely to sample points on a line/lines and then accesses respective values from the GeoTiff, therefore the extracted profile follows the direction of the line. Here is the method that I ended up with:
def extract_along_line(xarr, line, n_samples=256):
profile = []
for i in range(n_samples):
# get next point on the line
point = line.interpolate(i / n_samples - 1., normalized=True)
# access the nearest pixel in the xarray
value = xarr.sel(x=point.x, y=point.y, method="nearest").data
profile.append(value)
return profile
Here is a working example with data from the copernicus-dem database and the line is the diagonal of the received tile:
import rioxarray
import shapely.geometry
import matplotlib.pyplot as plt
sample_tif = ('https://elevationeuwest.blob.core.windows.net/copernicus-dem/'
'COP30_hh/Copernicus_DSM_COG_10_N35_00_E138_00_DEM.tif')
# Load xarray
tile = rioxarray.open_rasterio(sample_tif).squeeze()
# create a line (here its the diagonal of tile)
line = shapely.geometry.MultiLineString([[
[tile.x[-1],tile.y[-1]],
[tile.x[0], tile.y[0]]]])
# use the method from above to extract the profile
profile = extract_along_line(tile, line)
plt.plot(profile)
plt.show()
I have geotiff file which I have read into an numpy array as described in the link below:
Working with TIFFs (import, export) in Python using numpy
The size of the Geotiff array that I have is (465,465) and I have acquired the meta data of the file using gdalinfo and it's using WGS84 as it's CRS.
What I wish to do with the file is to translate the x y lat lon co-ordinates that I see in QGIS and Gdalinfo to actual positions of points in the imported numpy array, how would I go about doing this?
You need to use the geotransform, from an opened GDAL dataset you can get it with:
gt = ds.GetGeoTransform()
From the GDAL documentation:
The affine transform consists of six coefficients returned by
GDALDataset::GetGeoTransform() which map pixel/line coordinates into
georeferenced space using the following relationship:
Xgeo = GT(0) + Xpixel*GT(1) + Yline*GT(2)
Ygeo = GT(3) + Xpixel*GT(4) + Yline*GT(5)
https://gdal.org/user/raster_data_model.html
If your raster is not rotated, its fairly simple, just subtract the origin and divide by the resolution.
There are several libraries which can perform such an affine transformation, for example the aptly named 'affine' library. A detailed explanation on its usage can be found at:
http://www.perrygeo.com/python-affine-transforms.html
Are there any libraries in Python that have functions that compute for geoid heights using egm84, egm96 and egm2008?
I know that geographiclib has a function (http://geographiclib.sourceforge.net/html/classGeographicLib_1_1GravityModel.html) that computes for geoid heights using the three egms but I don't know how to implement them in Python (if they are really applicable). How do you implement that? Or if it isn't applicable to Python, are there any libraries that can be used?
Thank you.
There is some relevant work done here on this:
https://github.com/mrJean1/PyGeodesy
Their instructions on use were not the most up to date, but here is a TLDR:
First download your choice of geoid from here, like so:
wget https://sourceforge.net/projects/geographiclib/files/geoids-distrib/egm2008-5.tar.bz2
bzip2 -d egm2008-5.tar.bz2
tar -xvf egm2008-5.tar
Then use in your python script:
import pygeodesy
from pygeodesy.ellipsoidalKarney import LatLon
ginterpolator = pygeodesy.GeoidKarney("./geoids/egm2008-5.pgm")
# Make an example location
lat=51.416422
lon=-116.217151
# Get the geoid height
single_position=LatLon(lat, lon)
h = ginterpolator(single_position)
print(h)
This will give you the deviation from the ellipsoid at that location in meters:
-11.973145778529625
This roughly matches up with what we find when we use an online calculator.
I'm interested in using Spectral Python (SPy) to visualize and classify multiband raster GeoTIFF (not hyperspectral data). Currently it appaers that only .lan, .gis File Formats are readable.
I've tried to convert files to .lan with gdal_translate but the image format is not supported( IOError: Unable to determine file type or type not supported).
Any idea how to use this library for non hypersperctral dataset?
Convert the GeoTIFF file to a compatible format (e.g. LAN). This can be done in one of two ways. From a system shell, use gdal_translate:
gdal_translate -of LAN file.tif file.lan
Or similar within Python:
from osgeo import gdal
src_fname = 'file.tif'
dst_fname = 'file.lan'
driver = gdal.GetDriverByName('LAN')
sds = gdal.Open(src_fname)
dst = driver.CreateCopy(dst_fname, sds)
dst = None # close dataset; the file can now be used by other processes
Note that the first method is actually better, as it also transfers other metadata, such as the spatial reference system and possibly other data. To correctly do the same in Python would require adding more lines of code.