I am trying to read data from a nc file, which has the following variables:
['latitude',
'longitude',
'latitude_bnds',
'longitude_bnds',
'time',
'minimum',
'maximum',
'average',
'stddev',
'AirTemperature']
What I am trying to achieve is to extract the AirTemperature data for any given (time, latitude and longitude):
And for that, I am doing something like this:
df = Dataset('data_file.nc', 'r')
lat = df.variables['latitude'][:]
lon = df.variables['longitude'][:]
temp = df.variables['AirTemperature'][:,:,:]
#(lat, lon) for Coffee, TN
test_lat = 35.45
test_lon = -86.05
#getting the indices for the (lat, lon) using numpy.where
lat_idx = np.where(lat==test_lat)[0][0]
lon_idx = np.where(lon==test_lon)[0][0]
#extracting data for all the times for given indices
tmp_crd = temp[:,lat_idx,lon_idx]
Up till this point, it all goes fine. However, when I print the data, I see all the identical values being printed.. (for any lat, lon that I have been testing..)
print tmp_crd.data
>>> [-9999. -9999. -9999. ..., -9999. -9999. -9999.]
Which I don't seem to understand..why the air temperature is always shown as -9999.0? I have tested for a lot of other (lat, lon) points, and it seems for every location point, the air temperature is -9999.0. How can I extract the real data from this file?
Please help :-(.
Thank You
Okay..I think I figured out. Here is what was happening:
The nc file i have has a different precision for latitude and longitudes, and I was apparently passing much more rounded sets of (lat, lon) points. Once I figured out the right precision, it works fine for me. The -9999.0 value was basically the _fill_value for the numpy's masked array (which indicated that if there is no record matching the given set of lat and long, return the masked values).
Thanks every one.
Related
I have a Sentinel 3 image which is stored in a number of netcdf files. The variable is stored in the file "LST_in.nc" with dimensions = rows and columns of the image. The lat and long are in another file "geodetic_in.nc". I want to export the image with the lat and long to tiff format.
To my understanding, the names of dimensions and coordinates should be the same, while I failed to do this
here are my attempts
import rioxarray as rio
import xarray as xr
xds = xr.open_dataset('LST_in.nc')
coord =xr.open_dataset('geodetic_in.nc')
lat, lon = coord.latitude_in.data, coord.longitude_in.data
xds = xds.assign_coords({"lat":(["rows","columns"], lat), "lon":(["rows","columns"], lon)})
xds = xds.rename_dims({"rows": "lon", "columns": 'lat'})
Here I received this error
ValueError: Cannot rename rows to lon because lon already exists. Try using swap_dims instead.
Then I tried this
xds = xds.swap_dims({'rows' : 'lon', 'columns' : 'lat'})
but received another error
ValueError: replacement dimension 'lon' is not a 1D variable along the old dimension 'rows'
Also this one
lst = xds.LST
lst.rio.set_spatial_dims(x_dim = 'lon', y_dim = 'lat', inplace = True)
Error: MissingSpatialDimensionError: x dimension (lon) not found. Data variable: LST
The only one that works but with the wrong coordinates is
lst = xds.LST
lst.rio.set_spatial_dims(x_dim = 'columns', y_dim = 'rows', inplace = True)
lst.rio.write_crs("epsg:4326", inplace = True)
lst.rio.to_raster("lst.tif")
I would appreciate your help. attached is the image files
https://wetransfer.com/downloads/e3711adf56f73cd07119b43d19f7360820220117154330/c46b21
The short answer is: you can't. Because both netCDF and grib are gridded data format and the current data points positions can't be described using a regular latitude/longitude grid.
I plotted a sample of your data points latitude and longitude:
As you can see, the data points are not placed on lines of constant latitude and longitude, they do not follow any pattern that could be describe with projected grid, rotated grid or curvilinear grid either.
If you want to make a gridded file with the LST values and latitude and longitude as coordinates, you will have to reproject your data. You can use the rasterio.warp module, see here for more information.
I'm trying to open, read, and plot cloud cover data from a netCDF file. The file opens and plots fine in the Panoply viewer, so the data looks to be OK. But I can't figure out for the life in me how to convert the latitudes and longitudes into a 1-D array each, they seem to be 2-D. Which makes NaN sense to me...
Reading the file and variables works fine:
fh = Dataset("/home/ubuntu/HIMA8_CC/Himawari8_AHI_FLDK_2020171_0140_00_CLOUD_MASK_EN.nc", mode='r')
lon = fh.variables['Longitude'][:]
lat = fh.variables['Latitude'][:]
cloud_mask = fh.variables['CloudMask'][:]
However, the dimensions of the latitude and longitude variable are 2D, I would have expected those to be 1D:
print(lon.shape)
print(lat.shape)
print(np.mean(lon))
print(cloud_mask.shape)
print(np.mean(lon))
print(np.mean(cloud_mask))
prints
(5500, 5500)
(5500, 5500)
91.97970824333167
(5500, 5500)
91.97970824333167
1.8154066433116118
the mean values look as expected. Can anyone with netCDF experience shed some light on what it is I'm missing here?
You could try this.
import xarray as xr
fh = xr.open_dataset('/home/ubuntu/HIMA8_CC/Himawari8_AHI_FLDK_2020171_0140_00_CLOUD_MASK_EN.nc')
lon = fh['Longitude'].values.flatten()
lat = fh['Latitude'].values.flatten()
I'm starting in python programming and I would like to make a small script which displays the data of "Local temperature diagnosed at 2m height above the relief", "Zonal component of the west-east horizontal wind diagnosed at 10 m height" and the "Meridian component of the horizontal wind diagnosed at 10 m height" as a function of longitude and latitude.
For this, I download a file from the open data site of Meteofrance OPEN DATA by selecting:
Domain: "France - 0.01 °", Sub Package: "SP1 - Current surface parameters", Deadline group "0h" and Run date "2020-02-10 00 UTC"
So I have a file in grib2 format that I am trying to process with the pygrib library
To start simple, I'm just trying to get the temperature for a given point (longitude = 0.25, latitude = 49.21)
I created an index to read the file (It seems that it is the fastest)
indx = pygrib.index('./AROME_0.01_SP1_00H_2020021000.grib2', 'typeOfLevel', 'level', 'name')
I select the records which correspond to the temperature and I recover the values:
msg = indx.select(level=2, typeOfLevel="heightAboveGround", name="2 metre temperature")
temp2m = msg[0].values
The problem is that from there, I fumble and I don't find how to retrieve from this variable (numpy.ma.core.MaskedArray) the value that corresponds to my longitude = 0.25 and latitude = 49.21
If someone has an idea, I'm interested
import pygrib
indx = pygrib.index('./AROME_0.01_SP1_00H_2020021000.grib2', 'typeOfLevel', 'level', 'name')
msg = indx.select(level=2, typeOfLevel="heightAboveGround", name="2 metre temperature")
temp2m = msg[0].values
You could use the pygrib.data() method to create a bounding box around your lat/lon and then take the mean of the values within that box. Just adjust the tolerance to control how big the box is.
lat, lon, tolerence = 49.21, 0.25, 0.1
data, lats, lons = grb.data(lat1=lat-tolerence,lat2=lat+tolerence,
lon1=lon-tolerence,lon2=lon+tolerence)
data.mean()
You should be able to get your lat and lon by using .latlons() from your variable that you selected (not the values). From there you can get your index corresponding your point.
import pygrib
indx = pygrib.index('./AROME_0.01_SP1_00H_2020021000.grib2', 'typeOfLevel', 'level', 'name')
msg = indx.select(level=2, typeOfLevel="heightAboveGround", name="2 metre temperature")
lat,lon = msg.latlons()
temp2m = msg[0].values
I have FITS file in pixel coordinates. I want to basically read in its latitude and longitude.
I know we can use something following:
from astropy import WCS
w = WCS('sample.fits')
lat, long = w.all_pix2world(90, 38, 1)
But instead of one RA(=90), DEC(=38) value inside
w.all_pix2world
I want to fetch latitudes and longitudes (in Galactic coordinate system) of the entire field of sample.fits.
header
NAXIS = 2
NAXIS1 = 150
NAXIS2 = 150
Please let me know if any other information is needed. Any help is appreciated.
Thanks.
I am trying to find the equivalent (if there exists one) of an NCL function that returns the indices of two-dimensional latitude/longitude arrays closest to a user-specified latitude/longitude coordinate pair.
This is the link to the NCL function that I am hoping there is an equivalent to in python. I'm suspecting at this point that there is not, so any tips on how to get indices from lat/lon coordinates is appreciated
https://www.ncl.ucar.edu/Document/Functions/Contributed/getind_latlon2d.shtml
Right now , I have my coordinate values saved into an .nc file and are read by:
coords='coords.nc'
fh = Dataset(coords, mode='r')
lons = fh.variables['g5_lon_1'][:,:]
lats = fh.variables['g5_lat_0'][:,:]
rot = fh.variables['g5_rot_2'][:,:]
fh.close()
I found scipy spatial.KDTree can perform similar task. Here is my code of finding the model grid that is closest to the observation location
from scipy import spatial
from netCDF4 import Dataset
# read in the one dimensional lat lon info from a dataset
fname = '0k_T_ann_clim.nc'
fid = Dataset(fname, 'r')
lat = fid.variables['lat'][:]
lon = fid.variables['lon'][:]
# make them a meshgrid for later use KDTree
lon2d, lat2d = np.meshgrid(lon, lat)
# zip them together
model_grid = list( zip(np.ravel(lon2d), np.ravel(lat2d)) )
#target point location : 30.5N, 56.1E
target_pts = [30.5 56.1]
distance, index = spatial.KDTree(model_grid).query(target_pts)
# the nearest model location (in lat and lon)
model_loc_coord = [coord for i, coord in enumerate(model_grid) if i==index]
I'm not sure how lon/lat arrays are stored when read in python, so to use the following solution you may need to convert lon/lat to numpy arrays. You can just put the abs(array-target).argmin() in a function.
import numpy as np
# make a dummy longitude array, 0.5 degree resolution.
lon=np.linspace(0.5,360,720)
# find index of nearest longitude to 25.4
ind=abs(lon-25.4).argmin()
# check it works! this gives 25.5
lon[ind]