I am using python 3.6 to map climate model data that has the original longitude of (0,360). I using a basemap function called shiftgrid in order to shift all of the longitude values in my data set to (-180,180). However I am still getting an empty map. Any suggestions would be helpful. Thanks!
Here is my code so far:
#Longitude values:
a=0
b=360
prcp = np.load('data.npy')
data=np.average(prcp,axis=0)
plt.figure()
# create Basemap
x1 = np.linspace(a,b, data.shape[1])
y1 = np.linspace(-90, 90, data.shape[0])
xx1, yy1 = np.meshgrid(x1, y1)
data, x1 = shiftgrid(180., data, x1,start= False)
I've run into these kind of problems myself. My solution was to transform the co-ordinates of the input file using cdo (there is a python front end available for cdo). The shift can be done with:
cdo sellonlatbox,-180,180,-90,90 input.nc output.nc
or, if you have the python front end available, you can get the data directly as netCDF4.Dataset with
from cdo import Cdo
cdo = Cdo()
data = cdo.sellonlatbox(
'-180,180,-90,90',
input = 'input.nc',
)
Hope this helps.
Related
I'm trying to transform some points that are tabulated .csv in a netcdf file.
This is my .csv file: https://1drv.ms/u/s!AhZf0QH5jEVSjWfnPtJjJgmXf-i0?e=WEpMyU
In my spreadsheet, I have the unique location of each point, not regular for all area but points are spaced by 0.1 degree, an SP value per year up to 100 years forward.
To work with this data, I needed something like other sources that use netcdf data tabled in sp(time, lat, lon). So, I can evaluate and visualize the values of this specific region by year (using panoply or ncview for example).
For that, I came up with this code:
import pandas as pd
import xarray as xr
import numpy as np
csv_file = 'example.csv'
df = pd.read_csv(csv_file)
df = pd.melt(df, id_vars=["lon", "lat"], var_name="time", value_name="sp")
df['time']= pd.to_datetime(df['time'])
df = df.set_index(["time", "lat", "lon"])
df = df.astype('float32')
xr = df.to_xarray()
xc = xr.fillna(0)
xc.to_netcdf(csv_file + '.nc')
And I got a netcdf file like this: https://1drv.ms/u/s!AhZf0QH5jEVSjWfnPtJjJgmXf-i0?e=WEpMyU
At first, my code seems to work and create my netcdf file without problems, however, I noticed that in some places I am creating some "leakage" of points, or interpolating the same values in some direction (north-south and west-east) when it shouldn't happen.
If you do a simple plot before converting to xarray you can see there are 3 west segments and one south segment
xr.sp[0].plot()
And this ends up being masked a bit when I fill the NaN with 0 and plot it again:
xc.sp[0].plot()
Checking the netcdf file using panoply I got something similar as well:
So I've start to check every-step of my code to see if I miss something.. my first guess was the melt part but I not 100% sure because if I plot df I can't see any leaking or extrapolation in the same region:
joint_axes = seaborn.jointplot(
x="lon", y="lat", data=df, s=0.5
)
contextily.add_basemap(
joint_axes.ax_joint,
crs="EPSG:4326",
source=contextily.providers.CartoDB.PositronNoLabels,
);
So anyone have any idea what's happening here?
EDIT:
Now a solution that would help me at the moment would be to fill in the missing coordinates with a value equal to 0 within my domain area using the minimum and maximum latitudes and longitudes.
My first (and unconventional) idea was to create a 0.1 x 0.1 grid with values equal to zero and feed this grid with my existing values.
However, the method using reindex would help me and I would be able to execute it in a few lines. My doubt is whether I should do this before or after the df.melt in my code.
I'm in this situation:
csv_file = '/Users/helioguerraneto/Desktop/example.csv'
df = pd.read_csv(csv_file)
lonmin, lonmax = df['lon'].min(), df['lon'].max()
latmin, latmax = df['lat'].min(), df['lat'].max()
df = pd.melt(df, id_vars=["lon", "lat"], var_name="time", value_name="sp")
df['time']= pd.to_datetime(df['time'])
df = df.set_index(["time", "lat", "lon"])
df = df.astype('float32')
xr = df.to_xarray()
xc = xr.reindex(lat=np.arange(latmin, latmax, 0.1), lon=np.arange(lonmin, lonmax, 0.1), fill_value=0)
xc.to_netcdf(csv_file + '.nc')
Seems like reindex is the way but I need to keep original data. I was expecting some zeros but not in all area:
EDIT2:
I think I found something might help! My goal now could be same what's happing here: How to interpolate latitude/longitude and heading in Pandas
But instead of interpolation by the nearest I just could match with the exactly coordinates. Maybe the real problem here is mix 100 hundred grids in the end..
Any suggestions?
I am trying desperately to project some geostationary data from GOES-16 netCDF file to a different projection. I can get the background map to re-project but can't seem to get the data to follow.
I'm not super versed in this yet, but here is what I have thus far:
Reading the data through NetCDF4:
from netCDF4 import Dataset
nc = Dataset('OR_ABI-L1b-RadF-
M3C13_G16_s20182831030383_e20182831041161_c20182831041217.nc')
data = nc.variables['Rad'][:]
Here I'm trying to get the geostationary info:
sat_h = nc.variables['goes_imager_projection'].perspective_point_height
X = nc.variables['x'][:] * sat_h
Y = nc.variables['y'][:] * sat_h
# Satellite longitude
sat_lon =
nc.variables['goes_imager_projection'].longitude_of_projection_origin
# Satellite sweep
sat_sweep = nc.variables['goes_imager_projection'].sweep_angle_axis
Here I'm taking projection data from the .nc file:
proj_var = nc.variables['goes_imager_projection']
sat_height = proj_var.perspective_point_height
central_lon = proj_var.longitude_of_projection_origin
semi_major = proj_var.semi_major_axis
semi_minor = proj_var.semi_minor_axis
print proj_var
<type 'netCDF4._netCDF4.Variable'>
int32 goes_imager_projection()
long_name: GOES-R ABI fixed grid projection
grid_mapping_name: geostationary
perspective_point_height: 35786023.0
semi_major_axis: 6378137.0
semi_minor_axis: 6356752.31414
inverse_flattening: 298.2572221
latitude_of_projection_origin: 0.0
longitude_of_projection_origin: -75.0
sweep_angle_axis: x
unlimited dimensions:
current shape = ()
filling on, default _FillValue of -2147483647 used
And here is a small snippet of my code that's relevant:
fig = plt.figure(figsize=(30,20))
globe = ccrs.Globe(semimajor_axis=semi_major, semiminor_axis=semi_minor)
proj = ccrs.Geostationary(central_longitude=central_lon,
satellite_height=sat_height, globe=globe)
ax = fig.add_subplot(1, 1, 1, projection=proj)
IR_img = ax.imshow(data[:,:],origin='upper',extent=(X.min(), X.max(), Y.min(), Y.max()),
cmap=IR_cmap,interpolation='nearest',vmin=162.,vmax=330.)
And an image of everyone playing nicely:
Data and map working
When I try and get say a Plate Carree projection I try:
proj = ccrs.PlateCarree(central_longitude=central_lon,globe=globe)
And an image of my failure:
Data and map not working
I've tried messing with the extent in the imshow method, I've tried adding a
transform=proj
in the imshow and no luck, it just gets hung up and I have to restart the kernel.
Clearly it is a lack of understanding on my part. If anyone can quickly and easily help/explain the way I want to change my projection from geostationary, I would greatly appreciate that.
I'm running archaic python2.
Thanks for looking.
EDIT: Problem seems to be resolved thanks to insight from DopplerShift and ajdawson, I guess I was maybe a little impatient/ignorant of how long a full disk transformation would take.
It looks like you need to specify the transform keyword to imshow. This keyword tells cartopy what coordinates your data are in, which in this case should be geostationary.
I don't have your dataset so I cannot test this, but the snippet below illustrates the concept. The projection and the transform are independent so you should define both. The value of the transform argument (crs in the example below) is fixed for the data set, but the projection can be anything you like (including the same as crs).
See this example of reprojecting a geostationary image: https://scitools.org.uk/cartopy/docs/v0.16/gallery/geostationary.html#sphx-glr-gallery-geostationary-py. Also see the guide to projection and transform arguments here: https://scitools.org.uk/cartopy/docs/v0.16/tutorials/understanding_transform.html.
globe = ccrs.Globe(semimajor_axis=semi_major, semiminor_axis=semi_minor)
crs = ccrs.Geostationary(central_longitude=central_lon,
satellite_height=sat_height, globe=globe)
proj = ccrs.PlateCarree(central_longitude=central_lon, globe=globe)
ax = fig.add_subplot(1, 1, 1, projection=proj)
IR_img = ax.imshow(data[:,:], origin='upper',
extent=(X.min(), X.max(), Y.min(), Y.max()),
transform=crs,
cmap=IR_cmap,
interpolation='nearest', vmin=162., vmax=330.)
I would like to create plot images from a NetCDF at each time step.
My NetCDF files look like this:
netcdf file:/C:/home/data/cmorph/test/reduced_cmorph_adjusted_spi_pearson_01.nc {
dimensions:
time = UNLIMITED; // (240 currently)
lat = 120;
lon = 360;
variables:
float spi_pearson_01(time=240, lat=120, lon=360);
:_FillValue = NaNf; // float
:valid_min = -3.09; // double
:valid_max = 3.09; // double
:long_name = "Standard Precipitation Index (Pearson Type III distribution), 1-month scale";
:_ChunkSizes = 1, 120, 360; // int
int time(time=240);
:units = "days since 1800-01-01 00:00:00";
:_ChunkSizes = 1024; // int
:_CoordinateAxisType = "Time";
float lat(lat=120);
:units = "degrees_north";
:_CoordinateAxisType = "Lat";
float lon(lon=360);
:units = "degrees_east";
:_CoordinateAxisType = "Lon";
// global attributes:
:title = "CMORPH Version 1.0BETA Version, daily precip from 00Z-24Z";
:history = "Wed Feb 28 07:30:01 2018: C:\\home\\miniconda\\Library\\bin\\ncks.exe --dmn lon,0,,4 --dmn lat,0,,4 CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_1998_2017.nc cmorph_reduced_adjusted.nc";
:NCO = "4.7.1";
:_CoordSysBuilder = "ucar.nc2.dataset.conv.DefaultConvention";
}
I like the plots produced by Panoply but I haven't worked out how to script it (I don't want to go through the GUI for this since I'll have roughly 1500 plots to create). I'm not wedded to Panoply per se, so if someone has a better idea please advise. I could hammer this out in matplotlib but it'd take me quite a while and wouldn't look as good as the Panoply plots. I'm trying to avoid doing much if any of the plotting myself, but maybe there's something out there that provides easy plotting of NetCDFs which can be called from a script (I typically use Python and Bash).
Example using xarray:
import xarray as xr
import matplotlib
import matplotlib.pyplot as plt
matplotlib.use('Agg')
file_name = "reduced_cmorph_adjusted_spi_pearson_01.nc"
with xr.open_dataset(file_name) as ds:
for t in range(ds.time.shape[0]):
da = ds.spi_pearson_01.isel(time=t)
plt.figure()
da.plot()
plt.savefig('frame{}.png'.format(t))
Non-scripting method if you don't mind using a few clicks in Panoply: create a lat/lon plot and then choose File->Export Animation . You can output individual time steps as JPG or PNG.
I'm kind of assuming you don't want to insert 1500 figures in a report or talk and therefore the purpose of this is just to investigate the file slice by slice. If this is the case I would simply open the file using
ncview file.nc
This allows you to step through the slices, animate, pass the cursor over the slices to see the values and click on a point to see a timeseries. If you don't have it, you can install it easily with apt-get (ubuntu, mint etc) with
sudo apt-get install ncview
I'm reasonably new to Python, and I'm trying to plot long-term mean rainfall data for the African continent. I have various NetCDF files, which have already been cut to just contain the long term mean value - I just need to plot it.
My issue is that the data is only plotting to the right of the 0 degree longitude line. I gather this is due to Basemap wanting -180 to 180 coordinates, and my data is 0 to 360. However, nothing I've tried seems to work.
Here's the code (which gives the correct plot, just cut off to the left of 0 degrees):
nc = Dataset(GISS-E2-H_MAM_plots.nc)
prcp = nc.variables['pr'][0,:,:]
pr = 86400*prcp[:]
lon=nc.variables['lon']
lat=nc.variables['lat']
[lonall, latall] = np.meshgrid(lon, lat)
fig = plt.figure()
m = Basemap(projection='cyl', llcrnrlat=-25, urcrnrlat=15, llcrnrlon=-20, urcrnrlon=60)
m.drawcoastlines()
m.drawcountries()
m.drawparallels(np.arange(-90.,90.,10.), labels = [1,0,0,0], fontsize = 10)
m.drawmeridians(np.arange(-180., 180., 10.), labels = [0,0,0,1], fontsize = 10)
levels=np.arange(2, 11.6, 0.8)
mymapf = plt.contourf(lonall, latall, pr, levels, cmap=plt.cm.gist_rainbow_r)
I've tried to shift the data by 180 using the following, and then np.roll to move it all along.
lonall= lonall-180
nlon=len(lonall)
pr=np.roll(pr, nlon/2, axis=1)
This worked for a colleague in a similar instance, but hasn't worked for me.
Any help would be greatly appreciated!
I think the problem is that you don't have [:] after you read in latitude and longitude. I.e. change the above lines to:
lon=nc.variables['lon'][:]
lat=nc.variables['lat'][:]
Also, you don't need the brackets around [lonall,latall]
I have a text file with lots of data that is arranged in 2 columns. I need to use the data in the 2nd column in a formula (which outputs Energy). I need to plot that energy against the time which is all the data in the first column.
So far I have this, and it prints a very weird graph. I know that the energy should be oscillating and decaying exponentially.
import numpy as np
import matplotlib.pyplot as plt
m = 0.090
l = 0.089
g = 9.81
H = np.loadtxt("AngPosition_3p5cmSeparation.txt")
x, y = np.hsplit(H,2)
Ep = m*g*l*(1-np.cos(y))
plt.plot(x, Ep)
plt.show()
I'm struggling to see where I have gone wrong, but then again I am somewhat new to Python. Any help is much appreciated.
I managed to get it to work. My problem was that the angle data had to be converted into radians.
I couldn't do that automatically in Python using math.radians for some reason so I just edited the data in Excel and then back into Notepad.