We Keep Coding

Resampling and re-projecting weather satellite image

The current question is somewhat similar to a resampling question on:
Resample question on Stackoverflow
However, my specific problem is that I only have a partial satellite image and not the full image. As a result, I'm not sure how to proceed. Here's what I've tried so far:
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
def transform_extent_pts(extent_pts, map_proj, pt_crs):
xul, yul = map_proj.transform_point(
x = extent_pts[0],
y = extent_pts[3],
src_crs = pt_crs)
xlr, ylr = map_proj.transform_point(
x = extent_pts[1],
y = extent_pts[2],
src_crs = pt_crs)
return [xul, xlr, ylr, yul]
sat_image1 = ROOT
df = plt.imread(sat_image1)
# re-project to Mercator
map_proj = ccrs.Mercator()
# Image extent in Geostationary coordinates:
data_crs = ccrs.Geostationary(central_longitude=0.0)
ax2 = plt.axes(projection=data_crs)
img_extent_sat = ax2.get_extent(crs=data_crs)
img_extent_sat = [1.03*x for x in img_extent_sat]
#img_extent_sat=[-32.150361957, 30.150361957, 7.150361956, 42.150361956]
# Convert to Mercator
img_extent_merc = transform_extent_pts(img_extent_sat, map_proj, ccrs.Geodetic())
plt.close()
fig = plt.figure(figsize=(10,10))
ax = plt.axes(projection=map_proj)
ax.coastlines(color='blue')
ax.gridlines(color='black', alpha=0.5, linestyle='--', linewidth=0.75, draw_labels=True)
# Map extent in degrees (PlateCarree) coordinates:
map_extent_deg = (50., -20., -40., 40.) # African continent
map_extent_deg = (-31, 38.009232, 2.880476, 42)
# Convert to Mercator
map_extent_merc = transform_extent_pts(map_extent_deg, map_proj, ccrs.Geodetic())
ax.set_extent(map_extent_merc, map_proj)
plt.imshow(df, origin='upper', transform=data_crs, extent=img_extent_sat)
The main issue I'm facing is that the projection of my data seems to be incorrect, and there might also be a small misalignment. I'm now wondering how I can change the projection of my data. Could you provide some guidance on how to accomplish this?
I've tried a few things, but unfortunately the image is only available in PNG format. The "transform" function, such as using "geostationary," doesn't seem to work with this format. Are there any other options available? Can the projection be changed afterwards, or is that not possible? I assume that the exact coordinates can be obtained by shifting the image.
Perhaps the two red dots can be helpful. I know their coordinates, and they should align with the two white squares:
plt.plot(sta_lon, sta_lat, marker='o', color='red', markersize=8,
alpha=0.7, transform=ccrs.Geodetic())
plt.text(sta_lon, sta_lat+0.25, sta_name, ha='center', fontsize=18,
color='red', transform=ccrs.Geodetic())

Raster and Shapefiles not lining up using Geopandas, Rasterio, and Contextily

I am trying to get a DEM raster to line up with a shapefile in Python, but it will not show up no matter what I do. This is for lab exercise, the entire rest of the exercise relies on these lining up, as I will be extracting data from the raster and polygon layers to a point layer.
I know how to do all this "by hand" in ArcGIS, but the point of the exercise is to use R or Python (the professor did an example with R, but we can use whichever, and I have been learning Python the past couple of months for a work project). In the class notes, he says that both files are in EPSG 3847, but the shapefile was missing the CRS, so I added the CRS to it in geopandas.
The DEM appears to be EPSG 3006 (even though it was supposed to be in 3847), so I tried converting it to EPSG 3847 and it still does not show up. So then I tried going the other way and converting the shapefile to EPSG 3006, which did not help either.
import contextily as cx
import geopandas as gpd
import rasterio
from rasterio.plot import show
from rasterio.crs import CRS
from rasterio.plot import show as rioshow
import matplotlib.pyplot as plt
#data files
abisveg = gpd.read_file(r'/content/drive/MyDrive/Stackoverflow/Sweden/abisveg_polygon.shp')
abisveg_3847 = abisveg.set_crs(epsg = 3847)
abisveg_3006 = abisveg_3847.to_crs(epsg = 3006)
src = rasterio.open(r'/content/drive/MyDrive/Stackoverflow/Sweden/nh_75_6.tif')
DEM = src.read()
### creating plot grid
fig = plt.figure(figsize = (20,20), constrained_layout = True)
gs = fig.add_gridspec(1,3)
ax1 = fig.add_subplot(gs[0,0])
ax2 = fig.add_subplot(gs[0,1], sharex = ax1, sharey = ax1)
ax3 = fig.add_subplot(gs[0,2], sharex = ax1, sharey = ax1)
### Plot 1 - Basemap Only
abisveg_3006.plot(ax = ax1, color = 'none')
cx.add_basemap(ax1, crs = 3006)
ax1.set_aspect('equal')
ax1.set_title("Basemap of AOI")
### Plot 2 - DEM
# abisveg_3847.plot(ax = ax2, color = 'none')
show(DEM, ax=ax2, cmap = "Greys")
cx.add_basemap(ax2, crs = 3006)
ax2.set_aspect('equal')
ax2.set_title('Digitial Elevation Model of AOI')
### Plot 3 - Vegetation Types
abisveg_3006.plot(ax = ax3, column = "VEGKOD", cmap = "viridis")
cx.add_basemap(ax3, crs = 3006)
ax3.set_aspect('equal')
ax3.set_title("Vegetation Types")
3 Panel map with missing DEM:
https://i.imgur.com/taG2U9Q.jpg
Trying to plot the files in Matplotlib has not worked, b/c they do not align at all. I am using contextily for the basemap, and have set the basemap CRS to EPSG 3847 (or 3006, depending on which version of the GIS files I was using). The shapefile shows up in the correct location no matter the projection, but the Raster does not show up. What's weird is that if I open everything up in ArcGIS, it all lines up correctly.
If I plot just the DEM all by itself, it shows up, though I don't know where on the earth it is plotting.
fig = plt.figure(figsize = (10,10), constrained_layout = True)
show(DEM, cmap = "Greys")
DEM just by itself:
https://i.imgur.com/KyYu7jc.jpg
I have my code in a colab notebook here:
https://colab.research.google.com/drive/1VAZ3dgf0QS2PPBOl8KJ2FXtB2oRj0qJ8?usp=share_link
The files are here:
https://drive.google.com/drive/folders/1t-xvpIcLOIR9uYXOguJ7KyKqt7wuYSNc?usp=share_link

You could give EOmaps a try... it uses matplotlib/cartopy for plotting and handles re-projecting the data and shapes to the plot-crs
from pathlib import Path
from eomaps import Maps
import geopandas as gpd
p = Path(r"path to the data folder")
# read shapefile
abisveg = gpd.read_file(p / 'abisveg_polygon.shp').set_crs(epsg = 3847)
# create a map in epsg=3006
m = Maps(crs=3006, figsize=(10, 8))
# add stamen-terrain basemap
m.add_wms.OpenStreetMap.add_layer.stamen_terrain()
# plot shapefile (zorder=2 to be on top of the DEM)
m.add_gdf(abisveg, column=abisveg.VEGKOD, cmap="viridis", ec="k", lw=0.2, alpha=0.5, zorder=2)
# plot DEM
m2 = m.new_layer_from_file.GeoTIFF(p / "nh_75_6.tif", cmap="Greys", zorder=1)
m.ax.set_extent((589913.0408156103, 713614.6619114348, 7495264.310799116, 7618965.93189494),
Maps.CRS.epsg(3006))

How to adjust Matplotlib colorbar range in xarray plot?

I have a plot that looks like this
I cannot understand how to manually change or set the range of data values for the colorbar. I would like to experiment with ranges based on the data values shown in the plots and change the colorbar to (-4,4). I see that plt.clim, vmin and vmax are functions to possibly use.
Here is my code:
import cdsapi
import xarray as xr
import matplotlib.pyplot as plt
import numpy as np
import cartopy.crs as ccrs
# Also requires cfgrib library.
c = cdsapi.Client()
url = c.retrieve(
'reanalysis-era5-single-levels-monthly-means',
{
'product_type': 'monthly_averaged_reanalysis',
'format': 'grib',
'variable': ['100m_u_component_of_wind','100m_v_component_of_wind'],
'year': ['2006','2007','2008','2009','2010','2011','2012','2013','2014','2015','2016','2017','2018','2019','2020','2021'],
'month': ['01','02','03','04','05','06','07','08','09','10','11','12'],
'time': '00:00',
'grid': [0.25, 0.25],
'area': [70.00, -180.00, -40.00, 180.00],
},
"C:\\Users\\U321103\\.spyder-py3\\ERA5_MAPPING\\100m_wind_U_V.grib")
path = "C:\\Users\\U321103\\.spyder-py3\\ERA5_MAPPING\\100m_wind_U_V.grib"
ds = xr.load_dataset(path, engine='cfgrib')
wind_abs = np.sqrt(ds.u100**2 + ds.v100**2)
monthly_means = wind_abs.mean(dim='time')
wind_abs_clim = wind_abs.sel(time=slice('2006-01','2020-12')).groupby('time.month').mean(dim='time') # select averaging period
wind_abs_anom = ((wind_abs.groupby('time.month') / wind_abs_clim))-1 #deviation from climo
fg = wind_abs_anom.sel(time=slice('2021-01',None)).groupby('time.month').mean(dim='time').plot(col='month',
col_wrap=3,transform=ccrs.PlateCarree(),
cbar_kwargs={'orientation':'horizontal','shrink':0.6, 'aspect':40,'label':'Percent Deviation'},robust=False,subplot_kws={'projection': ccrs.Mercator()})
fg.map(lambda: plt.gca().coastlines())

I was able to reproduce your figure and found that I could add vmin and vmax as shown below. For some reason that meant I also had to specify the colormap, otherwise I ended up with viridis. But the code below works for me (with a bit of refactoring as I got it working — the only material change here is in the plotting section at the bottom).
First, loading the data:
import cdsapi
c = cdsapi.Client()
params = {
'product_type': 'monthly_averaged_reanalysis',
'format': 'grib',
'variable': ['100m_u_component_of_wind', '100m_v_component_of_wind'],
'year': [f'{n}' for n in range(2006, 2022)],
'month': [f'{n:02d}' for n in range(1, 13)],
'time': '00:00',
'grid': [0.25, 0.25],
'area': [70.00, -180.00, -40.00, 180.00],
}
path = '100m_wind_U_V.grib'
url = c.retrieve('reanalysis-era5-single-levels-monthly-means',
params,
path,
)
Then there's the data pipeline:
import xarray as xr
import numpy as np
# Also need cfgrib library.
ds = xr.load_dataset(path, engine='cfgrib')
wind_abs = np.sqrt(ds.u100**2 + ds.v100**2)
monthly_means = wind_abs.mean(dim='time')
wind_abs_clim = (wind_abs.sel(time=slice('2006-01','2020-12'))
.groupby('time.month')
.mean(dim='time'))
wind_abs_anom = ((wind_abs.groupby('time.month') / wind_abs_clim)) - 1
Finally the plotting:
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
cbar_kwargs = {'orientation':'horizontal', 'shrink':0.6, 'aspect':40, 'label':'Percent Deviation'}
subplot_kws = {'projection': ccrs.Mercator()}
fg = (wind_abs_anom.sel(time=slice('2021-01', None))
.groupby('time.month')
.mean(dim='time')
.plot(col='month',
col_wrap=3,
transform=ccrs.PlateCarree(),
cmap='RdBu_r', vmin=-3, vmax=3, # <-- New bit.
cbar_kwargs=cbar_kwargs,
robust=False,
subplot_kws=subplot_kws
))
fg.map(lambda: plt.gca().coastlines())
Sometimes I'll use a percentile to control the values for vmin and vmax automatically, like max_ = np.percentile(data, 99), then vmin=-max_, vmax=max_. This deals nicely with outliers that stretch the colormap, but it requires you to be able to calculate those values before making the plot.
If you want to start having more control over the plot, it might be a good idea to stop using the xarray plotting interface and use matplotlib and cartopy directly. Here's what that might look like (replacing all of the plotting code above):
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
sel = wind_abs_anom.sel(time=slice('2021-01', None))
left, *_, right = wind_abs_anom.longitude
top, *_, bottom = wind_abs_anom.latitude # Min and max latitude.
extent = [left, right, bottom, top]
fig, axs = plt.subplots(nrows=2, ncols=3,
figsize=(15, 6),
subplot_kw={'projection': ccrs.PlateCarree()},
)
for ax, (month, group) in zip(axs.flat, sel.groupby('time.month')):
mean = group.mean(dim='time')
im = ax.imshow(mean,
transform=ccrs.PlateCarree(),
extent=extent,
cmap='RdBu_r', vmin=-3, vmax=3)
ax.set_title(f'month = {month}')
ax.coastlines()
cbar_ax = fig.add_axes([0.2, 0.0, 0.6, 0.05]) # Left, bottom, width, height.
cbar = fig.colorbar(im, cax=cbar_ax, extend='both', orientation='horizontal')
cbar.set_label('Percent deviation')
plt.show()
For some reason, when I try to use ccra.Mercator() for the map, the data gets distorted; maybe you can figure that bit out.

How do I get the transformed data of a cartopy geodetic plot?

how do I get all the data of the transformed line of the "handle" - Line2D object in the following code:
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
ax = plt.axes(projection=ccrs.PlateCarree())
ax.stock_img()
ny_lon, ny_lat = -75, 43
delhi_lon, delhi_lat = 77.23, 28.61
handle = plt.plot([ny_lon, delhi_lon], [ny_lat, delhi_lat],
color='blue', linewidth=2, marker='o',
transform=ccrs.Geodetic(),
)
plt.show()
To be more clear:
I'm not looking for the output of "handle[0].get_data()", since this just prints my original longitude and latitude, but im looking for the the data of the geodetic line drawn on the map.

I found the answer!
According to this question, you can access the data of the transformation via the following code snippet:
[handle] = plt.plot([ny_lon, delhi_lon], [ny_lat, delhi_lat], color='blue', linewidth=2, marker='o', transform=ccrs.Geodetic())
t_path = handle._get_transformed_path()
path_in_data_coords, _ = t_path.get_transformed_path_and_affine()
print(path_in_data_coords.vertices)
In the answer to this question there is also a second approach.

Let me do some computation and plot checks on the code provided by the OP.
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
ax = plt.axes(projection=ccrs.PlateCarree())
ax.stock_img()
ny_lon, ny_lat = -75, 43
delhi_lon, delhi_lat = 77.23, 28.61
# Plot geodetic path in thick 'blue' line
handle = plt.plot([ny_lon, delhi_lon], [ny_lat, delhi_lat],
color='blue', linewidth=10, marker='o',
transform=ccrs.Geodetic(),
)
# Get the geodetic path's coordinates to plot on top in 'red'
t_path = handle[0]._get_transformed_path()
path_in_data_coords, _ = t_path.get_transformed_path_and_affine()
ax.plot(path_in_data_coords.vertices[:,0],
path_in_data_coords.vertices[:,1],
color='red', lw=2)
plt.show()
And, the output plot is:
Congratulations to the OP.
(Extension part 1)
Now, let us compute the length of the geodesic path using the coordinates obtained above. My proposed code is:
# (*** Continued from the code above ***)
import cartopy.geodesic as geodesic
import numpy as np
# defining the earth shape on which to make calculations
myGeod = geodesic.Geodesic(6378137.0, 1/298.257223563)
# get (lat,long) lists from (long,lat) of the geodesic path
latlonlists = []
[latlonlists.append([lat,lon]) for lon,lat in zip(path_in_data_coords.vertices[:,0], path_in_data_coords.vertices[:,1])]
#print(latlonlists)
# compute length of the geodesic
geodesic_in_meters = myGeod.geometry_length(np.array(latlonlists))
print(geodesic_in_meters) # output: 17554975.077432975

Plot polylines on top of OSMnx map

Using the OSMnx library, I'm trying to draw lines as polygons on top of a base map (with per-defined coordinates not adhering to the underlying network), but with no luck. I'm certain that the coordinates I have are inside of the boundary, and I get no error when adding them.
Here's my current code, which generates the base map and also adds a multi polygon layer below the network. So it's possible to add polygons, which makes me think there might a projection issue with my coordinates, but I haven't had any luck setting different projections.
Any help would be much appreciated!
import matplotlib.pyplot as plt
from descartes import PolygonPatch
from shapely.geometry import Polygon, MultiPolygon
import osmnx as ox
ox.config(log_console=True, use_cache=True)
ox.__version__
def plot(geometries):
# get the place shape
gdf = ox.gdf_from_place('Copenhagen Municipality,Denmark')
gdf = ox.project_gdf(gdf)
# get the street network, with retain_all=True to retain all the disconnected islands' networks
G = ox.graph_from_place('Copenhagen Municipality,Denmark', network_type='drive', retain_all=True)
G = ox.project_graph(G)
fig, ax = ox.plot_graph(G, fig_height=10, show=False, close=False, edge_color='#777777')
# Add shape from gdf
for geometry in gdf['geometry'].tolist():
if isinstance(geometry, (Polygon, MultiPolygon)):
if isinstance(geometry, Polygon):
geometry = MultiPolygon([geometry])
for polygon in geometry:
patch = PolygonPatch(polygon, fc='#cccccc', ec='k', linewidth=3, alpha=0.1, zorder=-1)
ax.add_patch(patch)
# Add lines:
for geometry in geometries:
if isinstance(geometry, (Polygon, MultiPolygon)):
if isinstance(geometry, Polygon):
geometry = MultiPolygon([geometry])
for polygon in geometry:
patch = PolygonPatch(polygon, fc='#148024', ec='#777777', linewidth=10, alpha=1, zorder=2)
ax.add_patch(patch)
plt.savefig('images/cph.png', alpha=True, dpi=300)
plot(geometries)
geometries is a list which contains polygons like these:
POLYGON ((55.6938796 12.5584122, 55.6929711 12.5585957, 55.6921317 12.5579927, 55.6916918 12.5564539, 55.6909246 12.5553629, 55.6901215 12.554119, 55.6891181 12.5531433, 55.6881469 12.5526575, 55.687502 12.5538862, 55.6866445 12.5530816, 55.6856769 12.5524416, 55.6848185 12.5515929, 55.6838506 12.551074, 55.6829915 12.5504047, 55.6821492 12.5498124, 55.6812104 12.5492503, 55.680311 12.5486803, 55.6792187 12.547724, 55.6783172 12.5472156, 55.6774282 12.5466767, 55.6765291 12.5461124, 55.6755652 12.5453961, 55.6747743 12.5445313, 55.6738159 12.5439029, 55.673417 12.5454132, 55.6733398 12.5470051, 55.6731045 12.5486561, 55.6726013 12.5501493, 55.6727833 12.5520672, 55.6716717 12.5525378, 55.6706619 12.5528382, 55.6698239 12.5521737))
POLYGON ((55.6693768 12.5509383, 55.6684025 12.5511539, 55.6677405 12.5500371, 55.6668188 12.5501435, 55.6658323 12.550075, 55.665264 12.5487917, 55.6649187 12.5473085, 55.6645313 12.5457653))

In geopandas dataframes, the geo-coordinates are (longitude, latitude). Here is a simple demonstration code that plots some sample data.
import geopandas as gpd
import pandas as pd
import matplotlib.pyplot as plt
import osmnx as ox
from shapely import wkt #need wkt.loads
# simple plot of OSM data
gdf = ox.gdf_from_place('Copenhagen Municipality,Denmark')
gdf = gpd.GeoDataFrame(gdf, crs={'init': 'epsg:4326'}) #set CRS
ax1 = gdf.plot(color='lightgray') # grab axis as `ax1` for reuse
# prep the polygons to plot on the axis `ax1`
# use (longitude latitude), and the last point must equal the 1st
pgon1 = "POLYGON((12.5584122 55.6938796, 12.5585957 55.6929711, 12.5579927 55.6921317, 12.5564539 55.6916918, 12.5553629 55.6909246, 12.554119 55.6901215, 12.5531433 55.6891181, 12.5526575 55.6881469, 12.5538862 55.687502, 12.5530816 55.6866445, 12.5524416 55.6856769, 12.5515929 55.6848185, 12.551074 55.6838506, 12.5504047 55.6829915, 12.5498124 55.6821492, 12.5492503 55.6812104, 12.5486803 55.680311, 12.547724 55.6792187, 12.5472156 55.6783172, 12.5466767 55.6774282, 12.5461124 55.6765291, 12.5453961 55.6755652, 12.5445313 55.6747743, 12.5439029 55.6738159, 12.5454132 55.673417, 12.5470051 55.6733398, 12.5486561 55.6731045, 12.5501493 55.6726013, 12.5520672 55.6727833, 12.5525378 55.6716717, 12.5528382 55.6706619, 12.5521737 55.6698239, 12.5584122 55.6938796))"
pgon2 = "POLYGON((12.5509383 55.6693768, 12.5511539 55.6684025, 12.5500371 55.6677405, 12.5501435 55.6668188, 12.550075 55.6658323, 12.5487917 55.665264, 12.5473085 55.6649187, 12.5457653 55.6645313, 12.5509383 55.6693768))"
# create dataframe of the 2 polygons
d = {'col1': [1, 2], 'wkt': [pgon1, pgon2]}
df = pd.DataFrame( data=d )
# make geo-dataframe from it
geometry = [wkt.loads(pgon) for pgon in df.wkt]
gdf2 = gpd.GeoDataFrame(df, \
crs={'init': 'epsg:4326'}, \
geometry=geometry)
# plot it as red polygons
gdf2.plot(ax=ax1, color='red', zorder=5)
The output plot: