I am working on gfs weather data to plot certain parameters using python and matplotlib. Since it is a grib2 file, I am using wgrib2 for extracting data from file (even though I am aware of pygrib). These extracted variables (lat, lon and temp) are convered in to numpy arrary successfully. I am attaching these lat, lon and data values for your review. Portion of the code I am using for plotting is:-
lat = # load from attached lat file
lon = # load from attached lon file
data = # load data from data file
plt.figure()
m = Basemap(projection='mill', lat_ts=10, llcrnrlon=lon.min(), urcrnrlon=lon.max(), llcrnrlat=lat.min(), urcrnrlat=lat.max(),
resolution='c')
x, y = m(lat, lon)
cs = m.contourf(x, y, data)
m.drawcoastlines()
m.fillcontinents()
m.drawmapboundary()
plt.show()
Now, when I use matplotlib countorf function for plotting, it produces the following error:-
File "wgrib2.py", line 145, in <module>
cs = m.contourf(x, y, data)
File "/usr/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py", line 521, in with_transform
return plotfunc(self,x,y,data,*args,**kwargs)
File "/usr/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py", line 3644, in contourf
xx = x[x.shape[0]/2,:]
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
Can someone help me to solve this issue?
Here is the minimal example as requested (actual files are uploaded in above link):-
#!/usr/bin/python2
# -*- coding: utf-8 -*-
from __future__ import print_function, unicode_literals, division
import os
import subprocess
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
lat = np.loadtxt("lat.txt")
lon = np.loadtxt("lon.txt")
data = np.loadtxt("data.txt")
plt.figure()
m = Basemap(projection='mill', lat_ts=10, llcrnrlon=lon.min(), urcrnrlon=lon.max(), llcrnrlat=lat.min(), urcrnrlat=lat.max(),
resolution='c')
x, y = m(lat, lon)
cs = m.contourf(x, y, data)
m.drawcoastlines()
m.fillcontinents()
m.drawmapboundary()
plt.show()
Edit 1
Uploaded files again.
It appears to be an issue with dimension of the data. The output generate by wgrib2 to numpy array is a single dimension with value of (259920,) for lat, lon and data. I have checked with pygrib and it produces the shape of (720, 361) which leads to 720 x 361 = 259920. Therefore, it seems that I have an issue with the data being converted in to numpy.
Your code does not produce an error on Python 3.5 with basemap 1.1.0, so this should be fixed if you upgrade your basemap. If you are actually running Python 2 (which is suggested by your minimal example), then it may be sufficient to remove division from your __future__ imports.
In particular, this MCVE does not produce an error:
from __future__ import print_function, unicode_literals, division
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
lat = np.linspace(-90, 90, 180)
lon = np.linspace(-180, 180, 360)
lat, lon = np.meshgrid(lat, lon)
data = np.random.randn(360, 180)
plt.figure()
m = Basemap(projection='mill', lat_ts=10, llcrnrlon=lon.min(), urcrnrlon=lon.max(), llcrnrlat=lat.min(), urcrnrlat=lat.max(),
resolution='c')
x, y = m(lon, lat)
cs = m.contourf(x, y, data)
m.drawcoastlines()
m.fillcontinents()
m.drawmapboundary()
plt.show()
If for some reason that doesn't work and you want to patch the issue erroring right here, you can go into the basemap source code and change xx = x[x.shape[0]/2,:] to xx = x[int(x.shape[0]/2),:]. What I heard from Ben suggests that if you do this, you're likely to continue to run in to other compatibility problems (possibly right after you make that fix), but it might get this code working for now.
Keep in mind that basemap does not support Python 3, and is deprecated. The core developer of Basemap (Ben Root) is urging users to stop using it, because it's been abandoned and will not be ported to Python 3, even when Python 2 stops receiving security updates in 2020. He suggests using cartopy instead. All this was said by Ben Root at Scipy 2017.
Related
I am trying to generate a heatmap from 3D-data in a csv-file. The csv-file has the format x,y,z for each line. The problem is when I create a array to link the values, I can't use float-numbers as keys. When setting the dtype to int in np.loadtext(), the code works fine; but this makes the resolution only half of what the csv-file can replicate. Is there another way of linking the values?
The code so far is:
import numpy as np
import seaborn as sb
import matplotlib.pyplot as plt
fname = 'test18.csv'
x, y, z = np.loadtxt(fname, delimiter=',', dtype=float).T
pltZ = np.zeros((y.max()+1, x.max()+1), dtype=float)
pltZ[y, x] = z
heat_map = sb.heatmap(pltZ, cmap=plt.cm.rainbow)
plt.show()
I want to plot a coloured contour graph with x,y,z from 3 columns of a comma delimited text file, but each time I try the code below, I get ValueError: too many values to unpack (expected 3) error. I would be grateful if that could be resolved.
I would also like to know if there is another (probably better) code for plotting the 3 independent columns.
This is the code:
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
import numpy as np
import scipy.interpolate
N = 100000
long_col, lat_col, Bouguer_col = np.genfromtxt(r'data.txt', unpack=True)
xi = np.linspace(long_col.min(), long_col.max(), N)
yi = np.linspace(lat_col.min(), lat_col.max(), N)
zi = scipy.interpolate.griddata((long_col, lat_col), Bouguer_col, (xi[None,:], yi[:,None]), method='cubic')
fig = plt.figure()
plt.contourf(xi, yi, zi)
plt.xlabel("Long")
plt.ylabel("Lat")
plt.show()
This is the 'data.txt' sample data.
Lat, Long, Elev, ObsGrav, Anomalies
6.671482000000001022e+00,7.372505999999999560e+00,3.612977999999999952e+02,9.780274000000000233e+05,-1.484474523360840976e+02
6.093078000000000216e+00,7.480882000000001142e+00,1.599972999999999956e+02,9.780334000000000233e+05,-1.492942383352201432e+02
6.092045999999999850e+00,7.278669999999999973e+00,1.462445999999999913e+02,9.780663000000000466e+05,-1.190960417173337191e+02
6.402087429999999912e+00,7.393360939999999992e+00,5.237939999999999827e+02,9.780468000000000466e+05,-8.033459449396468699e+01
6.264082730000000154e+00,7.518244540000000420e+00,2.990849999999999795e+02,9.780529000000000233e+05,-1.114865156192099676e+02
6.092975000000000030e+00,7.482914000000000065e+00,1.416474000000000046e+02,9.780338000000000466e+05,-1.525697779102483764e+02
6.383570999999999884e+00,7.289616999999999791e+00,2.590403000000000020e+02,9.780963000000000466e+05,-8.300666170357726514e+01
6.318417000000000172e+00,7.557638000000000744e+00,1.672036999999999978e+02,9.780693000000000466e+05,-1.246774551668204367e+02
6.253779999999999895e+00,7.268805999999999656e+00,1.059429999999999978e+02,9.781026999999999534e+05,-9.986763240839354694e+01
6.384635000000000282e+00,7.291032000000000401e+00,2.615624000000000251e+02,9.780963000000000466e+05,-8.256190758384764194e+01
If the data file looks exactly like in the question you first of all have 5 columns, which you cannot unpack to 3 variables.
Next, you have a header line which you do not want to be part of the data. Also the header line is separated by ,<space>, while the data is separated by ,.
So in total you need
import numpy as np
a,b,c,d,e = np.genfromtxt("data.txt", unpack=True, delimiter=",", skip_header=1)
I have run into problems when using Basemap.contour with certain projections. Based on the example given in the Basemap documentation, I created the following working code which produces the expected result. The example uses the 'tmerc' projection.
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np
m2 = Basemap(projection='tmerc',
lat_0=0, lon_0=3,
llcrnrlon=1.819757266426611,
llcrnrlat=41.583851612359275,
urcrnrlon=1.841589961763497,
urcrnrlat=41.598674173123)
##m2 = Basemap(projection='kav7',lon_0=0)
x = np.linspace(0, m2.urcrnrx, 100)
y = np.linspace(0, m2.urcrnry, 100)
xx, yy = np.meshgrid(x, y)
data = np.sin(xx/100)*np.cos(yy/100)
levels = np.linspace(-1,1,8)
m2.contour(xx, yy, data, levels)
plt.show()
However, if I switch to using the 'kav7' projection in the alternative m2=Basemap declaration (commented out in the example code), the code fails with the following error:
Traceback (most recent call last):
File "basemap_contour.py", line 20, in <module>
m2.contour(xx, yy, data, levels)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/mpl_toolkits/basemap/__init__.py", line 521, in with_transform
return plotfunc(self,x,y,data,*args,**kwargs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/mpl_toolkits/basemap/__init__.py", line 3542, in contour
xx = x[x.shape[0]/2,:]
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
Note that this also happens when I define lon and lat values 'properly', the example was only chosen to be as short as possible. Does anybody know how to resolve this?
EDIT:
In case this is relevant, I'm using python version 3.5.3 on an osx Sierra machine. The matplotlib version is 2.0.0 and the basemap version is 1.0.7 .
This behavior is according to python3 integer division. Look for examples:
1) python3:
n=100
print (n/2, (n+1)/2)
Output: 50.0 50.5
2) For python 2.7 this code returns 50 50
Solutions:
1) manually update contour and contourf function of basemap with division for python3.
You have to write for integer n: n//2 which is apply division from python2.
2) or run your program with python2.
I found a really simple workaround to this problem. Instead of calling Basemap.contour, one can call contour directly on the Axes instance of the Basemap:
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np
fig,ax = plt.subplots()
m2 = Basemap(projection='kav7',lon_0=0, ax=ax)
x = np.linspace(0, m2.urcrnrx, 100)
y = np.linspace(0, m2.urcrnry, 100)
xx, yy = np.meshgrid(x, y)
lon,lat = m2(xx,yy, inverse = True)
data = np.sin(np.pi*lon/180)*np.cos(np.pi*lat/90)
m2.drawcoastlines(linewidth=0.5)
levels = np.linspace(-1,1,8)
##m2.contour(xx, yy, data, levels)
ax.contour(xx,yy,data,levels)
plt.show()
This produces the following picture both under Python 2.7 and 3.6:
This bug has been fixed 2 years ago and does not occur in any basemap version >=1.1.0, independent of the use of python 2 or 3.
Does anyone know why memory use keeps increasing? An idealised case is below, I can't see why.
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np
Lon = np.linspace(-180,180,1440)
Lat = np.linspace(-90,90,721)
Lon,Lat = np.meshgrid(Lon,Lat)
m = Basemap()
X, Y = m(Lon, Lat)
matrix = np.random.rand(721,1440)
for i in range(0,100):
cs = m.contourf(X,Y,matrix)
plt.clf()
plt.close()
print i
May of been a memory leak - cannot replicate problem with the latest Matplotlib library.
I am trying to plot a contour plot with Python's Matplotlib package. I'm trying to get similar results to what can be seen in this other stack overflow post. However, I'm getting the problem of it saying that there is a type error and it tells me TypeError: Invalid dimensions for image data, which can be seen in the full error code below.
Traceback (most recent call last):
File "./plot_3.py", line 68, in <module>
plt.imshow(zi, vmin=temp.min(), vmax=temp.max(), origin="lower", extent=[x.min(), x.max(), y.min(), y.max()])
File "/usr/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 3022, in imshow
**kwargs)
File "/usr/lib64/python2.7/site-packages/matplotlib/__init__.py", line 1812, in inner
return func(ax, *args, **kwargs)
File "/usr/lib64/python2.7/site-packages/matplotlib/axes/_axes.py", line 4947, in imshow
im.set_data(X)
File "/usr/lib64/python2.7/site-packages/matplotlib/image.py", line 453, in set_data
raise TypeError("Invalid dimensions for image data")
TypeError: Invalid dimensions for image data
I'm unsure of what this means, as googling brought up no useful results on how to fix it. The code is listed below, and the data that I'm using can be found here. The code below simply runs the code which will parse the file and then return the data to the main where it's supposed to plot it then. To run the code, you have to use ./plot_3.py 20.0 to use it with the specific file that I posted above. x ranges from 0 to 0.3 with 61 grids, while y ranges from 0 to 0.4 with 81 grids. The data is in the format x,y,temperature where I want the temperature values to be the contour values.
from __future__ import print_function, division
import math
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import sys
import matplotlib.cm as cm
from matplotlib.mlab import griddata
import scipy.interpolate
def ParseFile(filename):
x = []
y = []
temp = []
infile = open(filename, 'r')
lines = [line.strip() for line in infile.readlines()]
for line in lines:
x.append(float(line.split(',')[0]))
y.append(float(line.split(',')[1]))
temp.append(float(line.split(',')[2]))
return np.array(x), np.array(y), np.array(temp)
time = str(sys.argv[1])
filename = time + "_sec.dat"
x,y,temp = ParseFile(filename)
xi = np.linspace(min(x), max(x))
yi = np.linspace(min(y), max(y))
zi = scipy.interpolate.griddata((x,y),temp,(xi,yi),method="linear")
matplotlib.rcParams['xtick.direction'] = 'out'
matplotlib.rcParams['ytick.direction'] = 'out'
plt.imshow(zi, vmin=temp.min(), vmax=temp.max(), origin="lower",
extent=[x.min(), x.max(), y.min(), y.max()])
plt.colorbar()
plt.show()
I think the problem is that that you need to have the points to be interpolated in a gridded format, not two 1D matrices for the interpolate.griddata function.
Adding this line to the (xi, yi) declaration I think fixes your problem:
x,y,temp = ParseFile(filename)
xi = np.linspace(min(x), max(x))
yi = np.linspace(min(y), max(y))
#create the 2D grid for interpolation:
xi, yi = np.meshgrid(xi,yi)
zi = scipy.interpolate.griddata((x,y),temp,(xi,yi),method="linear")