We Keep Coding

3D plot of Excel data

I'm trying to recreate this plot using some of my own excel data but I've hit a wall. So far I have:
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
df = pd.read_excel(r'/path/to/data.xlsx')
yr = df['Year']
jd = df['Jday']
dc = df['Discharge']
x = np.asarray(yr)
y = np.asarray(jd)
z = np.asarray(dc)
X,Y,Z = np.meshgrid(x,y,z)
ax = plt.figure().add_subplot(projection='3d')
ax.plot_surface(X,Y,Z, cmap='autumn')
ax.set_xlabel("Year")
ax.set_ylabel("Jday")
ax.set_zlabel("Discharge")
plt.show()
But when I run this I get:
Traceback (most recent call last):
File "/Users/Desktop/main.py", line 19, in <module>
ax.plot_surface(X,Y,Z, cmap='autumn')
File "/Users/venv/lib/python3.10/site-packages/matplotlib/_api/deprecation.py", line 412, in wrapper
return func(*inner_args, **inner_kwargs)
File "/Users/venv/lib/python3.10/site-packages/mpl_toolkits/mplot3d/axes3d.py", line 1581, in plot_surface
raise ValueError("Argument Z must be 2-dimensional.")
ValueError: Argument Z must be 2-dimensional.
Any help would be appreciated.
EDIT:
I changed my code to:
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
df = pd.read_excel(r'/path/to/data.xlsx')
yr = df['Year']
jd = df['Jday']
dc = df['Discharge']
X = np.asarray(yr).reshape(-1,2)
Y = np.asarray(jd).reshape(-1,2)
Z = np.asarray(dc).reshape(-1,2)
fig = plt.figure(figsize=(14,8))
ax = plt.axes(projection='3d')
my_cmap = plt.get_cmap('seismic')
surf = ax.plot_surface(X,Y,Z,
cmap = my_cmap,
edgecolor = 'none')
fig.colorbar(surf, ax=ax,
shrink = 0.5, aspect = 5)
plt.show()
When I run this it produces the following plot:
Which obviously doesn't match the other plot. It seems to be plotting the data from each year in a single line instead of creating filled in polygons which is what I think it's supposed to do. I have a feeling this issue has to do with the .reshape function but I'm not entirely sure.

Note: original answer completely rewritten!
The problem is, as your data stated, that the Z-argument must be two-dimensional. In your problem, you don't need np.meshgrid at all. This is typically used to make a 'grid' of all possible combinations of X/Y, after which you can use these combinations to calculate your response matrix Z. However, since all your data is read in, it is merely a reshaping of all 1d-arrays to 2d-arrays:
target_shape = (np.sqrt(X.shape[0]),-1)
X = np.reshape(X, target_shape)
Y = np.reshape(Y, target_shape)
Z = np.reshape(Z, target_shape)
Have a look at the documentation of np.reshape for some more information.

Simple plotting of log function in python

I wrote a simple function to plot log in python:
import matplotlib.pyplot as plt
import numpy as np
x = list(range(1, 10000, 1))
y = [-np.log(p/10000) for p in x]
plt.scatter(x, y) # also tried with plt.plot(x, y)
plt.show()
I just want to see how the plot looks.
fn.py:5: RuntimeWarning: divide by zero encountered in log
y = [-np.log(p/10000) for p in x]
I get the above error and on top of that I get a blank plot with even the ranges wrong.
It is strange why there is divide by zero warning, when I am dividing by a number?
How can I correctly plot the function?

Although you have tagged python-3.x, it seems that you are using python-2.x where p/10000 will result in 0 for values of p < 10000 because the division operator / performs integer division in python-2.x. If that is the case, you can explicitly use 10000.0 instead of 10000 to avoid that and get a float division.
Using .0 is not needed in python 3+ because by default it performs float division. Hence, your code works fine in python 3.6.5 though
import matplotlib.pyplot as plt
import numpy as np
x = list(range(1, 10000, 1))
y = [-np.log(p/10000.0) for p in x]
plt.scatter(x, y)
plt.show()
On a different note: You can simply use NumPy's arange to generate x and avoid the list completely and use vectorized operation.
x = np.arange(1, 10000)
y = -np.log(x/10000.0)

Why import numpy and then avoid using it? You could have simply done:
from math import log
import matplotlib.pyplot as plt
x = xrange(1, 10000)
y = [-log(p / 10000.0) for p in x]
plt.scatter(x, y)
plt.show()
If you're going to bring numpy into the picture, think about doing things in a numpy-like fashion:
import matplotlib.pyplot as plt
import numpy as np
f = lambda p: -np.log(p / 10000.0)
x = np.arange(1, 10000)
plt.scatter(x, f(x))
plt.show()

Valid numpy arrays produces IndexError:

I am working on gfs weather data to plot certain parameters using python and matplotlib. Since it is a grib2 file, I am using wgrib2 for extracting data from file (even though I am aware of pygrib). These extracted variables (lat, lon and temp) are convered in to numpy arrary successfully. I am attaching these lat, lon and data values for your review. Portion of the code I am using for plotting is:-
lat = # load from attached lat file
lon = # load from attached lon file
data = # load data from data file
plt.figure()
m = Basemap(projection='mill', lat_ts=10, llcrnrlon=lon.min(), urcrnrlon=lon.max(), llcrnrlat=lat.min(), urcrnrlat=lat.max(),
resolution='c')
x, y = m(lat, lon)
cs = m.contourf(x, y, data)
m.drawcoastlines()
m.fillcontinents()
m.drawmapboundary()
plt.show()
Now, when I use matplotlib countorf function for plotting, it produces the following error:-
File "wgrib2.py", line 145, in <module>
cs = m.contourf(x, y, data)
File "/usr/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py", line 521, in with_transform
return plotfunc(self,x,y,data,*args,**kwargs)
File "/usr/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py", line 3644, in contourf
xx = x[x.shape[0]/2,:]
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
Can someone help me to solve this issue?
Here is the minimal example as requested (actual files are uploaded in above link):-
#!/usr/bin/python2
# -*- coding: utf-8 -*-
from __future__ import print_function, unicode_literals, division
import os
import subprocess
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
lat = np.loadtxt("lat.txt")
lon = np.loadtxt("lon.txt")
data = np.loadtxt("data.txt")
plt.figure()
m = Basemap(projection='mill', lat_ts=10, llcrnrlon=lon.min(), urcrnrlon=lon.max(), llcrnrlat=lat.min(), urcrnrlat=lat.max(),
resolution='c')
x, y = m(lat, lon)
cs = m.contourf(x, y, data)
m.drawcoastlines()
m.fillcontinents()
m.drawmapboundary()
plt.show()
Edit 1
Uploaded files again.
It appears to be an issue with dimension of the data. The output generate by wgrib2 to numpy array is a single dimension with value of (259920,) for lat, lon and data. I have checked with pygrib and it produces the shape of (720, 361) which leads to 720 x 361 = 259920. Therefore, it seems that I have an issue with the data being converted in to numpy.

Your code does not produce an error on Python 3.5 with basemap 1.1.0, so this should be fixed if you upgrade your basemap. If you are actually running Python 2 (which is suggested by your minimal example), then it may be sufficient to remove division from your __future__ imports.
In particular, this MCVE does not produce an error:
from __future__ import print_function, unicode_literals, division
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
lat = np.linspace(-90, 90, 180)
lon = np.linspace(-180, 180, 360)
lat, lon = np.meshgrid(lat, lon)
data = np.random.randn(360, 180)
plt.figure()
m = Basemap(projection='mill', lat_ts=10, llcrnrlon=lon.min(), urcrnrlon=lon.max(), llcrnrlat=lat.min(), urcrnrlat=lat.max(),
resolution='c')
x, y = m(lon, lat)
cs = m.contourf(x, y, data)
m.drawcoastlines()
m.fillcontinents()
m.drawmapboundary()
plt.show()
If for some reason that doesn't work and you want to patch the issue erroring right here, you can go into the basemap source code and change xx = x[x.shape[0]/2,:] to xx = x[int(x.shape[0]/2),:]. What I heard from Ben suggests that if you do this, you're likely to continue to run in to other compatibility problems (possibly right after you make that fix), but it might get this code working for now.
Keep in mind that basemap does not support Python 3, and is deprecated. The core developer of Basemap (Ben Root) is urging users to stop using it, because it's been abandoned and will not be ported to Python 3, even when Python 2 stops receiving security updates in 2020. He suggests using cartopy instead. All this was said by Ben Root at Scipy 2017.

Algorithm equalivence from Matlab to Python

I've plotted a 3-d mesh in Matlab by below little m-file:
[x,n] = meshgrid(0:0.1:20, 1:1:100);
mu = 0;
sigma = sqrt(2)./n;
f = normcdf(x,mu,sigma);
mesh(x,n,f);
I am going to acquire the same result by utilization of Python and its corresponding modules, by below code snippet:
import numpy as np
from scipy.integrate import quad
import matplotlib.pyplot as plt
sigma = 1
def integrand(x, n):
return (n/(2*sigma*np.sqrt(np.pi)))*np.exp(-(n**2*x**2)/(4*sigma**2))
tt = np.linspace(0, 20, 2000)
nn = np.linspace(1, 100, 100)
T = np.zeros([len(tt), len(nn)])
for i,t in enumerate(tt):
for j,n in enumerate(nn):
T[i, j], _ = quad(integrand, -np.inf, t, args=(n,))
x, y = np.mgrid[0:20:0.01, 1:101:1]
plt.pcolormesh(x, y, T)
plt.show()
But the output of the Python is is considerably different with the Matlab one, and as a matter of fact is unacceptable.
I am afraid of wrong utilization of the functions just like linespace, enumerate or mgrid...
Does somebody have any idea about?!...
PS. Unfortunately, I couldn't insert the output plots within this thread...!
Best
..............................
Edit: I changed the linespace and mgrid intervals and replaced plot_surface method... The output is 3d now with the suitable accuracy and smoothness...

From what I see the equivalent solution would be:
import numpy as np
from scipy.stats import norm
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
x, n = np.mgrid[0:20:0.01, 1:100:1]
mu = 0
sigma = np.sqrt(2)/n
f = norm.cdf(x, mu, sigma)
fig = plt.figure()
ax = fig.gca(projection='3d')
ax.plot_surface(x, n, f, rstride=x.shape[0]//20, cstride=x.shape[1]//20, alpha=0.3)
plt.show()
Unfortunately 3D plotting with matplotlib is not as straight forward as with matlab.
Here is the plot from this code:

Your Matlab code generate 201 points through x:
[x,n] = meshgrid(0:0.1:20, 1:1:100);
While your Python code generate only 20 points:
tt = np.linspace(0, 19, 20)
Maybe it's causing accuracy problems?
Try this code:
tt = np.linspace(0, 20, 201)

The seminal points to resolve the problem was:
1- Necessity of the equivalence regarding the provided dimensions of the linespace and mgrid functions...
2- Utilization of a mesh with more density to make a bee line into a high degree of smoothness...
3- Application of a 3d plotter function, like plot_surf...
The current code is totally valid...

resampled time using scipy.signal.resample

I have a signal that is not sampled equidistant; for further processing it needs to be. I thought that scipy.signal.resample would do it, but I do not understand its behavior.
The signal is in y, corresponding time in x.
The resampled is expected in yy, with all corresponding time in xx. Does anyone know what I do wrong or how to achieve what I need?
This code does not work: xx is not time:
import numpy as np
from scipy import signal
import matplotlib.pyplot as plt
x = np.array([0,1,2,3,4,5,6,6.5,7,7.5,8,8.5,9])
y = np.cos(-x**2/4.0)
num=50
z=signal.resample(y, num, x, axis=0, window=None)
yy=z[0]
xx=z[1]
plt.plot(x,y)
plt.plot(xx,yy)
plt.show()

Even when you give the x coordinates (which corresponds to the t argument), resample assumes that the sampling is uniform.
Consider using one of the univariate interpolators in scipy.interpolate.
For example, this script:
import numpy as np
from scipy import interpolate
import matplotlib.pyplot as plt
x = np.array([0,1,2,3,4,5,6,6.5,7,7.5,8,8.5,9])
y = np.cos(-x**2/4.0)
f = interpolate.interp1d(x, y)
num = 50
xx = np.linspace(x[0], x[-1], num)
yy = f(xx)
plt.plot(x,y, 'bo-')
plt.plot(xx,yy, 'g.-')
plt.show()
generates this plot:
Check the docstring of interp1d for options to control the interpolation, and also check out the other interpolation classes.