Best way to plot 3D data over a large range in Python - python

I have some function f(x,y) --> z
The range of x is small, typically ~0.1 -1
The range of y is large, typically 1e8 - 1e14
The output of the function z also has a large range, from ~ 1e25 - 1e45
What is the best way to plot this function in Python?
Clearly I need some kind of logarithmic axes, but log scaling 3d plots in matplotlib lib is an issue (see here)
I have tried a colorbar plot, e.g.
fig, ax = plt.subplots()
x = np.linspace(1e8,1e14,500)
y = np.linspace(0.01,1,500)
X,Y = np.meshgrid(x,y)
Z = process(X,Y).T
im = plt.imshow(Z,cmap=cm.RdBu,norm=LogNorm())
im.set_interpolation('bilinear')
cb = fig.colorbar(im)
But the colours tend to 'wash out', e.g. the plot is mostly composed of just one colour and the x/y ticks go weird due to the normalisation which I have not found a way to fix.
Any ideas?

Related

Changing colors in a scatterplot using Matplotlib with python

I am currently taking a Matplotlib class. I was given an image to create the image as a 3D subplot 4 times at 4 different angles. It's a linear plot. As the data changes the plots change colors. As it's an image, I'm not certain where the actual changes start. I don't want an exact answer, just an explanation of how this would work. I have found many methods for doing this for a small list but this has 75 data points and I can't seem to do it without adding 75 entries.
I've also tried to understand cmap but I am confused on it as well.
Also, it needs to done without Seaborn.
This is part of the photo.
I am finding your question a little bit hard to understand. What I think you need is a function to map the input x/y argument onto a colour in your chosen colour map. See the below example:
import numpy as np
import matplotlib.pyplot
def number_to_colour(number, total_number):
return plt.cm.rainbow(np.linspace(0,1.,total_number))[list(number)]
x = np.arange(12)
y = x*-3.
z = x
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(x, y, z, c=number_to_colour(x, len(x)))
plt.show()
plt.cm.rainbow(np.linspace(0,1.,total_number)) creates an array of colours of length total_number evenly spaced spaced across the colour map (in this case rainbow). Modifying the indexing of this array (or changing np.linspace to another function with the desired scaling), should give you the colour scaling that you need.

Matplotlib 3D Waterfall Plot with Colored Heights

I'm trying to visualise a dataset in 3D which consists of a time series (along y) of x-z data, using Python and Matplotlib.
I'd like to create a plot like the one below (which was made in Python: http://austringer.net/wp/index.php/2011/05/20/plotting-a-dolphin-biosonar-click-train/), but where the colour varies with Z - i.e. so the intensity is shown by a colormap as well as the peak height, for clarity.
An example showing the colormap in Z is (apparently made using MATLAB):
This effect can be created using the waterfall plot option in MATLAB, but I understand there is no direct equivalent of this in Python.
I have also tried using the plot_surface option in Python (below), which works ok, but I'd like to 'force' the lines running over the surface to only be in the x direction (i.e. making it look more like a stacked time series than a surface). Is this possible?
Any help or advice greatly welcomed. Thanks.
I have generated a function that replicates the matlab waterfall behaviour in matplotlib, but I don't think it is the best solution when it comes to performance.
I started from two examples in matplotlib documentation: multicolor lines and multiple lines in 3d plot. From these examples, I only saw possible to draw lines whose color varies following a given colormap according to its z value following the example, which is reshaping the input array to draw the line by segments of 2 points and setting the color of the segment to the z mean value between the 2 points.
Thus, given the input matrixes n,m matrixes X,Y and Z, the function loops over the smallest dimension between n,m to plot each line like in the example, by 2 points segments, where the reshaping to plot by segments is done reshaping the array with the same code as the example.
def waterfall_plot(fig,ax,X,Y,Z):
'''
Make a waterfall plot
Input:
fig,ax : matplotlib figure and axes to populate
Z : n,m numpy array. Must be a 2d array even if only one line should be plotted
X,Y : n,m array
'''
# Set normalization to the same values for all plots
norm = plt.Normalize(Z.min().min(), Z.max().max())
# Check sizes to loop always over the smallest dimension
n,m = Z.shape
if n>m:
X=X.T; Y=Y.T; Z=Z.T
m,n = n,m
for j in range(n):
# reshape the X,Z into pairs
points = np.array([X[j,:], Z[j,:]]).T.reshape(-1, 1, 2)
segments = np.concatenate([points[:-1], points[1:]], axis=1)
lc = LineCollection(segments, cmap='plasma', norm=norm)
# Set the values used for colormapping
lc.set_array((Z[j,1:]+Z[j,:-1])/2)
lc.set_linewidth(2) # set linewidth a little larger to see properly the colormap variation
line = ax.add_collection3d(lc,zs=(Y[j,1:]+Y[j,:-1])/2, zdir='y') # add line to axes
fig.colorbar(lc) # add colorbar, as the normalization is the same for all, it doesent matter which of the lc objects we use
Therefore, plots looking like matlab waterfall can be easily generated with the same input matrixes as a matplotlib surface plot:
import numpy as np; import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
from mpl_toolkits.mplot3d import Axes3D
# Generate data
x = np.linspace(-2,2, 500)
y = np.linspace(-2,2, 40)
X,Y = np.meshgrid(x,y)
Z = np.sin(X**2+Y**2)
# Generate waterfall plot
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
waterfall_plot(fig,ax,X,Y,Z)
ax.set_xlabel('X') ; ax.set_xlim3d(-2,2)
ax.set_ylabel('Y') ; ax.set_ylim3d(-2,2)
ax.set_zlabel('Z') ; ax.set_zlim3d(-1,1)
The function assumes that when generating the meshgrid, the x array is the longest, and by default the lines have fixed y, and its the x coordinate what varies. However, if the size of the y dimension is larger, the matrixes are transposed, generating the lines with fixed x. Thus, generating the meshgrid with the sizes inverted (len(x)=40 and len(y)=500) yields:
with a pandas dataframe with the x axis as the first column and each spectra as another column
offset=0
for c in s.columns[1:]:
plt.plot(s.wavelength,s[c]+offset)
offset+=.25
plt.xlim([1325,1375])

Plotting KDE with logarithmic x-data in Matplotlib

I want to plot a KDE for some data with data that covers a large range in x-values. Therefore I want to use a logarithmic scale for the x-axis. For plotting I was using seaborn and the solution from Plotting 2D Kernel Density Estimation with Python, both of which fail once I set the xscale to logarithmic. When I take the logarithm of my x-data beforehand, everything looks fine, except the tics and ticlabels are still linear with the logarithm of the actual values as the labels. I could manually change the tics using something like:
labels = np.array(ax.get_xticks().tolist(), dtype=np.float64)
new_labels = [r'$10^{%.1f}$' % (labels[i]) for i in range(len(labels))]
ax.set_xticklabels(new_labels)
but in my eyes that looks just wrong and is nothing close to the axis labels (including the minor tics) when I would just use
ax.set_xscale('log')
Is there an easier way to plot a KDE with logarithmic x-data? Or is it possible to just change the tic- or label-scale without changing the scaling of the data, so that I could plot the logarithmic values of x and change the scaling of the labels afterwards?
Edit:
The plot I want to create looks like this:
The two right columns are what it is supposed to look like. There I used the the x data with the logarithm already applied. I don't like the labels on the x-axis, though.
The left column displays the plots, when the original data is used for the kde and all the other plots, and afterwards the scale is changed using
ax.set_xscale('log')
For some reason the kde, does not look like it is supposed to look. This is also not a result of erroneous data, since it looks just fine if the logarithmic data is used.
Edit 2:
A working example of code is
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
data = np.random.multivariate_normal((0, 0), [[0.8, 0.05], [0.05, 0.7]], 100)
x = np.power(10, data[:, 0])
y = data[:, 1]
fig, ax = plt.subplots(2, 1)
sns.kdeplot(data=np.log10(x), data2=y, ax=ax[0])
sns.kdeplot(data=x, data2=y, ax=ax[1])
ax[1].set_xscale('log')
plt.show()
The ax[1] plot is not displayed correctly for me (the x-axis is inverted), but the general behavior is the same as for the case described above. I believe the problem lies with the bandwidth of the kde, which should probably account for the logarithmic x-data.
I found an answer that works for me and wanted to post it in case someone else has a similar problem.
Based on the accepted answer from this post, I defined a function that first applies the logarithm to the x-data and after the KDE was performed, transforms the x-values back to the original values. Afterwards I can simply plot the contours and use ax.set_xscale('log')
import numpy as np
import scipy.stats as st
def logx_kde(x, y, xmin, xmax, ymin, ymax):
x = np.log10(x)
# Peform the kernel density estimate
xx, yy = np.mgrid[xmin:xmax:100j, ymin:ymax:100j]
positions = np.vstack([xx.ravel(), yy.ravel()])
values = np.vstack([x, y])
kernel = st.gaussian_kde(values)
f = np.reshape(kernel(positions).T, xx.shape)
return np.power(10, xx), yy, f

Setting both axes logarithmic in bar plot matploblib

I have already binned data to plot a histogram. For this reason I'm using the plt.bar() function. I'd like to set both axes in the plot to a logarithmic scale.
If I set plt.bar(x, y, width=10, color='b', log=True) which lets me set the y-axis to log but I can't set the x-axis logarithmic.
I've tried plt.xscale('log') unfortunately this doesn't work right. The x-axis ticks vanish and the sizes of the bars don't have equal width.
I would be grateful for any help.
By default, the bars of a barplot have a width of 0.8. Therefore they appear larger for smaller x values on a logarithmic scale. If instead of specifying a constant width, one uses the distance between the bin edges and supplies this to the width argument, the bars will have the correct width. One would also need to set the align to "edge" for this to work.
import matplotlib.pyplot as plt
import numpy as np; np.random.seed(1)
x = np.logspace(0, 5, num=21)
y = (np.sin(1.e-2*(x[:-1]-20))+3)**10
fig, ax = plt.subplots()
ax.bar(x[:-1], y, width=np.diff(x), log=True,ec="k", align="edge")
ax.set_xscale("log")
plt.show()
I cannot reproduce missing ticklabels for a logarithmic scaling. This may be due to some settings in the code that are not shown in the question or due to the fact that an older matplotlib version is used. The example here works fine with matplotlib 2.0.
If the goal is to have equal width bars, assuming datapoints are not equidistant, then the most proper solution is to set width as
plt.bar(x, y, width=c*np.array(x), color='b', log=True) for a constant c appropriate for the plot. Alignment can be anything.
I know it is a very old question and you might have solved it but I've come to this post because I was with something like this but at the y axis and I manage to solve it just using ax.set_ylim(df['my data'].min()+100, df['my data'].max()+100). In y axis I have some sensible information which I thouhg the best way was to show in log scale but when I set log scale I couldn't see the numbers proper (as this post in x axis) so I just leave the idea of use log and use the min and max argment. It sets the scale of my graph much like as log. Still looking for another way for doesnt need use that -+100 at set_ylim.
While this does not actually use pyplot.bar, I think this method could be helpful in achieving what the OP is trying to do. I found this to be easier than trying to calibrate the width as a function of the log-scale, though it's more steps. Create a line collection whose width is independent of the chart scale.
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.collections as coll
#Generate data and sort into bins
a = np.random.logseries(0.5, 1000)
hist, bin_edges = np.histogram(a, bins=20, density=False)
x = bin_edges[:-1] # remove the top-end from bin_edges to match dimensions of hist
lines = []
for i in range(len(x)):
pair=[(x[i],0), (x[i], hist[i])]
lines.append(pair)
linecoll = coll.LineCollection(lines, linewidths=10, linestyles='solid')
fig, ax = plt.subplots()
ax.add_collection(linecoll)
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_xlim(min(x)/10,max(x)*10)
ax.set_ylim(0.1,1.1*max(hist)) #since this is an unweighted histogram, the logy doesn't make much sense.
Resulting plot - no frills
One drawback is that the "bars" will be centered, but this could be changed by offsetting the x-values by half of the linewidth value ... I think it would be
x_new = x + (linewidth/2)*10**round(np.log10(x),0).

Wireframe joins the wrong way in numpy matplotlib mplot3d

I'm trying to create a 3D wireframe in Python using matplotlib.
When I get to the actual graph plotting, however, the wireframe joins the wrong way, as shown in the images below.
How can I force matplotlib to join the wireframe along a certain axis?
My code is below:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
def rossler(x_n, y_n, z_n, h, a, b, c):
#defining the rossler function
x_n1=x_n+h*(-y_n-z_n)
y_n1=y_n+h*(x_n+a*y_n)
z_n1=z_n+h*(b+z_n*(x_n-c))
return x_n1,y_n1,z_n1
#defining a, b, and c
a = 1.0/5.0
b = 1.0/5.0
c = 5
#defining time limits and steps
t_0 = 0
t_f = 32*np.pi
h = 0.01
steps = int((t_f-t_0)/h)
#3dify
c_list = np.linspace(5,10,6)
c_size = len(c_list)
c_array = np.zeros((c_size,steps))
for i in range (0, c_size):
for j in range (0, steps):
c_array[i][j] = c_list[i]
#create plotting values
t = np.zeros((c_size,steps))
for i in range (0, c_size):
t[i] = np.linspace(t_0,t_f,steps)
x = np.zeros((c_size,steps))
y = np.zeros((c_size,steps))
z = np.zeros((c_size,steps))
binvar, array_size = x.shape
#initial conditions
x[0] = 0
y[0] = 0
z[0] = 0
for j in range(0, c_size-1):
for i in range(array_size-1):
c = c_list[j]
#re-evaluate the values of the x-arrays depending on the initial conditions
[x[j][i+1],y[j][i+1],z[j][i+1]]=rossler(x[j][i],y[j][i],z[j][i],t[j][i+1]-t[j][i],a,b,c)
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot_wireframe(t,x,c_array, rstride=10, cstride=10)
plt.show()
I am getting this as an output:
The same output from another angle:
Whereas I'd like the wireframe to join along the wave-peaks. Sorry, I can't give you an image I'd like to see, that's my problem, but I guess it'd be more like the tutorial image.
If I understood, you want to link the 6 traces with polygons. You can do that by triangulating the traces 2 by 2, then plotting the surface with no edges or antialising. Maybe choosing a good colormap will also help.
Just keep in mind that this will be a very heavy plot. The exported SVG weight 10mb :)
import matplotlib.tri as mtri
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
for LineIndex in range(c_size-1):
# If plotting all at once, you get a MemoryError. I'll plot each 6 points
for Sample in range(0, array_size-1, 3):
# I switched x and c_array, because the surface and the triangles
# will look better by default
X = np.concatenate([t[LineIndex,Sample:Sample+3], t[LineIndex+1,Sample:Sample+3]])
Y = np.concatenate([c_array[LineIndex,Sample:Sample+3], c_array[LineIndex+1,Sample:Sample+3]])
Z = np.concatenate([x[LineIndex,Sample:Sample+3], x[LineIndex+1,Sample:Sample+3]])
T = mtri.Triangulation(X, Y)
ax.plot_trisurf(X, Y, Z, triangles=T.triangles, edgecolor='none', antialiased=False)
ax.set_xlabel('t')
ax.set_zlabel('x')
plt.savefig('Test.png', format='png', dpi=600)
plt.show()
Here is the resulting image:
I'm quite unsure about what you're exactly trying to achieve, but I don't think it will work.
Here's what your data looks like when plotted layer by layer (without and with filling):
You're trying to plot this as a wireframe plot. Here's how a wireframe plot looks like as per the manual:
Note the huge differene: a wireframe plot is essentially a proper surface plot, the only difference is that the faces of the surface are fully transparent. This also implies that you can only plot
single-valued functions of the form z(x,y), which are furthermore
specified on a rectangular mesh (at least topologically)
Your data is neither: your points are given along lines, and they are stacked on top of each other, so there's no chance that this is a single surface that can be plotted.
If you just want to visualize your functions above each other, here's how I plotted the above figures:
from mpl_toolkits.mplot3d.art3d import Poly3DCollection
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
for zind in range(t.shape[0]):
tnow,xnow,cnow = t[zind,:],x[zind,:],c_array[zind,:]
hplot = ax.plot(tnow,xnow,cnow)
# alternatively fill:
stride = 10
tnow,xnow,cnow = tnow[::stride],xnow[::stride],cnow[::stride]
slice_from = slice(None,-1)
slice_to = slice(1,None)
xpoly = np.array([tnow[slice_from],
tnow[slice_to],
tnow[slice_to],
tnow[slice_from]]
).T
ypoly = np.array([xnow[slice_from],
xnow[slice_to],
np.zeros_like(xnow[slice_to]),
np.zeros_like(xnow[slice_from])]
).T
zpoly = np.array([cnow[slice_from],
cnow[slice_to],
cnow[slice_to],
cnow[slice_from]]
).T
tmppoly = [tuple(zip(xrow,yrow,zrow)) for xrow,yrow,zrow in zip(xpoly,ypoly,zpoly)]
poly3dcoll = Poly3DCollection(tmppoly,linewidth=0.0)
poly3dcoll.set_edgecolor(hplot[0].get_color())
poly3dcoll.set_facecolor(hplot[0].get_color())
ax.add_collection3d(poly3dcoll)
plt.xlabel('t')
plt.ylabel('x')
plt.show()
There is one other option: switching your coordinate axes, such that the (x,t) pair corresponds to a vertical plane rather than a horizontal one. In this case your functions for various c values are drawn on parallel planes. This allows a wireframe plot to be used properly, but since your functions have extrema in different time steps, the result is as confusing as your original plot. You can try using very few plots along the t axis, and hoping that the extrema are close. This approach needs so much guesswork that I didn't try to do this myself. You can plot each function as a filled surface instead, though:
from matplotlib.collections import PolyCollection
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
for zind in range(t.shape[0]):
tnow,xnow,cnow = t[zind,:],x[zind,:],c_array[zind,:]
hplot = ax.plot(tnow,cnow,xnow)
# alternative to fill:
stride = 10
tnow,xnow,cnow = tnow[::stride],xnow[::stride],cnow[::stride]
slice_from = slice(None,-1)
slice_to = slice(1,None)
xpoly = np.array([tnow[slice_from],
tnow[slice_to],
tnow[slice_to],
tnow[slice_from]]
).T
ypoly = np.array([xnow[slice_from],
xnow[slice_to],
np.zeros_like(xnow[slice_to]),
np.zeros_like(xnow[slice_from])]
).T
tmppoly = [tuple(zip(xrow,yrow)) for xrow,yrow in zip(xpoly,ypoly)]
polycoll = PolyCollection(tmppoly,linewidth=0.5)
polycoll.set_edgecolor(hplot[0].get_color())
polycoll.set_facecolor(hplot[0].get_color())
ax.add_collection3d(polycoll,zdir='y',zs=cnow[0])
hplot[0].set_color('none')
ax.set_xlabel('t')
ax.set_zlabel('x')
plt.show()
This results in something like this:
There are a few things to note, however.
3d scatter and wire plots are very hard to comprehend, due to the lacking depth information. You might be approaching your visualization problem in a fundamentally wrong way: maybe there are other options with which you can visualize your data.
Even if you do something like the plots I showed, you should be aware that matplotlib has historically been failing to plot complicated 3d objects properly. Now by "properly" I mean "with physically reasonable apparent depth", see also the mplot3d FAQ note describing exactly this. The core of the problem is that matplotlib projects every 3d object to 2d, and draws these pancakes on the sreen one after the other. Sometimes the asserted drawing order of the pancakes doesn't correspond to their actual relative depth, which leads to artifacts that are both very obvious to humans and uncanny to look at. If you take a closer look at the first filled plot in this post, you'll see that the gold flat plot is behind the magenta one, even though it should be on top of it. Similar things often happen with 3d bar plots and convoluted surfaces.
When you're saying "Sorry, I can't give you an image I'd like to see, that's my problem", you're very wrong. It's not just your problem. It might be crystal clear in your head what you're trying to achieve, but unless you very clearly describe what you see in your head, the outside world will have to resort to guesswork. You can make the work of others and yourself alike easier by trying to be as informative as possible.

Categories