We Keep Coding

How to convert a matrix to heatmap image in torch [duplicate]

Using Matplotlib, I want to plot a 2D heat map. My data is an n-by-n Numpy array, each with a value between 0 and 1. So for the (i, j) element of this array, I want to plot a square at the (i, j) coordinate in my heat map, whose color is proportional to the element's value in the array.
How can I do this?

The imshow() function with parameters interpolation='nearest' and cmap='hot' should do what you want.
Please review the interpolation parameter details, and see Interpolations for imshow and Image antialiasing.
import matplotlib.pyplot as plt
import numpy as np
a = np.random.random((16, 16))
plt.imshow(a, cmap='hot', interpolation='nearest')
plt.show()

Seaborn is a high-level API for matplotlib, which takes care of a lot of the manual work.
seaborn.heatmap automatically plots a gradient at the side of the chart etc.
import numpy as np
import seaborn as sns
import matplotlib.pylab as plt
uniform_data = np.random.rand(10, 12)
ax = sns.heatmap(uniform_data, linewidth=0.5)
plt.show()
You can even plot upper / lower left / right triangles of square matrices. For example, a correlation matrix, which is square and is symmetric, so plotting all values would be redundant.
corr = np.corrcoef(np.random.randn(10, 200))
mask = np.zeros_like(corr)
mask[np.triu_indices_from(mask)] = True
with sns.axes_style("white"):
ax = sns.heatmap(corr, mask=mask, vmax=.3, square=True, cmap="YlGnBu")
plt.show()

I would use matplotlib's pcolor/pcolormesh function since it allows nonuniform spacing of the data.
Example taken from matplotlib:
import matplotlib.pyplot as plt
import numpy as np
# generate 2 2d grids for the x & y bounds
y, x = np.meshgrid(np.linspace(-3, 3, 100), np.linspace(-3, 3, 100))
z = (1 - x / 2. + x ** 5 + y ** 3) * np.exp(-x ** 2 - y ** 2)
# x and y are bounds, so z should be the value *inside* those bounds.
# Therefore, remove the last value from the z array.
z = z[:-1, :-1]
z_min, z_max = -np.abs(z).max(), np.abs(z).max()
fig, ax = plt.subplots()
c = ax.pcolormesh(x, y, z, cmap='RdBu', vmin=z_min, vmax=z_max)
ax.set_title('pcolormesh')
# set the limits of the plot to the limits of the data
ax.axis([x.min(), x.max(), y.min(), y.max()])
fig.colorbar(c, ax=ax)
plt.show()

For a 2d numpy array, simply use imshow() may help you:
import matplotlib.pyplot as plt
import numpy as np
def heatmap2d(arr: np.ndarray):
plt.imshow(arr, cmap='viridis')
plt.colorbar()
plt.show()
test_array = np.arange(100 * 100).reshape(100, 100)
heatmap2d(test_array)
This code produces a continuous heatmap.
You can choose another built-in colormap from here.

Here's how to do it from a csv:
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import griddata
# Load data from CSV
dat = np.genfromtxt('dat.xyz', delimiter=' ',skip_header=0)
X_dat = dat[:,0]
Y_dat = dat[:,1]
Z_dat = dat[:,2]
# Convert from pandas dataframes to numpy arrays
X, Y, Z, = np.array([]), np.array([]), np.array([])
for i in range(len(X_dat)):
X = np.append(X, X_dat[i])
Y = np.append(Y, Y_dat[i])
Z = np.append(Z, Z_dat[i])
# create x-y points to be used in heatmap
xi = np.linspace(X.min(), X.max(), 1000)
yi = np.linspace(Y.min(), Y.max(), 1000)
# Interpolate for plotting
zi = griddata((X, Y), Z, (xi[None,:], yi[:,None]), method='cubic')
# I control the range of my colorbar by removing data
# outside of my range of interest
zmin = 3
zmax = 12
zi[(zi<zmin) | (zi>zmax)] = None
# Create the contour plot
CS = plt.contourf(xi, yi, zi, 15, cmap=plt.cm.rainbow,
vmax=zmax, vmin=zmin)
plt.colorbar()
plt.show()
where dat.xyz is in the form
x1 y1 z1
x2 y2 z2
...

Use matshow() which is a wrapper around imshow to set useful defaults for displaying a matrix.
a = np.diag(range(15))
plt.matshow(a)
https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.matshow.html
This is just a convenience function wrapping imshow to set useful defaults for displaying a matrix. In particular:
Set origin='upper'.
Set interpolation='nearest'.
Set aspect='equal'.
Ticks are placed to the left and above.
Ticks are formatted to show integer indices.

Here is a new python package to plot complex heatmaps with different kinds of row/columns annotations in Python: https://github.com/DingWB/PyComplexHeatmap

How do I remove overflow along the z-axis for a 3D matplotlib surface?

I'm trying to graph a 3d mesh surface with matplotlib and constrain the limits of the graph. The X and Y axes are correctly constrained, but there is overflow in the Z-Axis.
What am I missing? Here's my code:
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits import mplot3d
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure(figsize=(10,10))
x = np.linspace(-6,6,100)
y = np.linspace(-6,6,100)
X,Y = np.meshgrid(x,y)
def f(x,y):
return x**2 + 3*y
Z = f(X,Y)
ax = plt.axes(projection = '3d')
ax.plot_surface(X,Y,Z,cmap='viridis')
ax.title.set_text("z=x**2+3y")
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
ax.set_zlim3d(zmin=-3,zmax=5)
ax.set_xlim3d(xmin=-6,xmax=6)
ax.set_ylim3d(ymin=-6,ymax=6)
plt.show()
The graph:
Edit:
When I add clipping/min/max to the Z values, the graph is a little better, but it sets z values outside the bounds to the bounds themselves. Both of the following suggestions do this. Perhaps it's because I'm on a mac?
z_tmp = np.maximum(np.minimum(5,Z),-3)
z_temp = np.clip(Z, -3, 5, None)

Your data is outside the axis boundaries. Try rotate the view and you will notice.
z = x**2 + 3*y
If you want to only show a defined area of the data you could add a max() min() limitation on the Z data to exclude the data outside your wanted limitations.
Z = f(X,Y)
z_tmp = np.maximum(np.minimum(5,Z),-3)
ax = plt.axes(projection = '3d')
ax.plot_surface(X,Y,z_tmp,cmap='viridis')
I'm not sure the matplotlib behaves as it should in your default case.

Plotting a heatmap with interpolation in Python using excel file

I need to plot a HEATMAP in python using x, y, z data from the excel file.
All the values of z are 1 except at (x=5,y=5). The plot should be red at point (5,5) and blue elsewhere. But I am getting false alarms which need to be removed. The COLORMAP I have used is 'jet'
X=[0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,6,6,6,6,6,6,6,6,6,6,7,7,7,7,7,7,7,7,7,7,8,8,8,8,8,8,8,8,8,8,9,9,9,9,9,9,9,9,9,9]
Y=[0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9]
Z=[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,9,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]
Code I have used is:
import matplotlib.pyplot as plt
import numpy as np
from numpy import ravel
from scipy.interpolate import interp2d
import pandas as pd
import matplotlib as mpl
excel_data_df = pd.read_excel('test.xlsx')
X= excel_data_df['x'].tolist()
Y= excel_data_df['y'].tolist()
Z= excel_data_df['z'].tolist()
x_list = np.array(X)
y_list = np.array(Y)
z_list = np.array(Z)
# f will be a function with two arguments (x and y coordinates),
# but those can be array_like structures too, in which case the
# result will be a matrix representing the values in the grid
# specified by those arguments
f = interp2d(x_list,y_list,z_list,kind="linear")
x_coords = np.arange(min(x_list),max(x_list))
y_coords = np.arange(min(y_list),max(y_list))
z= f(x_coords,y_coords)
fig = plt.imshow(z,
extent=[min(x_list),max(x_list),min(y_list),max(y_list)],
origin="lower", interpolation='bicubic', cmap= 'jet', aspect='auto')
# Show the positions of the sample points, just to have some reference
fig.axes.set_autoscale_on(False)
#plt.scatter(x_list,y_list,400, facecolors='none')
plt.xlabel('X Values', fontsize = 15, va="center")
plt.ylabel('Y Values', fontsize = 15,va="center")
plt.title('Heatmap', fontsize = 20)
plt.tight_layout()
plt.show()
For your ease you can also use the X, Y, Z arrays instead of reading excel file.
The result that I am getting is:
Here you can see dark blue regions at (5,0) and (0,5). These are the FALSE ALARMS I am getting and I need to REMOVE these.
I am probably doing some beginner's mistake. Grateful to anyone who points it out. Regards

There are at least three problems in your example:
x_coords and y_coords are not properly resampled;
the interpolation z does to fill in the whole grid leading to incorrect output;
the output is then forced to be plotted on the original grid (extent) that add to the confusion.
Leading to the following interpolated results:
On what you have applied an extra smoothing with imshow.
Let's create your artificial input:
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(0, 11)
y = np.arange(0, 11)
X, Y = np.meshgrid(x, y)
Z = np.ones(X.shape)
Z[5,5] = 9
Depending on how you want to proceed, you can simply let imshow smooth your signal by interpolation:
fig, axe = plt.subplots()
axe.imshow(Z, origin="lower", cmap="jet", interpolation='bicubic')
And you are done, simple and efficient!
If you aim to do it by yourself, then choose the interpolant that suits you best and resample on a grid with a higher resolution:
interpolant = interpolate.interp2d(x, y, Z.ravel(), kind="linear")
xlin = np.linspace(0, 10, 101)
ylin = np.linspace(0, 10, 101)
zhat = interpolant(xlin, ylin)
fig, axe = plt.subplots()
axe.imshow(zhat, origin="lower", cmap="jet")
Have a deeper look on scipy.interpolate module to pick up the best interpolant regarding your needs. Notice that all methods does not expose the same interface for imputing parameters. You may need to reshape your data to use another objects.
MCVE
Here is a complete example using the trial data generated above. Just bind it to your excel columns:
# Flatten trial data to meet your requirement:
x = X.ravel()
y = Y.ravel()
z = Z.ravel()
# Resampling on as square grid with given resolution:
resolution = 11
xlin = np.linspace(x.min(), x.max(), resolution)
ylin = np.linspace(y.min(), y.max(), resolution)
Xlin, Ylin = np.meshgrid(xlin, ylin)
# Linear multi-dimensional interpolation:
interpolant = interpolate.NearestNDInterpolator([r for r in zip(x, y)], z)
Zhat = interpolant(Xlin.ravel(), Ylin.ravel()).reshape(Xlin.shape)
# Render and interpolate again if necessary:
fig, axe = plt.subplots()
axe.imshow(Zhat, origin="lower", cmap="jet", interpolation='bicubic')
Which renders as expected:

Update interactive plot in Jupyter Notebook [duplicate]

I am trying to animate a pcolormesh in matplotlib. I have seen many of the examples using the package animation, most of them using a 1D plot routine, and some of them with imshow().
First, I wan to use the FuncAnimation routine. My problem is, first, that I do not know if I can initialize the plot
fig,ax = plt.subplots()
quad = ax.pcolormesh(X,Y,Z)
I have tried a few simple lines:
fig,ax = plt.subplots()
quad = ax.pcolormesh([])
def init():
quad.set_array([])
return quad,
def animate(ktime):
quad.set_array(X,Y,np.sin(Z)+ktime)
return quad,
anim = animation.FuncAnimation(fig,animate,init_func=init,frames=Ntime,interval=200,blit=True)
plt.show()
By the way, How do I set labels into and animated plot? Can I animate the title, if it is showing a number that changes in time?
Thanks

The problem was that I was wrongly using set_array() routine. It is very important to note that you must pass a 1D array to this routine. To do so, regarding that color, pcolormesh and so on usually plots multidimensional arrays, you should use .ravel() .
One more important thing: In order to animate different plots at the same time, the blitz option at animate.FuncAnimation must be False (See section "Animating selected plot elements" of this link).
Here I post the code that simple program with various subplots:
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.gridspec as gridspec
import matplotlib.animation as animation
y, x = np.meshgrid(np.linspace(-10, 10,100), np.linspace(-10, 10,100))
z = np.sin(x)*np.sin(x)+np.sin(y)*np.sin(y)
v = np.linspace(-10, 10,100)
t = np.sin(v)*np.sin(v)
tt = np.cos(v)*np.cos(v)
###########
fig = plt.figure(figsize=(16, 8),facecolor='white')
gs = gridspec.GridSpec(5, 2)
ax1 = plt.subplot(gs[0,0])
line, = ax1.plot([],[],'b-.',linewidth=2)
ax1.set_xlim(-10,10)
ax1.set_ylim(0,1)
ax1.set_xlabel('time')
ax1.set_ylabel('amplitude')
ax1.set_title('Oscillationsssss')
time_text = ax1.text(0.02, 0.95, '', transform=ax1.transAxes)
#############################
ax2 = plt.subplot(gs[1:3,0])
quad1 = ax2.pcolormesh(x,y,z,shading='gouraud')
ax2.set_xlabel('time')
ax2.set_ylabel('amplitude')
cb2 = fig.colorbar(quad1,ax=ax2)
#########################
ax3 = plt.subplot(gs[3:,0])
quad2 = ax3.pcolormesh(x, y, z,shading='gouraud')
ax3.set_xlabel('time')
ax3.set_ylabel('amplitude')
cb3 = fig.colorbar(quad2,ax=ax3)
############################
ax4 = plt.subplot(gs[:,1])
line2, = ax4.plot(v,tt,'b',linewidth=2)
ax4.set_xlim(-10,10)
ax4.set_ylim(0,1)
def init():
line.set_data([],[])
line2.set_data([],[])
quad1.set_array([])
return line,line2,quad1
def animate(iter):
t = np.sin(2*v-iter/(2*np.pi))*np.sin(2*v-iter/(2*np.pi))
tt = np.cos(2*v-iter/(2*np.pi))*np.cos(2*v-iter/(2*np.pi))
z = np.sin(x-iter/(2*np.pi))*np.sin(x-iter/(2*np.pi))+np.sin(y)*np.sin(y)
line.set_data(v,t)
quad1.set_array(z.ravel())
line2.set_data(v,tt)
return line,line2,quad1
gs.tight_layout(fig)
anim = animation.FuncAnimation(fig,animate,frames=100,interval=50,blit=False,repeat=False)
plt.show()
print 'Finished!!'

There is an ugly detail you need to take care when using QuadMesh.set_array(). If you intantiate your QuadMesh with X, Y and C you can update the values C by using set_array(). But set_array does not support the same input as the constructor. Reading the source reveals that you need to pass a 1d-array and what is even more puzzling is that depending on the shading setting you might need to cut of your array C.
Edit: There is even a very old bug report about the confusing array size for shading='flat'.
That means:
Using QuadMesh.set_array() with shading = 'flat'
'flat' is default value for shading.
# preperation
import numpy as np
import matplotlib.pyplot as plt
plt.ion()
y = np.linspace(-10, 10, num=1000)
x = np.linspace(-10, 10, num=1000)
X, Y = np.meshgrid(x, y)
C = np.ones((1000, 1000)) * float('nan')
# intantiate empty plot (values = nan)
pcmesh = plt.pcolormesh(X, Y, C, vmin=-100, vmax=100, shading='flat')
# generate some new data
C = X * Y
# necessary for shading='flat'
C = C[:-1, :-1]
# ravel() converts C to a 1d-array
pcmesh.set_array(C.ravel())
# redraw to update plot with new data
plt.draw()
Looks like:
Note that if you omit C = C[:-1, :-1] your will get this broken graphic:
Using QuadMesh.set_array() with shading = 'gouraud'
# preperation (same as for 'flat')
import numpy as np
import matplotlib.pyplot as plt
plt.ion()
y = np.linspace(-10, 10, num=1000)
x = np.linspace(-10, 10, num=1000)
X, Y = np.meshgrid(x, y)
C = np.ones((1000, 1000)) * float('nan')
# intantiate empty plot (values = nan)
pcmesh = plt.pcolormesh(X, Y, C, vmin=-100, vmax=100, shading='gouraud')
# generate some new data
C = X * Y
# here no cut of of last row/column!
# ravel() converts C to a 1d-array
pcmesh.set_array(C.ravel())
# redraw to update plot with new data
plt.draw()
If you cut off the last row/column with shade='gouraud' you will get:
ValueError: total size of new array must be unchanged

I am not sure why your quad = ax.pcolormesh(X,Y,Z) function is giving an error. Can you post the error?
Below is what I would do to create a simple animation using pcolormesh:
import matplotlib.pyplot as plt
import numpy as np
y, x = np.meshgrid(np.linspace(-3, 3,100), np.linspace(-3, 3,100))
z = np.sin(x**2+y**2)
z = z[:-1, :-1]
ax = plt.subplot(111)
quad = plt.pcolormesh(x, y, z)
plt.colorbar()
plt.ion()
plt.show()
for phase in np.linspace(0,10*np.pi,200):
z = np.sin(np.sqrt(x**2+y**2) + phase)
z = z[:-1, :-1]
quad.set_array(z.ravel())
plt.title('Phase: %.2f'%phase)
plt.draw()
plt.ioff()
plt.show()
One of the frames:
Does this help? If not, maybe you can clarify the question.

There is another answer presented here that looks simpler thus better (IMHO)
Here is a copy & paste of the alternative solution :
import matplotlib.pylab as plt
from matplotlib import animation
fig = plt.figure()
plt.hold(True)
#We need to prime the pump, so to speak and create a quadmesh for plt to work with
plt.pcolormesh(X[0:1], Y[0:1], C[0:1])
anim = animation.FuncAnimation(fig, animate, frames = range(2,155), blit = False)
plt.show()
plt.hold(False)
def animate( self, i):
plt.title('Ray: %.2f'%i)
#This is where new data is inserted into the plot.
plt.pcolormesh(X[i-2:i], Y[i-2:i], C[i-2:i])

Plotting surface of implicitly defined volume

Having a volume implicitly defined by
x*y*z <= 1
for
-5 <= x <= 5
-5 <= y <= 5
-5 <= z <= 5
how would I go about plotting its outer surface using available Python modules, preferably mayavi?
I am aware of the function mlab.mesh, but I don't understand its input. It requires three 2D arrays, that I don't understand how to create having the above information.
EDIT:
Maybe my problem lies with an unsufficient understanding of the meshgrid()-function or the mgrid-class of numpy. I see that I have to use them in some way, but I do not completely grasp their purpose or what such a grid represents.
EDIT:
I arrived at this:
import numpy as np
from mayavi import mlab
x, y, z = np.ogrid[-5:5:200j, -5:5:200j, -5:5:200j]
s = x*y*z
src = mlab.pipeline.scalar_field(s)
mlab.pipeline.iso_surface(src, contours=[1., ],)
mlab.show()
This results in an isosurface (for x*y*z=1) of a volume though, which is not quite what I was looking for. What I am looking for is basically a method to draw an arbitrary surface, like a "polygon in 3d" if there is such a thing.
I created the following code, which plots a surface (works with mayavi, too). I would need to modify this code to my particular problem, but to do that I need to understand why and how a 3d surface is defined by three 2d-arrays? What do these arrays (x, y and z) represent?
import numpy as np
from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import axes3d, Axes3D
phi, theta = np.mgrid[0:np.pi:11j, 0:2*np.pi:11j]
x = np.sin(phi) * np.cos(theta)
y = np.sin(phi) * np.sin(theta)
z = np.cos(phi)
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot_wireframe(x,y,z)
fig.show()

The outer surface, implicitly defined by
x*y*z = 1,
cannot be defined explicitly globally. To see this, consider x and y given, then:
z = 1/(x*y),
which is not defined for x = 0 or y = 0. Therefore, you can only define your surface locally for domains that do not include the singularity, e.g. for the domain
0 < x <= 5
0 < y <= 5
z is indeed defined (a hyperbolic surface). Similarly, you need to plot the surfaces for the other domains, until you have patched together
-5 <= x <= 5
-5 <= y <= 5
Note that your surface is not defined for x = 0 and y = 0, i.e. the axis of your coordinate system, so you cannot patch your surfaces together to get a globally defined surface.
Using numpy and matplotlib, you can plot one of these surfaces as follows (adopted from http://matplotlib.org/mpl_toolkits/mplot3d/tutorial.html#surface-plots):
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure()
ax = fig.gca(projection='3d')
X = np.arange(0.25, 5, 0.25)
Y = np.arange(0.25, 5, 0.25)
X, Y = np.meshgrid(X, Y)
Z = 1/(X*Y)
surf = ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=cm.coolwarm,
linewidth=0, antialiased=False)
ax.set_zlim(0, 10)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
fig.colorbar(surf, shrink=0.5, aspect=5)
plt.show()
I'm not familiar with mayavi, but I would assume that creating the meshes with numpy would work the same.

The test case in the Mayavi docs where the function test_mesh() is defined is capable of producing a sphere. This is done by replacing
r = sin(m0*phi)**m1 + cos(m2*phi)**m3 + sin(m4*theta)**m5 + cos(m6*theta)**m7
with r = 1.0 say.
However, your problem is you need to understand that the equations you are writing define a volume when you want to draw a sphere. You need to reformulate them to give a parametric equation of a sphere. This is essentially what is done in the above example, but it may be worth your while to try it yourself. As a hint consider the equation of a circle and extend it.