We Keep Coding

How to convert a matrix to heatmap image in torch [duplicate]

Using Matplotlib, I want to plot a 2D heat map. My data is an n-by-n Numpy array, each with a value between 0 and 1. So for the (i, j) element of this array, I want to plot a square at the (i, j) coordinate in my heat map, whose color is proportional to the element's value in the array.
How can I do this?

The imshow() function with parameters interpolation='nearest' and cmap='hot' should do what you want.
Please review the interpolation parameter details, and see Interpolations for imshow and Image antialiasing.
import matplotlib.pyplot as plt
import numpy as np
a = np.random.random((16, 16))
plt.imshow(a, cmap='hot', interpolation='nearest')
plt.show()

Seaborn is a high-level API for matplotlib, which takes care of a lot of the manual work.
seaborn.heatmap automatically plots a gradient at the side of the chart etc.
import numpy as np
import seaborn as sns
import matplotlib.pylab as plt
uniform_data = np.random.rand(10, 12)
ax = sns.heatmap(uniform_data, linewidth=0.5)
plt.show()
You can even plot upper / lower left / right triangles of square matrices. For example, a correlation matrix, which is square and is symmetric, so plotting all values would be redundant.
corr = np.corrcoef(np.random.randn(10, 200))
mask = np.zeros_like(corr)
mask[np.triu_indices_from(mask)] = True
with sns.axes_style("white"):
ax = sns.heatmap(corr, mask=mask, vmax=.3, square=True, cmap="YlGnBu")
plt.show()

I would use matplotlib's pcolor/pcolormesh function since it allows nonuniform spacing of the data.
Example taken from matplotlib:
import matplotlib.pyplot as plt
import numpy as np
# generate 2 2d grids for the x & y bounds
y, x = np.meshgrid(np.linspace(-3, 3, 100), np.linspace(-3, 3, 100))
z = (1 - x / 2. + x ** 5 + y ** 3) * np.exp(-x ** 2 - y ** 2)
# x and y are bounds, so z should be the value *inside* those bounds.
# Therefore, remove the last value from the z array.
z = z[:-1, :-1]
z_min, z_max = -np.abs(z).max(), np.abs(z).max()
fig, ax = plt.subplots()
c = ax.pcolormesh(x, y, z, cmap='RdBu', vmin=z_min, vmax=z_max)
ax.set_title('pcolormesh')
# set the limits of the plot to the limits of the data
ax.axis([x.min(), x.max(), y.min(), y.max()])
fig.colorbar(c, ax=ax)
plt.show()

For a 2d numpy array, simply use imshow() may help you:
import matplotlib.pyplot as plt
import numpy as np
def heatmap2d(arr: np.ndarray):
plt.imshow(arr, cmap='viridis')
plt.colorbar()
plt.show()
test_array = np.arange(100 * 100).reshape(100, 100)
heatmap2d(test_array)
This code produces a continuous heatmap.
You can choose another built-in colormap from here.

Here's how to do it from a csv:
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import griddata
# Load data from CSV
dat = np.genfromtxt('dat.xyz', delimiter=' ',skip_header=0)
X_dat = dat[:,0]
Y_dat = dat[:,1]
Z_dat = dat[:,2]
# Convert from pandas dataframes to numpy arrays
X, Y, Z, = np.array([]), np.array([]), np.array([])
for i in range(len(X_dat)):
X = np.append(X, X_dat[i])
Y = np.append(Y, Y_dat[i])
Z = np.append(Z, Z_dat[i])
# create x-y points to be used in heatmap
xi = np.linspace(X.min(), X.max(), 1000)
yi = np.linspace(Y.min(), Y.max(), 1000)
# Interpolate for plotting
zi = griddata((X, Y), Z, (xi[None,:], yi[:,None]), method='cubic')
# I control the range of my colorbar by removing data
# outside of my range of interest
zmin = 3
zmax = 12
zi[(zi<zmin) | (zi>zmax)] = None
# Create the contour plot
CS = plt.contourf(xi, yi, zi, 15, cmap=plt.cm.rainbow,
vmax=zmax, vmin=zmin)
plt.colorbar()
plt.show()
where dat.xyz is in the form
x1 y1 z1
x2 y2 z2
...

Use matshow() which is a wrapper around imshow to set useful defaults for displaying a matrix.
a = np.diag(range(15))
plt.matshow(a)
https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.matshow.html
This is just a convenience function wrapping imshow to set useful defaults for displaying a matrix. In particular:
Set origin='upper'.
Set interpolation='nearest'.
Set aspect='equal'.
Ticks are placed to the left and above.
Ticks are formatted to show integer indices.

Here is a new python package to plot complex heatmaps with different kinds of row/columns annotations in Python: https://github.com/DingWB/PyComplexHeatmap

Custom "multicolored lines" in matplotlib [duplicate]

I have two list as below:
latt=[42.0,41.978567980875397,41.96622693388357,41.963791391892457,...,41.972407378075879]
lont=[-66.706920989908909,-66.703116557977069,-66.707351643324543,...-66.718218142021925]
now I want to plot this as a line, separate each 10 of those 'latt' and 'lont' records as a period and give it a unique color.
what should I do?

There are several different ways to do this. The "best" approach will depend mostly on how many line segments you want to plot.
If you're just going to be plotting a handful (e.g. 10) line segments, then just do something like:
import numpy as np
import matplotlib.pyplot as plt
def uniqueish_color():
"""There're better ways to generate unique colors, but this isn't awful."""
return plt.cm.gist_ncar(np.random.random())
xy = (np.random.random((10, 2)) - 0.5).cumsum(axis=0)
fig, ax = plt.subplots()
for start, stop in zip(xy[:-1], xy[1:]):
x, y = zip(start, stop)
ax.plot(x, y, color=uniqueish_color())
plt.show()
If you're plotting something with a million line segments, though, this will be terribly slow to draw. In that case, use a LineCollection. E.g.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
xy = (np.random.random((1000, 2)) - 0.5).cumsum(axis=0)
# Reshape things so that we have a sequence of:
# [[(x0,y0),(x1,y1)],[(x0,y0),(x1,y1)],...]
xy = xy.reshape(-1, 1, 2)
segments = np.hstack([xy[:-1], xy[1:]])
fig, ax = plt.subplots()
coll = LineCollection(segments, cmap=plt.cm.gist_ncar)
coll.set_array(np.random.random(xy.shape[0]))
ax.add_collection(coll)
ax.autoscale_view()
plt.show()
For both of these cases, we're just drawing random colors from the "gist_ncar" coloramp. Have a look at the colormaps here (gist_ncar is about 2/3 of the way down): http://matplotlib.org/examples/color/colormaps_reference.html

Copied from this example:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
from matplotlib.colors import ListedColormap, BoundaryNorm
x = np.linspace(0, 3 * np.pi, 500)
y = np.sin(x)
z = np.cos(0.5 * (x[:-1] + x[1:])) # first derivative
# Create a colormap for red, green and blue and a norm to color
# f' < -0.5 red, f' > 0.5 blue, and the rest green
cmap = ListedColormap(['r', 'g', 'b'])
norm = BoundaryNorm([-1, -0.5, 0.5, 1], cmap.N)
# Create a set of line segments so that we can color them individually
# This creates the points as a N x 1 x 2 array so that we can stack points
# together easily to get the segments. The segments array for line collection
# needs to be numlines x points per line x 2 (x and y)
points = np.array([x, y]).T.reshape(-1, 1, 2)
segments = np.concatenate([points[:-1], points[1:]], axis=1)
# Create the line collection object, setting the colormapping parameters.
# Have to set the actual values used for colormapping separately.
lc = LineCollection(segments, cmap=cmap, norm=norm)
lc.set_array(z)
lc.set_linewidth(3)
fig1 = plt.figure()
plt.gca().add_collection(lc)
plt.xlim(x.min(), x.max())
plt.ylim(-1.1, 1.1)
plt.show()

See the answer here to generate the "periods" and then use the matplotlib scatter function as #tcaswell mentioned. Using the plot.hold function you can plot each period, colors will increment automatically.

Cribbing the color choice off of #JoeKington,
import numpy as np
import matplotlib.pyplot as plt
def uniqueish_color(n):
"""There're better ways to generate unique colors, but this isn't awful."""
return plt.cm.gist_ncar(np.random.random(n))
plt.scatter(latt, lont, c=uniqueish_color(len(latt)))
You can do this with scatter.

I have been searching for a short solution how to use pyplots line plot to show a time series coloured by a label feature without using scatter due to the amount of data points.
I came up with the following workaround:
plt.plot(np.where(df["label"]==1, df["myvalue"], None), color="red", label="1")
plt.plot(np.where(df["label"]==0, df["myvalue"], None), color="blue", label="0")
plt.legend()
The drawback is you are creating two different line plots so the connection between the different classes is not shown. For my purposes it is not a big deal. It may help someone.

matplotlib scatterplot: adding 4th dimension by the marker shape

I would like to add a fourth dimension to the scatter plot by defining the ellipticity of the markers depending on a variable. Is that possible somehow ?
EDIT:
I would like to avoid a 3D-plot. In my opinion these plots are usually not very informative.

You can place Ellipse patches directly onto your axes, as demonstrated in this matplotlib example. To adapt it to use eccentricity as your "third dimension") keeping the marker area constant:
from pylab import figure, show, rand
from matplotlib.patches import Ellipse
import numpy as np
import matplotlib.pyplot as plt
N = 25
# ellipse centers
xy = np.random.rand(N, 2)*10
# ellipse eccentrities
eccs = np.random.rand(N) * 0.8 + 0.1
fig = plt.figure()
ax = fig.add_subplot(111, aspect='equal')
A = 0.1
for pos, e in zip(xy, eccs):
# semi-minor, semi-major axes, b and a:
b = np.sqrt(A/np.pi * np.sqrt(1-e**2))
a = A / np.pi / b
ellipse = Ellipse(xy=pos, width=2*a, height=2*b)
ax.add_artist(ellipse)
ax.set_xlim(0, 10)
ax.set_ylim(0, 10)
show()
Of course, you need to scale your marker area to your x-, y- values in this case.

You can use colorbar as the 4th dimension to your 3D plot. One example is as shown below:
import matplotlib.cm as cmx
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import matplotlib
import numpy as np
def scatter3d(x,y,z, cs, colorsMap='jet'):
cm = plt.get_cmap(colorsMap)
cNorm = matplotlib.colors.Normalize(vmin=min(cs), vmax=max(cs))
scalarMap = cmx.ScalarMappable(norm=cNorm, cmap=cm)
fig = plt.figure()
ax = Axes3D(fig)
ax.scatter(x, y, z, c=scalarMap.to_rgba(cs))
scalarMap.set_array(cs)
fig.colorbar(scalarMap,label='Test')
plt.show()
x = np.random.uniform(0,1,50)
y = np.random.uniform(0,1,50)
z = np.random.uniform(0,1,50)
so scatter3D(x,y,z,x+y) produces:
with x+y being the 4th dimension shown in color. You can add your calculated ellipticity depending on your specific variable instead of x+y to get what you want.

To change the ellipticity of the markers you will have to create them manually as such a feature is not implemented yet. However, I believe you can show 4 dimensions with a 2D scatter plot by using color and size as additional dimensions. You will have to take care of the scaling from data to marker size yourself. I added a simple function to handle that in the example below:
import matplotlib.pyplot as plt
import numpy as np
data = np.random.rand(60,4)
def scale_size(data, data_min=None, data_max=None, size_min=10, size_max=60):
# if the data limits are set to None we will just infer them from the data
if data_min is None:
data_min = data.min()
if data_max is None:
data_max = data.max()
size_range = size_max - size_min
data_range = data_max - data_min
return ((data - data_min) * size_range / data_range) + size_min
plt.scatter(data[:,0], data[:,1], c=data[:,2], s=scale_size(data[:,3]))
plt.colorbar()
plt.show()
Result:

Matplotlib 2D line plot with color as a function of a third variable, plus colorbar [duplicate]

I have two list as below:
latt=[42.0,41.978567980875397,41.96622693388357,41.963791391892457,...,41.972407378075879]
lont=[-66.706920989908909,-66.703116557977069,-66.707351643324543,...-66.718218142021925]
now I want to plot this as a line, separate each 10 of those 'latt' and 'lont' records as a period and give it a unique color.
what should I do?

There are several different ways to do this. The "best" approach will depend mostly on how many line segments you want to plot.
If you're just going to be plotting a handful (e.g. 10) line segments, then just do something like:
import numpy as np
import matplotlib.pyplot as plt
def uniqueish_color():
"""There're better ways to generate unique colors, but this isn't awful."""
return plt.cm.gist_ncar(np.random.random())
xy = (np.random.random((10, 2)) - 0.5).cumsum(axis=0)
fig, ax = plt.subplots()
for start, stop in zip(xy[:-1], xy[1:]):
x, y = zip(start, stop)
ax.plot(x, y, color=uniqueish_color())
plt.show()
If you're plotting something with a million line segments, though, this will be terribly slow to draw. In that case, use a LineCollection. E.g.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
xy = (np.random.random((1000, 2)) - 0.5).cumsum(axis=0)
# Reshape things so that we have a sequence of:
# [[(x0,y0),(x1,y1)],[(x0,y0),(x1,y1)],...]
xy = xy.reshape(-1, 1, 2)
segments = np.hstack([xy[:-1], xy[1:]])
fig, ax = plt.subplots()
coll = LineCollection(segments, cmap=plt.cm.gist_ncar)
coll.set_array(np.random.random(xy.shape[0]))
ax.add_collection(coll)
ax.autoscale_view()
plt.show()
For both of these cases, we're just drawing random colors from the "gist_ncar" coloramp. Have a look at the colormaps here (gist_ncar is about 2/3 of the way down): http://matplotlib.org/examples/color/colormaps_reference.html

Copied from this example:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
from matplotlib.colors import ListedColormap, BoundaryNorm
x = np.linspace(0, 3 * np.pi, 500)
y = np.sin(x)
z = np.cos(0.5 * (x[:-1] + x[1:])) # first derivative
# Create a colormap for red, green and blue and a norm to color
# f' < -0.5 red, f' > 0.5 blue, and the rest green
cmap = ListedColormap(['r', 'g', 'b'])
norm = BoundaryNorm([-1, -0.5, 0.5, 1], cmap.N)
# Create a set of line segments so that we can color them individually
# This creates the points as a N x 1 x 2 array so that we can stack points
# together easily to get the segments. The segments array for line collection
# needs to be numlines x points per line x 2 (x and y)
points = np.array([x, y]).T.reshape(-1, 1, 2)
segments = np.concatenate([points[:-1], points[1:]], axis=1)
# Create the line collection object, setting the colormapping parameters.
# Have to set the actual values used for colormapping separately.
lc = LineCollection(segments, cmap=cmap, norm=norm)
lc.set_array(z)
lc.set_linewidth(3)
fig1 = plt.figure()
plt.gca().add_collection(lc)
plt.xlim(x.min(), x.max())
plt.ylim(-1.1, 1.1)
plt.show()

See the answer here to generate the "periods" and then use the matplotlib scatter function as #tcaswell mentioned. Using the plot.hold function you can plot each period, colors will increment automatically.

Cribbing the color choice off of #JoeKington,
import numpy as np
import matplotlib.pyplot as plt
def uniqueish_color(n):
"""There're better ways to generate unique colors, but this isn't awful."""
return plt.cm.gist_ncar(np.random.random(n))
plt.scatter(latt, lont, c=uniqueish_color(len(latt)))
You can do this with scatter.

I have been searching for a short solution how to use pyplots line plot to show a time series coloured by a label feature without using scatter due to the amount of data points.
I came up with the following workaround:
plt.plot(np.where(df["label"]==1, df["myvalue"], None), color="red", label="1")
plt.plot(np.where(df["label"]==0, df["myvalue"], None), color="blue", label="0")
plt.legend()
The drawback is you are creating two different line plots so the connection between the different classes is not shown. For my purposes it is not a big deal. It may help someone.

Matplotlib - add colorbar to a sequence of line plots

I have a sequence of line plots for two variables (x,y) for a number of different values of a variable z. I would normally add the line plots with legends like this:
import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.add_subplot(111)
# suppose mydata is a list of tuples containing (xs, ys, z)
# where xs and ys are lists of x's and y's and z is a number.
legns = []
for(xs,ys,z) in mydata:
pl = ax.plot(xs,ys,color = (z,0,0))
legns.append("z = %f"%(z))
ax.legends(legns)
plt.show()
But I have too many graphs and the legends will cover the graph. I'd rather have a colorbar indicating the value of z corresponding to the color. I can't find anything like that in the galery and all my attempts do deal with the colorbar failed. Apparently I must create a collection of plots before trying to add a colorbar.
Is there an easy way to do this? Thanks.
EDIT (clarification):
I wanted to do something like this:
import matplotlib.pyplot as plt
import matplotlib.cm as cm
fig = plt.figure()
ax = fig.add_subplot(111)
mycmap = cm.hot
# suppose mydata is a list of tuples containing (xs, ys, z)
# where xs and ys are lists of x's and y's and z is a number between 0 and 1
plots = []
for(xs,ys,z) in mydata:
pl = ax.plot(xs,ys,color = mycmap(z))
plots.append(pl)
fig.colorbar(plots)
plt.show()
But this won't work according to the Matplotlib reference because a list of plots is not a "mappable", whatever this means.
I've created an alternative plot function using LineCollection:
def myplot(ax,xs,ys,zs, cmap):
plot = lc([zip(x,y) for (x,y) in zip(xs,ys)], cmap = cmap)
plot.set_array(array(zs))
x0,x1 = amin(xs),amax(xs)
y0,y1 = amin(ys),amax(ys)
ax.add_collection(plot)
ax.set_xlim(x0,x1)
ax.set_ylim(y0,y1)
return plot
xs and ys are lists of lists of x and y coordinates and zs is a list of the different conditions to colorize each line. It feels a bit like a cludge though... I thought that there would be a more neat way to do this. I like the flexibility of the plt.plot() function.

(I know this is an old question but...) Colorbars require a matplotlib.cm.ScalarMappable, plt.plot produces lines which are not scalar mappable, therefore, in order to make a colorbar, we are going to need to make a scalar mappable.
Ok. So the constructor of a ScalarMappable takes a cmap and a norm instance. (norms scale data to the range 0-1, cmaps you have already worked with and take a number between 0-1 and returns a color). So in your case:
import matplotlib.pyplot as plt
sm = plt.cm.ScalarMappable(cmap=my_cmap, norm=plt.normalize(min=0, max=1))
plt.colorbar(sm)
Because your data is in the range 0-1 already, you can simplify the sm creation to:
sm = plt.cm.ScalarMappable(cmap=my_cmap)
EDIT: For matplotlib v1.2 or greater the code becomes:
import matplotlib.pyplot as plt
sm = plt.cm.ScalarMappable(cmap=my_cmap, norm=plt.normalize(vmin=0, vmax=1))
# fake up the array of the scalar mappable. Urgh...
sm._A = []
plt.colorbar(sm)
EDIT: For matplotlib v1.3 or greater the code becomes:
import matplotlib.pyplot as plt
sm = plt.cm.ScalarMappable(cmap=my_cmap, norm=plt.Normalize(vmin=0, vmax=1))
# fake up the array of the scalar mappable. Urgh...
sm._A = []
plt.colorbar(sm)
EDIT: For matplotlib v3.1 or greater simplifies to:
import matplotlib.pyplot as plt
sm = plt.cm.ScalarMappable(cmap=my_cmap, norm=plt.Normalize(vmin=0, vmax=1))
plt.colorbar(sm)

Here's one way to do it while still using plt.plot(). Basically, you make a throw-away plot and get the colorbar from there.
import matplotlib as mpl
import matplotlib.pyplot as plt
min, max = (-40, 30)
step = 10
# Setting up a colormap that's a simple transtion
mymap = mpl.colors.LinearSegmentedColormap.from_list('mycolors',['blue','red'])
# Using contourf to provide my colorbar info, then clearing the figure
Z = [[0,0],[0,0]]
levels = range(min,max+step,step)
CS3 = plt.contourf(Z, levels, cmap=mymap)
plt.clf()
# Plotting what I actually want
X=[[1,2],[1,2],[1,2],[1,2]]
Y=[[1,2],[1,3],[1,4],[1,5]]
Z=[-40,-20,0,30]
for x,y,z in zip(X,Y,Z):
# setting rgb color based on z normalized to my range
r = (float(z)-min)/(max-min)
g = 0
b = 1-r
plt.plot(x,y,color=(r,g,b))
plt.colorbar(CS3) # using the colorbar info I got from contourf
plt.show()
It's a little wasteful, but convenient. It's also not very wasteful if you make multiple plots as you can call plt.colorbar() without regenerating the info for it.

Here is a slightly simplied example inspired by the top answer given by Boris and Hooked (Thanks for the great idea!):
1. Discrete colorbar
Discrete colorbar is more involved, because colormap generated by mpl.cm.get_cmap() is not a mappable image needed as a colorbar() argument. A dummie mappable needs to generated as shown below:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
n_lines = 5
x = np.linspace(0, 10, 100)
y = np.sin(x[:, None] + np.pi * np.linspace(0, 1, n_lines))
c = np.arange(1, n_lines + 1)
cmap = mpl.cm.get_cmap('jet', n_lines)
fig, ax = plt.subplots(dpi=100)
# Make dummie mappable
dummie_cax = ax.scatter(c, c, c=c, cmap=cmap)
# Clear axis
ax.cla()
for i, yi in enumerate(y.T):
ax.plot(x, yi, c=cmap(i))
fig.colorbar(dummie_cax, ticks=c)
plt.show();
This will produce a plot with a discrete colorbar:
2. Continuous colorbar
Continuous colorbar is less involved, as mpl.cm.ScalarMappable() allows us to obtain an "image" for colorbar().
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
n_lines = 5
x = np.linspace(0, 10, 100)
y = np.sin(x[:, None] + np.pi * np.linspace(0, 1, n_lines))
c = np.arange(1, n_lines + 1)
norm = mpl.colors.Normalize(vmin=c.min(), vmax=c.max())
cmap = mpl.cm.ScalarMappable(norm=norm, cmap=mpl.cm.jet)
cmap.set_array([])
fig, ax = plt.subplots(dpi=100)
for i, yi in enumerate(y.T):
ax.plot(x, yi, c=cmap.to_rgba(i + 1))
fig.colorbar(cmap, ticks=c)
plt.show();
This will produce a plot with a continuous colorbar:
[Side note] In this example, I personally don't know why cmap.set_array([]) is necessary (otherwise we'd get error messages). If someone understand the principles under the hood, please comment :)

As other answers here do try to use dummy plots, which is not really good style, here is a generic code for a
Discrete colorbar
A discrete colorbar is produced in the same way a continuous colorbar is created, just with a different Normalization. In this case a BoundaryNorm should be used.
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors
n_lines = 5
x = np.linspace(0, 10, 100)
y = np.sin(x[:, None] + np.pi * np.linspace(0, 1, n_lines))
c = np.arange(1., n_lines + 1)
cmap = plt.get_cmap("jet", len(c))
norm = matplotlib.colors.BoundaryNorm(np.arange(len(c)+1)+0.5,len(c))
sm = plt.cm.ScalarMappable(norm=norm, cmap=cmap)
sm.set_array([]) # this line may be ommitted for matplotlib >= 3.1
fig, ax = plt.subplots(dpi=100)
for i, yi in enumerate(y.T):
ax.plot(x, yi, c=cmap(i))
fig.colorbar(sm, ticks=c)
plt.show()