Related
I need to plot a HEATMAP in python using x, y, z data from the excel file.
All the values of z are 1 except at (x=5,y=5). The plot should be red at point (5,5) and blue elsewhere. But I am getting false alarms which need to be removed. The COLORMAP I have used is 'jet'
X=[0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,6,6,6,6,6,6,6,6,6,6,7,7,7,7,7,7,7,7,7,7,8,8,8,8,8,8,8,8,8,8,9,9,9,9,9,9,9,9,9,9]
Y=[0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9]
Z=[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,9,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]
Code I have used is:
import matplotlib.pyplot as plt
import numpy as np
from numpy import ravel
from scipy.interpolate import interp2d
import pandas as pd
import matplotlib as mpl
excel_data_df = pd.read_excel('test.xlsx')
X= excel_data_df['x'].tolist()
Y= excel_data_df['y'].tolist()
Z= excel_data_df['z'].tolist()
x_list = np.array(X)
y_list = np.array(Y)
z_list = np.array(Z)
# f will be a function with two arguments (x and y coordinates),
# but those can be array_like structures too, in which case the
# result will be a matrix representing the values in the grid
# specified by those arguments
f = interp2d(x_list,y_list,z_list,kind="linear")
x_coords = np.arange(min(x_list),max(x_list))
y_coords = np.arange(min(y_list),max(y_list))
z= f(x_coords,y_coords)
fig = plt.imshow(z,
extent=[min(x_list),max(x_list),min(y_list),max(y_list)],
origin="lower", interpolation='bicubic', cmap= 'jet', aspect='auto')
# Show the positions of the sample points, just to have some reference
fig.axes.set_autoscale_on(False)
#plt.scatter(x_list,y_list,400, facecolors='none')
plt.xlabel('X Values', fontsize = 15, va="center")
plt.ylabel('Y Values', fontsize = 15,va="center")
plt.title('Heatmap', fontsize = 20)
plt.tight_layout()
plt.show()
For your ease you can also use the X, Y, Z arrays instead of reading excel file.
The result that I am getting is:
Here you can see dark blue regions at (5,0) and (0,5). These are the FALSE ALARMS I am getting and I need to REMOVE these.
I am probably doing some beginner's mistake. Grateful to anyone who points it out. Regards
There are at least three problems in your example:
x_coords and y_coords are not properly resampled;
the interpolation z does to fill in the whole grid leading to incorrect output;
the output is then forced to be plotted on the original grid (extent) that add to the confusion.
Leading to the following interpolated results:
On what you have applied an extra smoothing with imshow.
Let's create your artificial input:
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(0, 11)
y = np.arange(0, 11)
X, Y = np.meshgrid(x, y)
Z = np.ones(X.shape)
Z[5,5] = 9
Depending on how you want to proceed, you can simply let imshow smooth your signal by interpolation:
fig, axe = plt.subplots()
axe.imshow(Z, origin="lower", cmap="jet", interpolation='bicubic')
And you are done, simple and efficient!
If you aim to do it by yourself, then choose the interpolant that suits you best and resample on a grid with a higher resolution:
interpolant = interpolate.interp2d(x, y, Z.ravel(), kind="linear")
xlin = np.linspace(0, 10, 101)
ylin = np.linspace(0, 10, 101)
zhat = interpolant(xlin, ylin)
fig, axe = plt.subplots()
axe.imshow(zhat, origin="lower", cmap="jet")
Have a deeper look on scipy.interpolate module to pick up the best interpolant regarding your needs. Notice that all methods does not expose the same interface for imputing parameters. You may need to reshape your data to use another objects.
MCVE
Here is a complete example using the trial data generated above. Just bind it to your excel columns:
# Flatten trial data to meet your requirement:
x = X.ravel()
y = Y.ravel()
z = Z.ravel()
# Resampling on as square grid with given resolution:
resolution = 11
xlin = np.linspace(x.min(), x.max(), resolution)
ylin = np.linspace(y.min(), y.max(), resolution)
Xlin, Ylin = np.meshgrid(xlin, ylin)
# Linear multi-dimensional interpolation:
interpolant = interpolate.NearestNDInterpolator([r for r in zip(x, y)], z)
Zhat = interpolant(Xlin.ravel(), Ylin.ravel()).reshape(Xlin.shape)
# Render and interpolate again if necessary:
fig, axe = plt.subplots()
axe.imshow(Zhat, origin="lower", cmap="jet", interpolation='bicubic')
Which renders as expected:
Background:
I have a list_of_x_and_y_list that contains x and y values which looks like:
[[(44800, 14888), (132000, 12500), (40554, 12900)], [(None, 193788), (101653, 78880), (3866, 160000)]]
I have another data_name_list ["data_a","data_b"] so that
"data_a" = [(44800, 14888), (132000, 12500), (40554, 12900)]
"data_b" = [(None, 193788), (101653, 78880), (3866, 160000)]
The len of list_of_x_and_y_list / or len of data_name_list is > 20.
Question:
How can I create a scatter plot for each item (being the same colour) in the data_name_list?
What I have tried:
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax = plt.axes(facecolor='#FFFFFF')
prop_cycle = plt.rcParams['axes.prop_cycle']
colors = prop_cycle.by_key()['color']
print(list_of_x_and_y_list)
for x_and_y_list, data_name, color in zip(list_of_x_and_y_list, data_name_list, colors):
for x_and_y in x_and_y_list,:
print(x_and_y)
x, y = x_and_y
ax.scatter(x, y, label=data_name, color=color) # "label=data_name" creates
# a huge list as a legend!
# :(
plt.title('Matplot scatter plot')
plt.legend(loc=2)
file_name = "3kstc.png"
fig.savefig(file_name, dpi=fig.dpi)
print("Generated: {}".format(file_name))
The Problem:
The legend appears to be a very long list, which I don't know how to rectify:
Relevant Research:
Matplotlib scatterplot
Scatter Plot
Scatter plot in Python using matplotlib
The reason you get a long repeated list as a legend is because you are providing each point as a separate series, as matplotlib does not automatically group your data based on the labels.
A quick fix is to iterate over the list and zip together the x-values and the y-values of each series as two tuples, so that the x tuple contains all the x-values and the y tuple the y-values.
Then you can feed these tuples to the plt.plot method together with the labels.
I felt that the names list_of_x_and_y_list were uneccessary long and complicated, so in my code I've used shorter names.
import matplotlib.pyplot as plt
data_series = [[(44800, 14888), (132000, 12500), (40554, 12900)],
[(None, 193788), (101653, 78880), (3866, 160000)]]
data_names = ["data_a","data_b"]
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax = plt.axes(facecolor='#FFFFFF')
prop_cycle = plt.rcParams['axes.prop_cycle']
colors = prop_cycle.by_key()['color']
for data, data_name, color in zip(data_series, data_names, colors):
x,y = zip(*data)
ax.scatter(x, y, label=data_name, color=color)
plt.title('Matplot scatter plot')
plt.legend(loc=1)
To only get one entry per data_name, you should add data_name only once as a label. The rest of the calls should go with label=None.
The simplest you can achieve this using the current code, is to set data_name to None at the end of the loop:
from matplotlib import pyplot as plt
from random import randint
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax.set_facecolor('#FFFFFF')
# create some random data, suppose the sublists have different lengths
list_of_x_and_y_list = [[(randint(1000, 4000), randint(2000, 5000)) for col in range(randint(2, 10))]
for row in range(10)]
data_name_list = list('abcdefghij')
colors = plt.rcParams['axes.prop_cycle'].by_key()['color']
for x_and_y_list, data_name, color in zip(list_of_x_and_y_list, data_name_list, colors):
for x_and_y in x_and_y_list :
x, y = x_and_y
ax.scatter(x, y, label=data_name, color=color)
data_name = None
plt.legend(loc=2)
plt.show()
Some things can be simplified, making the code 'more pythonic', for example:
for x_and_y in x_and_y_list :
x, y = x_and_y
can be written as:
for x, y in x_and_y_list:
Another issue, is that with a lot of data calling scatter for every point could be rather slow. All the x and y belonging to the same list can be plotted together. For example using list comprehension:
for x_and_y_list, data_name, color in zip(list_of_x_and_y_list, data_name_list, colors):
xs = [x for x, y in x_and_y_list]
ys = [y for x, y in x_and_y_list]
ax.scatter(xs, ys, label=data_name, color=color)
scatter could even get a list of colors per point, but plotting all the points in one go, wouldn't allow for labels per data_name.
Very often, numpy is used to store numerical data. This has some advantages, such as vectorization for quick calculations. With numpy the code would look like:
import numpy as np
for x_and_y_list, data_name, color in zip(list_of_x_and_y_list, data_name_list, colors):
xys = np.array(x_and_y_list)
ax.scatter(xys[:,0], xys[:,1], label=data_name, color=color)
I would like to contour plot a function, f(x,y), against x and x-y. The spacing in the y grid is not the same as the x grid, so x-y is 2 dimensional, whereas x is one-dimensional.
I do not know how to set up the grids. The function, tricontourf, can handle non-uniform grids, but only it seems, if both the axes are one-dimensional. contour can handle matrices, but only for f(x,y), whereas I need one of the axes to be a matrix.
Pseudocode would look like the following:
import matplotlib.pyplot as plt
def twoDfunction(x,y):
return x + y # my function is more complicated than this
xaxis = np.linspace(0,10,100)
yaxis = np.linspace(0,10,22)
xminusyaxis = np.subtract(xaxis,yaxis)
functionsurfacevalues = twoDfunction(xaxis,yaxis)
fig =plt.figure(figsize=(10,10),dpi=300,facecolor='w')
ax1 = plt.subplot(111)
ax1.tricontourf(xaxis, xminusyaxis, functionsurfacevalues)
I would like the pseudocode to plot functionsurfacevalues versus x and xminusy.
What you need to do is create your grid using np.meshgrid() and then plot a contour or contourf plot.np.meshgrid will make irregular grids based on whatever you give it. You do not need a surface plot because your data isn't really a surface.
The main problem you are having is that because your x and y axis are different lengths, you can't subtract them. Otherwise the solution is easy and you can follow the following code.
import matplotlib.pyplot as plt
def twoDfunction(x,y):
return (x + y) # my function is more complicated than this
xaxis = np.linspace(0,10,100)
yaxis = np.linspace(0,5,100)
xminusyaxis = np.subtract(xaxis,yaxis)
xx,yy = np.meshgrid(xaxis,xminusyaxis)
fig =plt.figure(figsize=(10,10),dpi=300,facecolor='w')
ax1 = plt.subplot(111)
ax1.contourf(xx, yy, twoDfunction(xx,yy))
plt.show()
I am trying to have a polar heatmap appear incrementally. I want the plot to grow by adding a deltasector to the existing plot. The same maximal radius is always used.
For now I replot the old data as well, but that is only because I do not know how to add to the existing plot.
How do I add z values for the new angle to an existing heatmap?
The accepted answer here gives shows how to plot a polar heatmap:
Polar heatmaps in python
In the code below the z values are calculated as a function of the r and th. My situation is however that I read the values from a file instead.
How would I add them to the heatmap?
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
import time
fig = plt.figure()
ax = Axes3D(fig)
angle = 0
rad = np.linspace(0, 5, 100)
d_angle = np.pi/100
while angle < np.pi:
azm = np.linspace(0, angle, 100)
r, th = np.meshgrid(rad, azm)
z = r/5
plt.subplot(projection="polar")
plt.pcolormesh(th, r, z)
plt.plot(azm, r, color='k', ls='none')
plt.grid()
plt.ion()
plt.show()
plt.pause(0.0001)
plt.clf()
angle += d_angle
I do not know where to start. Any pointers to docs? Or other advices?
You can retrieve the data from a plot by looking into ax.lines. Add a "gid" to your curve like so plt.plot(azm, r, color='k', ls='none', gid="a custom name") then we have a little work to do:
def append_data_to_curve(ax, gid):
for line in ax.lines: # Check every curve.
if line.get_gid() == "a custom name": # If the one you seek is found:
X = line.get_xdata() # Get its X and Y data.
Y = line.get_ydata()
X.append(x) # Add the new point (x,y) you want.
Y.append(y)
line.set_xdata(X) # Put back the modified list as curve data.
line.set_ydata(Y)
You can call this function for every iteration of a loop and add a single new point by giving it its (x,y) coordinates.
I am trying to make a discrete colorbar for a scatterplot in matplotlib
I have my x, y data and for each point an integer tag value which I want to be represented with a unique colour, e.g.
plt.scatter(x, y, c=tag)
typically tag will be an integer ranging from 0-20, but the exact range may change
so far I have just used the default settings, e.g.
plt.colorbar()
which gives a continuous range of colours. Ideally i would like a set of n discrete colours (n=20 in this example). Even better would be to get a tag value of 0 to produce a gray colour and 1-20 be colourful.
I have found some 'cookbook' scripts but they are very complicated and I cannot think they are the right way to solve a seemingly simple problem
You can create a custom discrete colorbar quite easily by using a BoundaryNorm as normalizer for your scatter. The quirky bit (in my method) is making 0 showup as grey.
For images i often use the cmap.set_bad() and convert my data to a numpy masked array. That would be much easier to make 0 grey, but i couldnt get this to work with the scatter or the custom cmap.
As an alternative you can make your own cmap from scratch, or read-out an existing one and override just some specific entries.
import numpy as np
import matplotlib as mpl
import matplotlib.pylab as plt
fig, ax = plt.subplots(1, 1, figsize=(6, 6)) # setup the plot
x = np.random.rand(20) # define the data
y = np.random.rand(20) # define the data
tag = np.random.randint(0, 20, 20)
tag[10:12] = 0 # make sure there are some 0 values to show up as grey
cmap = plt.cm.jet # define the colormap
# extract all colors from the .jet map
cmaplist = [cmap(i) for i in range(cmap.N)]
# force the first color entry to be grey
cmaplist[0] = (.5, .5, .5, 1.0)
# create the new map
cmap = mpl.colors.LinearSegmentedColormap.from_list(
'Custom cmap', cmaplist, cmap.N)
# define the bins and normalize
bounds = np.linspace(0, 20, 21)
norm = mpl.colors.BoundaryNorm(bounds, cmap.N)
# make the scatter
scat = ax.scatter(x, y, c=tag, s=np.random.randint(100, 500, 20),
cmap=cmap, norm=norm)
# create a second axes for the colorbar
ax2 = fig.add_axes([0.95, 0.1, 0.03, 0.8])
cb = plt.colorbar.ColorbarBase(ax2, cmap=cmap, norm=norm,
spacing='proportional', ticks=bounds, boundaries=bounds, format='%1i')
ax.set_title('Well defined discrete colors')
ax2.set_ylabel('Very custom cbar [-]', size=12)
I personally think that with 20 different colors its a bit hard to read the specific value, but thats up to you of course.
You could follow this example below or the newly added example in the documentation
#!/usr/bin/env python
"""
Use a pcolor or imshow with a custom colormap to make a contour plot.
Since this example was initially written, a proper contour routine was
added to matplotlib - see contour_demo.py and
http://matplotlib.sf.net/matplotlib.pylab.html#-contour.
"""
from pylab import *
delta = 0.01
x = arange(-3.0, 3.0, delta)
y = arange(-3.0, 3.0, delta)
X,Y = meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = Z2 - Z1 # difference of Gaussians
cmap = cm.get_cmap('PiYG', 11) # 11 discrete colors
im = imshow(Z, cmap=cmap, interpolation='bilinear',
vmax=abs(Z).max(), vmin=-abs(Z).max())
axis('off')
colorbar()
show()
which produces the following image:
The above answers are good, except they don't have proper tick placement on the colorbar. I like having the ticks in the middle of the color so that the number -> color mapping is more clear. You can solve this problem by changing the limits of the matshow call:
import matplotlib.pyplot as plt
import numpy as np
def discrete_matshow(data):
# get discrete colormap
cmap = plt.get_cmap('RdBu', np.max(data) - np.min(data) + 1)
# set limits .5 outside true range
mat = plt.matshow(data, cmap=cmap, vmin=np.min(data) - 0.5,
vmax=np.max(data) + 0.5)
# tell the colorbar to tick at integers
cax = plt.colorbar(mat, ticks=np.arange(np.min(data), np.max(data) + 1))
# generate data
a = np.random.randint(1, 9, size=(10, 10))
discrete_matshow(a)
To set a values above or below the range of the colormap, you'll want to use the set_over and set_under methods of the colormap. If you want to flag a particular value, mask it (i.e. create a masked array), and use the set_bad method. (Have a look at the documentation for the base colormap class: http://matplotlib.org/api/colors_api.html#matplotlib.colors.Colormap )
It sounds like you want something like this:
import matplotlib.pyplot as plt
import numpy as np
# Generate some data
x, y, z = np.random.random((3, 30))
z = z * 20 + 0.1
# Set some values in z to 0...
z[:5] = 0
cmap = plt.get_cmap('jet', 20)
cmap.set_under('gray')
fig, ax = plt.subplots()
cax = ax.scatter(x, y, c=z, s=100, cmap=cmap, vmin=0.1, vmax=z.max())
fig.colorbar(cax, extend='min')
plt.show()
This topic is well covered already but I wanted to add something more specific : I wanted to be sure that a certain value would be mapped to that color (not to any color).
It is not complicated but as it took me some time, it might help others not lossing as much time as I did :)
import matplotlib
from matplotlib.colors import ListedColormap
# Let's design a dummy land use field
A = np.reshape([7,2,13,7,2,2], (2,3))
vals = np.unique(A)
# Let's also design our color mapping: 1s should be plotted in blue, 2s in red, etc...
col_dict={1:"blue",
2:"red",
13:"orange",
7:"green"}
# We create a colormar from our list of colors
cm = ListedColormap([col_dict[x] for x in col_dict.keys()])
# Let's also define the description of each category : 1 (blue) is Sea; 2 (red) is burnt, etc... Order should be respected here ! Or using another dict maybe could help.
labels = np.array(["Sea","City","Sand","Forest"])
len_lab = len(labels)
# prepare normalizer
## Prepare bins for the normalizer
norm_bins = np.sort([*col_dict.keys()]) + 0.5
norm_bins = np.insert(norm_bins, 0, np.min(norm_bins) - 1.0)
print(norm_bins)
## Make normalizer and formatter
norm = matplotlib.colors.BoundaryNorm(norm_bins, len_lab, clip=True)
fmt = matplotlib.ticker.FuncFormatter(lambda x, pos: labels[norm(x)])
# Plot our figure
fig,ax = plt.subplots()
im = ax.imshow(A, cmap=cm, norm=norm)
diff = norm_bins[1:] - norm_bins[:-1]
tickz = norm_bins[:-1] + diff / 2
cb = fig.colorbar(im, format=fmt, ticks=tickz)
fig.savefig("example_landuse.png")
plt.show()
I have been investigating these ideas and here is my five cents worth. It avoids calling BoundaryNorm as well as specifying norm as an argument to scatter and colorbar. However I have found no way of eliminating the rather long-winded call to matplotlib.colors.LinearSegmentedColormap.from_list.
Some background is that matplotlib provides so-called qualitative colormaps, intended to use with discrete data. Set1, e.g., has 9 easily distinguishable colors, and tab20 could be used for 20 colors. With these maps it could be natural to use their first n colors to color scatter plots with n categories, as the following example does. The example also produces a colorbar with n discrete colors approprately labelled.
import matplotlib, numpy as np, matplotlib.pyplot as plt
n = 5
from_list = matplotlib.colors.LinearSegmentedColormap.from_list
cm = from_list(None, plt.cm.Set1(range(0,n)), n)
x = np.arange(99)
y = x % 11
z = x % n
plt.scatter(x, y, c=z, cmap=cm)
plt.clim(-0.5, n-0.5)
cb = plt.colorbar(ticks=range(0,n), label='Group')
cb.ax.tick_params(length=0)
which produces the image below. The n in the call to Set1 specifies
the first n colors of that colormap, and the last n in the call to from_list
specifies to construct a map with n colors (the default being 256). In order to set cm as the default colormap with plt.set_cmap, I found it to be necessary to give it a name and register it, viz:
cm = from_list('Set15', plt.cm.Set1(range(0,n)), n)
plt.cm.register_cmap(None, cm)
plt.set_cmap(cm)
...
plt.scatter(x, y, c=z)
I think you'd want to look at colors.ListedColormap to generate your colormap, or if you just need a static colormap I've been working on an app that might help.