We Keep Coding

Legends are printing twice when calling matplotlib subplots

I'm writing a code in matplotlib to print multiple histograms under a subplot grid, however, when I call the fig.legend() function at the end, legends from each plot are printing twice. Any guidance on how to resolve this issue would be greatly appreciated:)
Here is my code:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('darkgrid')
def get_cmap(n, name='hsv'):
return plt.cm.get_cmap(name, n)
def isSqrt(n):
sq_root = int(np.sqrt(n))
return (sq_root*sq_root) == n
df = pd.read_csv('mpg.csv')
df2 = pd.read_csv('dm_office_sales.csv')
df['miles'] = df2['salary']
numericClassifier = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']
newdf = df.select_dtypes(numericClassifier)
columns = newdf.columns.tolist()
n = len(columns)
cmap = get_cmap(n)
if(isSqrt(n)):
nrows = ncols = int(np.sqrt(n))
else:
ncols = int(np.sqrt(n))
for i in range(ncols,50):
if ncols*i >= n:
nrows = i
break
else:
pass
fig,ax = plt.subplots(nrows,ncols)
count = 0
print(nrows,ncols)
for i in range(0,nrows,1):
for j in range(0,ncols,1):
print('ncols = {}'.format(j),'nrows = {}'.format(i),'count = {}'.format(count))
if count<=n-1:
plt_new = sns.histplot(df[columns[count]],ax=ax[i,j],facecolor=cmap(count),kde=True,edgecolor='black',label=df[columns[count]].name)
patches = plt_new.get_children()
for patch in patches:
patch.set_alpha(0.8)
color = patches[0].get_facecolor()
ax[i,j].set_xlabel('{}'.format(df[columns[count]].name))
ax[i,j].xaxis.label.set_fontsize(10)
ax[i,j].xaxis.label.set_fontname('ariel')
ax[i,j].set(xlabel=None)
ax[i,j].tick_params(axis='y', labelsize=8)
count+=1
else:
break
for i in range(0,nrows,1):
for j in range(0,ncols,1):
if not ax[i,j].has_data():
fig.delaxes(ax[i,j])
else:
pass
plt.suptitle('Histograms').set_fontname('ariel')
plt.tight_layout()
fig.legend(loc='upper right')
plt.show()
Here is the output:

sns.histplot seems to create two bar containers. First a dummy one, and then the real one. (Tested with seaborn 0.12.1; this might work different in other versions.) Therefore, the label gets assigned to both the dummy and the real bar container. A workaround would be to remove the label of the dummy bar container.
Here is the adapted code. Seaborn's mpg dataset is used to have an easily reproducible example. As the first and last color of the hls colormap is red, get_cmap(n + 1) ensures n different colors are choosen. Some superfluous code has been removed.
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
def get_cmap(n, name='hsv'):
return plt.cm.get_cmap(name, n)
sns.set_style('darkgrid')
df = sns.load_dataset('mpg')
numericClassifier = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']
newdf = df.select_dtypes(numericClassifier)
columns = newdf.columns.tolist()
n = len(columns)
cmap = get_cmap(n + 1)
ncols = int(np.sqrt(n))
nrows = int(np.ceil(n / ncols))
fig, ax = plt.subplots(nrows, ncols)
count = 0
print(nrows, ncols)
for i in range(0, nrows):
for j in range(0, ncols):
if count < n:
# print('ncols = {j}; nrows = {i}; count = {count}')
sns.histplot(df[columns[count]], ax=ax[i, j], facecolor=cmap(count), kde=True, edgecolor='black',
label=df[columns[count]].name)
ax[i, j].containers[0].set_label('') # seaborn seems to create a dummy bar container, remove its label
for patch in ax[i, j].get_children():
patch.set_alpha(0.8)
ax[i, j].tick_params(axis='y', labelsize=8)
count += 1
for i in range(0, nrows):
for j in range(0, ncols):
if not ax[i, j].has_data():
fig.delaxes(ax[i, j])
plt.suptitle('Histograms').set_fontname('ariel')
fig.legend(loc='upper right')
plt.tight_layout()
plt.subplots_adjust(right=0.75) # make extra space for the legend
plt.show()
Upon further investigation, it seems the dummy bar container isn't created when sns.histplot is called with color= instead of facecolor=.
The code could also be written a bit more "pythonic". This means a.o. trying to avoid repeating code and explicit indices. To achieve this, zip is an important helper. Alongside avoiding repetition, the code becomes shorter and easier to modify. Once you get used to it, it becomes easier to read and to reason about.
The main part could look like e.g.:
fig, axs = plt.subplots(nrows=nrows, ncols=ncols)
for column, ax, color in zip(columns, axs.flat, cmap(range(n))):
# using `color=` instead of `facecolor=` seems to avoid the creating of dummy bars
sns.histplot(df[column], ax=ax, color=color, kde=True, edgecolor='black', label=column)
for patch in ax.get_children():
patch.set_alpha(0.8)
ax.tick_params(axis='y', labelsize=8)
for ax in axs.flat:
if not ax.has_data():
fig.delaxes(ax)

Matplotlib clearing old axis labels when re-plotting data

I've got a script wherein I have two functions, makeplots() which makes a figure of blank subplots arranged in a particular way (depending on the number of subplots to be drawn), and drawplots() which is called later, drawing the plots (obviously). The functions are posted below.
The script does some analysis of data for a given number of 'targets' (which can number anywhere from one to nine) and creates plots of the linear regression for each target. When there are multiple targets, this works great. But when there's a single target (i.e. a single 'subplot' in the figure), the Y-axis label overlaps the axis itself (this does not happen when there are multiple targets).
Ideally, each subplot would be square, no labels would overlap, and it would work the same for one target as for multiple targets. But when I tried to decrease the size of the y-axis label and shift it over a bit, it appears that the actual axes object was drawn over the previously blank, square plot (whose axes ranged from 0 to 1), and the old tick mark labels are still visible. I'd like to have those old tick marks removed when calling drawplots(). I've tried changing the subplot_kw={} arguments in makeplots, as well as removing ax.set_aspect('auto') from drawplots, both to no avail. Note that there are also screenshots of various behaviors at the end, also.
def makeplots(targets, active=actwindow):
def rowcnt(y):
rownumb = y//3 if (y%3 == 0) else y//3+1
return rownumb
def colcnt(x):
if x <= 3: colnumb = x
elif x == 4: colnumb = 2
else: colnumb = 3
return colnumb
numsubs = len(targets)
numrow, numcol = rowcnt(numsubs), colcnt(numsubs)
if numsubs >= 1:
if numsubs == 1:
fig, axs = plt.subplots(num='LOD-95 Plots', nrows=1, ncols=1, figsize = [8,6], subplot_kw={'adjustable': 'box', 'aspect': 1})
# changed 'box' to 'datalim'
fig, axs = plt.subplots(num='LOD-95 Plots', nrows=numrow, ncols=numcol, figsize = [numcol*6,numrow*6], subplot_kw={'adjustable': 'box', 'aspect': 1})
fig.text(0.02, 0.5, 'Probit score\n $(\sigma + 5)$', va='center', rotation='vertical', size='16')
else:
raise ValueError(f'Error generating plots [call: makeplots({targets},{active}) - invalid numsubs value]')
axs = np.ravel(axs)
for i, ax in enumerate(axs):
ax.set_title(f'Limit of Detection: {targets[i]}', size=11)
ax.grid()
return fig, axs
and
def drawplots(ax, dftables, color1, color2):
y = dftables.probit
y95 = 6.6448536269514722
logreg = False
regfun = lambda m, x, b : (m*x) + b
regq = scipy.stats.linregress(dftables.qty,y)
regl = scipy.stats.linregress(dftables.log_qty,y)
if regq.rvalue**2 >= regl.rvalue**2:
regression = regq
x_label = 'input quantity'
x = dftables.qty
elif regq.rvalue**2 < regl.rvalue**2:
regression = regl
x_label = '$log_{10}$(input quantity)'
x = dftables.log_qty
logreg = True
slope, intercept, r = regression.slope, regression.intercept, regression.rvalue
r2 = r**2
lod = (y95-intercept)/slope
xr = [0, lod*1.2]
yr = [intercept, regfun(slope, xr[1], intercept)]
regeqn = "y = "+str(f"{slope:.2e}")+"x + "+str(f"{intercept:.3f}")
if logreg:
lodstr = f'log(LOD) = {lod:.2f}' if lod <= 100 else f'log(LOD) = {lod:.2e}'
elif not logreg:
lodstr = f'LOD = {lod:.2f}' if lod <= 100 else f'LOD = {lod:.2e}'
# raise ValueError(f'Error raised calling drawplots()')
ax.set_xlabel(x_label, fontweight='bold')
ax.plot(xr, yr, color=color1, linestyle='--') # plot regression line
ax.plot(lod, y95, marker='D', color=color2, markersize=7) # plot point for LoD
ax.plot(xr, [y95,y95], color=color2, linestyle=':') # horizontal crosshair
ax.plot([lod,lod],[0, 7.1], color=color2, linestyle=':') # vertical crosshair
ax.scatter(x, y, s=81, color=color1, marker='.') # actual data points
ax.annotate(f"{lodstr}", xy=(lod,0.1),
xytext=(0.9*lod,0.5), fontsize=8, arrowprops = dict(facecolor='black', headlength=5, width=2, headwidth=5))
ax.set_aspect('auto')
ax.set_xlim(left=0)
ax.set_ylim(bottom=0)
ax.plot()
if logreg: lod = 10 ** lod
return r2, lod, regeqn, logreg
The context they're called in:
fig, axs = makeplots(targets)
wg.SetForegroundWindow(actwindow)
with open(outName, 'a+') as f:
print(f"Lower Limit of Detection Analysis on {dt} at {tm}\n", file=f)
for i, tars in enumerate(targets):
data[tars] = stripThousands(data[tars])
# logans = checkyn(f"Analyze {tars} using log10(concentration/quantity)? (y/n): ")
for idx, val in enumerate(qtys):
tables[i,idx,2] = hitrate(val,data,tars)
tables[i,idx,3] = norm.ppf(tables[i,idx,2])+5
printtables[tars] = pd.DataFrame(tables[i,:,:], columns=["qty","log_qty","probability","probit"])
# construct dataframes from np.arrays and drop
# rows with infinite probit values:
dftables[tars] = pd.DataFrame(tables[i,:,:], columns=["qty","log_qty","probability","probit"])
dftables[tars].probit.replace([np.inf,-np.inf],np.nan, inplace=True)
dftables[tars].dropna(inplace=True)
r2, lod, eqn, logreg = drawplots(axs[i], dftables[tars], cbcolors[i], cbcolors[i+5])

You should clear the axes in each iteration using pyplot.cla().
You posted a lot of code, so I'm not 100% sure of the best location to place it in your code, but the general idea is to clear the axes before each new plot.
Here is a minimal demo without cla():
x = [[1,2,3], [3,2,1]]
fig, ax = plt.subplots()
for index, data in enumerate(x):
ax.plot(data)
And with cla():
for index, data in enumerate(x):
ax.cla()
ax.plot(data)

matplotlib: change axis ticks of ndim histogram plotted with seaborn.heatmap

Motivation:
I'm trying to visualize a dataset of many n-dimensional vectors (let's say i have 10k vectors with n=300 dimensions). What i'd like to do is calculate a histogram for each of the n dimensions and plot it as a single line in a bins*n heatmap.
So far i've got this:
import numpy as np
import matplotlib
from matplotlib import pyplot as plt
%matplotlib inline
import seaborn as sns
# sample data:
vectors = np.random.randn(10000, 300) + np.random.randn(300)
def ndhist(vectors, bins=500):
limits = (vectors.min(), vectors.max())
hists = []
dims = vectors.shape[1]
for dim in range(dims):
h, bins = np.histogram(vectors[:, dim], bins=bins, range=limits)
hists.append(h)
hists = np.array(hists)
fig = plt.figure(figsize=(16, 9))
sns.heatmap(hists)
axes = fig.gca()
axes.set(ylabel='dimensions', xlabel='values')
print(dims)
print(limits)
ndhist(vectors)
This generates the following output:
300
(-6.538069472429366, 6.52159540162285)
Problem / Question:
How can i change the axes ticks?
for the y-axis i'd like to simply change this back to matplotlib's default, so it picks nice ticks like 0, 50, 100, ..., 250 (bonus points for 299 or 300)
for the x-axis i'd like to convert the shown bin indices into the bin (left) boundaries, then, as above, i'd like to change this back to matplotlib's default selection of some "nice" ticks like -5, -2.5, 0, 2.5, 5 (bonus points for also including the actual limits -6.538, 6.522)
Own solution attempts:
I've tried many things like the following already:
def ndhist_axlabels(vectors, bins=500):
limits = (vectors.min(), vectors.max())
hists = []
dims = vectors.shape[1]
for dim in range(dims):
h, bins = np.histogram(vectors[:, dim], bins=bins, range=limits)
hists.append(h)
hists = np.array(hists)
fig = plt.figure(figsize=(16, 9))
sns.heatmap(hists, yticklabels=False, xticklabels=False)
axes = fig.gca()
axes.set(ylabel='dimensions', xlabel='values')
#plt.xticks(np.linspace(*limits, len(bins)), bins)
plt.xticks(range(len(bins)), bins)
axes.xaxis.set_major_locator(matplotlib.ticker.AutoLocator())
plt.yticks(range(dims+1), range(dims+1))
axes.yaxis.set_major_locator(matplotlib.ticker.AutoLocator())
print(dims)
print(limits)
ndhist_axlabels(vectors)
As you can see however, the axes labels are pretty wrong. My guess is that the extent or limits are somewhere stored in the original axis, but lost when switching back to the AutoLocator. Would greatly appreciate a nudge in the right direction.

Maybe you're overthinking this. To plot image data, one can use imshow and get the ticking and formatting for free.
import numpy as np
from matplotlib import pyplot as plt
# sample data:
vectors = np.random.randn(10000, 300) + np.random.randn(300)
def ndhist(vectors, bins=500):
limits = (vectors.min(), vectors.max())
hists = []
dims = vectors.shape[1]
for dim in range(dims):
h, _ = np.histogram(vectors[:, dim], bins=bins, range=limits)
hists.append(h)
hists = np.array(hists)
fig, ax = plt.subplots(figsize=(16, 9))
extent = [limits[0], limits[-1], hists.shape[0]-0.5, -0.5]
im = ax.imshow(hists, extent=extent, aspect="auto")
fig.colorbar(im)
ax.set(ylabel='dimensions', xlabel='values')
ndhist(vectors)
plt.show()

If you read the docs, you will notice that the xticklabels/yticklabels arguments are overloaded, such that if you provide an integer instead of a string, it will interpret the argument as xtickevery/ytickevery and place ticks only at the corresponding locations. So in your case, seaborn.heatmap(hists, yticklabels=50) fixes your y-axis problem.
Regarding your xtick labels, I would simply provide them explictly:
xtickevery = 50
xticklabels = ['{:.1f}'.format(b) if ii%xtickevery == 0 else '' for ii, b in enumerate(bins)]
sns.heatmap(hists, yticklabels=50, xticklabels=xticklabels)

Finally came up with a version that works for me for now and uses AutoLocator based on some simple linear mapping...
def ndhist(vectors, bins=1000, title=None):
t = time.time()
limits = (vectors.min(), vectors.max())
hists = []
dims = vectors.shape[1]
for dim in range(dims):
h, bs = np.histogram(vectors[:, dim], bins=bins, range=limits)
hists.append(h)
hists = np.array(hists)
fig = plt.figure(figsize=(16, 12))
sns.heatmap(
hists,
yticklabels=50,
xticklabels=False
)
axes = fig.gca()
axes.set(
ylabel=f'dimensions ({dims} total)',
xlabel=f'values (min: {limits[0]:.4g}, max: {limits[1]:.4g}, {bins} bins)',
title=title,
)
def val_to_idx(val):
# calc (linearly interpolated) index loc for given val
return bins*(val - limits[0])/(limits[1] - limits[0])
xlabels = [round(l, 3) for l in limits] + [
v for v in matplotlib.ticker.AutoLocator().tick_values(*limits)[1:-1]
]
# drop auto-gen labels that might be too close to limits
d = (xlabels[4] - xlabels[3])/3
if (xlabels[1] - xlabels[-1]) < d:
del xlabels[-1]
if (xlabels[2] - xlabels[0]) < d:
del xlabels[2]
xticks = [val_to_idx(val) for val in xlabels]
axes.set_xticks(xticks)
axes.set_xticklabels([f'{l:.4g}' for l in xlabels])
plt.show()
print(f'histogram generated in {time.time() - t:.2f}s')
ndhist(np.random.randn(100000, 300), bins=1000, title='randn')
Thanks to Paul for his answer giving me the idea.
If there's an easier or more elegant solution, i'd still be interested though.

How to set fixed spaces between ticks in maptlotlib

I am preparing a graph of latency percentile results. This is my pd.DataFrame looks like:
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
result = pd.DataFrame(np.random.randint(133000, size=(5,3)), columns=list('ABC'), index=[99.0, 99.9, 99.99, 99.999, 99.9999])
I am using this function (commented lines are different pyplot methods I have already tried to achieve my goal):
def plot_latency_time_bar(result):
ind = np.arange(4)
means = []
stds = []
for index, row in result.iterrows():
means.append(np.mean([row[0]//1000, row[1]//1000, row[2]//1000]))
stds.append(np .std([row[0]//1000, row[1]//1000, row[2]//1000]))
plt.bar(result.index.values, means, 0.2, yerr=stds, align='center')
plt.xlabel('Percentile')
plt.ylabel('Latency')
plt.xticks(result.index.values)
# plt.xticks(ind, ('99.0', '99.9', '99.99', '99.999', '99.99999'))
# plt.autoscale(enable=False, axis='x', tight=False)
# plt.axis('auto')
# plt.margins(0.8, 0)
# plt.semilogx(basex=5)
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)
fig = plt.gcf()
fig.set_size_inches(15.5, 10.5)
And here is the figure:
As you can see bars for all percentiles above 99.0 overlaps and are completely unreadable. I would like to set some fixed space between ticks to have a same space between all of them.

Since you're using pandas, you can do all this from within that library:
means = df.mean(axis=1)/1000
stds = df.std(axis=1)/1000
means.plot.bar(yerr=stds, fc='b')
# Make some room for the x-axis tick labels
plt.subplots_adjust(bottom=0.2)
plt.show()

Not wishing to take anything away from xnx's answer (which is the most elegant way to do things given that you're working in pandas, and therefore likely the best answer for you) but the key insight you're missing is that, in matplotlib, the x positions of the data you're plotting and the x tick labels are independent things. If you say:
nominalX = np.arange( 1, 6 ) ** 2
y = np.arange( 1, 6 ) ** 4
positionalX = np.arange(len(y))
plt.bar( positionalX, y ) # graph y against the numbers 1..n
plt.gca().set(xticks=positionalX + 0.4, xticklabels=nominalX) # ...but superficially label the X values as something else
then that's different from tying positions to your nominal X values:
plt.bar( nominalX, y )
Note that I added 0.4 to the x position of the ticks, because that's half the default width of the bars bar( ..., width=0.8 )—so the ticks end up in the middle of the bar.

Trying to set y axis labels and ticks aligned to 2D faces

This is my plot:
If I were to draw your attention to the axis labelled 'B' you'll see that everything is not as it should be.
The plots was produced using this:
def newPoly3D(self):
from matplotlib.cm import autumn
# This passes a pandas dataframe of shape (data on rows x 4 columns)
df = self.loadData()
fig = plt.figure(figsize=(10,10))
ax = fig.gca(projection='3d')
vels = [1.42,1.11,0.81,0.50]
which_joints = df.columns
L = len(which_joints)
dmin,dmax = df.min().min(),df.max().max()
dix = df.index.values
offset=-5
for i,j in enumerate(which_joints):
ax.add_collection3d(plt.fill_between(dix,df[j],
dmin,
lw=1.5,
alpha=0.3/float(i+1.),
facecolor=autumn(i/float(L))),
zs=vels[i],
zdir='y')
ax.grid(False)
ax.set_xlabel('A')
ax.set_xlim([0,df.index[-1]])
ax.set_xticks([])
ax.xaxis.set_ticklabels([])
ax.set_axis_off
ax.set_ylabel('B')
ax.set_ylim([0.4, max(vels)+0.075])
ax.set_yticks(vels)
ax.tick_params(direction='out', pad=10)
ax.set_zlabel('C')
ax.set_zlim([dmin,dmax])
ax.xaxis.labelpad = -10
ax.yaxis.labelpad = 15
ax.zaxis.labelpad = 15
# Note the inversion of the axis
plt.gca().invert_yaxis()
First I want to align the ticks on the yaxis (labelled B) with each coloured face. As you can see they are now offset slightly down.
Second I want to align the yaxis tick labels with the above, as you cans see they are currently very much offset downwards. I do not know why.
EDIT:
Here is some example data; each column represents one coloured face on the above plot.
-13.216256 -7.851065 -9.965357 -25.502654
-13.216253 -7.851063 -9.965355 -25.502653
-13.216247 -7.851060 -9.965350 -25.502651
-13.216236 -7.851052 -9.965342 -25.502647
-13.216214 -7.851038 -9.965324 -25.502639
-13.216169 -7.851008 -9.965289 -25.502623
-13.216079 -7.850949 -9.965219 -25.502592
-13.215900 -7.850830 -9.965078 -25.502529
Here we are again, with a simpler plot, reproduced with this data:
k = 10
df = pd.DataFrame(np.array([range(k),
[x + 1 for x in range(k)],
[x + 4 for x in range(k)],
[x + 9 for x in range(k)]]).T,columns=list('abcd'))
If you want to try this with the above function, comment out the df line in the function and change its argument as so def newPoly3D(df): so that you can pass the the test df above.