We Keep Coding

Refining dataframe-based stacked bar plot in Python

I am trying to create a stacked bar chart using PyCharm.
I am using matplotlib to explore at fullest its potentialities for simple data visualization.
My original code is for a group chart bar that displays cycle time for different teams. Such information come from a dataframe. The chart also includes autolabeling function (i.e. the height of each bar = continuous variable).
I am trying to convert such group bar chart in a stacked bar chart. The code below needs to be improved because of two factors:
labels for variables have too many decimals: this issue did not occur for the grouped bar chart. The csv file and the derived data frame weren't altered. I am struggling to understand if and where to use round command. I guess the issue is either related to the autolabeling function, because datatype used is float (I need to see at least 1 decimal).
data labels are displaced: as the auto labeling function was created for separated bars, the labels always matched the distance I wanted (based on the vertical offset). Unfortunately I did not figure out how to make sure that this distance is rather centered (see my example, the value for funnel time is at the height of squad time instead, and vice-versa). By logic, the issue should be that the height of each variable is defined ahead (see rects3 in the code, value of bottom) but I don't know how to reflect this in my auto-labeling variable.
The question is what exactly in the code must be altered in order to have the values of cycle time centered?
The code (notes for you are marked in bold):
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
'''PART 1 - Preprocess data -----------------------------------------------'''
#Directory or link of my CSV. This can be used also if you want to use API.
csv1 = r"C:\Users\AndreaPaviglianiti\Downloads\CT_Plot_DF.csv"
#Create and read dataframe. This is just to check the DF before plotting
df = pd.read_csv(csv1, sep=',', engine= 'python')
print(df, '\n')
#Extract columns as lists
squads = df['Squad_Name'].astype('str') #for our horizontal axis
funnel = df['Funnel_Time'].astype('float')
squadt = df['Squad_Time'].astype('float')
wait = df['Waiting_Time'].astype('float')
Here I tried to set the rounding but without success
'''PART 2 - Create the Bar Plot / Chart ----------------------------------'''
x = np.arange(len(squads)) #our labels on x will be the squads' names
width = 0.2 # the width of the bars. The bigger value, the larger bars
distance = 0.2
#Create objects that will be used as subplots (fig and ax).
#Each "rects" is the visualization of a yn value. first witdth is distance between X values,
# the second is the real width of bars.
fig, ax = plt.subplots()
rects1 = ax.bar(x, funnel, width, color='red', label='Funnel Time')
rects2 = ax.bar(x, squadt, width, color='green', bottom=funnel, label='Squad Time')
rects3 = ax.bar(x, wait, width, bottom=funnel+squadt, color='purple', label='Waiting Time')
# Add some text for labels, title and custom x-axis tick labels, etc.
ax.set_ylabel('Mean Cycle Time (h)')
ax.set_xlabel('\n Squads')
ax.set_title("Squad's Cycle Time Comparison in Dec-2020 \n (in mean Hours)")
ax.set_xticks(x)
ax.set_xticklabels(squads)
ax.legend()
fig.align_xlabels() #align labels to columns
# The function to display values above the bars
def autolabel(rects):
"""Attach a text label above each bar in *rects*, displaying its height."""
for rect in rects:
height = rect.get_height()
ax.annotate('{}'.format(height),
xy=(rect.get_x() + rect.get_width()/2, height),
xytext=(0, 3), # 3 points vertical offset
textcoords="offset points",
ha='center', va='bottom')
Here I tried to change xytext="center" but I get error, I am supposed to use coordinates only or is there an alternative to change the position from the height to the center?
#We will label only the most recent information. To label both add to the code "autolabel(rects1)"
autolabel(rects1)
autolabel(rects2)
autolabel(rects3)
fig.tight_layout()
'''PART 3 - Execute -------------------------------------------------------'''
plt.show()
Thank you for the help!

Get automatically coordinates of subplots in order to set them for automatic positioning of legend

I tried in a first time to set manually the location for the main legend of a main plot produced by Getdist tool.
The plot below represents the 1/2 sigma confidence levels coming from a covariance matrix with joint distributions. It is produced by Getdist tool.
The main routine that generates this plot is :
# g.settings
g = plots.get_subplot_plotter()
g.settings.figure_legend_frame = True
g.settings.legend_fontsize = 21
g.triangle_plot([matrix1, matrix2],
names,
filled = True,
contour_colors = ['darkblue','red'],
line_args = [{'lw':2, 'color':'darkblue'},
{'lw':2, 'color':'red'}]
)
g.add_legend(['Opt. Flat. No Gamma. - cross - standard situation - Criterion taking into accound a = 200',\
'Pess. Flat. No Gamma. - cross - standard situation - Criterion taking into account a = 300' ],\
bbox_to_anchor = [1.5, 8.5])
The value 1.5 seems to correspond to the x-coordinate (width) 8.5 corresponds to the y-coordinate of legend (height).
Now, I would like to automatically do this process instead of set manual at each time the position of the legend.
I want the top right of the legend to be positioned at the top border of the first left upper box (just at the level of top line border below the "1sigma ± 0.0012" title).
I would like also the legend to be pushed to the right of the figure (up to the right border for the right lower box of the figure: identified by sigma8 "1sigma ± 0.001" title ; Caution: I want it located before the 1.0 and 0.0 xticks, just at the x-coordinate of right line border).
Here what I tried to get the global coordinates (the entire plot) of the top border for this left upper box :
# First, get y coordinates of top border for first Likelihood
box1 = g.subplots[0,0]
box1_coords = box1._position.bounds
print('box1_coords = ', box1_coords)
and I get at the execution the following values :
box1_coords = (0.125, 0.7860975609756098, 0.09451219512195125, 0.09390243902439022)
As you can see, these values seem to be normalized, so I don't know how to handle if I want to insert these values into :
bbox_to_anchor = [box1_coords[0], box1_coords[1]]
This line of code produces a bad position for legend, as expected.
So, how can I manage to automatically assign the good values for bbox_to_anchor to get what I want (y-coordinate at level of top border of left upper box identified by the "1sigma ± 0.0012" title) and pushed on the right side up to the right border of right lower box (x-coordinate identified by sigma8 with "1sigma ± 0.001" title)?
Update 1
I tried to adapt them to my case, but issue still occurs. Here what I have done:
# g.settings
g = plots.get_subplot_plotter()
# get the max y position of the top left axis
top_left_plot = g.subplots[0,0].axes.get_position().ymax
# get the max x position of the bottom right axis
# it is -1 to reference the last plot
bottom_right_plot = g.subplots[-1,-1].axes.get_position().xmax
I don't know why the values of top_left_plot and bottom_right_plot are not the good ones.
I think that subplots[0,0] (for top y-coordinate of legend) refers to the top left subplot and subplots[-1,-1] to the bottom right subplot (for right x-coordinate of legend) but considering this, it doesn't work.
For example :
# g.settings
g = plots.get_subplot_plotter()
# Call triplot
g.triangle_plot([matrix1, matrix2],
names,
filled = True,
legend_labels = [],
contour_colors = ['darkblue','red'],
line_args = [{'lw':2, 'color':'darkblue'},
{'lw':2, 'color':'red'}])
g.add_legend(['Opt. Flat. No Gamma. - cross - standard situation - Criterion taking into accound a = 200',
'Pess. Flat. No Gamma. - cross - standard situation - Criterion taking into account a = 300'],
legend_loc='upper right',
bbox_to_anchor=(bottom_right_plot, top_left_plot)
)
I get :
legend_coords y_max, x_max 0.88 0.9000000000000001
I can't understand why these values (seems to be comprised between 0.0 and 1.0) are not taken into account with g.add_legend.
With #mullinscr's solution, I get the following figure :
If I take for the coordinates of legend position by forcing :
top_left_plot = 8.3
bottom_right_plot = 1.0
This looks like to the first figure of this post. But these 2 values are not comprised between 0.0 and 1.0 like it should.
Update 2
#mullinscr, thanks, I have followed your update and always get an issue. If I apply the same code snippet directly in my script, i.e :
g.add_legend(['An example legend - item 1'],
legend_loc='upper right', # we want to specify the location of this point
bbox_to_anchor=(bottom_right_plot, top_left_plot),
bbox_transform=plt.gcf().transFigure, # this is the x and y co-ords we extracted above
borderaxespad=0, # this means there is no padding around the legend
edgecolor='black')
Then I get the following figure :
As you can see, the coordinates are not really what is really expected : a slight shift on x-coordinate and y-coordinate is present.
If I apply your code snippet for my legend text, I get:
I give you the link of my entire script, this will be easier maybe for you to see an error compared what is expected:
My entire Python script

Here's my answer, it's the same as #scleronomic's answer, but I'll point out some of the things that tripped me up when figuring this out.
Below is my code to reproduce your desired positioning, I've tried to create the same subplot layout to you, but through matplotlib not getdist -- same result though.
As you discovered, the trick lies in extracting the position data of the first and last axes (top-left and lower-right), to reference from. The bounds method that you used returns: the x0, y0, width and height of the axes (see the docs). However what we want is the maximum x and y, so that our legend corner is in the top right. This can be achieved by using the xmax and ymax method:
axes.flatten()[-1].get_position().xmax
axes.flatten()[0].get_position().ymax
Once we have these variables they can be passed into the bbox_to_anchor parameter of the add_legend() function, as you did. However, if we use loc='upper right' too, it tells matplotlib that we want the upper right of the legend to be pinned to this top right corner. Finally, we need to set borderaxespad=0 otherwise the legend won't sit exactly where we want it to due to default padding.
Please see my example code below, as well as the resulting picture. Note that I left the top-right plot in so you can see that it lines up correctly.
Also, note that as #scleronomic says, calls to plt.tight_layout() etc will mess this positioning up.
import matplotlib.pyplot as plt
# code to layout subplots as in your example:
# --------------------------------------------
g, axes = plt.subplots(nrows=7, ncols=7,figsize=(10,10))
unwanted = [1,2,3,4,5,9,10,11,12,13,17,
18,19,20,25,26,27,33,34,41]
for ax in axes.flatten():
ax.plot([1,2], [1,2])
ax.set_yticks([])
ax.set_xticks([])
for n, ax in enumerate(axes.flatten()):
if n in unwanted:
ax.remove()
# Code to answer your question:
# ------------------------------
# get the max y position of the top left axis
top_left_plot = axes.flatten()[0].get_position().ymax
# get the max x position of the bottom right axis
# it is -1 to reference the last plot
bottom_right_plot = axes.flatten()[-1].get_position().xmax
# I'm using the matplotlib so it is g.legend() not g.add_legend
# but g.add_legend() should work the same as it is a wrapper of th ematplotlib func
g.legend(['Opt. Flat. No Gamma. - cross - standard situation - Criterion taking into accound a = 200',
'Pess. Flat. No Gamma. - cross - standard situation - Criterion taking into account a = 300'],
loc='upper right', # we want to specify the location of this point
bbox_to_anchor=(bottom_right_plot, top_left_plot), # this is the x and y co-ords we extracted above
borderaxespad=0, # this means there is no padding around the legend
edgecolor='black') # I set it black for this example
plt.show()
Update
After #youpilat13's comments, I investigated some more and installed getdist to try and recreate with that tool. Initially I got the same results, but found the trick is, unlike if you were making this in matplotlib, you have to transform the legend's coordinates to figure coordinates. This can be achieved with the following in the g.add_legend() call:
bbox_transform=plt.gcf().transFigure
Here is a complete example:
import getdist
from getdist import plots, MCSamples
from getdist.gaussian_mixtures import GaussianND
covariance = [[0.001**2, 0.0006*0.05, 0], [0.0006*0.05, 0.05**2, 0.2**2], [0, 0.2**2, 2**2]]
mean = [0.02, 1, -2]
gauss=GaussianND(mean, covariance)
g = plots.get_subplot_plotter(subplot_size=3)
g.triangle_plot(gauss,filled=True)
top_left_plot = g.subplots.flatten()[0].get_position().ymax
bottom_right_plot = g.subplots.flatten()[-1].get_position().xmax
g.add_legend(['An example legend - item 1'],
legend_loc='upper right', # we want to specify the location of this point
bbox_to_anchor=(bottom_right_plot, top_left_plot),
bbox_transform=plt.gcf().transFigure, # this is the x and y co-ords we extracted above
borderaxespad=0, # this means there is no padding around the legend
edgecolor='black')
And the resulting image:

It basically works as you described. The bboxes (xmin, ymin, width, height) of the axes are given in fractions of the figure and plt.legend() uses the same format so the two are compatible. By setting the upper right corner of the legend to the corner defined by the outer most axes you get the clean layout and don't have to worry about the exact size of the legend.
import matplotlib.pyplot as plt
n = 4
# Create the subplot grid
# Alternative: fig, ax = plt.subplots(n, n); ax[i, j].remove() for j > i
fig = plt.figure()
gs = fig.add_gridspec(nrows=n, ncols=n)
ax = np.zeros((n, n), dtype=object)
for i in range(n):
for j in range(n):
if j <= i:
ax[i, j] = fig.add_subplot(gs[i, j])
# add this to make the position of the legend easier to spot
ax[0, -1] = fig.add_subplot(gs[0, -1])
# Plot some dummy data
ax[0, 0].plot(range(10), 'b-o', label='Dummy Label 4x4')
# Set the legend
y_max = ax[0][0].get_position().ymax
x_max = ax[-1][-1].get_position().xmax
fig.legend(loc='upper right', bbox_to_anchor=(x_max, y_max),
borderaxespad=0)
plt.show()
Some pitfalls could be using the Constrained Layout
or using bbox_inches='tight' when saving the file as both screw up the position of the legend in unexpected ways.
For some more examples of legend placement I found this collection
very helpful.

Drawing a Table at the positions of yticks

I have been trouble with trying to find a way to display a 3 element list in the form of a table. What I actually care about is drawing the table. I would like to draw a 1by3 table for each ylabel in a plot.
Below is what I have so far. If I can get each Table instance to show up, I will have what I want. Right now a reference to a table appears and I'm not sure why. If you actually look in the center left where the reference locations appear, you can see one 1by3 table.
Is it possible using matplotlib to generate a new table for each ylabel? The table info is directly related to each row in the bar graph, so it's important that I have a way that they line up.
The number of rows in the bar graph is dynamic, so creating 1 table for the whole figure and trying to dynamically line up the rows with the corresponding bar graph is a difficult problem.
# initialize figure
fig = plt.figure()
gs = gridspec.GridSpec(1, 2, width_ratios=[2, 1])
fig.set_size_inches(18.5, 10.5)
ax = fig.add_subplot(gs[0])
#excluded bar graph code
# create ylabels
for row in range(1,len(data)):
ylabel = [str(data[row][0]),str(data[row][1]),str(data[row][2])]
ylabels.append(ylabel)
#attempting to put each ylabel in a 1by3 table to display
pos = np.arange(0.5,10.5,0.5)
axTables = [None] * len(ylabels)
for x in range(0,len(ylabels)):
axTables[x] = fig.add_subplot(gs[0])
ylabels[x] = axTables[x].table(cellText=[ylabels[x]], loc='left')
locsy, labelsy = plt.yticks(pos,ylabels)

First, yticks will expect text as input, it cannot handle other objects. Second, a table needs to sit within an axes.
So in order to get a table at the position of a tick(label) the idea can be to create an axes at the position of a y ticklabel. An option is the use of mpl_toolkits.axes_grid1.inset_locator.inset_axes. Now the difficulty is that this axes needs to be positionned in data coordinates along the y axis, and in figure (or pixel-) coorinates in the horizontal direction. For this one might use a blended transform. The inset_axes allows to give the width and height as absolute measures (in inches) or in relative, which is handy because we can set the width of the axes to 100% of the bounding box, while the height is still some absolute value (we don't want the axes height to depend on the data coordinates!).
In the following a function ax_at_posy creates such axes.
The table would then sit tight inside the axes, such that all columns are the same width. One would still need to make sure the same fontsize is used throughout.
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1.inset_locator import inset_axes
import matplotlib.transforms as mtrans
# General helper function to create an axes at the position of a yticklabel
def ax_at_posy(y, ax=None, width=0.3, leftspace=0.08, height=0.2):
ax = ax or plt.gca()
trans = mtrans.blended_transform_factory(ax.figure.transFigure, ax.transData)
axins = inset_axes(ax, "100%", height,
bbox_to_anchor=(leftspace, y, width-leftspace, 0.05),
bbox_transform=trans, loc="center right", borderpad=0.8)
axins.tick_params(left=False, bottom=False, labelleft=False, labelbottom=False)
axins.axis("off")
return axins
fig, ax = plt.subplots()
fig.subplots_adjust(left=0.4)
ax.scatter(np.random.rand(30), np.random.randint(7, size=30), c=np.random.rand(30))
get_data = lambda i: "".join(np.random.choice(list("abcdefgxyzt0"), size=i+2))
data = np.vectorize(get_data)(np.random.randint(2,6,size=(7,3)))
for i, row in enumerate(data):
axi = ax_at_posy(i, ax=ax, width=0.4)
tab = axi.table(cellText=[list(row)], loc='center', bbox=(0,0,1,1))
tab.auto_set_font_size(False)
tab.set_fontsize(9)
plt.setp(tab.get_celld().values(), linewidth=0.72)
plt.show()

How to remove a particular grid line corresponding to a custom xtick on a log scale axis?

I'd like to remove the vertical grid line corresponding to the custom xtick (displayed at x = 71 in the below picture). I could remove a horizontal grid line corresponding to the ytick 701 in the below picture by using a hack : since I have no minor tick on the y axis, I defined the custom ytick corresponding to the line that points toward the maximum and crosses the y axis as a minor tick, and then I disabled grid lines for minor ticks on the y axis. Unfortunately I cannot use the same hack on the x axis without disabling the grid lines of the minor ticks and that's something I'd like to avoid at all costs.
Below is a not so minimal albeit still WE.
There are many things I don't understand, the 2 majors are why does
locs, labels = plt.xticks()
not return the locs and labels that are plotted and why I don't get xticks labels displayed as 10^x where x = 0, 1, 2 and 3 but that's outside the scope of the original question.
import matplotlib.pyplot as plt
plt.grid(True)
import numpy as np
# Generate data
x_data = np.arange(1, 1000 , 10)
y_data = np.random.lognormal(1e-5, 3, len(x_data))
y_max = max(y_data)
# plot
plt.xscale('log')
import math
ratio_log = math.log(x_data[np.argmax(y_data)]) / math.log(max(x_data)) # I need to do this in order to plot a horizontal red dashed line that points to the max and do not extend any further.
plt.axhline(y=y_max, xmin=0, xmax = ratio_log, color='r', linestyle='--') # horizontal line pointing to the max y value.
axes = plt.gca()
axes.set_xlim([1, max(x_data)]) # Limits for the x axis.
# custom ticks and labels
# First yticks because I'm able to achieve what I seek
axes.set_yticks([int(y_max)], minor=True) # Sets the custom ytick as a minor one.
from matplotlib.ticker import FormatStrFormatter
axes.yaxis.set_minor_formatter(FormatStrFormatter("%.0f"))
axes.yaxis.grid(False, which='minor') # Removes minor yticks grid. Since I only have my custom yticks as a minor one, this will disable only the grid line corresponding to that ytick. That's a hack.
import matplotlib.ticker as plticker
loc = plticker.MultipleLocator(base=y_max / 3.3) # this locator puts ticks at regular intervals. I ensure the y axis ticks look ok.
axes.yaxis.set_major_locator(loc)
# Now xticks. I'm having a lot of difficulty here, unable to remove the grid of a particular custom xticks.
locs, labels = plt.xticks() # Strangely, this doesn't return the locs and labels that are plotted. There are indeed 2 values that aren't displayed in the plot, here 1.00000000e-01 and 1.00000000e+04. I've got to remove them before I can append my custom loc and label.
# This means that if I do: plt.xticks(locs, labels) right here, it would enlarge both the lower and upper limits on the x axis... I fail to see how that's intuitive or useful at all. Might this be a bug?
locs = np.append(locs[1:-1], np.asarray(x_data[np.argmax(y_data)])) # One of the ugliest hack I have ever seen... to get correct ticks and labels.
labels = (str(int(loc)) for loc in locs) # Just visuals to get integers on the axis.
plt.xticks(locs, labels) # updates the xticks and labels.
plt.plot((x_data[np.argmax(y_data)], x_data[np.argmax(y_data)]), (0, y_max), 'r--') # vertical line that points to the max. Non OO way to do it, so a bad way.
plt.plot(x_data, y_data)
plt.savefig('grid_prob.png')
plt.close()
Example picture below (the code outputs a different picture each time it is executed, but the problem appears in all pictures).

Credit for the idea goes to #ImportanceOfBeingErnest to whom I am extremely grateful.
I removed the grid with
axes.xaxis.grid(False, which='both')
, then I added a grid correspond to each xtick except the custom one with the following loop:
for loc in locs[1:-1]:
if loc != x_data[np.argmax(y_data)]:
plt.axvline(x=loc, color = 'grey', linestyle = '-', linewidth = 0.4)
Insert this code just before the line
plt.xticks(locs, labels) # updates the xticks and labels.
Example of output picture below.

How do I plot more than one set of bars per axis on a bar plot in python?

I currently use the align=’edge’ parameter and positive/negative widths in pyplot.bar() to plot the bar data of one metric to each axis. However, if I try to plot a second set of data to one axis, it covers the first set. Is there a way for pyplot to automatically space this data correctly?
lns3 = ax[1].bar(bucket_df.index,bucket_df.original_revenue,color='c',width=-0.4,align='edge')
lns4 = ax[1].bar(bucket_df.index,bucket_df.revenue_lift,color='m',bottom=bucket_df.original_revenue,width=-0.4,align='edge')
lns5 = ax3.bar(bucket_df.index,bucket_df.perc_first_priced,color='grey',width=0.4,align='edge')
lns6 = ax3.bar(bucket_df.index,bucket_df.perc_revenue_lift,color='y',width=0.4,align='edge')
This is what it looks like when I show the plot:
The data shown in yellow completely covers the data in grey. I'd like it to be shown next to the grey data.
Is there any easy way to do this? Thanks!

The first argument to the bar() plotting method is an array of the x-coordinates for your bars. Since you pass the same x-coordinates they will all overlap. You can get what you want by staggering the bars by doing something like this:
x = np.arange(10) # define your x-coordinates
width = 0.1 # set a width for your plots
offset = 0.15 # define an offset to separate each set of bars
fig, ax = plt.subplots() # define your figure and axes objects
ax.bar(x, y1) # plot the first set of bars
ax.bar(x + offset, y2) # plot the second set of bars
Since you have a few sets of data to plot, it makes more sense to make the code a bit more concise (assume y_vals is a list containing the y-coordinates you'd like to plot, bucket_df.original_revenue, bucket_df.revenue_lift, etc.). Then your plotting code could look like this:
for i, y in enumerate(y_vals):
ax.bar(x + i * offset, y)
If you want to plot more sets of bars you can decrease the width and offset accordingly.