We Keep Coding

How to fix overlapping matplotlib y-axis tick labels or autoscale the plot? [duplicate]

I am trying to make a series of matplotlib plots that plot timespans for different classes of objects. Each plot has an identical x-axis and plot elements like a title and a legend. However, which classes appear in each plot differs; each plot represents a different sampling unit, each of which only contains only a subset of all the possible classes.
I am having a lot of trouble determining how to set the figure and axis dimensions. The horizontal size should always remain the same, but the vertical dimensions need to be scaled to the number of classes represented in that sampling unit. The distance between each entry on the y-axis should be equal for every plot.
It seems that my difficulties lie in the fact that I can set the absolute size (in inches) of the figure with plt.figure(figsize=(w,h)), but I can only set the size of the axis with relative dimensions (e.g., fig.add_axes([0.3,0.05,0.6,0.85]) which leads to my x-axis labels getting cut off when the number of classes is small.
Here is an MSPaint version of what I'd like to get vs. what I'm getting.
Here is a simplified version of the code I have used. Hopefully it is enough to identify the problem/solution.
import pandas as pd
import matplotlib.pyplot as plt
import pylab as pl
from matplotlib import collections as mc
from matplotlib.lines import Line2D
import seaborn as sns
# elements for x-axis
start = 1
end = 6
interval = 1 # x-axis tick interval
xticks = [x for x in range(start, end, interval)] # create x ticks
# items needed for legend construction
lw_bins = [0,10,25,50,75,90,100] # bins for line width
lw_labels = [3,6,9,12,15,18] # line widths
def make_proxy(zvalue, scalar_mappable, **kwargs):
color = 'black'
return Line2D([0, 1], [0, 1], color=color, solid_capstyle='butt', **kwargs)
for line_subset in data:
# create line collection for this run through loop
lc = mc.LineCollection(line_subset)
# create plot and set properties
sns.set(style="ticks")
sns.set_context("notebook")
############################################################
# I think the problem lies here
fig = plt.figure(figsize=(11, len(line_subset.index)*0.25))
ax = fig.add_axes([0.3,0.05,0.6,0.85])
############################################################
ax.add_collection(lc)
ax.set_xlim(left=start, right=end)
ax.set_xticks(xticks)
ax.xaxis.set_ticks_position('bottom')
ax.margins(0.05)
sns.despine(left=True)
ax.set_yticks(line_subset['order_y'])
ax.set(yticklabels=line_subset['ylabel'])
ax.tick_params(axis='y', length=0)
# legend
proxies = [make_proxy(item, lc, linewidth=item) for item in lw_labels]
leg = ax.legend(proxies, ['0-10%', '10-25%', '25-50%', '50-75%', '75-90%', '90-100%'], bbox_to_anchor=(1.0, 0.9),
loc='best', ncol=1, labelspacing=3.0, handlelength=4.0, handletextpad=0.5, markerfirst=True,
columnspacing=1.0)
for txt in leg.get_texts():
txt.set_ha("center") # horizontal alignment of text item
txt.set_x(-23) # x-position
txt.set_y(15) # y-position

You can start by defining the margins on top and bottom in units of inches. Having a fixed unit of one data unit in inches allows to calculate how large the final figure should be.
Then dividing the margin in inches by the figure height gives the relative margin in units of figure size, this can be supplied to the figure using subplots_adjust, given the subplots has been added with add_subplot.
A minimal example:
import numpy as np
import matplotlib.pyplot as plt
data = [np.random.rand(i,2) for i in [2,5,8,4,3]]
height_unit = 0.25 #inch
t = 0.15; b = 0.4 #inch
for d in data:
height = height_unit*(len(d)+1)+t+b
fig = plt.figure(figsize=(5, height))
ax = fig.add_subplot(111)
ax.set_ylim(-1, len(d))
fig.subplots_adjust(bottom=b/height, top=1-t/height, left=0.2, right=0.9)
ax.barh(range(len(d)),d[:,1], left=d[:,0], ec="k")
ax.set_yticks(range(len(d)))
plt.show()

How to remove a particular grid line corresponding to a custom xtick on a log scale axis?

I'd like to remove the vertical grid line corresponding to the custom xtick (displayed at x = 71 in the below picture). I could remove a horizontal grid line corresponding to the ytick 701 in the below picture by using a hack : since I have no minor tick on the y axis, I defined the custom ytick corresponding to the line that points toward the maximum and crosses the y axis as a minor tick, and then I disabled grid lines for minor ticks on the y axis. Unfortunately I cannot use the same hack on the x axis without disabling the grid lines of the minor ticks and that's something I'd like to avoid at all costs.
Below is a not so minimal albeit still WE.
There are many things I don't understand, the 2 majors are why does
locs, labels = plt.xticks()
not return the locs and labels that are plotted and why I don't get xticks labels displayed as 10^x where x = 0, 1, 2 and 3 but that's outside the scope of the original question.
import matplotlib.pyplot as plt
plt.grid(True)
import numpy as np
# Generate data
x_data = np.arange(1, 1000 , 10)
y_data = np.random.lognormal(1e-5, 3, len(x_data))
y_max = max(y_data)
# plot
plt.xscale('log')
import math
ratio_log = math.log(x_data[np.argmax(y_data)]) / math.log(max(x_data)) # I need to do this in order to plot a horizontal red dashed line that points to the max and do not extend any further.
plt.axhline(y=y_max, xmin=0, xmax = ratio_log, color='r', linestyle='--') # horizontal line pointing to the max y value.
axes = plt.gca()
axes.set_xlim([1, max(x_data)]) # Limits for the x axis.
# custom ticks and labels
# First yticks because I'm able to achieve what I seek
axes.set_yticks([int(y_max)], minor=True) # Sets the custom ytick as a minor one.
from matplotlib.ticker import FormatStrFormatter
axes.yaxis.set_minor_formatter(FormatStrFormatter("%.0f"))
axes.yaxis.grid(False, which='minor') # Removes minor yticks grid. Since I only have my custom yticks as a minor one, this will disable only the grid line corresponding to that ytick. That's a hack.
import matplotlib.ticker as plticker
loc = plticker.MultipleLocator(base=y_max / 3.3) # this locator puts ticks at regular intervals. I ensure the y axis ticks look ok.
axes.yaxis.set_major_locator(loc)
# Now xticks. I'm having a lot of difficulty here, unable to remove the grid of a particular custom xticks.
locs, labels = plt.xticks() # Strangely, this doesn't return the locs and labels that are plotted. There are indeed 2 values that aren't displayed in the plot, here 1.00000000e-01 and 1.00000000e+04. I've got to remove them before I can append my custom loc and label.
# This means that if I do: plt.xticks(locs, labels) right here, it would enlarge both the lower and upper limits on the x axis... I fail to see how that's intuitive or useful at all. Might this be a bug?
locs = np.append(locs[1:-1], np.asarray(x_data[np.argmax(y_data)])) # One of the ugliest hack I have ever seen... to get correct ticks and labels.
labels = (str(int(loc)) for loc in locs) # Just visuals to get integers on the axis.
plt.xticks(locs, labels) # updates the xticks and labels.
plt.plot((x_data[np.argmax(y_data)], x_data[np.argmax(y_data)]), (0, y_max), 'r--') # vertical line that points to the max. Non OO way to do it, so a bad way.
plt.plot(x_data, y_data)
plt.savefig('grid_prob.png')
plt.close()
Example picture below (the code outputs a different picture each time it is executed, but the problem appears in all pictures).

Credit for the idea goes to #ImportanceOfBeingErnest to whom I am extremely grateful.
I removed the grid with
axes.xaxis.grid(False, which='both')
, then I added a grid correspond to each xtick except the custom one with the following loop:
for loc in locs[1:-1]:
if loc != x_data[np.argmax(y_data)]:
plt.axvline(x=loc, color = 'grey', linestyle = '-', linewidth = 0.4)
Insert this code just before the line
plt.xticks(locs, labels) # updates the xticks and labels.
Example of output picture below.

Is it possible to test if the legend is covering any data in matplotlib/pyplot

Python beginner so apologies if incorrect terminology at any point.
I am using the legend(loc='best', ...) method and it works 99% of the time. However, when stacking more than 9 plots (i.e. i>9 in example below) on a single figure, with individual labels, it defaults to center and covers the data.
Is there a way to run a test in the script that will give a true/false value if the legend is covering any data points?
Very simplified code:
fig = plt.figure()
for i in data:
plt.plot(i[x, y], label=LABEL)
fig.legend(loc='best')
fig.savefig()
Example of legend covering data

One way is to add some extra space at the bottom/top/left or right side of the axis (in your case I would prefer top or bottom), by changing the limits slightly. Doing so makes the legend fit below the data. Add extra space by setting a different y-limit with ax.set_ylim(-3e-4, 1.5e-4) (the upper limit is approximately what it is in your figure and -3 is a estimate of what you need).
What you also need to do is to add split the legend into more columns, with the keyword ncol=N when creating the legend.
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure()
ax = fig.add_subplot(111)
x = np.linspace(0, 1, 100)
y = 3.5 * x - 2
for i in range(9):
ax.plot(x, y + i / 10., label='iiiiiiiiiiii={}'.format(i))
ax.set_ylim(-3, 1.5)
ax.legend(loc='lower center', ncol=3) # ncol=3 looked nice for me, maybe you need to change this
plt.show()
EDIT
Another solution is to put the legend in a separate axis like I do in the code below. The data-plot does not need to care about making space for the legend or anything and you should have enough space in the axis below to put all your line-labels. If you need more space, you can easily change the ratio of the upper axis to the lower axis.
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure()
ax = fig.add_subplot(211)
ax_leg = fig.add_subplot(212)
x = np.linspace(0, 1, 100)
y = 3.5 * x - 2
lines = []
for i in range(9): #for plotting the actual data
li, = ax.plot(x, y + i / 10., label='iiiiiiiiiiii={}'.format(i))
lines.append(li)
for line in lines: # just to make the legend plot
ax_leg.plot([], [], line.get_color(), label=line.get_label())
ax_leg.legend(loc='center', ncol=3, ) # ncol=3 looked nice for me, maybe you need to change this
ax_leg.axis('off')
fig.show()

Adding secondary X-axis with user-defined list of coordinates

I have a function for plotting amount of graphics using pyplot. The code is here:
def plot_results(results, expers):
"""
type results: list[list[float]]
type expers: list[int]
"""
label_builder = lambda index: 'experiment ' + str(index + 1)
colors = ('green', 'blue')
x_indices = list(map(compute_filesize, list(range(np.shape(results)[1]))))
x_percents = list(map(compute_percent, list(range(np.shape(results)[1]))))
fig, ax1 = plt.subplots()
for i in range(expers):
ax1.plot(x_indices, results[i], color=colors[i], lw=2, label=label_builder(i))
ax1.legend()
plt.show()
For each value of expers list my function plots a chart.
For example, if len(results) == len (expers) == 2, I will get such graph:
I need to create the secondary X-axis (similarly to this, but it may be X-axis and it will be located on the top of graph).
Another difference is that I need to set list of coordinates manually (e.g. ax2.set_coords(x_percents)).
I created new axis using ax2 = ax1.twinx(). Then, I established coordinates list using ax2.set_xticks(x_percents).
But because of each x_percents[i] < x_indices[i], I got such picture:
(as you can see, all coordinates of new axis are located in left-bottom corner)
How can I change my function to make new X-axis:
located on the top side of the graph,
has its own scale, i.e. each value of x_percents corresponds to value of results[i] and
x_percents dispersed throughout interval?

Your code suggests that x_indices and x_percents are linearly related. To keep things clear and useful for others, I'll assume the following for these 2 variables:
x_indices = [0, 5, 10, 25, 50]
max_size = max(x_indices)
x_percents = [ n/max_size * 100 for n in x_indices]
One way you could achieve creating these dual axes that relate to the same data, but just have different labels goes like this: first create an axes, then create another one over that (the twinx/twiny methods could be used but not strictly necessary, I'll use them here for convenience and explaining an important issue that resulted in you setting the xticks for your first axes). Then ensure the limits of both x-axes are the same, set the position of the x-ticks the same as in the first axes and finally change the labels:
import matplotlib.pyplot as plt
vals = [1, 100, 14, 76, 33] # random data, aligned to `x_indices`
fig, ax1 = plt.subplots(1,1)
ax1.plot(x_indices, vals)
ax2 = ax1.twiny() # Remark: twiny will create a new axes
# where the y-axis is shared with ax1,
# but the x-axis is independant - important!
ax2.set_xlim(ax1.get_xlim()) # ensure the independant x-axes now span the same range
ax2.set_xticks(x_indices) # copy over the locations of the x-ticks from the first axes
ax2.set_xticklabels(x_percents) # But give them a different meaning
A graph like this is frequently encountered in physics, where e.g. wavelength and energy are inversely proportionate. On one axis, you would be able to read off the units in one scale (e.g. nanometers) whereas the other would represent the same data in a different scale (e.g. electron volts).

Subplots: tight_layout changes figure size

Changing the vertical distance between two subplot using tight_layout(h_pad=-1) changes the total figuresize. How can I define the figuresize using tight_layout?
Here is the code:
#define figure
pl.figure(figsize=(10, 6.25))
ax1=subplot(211)
img=pl.imshow(np.random.random((10,50)), interpolation='none')
ax1.set_xticklabels(()) #hides the tickslabels of the first plot
subplot(212)
x=linspace(0,50)
pl.plot(x,x,'k-')
xlim( ax1.get_xlim() ) #same x-axis for both plots
And here is the results:
If I write
pl.tight_layout(h_pad=-2)
in the last line, then I get this:
As you can see, the figure is bigger...

You can use a GridSpec object to control precisely width and height ratios, as answered on this thread and documented here.
Experimenting with your code, I could produce something like what you want, by using a height_ratio that assigns twice the space to the upper subplot, and increasing the h_pad parameter to the tight_layout call. This does not sound completely right, but maybe you can adjust this further ...
import numpy as np
from matplotlib.pyplot import *
import matplotlib.pyplot as pl
import matplotlib.gridspec as gridspec
#define figure
fig = pl.figure(figsize=(10, 6.25))
gs = gridspec.GridSpec(2, 1, height_ratios=[2,1])
ax1=subplot(gs[0])
img=pl.imshow(np.random.random((10,50)), interpolation='none')
ax1.set_xticklabels(()) #hides the tickslabels of the first plot
ax2=subplot(gs[1])
x=np.linspace(0,50)
ax2.plot(x,x,'k-')
xlim( ax1.get_xlim() ) #same x-axis for both plots
fig.tight_layout(h_pad=-5)
show()
There were other issues, like correcting the imports, adding numpy, and plotting to ax2 instead of directly with pl. The output I see is this:

This case is peculiar because of the fact that the default aspect ratios of images and plots are not the same. So it is worth noting for people looking to remove the spaces in a grid of subplots consisting of images only or of plots only that you may find an appropriate solution among the answers to this question (and those linked to it): How to remove the space between subplots in matplotlib.pyplot?.
The aspect ratios of the subplots in this particular example are as follows:
# Default aspect ratio of images:
ax1.get_aspect()
# 1.0
# Which is as it is expected based on the default settings in rcParams file:
matplotlib.rcParams['image.aspect']
# 'equal'
# Default aspect ratio of plots:
ax2.get_aspect()
# 'auto'
The size of ax1 and the space beneath it are adjusted automatically based on the number of pixels along the x-axis (i.e. width) so as to preserve the 'equal' aspect ratio while fitting both subplots within the figure. As you mentioned, using fig.tight_layout(h_pad=xxx) or the similar fig.set_constrained_layout_pads(hspace=xxx) is not a good option as this makes the figure larger.
To remove the gap while preserving the original figure size, you can use fig.subplots_adjust(hspace=xxx) or the equivalent plt.subplots(gridspec_kw=dict(hspace=xxx)), as shown in the following example:
import numpy as np # v 1.19.2
import matplotlib.pyplot as plt # v 3.3.2
np.random.seed(1)
fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 6.25),
gridspec_kw=dict(hspace=-0.206))
# For those not using plt.subplots, you can use this instead:
# fig.subplots_adjust(hspace=-0.206)
size = 50
ax1.imshow(np.random.random((10, size)))
ax1.xaxis.set_visible(False)
# Create plot of a line that is aligned with the image above
x = np.arange(0, size)
ax2.plot(x, x, 'k-')
ax2.set_xlim(ax1.get_xlim())
plt.show()
I am not aware of any way to define the appropriate hspace automatically so that the gap can be removed for any image width. As stated in the docstring for fig.subplots_adjust(), it corresponds to the height of the padding between subplots, as a fraction of the average axes height. So I attempted to compute hspace by dividing the gap between the subplots by the average height of both subplots like this:
# Extract axes positions in figure coordinates
ax1_x0, ax1_y0, ax1_x1, ax1_y1 = np.ravel(ax1.get_position())
ax2_x0, ax2_y0, ax2_x1, ax2_y1 = np.ravel(ax2.get_position())
# Compute negative hspace to close the vertical gap between subplots
ax1_h = ax1_y1-ax1_y0
ax2_h = ax2_y1-ax2_y0
avg_h = (ax1_h+ax2_h)/2
gap = ax1_y0-ax2_y1
hspace=-(gap/avg_h) # this divided by 2 also does not work
fig.subplots_adjust(hspace=hspace)
Unfortunately, this does not work. Maybe someone else has a solution for this.
It is also worth mentioning that I tried removing the gap between subplots by editing the y positions like in this example:
# Extract axes positions in figure coordinates
ax1_x0, ax1_y0, ax1_x1, ax1_y1 = np.ravel(ax1.get_position())
ax2_x0, ax2_y0, ax2_x1, ax2_y1 = np.ravel(ax2.get_position())
# Set new y positions: shift ax1 down over gap
gap = ax1_y0-ax2_y1
ax1.set_position([ax1_x0, ax1_y0-gap, ax1_x1, ax1_y1-gap])
ax2.set_position([ax2_x0, ax2_y0, ax2_x1, ax2_y1])
Unfortunately, this (and variations of this) produces seemingly unpredictable results, including a figure resizing similar to when using fig.tight_layout(). Maybe someone else has an explanation for what is happening here behind the scenes.