We Keep Coding

What is the correct matplotlib transform for a "virtual third axis" in my waterfall plot?

While working on improving my answer to this question, I have stumbled into a dead end.
What I want to achieve, is create a "fake" 3D waterfall plot in matplotlib, where individual line plots (or potentially any other plot type) are offset in figure pixel coordinates and plotted behind each other. This part works fine already, and using my code example (see below) you should be able to plot ten equivalent lines which are offset by fig.dpi/10. in x- and y-direction, and plotted behind each other via zorder.
Note that I also added fill_between()'s to make the "depth-cue" zorder more visible.
Where I'm stuck is that I'd like to add a "third axis", i.e. a line (later on perhaps formatted with some ticks) which aligns correctly with the base (i.e. [0,0] in data units) of each line.
This problem is perhaps further complicated by the fact that this isn't a one-off thing (i.e. the solutions should not only work in static pixel coordinates), but has to behave correctly on rescale, especially when working interactively.
As you can see, setting e.g. the xlim's allows one to rescale the lines "as expected" (best if you try it interactively), yet the red line (future axis) that I tried to insert is not transposed in the same way as the bases of each line plot.
What I'm not looking for are solutions which rely on mpl_toolkits.mplot3d's Axes3D, as this would lead to many other issues regarding to zorder and zoom, which are exactly what I'm trying to avoid by coming up with my own "fake 3D plot".
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
from matplotlib.transforms import Affine2D,IdentityTransform
def offset(myFig,myAx,n=1,xOff=60,yOff=60):
"""
this function will apply a shift of n*dx, n*dy
where e.g. n=2, xOff=10 would yield a 20px offset in x-direction
"""
## scale by fig.dpi to have offset in pixels!
dx, dy = xOff/myFig.dpi , yOff/myFig.dpi
t_data = myAx.transData
t_off = mpl.transforms.ScaledTranslation( n*dx, n*dy, myFig.dpi_scale_trans)
return t_data + t_off
fig,axes=plt.subplots(nrows=1, ncols=3,figsize=(10,5))
ys=np.arange(0,5,0.5)
print(len(ys))
## just to have the lines colored in some uniform way
cmap = mpl.cm.get_cmap('viridis')
norm=mpl.colors.Normalize(vmin=ys.min(),vmax=ys.max())
## this defines the offset in pixels
xOff=10
yOff=10
for ax in axes:
## plot the lines
for yi,yv in enumerate(ys):
zo=(len(ys)-yi)
ax.plot([0,0.5,1],[0,1,0],color=cmap(norm(yv)),
zorder=zo, ## to order them "behind" each other
## here we apply the offset to each plot:
transform=offset(fig,ax,n=yi,xOff=xOff,yOff=yOff)
)
### optional: add a fill_between to make layering more obvious
ax.fill_between([0,0.5,1],[0,1,0],0,
facecolor=cmap(norm(yv)),edgecolor="None",alpha=0.1,
zorder=zo-1, ## to order them "behind" each other
## here we apply the offset to each plot:
transform=offset(fig,ax,n=yi,xOff=xOff,yOff=yOff)
)
##################################
####### this is the important bit:
ax.plot([0,2],[0,2],color='r',zorder=100,clip_on=False,
transform=ax.transData+mpl.transforms.ScaledTranslation(0.,0., fig.dpi_scale_trans)
)
## make sure to set them "manually", as autoscaling will fail due to transformations
for ax in axes:
ax.set_ylim(0,2)
axes[0].set_xlim(0,1)
axes[1].set_xlim(0,2)
axes[2].set_xlim(0,3)
### Note: the default fig.dpi is 100, hence an offset of of xOff=10px will become 30px when saving at 300dpi!
# plt.savefig("./test.png",dpi=300)
plt.show()
Update:
I've now included an animation below, which shows how the stacked lines behave on zooming/panning, and how their "baseline" (blue circles) moves with the plot, instead of the static OriginLineTrans solution (green line) or my transformed line (red, dashed).
The attachment points observe different transformations and can be inserted by:
ax.scatter([0],[0],edgecolors="b",zorder=200,facecolors="None",s=10**2,)
ax.scatter([0],[0],edgecolors="b",zorder=200,facecolors="None",s=10**2,transform=offset(fig,ax,n=len(ys)-1,xOff=xOff,yOff=yOff),label="attachment points")

The question boils down to the following:
How to produce a line that
starts from the origin (0,0) in axes coordinates and
evolves at an angle angle in physical coordinates (pixel space)
by using a matpotlib transform?
The problem is that the origin in axes coordinates may vary depending on the subplot position. So the only option I see is to create some custom transform that
transforms to pixel space
translates to the origin in pixel space
skews the coordinate system (say, in x direction) by the given angle
translates back to the origin of the axes
That could look like this
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.transforms as mtrans
class OriginLineTrans(mtrans.Transform):
input_dims = 2
output_dims = 2
def __init__(self, origin, angle, axes):
self.axes = axes
self.origin = origin
self.angle = angle # in radiants
super().__init__()
def get_affine(self):
origin = ax.transAxes.transform_point(self.origin)
trans = ax.transAxes + \
mtrans.Affine2D().translate(*(-origin)) \
.skew(self.angle, 0).translate(*origin)
return trans.get_affine()
fig, ax = plt.subplots()
ax.plot([0,0], [0,1], transform=OriginLineTrans((0,0), np.arctan(1), ax))
plt.show()
Note that in the case of the original question, the angle would be np.arctan(dx/dy).

rotate x-axis text and move y-labels to the top of axes (multiple y-axes)

In the following python code I’m plotting time datas and multiple y-values out of a dataframe.
Now I want to:
- rotate the time values of the x-axis vertically
- move all y-labels (y1-y4) to the top of the axis
Does anyone have suggestions or solutions ?
import pandas as pd
from mpl_toolkits.axes_grid1 import host_subplot
import mpl_toolkits.axisartist as AA
import matplotlib.pyplot as plt
data = {'time':['00:00:00','18:00:00','23:59:00'],
'y1': [1,2,3],'y2': [4,5,6],'y3': [7,8,9],'y4': [10,11,12]}
df=pd.DataFrame(data,columns=['time','y1','y2','y3','y4'])
df['time']=pd.to_datetime(df['time'],format='%H:%M:%S')
host=host_subplot(111,axes_class=AA.Axes)
plt.subplots_adjust(right=0.75)
par1=host.twinx()
par2=host.twinx()
par3=host.twinx()
offset1=40
offset2=80
new_fixed_axis=par2.get_grid_helper().new_fixed_axis
par2.axis['right']=new_fixed_axis(loc='right',axes=par2,offset=(offset1,0))
par2.axis['right'].toggle(all=True)
par3.axis['right']=new_fixed_axis(loc='right',axes=par3,offset=(offset2,0))
par3.axis['right'].toggle(all=True)
host.set_xlabel('Time')
host.set_ylabel('y1')
par1.set_ylabel('y2')
par2.set_ylabel('y3')
par3.set_ylabel('y4')
p1,=host.plot(df['time'],df['y1'])
p2,=par1.plot(df['time'],df['y2'])
p3,=par2.plot(df['time'],df['y3'])
p4,=par3.plot(df['time'],df['y4'])
host.set_ylim(0,5)
par1.set_ylim(0,8)
par2.set_ylim(0,10)
par3.set_ylim(0,15)
host.legend(loc='upper left',bbox_to_anchor=(0,-.15),ncol=4)
host.axis['left'].label.set_color(p1.get_color())
host.axis["left"].label.set_rotation(90)
par1.axis['right'].label.set_color(p2.get_color())
par1.axis["right"].label.set_rotation(-90)
par2.axis['right'].label.set_color(p3.get_color())
par2.axis["right"].label.set_rotation(-90)
par3.axis['right'].label.set_color(p4.get_color())
par3.axis["right"].label.set_rotation(-90)
plt.draw()
plt.show()

I have a solution for the first problem, and an ugly hack for the second.
1) Rotate x-axis text
You need to set this using the host.axis property, and then move the legend and xlabel down a bit to make room:
host.legend(loc='upper left',bbox_to_anchor=(0,-.3),ncol=4) # Lower legend a bit
host.axis["bottom"].label.set_pad(50) # Lower x-label a bit
host.axis["bottom"].major_ticklabels.set( # This is the actual rotation command
rotation=90, verticalalignment='center', horizontalalignment='right', pad=-2)
2) Move y-labels to the top of axes
Nothing I tried here worked. Specifically, I expected the following to work:
par1.axis["right"].label.set_position((1.0, 1.0))
But the respective get_position command keeps returning (1.0, 0.5), in which 0.5 is the middle of the axis. For some reason the 'set' command simply won't stick. This is where the ugly hack comes in - adding blank lines below the label. So where you set the label text, change to:
new_line_pos_fix = ''.join(['\n'] * 9) # 9 is the number of blank lines; change as needed
host.set_ylabel('y1' + new_line_pos_fix)
par1.set_ylabel('y2' + new_line_pos_fix)
par2.set_ylabel('y3' + new_line_pos_fix)
par3.set_ylabel('y4' + new_line_pos_fix)
Now we just need to set the vertical alignment of the labels, so that the blank lines take effect, adding some padding for better readability:
host.axis['right'].label.set(verticalalignment='bottom', pad=7)
par1.axis['right'].label.set(verticalalignment='bottom', pad=7)
par2.axis['right'].label.set(verticalalignment='bottom', pad=7)
par3.axis['right'].label.set(verticalalignment='bottom', pad=7)
The above commands can be combined with the color and rotation settings, if you want more organized code, e.g.:
par1.axis['right'].label.set(color=p2.get_color(), rotation=-90, verticalalignment='bottom', pad=7)
Result:

How to move a tick label in matplotlib

I would like to move some ticks' labels horizontally along the x-axis, without moving the corresponding ticks.
More specifically, when rotating labels with plt.setp, the centers of the labels' text stay aligned with the ticks. I would like to shift those labels to the right, so that the near ends of the labels get aligned instead as suggested on the image below.
I am aware of this post and this one, however the answers are interesting kludges rather than strict answers to the question.
my code:
import matplotlib.pyplot as plt
import numpy as np
import datetime
# my fake data
dates = np.array([datetime.datetime(2000,1,1) + datetime.timedelta(days=i) for i in range(365*5)])
data = np.sin(np.arange(365*5)/365.0*2*np.pi - 0.25*np.pi) + np.random.rand(365*5) /3
# creates fig with 2 subplots
fig = plt.figure(figsize=(10.0, 6.0))
ax = plt.subplot2grid((2,1), (0, 0))
ax2 = plt.subplot2grid((2,1), (1, 0))
## plot dates
ax2.plot_date( dates, data )
# rotates labels
plt.setp( ax2.xaxis.get_majorticklabels(), rotation=-45 )
# try to shift labels to the right
ax2.xaxis.get_majorticklabels()[2].set_y(-.1)
ax2.xaxis.get_majorticklabels()[2].set_x(10**99)
plt.show()
Strangely enough, set_y behaves as expected, but even if I set x to a fantasillion, the labels would not move by one iota.
(The use of plot_date may introduce additional confusion, but the same actually happens with plot.)

First of all, let's use a mcve to show the problem.
import numpy as np
import datetime
import matplotlib.pyplot as plt
plt.rcParams["date.autoformatter.month"] = "%b %Y"
# my fake data
dates = np.array([datetime.datetime(2000,1,1) + datetime.timedelta(days=i) for i in range(365)])
data = np.sin(np.arange(365)/365.0*2*np.pi - 0.25*np.pi) + np.random.rand(365) /3
# creates fig with 2 subplots
fig, ax = plt.subplots(figsize=(6,2))
## plot dates
ax.plot_date( dates, data )
# rotates labels
plt.setp( ax.xaxis.get_majorticklabels(), rotation=-45 )
plt.tight_layout()
plt.show()
Now as other anwers pointed out already, you may use horizontal alignment of the text.
# rotates labels and aligns them horizontally to left
plt.setp( ax.xaxis.get_majorticklabels(), rotation=-45, ha="left" )
You may use the rotation_mode argument to let the rotation happen about the top left point of the text, giving a slightly nicer result in this case.
# rotates labels and aligns them horizontally to left
plt.setp( ax.xaxis.get_majorticklabels(), rotation=-45, ha="left", rotation_mode="anchor")
In case those options are not fine grained enough, i.e. you want to position the labels more accurately, e.g. shifting it to the side by some points, you may use a transform. The following would offset the label by 5 points in horizontal direction, using a matplotlib.transforms.ScaledTranslation.
import matplotlib.transforms
plt.setp( ax.xaxis.get_majorticklabels(), rotation=-45)
# Create offset transform by 5 points in x direction
dx = 5/72.; dy = 0/72.
offset = matplotlib.transforms.ScaledTranslation(dx, dy, fig.dpi_scale_trans)
# apply offset transform to all x ticklabels.
for label in ax.xaxis.get_majorticklabels():
label.set_transform(label.get_transform() + offset)
The advantage of this, compared to e.g. the solution provided by #explorerDude is that the offset is independent on the data in the graph, such that it is generally applicable to any plot and would look the same for a given fontsize.

Instead of
ax2.xaxis.get_majorticklabels()[2].set_y(-.1)
ax2.xaxis.get_majorticklabels()[2].set_x(10**99)
use the set_horizontalalignment() for each tick on the axis:
for tick in ax2.xaxis.get_majorticklabels():
tick.set_horizontalalignment("left")
resulting in:

I found a way to shift the tick labels of the x-axis by an arbitrary and exact amount, but this way runs dangerously close to the steep and slippery cliffs towering above the sea of madness. So only the very brave or desperate should read on...
That being said, the problem is that the x position of the labels are set when the drawing is rendered (I have not looked into that part of the code, but that is my understanding). So everything you do with set_x() is overridden later. However, there is a way around that: you can monkey patch set_x for certain ticks so that the labels are not drawn where the renderer wants to draw them:
import types
SHIFT = 10. # Data coordinates
for label in ax2.xaxis.get_majorticklabels():
label.customShiftValue = SHIFT
label.set_x = types.MethodType( lambda self, x: matplotlib.text.Text.set_x(self, x-self.customShiftValue ),
label, matplotlib.text.Text )
You can do this selectively only for the labels you want to shift and you can of course also use a different shift for every label.
If anybody knows how to do this on a lower madness level, I would be very interested...

Another way of doing a horizontal alignment:
plt.xticks(ha='left')

Subplots: tight_layout changes figure size

Changing the vertical distance between two subplot using tight_layout(h_pad=-1) changes the total figuresize. How can I define the figuresize using tight_layout?
Here is the code:
#define figure
pl.figure(figsize=(10, 6.25))
ax1=subplot(211)
img=pl.imshow(np.random.random((10,50)), interpolation='none')
ax1.set_xticklabels(()) #hides the tickslabels of the first plot
subplot(212)
x=linspace(0,50)
pl.plot(x,x,'k-')
xlim( ax1.get_xlim() ) #same x-axis for both plots
And here is the results:
If I write
pl.tight_layout(h_pad=-2)
in the last line, then I get this:
As you can see, the figure is bigger...

You can use a GridSpec object to control precisely width and height ratios, as answered on this thread and documented here.
Experimenting with your code, I could produce something like what you want, by using a height_ratio that assigns twice the space to the upper subplot, and increasing the h_pad parameter to the tight_layout call. This does not sound completely right, but maybe you can adjust this further ...
import numpy as np
from matplotlib.pyplot import *
import matplotlib.pyplot as pl
import matplotlib.gridspec as gridspec
#define figure
fig = pl.figure(figsize=(10, 6.25))
gs = gridspec.GridSpec(2, 1, height_ratios=[2,1])
ax1=subplot(gs[0])
img=pl.imshow(np.random.random((10,50)), interpolation='none')
ax1.set_xticklabels(()) #hides the tickslabels of the first plot
ax2=subplot(gs[1])
x=np.linspace(0,50)
ax2.plot(x,x,'k-')
xlim( ax1.get_xlim() ) #same x-axis for both plots
fig.tight_layout(h_pad=-5)
show()
There were other issues, like correcting the imports, adding numpy, and plotting to ax2 instead of directly with pl. The output I see is this:

This case is peculiar because of the fact that the default aspect ratios of images and plots are not the same. So it is worth noting for people looking to remove the spaces in a grid of subplots consisting of images only or of plots only that you may find an appropriate solution among the answers to this question (and those linked to it): How to remove the space between subplots in matplotlib.pyplot?.
The aspect ratios of the subplots in this particular example are as follows:
# Default aspect ratio of images:
ax1.get_aspect()
# 1.0
# Which is as it is expected based on the default settings in rcParams file:
matplotlib.rcParams['image.aspect']
# 'equal'
# Default aspect ratio of plots:
ax2.get_aspect()
# 'auto'
The size of ax1 and the space beneath it are adjusted automatically based on the number of pixels along the x-axis (i.e. width) so as to preserve the 'equal' aspect ratio while fitting both subplots within the figure. As you mentioned, using fig.tight_layout(h_pad=xxx) or the similar fig.set_constrained_layout_pads(hspace=xxx) is not a good option as this makes the figure larger.
To remove the gap while preserving the original figure size, you can use fig.subplots_adjust(hspace=xxx) or the equivalent plt.subplots(gridspec_kw=dict(hspace=xxx)), as shown in the following example:
import numpy as np # v 1.19.2
import matplotlib.pyplot as plt # v 3.3.2
np.random.seed(1)
fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 6.25),
gridspec_kw=dict(hspace=-0.206))
# For those not using plt.subplots, you can use this instead:
# fig.subplots_adjust(hspace=-0.206)
size = 50
ax1.imshow(np.random.random((10, size)))
ax1.xaxis.set_visible(False)
# Create plot of a line that is aligned with the image above
x = np.arange(0, size)
ax2.plot(x, x, 'k-')
ax2.set_xlim(ax1.get_xlim())
plt.show()
I am not aware of any way to define the appropriate hspace automatically so that the gap can be removed for any image width. As stated in the docstring for fig.subplots_adjust(), it corresponds to the height of the padding between subplots, as a fraction of the average axes height. So I attempted to compute hspace by dividing the gap between the subplots by the average height of both subplots like this:
# Extract axes positions in figure coordinates
ax1_x0, ax1_y0, ax1_x1, ax1_y1 = np.ravel(ax1.get_position())
ax2_x0, ax2_y0, ax2_x1, ax2_y1 = np.ravel(ax2.get_position())
# Compute negative hspace to close the vertical gap between subplots
ax1_h = ax1_y1-ax1_y0
ax2_h = ax2_y1-ax2_y0
avg_h = (ax1_h+ax2_h)/2
gap = ax1_y0-ax2_y1
hspace=-(gap/avg_h) # this divided by 2 also does not work
fig.subplots_adjust(hspace=hspace)
Unfortunately, this does not work. Maybe someone else has a solution for this.
It is also worth mentioning that I tried removing the gap between subplots by editing the y positions like in this example:
# Extract axes positions in figure coordinates
ax1_x0, ax1_y0, ax1_x1, ax1_y1 = np.ravel(ax1.get_position())
ax2_x0, ax2_y0, ax2_x1, ax2_y1 = np.ravel(ax2.get_position())
# Set new y positions: shift ax1 down over gap
gap = ax1_y0-ax2_y1
ax1.set_position([ax1_x0, ax1_y0-gap, ax1_x1, ax1_y1-gap])
ax2.set_position([ax2_x0, ax2_y0, ax2_x1, ax2_y1])
Unfortunately, this (and variations of this) produces seemingly unpredictable results, including a figure resizing similar to when using fig.tight_layout(). Maybe someone else has an explanation for what is happening here behind the scenes.

matplotbib figure horization axis label automatically alignment or rescale

I was trying to plot a time series data figure using matplotbib, the problem is that there are too many observations, therefore the labels have overlap and don't fit well within a sized figure.
I am thinking of three solutions, one is to shrink the label size of observations, one is to change the text into vertical order or skewed manner, last is only to specify the first and last a few observations with dots between them. The code is to demonstrate my point.
I wonder anyone can help? Thanks
from datetime import date
import numpy as np
from pandas import *
import matplotlib.pyplot as plt
N = 100
data = np.array(np.random.randn(N))
time_index = date_range(date.today(), periods = len(data))
plt.plot(time_index, data)

For your simple plot, you could do
plt.xticks(rotation=90).
Alternatively, you could specify what ticks you wanted to display with
plt.xticks(<certain range of values>)
plt.xticklabels(<labels for those values>)
Edit:
Personally, I would change to the object-oriented way of pyplot.
f = plt.figure()
ax = f.add_subplot(111)
ax.plot(<stuff>)
ax.tick_params(axis='x', labelsize='8')
plt.setp( ax.xaxis.get_majorticklabels(), rotation=90 )
# OR
xlabels = ax.get_xticklabels()
for label in xlabels:
label.set_rotation(90)
plt.show()