We Keep Coding

How to increase plottable space above a subplot in matplotlib?

I am currently making a plot on matplotlib, which looks like below.
The code for which is:
fig, ax1 = plt.subplots(figsize=(20,5))
ax2 = ax1.twinx()
# plt.subplots_adjust(top=1.4)
ax2.fill_between(dryhydro_df['Time'],dryhydro_df['Flow [m³/s]'],0,facecolor='lightgrey')
ax2.set_ylim([0,10])
AB = ax2.fill_between(dryhydro_df['Time'],[12]*len(dryhydro_df['Time']),9.25,facecolor=colors[0],alpha=0.5,clip_on=False)
ab = ax2.scatter(presence_df['Datetime'][presence_df['AB']==True],[9.5]*sum(presence_df['AB']==True),marker='X',color='black')
# tidal heights
ax1.plot(tide_df['Time'],tide_df['Tide'],color='dimgrey')
I want the blue shaded region and black scatter to be above the plot. I can move the elements above the plot by using clip_on=False but I think I need to extend the space above the plot to do visualise it. Is there a way to do this? Mock-up of what I need is below:

You can use clip_on=False to draw outside the main plot. To position the elements, an xaxis transform helps. That way, x-values can be used in the x direction, while the y-direction uses "axes coordinates". ax.transAxes() uses "axes coordinates" for both directions.
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
dates = pd.date_range('2018-07-01', '2018-07-31', freq='H')
xs = dates.to_numpy().astype(float)
ys = np.sin(xs * .091) * (np.sin(xs * .023) ** 2 + 1)
fig, ax1 = plt.subplots(figsize=(20, 5))
ax1.plot(dates, ys)
ax1.scatter(np.random.choice(dates, 10), np.repeat(1.05, 10), s=20, marker='*', transform=ax1.get_xaxis_transform(),
clip_on=False)
ax1.plot([0, 1], [1.05, 1.05], color='steelblue', lw=20, alpha=0.2, transform=ax1.transAxes, clip_on=False)
plt.tight_layout() # fit labels etc. nicely
plt.subplots_adjust(top=0.9) # make room for the additional elements
plt.show()

Customising the axis labels (Text & Position) in matplotlib

I have 2 sets of rectangular patches in a plot. I want to name them separately. "Layer-1" for the bottom part and similarly "Layer-2" for the upper part. I wanted to set coordinates for the Y-axis but it did not work. Moreover i was not able to add the "Layer-2" text into the label. Please help.
I tried with the below mentioned code but it did not work.
plt.ylabel("LAYER-1", loc='bottom')
yaxis.labellocation(bottom)

One solution is to create a second axis, so called twin axis that shares the same x axis. Then it is possbile to label them separately. Furthermore, you can adjust the location of the label via
axis.yaxis.set_label_coords(-0.1, 0.75)
Here is an example that you can adjust to your desires. The result can be found here: https://i.stack.imgur.com/1o2xl.png
%matplotlib notebook
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.patches as patches
import numpy as np
plt.rcParams['figure.dpi'] = 100
import matplotlib.pyplot as plt
x = np.arange(0, 10, 0.1)
y1 = 0.05 * x**2
y2 = -1 *y1
fig, ax1 = plt.subplots()
ax2 = ax1.twinx()
ax1.plot(x, y1, 'g-')
ax2.plot(x, y2, 'b-')
# common x axis
ax1.set_xlabel('X data')
# First y axis label
ax1.set_ylabel('LAYER-1', color='g')
# Second y [enter image description here][1]axis label
ax2.set_ylabel('LAYER-2', color='b')
# Adjust the label location
ax1.yaxis.set_label_coords(-0.075, 0.25)
ax2.yaxis.set_label_coords(-0.1, 0.75)
plt.show()

How to plot a contour plot if the difference between Zmax and Zmin is of the order of 10^(-5)?

I want to produce a ramachandran plot which would look like the following
basically it is a superposition of two plots: contour and scatter. I have the data file for plotting the contour and scatter plot. The data for contour plot is present as three different columns denoting x, y and z values. the value of x and y varies from -180 to 180. Whereas the value z varies from 0 to 1 and the difference between z values can be as low as 10^(-5). In my code I tried to plot the contour using tricontourf where the difference each entry of the level is 0.01. Whenever I tried to make gap between those levels to 0.00001, the code just doesn't get over. That's why I am unable to generate a graph that I want.
The code that I wrote is the following:
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.tri as tri
import matplotlib.cm as cm
x=[]
y=[]
z=[]
x1=[]
y1=[]
lst = []
plt.style.use('seaborn-whitegrid')
for line in open('rama_data.txt', 'r'):
values = [float(s) for s in line.split()]
x.append(values[0])
y.append(values[1])
z.append(values[2])
f=open('all_str_C-S-S-C_Acceptor.txt',"r")
lines=f.readlines()
for m in lines:
x1.append(m.split(' ')[8])
y1.append(m.split(' ')[9])
f.close()
norm = cm.colors.Normalize(vmax=max(z), vmin=min(z))
cmap = cm.OrRd
fig2, ax2 = plt.subplots()
#ax2.set_aspect('equal')
levels = np.arange(0, 1,0.01)
tcf = ax2.tricontourf(x, y, z, levels, cmap=cm.get_cmap(cmap, len(levels)-1),norm=norm)
ax2.set_xticks(np.arange(-180,181,45))
ax2.set_yticks(np.arange(-180,181,45))
ax2.set_xlabel('$\Phi$ Dihedral angle($\circ$)', fontsize=12, fontweight='bold')
ax2.set_ylabel('$\Psi\'$ Dihedral angle($\circ$)', fontsize=12, fontweight='bold')
#cbar=fig2.colorbar(tcf)
#cbar.ax.set_ylabel('Relative Electronic energy(kJ/mol)', fontsize=12, fontweight='bold')
ax2.autoscale(False) # To avoid that the scatter changes limits
ax2.scatter(x1,y1,s=0.15,c='black',zorder=1)
fig2.savefig("Ramachandran plot",dpi=300)
plt.show()
My code generates an image which looks this this:
What modifications should I do do produce the desirable plot?
I have attached the rama_data.txt file. Anyone can download and try it once.

The main problem seems to be that for 100 levels (as in levels = np.arange(0, 1,0.01)) the colors get very smoothed out. Just reducing the number of levels gets a plot much closer to the example plot.
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.tri as tri
xyz = np.loadtxt('rama.txt')
x = xyz[:, 0]
y = xyz[:, 1]
z = xyz[:, 2]
fig2, (ax1, ax2) = plt.subplots(ncols=2)
cmap = 'OrRd'
tcf = ax2.tricontourf(x, y, z, levels=5, cmap=cmap) # norm=norm)
filter = (z > 0.2) & (np.random.randint(0, 10, z.size) == 0)
ax2.scatter(x[filter], y[filter], marker='.', s=1, color='black')
ax1.scatter(x, y, c=z, cmap=cmap)
ax1.set_xticks(np.arange(-180, 181, 45))
ax1.set_yticks(np.arange(-180, 181, 45))
ax2.set_xticks(np.arange(-180, 181, 45))
ax2.set_yticks(np.arange(-180, 181, 45))
plt.show()
The plot shows a regular scatter plot of the given data at the left, and the contourf plot at the right.

Adding y=x to a matplotlib scatter plot if I haven't kept track of all the data points that went in

Here's some code that does scatter plot of a number of different series using matplotlib and then adds the line y=x:
import numpy as np, matplotlib.pyplot as plt, matplotlib.cm as cm, pylab
nseries = 10
colors = cm.rainbow(np.linspace(0, 1, nseries))
all_x = []
all_y = []
for i in range(nseries):
x = np.random.random(12)+i/10.0
y = np.random.random(12)+i/5.0
plt.scatter(x, y, color=colors[i])
all_x.extend(x)
all_y.extend(y)
# Could I somehow do the next part (add identity_line) if I haven't been keeping track of all the x and y values I've seen?
identity_line = np.linspace(max(min(all_x), min(all_y)),
min(max(all_x), max(all_y)))
plt.plot(identity_line, identity_line, color="black", linestyle="dashed", linewidth=3.0)
plt.show()
In order to achieve this I've had to keep track of all the x and y values that went into the scatter plot so that I know where identity_line should start and end. Is there a way I can get y=x to show up even if I don't have a list of all the points that I plotted? I would think that something in matplotlib can give me a list of all the points after the fact, but I haven't been able to figure out how to get that list.

You don't need to know anything about your data per se. You can get away with what your matplotlib Axes object will tell you about the data.
See below:
import numpy as np
import matplotlib.pyplot as plt
# random data
N = 37
x = np.random.normal(loc=3.5, scale=1.25, size=N)
y = np.random.normal(loc=3.4, scale=1.5, size=N)
c = x**2 + y**2
# now sort it just to make it look like it's related
x.sort()
y.sort()
fig, ax = plt.subplots()
ax.scatter(x, y, s=25, c=c, cmap=plt.cm.coolwarm, zorder=10)
Here's the good part:
lims = [
np.min([ax.get_xlim(), ax.get_ylim()]), # min of both axes
np.max([ax.get_xlim(), ax.get_ylim()]), # max of both axes
]
# now plot both limits against eachother
ax.plot(lims, lims, 'k-', alpha=0.75, zorder=0)
ax.set_aspect('equal')
ax.set_xlim(lims)
ax.set_ylim(lims)
fig.savefig('/Users/paul/Desktop/so.png', dpi=300)
Et voilà

In one line:
ax.plot([0,1],[0,1], transform=ax.transAxes)
No need to modify the xlim or ylim.

Starting with matplotlib 3.3 this has been made very simple with the axline method which only needs a point and a slope. To plot x=y:
ax.axline((0, 0), slope=1)
You don't need to look at your data to use this because the point you specify (i.e. here (0,0)) doesn't actually need to be in your data or plotting range.

If you set scalex and scaley to False, it saves a bit of bookkeeping. This is what I have been using lately to overlay y=x:
xpoints = ypoints = plt.xlim()
plt.plot(xpoints, ypoints, linestyle='--', color='k', lw=3, scalex=False, scaley=False)
or if you've got an axis:
xpoints = ypoints = ax.get_xlim()
ax.plot(xpoints, ypoints, linestyle='--', color='k', lw=3, scalex=False, scaley=False)
Of course, this won't give you a square aspect ratio. If you care about that, go with Paul H's solution.

Plotting two different sized grids. Smaller grid centred on the centre of the bigger

I am trying to get something like this (image the image is fully populated with the red squares (I only drew a few)): . Expanding on what I want: I want the RED squares to be centred int the YELLOW squares as shown in the picture (but with RED squares in ALL the YELLOW squares).
What is happening there is the bigger windows (yellow grid) are overlapped between each other by half their size, where the smaller windows in this case, half the size of the big window, (red square) are centred on the centre of the big window. The furthest I could get was using this Multiple grids on matplotlib I am basically using their code, but to make things absolutely clear, I include the code:
EDIT: Thanks to Rutgers I got what I wanted. Here is a slightly edited and shortened version. This code gives the first centre of the four yellow grid intersection where I want.
import matplotlib.pyplot as plt
from matplotlib.pyplot import subplot
from scipy.misc import imread
import numpy as np
import matplotlib.cm as cmps
import matplotlib.collections as collections
i = 1
initial_frame = 1
ax = subplot(111)
bg = imread("./png/frame_" + str("%05d" % (i + initial_frame) ) + ".png").astype(np.float64)
# define the normal (yellow) grid
ytcks = np.arange(16,bg.shape[0],32)
xtcks = np.arange(16,bg.shape[1],32)
# plot the sample data
ax.imshow(bg, cmap=plt.cm.Greys_r, interpolation='none')
ax.set_xticks(xtcks)
ax.set_xticks(xtcks+16, minor=True)
ax.set_yticks(ytcks)
ax.set_yticks(ytcks+16, minor=True)
ax.xaxis.grid(True,'minor', linestyle='--', lw=1., color='y')
ax.yaxis.grid(True,'minor', linestyle='--', lw=1., color='y')
ax.xaxis.grid(True,'major', linestyle='--', lw=0.5, color='g')
ax.yaxis.grid(True,'major', linestyle='--', lw=0.5, color='g')
plt.show()

Given the sample data z from my other answer:
# define the normal (yellow) grid
tcks = np.arange(0,90,10)
fig, ax = plt.subplots(figsize=(8,8))
# plot the sample data
ax.imshow(z, cmap=plt.cm.Greys_r, interpolation='none', vmin=0.4, vmax=1.5, extent=[0,z.shape[0],0,z.shape[1]])
ax.set_xticks(tcks)
ax.set_xticks(tcks+5, minor=True)
ax.set_yticks(tcks)
ax.set_yticks(tcks+5, minor=True)
ax.xaxis.grid(True,'minor', linestyle='--', lw=1., color='y')
ax.yaxis.grid(True,'minor', linestyle='--', lw=1., color='y')
ax.xaxis.grid(True,'major', linestyle='-', lw=1., color='r')
ax.yaxis.grid(True,'major', linestyle='-', lw=1., color='r')
ax.set_xlim(0,80)
ax.set_ylim(0,80)
I expect this to be much faster than drawing with polygons.

Mentioning bigger and smaller grid is a little bit confusing, since to me they seem of equal size, but i assume you mean the 'major' and 'minor' grid.
Well, to sort of mimic your picture with what i had in mind, see if this makes any sense:
import matplotlib.collections as collections
import numpy as np
import matplotlib.pyplot as plt
# generate some fake data, after:
# http://matplotlib.org/examples/images_contours_and_fields/pcolormesh_levels.html
dx, dy = 0.05, 0.05
y, x = np.mgrid[slice(1, 5, dy), slice(1, 5, dx)]
z = np.sin(x) ** 10 + np.cos(10 + y * x) * np.cos(x)
# define the normal (yellow) grid
tcks = np.arange(0,90,10)
fig, ax = plt.subplots(figsize=(8,8), subplot_kw={'xticks': tcks, 'yticks': tcks})
# plot the sample data
ax.imshow(z, cmap=plt.cm.Greys_r, interpolation='none', vmin=0.4, vmax=1.5, extent=[0,z.shape[0],0,z.shape[1]])
# plot the yellow grid
ax.grid(True, linestyle='--', color='y', lw=1.5, alpha=1.0)
# define some random 'red' grid cells
custom_grid = []
for i in range(10):
x = np.random.randint(0,7) * 10 + 5
y = np.random.randint(0,7) * 10 + 5
polygon = plt.Rectangle((x, y), 10, 10)
custom_grid.append(polygon)
p = collections.PatchCollection(custom_grid, facecolor='none', edgecolor='r', lw=1.5)
ax.add_collection(p)
Its stil a bit unclear for example when you want to show the 'red' grid cells and when not.