We Keep Coding

How to create a customized(non-linear, not log) x-axis in plot

I'm using matplotlib to plot a series data like the following, which is very uneven distributed in x axis (unit: second).
say, here , figure 1, very condensed between 0-100s,not so condensed after that. but in the linear x-axis, it's not obvious to figure the condensed fact out.
So I would like to customize the x-axis to zoom in some part and zoom out some other part. for example, here, to zoom in the 0-100s which may take almost half the axis, and zoom out 100-1750s for the other half axis.
Following figure 2 is a example of what I mean the 'customized' x-axis, which have [0.5s,1.3s,5s,10s,20s,30s,60s,2m,5m,10m,8h] on axis.
Sorry that I'm new here that can't attach the figure to the chatbox directly.

Different parts at different 'zoom' levels
You describe that you want your figure to be zoomed in between 0 and 100, and zoomed out between 100 to 1750. For this, you can create something similar to the 'broken axis' example available in the matplotlib documentation, but let me show you how you could apply it to your dataset.
First, let me generate a fake dataset that sort of looks like yours, as you have not made your dataset available.
large_peak = 10 * np.random.randn(100) + 10
other_data = 2000 * np.random.uniform(size=10)
data = np.concatenate((large_peak, other_data))
fig, ax = plt.subplots()
bins = np.arange(0, np.max(data), 20)
ax.hist(data, bins=bins, density=True)
This produces the following figure:
Now, to create this figure with a broken axis, we split the figure up in two subplots, plot the same data in these subplots, hide the line between the subplots, make the subplots connect, and set their x limits correctly.
# sharey to have the same y axis limits, wspace=0 to make the axes connect
fig, (ax1, ax2) = plt.subplots(1, 2, sharey=True, gridspec_kw={'wspace': 0})
# x limits of the axes
xlim1 = (0, 100)
xlim2 = (100, 1750)
# calculate the histogram bin width such that it is equal for both axes
# we choose 50 bins per axis, you can change this value
bins1 = np.arange(xlim1[0], xlim1[1] + 1, np.diff(xlim1) / 50)
bins2 = np.arange(xlim2[0], xlim2[1] + 1, np.diff(xlim2) / 50)
# plot the same data in both axes
ax1.hist(data, bins=bins1, density=True)
ax2.hist(data, bins=bins2, density=True)
# hide the spines and ticks
ax1.spines.right.set_visible(False)
ax2.spines.left.set_visible(False)
ax2.tick_params(axis='y', left=False)
# set the x limits
ax1.set_xlim(xlim1)
ax2.set_xlim(xlim2)
This produces the following figure:
Now, you can see that the left half of the figure is zoomed in, while the right half is zoomed out. You might want to clarify this change of axes, for example using a dotted gray line through the middle.
# add line to indicate the change in x scale
ax1.axvline(xlim1[1], clip_on=False, color='gray', linestyle=':')
Log scale?
The title of your question suggests that you do not want to use a log scale for your figure, and you give an example of a figure with a different x axis (the figure you call "figure2"). However, this figure's x axis has a log scale, its ticks have just been placed unconventionally (to indicate peaks in the data). See the following figure, where I have used the difference between the '60' tick (1 minute) and the '10m' tick (10 minutes) to find the size of the decades in the x axis:
Clearly, you can see that the x axis in this figure is actually scaled logarithmically. You can place the ticks at different locations using the ax.set_xticks() method, also in a log scaled figure.

Here is a way to create the second example, which uses a log scale. A custom formatter can be created to display a value for an x tick to either seconds, minutes or hours.
import matplotlib.pyplot as plt
from matplotlib.ticker import NullFormatter, FuncFormatter
import numpy as np
minutes = 60
hours = 60 * minutes
def hms_formatter(x, pos):
if x < 120:
return f'{x:g}'
elif x < 120 * minutes:
return f'{x / minutes:g}m'
else:
return f'{x / hours:g}h'
fig, ax = plt.subplots()
ax.set_xscale('log')
ax.set_xticks([.5, 1.3, 5, 10, 20, 30, 60, 2*minutes, 5*minutes, 10*minutes, 8*hours])
ax.set_xticks([i for i in range(1, 20)] + [i for i in range(20, 120, 10)]
+ [i * minutes for i in range(1, 20)] + [i * minutes for i in range(20, 120, 10)]
+ [i * hours for i in range(1, 10)],
minor=True)
ax.xaxis.set_major_formatter(FuncFormatter(hms_formatter))
ax.xaxis.set_minor_formatter(NullFormatter())
x = np.logspace(0, np.log10(8 * hours), 200)
y = np.random.normal(0.1, 1, 200).cumsum() + 5
ax.plot(x, y)
ax.margins(x=0)
ax.set_ylim(ymin=0)
plt.show()

How to force and edit major and minor log plot ticks of pyplot subplot

When I'm plotting my data, I want to be able to control the major and minor ticks of my subplots. However, whatever I try I seem to be unable modify the ticks of my second subplot. I've tried applying the advice from other stackoverflow questions, unfortunately to no avail. I think I'm doing something fundamentally wrong when constructing my boxplots.
As none of my colleagues have much experience with matplotlib, I'm turning to you stackoverflow! Any help would be greatly appreciated!
Currently, my figures look like this:
On the second boxplot, I also want to force a major tick on every 10^xth and show the default log minor ticks.
Currently, I generate my boxplots as follows:
def generateLogBoxPlot(title, plot_id, xlabels, data, initializion_time, fig):
# Create an axes instance
ax = fig.add_subplot(plot_id)
# Set Log Scale
ax.set_yscale('log')
# Create the boxplot
bp = ax.boxplot(data_of_plot)
# Show Initialization Time (as a Line)
line = plt.plot(...)
# Custom rotated x-axis labels
ax.set_xticklabels(xlabels)
plt.xticks(rotation=15)
#Set Labels
plt.ylabel('Time (ms)')
ax.set_title(title)
#Show Grid
ax.get_yaxis().grid()
And I call this function like this:
# Create a figure instance
fig = plt.figure(1, figsize=(9, 3))
# Generate first subplot
generateLogBoxPlot("No Context-Dependent A.C.\nBusiness Case", #title
121, #plot_id
["Add User", "Add Role", "Add Demarcation", "Add Permission"], #labels
results["AddEntities"], #data
40000, #initializion_time
fig) #figure
line = generateLogBoxPlot("Context-Dependent A.C.\nBusiness Case",
122, #plot_id
["Add User", "Add Role", "Add Demarcation", "Add Permission"], #labels
results["AddEntities2"], #data
153000, #initialization_time
fig) #figure
#Show Legend
plt.legend(plt.plot([], [],linestyle="--", color="#A9A9A9", label="Initialization
Time"),["Initialization Time"], loc='center left', bbox_to_anchor=(1, 0.5))
#Show
plt.tight_layout()
plt.show()
Whatever I try, I only seem to be able to modify the ticks of the fist subplot. How could I force/edit them on the second subbplot?

Matplotlib automatically shows or hides minor ticks of log scales depending on the range of values and to some extent the figure size as well. With regards to a y-axis base 10 log scale, here is what I have noticed from testing variations of the example shown further below (using matplotlib 3.3.2 with default settings):
For a figure height of 4 inches (default) or more: when the range of the y-axis covers 9 integer powers or more, the log scale switches from showing major ticks with labels at every power integer as well as all minor tick marks to showing major ticks every two (or more) power integers with no minor ticks (like in your plot on the right).
For a figure height of less than 4 inches (which seems to be your case): there is a more flexible adjustment of the ticks based on the range of the y-axis and the space available.
For your particular example, I would in any case start off by sharing the y-axis to make the plots more comparable. That then leaves two options: either leave the default tick formatting as it is and make do with no minor ticks or else force minor ticks for both plots.
Examples of matplotlib default log tick behavior and how to change it
First, here is an illustration of the matplotlib default behavior with log ticks:
import numpy as np # v 1.19.2
import matplotlib.pyplot as plt # v 3.3.2
import matplotlib.ticker as ticker
# Create sample data with exponentially increasing values for x and
# the y functions and where the y functions are nearly identical
x = 10**np.linspace(0, 3, 30)
y1 = x**2
y2 = x**2.5
# Create figure and subplots containing semilogy plots
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 3))
fig.suptitle('Matplotlib defaults with figure height of 3 inches', y=1.25)
ax1.set_title('Minor ticks are shown\n\n', pad=10)
ax1.semilogy(x, y1, label='y1')
ax1.legend(loc='lower right')
ax2.set_title(f'Minor ticks are not shown:\nthe y range covers less than 9 \
integer\npowers, but the figure height is <4 inches', pad=10)
ax2.semilogy(x, y2, label='y2')
ax2.legend(loc='lower right')
plt.show()
Now, what if the figure height is increased to make more space for the ticks?
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4))
fig.suptitle('Figure height of 4 inches', y=1.15)
ax1.set_title('Minor ticks are not shown:\nthe custom y-axis limits causes the\
\ny range to cover 9 integer powers', pad=10)
ax1.semilogy(x, y1, label='y1')
ax1.legend(loc='lower right')
ax1.set_ylim(1, 10**8) # the minor ticks disappear by changing the axis limits
ax2.set_title('Minor ticks are shown by default:\nthe y range covers less \
than 9 integer\npowers and the figure height is 4 inches', pad=10)
ax2.semilogy(x, y2, label='y2')
ax2.legend(loc='lower right')
plt.show()
This particular example shows that increasing the figure size can solve the problem of minor ticks not showing, but this may often not be the case.
Here is how to force minor ticks to be shown whatever the range of y and the figure size, by using the LogLocator from the matplotlib.ticker module (this example also includes a shared y-axis):
# Add sharey=True
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 3), sharey=True)
fig.suptitle('Figure height of 3 inches with\ncustomized tick locator and shared \
y-axis', y=1.25)
ax1.set_title('Ticks shared by ax2\n', pad=10)
ax1.semilogy(x, y1, label='y1')
ax1.legend(loc='lower right')
ax2.set_title('Customized LogLocator:\nminor ticks are forced to be shown', pad=10)
ax2.semilogy(x, y2, label='y2')
ax2.legend(loc='lower right')
# Set ax2 major and minor tick locators with custom parameters to show
# all major and minor ticks despite the small figure height and the large
# range of y: the numticks argument must be an arbitrary number at least
# one unit above the number of integer powers covering the range of y
nticks = 9
maj_loc = ticker.LogLocator(numticks=nticks)
min_loc = ticker.LogLocator(subs='all', numticks=nticks)
ax2.yaxis.set_major_locator(maj_loc)
ax2.yaxis.set_minor_locator(min_loc)
# The tick labels are formatted as one would expect, so no need to use
# log tick formatters for this example.
plt.show()
If you want to create a plotting function that always shows minor ticks for any range of values, you need to set numticks at a high value.
References: answer by ImportanceOfBeingErnest, matplotlib Log Demo, matplotlib Tick Locators

How to remove a particular grid line corresponding to a custom xtick on a log scale axis?

I'd like to remove the vertical grid line corresponding to the custom xtick (displayed at x = 71 in the below picture). I could remove a horizontal grid line corresponding to the ytick 701 in the below picture by using a hack : since I have no minor tick on the y axis, I defined the custom ytick corresponding to the line that points toward the maximum and crosses the y axis as a minor tick, and then I disabled grid lines for minor ticks on the y axis. Unfortunately I cannot use the same hack on the x axis without disabling the grid lines of the minor ticks and that's something I'd like to avoid at all costs.
Below is a not so minimal albeit still WE.
There are many things I don't understand, the 2 majors are why does
locs, labels = plt.xticks()
not return the locs and labels that are plotted and why I don't get xticks labels displayed as 10^x where x = 0, 1, 2 and 3 but that's outside the scope of the original question.
import matplotlib.pyplot as plt
plt.grid(True)
import numpy as np
# Generate data
x_data = np.arange(1, 1000 , 10)
y_data = np.random.lognormal(1e-5, 3, len(x_data))
y_max = max(y_data)
# plot
plt.xscale('log')
import math
ratio_log = math.log(x_data[np.argmax(y_data)]) / math.log(max(x_data)) # I need to do this in order to plot a horizontal red dashed line that points to the max and do not extend any further.
plt.axhline(y=y_max, xmin=0, xmax = ratio_log, color='r', linestyle='--') # horizontal line pointing to the max y value.
axes = plt.gca()
axes.set_xlim([1, max(x_data)]) # Limits for the x axis.
# custom ticks and labels
# First yticks because I'm able to achieve what I seek
axes.set_yticks([int(y_max)], minor=True) # Sets the custom ytick as a minor one.
from matplotlib.ticker import FormatStrFormatter
axes.yaxis.set_minor_formatter(FormatStrFormatter("%.0f"))
axes.yaxis.grid(False, which='minor') # Removes minor yticks grid. Since I only have my custom yticks as a minor one, this will disable only the grid line corresponding to that ytick. That's a hack.
import matplotlib.ticker as plticker
loc = plticker.MultipleLocator(base=y_max / 3.3) # this locator puts ticks at regular intervals. I ensure the y axis ticks look ok.
axes.yaxis.set_major_locator(loc)
# Now xticks. I'm having a lot of difficulty here, unable to remove the grid of a particular custom xticks.
locs, labels = plt.xticks() # Strangely, this doesn't return the locs and labels that are plotted. There are indeed 2 values that aren't displayed in the plot, here 1.00000000e-01 and 1.00000000e+04. I've got to remove them before I can append my custom loc and label.
# This means that if I do: plt.xticks(locs, labels) right here, it would enlarge both the lower and upper limits on the x axis... I fail to see how that's intuitive or useful at all. Might this be a bug?
locs = np.append(locs[1:-1], np.asarray(x_data[np.argmax(y_data)])) # One of the ugliest hack I have ever seen... to get correct ticks and labels.
labels = (str(int(loc)) for loc in locs) # Just visuals to get integers on the axis.
plt.xticks(locs, labels) # updates the xticks and labels.
plt.plot((x_data[np.argmax(y_data)], x_data[np.argmax(y_data)]), (0, y_max), 'r--') # vertical line that points to the max. Non OO way to do it, so a bad way.
plt.plot(x_data, y_data)
plt.savefig('grid_prob.png')
plt.close()
Example picture below (the code outputs a different picture each time it is executed, but the problem appears in all pictures).

Credit for the idea goes to #ImportanceOfBeingErnest to whom I am extremely grateful.
I removed the grid with
axes.xaxis.grid(False, which='both')
, then I added a grid correspond to each xtick except the custom one with the following loop:
for loc in locs[1:-1]:
if loc != x_data[np.argmax(y_data)]:
plt.axvline(x=loc, color = 'grey', linestyle = '-', linewidth = 0.4)
Insert this code just before the line
plt.xticks(locs, labels) # updates the xticks and labels.
Example of output picture below.

How to get rid of extra white space on subplots with shared axes?

I'm creating a plot using python 3.5.1 and matplotlib 1.5.1 that has two subplots (side by side) with a shared Y axis. A sample output image is shown below:
Notice the extra white space at the top and bottom of each set of axes. Try as I might I can't seem to get rid of it. The overall goal of the figure is to have a waterfall type plot on the left with a shared Y axes with the plot on the right.
Here's some sample code to reproduce the image above.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
%matplotlib inline
# create some X values
periods = np.linspace(1/1440, 1, 1000)
# create some Y values (will be datetimes, not necessarily evenly spaced
# like they are in this example)
day_ints = np.linspace(1, 100, 100)
days = pd.to_timedelta(day_ints, 'D') + pd.to_datetime('2016-01-01')
# create some fake data for the number of points
points = np.random.random(len(day_ints))
# create some fake data for the color mesh
Sxx = np.random.random((len(days), len(periods)))
# Create the plots
fig = plt.figure(figsize=(8, 6))
# create first plot
ax1 = plt.subplot2grid((1,5), (0,0), colspan=4)
im = ax1.pcolormesh(periods, days, Sxx, cmap='viridis', vmin=0, vmax=1)
ax1.invert_yaxis()
ax1.autoscale(enable=True, axis='Y', tight=True)
# create second plot and use the same y axis as the first one
ax2 = plt.subplot2grid((1,5), (0,4), sharey=ax1)
ax2.scatter(points, days)
ax2.autoscale(enable=True, axis='Y', tight=True)
# Hide the Y axis scale on the second plot
plt.setp(ax2.get_yticklabels(), visible=False)
#ax1.set_adjustable('box-forced')
#ax2.set_adjustable('box-forced')
fig.colorbar(im, ax=ax1)
As you can see in the commented out code I've tried a number of approaches, as suggested by posts like https://github.com/matplotlib/matplotlib/issues/1789/ and Matplotlib: set axis tight only to x or y axis.
As soon as I remove the sharey=ax1 part of the second subplot2grid call the problem goes away, but then I also don't have a common Y axis.

Autoscale tends to add a buffer to the data so that all of the data points are easily visible and not part-way cut off by the axes.
Change:
ax1.autoscale(enable=True, axis='Y', tight=True)
to:
ax1.set_ylim(days.min(),days.max())
and
ax2.autoscale(enable=True, axis='Y', tight=True)
to:
ax2.set_ylim(days.min(),days.max())
To get:

matplotlib: Creating two (stacked) subplots with SHARED X axis but SEPARATE Y axis values

I am using matplotlib 1.2.x and Python 2.6.5 on Ubuntu 10.0.4. I am trying to create a SINGLE plot that consists of a top plot and a bottom plot.
The X axis is the date of the time series. The top plot contains a candlestick plot of the data, and the bottom plot should consist of a bar type plot - with its own Y axis (also on the left - same as the top plot). These two plots should NOT OVERLAP.
Here is a snippet of what I have done so far.
datafile = r'/var/tmp/trz12.csv'
r = mlab.csv2rec(datafile, delimiter=',', names=('dt', 'op', 'hi', 'lo', 'cl', 'vol', 'oi'))
mask = (r["dt"] >= datetime.date(startdate)) & (r["dt"] <= datetime.date(enddate))
selected = r[mask]
plotdata = zip(date2num(selected['dt']), selected['op'], selected['cl'], selected['hi'], selected['lo'], selected['vol'], selected['oi'])
# Setup charting
mondays = WeekdayLocator(MONDAY) # major ticks on the mondays
alldays = DayLocator() # minor ticks on the days
weekFormatter = DateFormatter('%b %d') # Eg, Jan 12
dayFormatter = DateFormatter('%d') # Eg, 12
monthFormatter = DateFormatter('%b %y')
# every Nth month
months = MonthLocator(range(1,13), bymonthday=1, interval=1)
fig = pylab.figure()
fig.subplots_adjust(bottom=0.1)
ax = fig.add_subplot(111)
ax.xaxis.set_major_locator(months)#mondays
ax.xaxis.set_major_formatter(monthFormatter) #weekFormatter
ax.format_xdata = mdates.DateFormatter('%Y-%m-%d')
ax.format_ydata = price
ax.grid(True)
candlestick(ax, plotdata, width=0.5, colorup='g', colordown='r', alpha=0.85)
ax.xaxis_date()
ax.autoscale_view()
pylab.setp( pylab.gca().get_xticklabels(), rotation=45, horizontalalignment='right')
# Add volume data
# Note: the code below OVERWRITES the bottom part of the first plot
# it should be plotted UNDERNEATH the first plot - but somehow, that's not happening
fig.subplots_adjust(hspace=0.15)
ay = fig.add_subplot(212)
volumes = [ x[-2] for x in plotdata]
ay.bar(range(len(plotdata)), volumes, 0.05)
pylab.show()
I have managed to display the two plots using the code above, however, there are two problems with the bottom plot:
It COMPLETELY OVERWRITES the bottom part of the first (top) plot - almost as though the second plot was drawing on the same 'canvas' as the first plot - I can't see where/why that is happening.
It OVERWRITES the existing X axis with its own indice, the X axis values (dates) should be SHARED between the two plots.
What am I doing wrong in my code?. Can someone spot what is causing the 2nd (bottom) plot to overwrite the first (top) plot - and how can I fix this?
Here is a screenshot of the plot created by the code above:
[[Edit]]
After modifying the code as suggested by hwlau, this is the new plot. It is better than the first in that the two plots are separate, however the following issues remain:
The X axis should be SHARED by the two plots (i.e. the X axis should be shown only for the 2nd [bottom] plot)
The Y values for the 2nd plot seem to be formmated incorrectly
I think these issues should be quite easy to resolve however, my matplotlib fu is not great at the moment, as I have only recently started programming with matplotlib. any help will be much appreciated.

There seem to be a couple of problems with your code:
If you were using figure.add_subplots with the full
signature of subplot(nrows, ncols, plotNum) it may have
been more apparent that your first plot asking for 1 row
and 1 column and the second plot was asking for 2 rows and
1 column. Hence your first plot is filling the whole figure.
Rather than fig.add_subplot(111) followed by fig.add_subplot(212)
use fig.add_subplot(211) followed by fig.add_subplot(212).
Sharing an axis should be done in the add_subplot command using sharex=first_axis_instance
I have put together an example which you should be able to run:
import matplotlib.pyplot as plt
import matplotlib.ticker as mticker
import matplotlib.dates as mdates
import datetime as dt
n_pts = 10
dates = [dt.datetime.now() + dt.timedelta(days=i) for i in range(n_pts)]
ax1 = plt.subplot(2, 1, 1)
ax1.plot(dates, range(10))
ax2 = plt.subplot(2, 1, 2, sharex=ax1)
ax2.bar(dates, range(10, 20))
# Now format the x axis. This *MUST* be done after all sharex commands are run.
# put no more than 10 ticks on the date axis.
ax1.xaxis.set_major_locator(mticker.MaxNLocator(10))
# format the date in our own way.
ax1.xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d'))
# rotate the labels on both date axes
for label in ax1.xaxis.get_ticklabels():
label.set_rotation(30)
for label in ax2.xaxis.get_ticklabels():
label.set_rotation(30)
# tweak the subplot spacing to fit the rotated labels correctly
plt.subplots_adjust(hspace=0.35, bottom=0.125)
plt.show()
Hope that helps.

You should change this line:
ax = fig.add_subplot(111)
to
ax = fig.add_subplot(211)
The original command means that there is one row and one column so it occupies the whole graph. So your second graph fig.add_subplot(212) cover the lower part of the first graph.
Edit
If you dont want the gap between two plots, use subplots_adjust() to change the size of the subplots margin.

The example from #Pelson, simplified.
import matplotlib.pyplot as plt
import datetime as dt
#Two subplots that share one x axis
fig,ax=plt.subplots(2,sharex=True)
#plot data
n_pts = 10
dates = [dt.datetime.now() + dt.timedelta(days=i) for i in range(n_pts)]
ax[0].bar(dates, range(10, 20))
ax[1].plot(dates, range(10))
#rotate and format the dates on the x axis
fig.autofmt_xdate()
The subplots sharing an x-axis are created in one line, which is convenient when you want more than two subplots:
fig, ax = plt.subplots(number_of_subplots, sharex=True)
To format the date correctly on the x axis, we can simply use fig.autofmt_xdate()
For additional informations, see shared axis demo and date demo from the pylab examples.
This example ran on Python3, matplotlib 1.5.1