We Keep Coding

plot a combo chart with 2 secondary axes variables on matplotlb

I have a dataframe which stores the number of clients, predicted revenue, and actual revenue for a discrete set of products. I would like to plot a combo chart with number of clients on the first y axis as a bar plot, and both predicted and actual revenue plotted on the second y axis with the same scale.
I'm able to create a combo chart with a single secondary y axis using the following:
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
df = pd.DataFrame({
'product' : ['A','B','C','D'],
'number_of_clients' : [234,473,325,389],
'pred_turnover' : [1287,2311,5283,3211],
'act_turnover' : [1221,1927,5433,3888]})
df['number_of_clients'].plot.bar()
df['pred_turnover'].plot(secondary_y=True)
However, I am stuck on how to add a second variable to the secondary y axis using the same scale.
Here is what I would like to create as an end product:

I rewrote it in the general format instead of the df.plot format.
The point is that ax1=axtwinx() is a biaxial graph, and the difference between the maximum and minimum on the right axis is divided by the number of ticks on the left axis to adjust the ticks.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
df = pd.DataFrame({
'product' : ['A','B','C','D'],
'number_of_clients' : [234,473,325,389],
'pred_turnover' : [1287,2311,5283,3211],
'act_turnover' : [1221,1927,5433,3888]})
fig, ax = plt.subplots(figsize=(8,4))
ax.bar(df['product'], df['number_of_clients'], label='number_of_clients')
ax1 = ax.twinx()
ax1.plot(df['product'], df['pred_turnover'], lw=2, color='orange', label='pred_product')
ax1.set_yticks(np.arange(ax1.get_yticks()[0], ax1.get_yticks()[-1], (ax1.get_yticks()[-1] - ax1.get_yticks()[0])/(len(ax.get_yticks())-1)))
ax.grid(which='major', axis='y')
plt.show()

How to change what the axis of a plot is based on? (Python, Matplotlib)

I want to create a graph of 2 * height (which is the meter values in the index) versus the time squared (which are the decimal values in the columns). How can I go about doing this? (In matplotlib)
For clarity, I want the y-axis to be 2 * index values, and the x-axis to be the times squared from within the columns. I would like this to be a series of line graphs
It should end up looking something like this:

In your comment you say you use df1.plot() to draw lines. df.plot() uses dataframe index as x values by default. You say you want the y-axis to be 2 * index values, and the x-axis to be the times squared from within the columns. Your demand involves changes to dataframe values, so I suggest you use ax.plot() for better customization.
Here is a program uses numpy.linalg.lstsq which adopts Least squares internally to get a matched line among given points.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from io import StringIO
TESTDATA = StringIO("""Height Trial:1 Trial:2 Trial:3 Trial:4 Trial:5 Trial:6 Trial:7
1.029 0.4667 0.4616 0.4569 0.4579 0.4653 0.4578 0.4484
1.095 0.4752 0.4773 0.4721 0.4738 0.4713 0.4745 0.4663
1.168 0.4836 0.4834 0.4873 0.4890 0.4890 0.4904 0.4902
1.315 0.5139 0.5117 0.5161 0.5108 0.5224 0.5129 0.5187
1.540 0.5644 0.5677 0.5804 0.5535 0.5636 0.5605 0.5609
1.807 0.6051 0.6124 0.6014 0.6035 0.5977 0.6012 0.6209
""")
df = pd.read_csv(TESTDATA, delim_whitespace=True)
df.set_index(['Height'], inplace=True)
fig, ax = plt.subplots()
for column in df:
x = df[column]**2
y = df.index*2
A = np.vstack([x, np.ones(len(x))]).T
k, b = np.linalg.lstsq(A, y)[0]
line = ax.plot(x, y, 'o')
ax.plot(x, k*x+b, label=f'y={k:.5f}x+{b:.5f}', color=line[0].get_color(), linestyle='dashed')
plt.legend()
plt.xlabel('Fall time, squared (s²)')
plt.ylabel('Twice the height (m)')
plt.title('Measurement of Acceleration due to Gravity on Earth')
plt.show()

import matplotlib.pyplot as plt
plt.plot(list of things on x-axis, list of things on y-axs)
plt.show

import matplotlib.pyplot as plt
plt.plot(times_squared_variable, 2_height_variable, '--', color='choose_a_color')
# Label axis and the plot
plt.xlabel('Name_x_axis')
plt.ylabel('Name_y_axis')
plt.title('Plot_name')
# Show the plot
plt.show()

Pandas plotting--ignore time range

Let's say I have one-minute data during business hours of 8am to 4pm over three days. I would like to plot these data using the pandas plot function:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(51723)
dates = pd.date_range("11/8/2018", "11/11/2018", freq = "min")
df = pd.DataFrame(np.random.rand(len(dates)), index = dates, columns = ['A'])
df = df[(df.index.hour >= 8) & (df.index.hour <= 16)] # filter for business hours
fig, ax = plt.subplots()
df.plot(ax = ax)
plt.show()
However, the plot function also includes overnight hours in the plot, resulting in unintended plotting during this time:
I would the data to be plotted contiguously, ignoring the overnight time (something like this):
What is a good way to plot only the intended hours of 8am to 4pm?

This can be done by plotting each date on a different axis. But things like the labels will get cramped in certain cases.
import datetime
import matplotlib.pyplot as plt
pdates = np.unique(df.index.date) # Unique Dates
fig, ax = plt.subplots(ncols=len(pdates), sharey=True, figsize=(18,6))
# Adjust spacing between suplots
# (Set to 0 for continuous, though labels will overlap)
plt.subplots_adjust(wspace=0.05)
# Plot all data on each subplot, adjust the limits of each accordingly
for i in range(len(pdates)):
df.plot(ax=ax[i], legend=None)
# Hours 8-16 each day:
ax[i].set_xlim(datetime.datetime.combine(pdates[i], datetime.time(8)),
datetime.datetime.combine(pdates[i], datetime.time(16)))
# Deal with spines for each panel
if i !=0:
ax[i].spines['left'].set_visible(False)
ax[i].tick_params(right=False,
which='both',
left=False,
axis='y')
if i != len(pdates)-1:
ax[i].spines['right'].set_visible(False)
plt.show()

how to highlight weekends for time series line plot in python

I am trying to do analysis on a bike share dataset. Part of the analysis includes showing the weekends' demand in date wise plot.
My dataframe in pandas with last 5 row looks like this.
Here is my code for date vs total ride plot.
import seaborn as sns
sns.set_style("darkgrid")
plt.plot(d17_day_count)
plt.show()
.
I want to highlight weekends in the plot. So that it could look something similar to this plot.
I am using Python with matplotlib and seaborn library.

You can easily highlight areas by using axvspan, to get the areas to be highlighted you can run through the index of your dataframe and search for the weekend days. I've also added an example for highlighting 'occupied hours' during a working week (hopefully that doesn't confuse things).
I've created dummy data for a dataframe based on days and another one for hours.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# dummy data (Days)
dates_d = pd.date_range('2017-01-01', '2017-02-01', freq='D')
df = pd.DataFrame(np.random.randint(1, 20, (dates_d.shape[0], 1)))
df.index = dates_d
# dummy data (Hours)
dates_h = pd.date_range('2017-01-01', '2017-02-01', freq='H')
df_h = pd.DataFrame(np.random.randint(1, 20, (dates_h.shape[0], 1)))
df_h.index = dates_h
#two graphs
fig, axes = plt.subplots(nrows=2, ncols=1, sharex=True)
#plot lines
dfs = [df, df_h]
for i, df in enumerate(dfs):
for v in df.columns.tolist():
axes[i].plot(df[v], label=v, color='black', alpha=.5)
def find_weekend_indices(datetime_array):
indices = []
for i in range(len(datetime_array)):
if datetime_array[i].weekday() >= 5:
indices.append(i)
return indices
def find_occupied_hours(datetime_array):
indices = []
for i in range(len(datetime_array)):
if datetime_array[i].weekday() < 5:
if datetime_array[i].hour >= 7 and datetime_array[i].hour <= 19:
indices.append(i)
return indices
def highlight_datetimes(indices, ax):
i = 0
while i < len(indices)-1:
ax.axvspan(df.index[indices[i]], df.index[indices[i] + 1], facecolor='green', edgecolor='none', alpha=.5)
i += 1
#find to be highlighted areas, see functions
weekend_indices = find_weekend_indices(df.index)
occupied_indices = find_occupied_hours(df_h.index)
#highlight areas
highlight_datetimes(weekend_indices, axes[0])
highlight_datetimes(occupied_indices, axes[1])
#formatting..
axes[0].xaxis.grid(b=True, which='major', color='black', linestyle='--', alpha=1) #add xaxis gridlines
axes[1].xaxis.grid(b=True, which='major', color='black', linestyle='--', alpha=1) #add xaxis gridlines
axes[0].set_xlim(min(dates_d), max(dates_d))
axes[0].set_title('Weekend days', fontsize=10)
axes[1].set_title('Occupied hours', fontsize=10)
plt.show()

I tried using the code in the accepted answer but the way the indices are used, the last weekend in the time series does not get highlighted entirely, despite what the image currently shown suggests (this is noticeable mainly with a frequency of 6 hours or more). Also, it does not work if the frequency of the data is higher than daily. This is why I share here a solution that uses the x-axis units so that weekends (or any other recurring time period) can be highlighted without any problem related to the index.
This solution takes only 6 lines of code and it works with any frequency. In the example below, it highlights full weekend days which makes it more efficient than the accepted answer where small frequencies (e.g. 30 minutes) will produce many polygons to cover the whole weekend.
The x-axis limits are used to compute the range of time covered by the plot in terms of days, which is the unit used for matplotlib dates. Then a weekends mask is computed and passed to the where argument of the fill_between plotting function. The masks are processed as right-exclusive so in this case, they must contain Mondays for the highlights to be drawn up to Mondays 00:00. Because plotting these highlights can alter the x-axis limits when weekends occur near the limits, the x-axis limits are set back to the original values after plotting.
Note that contrary to axvspan, the fill_between function needs the y1 and y2 arguments. For some reason, using the default y-axis limits leaves a small gap between the plot frame and the tops and bottoms of the weekend highlights. This issue is solved by running ax.set_ylim(*ax.get_ylim()) just after creating the plot.
import numpy as np # v 1.19.2
import pandas as pd # v 1.1.3
import matplotlib.pyplot as plt # v 3.3.2
import matplotlib.dates as mdates
# Create sample dataset
rng = np.random.default_rng(seed=1234) # random number generator
dti = pd.date_range('2017-01-01', '2017-05-15', freq='D')
counts = 5000 + np.cumsum(rng.integers(-1000, 1000, size=dti.size))
df = pd.DataFrame(dict(Counts=counts), index=dti)
# Draw pandas plot: x_compat=True converts the pandas x-axis units to matplotlib
# date units (not strictly necessary when using a daily frequency like here)
ax = df.plot(x_compat=True, figsize=(10, 5), legend=None, ylabel='Counts')
ax.set_ylim(*ax.get_ylim()) # reset y limits to display highlights without gaps
# Highlight weekends based on the x-axis units
xmin, xmax = ax.get_xlim()
days = np.arange(np.floor(xmin), np.ceil(xmax)+2)
weekends = [(dt.weekday()>=5)|(dt.weekday()==0) for dt in mdates.num2date(days)]
ax.fill_between(days, *ax.get_ylim(), where=weekends, facecolor='k', alpha=.1)
ax.set_xlim(xmin, xmax) # set limits back to default values
# Create appropriate ticks using matplotlib date tick locators and formatters
ax.xaxis.set_major_locator(mdates.MonthLocator())
ax.xaxis.set_minor_locator(mdates.MonthLocator(bymonthday=np.arange(5, 31, step=7)))
ax.xaxis.set_major_formatter(mdates.DateFormatter('\n%b'))
ax.xaxis.set_minor_formatter(mdates.DateFormatter('%d'))
# Additional formatting
ax.figure.autofmt_xdate(rotation=0, ha='center')
title = 'Daily count of trips with weekends highlighted from SAT 00:00 to MON 00:00'
ax.set_title(title, pad=20, fontsize=14);
As you can see, the weekends are always highlighted to the full extent, regardless of where the data starts and ends.
You can find more examples of this solution in the answers I have posted here and here.

I have another suggestion to make in this regard, which takes inspirations from previous posts by other contributors. The code is as follows:
import datetime
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
rng = np.random.default_rng(seed=42) # random number generator
dti = pd.date_range('2021-08-01', '2021-08-31', freq='D')
counts = 5000 + np.cumsum(rng.integers(-1000, 1000, size=dti.size))
df = pd.DataFrame(dict(Counts=counts), index=dti)
weekends = [d for d in df.index if d.isoweekday() in [6,7]]
weekend_list = []
for weekendday in weekends:
d1 = weekendday
d2 = weekendday + datetime.timedelta(days=1)
weekend_list.append((d1, d2))
weekend_df = pd.DataFrame(weekend_list)
sns.set()
plt.figure(figsize=(15, 10), dpi=100)
df.plot()
plt.legend(bbox_to_anchor=(1.02, 0), loc="lower left", borderaxespad=0)
plt.ylabel("Counts")
plt.xlabel("Date of visit")
plt.xticks(rotation = 0)
plt.title("Daily counts of shop visits with weekends highlighted in green")
ax = plt.gca()
for d in weekend_df.index:
print(weekend_df[0][d], weekend_df[1][d])
ax.axvspan(weekend_df[0][d], weekend_df[1][d], facecolor="g", edgecolor="none", alpha=0.5)
ax.relim()
ax.autoscale_view()
plt.savefig("junk.png", dpi=100, bbox_inches='tight', pad_inches=0.2)
The result would be something like the following diagram:

How to plot time series that consists of different dates but same timestamps on one graph in matplotlib

I have data that shows some values collected on three different dates: 2015-01-08, 2015-01-09 and 2015-01-12. For each date there are several data points that have timestamps.
Date/times are in a list and it looks as follows:
['2015-01-08-09:00:00', '2015-01-08-10:00:00', '2015-01-08-11:00:00', '2015-01-08-12:00:00', '2015-01-08-13:00:00', '2015-01-09-14:00:00', '2015-01-09-15:00:00', '2015-01-09-16:00:00', '2015-01-12-09:00:00', '2015-01-12-10:00:00', '2015-01-12-11:00:00']
On the other hand I have corresponding values (floats) in another list:
[12210.0, 12210.0, 12180.0, 12240.0, 12250.0, 12420.0, 12390.0, 12400.0, 12380.0, 12450.0, 12460.0]
To put all this together and plot a graph I use following code:
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import matplotlib.dates as md
import dateutil
from matplotlib.font_manager import FontProperties
timestamps = ['2015-01-08-09:00:00', '2015-01-08-10:00:00', '2015-01-08-11:00:00', '2015-01-08-12:00:00', '2015-01-08-13:00:00', '2015-01-09-14:00:00', '2015-01-09-15:00:00', '2015-01-09-16:00:00', '2015-01-12-09:00:00', '2015-01-12-10:00:00', '2015-01-12-11:00:00']
ticks = [12210.0, 12210.0, 12180.0, 12240.0, 12250.0, 12420.0, 12390.0, 12400.0, 12380.0, 12450.0, 12460.0]
plt.subplots_adjust(bottom=0.2)
plt.xticks( rotation=90 )
dates = [dateutil.parser.parse(s) for s in timestamps]
ax=plt.gca()
ax.set_xticks(dates)
ax.tick_params(axis='x', labelsize=8)
xfmt = md.DateFormatter('%H:%M:%S')
ax.xaxis.set_major_formatter(xfmt)
plt.plot(dates, ticks, label="Price")
plt.xlabel("Date and time", fontsize=12)
plt.ylabel("Price", fontsize=12)
plt.suptitle("Price during last three days", fontsize=12)
plt.legend(loc=0,prop={'size':8})
plt.savefig("figure.pdf")
When I try to plot these datetimes and values I get a messy graph with the line going back and forth.
It looks like the dates are being ignored and only timestamps are taken in account which is the reason for the messy chart. I tried to edit the datetimes to have the same date and consecutive timestamps and it fixed the chart. However, I must have dates as well..
What am I doing wrong?

When I try to plot these datetimes and values I get a messy graph with the line going back and forth.
Your plots are going all over the place because plt.plot connects the dots in the order you give it. If this order is not monotonically increasing in x, then it looks "messy". You can sort the points by x first to fix this. Here is a minimal example:
import numpy as np
import pylab as plt
X = np.random.random(20)
Y = 2*X+np.random.random(20)
idx = np.argsort(X)
X2 = X[idx]
Y2 = Y[idx]
fig,ax = plt.subplots(2,1)
ax[0].plot(X,Y)
ax[1].plot(X2,Y2)
plt.show()