Since pandas last update, the x axis is not reading the index as a date. Any clues on what changed? As an example, the following code (Source) creates a random df. The matplotlib part is exactly what I'm have been doing with my real dataset (dates in my data where made using time.strftime("%Y-%m-%d")):
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
date_today = datetime.now()
days = pd.date_range(date_today, date_today + timedelta(7), freq='D')
np.random.seed(seed=1111)
data = np.random.randint(1, high=100, size=len(days))
df = pd.DataFrame({'test': days, 'col2': data})
df = df.set_index('test')
# creates graph:
import matplotlib.pyplot as plt
fig = plt.plot(df.index, df["col2"])
fig = plt.xticks(rotation=30), plt.legend(loc='best'), plt.xlabel("Weeks")
fig = plt.style.use(['bmh', 'seaborn-paper'])
fig = plt.title("Index", fontsize=14, fontweight='bold')
plt.show()
The resulting graph has the x axis in number format. Before updating, my graphs automatically had dates in the index (because the index is in date format).
Pandas used to import the units handlers for datetime64, but as of 0.21 stopped (though it may be back for 0.22). The way to get the old behaviour without explicit conversion is
from pandas.tseries import converter as pdtc
pdtc.register()
Solution 1
Use pandas .plot on the dataframe:
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
date_today = datetime.now()
days = pd.date_range(date_today, date_today + timedelta(7), freq='D')
np.random.seed(seed=1111)
data = np.random.randint(1, high=100, size=len(days))
df = pd.DataFrame({'test': days, 'col2': data})
df = df.set_index('test')
# creates graph:
import matplotlib.pyplot as plt
sub = df.plot()
fig = plt.xticks(rotation=30), plt.legend(loc='best'), plt.xlabel("Weeks")
fig = plt.style.use(['bmh', 'seaborn-paper'])
fig = plt.title("Index", fontsize=14, fontweight='bold')
Solution 2
Convert them Python datetime objects:
fig = plt.plot(df.index.to_pydatetime(), df["col2"])
Result of both approaches
Related
I have the following code:
import pandas as pd
from pandas import DataFrame as df
import matplotlib
from pandas_datareader import data as web
import matplotlib.pyplot as plt
import datetime
import warnings
warnings.filterwarnings("ignore")
start = datetime.date(2020,1,1)
end = datetime.date.today()
stock = 'fb'
data = web.DataReader(stock, 'yahoo', start, end)
data.index = pd.to_datetime(data.index, format ='%Y-%m-%d')
data = data[~data.index.duplicated(keep='first')]
data['year'] = data.index.year
data['month'] = data.index.month
data['week'] = data.index.week
data['day'] = data.index.day
data.set_index('year', append=True, inplace =True)
data.set_index('month',append=True,inplace=True)
data.set_index('week',append=True,inplace=True)
data.set_index('day',append=True,inplace=True)
fig, ax = plt.subplots(dpi=300, figsize =(30,4))
data.plot(y='Close', ax=ax, xlabel= 'Date')
plt.show()
What can I do to interpret the multiindex dates as the x axis in more readable year and month format? Such as in a format like strftime('%y -%m'). A similar question was asked here: Renaming months from number to name in pandas
But I am unable to see how I can use this to rename the x axis. Any help would be appreciated.
You can use the dates from matplotlib. See the following link for more details:
https://matplotlib.org/stable/api/dates_api.html#matplotlib.dates.ConciseDateFormatter
Here is the modified code:
import pandas as pd
from pandas import DataFrame as df
import matplotlib
from pandas_datareader import data as web
import matplotlib.pyplot as plt
import datetime
import warnings
warnings.filterwarnings("ignore")
from matplotlib import dates as mdates
start = datetime.date(2020,1,1)
end = datetime.date.today()
stock = 'fb'
data = web.DataReader(stock, 'yahoo', start, end)
data.index = pd.to_datetime(data.index, format ='%Y-%m-%d')
data = data[~data.index.duplicated(keep='first')]
data['year'] = data.index.year
data['month'] = data.index.month
data['week'] = data.index.week
data['day'] = data.index.day
data.set_index('year', append=True, inplace =True)
data.set_index('month',append=True,inplace=True)
data.set_index('week',append=True,inplace=True)
data.set_index('day',append=True,inplace=True)
fig, ax = plt.subplots(dpi=300, figsize =(15,4))
plt.plot(data.index.get_level_values('Date'), data['Close'])
#--------------------------------------
#Feel free to try different options
#--------------------------------------
#locator = mdates.AutoDateLocator()
locator = mdates.MonthLocator()
formatter = mdates.ConciseDateFormatter(locator)
ax.xaxis.set_major_locator(locator)
ax.xaxis.set_major_formatter(formatter)
plt.show()
Here is the
output.
The formatting of the x-axis works just fine for daily data but fails for monthly data. Then, it just uses 1970 as the base year. Why is that and how can I solve this issue?
import numpy as np
import pandas as pd
import matplotlib.dates as mdates
# generate time series
# switch between 'M' and 'D'
years = pd.date_range(start='1/1/2018', periods=rows, freq='D') #'M')
# randomly generated dummy data
np.random.seed(0)
data1 = np.random.randn(len(years))
# put data together
ts1 = pd.Series(data=data1, index=years)
data = {'check': ts1}
df = pd.concat(data, axis=1)
# generate a plot
ax = df.plot(figsize=(20, 5))
# format xaxis
ax.xaxis.set_major_formatter(mdates.DateFormatter("%Y %b"))
Daily
Monthly
I have a simple dataframe with the time as index and dummy values as example.[]
I did a simple scatter plot as you see here:
Simple question: How to adjust the xaxis, so that all time values from 00:00 to 23:00 are visible in the xaxis? The rest of the plot is fine, it shows all the datapoints, it is just the labeling. Tried different things but didn't work out.
All my code so far is:
import pandas as pd
import seaborn as sns
import matplotlib.dates as mdates
from datetime import time
data = []
for i in range(0, 24):
temp_list = []
temp_list.append(time(i))
temp_list.append(i)
data.append(temp_list)
my_df = pd.DataFrame(data, columns=["time", "values"])
my_df.set_index(['time'],inplace=True)
my_df
fig = sns.scatterplot(my_df.index, my_df['values'])
fig.set(xlabel='time', ylabel='values')
I think you're gonna have to go down to the matplotlib level for this:
import pandas as pd
import seaborn as sns
import matplotlib.dates as mdates
from datetime import time
import matplotlib.pyplot as plt
data = []
for i in range(0, 24):
temp_list = []
temp_list.append(time(i))
temp_list.append(i)
data.append(temp_list)
df = pd.DataFrame(data, columns=["time", "values"])
df.time = pd.to_datetime(df.time, format='%H:%M:%S')
df.set_index(['time'],inplace=True)
ax = sns.scatterplot(df.index, df["values"])
ax.set(xlabel="time", ylabel="measured values")
ax.set_xlim(df.index[0], df.index[-1])
ax.xaxis.set_major_locator(mdates.HourLocator())
ax.xaxis.set_major_formatter(mdates.DateFormatter("%H:%M:%S"))
ax.tick_params(axis="x", rotation=45)
This produces
i think you have 2 options:
convert the time to hour only, for that just extract the hour to new column in your df
df['hour_'] = datetime.hour
than use it as your xaxis
if you need the time in the format you described, it may cause you a visibility problem in which timestamps will overlay each other. i'm using the
plt.xticks(rotation=45, horizontalalignment='right')
ax.xaxis.set_major_locator(plt.MaxNLocator(12))
so first i rotate the text then i'm limiting the ticks number.
here is a full script where i used it:
sns.set()
sns.set_style("whitegrid")
sns.axes_style("whitegrid")
for k, g in df_forPlots.groupby('your_column'):
fig = plt.figure(figsize=(10,5))
wide_df = g[['x', 'y', 'z']]
wide_df.set_index(['x'], inplace=True)
ax = sns.lineplot(data=wide_df)
plt.xticks(rotation=45,
horizontalalignment='right')
ax.yaxis.set_major_locator(plt.MaxNLocator(14))
ax.xaxis.set_major_locator(plt.MaxNLocator(35))
plt.title(f"your {k} in somthing{g.z.unique()}")
plt.tight_layout()
hope i halped
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as dts
def use_matplot():
ax = df.plot(x='year', kind="area" )
years = dts.YearLocator(20)
ax.xaxis.set_major_locator(years)
fig = ax.get_figure()
fig.savefig('output.pdf')
dates = np.arange(1990,2061, 1)
dates = dates.astype('str').astype('datetime64')
df = pd.DataFrame(np.random.randint(0, dates.size, size=(dates.size,3)), columns=list('ABC'))
df['year'] = dates
cols = df.columns.tolist()
cols = [cols[-1]] + cols[:-1]
df = df[cols]
use_matplot()
In the above code, I get an error, "ValueError: year 0 is out of range" when trying to set the YearLocator so as to ensure the X-Axis has year labels for every 20th year. By default the plot has the years show up every 10 years. What am I doing wrong? Desired outcome is simply a plot with 1990, 2010, 2030, 2050 on the bottom. (Instead of default 1990, 2000, 2010, etc.)
Since the years are simple numbers, you may opt for not using them as dates at all and keeping them as numbers.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
dates = np.arange(1990,2061, 1)
df = pd.DataFrame(np.random.randint(0,dates.size,size=(dates.size,3)),columns=list('ABC'))
df['year'] = dates
cols = df.columns.tolist()
cols = [cols[-1]] + cols[:-1]
df = df[cols]
ax = df.plot(x='year', kind="area" )
ax.set_xticks(range(2000,2061,20))
plt.show()
Apart from that, using Matplotlib locators and formatters on date axes created via pandas will most often fail. This is due to pandas using a completely different datetime convention. In order to have more freedom for setting custom tickers for datetime axes, you may use matplotlib. A stackplot can be plotted with plt.stackplot. On such a matplotlib plot, the use of the usual matplotlib tickers is unproblematic.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as dts
dates = np.arange(1990,2061, 1)
df = pd.DataFrame(np.random.randint(0,dates.size,size=(dates.size,3)),columns=list('ABC'))
df['year'] = pd.to_datetime(dates.astype(str))
cols = df.columns.tolist()
cols = [cols[-1]] + cols[:-1]
df = df[cols]
plt.stackplot(df["year"].values, df[list('ABC')].values.T)
years = dts.YearLocator(20)
plt.gca().xaxis.set_major_locator(years)
plt.margins(x=0)
plt.show()
Consider using set_xticklabels to specify values of x axis tick marks:
ax.set_xticklabels(sum([[i,''] for i in range(1990, 2060, 20)], []))
# [1990, '', 2010, '', 2030, '', 2050, '']
I have the following sample codes:
import pandas as pd
import matplotlib.pyplot as plt
dates = ['01/02/2007 00:02:00','01/02/2007 00:04:00','02/02/2007
00:02:00','02/02/2007 00:04:00']
x = pd.to_datetime(dates, format='%d/%m/%Y %H:%M:%S')
y = [0.32,0.33,0.32,0.34]
plt.plot(x,y)
I would like to have the xtick to be just 'Thu' for 01/02/2007 and 'Fri' for 02/02/2007. What is the best possible way to do that?
One possible solution is to change the X-axis format:
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
dates = ['01/02/2007 00:02:00','01/02/2007 00:04:00','02/02/2007 00:02:00','02/02/2007 00:04:00']
x = pd.to_datetime(dates, format='%d/%m/%Y %H:%M:%S')
y = [0.32,0.33,0.32,0.34]
fig, ax = plt.subplots()
ax.plot(x,y)
yearsFmt = mdates.DateFormatter('%a')
ax.xaxis.set_major_formatter(yearsFmt)
plt.show()
The key idea is to get the dayofweek from the DateTime object, like: x.dayofweek. This returns the numeric dayofweek. We can easily get the corresponding name np.array(['Mon','Tue','Wed','Thu','Fri','Sat', 'Sun'])[x.dayofweek]
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
dates = ['01/02/2007 00:02:00','01/02/2007 00:04:00','02/02/2007 00:02:00','02/02/2007 00:04:00']
x = pd.to_datetime(dates, format='%d/%m/%Y %H:%M:%S')
x_d = np.array(['Mon','Tue','Wed','Thu','Fri','Sat', 'Sun'])[x.dayofweek]
y = [0.32,0.33,0.32,0.34]
ser = pd.Series(y, index=x_d)
ser.plot()