I need to generate a heat map Where I have to arrange days as columns and week_num as rows and Green for a positive day and red for the negative day.
It should have break for each day and each week.
I have tried using seaborn library but couldn't succeed in plotting this. Can anyone help me with this?
week_num day color_code
1 2020-05-01 red
1 2020-05-02 green
2 2020-05-05 red
2 2020-05-06 red
3 2020-05-13 green
3 2020-05-14 green
3 2020-05-15 red
I am guessing you refer to the day of the week, otherwise it will be a really weird heatmap. You can try something like below, basically in something like your data.frame, get the day of week as another column, then pivot this into a wide format and plot. sns.heatmap does not take in categorical values so you need to replace this with 0,1 and label them accordingly in the legend:
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
dates = pd.date_range(start='1/1/2018', periods=60, freq='1D')
color_code = np.random.choice(['green','red'],60)
df = pd.DataFrame({'dates':dates ,'color_code':color_code})
df['week_num'] = df['dates'].dt.strftime("%W")
df['day_num'] = df['dates'].dt.weekday
fig, ax = plt.subplots(1, 1, figsize = (5, 3))
df_wide = df.pivot_table(index='week_num',columns='day_num',values='color_code',
aggfunc=lambda x:x)
sns.heatmap(df_wide.replace({'green':0,'red':1}),cmap=["#2ecc71","#e74c3c"],
linewidths=1.0,ax=ax)
colorbar = ax.collections[0].colorbar
colorbar.set_ticks([0.25,0.75])
colorbar.set_ticklabels(['green','red'])
Related
I tried drawing subplot through relplot method of seaborn. Now the question is, due to the original dataset is varying, sometimes I don't know how much final subplots will be.
I set col_wrap to limit it, but sometimes the results looks not so good. For example, I set col_wrap = 3, while there are 5 subplots as below:
As the figure shows, the x_axis only occurs in the C D E, which seems strange. I want x axis label is shown in all subplots(from A to E).
Now I already know that facet_kws={'sharex': 'col'} allows plots to have independent axis scales(according to set axis limits on individual facets of seaborn facetgrid).
But I want set labels for x axis of all subplots.I haven't found any solution for it.
Any keyword like set_xlabels in object FacetGrid seems to be useless, because official document announces they only control "on the bottom row of the grid".
FacetGrid.set_xlabels(label=None, clear_inner=True, **kwargs)
Label the x axis on the bottom row of the grid.
The following are my example data and my code:
city date value
0 A 1 9
1 B 1 20
2 C 1 4
3 D 1 33
4 E 1 2
5 A 2 22
6 B 2 32
7 C 2 27
8 D 2 32
9 E 2 18
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
df = pd.read_excel("data/example_data.xlsx")
# print(df)
g = sns.relplot(data=df, x="date", y="value", kind="line", col="city", col_wrap=3,
errorbar=None, facet_kws={'sharex': 'col'})
(g.set_axis_labels("x_axis", "y_axis", )
.set_titles("{col_name}")
.tight_layout()
.add_legend()
)
plt.subplots_adjust(top=0.94, wspace=None, hspace=0.4)
plt.show()
Thanks in advance.
In order to reduce superfluous information, Seaborn makes these inner labels invisible. You can make them visible again:
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
df = pd.DataFrame({'date': np.repeat([1, 2], 5),
'value': np.random.randint(1, 20, 10),
'city': np.tile([*'abcde'], 2)})
# print(df)
g = sns.relplot(data=df, x="date", y="value", kind="line", col="city", col_wrap=3,
errorbar=None, facet_kws={'sharex': 'col'})
g.set_titles("{col_name}")
g.add_legend()
for ax in g.axes.flat:
ax.set_xlabel('x axis', visible=True)
ax.set_ylabel('y axis', visible=True)
plt.subplots_adjust(top=0.94, wspace=None, hspace=0.4)
plt.show()
I want to annotate a plot of multivariate time-series with time intervals (in colour for each type of annotation).
data overview
An example dataset looks like this:
metrik_0 metrik_1 metrik_2 geospatial_id topology_id \
2020-01-01 -0.848009 1.305906 0.924208 12 4
2020-01-01 -0.516120 0.617011 0.623065 8 3
2020-01-01 0.762399 -0.359898 -0.905238 19 3
2020-01-01 0.708512 -1.502019 -2.677056 8 4
2020-01-01 0.249475 0.590983 -0.677694 11 3
cohort_id device_id
2020-01-01 1 1
2020-01-01 1 9
2020-01-01 2 13
2020-01-01 2 8
2020-01-01 1 12
The labels look like this:
cohort_id marker_type start end
0 1 a 2020-01-02 00:00:00 NaT
1 1 b 2020-01-04 05:00:00 2020-01-05 16:00:00
2 1 a 2020-01-06 00:00:00 NaT
desired result
multivariate plot of all the time-series of a cohort_id
highlighting for the markers (different color for each type)
notice the markers might overlay / transparency is useful
there will be attenuation around the marker type a (configured by the number of hours)
I thought about using seaborn/matplotlib for this task.
So far I have come around:
%pylab inline
import seaborn as sns; sns.set()
import matplotlib.dates as mdates
aut_locator = mdates.AutoDateLocator(minticks=3, maxticks=7)
aut_formatter = mdates.ConciseDateFormatter(aut_locator)
g = df[df['cohort_id'] == 1].plot(figsize=(8,8))
g.xaxis.set_major_locator(aut_locator)
g.xaxis.set_major_formatter(aut_formatter)
plt.show()
which is rather chaotic.
I fear, it will not be possible to fit the metrics (multivariate data) into a single plot.
It should be facetted by each column.
However, this again would require to reshape the dataframe for seaborn FacetGrid to work, which also doesn`t quite feel right - especially if the number of elements (time-series) in a cohort_id gets larger.
If FacetGrid is the right way, then something along the lines of: https://seaborn.pydata.org/examples/timeseries_facets.html would be the first part, but the labels would still be missing.
How could the labels be added?
How should the first part be accomplished?
An example of the desired result:
https://imgur.com/9J1EcmI, i.e. one of
for each metric value
code for the example data
The datasets are generated from the code snippet below:
import pandas as pd
import numpy as np
import random
random_seed = 47
np.random.seed(random_seed)
random.seed(random_seed)
def generate_df_for_device(n_observations, n_metrics, device_id, geo_id, topology_id, cohort_id):
df = pd.DataFrame(np.random.randn(n_observations,n_metrics), index=pd.date_range('2020', freq='H', periods=n_observations))
df.columns = [f'metrik_{c}' for c in df.columns]
df['geospatial_id'] = geo_id
df['topology_id'] = topology_id
df['cohort_id'] = cohort_id
df['device_id'] = device_id
return df
def generate_multi_device(n_observations, n_metrics, n_devices, cohort_levels, topo_levels):
results = []
for i in range(1, n_devices +1):
#print(i)
r = random.randrange(1, n_devices)
cohort = random.randrange(1, cohort_levels)
topo = random.randrange(1, topo_levels)
df_single_dvice = generate_df_for_device(n_observations, n_metrics, i, r, topo, cohort)
results.append(df_single_dvice)
#print(r)
return pd.concat(results)
# hourly data, 1 week of data
n_observations = 7 * 24
n_metrics = 3
n_devices = 20
cohort_levels = 3
topo_levels = 5
df = generate_multi_device(n_observations, n_metrics, n_devices, cohort_levels, topo_levels)
df = df.sort_index()
df.head()
marker_labels = pd.DataFrame({'cohort_id':[1,1, 1], 'marker_type':['a', 'b', 'a'], 'start':['2020-01-2', '2020-01-04 05', '2020-01-06'], 'end':[np.nan, '2020-01-05 16', np.nan]})
marker_labels['start'] = pd.to_datetime(marker_labels['start'])
marker_labels['end'] = pd.to_datetime(marker_labels['end'])
In general, you can use either plt.fill_between for horizontal and plt.fill_betweenx for vertical bands. For "bands-within-bands" you can just call the method twice.
A basic example using your data would look like this. I've used fixed values for the position of the bands, but you can put them on the main dataframe and reference them dynamically inside the loop.
import matplotlib.pyplot as plt
fig, ax = plt.subplots(3 ,figsize=(20, 9), sharex=True)
plt.subplots_adjust(hspace=0.2)
metriks = ["metrik_0", "metrik_1", "metrik_2"]
colors = ['#66c2a5', '#fc8d62', '#8da0cb'] #Set2 palette hexes
for i, metric in enumerate(metriks):
df[[metric]].plot(ax=ax[i], color=colors[i], legend=None)
ax[i].set_ylabel(metric)
ax[i].fill_betweenx(y=[-3, 3], x1="2020-01-04 05:00:00",
x2="2020-01-05 16:00:00", color='gray', alpha=0.2)
ax[i].fill_betweenx(y=[-3, 3], x1="2020-01-04 15:00:00",
x2="2020-01-05 00:00:00", color='gray', alpha=0.4)
I have a dataframe with a datetime index:
A B
date
2020-05-04 0 0
2020-05-05 5 0
2020-05-07 2 0
2020-05-09 2 0
2020-05-18 -5 0
2020-05-19 -1 0
2020-05-20 0 0
2020-05-21 1 0
2020-05-22 0 0
2020-05-23 3 0
2020-05-24 1 1
2020-05-25 0 1
2020-05-26 4 1
2020-05-27 3 1
I want to make a lineplot to track A over time and colour the background of the plot red when the values of B are 1. I have implemented this code to make the graph:
from matplotlib import dates as mdates
from matplotlib.colors import ListedColormap
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
cmap = ListedColormap(['white','red'])
ax.plot(data['A'])
ax.set_xlabel('')
plt.xticks(rotation = 30)
ax.xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d'))
ax.pcolorfast(ax.get_xlim(), ax.get_ylim(),
data['B'].values[np.newaxis],
cmap = cmap, alpha = 0.4)
plt.axhline(y = 0, color = 'black')
plt.tight_layout()
This gives me this graph:
But the red region incorrectly starts from 2020-05-21 rather than 2020-05-24 and it doesn't end at the end date in the dataframe. How can I alter my code to fix this?
If you change ax.pcolorfast(ax.get_xlim(), ... by ax.pcolor(data.index, ... you get what you want. The problem with the current code is that by using ax.get_xlim(), it creates a uniform rectangular grid while your index is not uniform (dates are missing), so the coloredmeshed is not like expected. The whole thing is:
from matplotlib import dates as mdates
from matplotlib.colors import ListedColormap
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
cmap = ListedColormap(['white','red'])
fig = plt.figure()
ax = fig.add_subplot()
ax.plot(data['A'])
ax.set_xlabel('')
plt.xticks(rotation = 30)
ax.xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d'))
#here are the two changes use pcolor
ax.pcolor(data.index, #use data.index to create the proper grid
ax.get_ylim(),
data['B'].values[np.newaxis],
cmap = cmap, alpha = 0.4,
linewidth=0, antialiased=True)
plt.axhline(y = 0, color = 'black')
plt.tight_layout()
and you get
I prefer axvspan in this case, see here for more information.
This adaptation will color the areas where data.B==1, including the potential where data.B might not be a continuous block.
With a modified dataframe data from data1.csv (added some more points that are 1):
date A B
5/4/2020 0 0
5/5/2020 5 0
5/7/2020 2 1
5/9/2020 2 1
5/18/2020 -5 0
5/19/2020 -1 0
5/20/2020 0 0
5/21/2020 1 0
5/22/2020 0 0
5/23/2020 3 0
5/24/2020 1 1
5/25/2020 0 1
5/26/2020 4 1
5/27/2020 3 1
from matplotlib import dates as mdates
import pandas as pd
import matplotlib.pyplot as plt
data = pd.read_csv('data1.csv',index_col='date')
data.index = pd.to_datetime(data.index)
fig = plt.figure()
ax = fig.add_subplot()
ax.plot(data['A'])
plt.xticks(rotation = 30)
ax.xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d'))
plt.axhline(y = 0, color = 'black')
# in this case I'm looking for a pair of ones to determine where to color
for i in range(1,len(data.B)):
if data.B[i]==True and data.B[i-1]==True:
plt.axvspan(data.index[i-1], data.index[i], color='r', alpha=0.4, lw=0)
plt.tight_layout()
If data.B==1 will always be "one block" you can do away with the for loop and just use something like this in its place:
first = min(idx for idx, val in enumerate(data.B) if val == 1)
last = max(idx for idx, val in enumerate(data.B) if val == 1)
plt.axvspan(data.index[first], data.index[last], color='r', alpha=0.4, lw=0)
Regarding "why" your data does not align, #Ben.T has this solution.
UPDATE: as pointed out, the for loop could be too crude for large datasets. The following uses numpy to find the falling and rising edges of data.B and then loops on those results:
import numpy as np
diffB = np.append([0], np.diff(data.B))
up = np.where(diffB == 1)[0]
dn = np.where(diffB == -1)[0]
if diffB[np.argmax(diffB!=0)]==-1:
# we have a falling edge before rising edge, must have started 'up'
up = np.append([0], up)
if diffB[len(diffB) - np.argmax(diffB[::-1]) - 1]==1:
# we have a rising edge that never fell, force it 'dn'
dn = np.append(dn, [len(data.B)-1])
for i in range(len(up)):
plt.axvspan(data.index[up[i]], data.index[dn[i]], color='r', alpha=0.4, lw=0)
I have a dataframe that looks like below:
DateTime ID Temperature
2019-03-01 18:36:01 3 21
2019-04-01 18:36:01 3 21
2019-18-01 08:30:01 2 18
2019-12-01 18:36:01 2 12
I would like to visualize this as a plot, where I need the datetime in x-axis, and Temperature on the y axis with a hue of IDs, I tried the below, but i need to see the Temperature distribution for every point more clearly. Is there any other visualization technique?
x= df['DateTime'].values
y= df['Temperature'].values
hue=df['ID'].values
plt.scatter(x, y,hue,color = "red")
you can try:
df.set_index('DateTime').plot()
output:
or you can use:
df.set_index('DateTime').plot(style="x-", figsize=(15, 10))
output:
(This question can be read alone, but is a sequel to: Timeseries from CSV data (Timestamp and events))
I would like to visualize CSV data (from 2 files) as shown below, by a timeseries representation, using python's pandas module (see links below).
Sample data of df1:
TIMESTAMP eventid
0 2017-03-20 02:38:24 1
1 2017-03-21 05:59:41 1
2 2017-03-23 12:59:58 1
3 2017-03-24 01:00:07 1
4 2017-03-27 03:00:13 1
The 'eventid' column always contains the value of 1, and I am trying to show the sum of events for each day in the dataset.
The 2nd dataset, df0, has similar structure but contains only zeros:
Sample data of df0:
TIMESTAMP eventid
0 2017-03-21 01:38:24 0
1 2017-03-21 03:59:41 0
2 2017-03-22 11:59:58 0
3 2017-03-24 01:03:07 0
4 2017-03-26 03:50:13 0
The x-axis label only shows the same date, and my question is: How can the different dates be shown? (What causes the same date to be shown multiple times on x labels?)
script so far:
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
df1 = pd.read_csv('timestamp01.csv', parse_dates=True, index_col='TIMESTAMP')
df0 = pd.read_csv('timestamp00.csv', parse_dates=True, index_col='TIMESTAMP')
f, (ax1, ax2) = plt.subplots(1, 2)
ax1.plot(df0.resample('D').size())
ax1.set_xlim([pd.to_datetime('2017-01-27'), pd.to_datetime('2017-04-30')])
ax1.xaxis.set_major_formatter(ticker.FixedFormatter
(df0.index.strftime('%Y-%m-%d')))
plt.setp(ax1.xaxis.get_majorticklabels(), rotation=15)
ax2.plot(df1.resample('D').size())
ax2.set_xlim([pd.to_datetime('2017-03-22'), pd.to_datetime('2017-04-29')])
ax2.xaxis.set_major_formatter(ticker.FixedFormatter(df1.index.strftime
('%Y-%m-%d')))
plt.setp(ax2.xaxis.get_majorticklabels(), rotation=15)
plt.show()
Output: (https://www.dropbox.com/s/z21koflkzglm6c3/figure_1.png?dl=0)
Links I have tried to follow:
http://pandas.pydata.org/pandas-docs/stable/visualization.html
Multiple timeseries plots from Pandas Dataframe
Pandas timeseries plot setting x-axis major and minor ticks and labels
Any help is much appreciated.
Making the example reproducible, we can create the following text file (data/timestamp01.csv):
TIMESTAMP;eventid
2017-03-20 02:38:24;1
2017-03-21 05:59:41;1
2017-03-23 12:59:58;1
2017-03-24 01:00:07;1
2017-03-27 03:00:13;1
(same for data/timestamp00.csv). We can then read them in
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
df1 = pd.read_csv('data/timestamp01.csv', parse_dates=True, index_col='TIMESTAMP', sep=";")
df0 = pd.read_csv('data/timestamp00.csv', parse_dates=True, index_col='TIMESTAMP', sep=";")
Plotting them
f, (ax1, ax2) = plt.subplots(1, 2)
ax1.plot(df0.resample('D').size())
ax2.plot(df1.resample('D').size())
plt.setp(ax1.xaxis.get_majorticklabels(), rotation=30, ha="right")
plt.setp(ax2.xaxis.get_majorticklabels(), rotation=30, ha="right")
plt.show()
results in
which is the desired plot.