Related
This question already has answers here:
Selecting a column to make histogram
(1 answer)
How to plot a histogram of a single dataframe column and exclude 0s
(1 answer)
How do I only plot histogram for only certain columns of a data-frame in pandas
(1 answer)
Closed 7 months ago.
I would like to construct a histogram (or empirical distribution function) for a dataframe column (=a column contatining a number of daily observations).
The dataframe column has the following structure (below)
Thanks in advance!
df1 = pd.DataFrame({"date": pd.to_datetime(["2021-3-22", "2021-4-7", "2021-4-18", "2021-5-12","2022-3-22", "2022-4-7", "2022-4-18", "2022-5-12"]),
"x": [1, 1, 1, 3, 2, 3,4,2 ]})
date x
0 2021-03-22 1
1 2021-04-07 1
2 2021-04-18 1
3 2021-05-12 3
4 2022-03-22 2
5 2022-04-07 3
6 2022-04-18 4
7 2022-05-12 2
Pandas has plotting feature with matplotlib backend as default, so you can do it like this:
df1.x.hist()
More: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.plot.html
You can do this with pyplot:
from matplotlib import pyplot as plt
plt.hist(df1.x)
#if you just want to look at the plot
plt.show()
#if you want to save the plot to a file
plt.savefig('filename.png')
Here's the documentation with all the options: https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.hist.html.
I have a dataframe with the weight and the number of measures of each user. The df looks like:
id_user
weight
number_of_measures
1
92.16
4
2
80.34
5
3
71.89
11
4
81.11
7
5
77.23
8
6
92.37
2
7
88.18
3
I would like to see an histogram with the attribute of the table (weight, but I want to do it for both cases) at the x-axis and the frequency in the y-axis.
Does anyone know how to do it with matplotlib?
Ok, it seems to be quite easy:
import pandas as pd
import matplotlib.pyplot as plt
hist = df.hist(bins=50)
plt.show()
I have the code as below to plot multiple plots on the same figure
fig, ax = plt.subplots(figsize=(25, 10))
def wl_ratioplot(wavelength1,wavelength2, dataframe, x1=0.1,x2=1.5,y1=-500,y2=25000):
a=dataframe[['asphalt_index','layer_thickness',wavelength1,wavelength2]].copy()
sns.scatterplot(x=a[wavelength1]/a[wavelength2],y=a['layer_thickness'],data=a)
ax.set_xlim(x1,x2)
ax.set_ylim(y1,y2)
leg = "{} vs {}".format(wavelength1,wavelength2)
print(leg) #this line is only to see the variable legend has the proper content
ax.legend(leg)
wl_ratioplot(wave_lengths[2],wave_lengths[0],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
wl_ratioplot(wave_lengths[0],wave_lengths[1],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
wl_ratioplot(wave_lengths[3],wave_lengths[1],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
wl_ratioplot(wave_lengths[3],wave_lengths[0],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
wl_ratioplot(wave_lengths[2],wave_lengths[1],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
I get the plot as the below pic where the legend seems to be first 5 letters separately even though the variable legend has the right content
There was another similar question & the solution was to put a square bracket to the variable legend. I tried this with the code as below.
fig, ax = plt.subplots(figsize=(25, 10))
def wl_ratioplot(wavelength1,wavelength2, dataframe, x1=0.1,x2=1.5,y1=-500,y2=25000):
a=dataframe[['asphalt_index','layer_thickness',wavelength1,wavelength2]].copy()
sns.scatterplot(x=a[wavelength1]/a[wavelength2],y=a['layer_thickness'],data=a)
ax.set_xlim(x1,x2)
ax.set_ylim(y1,y2)
leg = "{} vs {}".format(wavelength1,wavelength2)
print(leg)#this line is only to see the variable legend has the proper content
ax.legend([leg])
wl_ratioplot(wave_lengths[2],wave_lengths[0],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
wl_ratioplot(wave_lengths[0],wave_lengths[1],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
wl_ratioplot(wave_lengths[3],wave_lengths[1],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
wl_ratioplot(wave_lengths[3],wave_lengths[0],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
wl_ratioplot(wave_lengths[2],wave_lengths[1],dataframe=train_df_wo_outliers,x1=-.1,x2=3)
Now I get the full legend but only the first legend is shown as the pic below
Can someone let me know how to get the full legend for all the plots? Thanks.
dummy data (the plot in pic will NOT match)
14nm 15nm 16nm 17nm 18nm 19nm layer_thickness
1 2 3 4 5 6 0
1 2 3 4 5 6 0
3 5 7 9 11 13 5700
1 2 3 4 5 6 0
3 5 7 9 11 13 8600
1 2 3 4 5 6 0
3 5 7 9 11 13 5000
1 2 3 4 5 6 0
45 55 65 75 85 95 100
1 2 3 4 5 6 0
8 15 22 29 36 43 16600
wave_lengths=['15nm','16nm','14nm','18nm']
Answer Update
Based on answer from Quang Hoang. The output pics using scatter plot from matplotlib & sns.scatterplot
With plt it is pretty natural:
def wl_ratioplot(wavelength1,wavelength2, dataframe,
x1=0.1,x2=1.5,y1=-500,y2=25000,
ax=None):
leg = "{} vs {}".format(wavelength1,wavelength2)
# set the label here, and let plt deal with it
# also, you don't need to copy the dataframe:
ax.scatter(x=dataframe[wavelength1]/dataframe[wavelength2],
y=dataframe['layer_thickness'],label=leg)
ax.set_xlim(x1,x2)
ax.set_ylim(y1,y2)
fig, ax = plt.subplots(figsize=(25, 10))
wl_ratioplot(wave_lengths[2],wave_lengths[0],dataframe=df,x1=-.1,x2=3, ax=ax)
wl_ratioplot(wave_lengths[0],wave_lengths[1],dataframe=df,x1=-.1,x2=3, ax=ax)
wl_ratioplot(wave_lengths[3],wave_lengths[1],dataframe=df,x1=-.1,x2=3, ax=ax)
wl_ratioplot(wave_lengths[3],wave_lengths[0],dataframe=df,x1=-.1,x2=3, ax=ax)
wl_ratioplot(wave_lengths[2],wave_lengths[1],dataframe=df,x1=-.1,x2=3, ax=ax)
ax.legend()
Output:
every time you call the function wl_ratioplot the legend is being reset the final value. use a array to store all the legends then access it all through a loop.
ax.legend([leg]) #it is resetting the legend after each call.
use a legends = [];
legends.append([leg])
after all function calls, draw the legend differently
ax.legend(legends)
My data has three categorical variables I'm trying to visualize:
City (one of five)
Occupation (one of four)
Blood type (one of four)
So far, I've succeeded in grouping the data in a way that I think will be easy to work with:
import numpy as np, pandas as pd
# Make data
cities = ['Tijuana','Las Vegas','Los Angeles','Anaheim','Atlantis']
occupations = ['Doctor','Lawyer','Engineer','Drone security officer']
bloodtypes = ['A','B','AB','O']
df = pd.DataFrame({'City': np.random.choice(cities,500),
'Occupation': np.random.choice(occupations,500),
'Blood Type':np.random.choice(bloodtypes,500)})
# You need to make a dummy column, otherwise the groupby returns an empty df
df['Dummy'] = np.ones(500)
# This is now what I'd like to plot
df.groupby(by=['City','Occupation','Blood Type']).count().unstack(level=1)
Returns:
Dummy
Occupation Doctor Drone security officer Engineer Lawyer
City Blood Type
Anaheim A 7 7 7 7
AB 6 10 8 5
B 2 10 4 2
O 4 3 3 6
Atlantis A 6 5 5 7
AB 12 7 7 10
B 7 4 7 3
O 7 4 6 4
Las Vegas A 8 4 8 5
AB 5 6 8 9
B 6 10 6 6
O 6 9 5 9
Los Angeles A 7 4 8 8
AB 9 8 8 8
B 3 6 4 1
O 9 11 11 9
Tijuana A 3 4 5 3
AB 9 5 5 7
B 3 6 4 9
O 3 5 5 8
My goal is to create something like the Seaborn swarmplot shown below, which comes from the Seaborn documentation. Seaborn applies jitter to the quantitative data so that you can see the individual data points and their hues:
With my data, I'd like to plot City on the x-axis and Occupation on the y-axis, applying jitter to each, and then hue by Blood type. However, sns.swarmplot requires one of the axes to be quantitative:
sns.swarmplot(data=df,x='City',y='Occupation',hue='Blood Type')
returns an error.
An acceptable alternative might be to create 20 categorical bar plots, one for each intersection of City and Occupation, which I would do by running a for loop over each category, but I can't imagine how I'd feed that to matplotlib subplots to get them in a 4x5 grid.
The most similar question I could find was in R, and the asker only wanted to indicate the most common value for the third variable, so I didn't get any good ideas from there.
Thanks for any help you can provide.
Alright, I got to work on the "acceptable alternative" today and I have found a solution using basically pure matplotlib (but I stuck the Seaborn styling on top of it, just because).
import numpy as np, pandas as pd
import matplotlib.pyplot as plt
from matplotlib.cm import get_cmap
from matplotlib.patches import Patch
import seaborn as sns
# Make data
cities = ['Tijuana','Las Vegas','Los Angeles','Anaheim','Atlantis']
occupations = ['Doctor','Lawyer','Engineer','Drone security officer']
bloodtypes = ['A','B','AB','O']
df = pd.DataFrame({'City': np.random.choice(cities,500),
'Occupation': np.random.choice(occupations,500),
'Blood Type':np.random.choice(bloodtypes,500)})
# Make a dummy column, otherwise the groupby returns an empty df
df['Dummy'] = np.ones(500)
# This is now what I'd like to plot
grouped = df.groupby(by=['City','Occupation','Blood Type']).count().unstack()
# List of blood types, to use later as categories in subplots
kinds = grouped.columns.levels[1]
# colors for bar graph
colors = [get_cmap('viridis')(v) for v in np.linspace(0,1,len(kinds))]
sns.set(context="talk")
nxplots = len(grouped.index.levels[0])
nyplots = len(grouped.index.levels[1])
fig, axes = plt.subplots(nxplots,
nyplots,
sharey=True,
sharex=True,
figsize=(10,12))
fig.suptitle('City, occupation, and blood type')
# plot the data
for a, b in enumerate(grouped.index.levels[0]):
for i, j in enumerate(grouped.index.levels[1]):
axes[a,i].bar(kinds,grouped.loc[b,j],color=colors)
axes[a,i].xaxis.set_ticks([])
axeslabels = fig.add_subplot(111, frameon=False)
plt.tick_params(labelcolor='none', top=False, bottom=False, left=False, right=False)
plt.grid(False)
axeslabels.set_ylabel('City',rotation='horizontal',y=1,weight="bold")
axeslabels.set_xlabel('Occupation',weight="bold")
# x- and y-axis labels
for i, j in enumerate(grouped.index.levels[1]):
axes[nyplots,i].set_xlabel(j)
for i, j in enumerate(grouped.index.levels[0]):
axes[i,0].set_ylabel(j)
# Tune this manually to make room for the legend
fig.subplots_adjust(right=0.82)
fig.legend([Patch(facecolor = i) for i in colors],
kinds,
title="Blood type",
loc="center right")
Returns this:
I'd appreciate any feedback, and I'd still love it if someone could provide the preferred solution.
Python beginner/first poster here.
I'm running into trouble adding color bars to scatter plots. I have two types of plot: one that shows all the data color-coded by date, and one that shows just the maximum values of my data color-coded by date. In the first case, I can use the df.index (which is datetime) to make my color bar, but in the second case, I am using df2['col'].idxmax to generate the colors because my df2 is a df.groupby object which I'm using to generate the daily maximums in my data, and it does not have an accessible index.
For the first type of plot, I have succeeded in generating a date-based color bar with the code below, cobbled together from online examples:
fig, ax = plt.subplots(1,1, figsize=(20,20))
smap=plt.scatter(df.col1, df.col2, s=140,
c=[date2num(i.date()) for i in df.index],
marker='.')
cb = fig.colorbar(smap, orientation='vertical',
format=DateFormatter('%d %b %y'))
However for the second type of plot, where I am trying to use df2['col'].idxmax to create the date series instead of df.index, the following does not work:
for n in cols1:
for m in cols2:
fig, ax = plt.subplots(1,1, figsize=(15,15))
maxTimes=df2[n].idxmax()
PlottableTimes=maxTimes.dropna() #some NaNs in the
#.idxmax series were giving date2num trouble
smap2=plt.scatter(df2[n].max(), df2[m].max(),
s=160, c=[date2num(i.date()) for i in PlottableTimes],
marker='.')
cb2 = fig.colorbar(smap2, orientation='vertical',
format=DateFormatter('%d %b %y'))
plt.show()
The error is: 'length of rgba sequence should be either 3 or 4'
Because the error was complaining of the color argument, I separately checked the output of the color (that is, c=) arguments in the respective plotting commands, and both look similar to me, so I can't figure out why one color argument works and the other doesn't:
one that works:
[736809.0,
736809.0,
736809.0,
736809.0,
736809.0,
736809.0,
736809.0,
736809.0,
736809.0,
736809.0,
...]
one that doesn't work:
[736845.0,
736846.0,
736847.0,
736848.0,
736849.0,
736850.0,
736851.0,
736852.0,
736853.0,
736854.0,
...]
Any suggestions or explanations? I'm running python 3.5.2. Thank you in advance for helping me understand this.
Edit 1: I made the following example for others to explore, and in the process realized the crux of the issue is different than my first question. The code below works the way I want it to:
df=pd.DataFrame(np.random.randint(low=0, high=10, size=(169, 8)),
columns=['a', 'b', 'c', 'd', 'e','f','g','h']) #make sample data
date_rng = pd.date_range(start='1/1/2018', end='1/8/2018', freq='H')
df['i']=date_rng
df = df.set_index('i') #get a datetime index
df['ts']=date_rng #get a datetime column to group by
from pandas import Grouper
df2=df.groupby(Grouper(key='ts', freq='D'))
for n in ['a','b','c','d']: #now make some plots
for m in ['e','f','g','h']:
print(m)
print(n)
fig, ax = plt.subplots(1,1, figsize=(5,5))
maxTimes=df2[n].idxmax()
PlottableTimes=maxTimes.dropna()
smap=plt.scatter(df2[n].max(), df2[m].max(), s=160,
c=[date2num(i.date()) for i in PlottableTimes],
marker='.')
cb = fig.colorbar(smap, orientation='vertical',
format=DateFormatter('%d %b %y'))
plt.show()
The only difference between my real data and this example is that my real data has many NaNs scattered throughout. So, I think what is going wrong is that the 'c=' argument isn't long enough for the plotting command to interpret it as covering the whole date range...? For example, if I manually put in the output of the c= command, I get the following code which also works:
for n in ['a','b','c','d']:
for m in ['e','f','g','h']:
print(m)
print(n)
fig, ax = plt.subplots(1,1, figsize=(5,5))
maxTimes=df2[n].idxmax()
PlottableTimes=maxTimes.dropna()
smap=plt.scatter(df2[n].max(), df2[m].max(), s=160,
c=[736809.0, 736810.0, 736811.0, 736812.0, 736813.0, 736814.0, 736815.0, 736816.0],
marker='.')
cb = fig.colorbar(smap, orientation='vertical',
format=DateFormatter('%d %b %y'))
plt.show()
But, if I shorten the c= array by some amount, to emulate what is happening in my code when NaNs are being dropped from idxmax, it gives the same error I am seeing:
for n in ['a','b','c','d']:
for m in ['e','f','g','h']:
print(m)
print(n)
fig, ax = plt.subplots(1,1, figsize=(5,5))
maxTimes=df2[n].idxmax()
PlottableTimes=maxTimes.dropna()
smap=plt.scatter(df2[n].max(), df2[m].max(), s=160,
c=[736809.0, 736810.0, 736811.0, 736812.0, 736813.0, 736814.0],
marker='.')
cb = fig.colorbar(smap, orientation='vertical',
format=DateFormatter('%d %b %y'))
plt.show()
So this means the real question is: how can I grab the grouper column after grouping from the groupby object, when none of the columns appear to be grab-able with df2.col? I would like to be able to grab 'ts' from the following and use it to be the color data, instead of using idxmax:
df2['a'].max()
ts
2018-01-01 9
2018-01-02 9
2018-01-03 9
2018-01-04 9
2018-01-05 9
2018-01-06 9
2018-01-07 9
2018-01-08 8
Freq: D, Name: a, dtype: int64
Essentially, your Grouper call is similar to indexing on your date time column and callingpandas.DataFrame.resample specifying the aggregate function:
df.set_index('ts').resample('D').max()
# a b c d e f g h
# ts
# 2018-01-01 9 9 8 9 9 9 9 9
# 2018-01-02 9 9 9 9 9 9 9 9
# 2018-01-03 9 9 9 9 9 9 9 9
# 2018-01-04 9 9 9 9 9 9 9 9
# 2018-01-05 9 9 9 9 9 9 9 9
# 2018-01-06 9 9 9 8 9 9 9 9
# 2018-01-07 9 9 9 9 9 9 9 9
# 2018-01-08 2 8 6 3 1 3 2 7
Therefore, the return of df2['a'].max() is a Pandas Resampler object, very similar to a Pandas Series and hence carries the index property which you can use for color bar specification:
df['a'].max().index
# DatetimeIndex(['2018-01-01', '2018-01-02', '2018-01-03', '2018-01-04',
# '2018-01-05', '2018-01-06', '2018-01-07', '2018-01-08'],
# dtype='datetime64[ns]', name='ts', freq='D')
From there you can pass into date2num without list comprehension:
date2num(df2['a'].max().index)
# array([736695., 736696., 736697., 736698., 736699., 736700., 736701., 736702.])
Altogether, simply use above in loop without needing maxTimes or PlottableTimes:
fig, ax = plt.subplots(1, 1, figsize = (5,5))
smap = plt.scatter(df2[n].max(), df2[m].max(), s = 160,
c = date2num(df2[n].max().index),
marker = '.')
cb = fig.colorbar(smap, orientation = 'vertical',
format = DateFormatter('%d %b %y'))