This question already has answers here:
How to set the hue order in Seaborn plots
(1 answer)
Seaborn set color for unique categorical over several pair-plots
(1 answer)
Closed 5 months ago.
I have a pandas dataframe which contains information about distances moved by men and women in different provinces. Apart from an id and the distance, there is also a column for their gender in numerical form (0=men, 1=women, 2=unknown), and a gender label for the legend ('gender_legend' with 'male' and 'female').
I'm trying to plot the relative densities for men and women for each province, and I observed some annoying behaviour: sometimes, the plot for men is drawn in blue and the one for women in orange and sometimes the other way around, with the legends sometimes starting with men and sometimes starting with women (see images). Does anybody have any idea why this is the case, and how to force seaborn to always use the same color for the same gender?
Additionally, if anyone knows how to remove the legend title (here: 'gender_legend'), I'd appreciate this, too. I've already unsuccessfully tried these options.
for province in provinces:
fig, ax = plt.subplots()
sns.kdeplot(data=df[(-(df['gender'] == 2)) & (df['province'] == province)], x='distance', hue='gender_legend', ax=ax)
ax.set(xlabel='Distance (km)', ylabel='density', title=province)
plt.show()
Image 1: men=blue; Image 2: men=orange
for province in provinces:
fig, ax = plt.subplots()
# to sort dataframe by gender so male is always on top
df = df.sort_values(by=['gender'], ascending=True)
# add legend = False to remove legend
sns.kdeplot(data=df[(-(df['gender'] == 2)) & (df['province'] == province)], x='distance', hue='gender_legend', ax=ax, legend=False)
ax.set(xlabel='Distance (km)', ylabel='density', title=province)
plt.show()
Answer explanation:
Seaborn puts male or female on top based on the top row of your dataframe. In your case, it is changing. You need to make sure male is always on top by sorting using gender. Then you will always have blue line for male.
The answer you linked for legend removal actually explains how to remove legend title not legend itself. You just need to provide "legend=False" as a parameter to remove legend.
I have a very huge dataset with a lot of subsidiaries serving three customer groups in various countries, something like this (in reality there are much more subsidiaries and dates):
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame({'subsidiary': ['EU','EU','EU','EU','EU','EU','EU','EU','EU','US','US','US','US','US','US','US','US','US'],'date': ['2019-03','2019-04', '2019-05','2019-03','2019-04', '2019-05','2019-03','2019-04', '2019-05','2019-03','2019-04', '2019-05','2019-03','2019-04', '2019-05','2019-03','2019-04', '2019-05'],'business': ['RETAIL','RETAIL','RETAIL','CORP','CORP','CORP','PUBLIC','PUBLIC','PUBLIC','RETAIL','RETAIL','RETAIL','CORP','CORP','CORP','PUBLIC','PUBLIC','PUBLIC'],'value': [500.36,600.45,700.55,750.66,950.89,1300.13,100.05,120.00,150.01,800.79,900.55,1000,3500.79,5000.36,4500.25,50.17,75.25,90.33]})
print(df)
I'd like to make an analysis per subsidiary by producing a stacked bar chart. To do this, I started by defining the x-axis to be the unique months and by defining a subset per business type in a country like this:
x=df['date'].drop_duplicates()
EUCORP = df[(df['subsidiary']=='EU') & (df['business']=='CORP')]
EURETAIL = df[(df['subsidiary']=='EU') & (df['business']=='RETAIL')]
EUPUBLIC = df[(df['subsidiary']=='EU') & (df['business']=='PUBLIC')]
I can then make a bar chart per business type:
plotEUCORP = plt.bar(x=x, height=EUCORP['value'], width=.35)
plotEURETAIL = plt.bar(x=x, height=EURETAIL['value'], width=.35)
plotEUPUBLIC = plt.bar(x=x, height=EUPUBLIC['value'], width=.35)
However, if I try to stack all three together in one chart, I keep failing:
plotEURETAIL = plt.bar(x=x, height=EURETAIL['value'], width=.35)
plotEUCORP = plt.bar(x=x, height=EUCORP['value'], width=.35, bottom=EURETAIL)
plotEUPUBLIC = plt.bar(x=x, height=EUPUBLIC['value'], width=.35, bottom=EURETAIL+EUCORP)
plt.show()
I always receive the below error message:
ValueError: Missing category information for StrCategoryConverter; this might be caused by unintendedly mixing categorical and numeric data
ConversionError: Failed to convert value(s) to axis units: subsidiary date business value
0 EU 2019-03 RETAIL 500.36
1 EU 2019-04 RETAIL 600.45
2 EU 2019-05 RETAIL 700.55
I tried converting the months into the dateformat and/or indexing it, but it actually confused me further...
I would really appreciate any help/support on any of the following, as I a already spend a lot of hours to try to figure this out (I am still a python noob, sry):
How can I fix the error to create a stacked bar chart?
Assuming, the error can be fixed, is this the most efficient way to create the bar chart (e.g. do I really need to create three sub-dfs per subsidiary, or is there a more elegant way?)
Would it be possible to code an iteration, that produces a stacked bar chart by country, so that I don't need to create one per subsidiary?
As an FYI, stacked bars are not the best option, because they can make it difficult to compare bar values and can easily be misinterpreted. The purpose of a visualization is to present data in an easily understood format; make sure the message is clear. Side-by-side bars are often a better option.
Side-by-side stacked bars are a difficult manual process to construct, it's better to use a figure-level method like seaborn.catplot, which will create a single, easy to read, data visualization.
Bar plot ticks are located by 0 indexed range (not datetimes), the dates are just labels, so it is not necessary to convert them to a datetime dtype.
Tested in python 3.8.11, pandas 1.3.2, matplotlib 3.4.3, seaborn 0.11.2
seaborn
import seaborn as sns
sns.catplot(kind='bar', data=df, col='subsidiary', x='date', y='value', hue='business')
Create grouped and stacked bars
See Stacked Bar Chart and Grouped bar chart with labels
The issue with the creation of the stacked bars in the OP is bottom is being set on the entire dataframe for that group, instead of only the values that make up the bar height.
do I really need to create three sub-dfs per subsidiary. Yes, a DataFrame is needed for every group, so 6, in this case.
Creating the data subsets can be automated using a dict-comprehension to unpack the .groupby object into a dict.
data = {''.join(k): v for k, v in df.groupby(['subsidiary', 'business'])} to create a dict of DataFrames
Access the values like: data['EUCORP'].value
Automating the plot creation is more arduous, as can be seen x depends on how many groups of bars for each tick, and bottom depends on the values for each subsequent plot.
import numpy as np
import matplotlib.pyplot as plt
labels=df['date'].drop_duplicates() # set the dates as labels
x0 = np.arange(len(labels)) # create an array of values for the ticks that can perform arithmetic with width (w)
# create the data groups with a dict comprehension and groupby
data = {''.join(k): v for k, v in df.groupby(['subsidiary', 'business'])}
# build the plots
subs = df.subsidiary.unique()
stacks = len(subs) # how many stacks in each group for a tick location
business = df.business.unique()
# set the width
w = 0.35
# this needs to be adjusted based on the number of stacks; each location needs to be split into the proper number of locations
x1 = [x0 - w/stacks, x0 + w/stacks]
fig, ax = plt.subplots()
for x, sub in zip(x1, subs):
bottom = 0
for bus in business:
height = data[f'{sub}{bus}'].value.to_numpy()
ax.bar(x=x, height=height, width=w, bottom=bottom)
bottom += height
ax.set_xticks(x0)
_ = ax.set_xticklabels(labels)
As you can see, small values are difficult to discern, and using ax.set_yscale('log') does not work as expected with stacked bars (e.g. it does not make small values more readable).
Create only stacked bars
As mentioned by #r-beginners, use .pivot, or .pivot_table, to reshape the dataframe to a wide form to create stacked bars where the x-axis is a tuple ('date', 'subsidiary').
Use .pivot if there are no repeat values for each category
Use .pivot_table, if there are repeat values that must be combined with aggfunc (e.g. 'sum', 'mean', etc.)
# reshape the dataframe
dfp = df.pivot(index=['date', 'subsidiary'], columns=['business'], values='value')
# plot stacked bars
dfp.plot(kind='bar', stacked=True, rot=0, figsize=(10, 4))
I am currently learning K-means, so now I am writing a program in Python to determine different clusters of text that are similar to each other.
So now I got the results for two different clusters (using some fictional words but everything else is the same).
print(dfs) = [ features score
0 America 0.577350
1 new 0.288675
2 president 0.288675
3 Biden 0.288675
, features score
0 Corona 0.593578
1 COVID-19 0.296789
2 research 0.296789
3 health 0.158114]
And dfs is the following type
type(dfs) = list
And the following:
type(dfs[0]) = pandas.core.frame.DataFrame
But how can I easily create bar plots for each cluster in dfs where you see the score attached to each word?
Thanks in advance!
Iterate over the dfs list to access individual dataframes, then, use df.plot.bar with x='features and y='score' as arguments to plot the bar chart relative to that same dataframe. Use the resulting axis from plot function to attach the scores for each bar in the features column. For that, iterate over each patch from the bar plot axis using the x from the rectangle anchor point and the height of the bar as arguments to the annotate function.
...
...
fig, axes = plt.subplots(1, len(dfs))
for num, df in enumerate(dfs):
ax = df.plot.bar(x='features', y='score', ax=axes[num])
for p in ax.patches:
ax.annotate(f'{p.get_height():.4f}', xy=(p.get_x() * 1.01, p.get_height() * 1.01))
axes[num].tick_params(axis='x', labelrotation=30)
axes[num].set_title(f'Dataframe #{num}')
plt.show()
I want to create a Pie chart using single column of my dataframe, say my column name is 'Score'. I have stored scores in this column as below :
Score
.92
.81
.21
.46
.72
.11
.89
Now I want to create a pie chart with the range in percentage.
Say 0-0.4 is 30% , 0.4-0.7 is 35 % , 0.7+ is 35% .
I am using the below code using
df1['bins'] = pd.cut(df1['Score'],bins=[0,0.5,1], labels=["0-50%","50-100%"])
df1 = df.groupby(['Score', 'bins']).size().unstack(fill_value=0)
df1.plot.pie(subplots=True,figsize=(8, 3))
With the above code I am getting the Pie chart, but i don’t know how i can do this using percentage.
my pie chart look like this for now
Cutting the dataframe up into bins is the right first step. After which, you can use value_counts with normalize=True in order to get relative frequencies of values in the bins column. This will let you see percentage of data across ranges that are defined in the bins.
In terms of plotting the pie chart, I'm not sure if I understood correctly, but it seemed like you would like to display the correct legend values and the percentage values in each slice of the pie.
pandas.DataFrame.plot is a good place to see all parameters that can be passed into the plot method. You can specify what are your x and y columns to use, and by default, the dataframe index is used as the legend in the pie plot.
To show the percentage values per slice, you can use the autopct parameter as well. As mentioned in this answer, you can use all the normal matplotlib plt.pie() flags in the plot method as well.
Bringing everything together, this is the resultant code and the resultant chart:
df = pd.DataFrame({'Score': [0.92,0.81,0.21,0.46,0.72,0.11,0.89]})
df['bins'] = pd.cut(df['Score'], bins=[0,0.4,0.7,1], labels=['0-0.4','0.4-0.7','0.7-1'], right=True)
bin_percent = pd.DataFrame(df['bins'].value_counts(normalize=True) * 100)
plot = bin_percent.plot.pie(y='bins', figsize=(5, 5), autopct='%1.1f%%')
Plot of Pie Chart
I have a dataframe which has a column named genres. Each genres has multiple values as movie name. The format is given below:
Movie_val Genre
2 Fantasy
11 Adventure
12 Comedy
2 Fantasy
2 Adventure
11 Adventure
13 Thriller
12 Fantasy
10 Thriller
11 Drama
1 Fantasy
I need to group_by each of the genres based on movie_val and plot each group in a scatter plot like a cluster (Eg: Action genre movies in one cluster or color, Adventure in another, etc.,). I checked the matplot lib library and it expects two values X and Y for a cluster graph. My group_by command will have lot of movie values (eg,. Adventure genres have many values and I am not sure how to plot the values as a group).
Also each of these group_by values should be represented in different color.
I tried the below code for bar plot. But I am looking for scatter one, as below format doesnt allow for scatter.
result = df.groupby(['genres'])['Movie_val'].quantile(0.5)
result.sort_values().plot(kind='barh')
I am trying this in python using pandas library. Any help would be greatly appreciated.
The seaborn library can probably give you what you're after. Of course you still need to pick which columns of your data frame will provide the coordinates for the scatter plot.
import seaborn as sns
g = sns.FacetGrid(df, hue="Genre", size=5)
g.map(plt.scatter, "column name for x dimension", "column name for y dimension", s=50, alpha=.7)
g.add_legend();
See also the examples with more complex faceting here:
https://stanford.edu/~mwaskom/software/seaborn/tutorial/axis_grids.html