I have two numeric arrays of equal length, with one array always having the element value >= to the corresponding (same index) element in the second array.
I am trying to visualize in a single graph:
i) difference between the corresponding elements,
ii) values of the corresponding elements in the two arrays.
I have tried plotting the CDF as below:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
arr1 = np.random.uniform(1,20,[25,1])
arr2 = arr1 + np.random.uniform(1,10,[25,1])
df1 = pd.DataFrame(arr1)
df2 = pd.DataFrame(arr2)
fix, ax = plt.subplots()
sns.kdeplot(df1[0], cumulative=True, color='orange', label='arr1')
sns.kdeplot(df2[0], cumulative=True, color='b', label='arr2')
sns.kdeplot(df2[0]-df1[0], cumulative=True, color='r', label='difference')
plt.show()
which gives the following output:
However, it does not capture the difference, and values of the corresponding elements together. For example, suppose the difference between two elements is 3. The two numbers can be 2 and 5, but they can also be 15 and 18, and this can not be determined from the CDF.
Which kind of plotting can visualize both the difference between the elements and the values of the elements?
I do not wish to line plot as below because not much statistical insights can be derived from the visualization.
ax.plot(df1[0])
ax.plot(df2[0])
ax.plot(df2[0]-df1[0])
There are lots of ways to show difference between two values. It really depends on your goal for the chart, how quantitative or qualitative you want to be, or if you want to show the raw data somehow. Here are a few ideas that come to mind that do not involve simple line plots or density functions. I strongly recommend the book Better Data Visualization by Johnathan Schwabish. He discusses interesting considerations regarding data presentation.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib import ticker
arr1 = np.random.uniform(1,20, size=25)
arr2 = arr1 + np.random.uniform(1,10, size=25)
df = pd.DataFrame({
'col1' : arr1,
'col2' : arr2
})
df['diff'] = df.col2 - df.col1
df['sum'] = df.col1 + df.col2
fig, axes = plt.subplots(ncols=2, nrows=3, figsize=(15,15))
axes = axes.flatten()
# Pyramid chart
df_sorted = df.sort_values(by='sum', ascending=True)
axes[0].barh(
y = np.arange(1,26),
width = -df_sorted.col1
)
axes[0].barh(
y = np.arange(1,26),
width = df_sorted.col2
)
# Style axes[0]
style_func(axes[0], 'Pyramid Chart')
# Dot Plot
axes[1].scatter(df.col1, np.arange(1, 26), label='col1')
axes[1].scatter(df.col2, np.arange(1, 26), label='col2')
axes[1].hlines(
y = np.arange(1, 26),
xmin = df.col1, xmax = df.col2,
zorder=0, linewidth=1.5, color='k'
)
# Style axes[1]
legend = axes[1].legend(ncol=2, loc='center', bbox_to_anchor=(0.14,1.025), edgecolor='w')
style_func(axes[1], 'Dot Plot')
set_xlim = axes[1].set_xlim(0,25)
# Dot Plot 2
df_sorted = df.sort_values(by=['col1', 'diff'], ascending=False)
axes[2].scatter(df_sorted.col1, np.arange(1, 26), label='col1')
axes[2].scatter(df_sorted.col2, np.arange(1, 26), label='col2')
axes[2].hlines(
y = np.arange(1, 26),
xmin = df_sorted.col1, xmax = df_sorted.col2,
zorder=0, linewidth=1.5, color='k'
)
# Style axes[2]
legend = axes[2].legend(ncol=2, loc='center', bbox_to_anchor=(0.14,1.025), edgecolor='w')
style_func(axes[2], 'Dot Plot')
set_xlim = axes[2].set_xlim(0,25)
# Dot Plot 3
df_sorted = df.sort_values(by='sum', ascending=True)
axes[3].scatter(-df_sorted.col1, np.arange(1, 26), label='col1')
axes[3].scatter(df_sorted.col2, np.arange(1, 26), label='col2')
axes[3].vlines(x=0, ymin=-1, ymax=27, linewidth=2.5, color='k')
axes[3].hlines(
y = np.arange(1, 26),
xmin = -df_sorted.col1, xmax = df_sorted.col2,
zorder=0, linewidth=2
)
# Style axes[3]
legend = axes[3].legend(ncol=2, loc='center', bbox_to_anchor=(0.14,1.025), edgecolor='w')
style_func(axes[3], 'Dot Plot')
# Strip plot
axes[4].scatter(df.col1, [4] * 25)
axes[4].scatter(df.col2, [6] * 25)
axes[4].set_ylim(0, 10)
axes[4].vlines(
x = [df.col1.mean(), df.col2.mean()],
ymin = [3.5, 5.5], ymax=[4.5,6.5],
color='black', linewidth =2
)
# Style axes[4]
axes[4].yaxis.set_major_locator(ticker.FixedLocator([4,6]))
axes[4].yaxis.set_major_formatter(ticker.FixedFormatter(['col1','col2']))
hide_spines = [axes[4].spines[x].set_visible(False) for x in ['left','top','right']]
set_title = axes[4].set_title('Strip Plot', fontweight='bold')
tick_params = axes[4].tick_params(axis='y', left=False)
grid = axes[4].grid(axis='y', dashes=(8,3), alpha=0.3, color='gray')
# Slope chart
for i in range(25):
axes[5].plot([0,1], [df.col1[i], df.col2[i]], color='k')
align = ['left', 'right']
for i in range(1,3):
axes[5].text(x = i - 1, y = 0, s = 'col' + str(i),
fontsize=14, fontweight='bold', ha=align[i-1])
set_title = axes[5].set_title('Slope chart', fontweight='bold')
axes[5].axis('off')
def style_func(ax, title):
hide_spines = [ax.spines[x].set_visible(False) for x in ['left','top','right']]
set_title = ax.set_title(title, fontweight='bold')
set_xlim = ax.set_xlim(-25,25)
x_locator = ax.xaxis.set_major_locator(ticker.MultipleLocator(5))
y_locator = ax.yaxis.set_major_locator(ticker.FixedLocator(np.arange(1,26, 2)))
spine_width = ax.spines['bottom'].set_linewidth(1.5)
x_tick_params = ax.tick_params(axis='x', length=8, width=1.5)
x_tick_params = ax.tick_params(axis='y', left=False)
What about a parallel coordinates plot with plotly? This will allow to see the distinct values of each original array but then also if they converge on the same diffrence?
https://plot.ly/python/parallel-coordinates-plot/
Related
I try to produce a plot and want to automatically add text (in this case is percentage) to each circle in correspond to each y axis types. Any help would be very helpful.
# import libraries
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
# Make some data
index=['Stream flow',
'Soil moisture',
'Water indices',
'Others',
'Temperature',
'Precipitation',
'Vegetative indices']
value=[2.13, 6.38, 10.64, 12.77, 17.73, 21.99, 28.37]
# create dataframe
percentages = pd.Series(value,index=index)
df = pd.DataFrame({'percentage' : percentages})
df = df.sort_values(by='percentage')
# we first need a numeric placeholder for the y axis
my_range=list(range(1,len(df.index)+1))
fig, ax = plt.subplots(figsize=(15,8))
# create for each expense type an horizontal line that starts at x = 0 with the length
plt.hlines(y=my_range, xmin=0, xmax=df['percentage']-0.5, color='black', alpha=0.8, linewidth=1)
# create for each expense type a dot at the level of the expense percentage value
line=plt.plot(df['percentage'], my_range, "o", markersize=30, color='#fd8c00', alpha=0.6, linewidth=0.3)
# set labels
ax.set_xlabel('Percentage', fontsize=15)
ax.set_ylabel('')
# set axis
ax.tick_params(axis='both', which='major', labelsize=14)
plt.yticks(my_range, df.index)
ax.set_xlim(0,30)
You can use matplotlib.axes.Axes.text:
x_space = 0.4
y_space = 0.05
fontsize = 7
for y_i, val in enumerate(value, 1):
ax.text(x = val - x_space, y = y_i - y_space, s = f'{val}%', fontsize = fontsize)
You have to adjust x_space, y_space and fontsize in order to fit properly the text within the circles.
Complete code
# import libraries
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
# Make some data
index=['Stream flow',
'Soil moisture',
'Water indices',
'Others',
'Temperature',
'Precipitation',
'Vegetative indices']
value=[2.13, 6.38, 10.64, 12.77, 17.73, 21.99, 28.37]
# create dataframe
percentages = pd.Series(value,index=index)
df = pd.DataFrame({'percentage' : percentages})
df = df.sort_values(by='percentage')
# we first need a numeric placeholder for the y axis
my_range=list(range(1,len(df.index)+1))
fig, ax = plt.subplots(figsize=(15,8))
# create for each expense type an horizontal line that starts at x = 0 with the length
plt.hlines(y=my_range, xmin=0, xmax=df['percentage']-0.5, color='black', alpha=0.8, linewidth=1)
# create for each expense type a dot at the level of the expense percentage value
line=plt.plot(df['percentage'], my_range, "o", markersize=30, color='#fd8c00', alpha=0.6, linewidth=0.3)
# set labels
ax.set_xlabel('Percentage', fontsize=15)
ax.set_ylabel('')
# set axis
ax.tick_params(axis='both', which='major', labelsize=14)
plt.yticks(my_range, df.index)
ax.set_xlim(0,30)
x_space = 0.4
y_space = 0.05
for y_i, val in enumerate(value, 1):
ax.text(x = val - x_space, y = y_i - y_space, s = f'{val:>5.2f}%', fontsize = 7)
plt.show()
Same code as above, but with increased circle radius and font, in order to improve readability.
# import libraries
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
# Make some data
index=['Stream flow',
'Soil moisture',
'Water indices',
'Others',
'Temperature',
'Precipitation',
'Vegetative indices']
value=[2.13, 6.38, 10.64, 12.77, 17.73, 21.99, 28.37]
# create dataframe
percentages = pd.Series(value,index=index)
df = pd.DataFrame({'percentage' : percentages})
df = df.sort_values(by='percentage')
# we first need a numeric placeholder for the y axis
my_range=list(range(1,len(df.index)+1))
fig, ax = plt.subplots(figsize=(15,8))
# create for each expense type an horizontal line that starts at x = 0 with the length
plt.hlines(y=my_range, xmin=0, xmax=df['percentage']-0.85, color='black', alpha=0.8, linewidth=1)
# create for each expense type a dot at the level of the expense percentage value
line=plt.plot(df['percentage'], my_range, "o", markersize=50, color='#fd8c00', alpha=0.6, linewidth=0.3)
# set labels
ax.set_xlabel('Percentage', fontsize=15)
ax.set_ylabel('')
# set axis
ax.tick_params(axis='both', which='major', labelsize=14)
plt.yticks(my_range, df.index)
ax.set_xlim(0,30)
ax.set_ylim(0, len(value) + 1)
x_space = 0.75
y_space = 0.06
fontsize = 12
for y_i, val in enumerate(value, 1):
ax.text(x = val - x_space, y = y_i - y_space, s = f'{val:>5.2f}%', fontsize = fontsize)
plt.show()
Even better, you can use matplotlib.axes.Axes.annotate to get rid of x_space and y_space:
fontsize = 12
for y_i, x_i in enumerate(value, 1):
ax.annotate(f'{x_i:>5.2f}%', xy = (x_i, y_i), xytext = (0, 0), textcoords = 'offset points', ha = 'center', va = 'center', fontsize = fontsize)
You still have to adjust the fontsize to properly fit the radius of the circles.
I am plotting separate figures for each attribute and label for each data sample. Here is the illustration:
As illustrated in the the last subplot (Label), my data contains seven classes (numerically) (0 to 6). I'd like to visualize these classes using a different fancy colors and a legend. Please note that I just want colors for last subplot. How should I do that?
Here is the code of above plot:
x, y = test_data["x"], test_data["y"]
# determine the total number of plots
n, off = x.shape[1] + 1, 0
plt.rcParams["figure.figsize"] = (40, 15)
# plot all the attributes
for i in range(6):
plt.subplot(n, 1, off + 1)
plt.plot(x[:, off])
plt.title('Attribute:' + str(i), y=0, loc='left')
off += 1
# plot Labels
plt.subplot(n, 1, n)
plt.plot(y)
plt.title('Label', y=0, loc='left')
plt.savefig(save_file_name, bbox_inches="tight")
plt.close()
First, just to set up a similar dataset:
import matplotlib.pyplot as plt
import numpy as np
x = np.random.random((100,6))
y = np.random.randint(0, 6, (100))
fig, axs = plt.subplots(6, figsize=(40,15))
We could use plt.scatter() to give individual points different marker styles:
for i in range(x.shape[-1]):
axs[i].scatter(range(x.shape[0]), x[:,i], c=y)
Or we could mask the arrays we're plotting:
for i in range(x.shape[-1]):
for j in np.unique(y):
axs[i].plot(np.ma.masked_where(y!=j, x[:,i]), 'o')
Either way we get the same results:
Edit: Ah you've edited your question! You can do exactly the same thing for your last plot only, just modify my code above to take it out of the loop of subplots :)
As suggested, we imitate the matplotlib step function by creating a LineCollection to color the different line segments:
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.collections import LineCollection
from matplotlib.patches import Patch
#random data generation
np.random.seed(12345)
number_of_categories=4
y = np.concatenate([np.repeat(np.random.randint(0, number_of_categories), np.random.randint(1, 30)) for _ in range(20)])
#check the results with less points
#y = y[:10]
x = y[None] * np.linspace(1, 5, 3)[:, None]
x += 2 * np.random.random(x.shape) - 1
#your initial plot
num_plots = x.shape[0] + 1
fig, axes = plt.subplots(num_plots, 1, sharex=True, figsize=(10, 8))
for i, ax in enumerate(axes.flat[:-1]):
ax.plot(x[i,:])
#first we create the matplotlib step function with x-values as their midpoint
axes.flat[-1].step(np.arange(y.size), y, where="mid", color="lightgrey", zorder=-1)
#then we plot colored segments with shifted index simulating the step function
shifted_x = np.arange(y.size+1)-0.5
#and identify the step indexes
idx_steps, = np.nonzero(np.diff(y, prepend=np.inf, append=np.inf))
#create collection of plateau segments
colored_segments = np.zeros((idx_steps.size-1, 2, 2))
colored_segments[:, :, 0] = np.vstack((shifted_x[idx_steps[:-1]], shifted_x[idx_steps[1:]])).T
colored_segments[:, :, 1] = np.repeat(y[idx_steps[:-1]], 2).reshape(-1, 2)
#generate discrete color list
n_levels, idx_levels = np.unique(y[idx_steps[:-1]], return_inverse=True)
colorarr = np.asarray(plt.cm.tab10.colors[:n_levels.size])
#and plot the colored segments
lc_cs = LineCollection(colored_segments, colors=colorarr[idx_levels, :], lw=10)
lines_cs = axes.flat[-1].add_collection(lc_cs)
#scaling and legend generation
axes.flat[-1].set_ylim(n_levels.min()-0.5, n_levels.max()+0.5)
axes.flat[-1].legend([Patch(color=colorarr[i, :]) for i, _ in enumerate(n_levels)],
[f"cat {i}" for i in n_levels],
loc="upper center", bbox_to_anchor=(0.5, -0.15),
ncol=n_levels.size)
plt.show()
Sample output:
Alternatively, you can use broken barh plots or color this axis or even all axes using axvspan.
My end goal is to create a graph that quickly communicates that two data points are between their respective bounds. I could instead of having this information on one graph, create two separate graphs; the chart the data and illustrate the bounds with horizontal lines. If I could have it so that this basic function is done with one graph, it would be much more elegant.
Is there some method I can use to sync the two y-axes so that a certain value A1 on y-axis 1 and A2 on y-axis 2 appear on the same place vertically within the graph, while at the same time, ensuring that another certain value B1 on y-axis 1 and B2 on y-axis 2appears on a separate distinct place vertically within the graph?
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import datetime
### Generate linear data
Temp = 20
pH = 6
DataCopy = pd.DataFrame({'Temp': [], 'pH': [], 'Time': []})
for i in range(10):
DataTime = datetime.datetime.now()
DataCopy = DataCopy.append({'Temp': Temp, 'pH': pH, 'Time': DataTime}, ignore_index=True)
Temp += (-0.5)
pH += (0.2)
### Plot data unto graph w/ double y-axis
sns.lineplot(data=DataCopy, x='Time', y='pH', color = 'red', label = 'Temp')
ax2 = plt.twinx()
sns.lineplot(data=DataCopy, x='Time', y='Temp', color = 'blue', label = 'pH', ax=ax2)
plt.legend()
plt.show()
How this implementation would look if done on separate graphs:
Desired effect:
You seem to want to align two positions on the left axis with two positions on the right axis.
The following approach measures the distance factor of the lower limit vs the two lines, for both axes. Then it applies the largest factor to the axis with the lowest factor. The analog happens for the upper limit of the axes.
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
np.random.seed(2021)
ph_low, ph_high = 7, 9
temp_low, temp_high = 12, 18
data_copy = pd.DataFrame({'Temp': np.random.normal(0.04, 0.4, 100).cumsum() + 10,
'pH': np.random.normal(0.02, 0.2, 100).cumsum() + 6,
'Time': pd.date_range('20211211 08:00:00', freq='1min', periods=100)})
plt.figure(figsize=(12, 5))
ax1 = sns.lineplot(data=data_copy, x='Time', y='pH', color='red', label='Temp')
ax1.axhline(ph_low, color='red', ls=(0, (5, 5, 0)))
ax1.axhline(ph_high, color='red', ls=(0, (5, 5, 0)))
ax2 = ax1.twinx()
sns.lineplot(data=data_copy, x='Time', y='Temp', color='blue', label='pH', ax=ax2)
ax2.axhline(temp_low, color='blue', ls=(0, (0, 5, 5)))
ax2.axhline(temp_high, color='blue', ls=(0, (0, 5, 5)))
handles1, labels1 = ax1.get_legend_handles_labels()
handles2, labels2 = ax2.get_legend_handles_labels()
ax1.legend_.remove()
ax2.legend(handles=handles1 + handles2, labels=labels1 + labels2)
ymin1, ymax1 = ax1.get_ylim()
ymin2, ymax2 = ax2.get_ylim()
fymin1 = (ph_low - ymin1) / (ph_high - ph_low)
fymin2 = (temp_low - ymin2) / (temp_high - temp_low)
if fymin1 < fymin2: # move ymin1 using fymin2
ymin1 = ph_low - fymin2 * (ph_high - ph_low)
else: # move ymin2 using fymin1
ymin2 = temp_low - fymin1 * (temp_high - temp_low)
fymax1 = (ymax1 - ph_high) / (ph_high - ph_low)
fymax2 = (ymax2 - temp_high) / (temp_high - temp_low)
if fymax1 < fymax2: # move ymax1 using fymax2
ymax1 = ph_high + fymax2 * (ph_high - ph_low)
else: # move ymax2 using fymax1
ymax2 = temp_high + fymax1 * (temp_high - temp_low)
ax1.set_ylim(ymin1, ymax1)
ax2.set_ylim(ymin2, ymax2)
plt.show()
I have this piece of code that compares chess openings to their outcomes:
z = df2.groupby(["winner", "opening_name"]).size().unstack().fillna(0).astype(int)
fig, ax = plt.subplots(figsize=(32, 16))
sns.heatmap(z.apply(lambda x: x/x.sum()), xticklabels=True, yticklabels=True, cmap='YlOrBr',
annot=True, linewidths=0.005, linecolor='black', annot_kws={"fontsize":16}, fmt='.2f', cbar=False)
plt.xticks(fontsize = 16)
plt.yticks(fontsize=16)
plt.show()
del z
This is the result:
Is there a way to change seaborn's sns.heatmap's configurations so that it applies the color scaling horizontally instead of vertically? Without changing the given values?
If I change the method's axis(z.apply(lambda x: x/x.sum(), axis = 1), it also changes the actual outcomes:
I want to apply the horizontal color scaling of the 2nd picture to the data of the first picture.
Instead of just setting annot=True, it can also be an array of values (or o labels). You can use a different data= parameter to define the coloring.
from matplotlib import pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
np.random.seed(31416)
weights = np.random.rand(20) ** 1.5
weights /= weights.sum() # random weights summing to 1
N = 2000
df2 = pd.DataFrame({'winner': np.random.choice(['white', 'draw', 'black'], N, p=[0.47, 0.07, 0.46]),
'opening_name': np.random.choice([*'ABCDEFGHIJKLMNOPQRST'], N, p=weights)})
z = df2.groupby(["winner", "opening_name"]).size().unstack().fillna(0).astype(int)
fig, ax = plt.subplots(figsize=(32, 16))
sns.heatmap(data=z.apply(lambda x: x / x.sum(), axis=1),
annot=z.apply(lambda x: x / x.sum()),
xticklabels=True, yticklabels=True, cmap='YlOrBr',
linewidths=0.005, linecolor='black', annot_kws={"fontsize": 16}, fmt='.2f', cbar=False, ax=ax)
ax.tick_params(labelsize=16)
plt.tight_layout()
plt.show()
I'm creating a chart with matplotlib, here is my code:
fig = plt.figure(facecolor='#131722',dpi=155, figsize=(8, 4))
ax1 = plt.subplot2grid((1,2), (0,0), facecolor='#131722')
Colors = [['#0400ff', '#FF0000'], ['#09ff00', '#ff8c00']]
for x in List:
Index = List.index(x)
rate_buy = []
total_buy = []
for y in x['data']['bids']:
rate_buy.append(y[0])
total_buy.append(y[1])
rBuys = pd.DataFrame({'buy': rate_buy})
tBuys = pd.DataFrame({'total': total_buy})
ax1.plot(rBuys.buy, tBuys.total, color=Colors[Index][0], linewidth=0.5, alpha=0.8)
ax1.fill_between(rBuys.buy, 0, tBuys.total, facecolor=Colors[Index][0], alpha=1)
And here is the output:
The problem with the current output is that the colors of the two areas are "merging": basically the area BELOW the blue line should be blue, but instead it's green. How can i set it to be blue, for example, like in my example?
Example List data:
[[9665, 0.07062500000000001], [9666, 0.943708], [9667, 5.683787000000001], [9668, 9.802289], [9669, 11.763305], [9670, 14.286004], [9671, 16.180122], [9672, 23.316723000000003], [9673, 30.915156000000003], [9674, 33.44226200000001], [9675, 36.14526200000001], [9676, 45.76024100000001], [9677, 51.85294700000001], [9678, 58.79529300000001], [9679, 59.05322900000001], [9680, 60.27704500000001], [9681, 60.743885000000006], [9682, 66.75103700000001], [9683, 71.86412600000001], [9684, 73.659636], [9685, 78.08502800000001], [9686, 78.19614200000001], [9687, 79.98396400000001], [9688, 90.55855800000002]]
I guess the hint of #JohanC is correct, you are plotting in the wrong order and overlay your previous plots with new ones.
I tried to recreate a small example where total_buy1 > total_buy0, so in order to get the desired result you first have to plot total_buy1
and then total_buy0:
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
Colors = [['#0400ff', '#FF0000'],
['#09ff00', '#ff8c00']]
n = 100
rate_buy = np.linspace(0, 1000, 100)
total_buy0 = np.linspace(0, 300, n)[::-1] + np.random.normal(scale=10, size=n)
total_buy1 = np.linspace(0, 600, n)[::-1] + np.random.normal(scale=10, size=n)
ax.plot(rate_buy, total_buy1, color=Colors[1][1], linewidth=0.5, alpha=0.8)
ax.fill_between(rate_buy, 0, total_buy1, facecolor=Colors[1][0], alpha=1)
ax.plot(rate_buy, total_buy0, color=Colors[0][1], linewidth=0.5, alpha=0.8)
ax.fill_between(rate_buy, 0, total_buy0, facecolor=Colors[0][0], alpha=1)
I noticed that you use Colors[Index][0] for both plotting calls, so the line and the area will not have different colors.