We Keep Coding

use a dataframe column for the color of a graph line with matplotlib

I draw a graph like this:
p1=ax1.plot(df['timestamp'], df['break_even'], color='blue', zorder = 0)
but I would like the line to change color based on another column:
p1=ax1.plot(df['timestamp'], df['break_even'], color=df['trade_color'], zorder = 0)
this will not work, I get:
ValueError: Invalid RGBA argument: 0 red
1 green
2 red
3 red
4 green
...
how can this be achieved?
this is an example to test:
data = [[1, 10, 'red'], [2, 15, 'green'], [3, 14, 'blue']]
df = pd.DataFrame(data, columns = ['x', 'y', 'color'])
fig, ax = plt.subplots()
ax.plot(df['x'], df['y'], color='darkorange', zorder = 0)
this will work, but:
ax.plot(df['x'], df['y'], color=df['color'], zorder = 0)
will not. How can I get each line segment to use the color I need? (I have just 2 colors if it makes a difference)

Just plot part of the dataframe each time with the color you want:
import pandas as pd
import matplotlib.pyplot as plt
data = [[1, 10, 'red'], [2, 15, 'green'], [3, 14, 'blue']]
df = pd.DataFrame(data, columns = ['x', 'y', 'color'])
fig, ax = plt.subplots()
for i in df.index:
'''
Get two rows each time, every row has a point (x, y)
Two points can draw a line, use the color defined by first row
'''
partial = df.iloc[i:i+2, :]
ax.plot(partial['x'], partial['y'], color=partial['color'].iloc[0], zorder = 0)
plt.show()

bar chart legend based on coloring of bars by group not value

I've created a bar chart as described here where I have multiple variables (indicated in the 'value' column) and they belong to repeat groups. I've colored the bars by their group membership.
I want to create a legend ultimately equivalent to the colors dictionary, showing the color corresponding to a given group membership.
Code here:
d = {'value': [1, 2, 4, 5, 7 ,10], 'group': [1, 2, 3, 2, 2, 3]}
df = pd.DataFrame(data=d)
colors = {1: 'r', 2: 'b', 3: 'g'}
df['value'].plot(kind='bar', color=[colors[i] for i in df['group']])
plt.legend(df['group'])
In this way, I just get a legend with one color (1) instead of (1, 2, 3).
Thanks!

You can use sns:
sns.barplot(data=df, x=df.index, y='value',
hue='group', palette=colors, dodge=False)
Output:

With pandas, you could create your own legend as follows:
from matplotlib import pyplot as plt
from matplotlib import patches as mpatches
import pandas as pd
d = {'value': [1, 2, 4, 5, 7 ,10], 'group': [1, 2, 3, 2, 2, 3]}
df = pd.DataFrame(data=d)
colors = {1: 'r', 2: 'b', 3: 'g'}
df['value'].plot(kind='bar', color=[colors[i] for i in df['group']])
handles = [mpatches.Patch(color=colors[i]) for i in colors]
labels = [f'group {i}' for i in colors]
plt.legend(handles, labels)
plt.show()

Animation multiple columns as dots with matplotlib very slow for large dataset with networkx graph as background

In my previous question, (How to Animate multiple columns as dots with matplotlib from pandas dataframe with NaN in python), I managed to animate multiple dots from a dataframe as an animation.
However, I wanted to set a background for the animation as a network graph, so that it seems that the dots are moving on the lines of the network.
Using the code from How to Animate multiple columns as dots with matplotlib from pandas dataframe with NaN in python
I've created a new MCV example,
the code is listed below:
import random
import networkx as nx
import matplotlib.pyplot as plt
import numpy as np
import math
import pandas as pd
from matplotlib import animation
#from JSAnimation import IPython_display
%matplotlib inline
# initialise graph object
G = nx.Graph()
color_map =[]
G.add_node(1, pos=(1, 0)); color_map.append('r')
G.add_node(2, pos=(2, 0)); color_map.append('r')
G.add_node(3, pos=(3, -1)); color_map.append('r')
G.add_node(4, pos=(3, 1)); color_map.append('r')
G.add_node(5, pos=(4, -1)) ;color_map.append('r')
G.add_node(6, pos=(4, 1)); color_map.append('r')
G.add_node(7, pos=(5, 0)); color_map.append('r')
G.add_node(8, pos=(6, 0)); color_map.append('r')
e = [(1, 2, 1),
(2, 3, 1),
(2, 4, 2),
(3, 5, 5),
(4, 6, 2),
(5, 7, 1),
(6, 7, 2),
(7, 8, 1)]
G.add_weighted_edges_from(e)
labels = nx.get_edge_attributes(G,'weight')
nx.draw(G,nx.get_node_attributes(G, 'pos'))
nx.draw_networkx_edge_labels(G,nx.get_node_attributes(G, 'pos'),edge_labels=labels)
nx.draw_networkx_labels(G,nx.get_node_attributes(G, 'pos'))
df_x = pd.DataFrame(data=
np.array(
[[np.NaN, np.NaN, np.NaN, np.NaN],
[1, np.nan, np.NaN,np.NaN],
[1.5, 4, np.NaN,np.NaN],
[2, 5, 3,4]]
), index= [1, 2, 3, 4], columns=[1, 2, 3, 4])
print(df_x)
df_y = pd.DataFrame(data=np.array(
[[np.NaN, np.NaN, np.NaN, np.NaN],
[0, np.nan, np.NaN,np.NaN],
[0, -1, np.NaN,np.NaN],
[0, 0, 1,1]]
), index= [1, 2, 3, 4], columns=[1, 2, 3, 4])
%matplotlib notebook
from matplotlib import animation
#from JSAnimation import IPython_display
#from IPython.display import HTML
fig = plt.figure(figsize=(10,10))
ax = plt.axes()
nx.draw(G,nx.get_node_attributes(G, 'pos'),node_size = 10)
n_steps = df_x.index
graph, = plt.plot([],[],'o')
def get_data_x(i):
return df_x.loc[i]
def get_data_y(i):
return df_y.loc[i]
def animate(i):
x = get_data_x(i)
y= get_data_y(i)
graph.set_data(x,y)
return graph,
animation.FuncAnimation(fig, animate, frames=n_steps, repeat=True, blit = True)
This creates a workable animation, which works. But however, when I use a very large dataset ( pandas dataframe index is ~8000 rows * 800 columns instead of the example pandas dataset I posted), the animation takes very long(hour or so) to render and most of the times my browser( google chrome) crashes.
So I thought is maybe due to it needs to redraw the networks graph each frame? How can I set the background as the networkx graph? From there on it is just plotting points right? My actual graph is a bit larger (~5000 nodes, ~6000 edges).
Hopes anyone can help me speed the rendering of the animation up!

After some digging around, I found no 'easy' solution to this problem when trying to animate large datasets into an animation with matplotlib in a jupyter notebook. I just decided to write everything to an mp4 file, which works just as good for animations.
My code for this including the MVC example:
import random
import networkx as nx
import matplotlib.pyplot as plt
import numpy as np
import math
import pandas as pd
from matplotlib import animation
#from JSAnimation import IPython_display
%matplotlib inline
# initialise graph object
G = nx.Graph()
color_map =[]
G.add_node(1, pos=(1, 0)); color_map.append('r')
G.add_node(2, pos=(2, 0)); color_map.append('r')
G.add_node(3, pos=(3, -1)); color_map.append('r')
G.add_node(4, pos=(3, 1)); color_map.append('r')
G.add_node(5, pos=(4, -1)) ;color_map.append('r')
G.add_node(6, pos=(4, 1)); color_map.append('r')
G.add_node(7, pos=(5, 0)); color_map.append('r')
G.add_node(8, pos=(6, 0)); color_map.append('r')
e = [(1, 2, 1),
(2, 3, 1),
(2, 4, 2),
(3, 5, 5),
(4, 6, 2),
(5, 7, 1),
(6, 7, 2),
(7, 8, 1)]
G.add_weighted_edges_from(e)
labels = nx.get_edge_attributes(G,'weight')
nx.draw(G,nx.get_node_attributes(G, 'pos'))
nx.draw_networkx_edge_labels(G,nx.get_node_attributes(G, 'pos'),edge_labels=labels)
nx.draw_networkx_labels(G,nx.get_node_attributes(G, 'pos'))
df_x = pd.DataFrame(data=
np.array(
[[np.NaN, np.NaN, np.NaN, np.NaN],
[1, np.nan, np.NaN,np.NaN],
[1.5, 4, np.NaN,np.NaN],
[2, 5, 3,4]]
), index= [1, 2, 3, 4], columns=[1, 2, 3, 4])
print(df_x)
df_y = pd.DataFrame(data=np.array(
[[np.NaN, np.NaN, np.NaN, np.NaN],
[0, np.nan, np.NaN,np.NaN],
[0, -1, np.NaN,np.NaN],
[0, 0, 1,1]]
), index= [1, 2, 3, 4], columns=[1, 2, 3, 4])
def get_data_x(i):
return df_x.loc[i]
def get_data_y(i):
return sdf_y.loc[i]
def animate(i):
x = get_data_x(i)
y= get_data_y(i)
graph.set_data(x,y)
return graph,
# Set up formatting for the movie files
Writer = animation.writers['ffmpeg']
writer = Writer(fps=15, metadata=dict(artist='Me'), bitrate=1800)
fig = plt.figure(figsize=(20,20))
ax = plt.axes()
nx.draw(G,nx.get_node_attributes(G, 'pos'),node_size = 1)
n_steps = df_x.index
graph, = plt.plot([],[],'o')
ani = animation.FuncAnimation(fig, animate, frames= n_steps, interval=1, repeat=True, blit = True)
ani.save('path/file.mp4', writer=writer)

Dynamic treshold line in a bar chart

I have a stacked bar char where I want to add a dynamic threshold line. The threshold is calculated via a simple formular (90% of each specific value)
Graphic attached. The green line is what I am looking for. Looking forward for any idea how to approach this problem.

Here is what I came up with:
The idea was to have a continuous segment of Xs projected with a constant y value with a 0.5 excess before and after:
import numpy as np
import matplotlib.pyplot as plt
groups = 9
X = list(range(1, groups))
y = [1, 1, 2, 2, 1, 2, 1, 1]
threshold_interval_x = np.arange(min(X) - 0.5, max(X) + 0.5, 0.01).tolist()
threshold_y = []
for y_elt in y:
for i in range(0, int(len(threshold_interval_x) / (groups - 1))):
threshold_y.append(y_elt * 0.9)
plt.bar(X, y, width=0.4, align='center', color='yellow')
plt.plot(threshold_interval_x, threshold_y, color='green')
labels_X = ['PD', 'PZV', 'PP', 'FW', 'BA', 'IA', 'EA', 'NA']
plt.xticks(X, labels_X, rotation='horizontal')
plt.show()
And here's the output:

You could use matplotlibs step-function for this:
import pandas as pd
import matplotlib.pyplot as plt
supposed your data is structured like this:
df = pd.DataFrame({'In': [1, 1, 1, 2 , 0, 2, 0, 0], 'Out': [0, 0, 1, 0, 1, 0, 1, 1]}, index=['PD', 'PZV', 'PP', 'FW', 'BA', 'IA', 'EA', 'NA'])
In Out
PD 1 0
PZV 1 0
PP 1 1
FW 2 0
BA 0 1
IA 2 0
EA 0 1
NA 0 1
Then plotting the bars would be
df.plot(kind='bar', stacked=True, rot=0, color=['gold', 'beige'])
and plotting the threshold line at 90% of the sum would be
plt.step(df.index, df.sum(1) * .9, 'firebrick', where='mid', label = 'Ziel: 90%')
add legend:
plt.legend()
leads to:

Pandas Dataframe plot rows as x values and column header as y values

I have a Pandas DataFrame with displacements for different times (rows) and specific vertical locations (columns names). The goal is to plot the displacements (x axis) for the vertical location (y axis) for a given time (series).
According to the next example (time = 0, 1, 2, 3, 4 and vertical locations = 0.5, 1.5, 2.5, 3.5), how can the displacements be plotted for the times 0 and 3?
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(88)
df = pd.DataFrame({
'time': np.arange(0, 5, 1),
'0.5': np.random.uniform(-1, 1, size = 5),
'1.5': np.random.uniform(-2, 2, size = 5),
'2.5': np.random.uniform(-3, 3, size = 5),
'3.5': np.random.uniform(-4, 4, size = 5),
})
df = df.set_index('time')

You can filter your dataframe to only contain the desired rows. Either by using the positional index
filtered = df.iloc[[0,3],:]
or by using the actualy index of the dataframe,
filtered = df.iloc[(df.index == 3) | (df.index == 0),:]
You can then plot a scatter plot like this:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(88)
df = pd.DataFrame({
'time': np.arange(0, 5, 1),
'0.5': np.random.uniform(-1, 1, size = 5),
'1.5': np.random.uniform(-2, 2, size = 5),
'2.5': np.random.uniform(-3, 3, size = 5),
'3.5': np.random.uniform(-4, 4, size = 5),
})
df = df.set_index('time')
filtered_df = df.iloc[[0,3],:]
#filtered_df = df.iloc[(df.index == 3) | (df.index == 0),:]
loc = list(map(float, df.columns))
fig, ax = plt.subplots()
for row in filtered_df.iterrows():
ax.scatter(row[1], loc, label=row[1].name)
plt.legend()
plt.show()