We Keep Coding

Python: Creating AreaChart3D in a recursive way

My target is to create some AreaChart3D plots in an automatically way.
Precisely, for example I have the following picture:
This table is automatically outputed by a tool.
I can have only one graph, maybe 2 graphs or even 100 graphs (does not matter so much), it is important every time I will have this kind of behavior with Location, Speed, and some times inside.
Now, I would like to have in the second sheet(ws2_obj) 4 graphs or maybe 2 graphs depends how many graphs will be outputed by the tool.
If I would have had a fixed number of graph it would have been easier.
Because this graphs are not fixed i have to cover the entire sheet and I do not know how to do it.
Also, there is another question: how to handle Depth (% of base) using Python?
from openpyxl.chart import (
AreaChart3D,
Reference,
)
wb_obj = xl.load_workbook('Plots.xlsx')
ws_obj = wb_obj.active
ws2_obj = wb_obj.create_sheet("Graphs")
c1 = AreaChart3D()
c1.legend = None
c1.style = 15
cats = Reference(ws_obj, min_col=1, min_row=7, max_row=200)
data = Reference(ws_obj, min_col=2, min_row=6, max_col=8, max_row=200)
c1.add_data(data, titles_from_data=True)
c1.set_categories(cats)
ws2_obj.add_chart(c1, "A1")
wb_obj.save("Plots.xlsx")
The Code above produces only one graph, but how should I proceed to create 2 or 4 or 100 graphs?
Later edit 1:
I tried something like this and it is almost working:
for i in range(1, 4):
c1 = AreaChart3D()
cats = Reference(ws_obj, min_col=1, min_row=7, max_row=200)
data = Reference(ws_obj, min_col=2, min_row=6, max_col=i * int(step), max_row=200)
c1.title = ws_obj.cell(row=1, column=i * int(step)).value
c1.legend = None
c1.style = 15
c1.y_axis.title = 'Fire Time'
c1.x_axis.title = 'Temperature'
c1.z_axis.title = "Velocity"
c1.add_data(data, titles_from_data=True)
c1.set_categories(cats)
ws2_obj.add_chart(c1, "A2")
For me the last ws2_obj.add_chart(c1, "A2") seems to be the problematic one.
Instead of A2 I would like to use something like ws2_obj.add_chart(c1, cell(row=2, column=i)).value but does not working.
Later Edit 2
I have observed if you want to add a chart to a certain cell, you have to use something like: ws2_obj.add_chart(my_chart, "R2")
In order to use the for loop I tried to find out a way to get this value R2.
Please, see below:
my_cells = []
for i in range(1, 4):
my_cell = ws2_obj.cell(row=1, column=i * int(step) - (int(step) - 1))
my_cells.append(my_cell)
print("My_Cell:", my_cells)
new_cells = []
for i in my_cells:
new_cells.append(re.findall("\W\w\d", str(i)))
new_new_cells = []
for i in new_cells:
new_new_cells.append(i[0])
print("new_new_cells:", new_new_cells)
final_list = [re.sub('[^a-zA-Z0-9]+', '', _) for _ in new_new_cells]
print("final list:", final_list)
And the output will be ['A1', 'H1', 'O1']
and then I can output the graph:
for i in range(1, 4):
c1 = AreaChart3D()
# my_cell = ws2_obj.cell(row=i, column=i * int(step))
cats = Reference(ws_obj, min_col=1, min_row=7, max_row=255)
data = Reference(ws_obj, min_col=2, min_row=6, max_col=i * int(step), max_row=255)
c1.title = ws_obj.cell(row=1, column=i * int(step)).value
c1.legend = None
c1.style = 20
c1.y_axis.title = 'Time'
c1.x_axis.title = 'Location'
c1.z_axis.title = "Velocity"
c1.add_data(data, titles_from_data=True)
c1.set_categories(cats)
c1.x_axis.scaling.max = 75
c1.y_axis.scaling.max = 50
c1.z_axis.scaling.max = 25
ws2_obj.add_chart(c1, str(final_list[i - 1]))

You can create a list of the series data (position where the data series starts). The list has 1 element per series. Iterate the list creating a chart for each and ensure you have some means to place the chart in a unique position.
Example code with comments below.
import openpyxl as xl
from openpyxl.chart import (
AreaChart3D,
Reference,
)
def create_chart(tl, maxr, hdr, x_ax):
"""
Creates a standard Area 3D Chart
"""
cht = AreaChart3D()
cht.legend = None
cht.style = 15
cht.title = hdr + " Chart"
cht.x_axis.title = x_ax
cht.y_axis.title = 'Something' # Some text for the y axis
data = Reference(ws_obj, min_col=tl[0], min_row=tl[1], max_col=tl[0]+1, max_row=maxr-1)
cht.add_data(data, titles_from_data=True)
return cht
## Sheet constants
chart_header = 'Speed' # It is assumed this is located in a merged cell
x_axis_header = 'Location'
series_topleft_header = 25
## Load Workbook and Sheet of Excel with data series
wb_obj = xl.load_workbook('Plots.xlsx')
ws_obj = wb_obj.active
## Get the total used rows in the sheet (end of the series table)
maxrows = ws_obj.max_row
speed_row = ''
speed_col_start = ''
speed_col_end = ''
speed_col_letter = ''
## Get a list of Merged cell in the sheet these contain the Headers for position referencing
merge_list = [m.coord for m in ws_obj.merged_cells.ranges]
## Search for the row with Header name 'Speed' to use as reference for series data postioning
for merge_element in ws_obj.merged_cells:
merge_cell_val = merge_element.start_cell.internal_value
if merge_cell_val.lower() == chart_header.lower():
speed_row = merge_element.max_row
speed_col_start = merge_element.min_col
speed_col_end = merge_element.max_col
speed_col_letter = merge_element.start_cell.column_letter
series_header_row = speed_row + 1
series1_start = speed_col_letter + str(series_header_row+1)
"""
Obtain the location of the top left cell where the series data exists
This searches the row below the header (containing the text 'Speed') for the first
series header (i.e. 25 in the example) and adds each position to the series_postion_list
"""
series_position_list = []
for row in ws_obj.iter_rows(min_row=series_header_row,
max_row=series_header_row,
min_col=speed_col_start,
max_col=speed_col_end):
for cell in row:
if cell.value == series_topleft_header:
series_position_list.append([cell.column, series_header_row])
## Create the Charts
"""
With the series_position_list indicating the top left cell of the series data
and the number of rows in the series determined be the maxrows - 1. This data
can be passed to the create_chart function to create the chart.
Charts are placed below the series data table from Column A with two charts
per row. First row for chart location is 2 rows below the series table.
"""
chart_start_row = maxrows + 2
chart_col = 'A'
"""
The series_position_list is used to create 1 chart per series
The chart creation function takes the top left coordinate and max rows along
with Chart header name and x axis header name
"""
for enum, top_left in enumerate(series_position_list, 1):
chart_obj = create_chart(top_left,
maxrows,
chart_header + ' ' + str(enum),
x_axis_header)
## This sets the position the chart will be placed. Based on standard size
## of plot area the charts are 16 rows and 10 columns apart
if enum == 1:
pass
elif enum % 2 == 1:
chart_col = 'A'
chart_start_row += 16
else:
chart_col = 'J'
## Adds chart to the Excel sheet
print(f"Adding chart {chart_header + ' ' + str(enum)} to Excel:")
print(f"Series Data Start; Row:{str(top_left[1]+1)} Column:{top_left[0]}")
ws_obj.add_chart(chart_obj, chart_col + str(chart_start_row))
print("--------------\n")
wb_obj.save("Plots.xlsx")
-----------------Additional Information--------------
add_chart is a method that accepts two arguments; the chart object and optionally an anchor point (i.e the top left cell where the chart is placed in the sheet). Use of .value at the end of
ws2_obj.add_chart(c1, cell(row=2, column=i)).value
is invalid as you are not entering the method into the cell you are using the method to add the chart object c1 at position cell(row=2, column=i). Using cell(row=2, column=i) is also an invalid syntax. You may have meant to use ws2_obj.cell(row=2, column=i) as the anchor. This would be accepted by the add_chart method however when saving the worksheet there would be an error on checking the anchor point as this expects the anchor to be an "Excel style coordinate" i.e. a string like 'A2' rather than a cell object like ws2_obj.cell(row=2, column=i). Even using (2, 1) would fail the same check.
To set the anchor points I will show how to do two options; All charts on the same row and X charts across the row then start next X charts on the next row etc.
Place all charts on same row;
If you are going to put all charts on the same row then the row coord will not change and only the column position needs adjustment for each chart.
You can generate the anchor points like below, the example code uses a for loop with 18 elements;
from openpyxl.utils.cell import coordinate_to_tuple
from openpyxl.utils import get_column_letter
anchor = 'A2' # Position of anchor, first anchor point is 'A2'
column_separation = 9 # Number of columns to separate each chart
for i in range(0, 18):
coord_tuple = coordinate_to_tuple(anchor)
row = coord_tuple[0]
col_offset = column_separation if i > 0 else 0
col_new = get_column_letter(coord_tuple[1] + col_offset)
anchor = f'{col_new}{row}'
print(f'Adding chart at Anchor point {anchor}')
ws2_obj.add_chart(c1, anchor)
This will put the chart at the following achor points;
A2, J2, S2, AB2, AK2, AT2, BC2, BL2, BU2, CD2, CM2, CV2, DE2, DN2, DW2, EF2, EX2, EO2
Placing the charts is a pattern.
Placing the charts is a pattern of rows and columns is similar to the previous code however when the number of charts reaches your limit the 'row' value has to change and the column resets back to 'A'.
The example code again uses a for loop with 18 elements and splits the charts into rows of max_chart_row, set to 5 in this case;
from openpyxl.utils.cell import coordinate_to_tuple
from openpyxl.utils import get_column_letter
anchor = 'A2'
column_separation = 9
max_chart_row = 5
for i in range(0, 18):
coord_tuple = coordinate_to_tuple(anchor)
row = coord_tuple[0]
col_offset = column_separation if i > 0 else 0
# When the number of charts across the row is reached, set the row to 16 more than the current
# and reset the column offset to 0
if i % (max_chart_row) == 0 and i != 0:
row = row + 16
col_offset = 0
col_new = get_column_letter(col_offset+1)
else:
col_new = get_column_letter(coord_tuple[1] + col_offset)
anchor = f'{col_new}{row}'
print(f'Adding chart at Anchor point {anchor}')
ws2_obj.add_chart(c1, anchor)
This will put the chart at the following achor points;
A2, J2, S2, AB2, AK2,
A18, J18, S18, AB18, AK18,
A34, J34, S34, AB34, AK34,
A50, J50, S50

Tkinter Formatting Buttons in Same Row using .grid

How can I display a row of buttons across the bottom of the screen using Tkinter? The number of buttons is a variable. Could be 2 or could be 10. The number of items being displayed is also a variable. In this case it's 5, but could be 2 or could be 10. I am using Tkinter and currently have a working program that outputs a grid that looks like this:
-------------------------
| |Title| |
|Item1| |Quantity1|
|Item2| |Quantity2|
|Item3| |Quantity3|
|Item4| |Quantity4|
| | (Intentionally blank)
|Item5| |Quantity5|
-------------------------- (end)
I am outputting like so:
from tkinter import *
window = Tk()
itemLabel = Label(
window,
text = "Item1",
).grid(row = 2, column = 0, sticky = 'w')
However, when I try to add buttons, I can't seem to get the formatting correct. If I do it similarly to the label, with "sticky = 'w'", then it overlaps on the left. I only have 3 columns so if I have more than 3 buttons I run out of columns. Below is my desired output (hopefully all of the buttons will be of equal width):
-------------------------
| |Title| |
|Item1| |Quantity1|
|Item2| |Quantity2|
|Item3| |Quantity3|
|Item4| |Quantity4|
| | (Intentionally blank)
|Item5| |Quantity5|
--------------------------
|B#1| B#2|B#3|B#4| B#5|B#6|
--------------------------- (end)

I had a similar problem, but for radiobuttons - a variable number of options which can change, in this case updated on the click of a button. I set the max number of columns as I spill over into more rows if need be.
def labelling_add_radiobutton(self):
'''add selected values as radiobuttons for labelling'''
# destroy any existing radiobuttons in frame
for child in self.frame3_labelling_labels.winfo_children():
if isinstance(child, ttk.Radiobutton):
child.destroy()
# label variable and create radio button
self.checkbox_validation_output = tk.StringVar(value = '')
num_cols = 8 # sets max number of columns, rows are variable
for count, value in enumerate(list_of_buttons):
row = int((count + .99) / num_cols) + 1
col = (((row - 1) + count) - ((row - 1) * num_cols) - (row - 1)) + 1
ttk.Radiobutton(self.frame3_labelling_labels, text = value,
variable = self.checkbox_validation_output, value = value,
style = 'UL_radiobutton.TRadiobutton').grid(column = col, row = row,
sticky = 'w', padx = 10, pady = 10)

Switch color of sns.swarnplot

Does anybody know, if its possible to switch the colors, so that i can distinguish every row instead of every column ? And how do I add in a legend, where i can see which player (one color for each player) has e.g. which pace?
My code is:
feldspieler = feldspieler["sofifa_id"]
skills = ['pace','shooting','passing','dribbling','defending','physic']
diagramm = plt.figure(figsize=(40,20))
plt.xticks(rotation=90,fontsize=20)
plt.yticks(fontsize=20)
plt.xlabel('Skills', fontsize=30)
plt.ylabel('Skill value', fontsize=30)
plt.title('Spielervergleich', fontsize = 40)
sns.set_palette("pastel")
for i in feldspieler:
i = fifa_21.loc[fifa_21['sofifa_id'] == i]
i = pd.DataFrame(i, columns = skills)
sns.swarmplot(data=i,size=12)

Thanks a lot #Trevis.
Unfortunately, it still does not work.
Here you can find a screenshot of the dataset and the code that the graphic accesses.
while True:
team = input("Welches Team suchen Sie?: ")
if team in fifa_21.values:
break
else:
print("Dieser Verein existiert nicht. Bitte achten Sie auf eine korrekte Schreibweise.")
gesuchtes_team = fifa_21.loc[(fifa_21['club_name'] == team)]
spieler_verein = gesuchtes_team[["sofifa_id","short_name","nationality","age","player_positions","overall","value_eur"]]
spieler_verein = pd.DataFrame(spieler_verein)
spieler_verein = spieler_verein.reset_index(drop=True)
spieler_verein
feldspieler = spieler_verein.loc[spieler_verein.player_positions != "GK", :]
feldspieler = feldspieler.reset_index(drop=True)
feldspieler
feldspieler = feldspieler["sofifa_id"]
skills = ['pace','shooting','passing','dribbling','defending','physic']
diagramm = plt.figure(figsize=(40,20))
plt.xticks(rotation=90,fontsize=20)
plt.yticks(fontsize=20)
plt.xlabel('Skills', fontsize=30)
plt.ylabel('Skill value', fontsize=30)
plt.title('Spielervergleich', fontsize = 40)
sns.set_palette("pastel")
for i in feldspieler:
i = fifa_21.loc[fifa_21['sofifa_id'] == i]
i = pd.DataFrame(i, columns = skills)
sns.swarmplot(data=fifa_21, x="skills", y="skill_value", hue="sofifa_id")
#sns.swarmplot(x =skills, y= pd.DataFrame(i, columns == skills) ,hue= "sofifa_id", data=i,size=12)

Set the hue parameter to the value of the column you're interested in (sofifa_id). You can then provide the whole dataset at once to plot the data. The legend will be added automatically.
So you should have a DataFrame with a 'skills' column containing the different skills you have in x-axis here. If necessary, see the documentation for pd.melt, in particular the third example.
Then, assuming the default column name value for the value after melting, call
sns.swarmplot(data=fifa_21, x="skills", y="value", hue="sofifa_id")
This is from the official swarmplot function documentation (here).
Edit: So, seeing your data, you should really use pd.melt like this:
(I'm considering one row per player, with distinct short_name values).
data = pd.melt(fifa_21, id_vars='short_name', var_name='skill',
value_vars=['pace', 'shooting', 'passing', 'dribbling',
'defending', 'physic'])
sns.swarmplot(x='skill', y='value', hue='short_name', data=data)
melt will transform to columns and value from a wide format
short_name
pace
shooting
a_name
85
92
to a long table format
short_name
skill
value
a_name
pace
85
a_name
shooting
92

Most efficient way to use TrueSkill Algo to backfill a dataset in Pandas

I have the following gist that implements the TrueSkill algorithm over a dataset that is a collection n-player free-for-all games, where there is a winner, then second, third, fourth place etc.
Basically what I am asking is:
Have I implemented the algo correctly
Whats the most efficient way to backfill data in a Pandas dataframe

Another solution:
# Fetch the data
df = pd.read_csv('http://horse-data-abcd.s3.amazonaws.com/game_results.csv')
#sample_df = df[df['game_id'].isin([1592008, 1592012, 1592238, 1500903])].copy()
sample_df = df.head(10000)
def rate_games(df):
# store ratings for each user, this way it is easier to keep track of Rating between games
# i've decided that it will be easier compared to earlier implementation
# Ratings are initialized with default values mu=25.0 sigma=8.333
trueskills = {user_id: Rating() for user_id in df['player_id'].unique()}
# Group by the game_id
games = df.sort_values('date').groupby('game_id')
dataframes_list = []
# Now iterate the games
for game_id, game in games:
# sorting by postion (from best to last) this way rate function will figure out which player was best automatically
sorted_players = game.sort_values('position').iterrows()
# rate function allows input in form of [{some_label: Rating}, ...] and also it will return list
# in the same form with new Ratings. having [{row_id: Rating}, ...] will allow merging back the rating data
# back into original dataframee
trueskills_dicts = [{index: trueskills.get(row['player_id'], Rating())} for index, row in sorted_players]
flat_trueskill_dict = dict(ChainMap(*trueskills_dicts))
# Get the results from the rate method
try:
results = rate(trueskills_dicts) # returns [{'row_id': new rating}, ...]
except ValueError:
results = trueskills_dicts
# This converts results list of dicts into dict
# {'row_id_1': rating, 'row_id_2': rating}
flat_results_dict = dict(ChainMap(*results))
# This section prepares small dataframe looking like this, sorted from best to worst:
# index | mu | sigma | post_mu | post_sigma
# 3245 | 25 | 8.33 | 27.0 | 5.0
# 1225 | 25 | 8.33 | 26.0 | 5.0
df_columns = defaultdict(list)
df_index = []
for index, new_rating in flat_results_dict.items():
df_index.append(index)
previous_rating = flat_trueskill_dict.get(index, Rating())
df_columns['mu'].append(previous_rating.mu)
df_columns['sigma'].append(previous_rating.sigma)
df_columns['post_mu'].append(new_rating.mu)
df_columns['post_sigma'].append(new_rating.sigma)
# this dataframe has the same index column as the main one
# because of that we will be able to easily join it at the end of the function
df_results = pd.DataFrame(
df_columns,
index=df_index
)
dataframes_list.append(df_results)
# ok, all calclulations done, lets update the trueskills and advance to the next game
trueskills.update({game.loc[index, 'player_id']: rating for index, rating in flat_results_dict.items()})
# last thing get list of dataframes with calculated ratings and join it with main dataframe
concatenated_df = pd.concat(dataframes_list)
df = df.join(concatenated_df)
df.loc[:, 'player_id'].fillna(0, inplace=True)
return df
sample_df = rate_games(sample_df)
sample_df

Here is what I have come up with, can probably be optimised for speed.
# Fetch the data
df_raw = pd.read_csv('http://horse-data-abcd.s3.amazonaws.com/game_results.csv')
# Create a holding DataFrame for our TrueRank
df_truerank_columns = ['game_id', 'player_id', 'position', 'mu', 'sigma', 'post_mu', 'post_sigma']
df_truerank = pd.DataFrame(columns=df_truerank_columns)
# Use a sample of 1000
df = df_raw.head(10000)
# Group by the game_id
games = df.groupby('game_id')
# Now iterate the games
for game_id, game in games:
# Setup lists so we can zip them back up at the end
trueskills = []
player_ids = []
game_ids = []
mus = []
sigmas = []
post_mus = []
post_sigmas = []
# Now iterate over each player in a game
for index, row in game.iterrows():
# Create a game_ids arary for zipping up
game_ids.append(game_id)
# Now push the player_id onto the player_ids array for zipping up
player_ids.append(int(row['player_id']))
# Get the players last game, hence tail(1)
filter = (df_truerank['game_id'] < game_id) & (df_truerank['player_id'] == row['player_id'])
df_player = df_truerank[filter].tail(1)
# If there isnt a game then just use the TrueSkill defaults
if (len(df_player) == 0):
mu = 25
sigma = 8.333
else:
# Otherwise get the mu and sigma from the players last game
row = df_player.iloc[0]
mu = row['post_mu']
sigma = row['post_sigma']
# Keep lists of pre mu and sigmas
mus.append(mu)
sigmas.append(sigma)
# Now create a TrueSkull Rating() class and pass it into the trueskills dictionary
trueskills.append(Rating(mu=mu, sigma=sigma))
# Use the positions as ranks, they are 0 based so -1 from all of them
ranks = [x - 1 for x in list(game['position'])]
# Create tuples out of the trueskills array
trueskills_tuples = [(x,) for x in trueskills]
try:
# Get the results from the TrueSkill rate method
results = rate(trueskills_tuples, ranks=ranks)
# Loop the TrueSkill results and get the new mu and sigma for each player
for result in results:
post_mus.append(round(result[0].mu, 2))
post_sigmas.append(round(result[0].sigma, 2))
except:
# If the TrusSkill rate method blows up, just use the previous
# games mus and sigmas
post_mus = mus
post_sigmas = sigmas
# Change the positions back to non 0 based
positions = [x + 1 for x in ranks]
# Now zip together all our lists
data = list(zip(game_ids, player_ids, positions, mus, sigmas, post_mus, post_sigmas))
# Create a temp DataFrame the same as df_truerank and add data to the DataFrame
df_temp = pd.DataFrame(data, columns=df_truerank_columns)
# Add df_temp to our df_truerank
df_truerank = df_truerank.append(df_temp)

How to handle 2 widget events to control a dataframe?

I am trying to use 2 ipywidgets to control a dataframe and refresh as a change the 2 ipywidgets.
Create a new column (Check_value) that shifts down the Values column by the value in widget A
Filter and show only the rows where the Check_value is bigger than the Value column by the value in widget B
The dataframe :
Dates | Values
Day 1 | 5
Day 2 | 9
Day 3 | 14
Day 4 | 40
Day 5 | 80
## Widget A
A = widgets.IntSlider(
value=1,
min=1,
max=30,
step=1)
## Widget B
B = widgets.IntSlider(
value=1,
min=1,
max=30,
step=1)
out = widgets.Output()
## Processing
def common_processing(period, filters):
out.clear_output()
with out:
df = pd.read_csv(data.csv)
df['Dates'] = pd.to_datetime(df['Dates'])
df['check_value'] = df['Value'].shift(-period)
df['delta'] = df['check_value'] - df['Value']
display(df[df['delta'] > filters])
def A_eventhandler(change):
common_processing(change.new, B.value)
def B_eventhandler(change):
common_processing(A.value, change.new)
A.observe(A_eventhandler, names='value')
B.observe(B_eventhandler, names='value')
display(A)
display(B)
The data frame displayed does not change with changes in the widget values.

I tried running your code, the A_eventhandler function is never called when you change the widget value (you can check this by running print(change) before the common_processing call.)
The reason is your names keyword needs to be a list, rather than a string. So try:
A.observe(A_eventhandler, names=['value'])
B.observe(B_eventhandler, names=['value'])
When I set up an observe on a widget, I always do it without any filtering first, just printing the result in the observed function. Then you can add keywords to filter down to just the events and values you need.
Also, don't forget to display(out) somewhere in your code, as you are capturing the output here. Otherwise you will never see anything!