We Keep Coding

Rolling Year Based on Condition

Hello, I have the following code:
# Import Libraies
import numpy as np
import pandas as pd
import datetime as dt
# Conect to Drive
from google.colab import drive
drive.mount('/content/drive')
# Read Data
ruta = '/content/drive/MyDrive/example.csv'
df = pd.read_csv(ruta)
df.head(15)
d = pd.date_range(start="2015-01-01",end="2022-01-01", freq='MS')
dates = pd.DataFrame({"DATE":d})
df["DATE"] = pd.to_datetime(df["DATE"])
df_merge = pd.merge(dates, df, how='outer', on='DATE')
The data that I am using, you could download here: DATA
What I am trying to achieve is something known as Rolling Year.
First I create this metric gruped for each category:
# ROLLING YEAR
##################################################################################################
# I want to make a Roling Year for each category. Thats mean how much sell each category since 12 moths ago TO current month
# RY_ACTUAL One year have 12 months so I pass as parameter in the rolling 12
f = lambda x:x.rolling(12).sum()
df_merge["RY_ACTUAL"] = df_merge.groupby(["CATEGORY"])['Sales'].apply(f)
# RY_24 I create a rolling with 24 as parameter to compare actual RY vs last RY
f_1 = lambda x:x.rolling(24).sum()
df_merge["RY_24"] = df_merge.groupby(["CATEGORY"])['Sales'].apply(f_1)
#RY_LAST Substract RY_24 - RY_Actual to get the correct amount. Thats mean amount of RY vs the amount of RY-1
df_merge["RY_LAST"] = df_merge["RY_24"] - df_merge["RY_ACTUAL"]
##################################################################################################
df_merge.head(30)
And it works perfectly, ´cause if you download the file and then filter for example for "Blue" category, you could see, something like this:
Thats mean, if you stop in the row 2015-November, you could see in the column RY_ACTUAL the sum of all the values 12 records before.
Mi next goal is to create a similar column using the rollig function but with the next condition:
The column must sum all the sales of ALL the categories, as long as
the Color/Animal column is equal to Colour. For example if I am
stopped in 2016-December, it should give me the sum of ALL the sales
of the colors from 2016-January to 2016-December
This was my attempt:
df_merge.loc[(df_merge['Colour/Animal'] == 'Colour'),'Sales'].apply(f)
Cold anyone help me to code correctly this example?.
Thanks in advance comunity!!!

Update plotly chart based on different button click (load different data)

I am trying to create a plotyly chart with some subplots based on Use button to filter different data in plotly python
The chart generation function takes as input a stock symbol, dict of periods (m1, m3, m5... for different minutes) and stock specific period dataframes.
I am trying the put the periods as buttons, so that on a period button click, the corresponding dataframe (OHLC) can be loaded along with period dependent indicators - MACD, RSI and ADX.
The issue is only the last period df is loaded and the buttons are not showing/ loading the period specific OHLCV.
Below is the function
def plot_plotly_v3(in_stock, in_period_stock_mdf_df_dict,n):
f_stock = in_stock
f_period_stock_mdf_df_dict = in_period_stock_mdf_df_dict
period_buttons = []
i = 0
period_length = len(f_period_stock_mdf_df_dict) # to calculate visible args
period_frequency_dict = config.c_g_period_python_freq_dict # broker period to python period
for period, stock_period_df in f_period_stock_mdf_df_dict.items():
stock_period_df.index = stock_period_df.index.droplevel([1, 2])
fig = make_subplots(rows=4, cols=1, shared_xaxes=True, vertical_spacing=0.007, row_heights=[.35, .20, .20, .25],
subplot_titles=('', 'MACD', 'RSI', 'ADX'))
# removing all empty dates and build complete timeline from start date to end date
py_frequency = period_frequency_dict.get(period) # broker period to python period mapping
dt_all = pd.date_range(start=stock_period_df.index[0], end=stock_period_df.index[-1], freq=py_frequency)
# retrieve the dates that ARE in the original datset
dt_obs = [d.strftime("%Y-%m-%d %H:%M:%S") for d in pd.to_datetime(stock_period_df.index)]
# define dates with missing values
dt_breaks = [d for d in dt_all.strftime("%Y-%m-%d %H:%M:%S").tolist() if not d in dt_obs]
in_period_int = int(config.g_period_2_period_int_dict.get(period))
dvalue_ms = in_period_int * 60 * 1000
fig.update_xaxes(rangebreaks=[dict(values=dt_breaks, dvalue=dvalue_ms)])
fig_title = in_stock + ' for period ' + period + ' for range ' + 'From: ' + \
str(stock_period_df.index[0]) + ' To: ' + str(stock_period_df.index[-1])
for annotation in fig['layout']['annotations']:
annotation['textangle'] = -90
fig.update_annotations(xshift=-620, yshift=-100)
# Plot OHLC and MAs on 1st subplot
# Plot MACD trace on 2nd row
# Plot RSI trace on 3rd row
# Plot ADX trace on 4th row
# create visible args - True/ False list depending upon period/df dict
visible_args = create_true_false_list_v0(i, period_length)
# create a button object for the period we are on
button = dict(label=period, method="update", args=[{"visible": visible_args}])
# add the button to our list of buttons
period_buttons.append(button)
# i is an iterable used to tell our "args" list which value to set to True
i += 1
fig.update_layout(updatemenus=[dict(type="buttons",direction="right", x = 1,y = 1,buttons = period_buttons)],
height=800, width=1350, title=fig_title, xaxis_rangeslider_visible=False)
fig.show()
Would appreciate any support/ guidance.
Question:
1] Is it possible to display the title in the same level as toolbar/modebar.
2] Is it possible to display the period buttons in the same level as the toolbar.

How to process this data to create a stacked bar chart?

so I am writing a program that monitors and records your usage time of foreground applications and saves them in a SQL database. I then want to retrieve the data from previous days and compile it all together into a stacked bar chart. Here, the x-axis will have the different days over which usage was recorded, and the various stacks in each bar will represent each app that was used.
In my program, I created 2 tables, one to record each day's app usage (with each new day's data having a different primary key id), and another table to record the primary key for each day.
Table 1:
_id
Application
usage_time
0
Google Chrome
245.283942928347
1
PyCharm
450.3939754962921
1
SQLiteStudio
140.2376308441162
1
Google Chrome
5.008131980896
Table 2:
Date
daily_id
2021-07-18 07:25:25.376734
0
2021-07-18 07:27:57.419574
1
Within my stacked bar chart program, I have come up with this code to refine the data to put into the stacked bar chart:
conn = sqlite3.connect('daily_usage_monitor.sqlite', detect_types=sqlite3.PARSE_DECLTYPES)
all_app_data = conn.execute('SELECT all_usage_information.date, monitor.application, monitor.usage_time FROM all_usage_information INNER JOIN monitor ON all_usage_information.daily_id = monitor._id ORDER BY all_usage_information.date, monitor.usage_time ASC').fetchall()
for date, app, usage_time in all_app_data:
print(f'{date} - {app}: {usage_time}')
conn.close()
daily_data = {}
# Create nested dictionary - key = each date, value = dictionary of different apps & their time usage durations
for date, app, time in all_app_data:
conditions = [date not in daily_data, app != 'loginwindow']
if all(conditions):
daily_data[date] = {app: time}
elif not conditions[0] and conditions[1]:
daily_data[date].update({app: time})
print(daily_data) # TODO: REMOVE AFTER TESTING
total_time = 0
# Club any applications that account for <5% of total time into 1 category called 'Other'
for date, app_usages in daily_data.items():
total_time = sum(time for app, time in app_usages.items())
refined_data = {}
for key, value in app_usages.items():
if value/total_time < 0.05:
refined_data['Others'] = refined_data.setdefault('Others', 0) + value
else:
refined_data[key] = value
daily_data[date] = refined_data
print(daily_data) # TODO: REMOVE AFTER TESTING
# Takes the nested dictionary and breaks it into a labels list and a dictionary with apps & time usages for each day
# Sorts data so it can be used to create composite bar chart
final_data = {}
labels = []
for date, app_usages in daily_data.items():
labels.append(date)
for app, time in app_usages.items():
if app not in final_data:
final_data[app] = [time]
else:
final_data[app].append(time)
This is the kind of output I am currently getting:
{'Google Chrome': [245.283942928347, 190.20031905174255], 'SQLiteStudio': [145.24058270454407], 'PyCharm': [1166.0021023750305]}
The problem here is that for the days where an application had 0 usage time, it is not being recorded in the list. Therefore, the insertion order of the stacked bar chart will not be correct and will show the wrong apps for the wrong dates. How can I fix this?
This is one method I tried, but of course it's not working because you cannot index into a dictionary:
for app, usage in final_data.items():
for date, app_usages in daily_data.items():
if app not in app_usages:
usage.insert(app_usages.index(app), 0)

How do i plot multiple axvspan OR vspan's on a line plot when value is True

I'm trying to highlight on a line chart that has a line of stock price, and then a band above and below the stock price line.
I have a column whose value is "1" every time the stock price exceeds the bands, and Nan when the stock price is in between the lines.
I want to highlight the regions where the value is "1", or when the stock price exceeds the bands. This happens multiple times, and I am unable to find a way to create more than one vspan correctly.
I am using pandas. The dates are the x-axis and are indexed. Any help with this?
I've been looking at plotly extensively for a day now, but don't see it being possible. I am struggling to figure out how to use matplotlib for this.
In the code below, i am trying to compare "OPEN" (the column that is a 1 if price exceeds a band) and "AAPL.O", which is the ric variable that is the closing price
def add_SMA_bands(data,ric):
df = pd.DataFrame(data[ric])
SMAname = "SMA_{}".format(ric)
SMAstd = "SMAstd_{}".format(ric)
SMAUpper = "UpperBand_{}".format(ric)
SMALower = "LowerBand_{}".format(ric)
df[SMAname] = df["{}".format(ric)].rolling(SMA).mean()
df[SMAUpper] = df[SMAname] + (df["{}".format(ric)].rolling(SMA).std())*2
df[SMALower] = df[SMAname] - (df["{}".format(ric)].rolling(SMA).std())*2
return df
dfs={}
for ric in rics:
df = add_SMA_bands(data,ric)
df.dropna(inplace=True)
dfs[ric]=df
for ric in rics:
dfs[ric]["OVER"] = np.where(dfs[ric][ric]<dfs[ric]["LowerBand_{}".format(ric)],1, (np.where(dfs[ric][ric]>dfs[ric]["UpperBand_{}".format(ric)],1,0)))
dfs[ric]["OVER"][dfs[ric]["OVER"]==0]=np.nan

updating a Dash plot with drop-down filters

I am looking to have 2 dropdowns (one for dates and another sect_id) to filter and update the plot on my dashboard and I am feeling quite confused about the callback and function to create. Here are my data :
sect_id Date Measure1 Measure2 Measure3 Total %
L19801 01-01-17 12 65 0 33
L19801 01-01-17 19 81 7 45
M18803 01-01-17 15 85 7 45
M19803 01-01-17 20 83 2 52
xxxxxx xxxxxxx xx xx x xx
xxxxxx xxxxxxx xx xx x xx
I am looking to scatter Measure1 against Measure3 and have two dropdowns. here what I have done:
year_options = []
for date in df['Date'].unique():
date_options.append({'label':str(date),'value':date})
app=dash.Dash()
app.layout = html.Div([
dcc.Graph(id='graph'),
dcc.Dropdown(id='date_picker',options=date_options,value=df['Date'][1])])
I am currently trying with only one dropdown (the date) and I am very confused about what I have done the layout, the dropdown and the callback.

Layout is the contents of your dashboard, your graph and your dropdown(s). Callbacks are interactions between said components. Please refer to the Dash documentation:
For basic info about layout and for callbacks
You can create callbacks fairly simply, just define a function and add an callback decorator to them like this:
import plotly.graph_objs as go
from dash.dependencies import Input, Output
#app.callback(
# What does the callback change? Right now we want to change the figure of the graph.
# You can assign only one callback for each property of each component.
Output(component_id='graph', component_property='figure'),
# Any components that modify the outcome of the callback
# (sect_id picker should go here as well)
[Input(component_id='date_picker', component_property='value')])
def create_graph_figure(date_picker_value):
# you should define a function here that returns your plot
df_filtered = df[df['Date'] == date_picker_value]
return go.Scatter(x=df['Measure1'], y=df['Measure2'])