We Keep Coding

Prophet search trial freezes with optuna - python

I am currently using Prophet for a forecasting prototype, and I've done the search for the hyperparameters using optuna TPESampler. I have around 300 dataframes on which I have to compute the hyperparameter optimizations for. After completing a couple of dataframes the search suddenly freezes without any warning message and without exiting the proccess. It just gets blocked at a certain nr of trial (for the current search it has stopped, after completing 130 dataframes hyperparameter optimization at trial 786 out of 1000). I've tried to increase and decrease the number of trials per dataframe & (as expected) I've seen that as long as the number of trials goes down, more dataframes gets to be computed & the more the number of trials goes up (like 2000) less dataframes gets to be computed. Has everyone encountered this before? It really gets annoying because I have to restart the search because it can never get to an end. I am not quite sure if this is a problem with optuna or Prophet so I will post this on their github issue too.
It gets stucked inside the search function I've created, in the cross_validation() method from Prophet (I can tell that because in here it creates parallel processes = how many folds we want to test the subset of hyperparameters, in my code are defined as cutoffs)
Optuna version: 3.0.3
Python version: 3.9.13
OS: Linux-5.15.0-56-generic-x86_64-with-glibc2.31
numpy: 1.23.5
pandas: 1.5.2
prophet: 1.1.1
scikit_learn: 1.2.0
Here is the code I am using:
def generate_cutoffs(df, weeks_to_forecast):
cutoffs = pd.to_datetime(
[
df.at[len(df) - (weeks_to_forecast*9+1), "ds"],
df.at[len(df) - (weeks_to_forecast*8+1), "ds"],
df.at[len(df) - (weeks_to_forecast*7+1), "ds"],
df.at[len(df) - (weeks_to_forecast*6+1), "ds"],
df.at[len(df) - (weeks_to_forecast*5+1), "ds"],
df.at[len(df) - (weeks_to_forecast*4+1), "ds"],
df.at[len(df) - (weeks_to_forecast*3+1), "ds"],
df.at[len(df) - (weeks_to_forecast*2+1), "ds"],
df.at[len(df) - (weeks_to_forecast+1), "ds"],
]
)
return cutoffs
def get_best_hp_prophet_cv(df_for_cv, weeks_to_forecast, freq, holidays_df):
logging.getLogger('prophet').setLevel(logging.ERROR)
logging.getLogger('fbprophet').setLevel(logging.ERROR)
# print('DF entered for search: ', df_for_cv)
cutoffs = generate_cutoffs(df_for_cv, weeks_to_forecast, freq)
def objective(trial):
# print(cutoffs)
# print(df.tail(30))
param_grid = {
"changepoint_prior_scale": trial.suggest_categorical(
"changepoint_prior_scale", [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0]
),
"seasonality_prior_scale": trial.suggest_categorical(
"seasonality_prior_scale", [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0]
),
"seasonality_mode": trial.suggest_categorical(
"seasonality_mode", ["multiplicative", "additive"]
),
"growth": trial.suggest_categorical("growth", ["linear"]),
"yearly_seasonality": trial.suggest_categorical(
"yearly_seasonality", [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17]
),
"daily_seasonality": trial.suggest_categorical("daily_seasonality",[False]),
"weekly_seasonality": trial.suggest_categorical("weekly_seasonality",[False]),
"uncertainty_samples": trial.suggest_categorical("uncertainty_samples",[0]),
}
prior_scale_month = trial.suggest_categorical('prior_scale_month', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_week_num = trial.suggest_categorical('prior_scale_week_num', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_month_qty_over_df = trial.suggest_categorical('prior_scale_avg_month_qty_over_df', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_week_nr_qty_over_df = trial.suggest_categorical('prior_scale_avg_week_nr_qty_over_df', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_solar_rad_of_month = trial.suggest_categorical('prior_scale_avg_solar_rad_of_month', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_solar_rad_of_week_nr = trial.suggest_categorical('prior_scale_avg_solar_rad_of_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_temp_of_month = trial.suggest_categorical('prior_scale_avg_temp_of_month', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_temp_of_week_nr = trial.suggest_categorical('prior_scale_avg_temp_of_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_pres_of_month = trial.suggest_categorical('prior_scale_avg_pres_of_month', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_pres_of_week_nr = trial.suggest_categorical('prior_scale_avg_pres_of_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_solar_rad_last_week_week_nr = trial.suggest_categorical('prior_scale_avg_solar_rad_last_week_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_temp_last_week_week_nr = trial.suggest_categorical('prior_scale_avg_temp_last_week_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_pres_last_week_week_nr = trial.suggest_categorical('prior_scale_avg_pres_last_week_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_solar_rad_last_2weeks_week_nr = trial.suggest_categorical('prior_scale_avg_solar_rad_last_2weeks_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_temp_last_2weeks_week_nr = trial.suggest_categorical('prior_scale_avg_temp_last_2weeks_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_pres_last_2weeks_week_nr = trial.suggest_categorical('prior_scale_avg_pres_last_2weeks_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_solar_rad_next_week_week_nr = trial.suggest_categorical('prior_scale_avg_solar_rad_next_week_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_temp_next_week_week_nr = trial.suggest_categorical('prior_scale_avg_temp_next_week_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_pres_next_week_week_nr = trial.suggest_categorical('prior_scale_avg_pres_next_week_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_solar_rad_of_last_month = trial.suggest_categorical('prior_scale_avg_solar_rad_of_last_month', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_temp_of_last_month = trial.suggest_categorical('prior_scale_avg_temp_of_last_month', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_pres_of_last_month = trial.suggest_categorical('prior_scale_avg_pres_of_last_month', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_qty_last_2weeks_week_nr = trial.suggest_categorical('prior_scale_avg_qty_last_2weeks_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
prior_scale_avg_qty_next_week_week_nr = trial.suggest_categorical('prior_scale_avg_qty_next_week_week_nr', [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.09, 0.1, 0.2, 0.5, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 5.0, 9.0, 10.0])
# I ve used this only for testing to see if everything works fine
# param_grid = {
# 'changepoint_prior_scale': trial.suggest_categorical('changepoint_prior_scale', [0.001]),
# 'seasonality_prior_scale': trial.suggest_categorical('seasonality_prior_scale',[0.01, 0.1]),
# 'seasonality_mode' : trial.suggest_categorical('seasonality_mode',['additive']),
# 'growth': trial.suggest_categorical('growth',['linear']),
# 'yearly_seasonality': trial.suggest_categorical('yearly_seasonality',[14,15]),
# 'holidays_prior_scale' : trial.suggest_categorical('holidays_prior_scale',[10])
# }
# all_params = [dict(zip(param_grid.keys(), v)) for v in itertools.product(*param_grid.values())]
# mses = [] # Store the MSEs for each params here
# Use cross validation to evaluate all parameters
# for params in all_params:
m = Prophet(**param_grid, holidays = holidays_df)
m.add_regressor('month', prior_scale = prior_scale_month)
m.add_regressor('week_num', prior_scale = prior_scale_week_num)
m.add_regressor('avg_month_qty_over_df', prior_scale = prior_scale_avg_month_qty_over_df)
m.add_regressor('avg_week_nr_qty_over_df', prior_scale = prior_scale_avg_week_nr_qty_over_df)
m.add_regressor('avg_solar_rad_of_month', prior_scale = prior_scale_avg_solar_rad_of_month)
m.add_regressor('avg_solar_rad_of_week_nr', prior_scale = prior_scale_avg_solar_rad_of_week_nr)
m.add_regressor('avg_pres_of_month', prior_scale = prior_scale_avg_pres_of_month)
m.add_regressor('avg_pres_of_week_nr', prior_scale = prior_scale_avg_pres_of_week_nr)
m.add_regressor('avg_temp_of_month', prior_scale = prior_scale_avg_temp_of_month)
m.add_regressor('avg_temp_of_week_nr', prior_scale = prior_scale_avg_temp_of_week_nr)
m.add_regressor('avg_solar_rad_last_week_week_nr', prior_scale = prior_scale_avg_solar_rad_last_week_week_nr)
m.add_regressor('avg_temp_last_week_week_nr', prior_scale = prior_scale_avg_temp_last_week_week_nr)
m.add_regressor('avg_pres_last_week_week_nr', prior_scale = prior_scale_avg_pres_last_week_week_nr)
m.add_regressor('avg_solar_rad_last_2weeks_week_nr', prior_scale = prior_scale_avg_solar_rad_last_2weeks_week_nr)
m.add_regressor('avg_temp_last_2weeks_week_nr', prior_scale = prior_scale_avg_temp_last_2weeks_week_nr)
m.add_regressor('avg_pres_last_2weeks_week_nr', prior_scale = prior_scale_avg_pres_last_2weeks_week_nr)
m.add_regressor('avg_solar_rad_next_week_week_nr', prior_scale = prior_scale_avg_solar_rad_next_week_week_nr)
m.add_regressor('avg_temp_next_week_week_nr', prior_scale = prior_scale_avg_temp_next_week_week_nr)
m.add_regressor('avg_pres_next_week_week_nr', prior_scale = prior_scale_avg_pres_next_week_week_nr)
m.add_regressor('avg_solar_rad_of_last_month', prior_scale = prior_scale_avg_solar_rad_of_last_month)
m.add_regressor('avg_temp_of_last_month', prior_scale = prior_scale_avg_temp_of_last_month)
m.add_regressor('avg_pres_of_last_month', prior_scale = prior_scale_avg_pres_of_last_month)
m.add_regressor('avg_qty_last_2weeks_week_nr', prior_scale = prior_scale_avg_qty_last_2weeks_week_nr)
m.add_regressor('avg_qty_next_week_week_nr', prior_scale = prior_scale_avg_qty_next_week_week_nr)
print("Model_params currently are: ", param_grid)
dict_with_non_model_params_to_print = {
'month': prior_scale_month,
'week_num': prior_scale_week_num,
'avg_month_qty_over_df': prior_scale_avg_month_qty_over_df,
'avg_week_nr_qty_over_df': prior_scale_avg_week_nr_qty_over_df,
'avg_solar_rad_of_month': prior_scale_avg_solar_rad_of_month,
'avg_solar_rad_of_week_nr': prior_scale_avg_solar_rad_of_week_nr,
'avg_pres_of_month': prior_scale_avg_pres_of_month,
'avg_pres_of_week_nr': prior_scale_avg_pres_of_week_nr,
'avg_temp_of_month': prior_scale_avg_temp_of_month,
'avg_temp_of_week_nr': prior_scale_avg_temp_of_week_nr,
'avg_solar_rad_last_week_week_nr': prior_scale_avg_solar_rad_last_week_week_nr,
'avg_temp_last_week_week_nr': prior_scale_avg_temp_last_week_week_nr,
'avg_pres_last_week_week_nr': prior_scale_avg_pres_last_week_week_nr,
'avg_solar_rad_last_2weeks_week_nr':prior_scale_avg_solar_rad_last_2weeks_week_nr,
'avg_temp_last_2weeks_week_nr': prior_scale_avg_temp_last_2weeks_week_nr,
'avg_pres_last_2weeks_week_nr': prior_scale_avg_pres_last_2weeks_week_nr,
'avg_solar_rad_next_week_week_nr': prior_scale_avg_solar_rad_next_week_week_nr,
'avg_temp_next_week_week_nr': prior_scale_avg_temp_next_week_week_nr,
'avg_pres_next_week_week_nr': prior_scale_avg_pres_next_week_week_nr,
'avg_solar_rad_of_last_month': prior_scale_avg_solar_rad_of_last_month,
'avg_temp_of_last_month': prior_scale_avg_temp_of_last_month,
'avg_pres_of_last_month': prior_scale_avg_pres_of_last_month,
'avg_qty_last_2weeks_week_nr': prior_scale_avg_qty_last_2weeks_week_nr,
'avg_qty_next_week_week_nr': prior_scale_avg_qty_next_week_week_nr
}
print("Non_model_params_currently are: ", dict_with_non_model_params_to_print)
# m.add_country_holidays(country_name='UK')
# m.add_regressor('month')
# m.add_regressor('week_num')
# m.add_regressor('avg_month_qty_over_df')
# m.add_regressor('avg_week_nr_qty_over_df')
# m.add_regressor('BH_Minus_1_Week')
# m.add_regressor('BH_Minus_4_Weekday')
# m.add_regressor('BH_Minus_3_Weekday')
# m.add_regressor('BH_Minus_2_Weekday')
# m.add_regressor('BH_Minus_1_Weekday')
# m.add_regressor('BH_Nearest_Sat')
# m.add_regressor('Between_Xmas_NY')
# m.add_regressor('Xmas_NY')
# m.add_regressor('BH')
# m.add_regressor('BH_Plus_1_Weekday')
# m.add_regressor('BH_Plus_2_Weekday')
# m.add_regressor('BH_Plus_3_Weekday')
# m.add_regressor('BH_Plus_4_Weekday')
# m.add_regressor('BH_Plus_1_Week')
# m.add_regressor('Holiday_Xmas_Before_After')
# m.add_regressor('Holiday_Xmas')
# m.add_regressor('Holiday_Easter_Before_After')
# m.add_regressor('Holiday_Easter')
# m.add_regressor('Holiday_Summer_Before_After')
# m.add_regressor('Holiday_Summer')
# m.add_regressor('Half_Term')
m.fit(df_for_cv)
df_cv = cross_validation(
m, cutoffs=cutoffs, horizon="{} days".format(weeks_to_forecast*7), parallel="processes"
)
df_p = performance_metrics(df_cv, rolling_window=1)
return df_p["mse"].values[0]
# Find the best parameters
optuna_prophet = optuna.create_study(
direction="minimize", sampler=TPESampler(seed=321)
)
# * n_trials optuna hyperparameter.
# optuna_prophet.optimize(objective, n_trials=5000)
optuna_prophet.optimize(objective, n_trials=1000)
prophet_trial = optuna_prophet.best_trial
prophet_trial_params = prophet_trial.params
list_of_variables_outside_the_param_grid = ['prior_scale_month',
'prior_scale_week_num',
'prior_scale_avg_month_qty_over_df',
'prior_scale_avg_week_nr_qty_over_df',
'prior_scale_avg_solar_rad_of_month',
'prior_scale_avg_solar_rad_of_week_nr',
'prior_scale_avg_temp_of_month',
'prior_scale_avg_temp_of_week_nr',
'prior_scale_avg_pres_of_month',
'prior_scale_avg_pres_of_week_nr',
'prior_scale_avg_solar_rad_last_week_week_nr',
'prior_scale_avg_temp_last_week_week_nr',
'prior_scale_avg_pres_last_week_week_nr',
'prior_scale_avg_solar_rad_last_2weeks_week_nr',
'prior_scale_avg_temp_last_2weeks_week_nr',
'prior_scale_avg_pres_last_2weeks_week_nr',
'prior_scale_avg_solar_rad_next_week_week_nr',
'prior_scale_avg_temp_next_week_week_nr',
'prior_scale_avg_pres_next_week_week_nr',
'prior_scale_avg_solar_rad_of_last_month',
'prior_scale_avg_temp_of_last_month',
'prior_scale_avg_pres_of_last_month',
'prior_scale_avg_qty_last_2weeks_week_nr',
'prior_scale_avg_qty_next_week_week_nr']
params_outside_the_param_grid={}
param_grid = {}
for param_name in prophet_trial_params.keys():
if param_name in list_of_variables_outside_the_param_grid:
params_outside_the_param_grid.update({param_name : prophet_trial_params[param_name]})
else:
param_grid.update({param_name : prophet_trial_params[param_name]})
return param_grid, params_outside_the_param_grid
The 300 dataframes on which I tried to optimize the Prophet model for, are about >280 rows & < 290 rows each. I do a 9 fold cross_validation using cuttofs (a list datetime objects, they are created using the generate_cutoffs() where df is the dataframe I want to compute the hyperparameter optimization, and weeks_to_forecast is always 12)
Has anybody experienced something similar? Or does anybody know any workarounds?

Related

I'm working with itertools product and trying to return possible permutations.
This is what I'm trying to pass:
from itertools import product
df = {'A':[0,0.03,0.5],
'B':[0,0.02,0.5],
'C':[0,0.015,0.03],
'D':[0,0.15,0.3],
'E':[0,0.1,0.4],
'F':[0,0.06,0.5],
'G':[0,0.07,0.5]}
for element in product(list(df.values())):
print(element)
What I get back:
([0, 0.03, 0.5],)
([0, 0.02, 0.5],)
([0, 0.015, 0.03],)
([0, 0.15, 0.3],)
([0, 0.1, 0.4],)
([0, 0.06, 0.5],)
([0, 0.07, 0.5],)
What I want:
All possible permutations of the numbers, one possible solution would look like:
[0,0.02,0.015,0.15,0.4,0.06,0.07]
Why is the product method not working as desired. Working out of Jupyter Notebooks - Python 3.

Are you looking for this?
from itertools import product
df = {'A':[0,0.03,0.5],
'B':[0,0.02,0.5],
'C':[0,0.015,0.03],
'D':[0,0.15,0.3],
'E':[0,0.1,0.4],
'F':[0,0.06,0.5],
'G':[0,0.07,0.5]}
for element in product(*df.values()):
print(element)
example output:
...
(0.5, 0.5, 0.03, 0.3, 0.1, 0.5, 0)
(0.5, 0.5, 0.03, 0.3, 0.1, 0.5, 0.07)
(0.5, 0.5, 0.03, 0.3, 0.1, 0.5, 0.5)
(0.5, 0.5, 0.03, 0.3, 0.4, 0, 0)
(0.5, 0.5, 0.03, 0.3, 0.4, 0, 0.07)
(0.5, 0.5, 0.03, 0.3, 0.4, 0, 0.5)
(0.5, 0.5, 0.03, 0.3, 0.4, 0.06, 0)
(0.5, 0.5, 0.03, 0.3, 0.4, 0.06, 0.07)
(0.5, 0.5, 0.03, 0.3, 0.4, 0.06, 0.5)
(0.5, 0.5, 0.03, 0.3, 0.4, 0.5, 0)
(0.5, 0.5, 0.03, 0.3, 0.4, 0.5, 0.07)
(0.5, 0.5, 0.03, 0.3, 0.4, 0.5, 0.5)

You need to flatten the list as below:
for element in product([item for data in df.values() for item in data)):
print(element)

I'm looking for a similar function to tf.unsorted_segment_sum, but I don't want to sum the segments, I want to get every segment as a tensor.
So for example, I have this code:
(In real, I have a tensor with shapes of (10000, 63), and the number of segments would be 2500)
to_be_sliced = tf.constant([[0.1, 0.2, 0.3, 0.4, 0.5],
[0.3, 0.2, 0.2, 0.6, 0.3],
[0.9, 0.8, 0.7, 0.6, 0.5],
[2.0, 2.0, 2.0, 2.0, 2.0]])
indices = tf.constant([0, 2, 0, 1])
num_segments = 3
tf.unsorted_segment_sum(to_be_sliced, indices, num_segments)
The output would be here
array([sum(row1+row3), row4, row2]
What I am looking for is 3 tensor with different shapes (maybe a list of tensors), first containing the first and third rows of the original (shape of (2, 5)), the second contains the 4th row (shape of (1, 5)), the third contains the second row, like this:
[array([[0.1, 0.2, 0.3, 0.4, 0.5],
[0.9, 0.8, 0.7, 0.6, 0.5]]),
array([[2.0, 2.0, 2.0, 2.0, 2.0]]),
array([[0.3, 0.2, 0.2, 0.6, 0.3]])]
Thanks in advance!

You can do that like this:
import tensorflow as tf
to_be_sliced = tf.constant([[0.1, 0.2, 0.3, 0.4, 0.5],
[0.3, 0.2, 0.2, 0.6, 0.3],
[0.9, 0.8, 0.7, 0.6, 0.5],
[2.0, 2.0, 2.0, 2.0, 2.0]])
indices = tf.constant([0, 2, 0, 1])
num_segments = 3
result = [tf.boolean_mask(to_be_sliced, tf.equal(indices, i)) for i in range(num_segments)]
with tf.Session() as sess:
print(*sess.run(result), sep='\n')
Output:
[[0.1 0.2 0.3 0.4 0.5]
[0.9 0.8 0.7 0.6 0.5]]
[[2. 2. 2. 2. 2.]]
[[0.3 0.2 0.2 0.6 0.3]]

For your case, you can do Numpy slicing in Tensorflow. So this will work:
sliced_1 = to_be_sliced[:3, :]
# [[0.4 0.5 0.5 0.7 0.8]
# [0.3 0.2 0.2 0.6 0.3]
# [0.3 0.2 0.2 0.6 0.3]]
sliced_2 = to_be_sliced[3, :]
# [0.3 0.2 0.2 0.6 0.3]
Or a more general option, you can do it in the following way:
to_be_sliced = tf.constant([[0.1, 0.2, 0.3, 0.4, 0.5],
[0.3, 0.2, 0.2, 0.6, 0.3],
[0.9, 0.8, 0.7, 0.6, 0.5],
[2.0, 2.0, 2.0, 2.0, 2.0]])
first_tensor = tf.gather_nd(to_be_sliced, [[0], [2]])
second_tensor = tf.gather_nd(to_be_sliced, [[3]])
third_tensor = tf.gather_nd(to_be_sliced, [[1]])
concat = tf.concat([first_tensor, second_tensor, third_tensor], axis=0)

I have an array of probabilities. I would like the columns to sum to 1 (representing probability) and the rows to sum to X (where X is an integer, say 9 for example).
I thought that I could normalize the columns, and then normalize the rows and times by X. But this didn't work, the resulting sums of the rows and columns were not perfectly 1.0 and X.
This is what I tried:
# B is 5 rows by 30 columns
# Normalizing columns to 1.0
col_sum = []
for col in B.T:
col_sum.append(sum(col))
for row in range(B.shape[0]):
for col in range(B.shape[1]):
if B[row][col] != 0.0 and B[row][col] != 1.0:
B[row][col] = (B[row][col] / col_sum[col])
# Normalizing rows to X (9.0)
row_sum = []
for row in B:
row_sum.append(sum(row))
for row in range(B.shape[0]):
for col in range(B.shape[1]):
if B[row][col] != 0.0 and B[row][col] != 1.0:
B[row][col] = (B[row][col] / row_sum[row]) * 9.0

I'm not sure if I understood correctly, but it seems like what you're trying to accomplish might mathematically not be feasible?
Imagine you have a 2x2 matrix where you want the rows to sum up to 1 and the columns to 10. Even if you made all the numbers in the columns 1 (their max possible value) you would still not be able to sum them up to 10 in their columns?

This can only work if your matrix's number of columns is X times the number of rows. For example, if X = 3 and you have 5 rows, then you must have 15 columns. So, you could make your 5x30 matrix work for X=6 but not X=9.
The reason for this is that, if each column sums up to 1.0, the total of all values in the matrix will be 1.0 times the number of columns. And since you want each row to sum up to X, then the total of all values must also be X times the number of rows.
So: Columns * 1.0 = X * Rows
If that constraint is met, you only have to adjust all values proportionally to X/sum(row) and both dimensions will work automatically unless the initial values are not properly balanced. If the matrix is not already balanced, adjusting the values would be similar to solving a sudoku (allegedly an NP problem) and the result would largely be unrelated to the initial values. The matrix is balanced when all rows, adjusted to have the same sum, result in all columns having the same sum.
[0.7, 2.1, 1.4, 0.7, 1.4, 1.4, 0.7, 1.4, 1.4, 2.1, 0.7, 2.1, 1.4, 2.1, 1.4] 21
[2.8, 1.4, 0.7, 2.1, 1.4, 2.1, 0.7, 1.4, 2.1, 1.4, 0.7, 0.7, 1.4, 0.7, 1.4] 21
[1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 0.7, 2.8, 0.7, 0.7, 1.4, 2.1] 21
[1.4, 1.4, 1.4, 1.4, 2.1, 1.4, 1.4, 1.4, 0.7, 0.7, 2.1, 1.4, 1.4, 1.4, 1.4] 21
[0.7, 0.7, 2.1, 1.4, 0.7, 0.7, 2.8, 1.4, 1.4, 2.1, 0.7, 2.1, 2.1, 1.4, 0.7] 21
apply x = x * 3 / 21 to all elements ...
[0.1, 0.3, 0.2, 0.1, 0.2, 0.2, 0.1, 0.2, 0.2, 0.3, 0.1, 0.3, 0.2, 0.3, 0.2] 3.0
[0.4, 0.2, 0.1, 0.3, 0.2, 0.3, 0.1, 0.2, 0.3, 0.2, 0.1, 0.1, 0.2, 0.1, 0.2] 3.0
[0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.1, 0.4, 0.1, 0.1, 0.2, 0.3] 3.0
[0.2, 0.2, 0.2, 0.2, 0.3, 0.2, 0.2, 0.2, 0.1, 0.1, 0.3, 0.2, 0.2, 0.2, 0.2] 3.0
[0.1, 0.1, 0.3, 0.2, 0.1, 0.1, 0.4, 0.2, 0.2, 0.3, 0.1, 0.3, 0.3, 0.2, 0.1] 3.0
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]

I would like to plot a histogram with a non-uniform x-axis using Matplotlib.
For example, consider the following histogram:
import matplotlib.pyplot as plt
values = [0.68, 0.28, 0.31, 0.5, 0.25, 0.5, 0.002, 0.13, 0.002, 0.2, 0.3, 0.45,
0.56, 0.53, 0.001, 0.44, 0.008, 0.26, 0., 0.37, 0.03, 0.002, 0.19, 0.18,
0.04, 0.31, 0.006, 0.6, 0.19, 0.3, 0., 0.46, 0.2, 0.004, 0.06, 0.]
plt.hist(values)
plt.show()
The first bin has high density, so I would like to zoom in there.
Ideally, I would like to change the values in the x-axis to something like [0, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1], keeping the bin widths constant within the graph (but not numerically, of course). Is there an easy way to achieve this?
Any comments or suggestions are welcome.

Using bins will solve the problems. The bins are the values to which you assign the values for example 0.28 will be assigned to bin 0.3. The code below provides you an example of using bins:
import matplotlib.pyplot as plt
values = [0.68, 0.28, 0.31, 0.5, 0.25, 0.5, 0.002, 0.13, 0.002, 0.2, 0.3, 0.45,
0.56, 0.53, 0.001, 0.44, 0.008, 0.26, 0., 0.37, 0.03, 0.002, 0.19, 0.18,
0.04, 0.31, 0.006, 0.6, 0.19, 0.3, 0., 0.46, 0.2, 0.004, 0.06, 0.]
plt.hist(values, bins=[0, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1])
plt.show()
To plot it in a more suitable way, it can be handy to convert the x axis into a logaritmic scale:
plt.hist(values, bins=[0, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1], log=True)
changes the log scale on the y axis. Adding the following line to your code will make a logaritmic x axis for your histogram:
plt.xscale('log')

The solution from André is nice, but the bin widths are not constant. Working with a log2 x-axis suits what I was looking for. I use np.logspace to make the bin widths constant in the graph.
That's what I ended up doing:
import matplotlib.pyplot as plt
values = [0.68, 0.28, 0.31, 0.5, 0.25, 0.5, 0.002, 0.13, 0.002, 0.2, 0.3, 0.45,
0.56, 0.53, 0.001, 0.44, 0.008, 0.26, 0., 0.37, 0.03, 0.002, 0.19, 0.18,
0.04, 0.31, 0.006, 0.6, 0.19, 0.3, 0., 0.46, 0.2, 0.004, 0.06, 0.]
bins = np.logspace(-10, 1, 20, base=2)
bins[0]=0
fig, ax = plt.subplots()
plt.hist(values, bins=bins)
ax.set_xscale('log', basex=2)
ax.set_xlim(2**-10, 1)
plt.show()

I have a coding interface which has a counter component. It simply increments by 1 with every update. Consider it an infinite generator of {1,2,3,...} over time which I HAVE TO use.
I need to use this value and iterate from -1.5 to 1.5. So, the iteration should start from -1.5 and reach 1.5 and then from 1.5 back to -1.5.
How should I use this infinite iterator to generate an iteration in that range?

You can use cycle from itertools to repeat a sequence.
from itertools import cycle
# build the list with 0.1 increment
v = [(x-15)/10 for x in range(31)]
v = v + list(reversed(v))
cv = cycle(v)
for c in my_counter:
x = next(cv)
This will repeat the list v:
-1.5, -1.4, -1.3, -1.2, -1.1, -1.0, -0.9, -0.8, -0.7, -0.6, -0.5, -0.4,
-0.3, -0.2, -0.1, 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0,
1.1, 1.2, 1.3, 1.4, 1.5, 1.5, 1.4, 1.3, 1.2, 1.1, 1.0, 0.9, 0.8, 0.7,
0.6, 0.5, 0.4, 0.3, 0.2, 0.1, 0.0, -0.1, -0.2, -0.3, -0.4, -0.5, -0.6,
-0.7, -0.8, -0.9, -1.0, -1.1, -1.2, -1.3, -1.4, -1.5, -1.5, -1.4, -1.3,
-1.2, -1.1, -1.0, -0.9, -0.8, -0.7, -0.6, -0.5, -0.4, -0.3, -0.2, -0.1,
0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3,
1.4, 1.5, 1.5, 1.4, 1.3, 1.2, 1.1, 1.0, 0.9 ...

Something like:
import itertools
infGenGiven = itertools.count() # This is similar your generator
def func(x):
if x%2==0:
return 1.5
else:
return -1.5
infGenCycle = itertools.imap(func, infGenGiven)
count=0
while count<10:
print infGenCycle.next()
count+=1
Output:
1.5
-1.5
1.5
-1.5
1.5
-1.5
1.5
-1.5
1.5
-1.5
Note that this starts 1.5 because the first value in infGenGiven is 0, although for your generator it is 1 and so the infGenCycle output will give you what you want.

Thank you all.
I guess the best approach is to use the trigonometric functions (sine or cosine) which oscillate between plus and minus one.
More details at: https://en.wikipedia.org/wiki/Trigonometric_functions
Cheers