We Keep Coding

Pandas: how to incrementally add one to column while sum is less than corresponding column?

I am trying to increment a column by 1 while the sum of that column is less than or equal to a total supply figure. I also need that column to be less than the corresponding value in the 'allocation' column. The supply variable will be dynamic from 1-400 based on user input. Below is the desired output (Allocation Final column).
supply = 14
| rank | allocation | Allocation Final |
| ---- | ---------- | ---------------- |
| 1 | 12 | 9 |
| 2 | 3 | 3 |
| 3 | 1 | 1 |
| 4 | 1 | 1 |
Below is the code I have so far:
data = [[1.05493,12],[.94248,3],[.82317,1],[.75317,1]]
df = pd.DataFrame(data,columns=['score','allocation'])
df['rank'] = df['score'].rank()
df['allocation_new'] = 0
#static for testing
supply = 14
for index in df.index:
while df.loc[index, 'allocation_new'] < df.loc[index, 'allocation'] and df.loc[index, 'allocation_new'].sum() < supply:
df.loc[index, 'allocation_new'] += 1
print(df)

This should do:
def allocate(df, supply):
if supply > df['allocation'].sum():
raise ValueError(f'Unacheivable supply {supply}, maximal {df["allocation"].sum()}')
under_alloc = pd.Series(True, index=df.index)
df['allocation final'] = 0
while (missing := supply - df['allocation final'].sum()) >= 0:
assert under_alloc.any()
if missing <= under_alloc.sum():
df.loc[df.index[under_alloc][:missing], 'allocation final'] += 1
return df
df.loc[under_alloc, 'allocation final'] = (
df.loc[under_alloc, 'allocation final'] + missing // under_alloc.sum()
).clip(upper=df.loc[under_alloc, 'allocation'])
under_alloc = df['allocation final'] < df['allocation']
return df
At every iteration, we add the missing quotas to any rows that did not reach the allocation yet (rounded down, that’s missing // under_alloc.sum()), then using pd.Series.clip() to ensure we stay below the allocation.
If there’s less missing quotas than available ranks to which to allocate (e.g. run the same dataframe with supply=5 or 6), we allocate to the first missing ranks.
>>> df = pd.DataFrame( {'allocation': {0: 12, 1: 3, 2: 1, 3: 1}, 'rank': {0: 1, 1: 2, 2: 3, 3: 4}})
>>> print(allocate(df, 14))
allocation rank allocation final
0 12 1 9
1 3 2 3
2 1 3 1
3 1 4 1
>>> print(allocate(df, 5))
allocation rank allocation final
0 12 1 2
1 3 2 1
2 1 3 1
3 1 4 1

Here is a simpler version:
def allocate(series, supply):
allocated = 0
values = [0]*len(series)
while True:
for i in range(len(series)):
if allocated >= supply:
return values
if values[i] < series.iloc[i]:
values[i]+=1
allocated+=1
pass
allocate(df['allocation'], 14)
output:
[9,3,1,1]

Intersection over Union over two segments

Let's say I have this two Dataframes : Groundtruth and Prediction.
Each Dataframe has 3 columns ; Action, Start and End.
**Prediction :**
Action | Start | End
-------------------------
3 | 0 | 10
2 | 10 | 70
3 | 80 | 120
0 | 120 | 350
7 | 400 | 610
...
**Groundtruth :**
Action | Start | End
-------------------------
2 | 20 | 140
0 | 150 | 340
6 | 420 | 600
...
I want to compute the Intersection-over-Union (IoU) over these two dataframes using all columns, meaning Action first to see if it's a correct prediction or not plus the start and the end segments for each action to see if starts and ends correctly.
Here's my code :
def compute_iou(y_pred, y_true):
y_pred = y_pred.flatten()
y_true = y_true.flatten()
cm = confusion_matrix(y_true, y_pred)
intersection = np.diag(cm)
ground_truth_set = cm.sum(axis=1)
predicted_set = cm.sum(axis=0)
union = ground_truth_set + predicted_set - intersection
IoU = intersection / union
for i in range(len(IoU)):
if (IoU[i]>0.5):
IoU[i] = 1
return round(np.mean(IoU)*100, 3)
This works when I want to calculate the IoU over the actions column.
Now how can I adapt this so I can get IoU to get the overlapping segments over the start and end columns ?
PS : Groundtruth and Prediction dataframes don't have the same number of rows.

(post edit)
The calculation is broken into three cases:
Overlap: where the activity matches, and there's an overlap between the ground truth interval and the prediction interval.
No overlap: the activity matches, but there's no such overlap.
No hit: the activity wasn't predicted at all, or there's a wrong activity.
Here's the code:
df = pd.merge(pred, groundtruth, on = "Action", how = "outer", suffixes = ["_pred", "_gt"])
overlap = df[(df.Start_pred < df.End_gt) & (df.Start_gt < df.End_pred)]
intersection = (overlap[["End_pred", "End_gt"]].min(axis=1) - overlap[["Start_pred", "Start_gt"]].max(axis=1)).sum()
union_where_overlap = (overlap[["End_pred", "End_gt"]].max(axis=1) - overlap[["Start_pred", "Start_gt"]].\
min(axis=1)).sum()
no_hit = df[df.isna().sum(axis=1) > 0]
union_no_hit = (no_hit[["End_pred", "End_gt"]].max(axis=1) - no_hit[["Start_pred", "Start_gt"]].min(axis=1)).sum()
no_overlap = df[~((df.Start_pred < df.End_gt) & (df.Start_gt < df.End_pred))].dropna()
union_no_overlap = ((no_overlap.End_pred - no_overlap.Start_pred) + (no_overlap.End_gt - no_overlap.Start_gt)).sum()
IoU = intersection / (union_no_hit + union_where_overlap + union_no_overlap)

python Reverse Collatz Conjecture

What the program should do is take steps and a number and than output you how many unique sequences there are with exactly x steps to create number.
Does someone know how I can save some memory - as I should make this work for pretty huge numbers within a 4 second limit.
def IsaacRule(steps, number):
if number in IsaacRule.numbers:
return 0
else:
IsaacRule.numbers.add(number)
if steps == 0:
return 1
counter = 0
if ((number - 1) / 3) % 2 == 1:
counter += IsaacRule(steps-1, (number - 1) / 3)
if (number * 2) % 2 == 0:
counter += IsaacRule(steps-1, number*2)
return counter
IsaacRule.numbers = set()
print(IsaacRule(6, 2))
If someone knows a version with memoization I would be thankful, right now it works, but there is still room for improvement.

Baseline: IsaacRule(50, 2) takes 6.96s
0) Use the LRU Cache
This made the code take longer, and gave a different final result
1) Eliminate the if condition: (number * 2) % 2 == 0 to True
IsaacRule(50, 2) takes 0.679s. Thanks Pm2Ring for this one.
2) Simplify ((number - 1) / 3) % 2 == 1 to number % 6 == 4 and use floor division where possible:
IsaacRule(50, 2) takes 0.499s
Truth table:
| n | n-1 | (n-1)/3 | (n-1)/3 % 2 | ((n-1)/3)%2 == 1 |
|---|-----|---------|-------------|------------------|
| 1 | 0 | 0.00 | 0.00 | FALSE |
| 2 | 1 | 0.33 | 0.33 | FALSE |
| 3 | 2 | 0.67 | 0.67 | FALSE |
| 4 | 3 | 1.00 | 1.00 | TRUE |
| 5 | 4 | 1.33 | 1.33 | FALSE |
| 6 | 5 | 1.67 | 1.67 | FALSE |
| 7 | 6 | 2.00 | 0.00 | FALSE |
Code:
def IsaacRule(steps, number):
if number in IsaacRule.numbers:
return 0
else:
IsaacRule.numbers.add(number)
if steps == 0:
return 1
counter = 0
if number % 6 == 4:
counter += IsaacRule(steps-1, (number - 1) // 3)
counter += IsaacRule(steps-1, number*2)
return counter
3) Rewrite code using sets
IsaacRule(50, 2) takes 0.381s
This lets us take advantage of any optimizations made for sets. Basically I do a breadth first search here.
4) Break the cycle so we can skip keeping track of previous states.
IsaacRule(50, 2) takes 0.256s
We just need to add a check that number != 1 to break the only known cycle. This gives a speed up, but you need to add a special case if you start from 1. Thanks Paul for suggesting this!
START = 2
STEPS = 50
# Special case since we broke the cycle
if START == 1:
START = 2
STEPS -= 1
current_candidates = {START} # set of states that can be reached in `step` steps
for step in range(STEPS):
# Get all states that can be reached from current_candidates
next_candidates = set(number * 2 for number in current_candidates if number != 1) | set((number - 1) // 3 for number in current_candidates if number % 6 == 4)
# Next step of BFS
current_candidates = next_candidates
print(len(next_candidates))

Gurobi: How can I sum just a part of a variable?

I have the following model:
from gurobipy import *
n_units = 1
n_periods = 3
n_ageclasses = 4
units = range(1,n_units+1)
periods = range(1,n_periods+1)
periods_plus1 = periods[:]
periods_plus1.append(max(periods_plus1)+1)
ageclasses = range(1,n_ageclasses+1)
nothickets = ageclasses[1:]
model = Model('MPPM')
HARVEST = model.addVars(units, periods, nothickets, vtype=GRB.INTEGER, name="HARVEST")
FOREST = model.addVars(units, periods_plus1, ageclasses, vtype=GRB.INTEGER, name="FOREST")
model.addConstrs((quicksum(HARVEST[(k+1), (t+1), nothicket] for k in range(n_units) for t in range(n_periods) for nothicket in nothickets) == FOREST[unit, period+1, 1] for unit in units for period in periods if period < max(periods_plus1)), name="A_Thicket")
I have a problem with formulating the constraint. I want for every unit and every period to sum the nothickets part of the variable HARVEST. Concretely I want xk=1,t=1,2 + xk=1,t=1,3 + xk=1,t=1,4
and so on. This should result in only three ones per row of the constraint matrix. But with the formulation above I get 9 ones.
I tried to use a for loop outside of the sum, but this results in another problem:
for k in range(n_units):
for t in range(n_periods):
model.addConstrs((quicksum(HARVEST[(k+1), (t+1), nothicket] for nothicket in nothickets) == FOREST[unit,period+1, 1] for unit in units for period in periods if period < max(periods_plus1)), name="A_Thicket")
With this formulation I get this matrix:
constraint matrix
But what I want is:
row_idx | col_idx | coeff
0 | 0 | 1
0 | 1 | 1
0 | 2 | 1
0 | 13 | -1
1 | 3 | 1
1 | 4 | 1
1 | 5 | 1
1 | 17 | -1
2 | 6 | 1
2 | 7 | 1
2 | 8 | 1
2 | 21 | -1
Can anybody please help me to reformulate this constraint?

This worked for me:
model.addConstrs((HARVEST.sum(unit, period, '*') == ...

How do I put data from a while loop into a table?

Basically I'm estimating pi using polygons. I have a loop which gives me a value for n, ann and bnn before running the loop again. here is what I have so far:
def printPiTable(an,bn,n,k):
"""Prints out a table for values n,2n,...,(2^k)n"""
u = (2**k)*n
power = 0
t = ((2**power)*n)
while t<=u:
if power < 1:
print(t,an,bn)
power = power + 1
t = ((2**power)*n)
else:
afrac = (1/2)*((1/an)+(1/bn))
ann = 1/afrac
bnn = sqrt(ann*bn)
print(t,ann,bnn)
an = ann
bn = bnn
power = power + 1
t = ((2**power)*n)
return
This is what I get if I run it with these values:
>>> printPiTable(4,2*sqrt(2),4,5)
4 4 2.8284271247461903
8 3.3137084989847607 3.0614674589207187
16 3.1825978780745285 3.121445152258053
32 3.1517249074292564 3.1365484905459398
64 3.1441183852459047 3.1403311569547534
128 3.1422236299424577 3.1412772509327733
I want to find a way to make it instead of printing out these values, just print the values in a nice neat table, any help?

Use string formatting. For example,
print('{:<4}{:>20f}{:>20f}'.format(t,ann,bnn))
produces
4 4.000000 2.828427
8 3.313708 3.061467
16 3.182598 3.121445
32 3.151725 3.136548
64 3.144118 3.140331
128 3.142224 3.141277
{:<4} is replaced by t, left-justified, formatted to a string of length 4.
{:>20f} is replaced by ann, right-justified, formatted as a float to a string of length 20.
The full story on the format string syntax is explained here.
To add column headers, just add a print statement like
print('{:<4}{:>20}{:>20}'.format('t','a','b'))
For fancier ascii tables, consider using a package like prettytable:
import prettytable
def printPiTable(an,bn,n,k):
"""Prints out a table for values n,2n,...,(2^k)n"""
table = prettytable.PrettyTable(['t', 'a', 'b'])
u = (2**k)*n
power = 0
t = ((2**power)*n)
while t<=u:
if power < 1:
table.add_row((t,an,bn))
power = power + 1
t = ((2**power)*n)
else:
afrac = (1/2)*((1/an)+(1/bn))
ann = 1/afrac
bnn = sqrt(ann*bn)
table.add_row((t,ann,bnn))
an = ann
bn = bnn
power = power + 1
t = ((2**power)*n)
print(table)
printPiTable(4,2*sqrt(2),4,5)
yields
+-----+---------------+---------------+
| t | a | b |
+-----+---------------+---------------+
| 4 | 4 | 2.82842712475 |
| 8 | 3.31370849898 | 3.06146745892 |
| 16 | 3.18259787807 | 3.12144515226 |
| 32 | 3.15172490743 | 3.13654849055 |
| 64 | 3.14411838525 | 3.14033115695 |
| 128 | 3.14222362994 | 3.14127725093 |
+-----+---------------+---------------+
Perhaps it is overkill for this sole purpose, but Pandas can make nice tables too, and can export them in other formats, such as HTML.

You can use output formatting to make it look pretty. Look here for an example:
http://docs.python.org/release/1.4/tut/node45.html