I have a constraint problem that I'm trying to solve with python-constraint
So let's say I have 3 locations: loc1,...loc3
Also, I have 7 devices: device1,...device7
Max amount of devices in each location: loc1:3, loc2:4, loc3:2
(for example maximum of 3 devices in loc1 and so on...)
And some constraints about the locations and the devices:
loc1: device1, device3, device7,
loc2: device1, device3, device4, device5, device6, device7
loc3: device2, device4, device5, device6
(meaning for example only device1, device3 and device7 can be in loc1.)
I'm trying to get a set of possible options for devices in locations.
from constraint import *
problem = Problem()
for key in locations_devices_dict:
problem.addVariable(key,locations_devices_dict[key])
# problem.addVariable("loc1", ['device1', 'device3', 'device7'])
problem.addConstraint(AllDifferentConstraint())
and I'm stuck on how to do the constrains. I've tried:
problem.addConstraint(MaxSumConstraint(3), 'loc1')
but it doesn't work, MaxSumConstraint does not sum what I need.
All devices must be placed somewhere
possible solution:
loc1: device1, device3
loc2: device4, device6, device7
loc3: device2, device5
Anyone has an idea?
(another python package/not to use any package, is also good idea if someone has any suggestions...)
This is simple assignment-like model:
So we have a binary variable indicating if device d is assigned to location L. The linear constraints are just:
assign each device to one location
each location has a maximum number of devices
make sure to use only allowed assignments (modeled above by allowed(L,d))
This problem can be handled by any constraint solver.
Enumerating all possible solutions is a bit dangerous. For large instances there are just way too many. Even for this small problem we already have 25 solutions:
For large problems this number will be astronomically large.
Using the Python constraint package this can look like:
from constraint import *
D = 7 # number of devices
L = 3 # number of locations
maxdev = [3,4,2]
allowed = [[1,3,7],[1,3,4,5,6,7],[2,4,5,6]]
problem = Problem()
problem.addVariables(["x_L%d_d%d" %(loc+1,d+1) for loc in range(L) for d in range(D) if d+1 in allowed[loc]],[0,1])
for loc in range(L):
problem.addConstraint(MaxSumConstraint(maxdev[loc]),["x_L%d_d%d" %(loc+1,d+1) for d in range(D) if d+1 in allowed[loc]])
for d in range(D):
problem.addConstraint(ExactSumConstraint(1),["x_L%d_d%d" %(loc+1,d+1) for loc in range(L) if d+1 in allowed[loc]])
S = problem.getSolutions()
n = len(S)
n
For large problems you may want to use dicts to speed things up.
edit: I wrote this answer before I saw #ErwinKalvelagen's code. So I did not check his solution...
So I used #ErwinKalvelagen approach and created a matrix that represented the probelm.
for each (i,j), x[i,j]=1 if device i can go to location j, 0 otherwise.
Then, I used addConstraint(MaxSumConstraint(maxAmount[i]), row) for each row - this is the constraint that represent the maximum devices in each location.
and addConstraint(ExactSumConstraint(1), col) for each column - this is the constraint that each device can be placed only in one location.
next, I took all x[i,j]=0 (device i can not be in location j) and for each t(i,j) addConstraint(lambda var, val=0: var == val, (t,))
This problem is similar to the sudoku problem, and I used this example for help
The matrix for my example above is:
(devices:) 1 2 3 4 5 6 7
loc1: 1 0 1 0 0 0 1
loc2: 1 0 1 1 1 1 1
loc3: 0 1 0 1 1 1 0
My code:
problem = Problem()
rows = range(locations_amount)
cols = range(devices_amount)
matrix = [(row, col) for row in rows for col in cols]
problem.addVariables(matrix, range(0, 2)) #each cell can get 0 or 1
rowSet = [zip([el] * len(cols), cols) for el in rows]
colSet = [zip(rows, [el] * len(rows)) for el in cols]
rowsConstrains = getRowConstrains() # list that has the maximum amount in each location(3,4,2)
#from my example: loc1:3, loc2:4, loc3:2
for i,row in enumerate(rowSet):
problem.addConstraint(MaxSumConstraint(rowsConstrains[i]), row)
for col in colSet:
problem.addConstraint(ExactSumConstraint(1), col)
s = getLocationsSet() # set that has all the tuples that x[i,j] = 1
for i, loc in enumerate(locations_list):
for j, iot in enumerate(devices_list):
t=(i,j)
if t in s:
continue
problem.addConstraint(lambda var, val=0: var == val, (t,)) # the value in these cells must be 0
solver = problem.getSolution()
example for a solution:
(devices:) 1 2 3 4 5 6 7
loc1: 1 0 1 0 0 0 1
loc2: 0 0 0 1 1 1 0
loc3: 0 1 0 0 0 0 0
Related
I am assigning stations to tasks with different types of stations/machines, where every station can have exactly one of the types available. My problem is that in the solution point Output I get the following:
# Objective value = 8.0028043478260865e+04
y[1,BTC1] 0
y[1,BOC1] 1
y[2,BTC1] 0
y[2,BOC1] 1
y[3,BTC1] 1
y[3,BOC1] 0
x[1,BTC1,w01] 0.5
x[1,BTC1,w02] 2.1739130434782608e-01
x[1,BTC1,w03] 0
x[1,BOC1,w01] 0
x[1,BOC1,w02] 0
x[1,BOC1,w03] 0
x[2,BTC1,w01] 0.5
x[2,BTC1,w02] 2.1739130434782608e-01
x[2,BTC1,w03] 0
x[2,BOC1,w01] 0
x[2,BOC1,w02] 0
x[2,BOC1,w03] 0
x[3,BTC1,w01] 0
x[3,BTC1,w02] 5.6521739130434789e-01
x[3,BTC1,w03] 0
x[3,BOC1,w01] 0
x[3,BOC1,w02] 0
x[3,BOC1,w03] 1
Note: [station,type,task]. My problem is now that there is a assignment to x [1,BTC1,'w01'] for example but indeed the station y[1,BTC1] doesnt even exist. Therefore it should not be possible to assign anything like x [1,BTC1,every task]. To avoid this I thought of making a if staitment.
In python gurobi I saw many articles of how to implement if to constraints and I know that it is not possible in a direct way. In the examples I saw something like if x >= y do this and else do this. But for my problem I need to say if y[i,j] == 0 than x[i,j,k] should be also 0 and if y != 0 than x is also != 0. So with this I want to ensure that no assignment to x is made until the y variable is not 1 for that exact combination of station and type. How would I do this?
I went through google ortools scheduling tutorial and created slightly different constraint.
Say, that we have a df, that indicates the following:
Each nurse can work only on the preassigned shift. According to the table, nurse 0 must work either on shift 0 or 1, the nurse 2 must work only on the shift 2 etc.
Furthermore, I added the following constraint: "If nurse_0 takes shift_0, then she must take shif_1". It's shown by column "is_child" - shift_1 is child_shift for shift_0 for nurse_0.
df_dict = {
"slot_number":[1,1,2,2,3],
"asset_name":['0','1','0','1','2'],
"is_child":["No","No","Yes",'No','No']
}
df = pd.DataFrame.from_dict(df_dict)
Create variables and a model:
num_nurses = 3
num_shifts = 3
all_nurses = range(num_nurses)
all_shifts = range(num_shifts)
model = cp_model.CpModel()
shifts = {}
for n in all_nurses:
for s in all_shifts:
shifts[(n, s)] = model.NewBoolVar('shift_n%is%i' % (n, s))
One nurse can take only one shift:
for s in all_shifts:
model.Add(sum(shifts[(n, s)] for n in all_nurses) == 1)
My constraint:
# whether the slot (key) have any children in other slots
children = {0: [1], 1: [] ,2: []}
# what nurses are considered the children (values) in which slot (key)
nurse_child_sched = {0:[], 1:[0], 2:[]}
# In this case if nurse 0 take slot 0, then she must take slot 1 too.
for s in all_shifts:
for n in all_nurses:
if (children[s]):
for child in children[s]:
if n in nurse_child_sched[child]:
model.Add((shifts[(n,s)] + shifts[(n,child)])==2)
print(f"{s}-parent_slot and {child}-child_slot were connected for nurse {n}")
The code to create schedules and show solutions:
class NursesPartialSolutionPrinter(cp_model.CpSolverSolutionCallback):
"""Print intermediate solutions."""
def __init__(self, shifts, num_nurses, num_shifts, sols):
cp_model.CpSolverSolutionCallback.__init__(self)
self._shifts = shifts
self._num_nurses = num_nurses
self._num_shifts = num_shifts
self._solutions = set(sols)
self._solution_count = 0
def on_solution_callback(self):
if self._solution_count in self._solutions:
print('Solution %i' % self._solution_count)
for n in range(self._num_nurses):
is_working = False
for s in range(self._num_shifts):
if self.Value(self._shifts[(n, s)]):
is_working = True
print(' Nurse %i works shift %i' % (n, s))
if not is_working:
print(' Nurse {} does not work'.format(n))
print()
self._solution_count += 1
def solution_count(self):
return self._solution_count
solver = cp_model.CpSolver()
solver.parameters.linearization_level = 0
a_few_solutions = range(5)
solution_printer = NursesPartialSolutionPrinter(shifts, num_nurses, num_shifts,a_few_solutions)
solver.SearchForAllSolutions(model, solution_printer)
And, finally, I have the following result:
Solution 0
Nurse 0 works shift 0
Nurse 0 works shift 1
Nurse 1 works shift 2
Nurse 2 does not work
Solution 1
Nurse 0 works shift 0
Nurse 0 works shift 1
Nurse 1 does not work
Nurse 2 works shift 2
Solution 2
Nurse 0 works shift 0
Nurse 0 works shift 1
Nurse 0 works shift 2
Nurse 1 does not work
Nurse 2 does not work
However, according to my logic, there must be also at least the solution below. I realize, that my constraint says the model to put nurse_0 on shift_0, but I need to just put the relation between two slots in case if nurse_0 is putted on first shift and no constraints otherwise. Thanks in advance.
Solution x
Nurse 1 works shift 0
Nurse 0 works shift 1
Nurse 0 works shift 2
Nurse 1 does not work
Nurse 2 does not work
Side note:
One nurse can take only one shift:
for s in all_shifts:
model.Add(sum(shifts[(n, s)] for n in all_nurses) == 1)
Here you have written "each shift is assigned to exactly one nurse"
If you wan to express "one nurse can take only one shift" you should write:
for n in all_nurses:
model.Add(sum(shifts[(n, s)] for s in all_shifts) == 1)
note: but in this case since nurse exactly work one shift, child shift is impossible so I guess your code is OK and not the comment...
I want to build a scheduling app in python using pandas.
The following DataFrame is initialised where 0 denotes if a person is busy and 1 if a person is available.
import pandas as pd
df = pd.DataFrame({'01.01.': [1,1,0], '02.01.': [0,1,1], '03.01.': [1,0,1]}, index=['Person A', 'Person B', 'Person C'])
>>> df
01.01. 02.01. 03.01.
Person A 1 0 1
Person B 1 1 0
Person C 0 1 1
I now want to randomly schedule n number of people per day if they are available. In other words, for every day, if people are available (1), randomly set n number of people to scheduled (2).
I tried something as follows:
# Required number of people across time / columns
required_number = [0, 1, 2]
# Iterate through time / columns
for col in range(len(df.columns)):
# Current number of scheduled people
current_number = (df.iloc[:, [col]].values==2).sum()
# Iterate through indices / rows / people
for ind in range(len(df.index)):
# Check if they are available (1) and
# if the required number of people has not been met yet
if (df.iloc[ind, col]==1 and
current_number<required_number[col]):
# Change "free" / 1 person to "scheduled" / 2
df.iloc[ind, col] = 2
# Increment scheduled people by one
current_number += 1
>>> df
01.01. 02.01. 03.01.
Person A 1 0 2
Person B 1 2 0
Person C 0 1 2
This works as intended but – because I'm simply looping, I have no way of adding randomness (ie. that Person A / B / C) are randomly selected so long as they are available. Is there a way of directly doing so in pandas?
Thanks. BBQuercus
You can randomly choose proper indices in a series and then change values corresponding to the chosen indices:
for i in range(len(df.columns)):
if sum(df.iloc[:,i] == 1) >= required_number[i]:
column = df.iloc[:,i].reset_index(drop=True)
#We are going to store indices in a list
a = [j for j in column.index if column[j] == 1]
random_indexes = np.random.choice(a, required_number[i], replace = False)
df.iloc[:,i] = [column[j] if j not in random_indexes else 2 for j in column.index]
Now df is the wanted result.
I have a dataset that I clustered using two different clustering algorithms. The results are about the same, but the cluster numbers are permuted.
Now for displaying the color coded labels, I want the label ids to be same for the same clusters.
How can I get correct permutation between the two label ids?
I can do this using brute force, but perhaps there is a better/faster method. I would greatly appreciate any help or pointers. If possible I am looking for a python function.
The most well-known algorithm for finding the optimum matching is the hungarian method.
Because it cannot be explained in a few sentences, I have to refer you to a book of your choice, or Wikipedia article "Hungarian algorithm".
You can probably get good results (even perfect if the difference is indeed tiny) by simply picking the maximum of the correspondence matrix and then removing that row and column.
I have a function that works for me. But it may fail when the two cluster results are very inconsistent, which leads to duplicated max values in the contingency matrix. If your cluster results are about the same, it should work.
Here is my code:
from sklearn.metrics.cluster import contingency_matrix
def align_cluster_index(ref_cluster, map_cluster):
"""
remap cluster index according the the ref_cluster.
both inputs must be nparray and have same number of unique cluster index values.
Xin Niu Jan-15-2020
"""
ref_values = np.unique(ref_cluster)
map_values = np.unique(map_cluster)
print(ref_values)
print(map_values)
num_values = ref_values.shape[0]
if ref_values.shape[0]!=map_values.shape[0]:
print('error: both inputs must have same number of unique cluster index values.')
return()
switched_col = set()
while True:
cont_mat = contingency_matrix(ref_cluster, map_cluster)
print(cont_mat)
# divide contingency_matrix by its row and col sums to avoid potential duplicated values:
col_sum = np.matmul(np.ones((num_values, 1)), np.sum(cont_mat, axis = 0).reshape(1, num_values))
row_sum = np.matmul(np.sum(cont_mat, axis = 1).reshape(num_values, 1), np.ones((1, num_values)))
print(col_sum)
print(row_sum)
cont_mat = cont_mat/(col_sum+row_sum)
print(cont_mat)
# ignore columns that have been switched:
cont_mat[:, list(switched_col)]=-1
print(cont_mat)
sort_0 = np.argsort(cont_mat, axis = 0)
sort_1 = np.argsort(cont_mat, axis = 1)
print('argsort contmat:')
print(sort_0)
print(sort_1)
if np.array_equal(sort_1[:,-1], np.array(range(num_values))):
break
# switch values according to the max value in the contingency matrix:
# get the position of max value:
idx_max = np.unravel_index(np.argmax(cont_mat, axis=None), cont_mat.shape)
print(cont_mat)
print(idx_max)
if (cont_mat[idx_max]>0) and (idx_max[0] not in switched_col):
cluster_tmp = map_cluster.copy()
print('switch', map_values[idx_max[1]], 'and:', ref_values[idx_max[0]])
map_cluster[cluster_tmp==map_values[idx_max[1]]]=ref_values[idx_max[0]]
map_cluster[cluster_tmp==map_values[idx_max[0]]]=ref_values[idx_max[1]]
switched_col.add(idx_max[0])
print(switched_col)
else:
break
print('final argsort contmat:')
print(sort_0)
print(sort_1)
print('final cont_mat:')
cont_mat = contingency_matrix(ref_cluster, map_cluster)
col_sum = np.matmul(np.ones((num_values, 1)), np.sum(cont_mat, axis = 0).reshape(1, num_values))
row_sum = np.matmul(np.sum(cont_mat, axis = 1).reshape(num_values, 1), np.ones((1, num_values)))
cont_mat = cont_mat/(col_sum+row_sum)
print(cont_mat)
return(map_cluster)
And here is some test code:
ref_cluster = np.array([2,2,3,1,0,0,0,1,2,1,2,2,0,3,3,3,3])
map_cluster = np.array([0,0,0,1,1,3,2,3,2,2,0,0,0,2,0,3,3])
c = align_cluster_index(ref_cluster, map_cluster)
print(ref_cluster)
print(c)
>>>[2 2 3 1 0 0 0 1 2 1 2 2 0 3 3 3 3]
>>>[2 2 2 1 1 3 0 3 0 0 2 2 2 0 2 3 3]
I've been reading about the Metropolis-Hastings (MH) algorithm. Theoretically, I understood how the algorithm works. Now, I am trying to implement the MH algorithm using python.
I came across the following notebook. It suits exactly my problem since I want to fit my data by a straight line taking into consideration the measurement errors on my data. I am going to paste the code I am finding difficulties to understand:
# initial m, b
m,b = 2, 0
# step sizes
mstep, bstep = 0.1, 10.
# how many steps?
nsteps = 10000
chain = []
probs = []
naccept = 0
print 'Running MH for', nsteps, 'steps'
# First point:
L_old = straight_line_log_likelihood(x, y, sigmay, m, b)
p_old = straight_line_log_prior(m, b)
prob_old = np.exp(L_old + p_old)
for i in range(nsteps):
# step
mnew = m + np.random.normal() * mstep
bnew = b + np.random.normal() * bstep
# evaluate probabilities
# prob_new = straight_line_posterior(x, y, sigmay, mnew, bnew)
L_new = straight_line_log_likelihood(x, y, sigmay, mnew, bnew)
p_new = straight_line_log_prior(mnew, bnew)
prob_new = np.exp(L_new + p_new)
if (prob_new / prob_old > np.random.uniform()):
# accept
m = mnew
b = bnew
L_old = L_new
p_old = p_new
prob_old = prob_new
naccept += 1
else:
# Stay where we are; m,b stay the same, and we append them
# to the chain below.
pass
chain.append((b,m))
probs.append((L_old,p_old))
print 'Acceptance fraction:', naccept/float(nsteps)
The code is simple and easy, but I have difficulties in understanding how the MH is being implemented.
My question is in the chain.append (the third line from the bottom). The author is appending m and b whether they were accepted or rejected. Why? Shouldn't he append only the accepted points?
The following R code demonstrates why it is important to capture the rejected case:
# 20 samples from 0 or 1. 1 has an 80% probability of being chosen.
the.population <- sample(c(0,1), 20, replace = TRUE, prob=c(0.2, 0.8))
# Create a new sample that only catches changes
the.sample <- c(the.population[1])
# Loop though the.population,
# but only copy the.population to the.sample if the value changes
for( i in 2:length(the.population))
{
if(the.population[i] != the.population[i-1])
the.sample <- append(the.sample, the.population[i])
}
When this code runs, the.population gets 20 values, for example:
0 1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 1 1 1 1
The probability of a 1 in this population is 16/20 or 0.8. Exactly the probability we expected...
The sample, on the other hand, which only records changes, looks like this:
0 1 0 1 0 1
The probability of a 1 in the sample is 3/6 or 0.5.
We are trying to build a distribution, rejecting the new values means that the old values are more likely than the new values. That needs to be captured so our distribution is correct.
From a quick reading of the algorithm description: When a candidate is rejected, it still counts as a step, but the value is the same as the old step. I.e. b, m are appended either way, but they only get updated (to bnew, mnew) in the case where the candidate is accepted.