We Keep Coding

How can I use my helper functions to get the correct output

So I created a helper function to help my main function in extracting stuff from a dictionary...
and here is my code and function
def rdict(recipes):
recipes_splitted = {}
for r in recipes:
recipe_name, parts = r.split(":")
recipe_parts = {}
for part in parts.split(','):
product, number = part.split('*')
recipe_parts[product] = int(number)
recipes_splitted[recipe_name] = recipe_parts
return recipes_splitted
def extract(recipes, data):
result = []
for r in recipes:
tmp = []
for key in data[r]:
tmp.append(f"{key}:{data[r][key]}")
final_string = ""
for i in range(len(tmp)):
if i < len(tmp) - 1:
final_string += tmp[i] + ", "
else:
final_string += tmp[i]
result.append(final_string)
return result
So what I'm trying to do is make sure data in extract(recipe, data) go through rdict(data) since rdict will convert data into a dictionary, which is what I need.. However, when I tried doing for key in rdict(data[r]): the output returns Error. String is not supscriptable..
what should I do to successfully implement the changes??
Edit
So from my current code, here is a sample input..
print(extract(recipes = ['T-Bone', 'Green Salad1'],data = ["Pork Stew:Cabbage*5,Carrot*1,Fatty Pork*10",
"Green Salad1:Cabbage*10,Carrot*2,Pineapple*5",
"T-Bone:Carrot*2,Steak Meat*1"]
))
and in order for my code to work, it has to be like this
print(extract(recipes = ['T-Bone', 'Green Salad1'], data = {'Pork Stew': {'Cabbage': 5, 'Carrot': 1, 'Fatty Pork': 10}, 'Green Salad1': {'Cabbage': 10, 'Carrot': 2, 'Pineapple': 5},'T-Bone': {'Carrot': 2, 'Steak Meat': 1}}))
So from the input, data should be changed from
data = ["Pork Stew:Cabbage*5,Carrot*1,Fatty Pork*10",
"Green Salad1:Cabbage*10,Carrot*2,Pineapple*5",
"T-Bone:Carrot*2,Steak Meat*1"]
to
data = {'Pork Stew': {'Cabbage': 5, 'Carrot': 1, 'Fatty Pork': 10}, 'Green Salad1': {'Cabbage': 10, 'Carrot': 2, 'Pineapple': 5},'T-Bone': {'Carrot': 2, 'Steak Meat': 1}}

Convert the data to dict in extract().
recipes = ['T-Bone', 'Green Salad1']
data = ["Pork Stew:Cabbage*5,Carrot*1,Fatty Pork*10",
"Green Salad1:Cabbage*10,Carrot*2,Pineapple*5",
"T-Bone:Carrot*2,Steak Meat*1"]
def rdict(recipes):
recipes_splitted = {}
for r in recipes:
recipe_name, parts = r.split(":")
recipe_parts = {}
for part in parts.split(','):
product, number = part.split('*')
recipe_parts[product] = int(number)
recipes_splitted[recipe_name] = recipe_parts
return recipes_splitted
def extract(recipes, data):
data = rdict(data) # convert data to dict first
result = []
for r in recipes:
tmp = []
for key in data[r]:
tmp.append(f"{key}:{data[r][key]}")
final_string = ""
for i in range(len(tmp)):
if i < len(tmp) - 1:
final_string += tmp[i] + ", "
else:
final_string += tmp[i]
result.append(final_string)
return result
print(extract(recipes, data))
Output:
['Carrot:2, Steak Meat:1', 'Cabbage:10, Carrot:2, Pineapple:5']

Renamed rdict to parse_recipe, and modified it to return a tuple that is lighter and easier to process
In extract:
a) Build a dict of recipes: data_recipes
b) Built result by getting the wanted recipes, with a guard against missing recipe (which be an empty dict:{} )
def parse_recipe(s):
recipe, ings_s = s.split(':')
ings_l = ings_s.split(',')
ings_d= {}
for ing in ings_l:
i,q = ing.split('*')
ings_d[i.strip()] = q.strip()
return recipe.strip(), ings_d
def extract(recipes, data):
data_recipes = {}
for s in data:
recipe, ings_d = parse_recipe(s)
data_recipes[recipe] = ings_d
return {r: data_recipes.get(r, dict()) for r in recipes}

OR-TOOLS: PCVRP with delivery patterns

Please help me on my periodic capacitated vehicle routing problem.
Find the same question here on google-groups.
What I am trying to model:
1 depot
multiple customers (let's assume 4)
multiple days (let's assume 3)
Multiple vehicle types with price per km and specific capacity
Every customer has: delivery frequency and demand per delivery
Every customer has to be assigned to a delivery pattern. Multiple delivery patterns are possible for each frequency. The pattern indicates if a delivery is possible on that day or not. For a frequency of 1 and 3 working days: [[1,0,0],[0,1,0],[0,0,1]], a frequency of 2 [[1,1,0],[1,0,1],[1,1,0]], a frequency of 3: [1,1,1]these are the possible patterns.
The demand of each customer per delivery is the same, for every day a delivery is performed
So we have a set of customers that have to be delivered by one depo. Multiple Vehicle Types are available for the job. The day of service is flexible because only the frequency is fixed to a customer, but there are multiple possibilities to fulfill the demand.
What I did so far:
I copied every node by the number of working days.
I restrict the start and end to the depo nodes of each day.
I multiply the vehicles by the number of days.
Furthermore, I handle the days as different kinds of cargo. I do assign a capacity of zero to the vehicle when it should not be used on that day.
[[10, 15, 15, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 10, 15, 15, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 10, 15, 15]]
The first 3 vehicles are for day one, the second three are day two
and the last 3 are day 3.
For the demand matrix I use the same trick. In a case of 4 customers + 1 depot the demand matrix could look like this
[[0, 3, 0, 5, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 3, 4, 5, 2, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 2]]
The second entry in the first list (3) and the 7th entry in the second list (3) are for the same customer just on two different days. The nodes were copied.
This works good so far. But it's not what I want to do.
I do assign a demand for every customer and day by hand, but I want to assign the demand by a chosen delivery pattern for each customer.
The model could assign pattern [1,0,1] to one customer with a frequency of 2 and a pattern of [0,0,1] to another customer. This would result in the possibility to plan a tour on day 3 with both demands combined.
I need help to define a dimension that manages the assignment of patterns to customers which results in "flexible" demand matrix.
Thank you for your help.
Full Code:
"""Periodic Capacited Vehicles Routing Problem (PCVRP)."""
from ortools.constraint_solver import routing_enums_pb2
from ortools.constraint_solver import pywrapcp
import numpy as np
def dublicate_nodes(base_distance_matrix, num_days):
matrix = []
for i in range(num_days):
day_element = []
for node in range(len(base_distance_matrix)):
node_element = []
for day in range(num_days):
node_element.append(base_distance_matrix[node])
day_element.append(np.concatenate(node_element).tolist())
matrix.extend(day_element)
#print(matrix)
return matrix
def depo_nodes(num_days, num_verhicles, num_nodes):
depo_array = []
for day in range(num_days):
for veh in range(int(num_verhicles/num_days)):
depo_array.append(day*num_nodes)
#print(depo_array)
return depo_array
def veh_types_costs(cost_array, num_per_type, num_days):
cost_array_km = []
for day in range(num_days):
for i in range(len(cost_array)):
for num in range(num_per_type[i]):
cost_array_km.append(cost_array[i])
#print(cost_array_km)
return(cost_array_km)
def veh_capacity_matrix(num_vehicle_per_type, vehicle_capacity_type, num_days):
matrix= []
for index in range(num_days):
matrix_element = []
for day in range(num_days):
for i, num in enumerate(num_vehicle_per_type):
for times in range(num):
if day == index:
matrix_element.append(vehicle_capacity_type[i]);
else:
matrix_element.append(0)
matrix.append(matrix_element)
#print(matrix)
return matrix
def create_data_model():
data = {}
data["num_days"] = 3 #Anzahl Tage
### Definition der Fahrezugtypen
data["num_vehicle_per_type"] = [1, 2] #Anzahl Fahrzeuge pro Typ
data["vehicle_costs_type_per_km"] = [1, 0.01] #Kosten pro km pro Fahrzeugtyp
data["vehicle_costs_type_per_stop"] = [1, 1000] #Kosten pro Stop pro Fahrzeugtyp
data['price_per_km'] = veh_types_costs(data["vehicle_costs_type_per_km"], data["num_vehicle_per_type"], data["num_days"]) #Matrix für price_per_km je Fahrzeug ertsellen
data["price_per_stop"] = veh_types_costs(data["vehicle_costs_type_per_stop"], data["num_vehicle_per_type"], data["num_days"])
data["vehicle_capacity_type"] = [10, 15] # Kapaität pro Fahrzeugtyp
data['vehicle_capacities_matrix'] = veh_capacity_matrix(data["num_vehicle_per_type"], data["vehicle_capacity_type"], data["num_days"]) #Kapazitäten pro Fahrzeugs pro Tag
print('vehicle_capacities_matrix')
print(data['vehicle_capacities_matrix'])
data["num_vehicles_per_day"] = sum(data["num_vehicle_per_type"]) #Gesamtanzahl der Fahrzeuge pro Tag
data['num_vehicles'] = data["num_days"] * data["num_vehicles_per_day"]# Gesamtanzahl der Fahrzeuge in gesamten Zeitraum
###Distanzmatrix bestimmen
data["base_distance_matrix"] = [
[
0, 548, 776, 696, 582
],
[
548, 0, 684, 308, 194
],
[
776, 684, 0, 992, 878
],
[
696, 308, 992, 0, 114
],
[
582, 194, 878, 114, 0
]
] #Distanzmatrix mit allen Kunden einzeln
data['distance_matrix'] = dublicate_nodes(data["base_distance_matrix"], data["num_days"]) # Distanzmatrix mit mehrfachen Nodes pro Kunden, je Tag ein Node
###Start und Ende festlegen
data["num_nodes"] = len(data["base_distance_matrix"]) # Anzahl Kunden
data['starts'] = depo_nodes(data["num_days"], data['num_vehicles'], data["num_nodes"]) #Deponodes (Start) für die einzelnen Fahrzeuge (Tage)
data['ends'] = depo_nodes(data["num_days"], data['num_vehicles'], data["num_nodes"]) #Deponodes (Ende) für die einzelnen Fahrzeuge (Tage)
###Demand pro Kunde
data['demands_matrix'] = [[0, 3, 0, 5, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 0, 3, 4, 5, 2, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 2]] #Demandmatrix mit je einer list pro Tag
return data
def print_solution(data, manager, routing, solution):
"""Prints solution on console."""
total_distance = 0
total_load = 0
for vehicle_id in range(data['num_vehicles']):
if vehicle_id % data["num_vehicles_per_day"] == 0:
print(("------Tag {}------").format(int(vehicle_id/data["num_vehicles_per_day"])))
if routing.IsVehicleUsed(assignment= solution, vehicle=vehicle_id) == False:
continue
index = routing.Start(vehicle_id)
if vehicle_id >= data["num_vehicles_per_day"]:
plan_output = 'Route for vehicle {}:\n'.format(abs(vehicle_id-data["num_vehicles_per_day"]*int(vehicle_id/data["num_vehicles_per_day"])))
else:
plan_output = 'Route for vehicle {}:\n'.format(vehicle_id)
route_costs = 0
route_load = 0
while not routing.IsEnd(index):
node_index = manager.IndexToNode(index)
capacity_ID = int(vehicle_id/data["num_vehicles_per_day"])
route_load += data['demands_matrix'][capacity_ID][node_index]
plan_output += ' {0} Load({1}) -> '.format(node_index, route_load)
previous_index = index
index = solution.Value(routing.NextVar(index))
route_costs += routing.GetArcCostForVehicle(
previous_index, index, vehicle_id)
plan_output += ' {0} Load({1})\n'.format(manager.IndexToNode(index),
route_load)
plan_output += 'Costs of the route: {}€\n'.format(route_costs)
plan_output += 'Load of the route: {}\n'.format(route_load)
print(plan_output)
total_distance += route_costs
total_load += route_load
print('Total costs of all routes: {}€'.format(total_distance))
print('Total load of all routes: {}'.format(total_load))
def main():
"""Periodic CVRP problem."""
# Instantiate the data problem.
data = create_data_model()
# Create the routing index manager.
manager = pywrapcp.RoutingIndexManager(len(data['distance_matrix']), data['num_vehicles'], data['starts'],
data['ends'])
# Create Routing Model.
routing = pywrapcp.RoutingModel(manager)
### Kostenfunktion je Fahrzeug festlegen ###
def create_cost_callback(dist_matrix, km_costs, stop_costs):
# Create a callback to calculate distances between cities.
def distance_callback(from_index, to_index):
from_node = manager.IndexToNode(from_index)
to_node = manager.IndexToNode(to_index)
return int(dist_matrix[from_node][to_node]) * (km_costs) + (stop_costs)
return distance_callback
for i in range(data['num_vehicles']):
cost_callback = create_cost_callback(data['distance_matrix'], data["price_per_km"][i],
data["price_per_stop"][i]) # Callbackfunktion erstellen
cost_callback_index = routing.RegisterTransitCallback(cost_callback) # registrieren
routing.SetArcCostEvaluatorOfVehicle(cost_callback_index, i) # Vehicle zuordnen
#Define Capacities for Vehicles for every Day
def create_demand_callback(demand_matrix, demand_index):
# Create a callback to calculate capacity usage.
def demand_callback(from_index):
#Returns the demand of the node.
# Convert from routing variable Index to demands NodeIndex.
from_node = manager.IndexToNode(from_index)
return demand_matrix[demand_index][from_node]
return demand_callback
for i in range(data["num_days"]): #Jedes Fahrzeug hat pro Tag eine andere Kapazität
demand_callback = create_demand_callback(data['demands_matrix'], i)
demand_callback_index = routing.RegisterUnaryTransitCallback(demand_callback)
dimension_name = 'Capacity_day_{}'.format(i)
routing.AddDimensionWithVehicleCapacity(
demand_callback_index,
0, # null capacity slack
data['vehicle_capacities_matrix'][i], # vehicle maximum capacities
True, # start cumul to zero
dimension_name)
# Setting first solution heuristic.
search_parameters = pywrapcp.DefaultRoutingSearchParameters()
search_parameters.first_solution_strategy = (
routing_enums_pb2.FirstSolutionStrategy.AUTOMATIC)
search_parameters.local_search_metaheuristic = (
routing_enums_pb2.LocalSearchMetaheuristic.AUTOMATIC)
search_parameters.time_limit.FromSeconds(1)
# Solve the problem.
solution = routing.SolveWithParameters(search_parameters)
# Print solution on console.
if solution:
print_solution(data, manager, routing, solution)
return solution
if __name__ == '__main__':
solution_array = []
solution = main()

So I found a solution for my problem.
The solving time for problems greater than 40 customers could be better.
I would appreciate improvement suggestions.
"""Periodic Capacited Vehicles Routing Problem with delivery pattern(PCVRP)."""
from ortools.constraint_solver import routing_enums_pb2
from ortools.constraint_solver import pywrapcp
import numpy as np
import random
import math
import pandas as pd
def dublicate_nodes(base_distance_matrix, num_days):
matrix = []
for i in range(num_days):
day_element = []
for node in range(len(base_distance_matrix)):
node_element = []
for day in range(num_days):
node_element.append(base_distance_matrix[node])
day_element.append(np.concatenate(node_element).tolist())
matrix.extend(day_element)
#print(matrix)
return matrix
def depo_nodes(num_days, num_verhicles, num_nodes):
depo_array = []
for day in range(num_days):
for veh in range(int(num_verhicles/num_days)):
depo_array.append(day*num_nodes)
#print(depo_array)
return depo_array
def veh_types_costs(cost_array, num_per_type, num_days):
cost_array_km = []
for day in range(num_days):
for i in range(len(cost_array)):
for num in range(num_per_type[i]):
cost_array_km.append(cost_array[i])
#print(cost_array_km)
return(cost_array_km)
def veh_capacity_matrix(num_vehicle_per_type, vehicle_capacity_type, num_days):
matrix= []
for index in range(num_days):
matrix_element = []
for day in range(num_days):
for i, num in enumerate(num_vehicle_per_type):
for times in range(num):
if day == index:
matrix_element.append(vehicle_capacity_type[i]);
else:
matrix_element.append(0)
matrix.append(matrix_element)
#print(matrix)
return matrix
def dist_matrix(koordianten):
matrix =[]
for k_1 in koordianten:
matrix_element = []
for k_2 in koordianten:
matrix_element.append(np.linalg.norm(k_1-k_2))
matrix.append(matrix_element)
#print(matrix)
return matrix
def demand_matrix(demand_matrix_day,num_days):
matrix = []
for index in range(num_days):
matrix_element = []
for day in range(num_days):
if day == index:
matrix_element = matrix_element +demand_matrix_day
else:
matrix_element = matrix_element +(list(np.zeros(len(demand_matrix_day))))
matrix.append(matrix_element)
return matrix
def kunden_indizes(ID_list, num_days):
indizes = {}
for ID in ID_list:
indizes[ID] = []
for index, key in enumerate(indizes):
for i in range(len(ID_list)*num_days):
if (i % len(ID_list)) == index:
indizes[key].append(i)
#print(indizes)
return indizes
def create_data_model(node_num):
data = {}
data["num_days"] = 5 #Anzahl Tage
###Kundenset
num_IDs = node_num
base = [0]
data["IDs"] = list(range(0, num_IDs))
#data["IDs"] = [0, 1, 2, 3, 4]
print(data["IDs"])
data["demand"] = [random.randint(1,30) for i in range(0,num_IDs)]
#data["demand"] =[0, 16, 8, 1, 7]
data["demand"][0] = 0
print("demand", data["demand"])
data["frequenz"] = [random.randint(1,5) for i in range(0,num_IDs)]
#data["frequenz"] =[0, 3, 5, 2, 1]
data["frequenz"][0] = 0
print("freq ",data["frequenz"])
summe_dem =0
for index, demand in enumerate(data["demand"]):
summe_dem += data["demand"][index] * data["frequenz"][index]
print("DEMANDSUMME: ",summe_dem)
data["koordianten"] = np.random.rand(num_IDs, 2)*1000
#data["koordianten"] = np.array([(0, 0), (4, 0), (4, 4), (0, 4), (3, 3)])*100
data["koordianten"][0] = (0,0)
#print("koord ", data["koordianten"])
data["PAT"] = {
5:
[[1, 1, 1, 1, 1]],
4:
[[0, 1, 1, 1, 1],
[1, 0, 1, 1, 1],
[1, 1, 0, 1, 1],
[1, 1, 1, 0, 1],
[1, 1, 1, 1, 0]],
3: [[0, 1, 0, 1, 1],
[1, 0, 1, 0, 1],
[1, 1, 0, 1, 0],
[0, 1, 1, 0, 1],
[1, 0, 1, 1, 0],
],
2: [[1, 0, 1, 0, 0],
[0, 1, 0, 1, 0],
[0, 0, 1, 0, 1],
[1, 0, 0, 1, 0],
[0, 1, 0, 0, 1]],
1: [[1, 0, 0, 0, 0],
[0, 1, 0, 0, 0],
[0, 0, 1, 0, 0],
[0, 0, 0, 1, 0],
[0, 0, 0, 0, 1]],
}
### Definition der Fahrezugtypen
data["vehicle_costs_type_per_km"] = [1, 0.01] #Kosten pro km pro Fahrzeugtyp
data["vehicle_costs_type_per_stop"] = [1, 1000] #Kosten pro Stop pro Fahrzeugtyp
data["vehicle_capacity_type"] = [100, 200] # Kapaität pro Fahrzeugtyp
data["num_vehicle_per_type"] = [math.ceil(summe_dem /data["vehicle_capacity_type"][0]), math.ceil(summe_dem /data["vehicle_capacity_type"][1])] # Anzahl Fahrzeuge pro Typ
data['price_per_km'] = veh_types_costs(data["vehicle_costs_type_per_km"], data["num_vehicle_per_type"], data["num_days"]) #Matrix für price_per_km je Fahrzeug ertsellen
data["price_per_stop"] = veh_types_costs(data["vehicle_costs_type_per_stop"], data["num_vehicle_per_type"], data["num_days"])
data['vehicle_capacities_matrix'] = veh_capacity_matrix(data["num_vehicle_per_type"], data["vehicle_capacity_type"], data["num_days"]) #Kapazitäten pro Fahrzeugs pro Tag
#print('vehicle_capacities_matrix')
#print(data['vehicle_capacities_matrix'])
data["num_vehicles_per_day"] = sum(data["num_vehicle_per_type"]) #Gesamtanzahl der Fahrzeuge pro Tag
data['num_vehicles'] = data["num_days"] * data["num_vehicles_per_day"]# Gesamtanzahl der Fahrzeuge in gesamten Zeitraum
###Distanzmatrix bestimmen
data["base_distance_matrix"] = dist_matrix(data["koordianten"])
data['distance_matrix'] = dublicate_nodes(data["base_distance_matrix"], data["num_days"]) # Distanzmatrix mit mehrfachen Nodes pro Kunden, je Tag ein Node
###Start und Ende festlegen
data["num_nodes"] = len(data["base_distance_matrix"]) # Anzahl Kunden
data['starts'] = depo_nodes(data["num_days"], data['num_vehicles'], data["num_nodes"]) #Deponodes (Start) für die einzelnen Fahrzeuge (Tage)
data['ends'] = depo_nodes(data["num_days"], data['num_vehicles'], data["num_nodes"]) #Deponodes (Ende) für die einzelnen Fahrzeuge (Tage)
###Demand pro Kunde
data["kunden_indizes"] = kunden_indizes(data["IDs"], data["num_days"])
data["demands_matrix"] = demand_matrix(demand_matrix_day=data["demand"], num_days=data["num_days"])
#print(data["demands_matrix"])
return data
def print_solution(data, manager, routing, solution):
"""Prints solution on console."""
total_distance = 0
total_load = 0
for vehicle_id in range(data['num_vehicles']):
if vehicle_id % data["num_vehicles_per_day"] == 0:
print(("------Tag {}------").format(int(vehicle_id/data["num_vehicles_per_day"])))
if routing.IsVehicleUsed(assignment= solution, vehicle=vehicle_id) == False:
continue
index = routing.Start(vehicle_id)
if vehicle_id >= data["num_vehicles_per_day"]:
plan_output = 'Route for vehicle {}:\n'.format(abs(vehicle_id-data["num_vehicles_per_day"]*int(vehicle_id/data["num_vehicles_per_day"])))
else:
plan_output = 'Route for vehicle {}:\n'.format(vehicle_id)
route_costs = 0
route_load = 0
while not routing.IsEnd(index):
node_index = manager.IndexToNode(index)
capacity_ID = int(vehicle_id/data["num_vehicles_per_day"])
route_load += data['demands_matrix'][capacity_ID][node_index]
plan_output += ' {0} Load({1}) -> '.format(node_index, route_load)
previous_index = index
index = solution.Value(routing.NextVar(index))
route_costs += routing.GetArcCostForVehicle(
previous_index, index, vehicle_id)
plan_output += ' {0} Load({1})\n'.format(manager.IndexToNode(index),
route_load)
plan_output += 'Costs of the route: {}€\n'.format(route_costs)
plan_output += 'Load of the route: {}\n'.format(route_load)
print(plan_output)
total_distance += route_costs
total_load += route_load
print('Total costs of all routes: {}€'.format(total_distance))
print('Total load of all routes: {}'.format(total_load))
def main(node_num):
"""Periodic CVRP problem."""
# Instantiate the data problem.
data = create_data_model(node_num)
# Create the routing index manager.
manager = pywrapcp.RoutingIndexManager(len(data['distance_matrix']), data['num_vehicles'], data['starts'],
data['ends'])
# Create Routing Model.
routing = pywrapcp.RoutingModel(manager)
### Kostenfunktion je Fahrzeug festlegen ###
def create_cost_callback(dist_matrix, km_costs, stop_costs):
# Create a callback to calculate distances between cities.
def distance_callback(from_index, to_index):
from_node = manager.IndexToNode(from_index)
to_node = manager.IndexToNode(to_index)
return int(dist_matrix[from_node][to_node]) * (km_costs) + (stop_costs)
return distance_callback
for i in range(data['num_vehicles']):
cost_callback = create_cost_callback(data['distance_matrix'], data["price_per_km"][i],
data["price_per_stop"][i]) # Callbackfunktion erstellen
cost_callback_index = routing.RegisterTransitCallback(cost_callback) # registrieren
routing.SetArcCostEvaluatorOfVehicle(cost_callback_index, i) # Vehicle zuordnen
#Define Capacities for Vehicles for every Day
def create_demand_callback(demand_matrix, demand_index):
# Create a callback to calculate capacity usage.
def demand_callback(from_index):
#Returns the demand of the node.
# Convert from routing variable Index to demands NodeIndex.
from_node = manager.IndexToNode(from_index)
return demand_matrix[demand_index][from_node]
return demand_callback
for i in range(data["num_days"]): #Jedes Fahrzeug hat pro Tag eine andere Kapazität
demand_callback = create_demand_callback(data['demands_matrix'], i)
demand_callback_index = routing.RegisterUnaryTransitCallback(demand_callback)
dimension_name = 'Capacity_day_{}'.format(i)
routing.AddDimensionWithVehicleCapacity(
demand_callback_index,
0, # null capacity slack
data['vehicle_capacities_matrix'][i], # vehicle maximum capacities
True, # start cumul to zero
dimension_name)
#Drop visits that would exceed the frequency
for index, key in enumerate(data["kunden_indizes"]):
if index == 0:
continue
#routing.solver().Add(sum(routing.ActiveVar(manager.NodeToIndex(i)) for i in data["kunden_indizes"][key]) == data["frequenz"][index])
bool_array = []
for index, freq in enumerate(data["frequenz"]):
if index == 0:
continue
bool_array_part =[]
for index_pat, pat in enumerate(data["PAT"][freq]):
#print(index,freq,index_pat)
bool_name = str(index) +str(freq) + str(index_pat)
bool_array_part.append(routing.solver().BoolVar(bool_name))
bool_array.append(bool_array_part)
#print(bool_array)
for i in bool_array:
routing.solver().Add(sum(i) == 1)
node_array = []
for index, freq in enumerate(data["frequenz"]):
if index == 0:
continue
node_array_part = []
for index_pat, pat in enumerate(data["PAT"][freq]):
node_array_sub_part = []
for index_day, day_value in enumerate(pat):
if day_value == 1:
node_array_sub_part.append(data["kunden_indizes"][index][index_day])
#print(node_array_part)
node_array_part.append(node_array_sub_part)
node_array.append(node_array_part)
#print(node_array)
for index, bool in enumerate(bool_array):
for i in range(len(bool)):
#print(node_array[index][i])
#print(bool_array[index][i])
routing.solver().Add(sum(routing.ActiveVar(manager.NodeToIndex(i)) for i in node_array[index][i]) * bool_array[index][i] == bool_array[index][i] * len(node_array[index][i]))
#routing.solver().Add(routing.ActiveVar(manager.NodeToIndex(5)) == 1)
penalty = 0
for node in range(0, len(data['distance_matrix'])):
if node in data["kunden_indizes"][0]:
continue
else:
routing.AddDisjunction([manager.NodeToIndex(node)], penalty)
# Setting first solution heuristic.
search_parameters = pywrapcp.DefaultRoutingSearchParameters()
search_parameters.first_solution_strategy = (
routing_enums_pb2.FirstSolutionStrategy.AUTOMATIC)
search_parameters.local_search_metaheuristic = (
routing_enums_pb2.LocalSearchMetaheuristic.AUTOMATIC)
#search_parameters.time_limit.FromSeconds(300)
# Solve the problem.
solution = routing.SolveWithParameters(search_parameters)
# Print solution on console.
if solution:
print_solution(data, manager, routing, solution)
print()
print("Statistics")
print(' - wall time : %f s' % (int(routing.solver().WallTime())/1000))
solvingtime = int(routing.solver().WallTime())/1000
return solvingtime
if __name__ == '__main__':
node_num_array = [10,20,30,40,50,60,70,80,90,100]
solvingtime_array = []
num_array =[]
for i in node_num_array:
for j in range(0,4):
solvingtime = main(i)
solvingtime_array.append(solvingtime)
num_array.append(i)
print(i, "-->",solvingtime)
print(solvingtime_array)
print(num_array)
df_results = pd.DataFrame(data= {"num_nodes":num_array, "solvingtime":solvingtime_array})
#encoding="latin_1", sep=";")
print(df_results)

Count values from rowlist divided by key in nested dict

Here is an example dataset
Firstly, I try to create a dict from values in rows:
import csv
who = set()
figure = set()
date = set()
action = []
activity = {'play': 0, 'throw': 0, 'pin': 0, 'tap': 0}
with open(r'ShtrudelT.csv',
mode = 'r') as csv_file:
lines = csv_file.readlines()
for row in lines:
data = row.split(',')
who.add(data[1])
figure.add(data[2])
date.add(data[3][:7])
action.append(data[4].strip())
xdict = dict.fromkeys(who,
dict.fromkeys(figure,
dict.fromkeys(date, activity)))
The result is:
{'Googenhaim': {'Circle': {'2020-04': {'play': 0,'throw': 0, 'pin': 0, 'tap': 0},
'2020-06': {'play': 0, 'throw': 0, 'pin': 0, 'tap': 0},
'2020-05': {'play': 0, 'throw': 0, 'pin': 0, 'tap': 0}},
'Rectangle': {'2020-04': {'play': 0, 'throw': 0, 'pin': 0, 'tap': 0}...}
Secondly, I need to count actions divided by key to analyze data. For example, how many times Googenhaim use Circle by any type of action in every month.
Is there a solution without using Pandas?

import csv
count_dict = {}
with open(r'ShtrudelT.csv',
mode = 'r') as csv_file:
lines = csv_file.readlines()
for row in lines:
data = row.split(',')
key = data[1] + "_" + data[2] + "_" + data[3][:7] + "_" + data[4].strip()
if key in count_dict:
count_dict[key] += 1
else:
count_dict[key] = 1
print("\t".join(["Name", "Shape", "Month", "Action", "Count"]))
for element, count in count_dict.items():
items = element.split("_")
print("\t".join(items) + "\t" + str(count))
We use a dictionary where every key is the combination that we want to count. This combination is formed from name of the user, shape, month and the action. While processing every line, we form the key and store it in the dictionary. If it is encountered for the first time then we insert it or else we update the count.
After all the lines are processed, we can do any kind of post processing we want to do.
Hope that solves it.

How to add the value from list of tuples

I am extracting from the log file and print using the below code
for line in data:
g = re.findall(r'([\d.]+).*?(GET|POST|PUT|DELETE)', line)
print (g)
[('1.1.1.1', 'PUT')]
[('2.2.2.2', 'GET')]
[('1.1.1.1', 'PUT')]
[('2.2.2.2', 'POST')]
How to add to the output
output
1.1.1.1: PUT = 2
2.2.2.2: GET = 1,POST=1

You could use a dictionary to count:
# initialize the count dict
count_dict= dict()
for line in data:
g = re.findall(r'([\d.]+).*?(GET|POST|PUT|DELETE)', line)
for tup in g:
# get the counts for tuple tup if we don't have it yet
# use 0 (second argument to .get)
num= count_dict.get(tup, 0)
# increase the count and write it back
count_dict[tup]= num+1
# now iterate over the key (tuple) - value (counts)-pairs
# and print the result
for tup, count in count_dict.items():
print(tup, count)
Ok, I have to admit this doesn't give the exact output, you want, but from this you can do in a similar manner:
out_dict= dict()
for (comma_string, request_type), count in count_dict.items():
out_str= out_dict.get(comma_string, '')
sep='' if out_str == '' else ', '
out_str= f'{out_str}{sep}{request_type} = {count}'
out_dict[comma_string]= out_str
for tup, out_str in out_dict.items():
print(tup, out_str)
From your data that outputs:
1.1.1.1 PUT = 2
2.2.2.2 GET = 1, POST = 1

I would look towards Counter.
from collections import Counter
results = []
for line in data:
g = re.findall(r'([\d.]+).*?(GET|POST|PUT|DELETE)', line)
results.append(g[0])
ip_list = set(result[0] for result in results)
for ip in ip_list:
print(ip, Counter(result[1] for result in results if result[0] == ip ))

You can use collection.defaultdict
Ex:
from collections import defaultdict
result = defaultdict(list)
for line in data:
for ip, method in re.findall(r'([\d.]+).*?(GET|POST|PUT|DELETE)', line):
result[ip].append(method)
for k, v in result.items():
temp = ""
for i in set(v):
temp += " {} = {}".format(i, v.count(i))
print("{}{}".format(k, temp))

from collections import Counter
x = [[('1.1.1.1', 'PUT')],[('2.2.2.2', 'GET')],[('1.1.1.1', 'PUT')],[('2.2.2.2', 'POST')]]
# step 1: convert x into a dict.
m = {}
for i in x:
a, b = i[0]
if a not in m.keys():
m[a] = [b]
else:
x = m[a]
x.append(b)
m[a] = x
print('new dict is {}'.format(m))
# step 2 count frequency
m_values = list(m.values())
yy = []
for i in m_values:
x = []
k = list(Counter(i).keys())
v = list(Counter(i).values())
for i in range(len(k)):
x.append(k[i] + '=' + str(v[i]))
yy.append(x)
# step 3, update the value of the dict
m_keys = list(m.keys())
n = len(m_keys)
for i in range(n):
m[m_keys[i]] = yy[i]
print("final dict is{}".format(m))
Output is
new dict is {'1.1.1.1': ['PUT', 'PUT'], '2.2.2.2': ['GET', 'POST']}
final dict is{'1.1.1.1': ['PUT=2'], '2.2.2.2': ['GET=1', 'POST=1']}

Without dependencies and using a dict for counting, in a very basic way. Given the data_set:
data_set = [[('1.1.1.1', 'PUT')],
[('2.2.2.2', 'GET')],
[('2.2.2.2', 'POST')],
[('1.1.1.1', 'PUT')]]
Initialize the variables (manually, just few verbs) then iterate over the data:
counter = {'PUT': 0, 'GET': 0, 'POST': 0, 'DELETE': 0}
res = {}
for data in data_set:
ip, verb = data[0]
if not ip in res:
res[ip] = counter
else:
res[ip][verb] += 1
print(res)
#=> {'1.1.1.1': {'PUT': 1, 'GET': 0, 'POST': 1, 'DELETE': 0}, '2.2.2.2': {'PUT': 1, 'GET': 0, 'POST': 1, 'DELETE': 0}}
It's required to format the output to better fits your needs.

OverflowError mktime argument out of range

After solving a naive datetime problem I am facing a new problem on a view to generate graphs. Now I get mktime argument out of range.
I have no idea how to solve it. I didn't write the code, I am using it from a colleague of mine and I can't seem o understand why it fails. I think it has to do with a function that runs overtime and the error pops out.
#login_required(login_url='/accounts/login/')
def loggedin(request):
data = []
data2 = []
data3 = []
dicdata2 = {}
dicdata3 = {}
datainterior = []
today = timezone.localtime(timezone.now()+timedelta(hours=1)).date()
tomorrow = today + timedelta(1)
semana= today - timedelta(7)
today = today - timedelta(1)
semana_start = datetime.combine(today, time())
semana_start = timezone.make_aware(semana_start, timezone.utc)
today_start = datetime.combine(today, time())
today_start = timezone.make_aware(today_start, timezone.utc)
today_end = datetime.combine(tomorrow, time())
today_end = timezone.make_aware(today_end, timezone.utc)
for modulo in Repository.objects.values("des_especialidade").distinct():
dic = {}
mod = str(modulo['des_especialidade'])
dic["label"] = str(mod)
dic["value"] = Repository.objects.filter(des_especialidade__iexact=mod).count()
data.append(dic)
for modulo in Repository.objects.values("modulo").distinct():
dic = {}
mod = str(modulo['modulo'])
dic["label"] = str(mod)
dic["value"] = Repository.objects.filter(modulo__iexact=mod, dt_diag__gte=semana_start).count()
datainterior.append(dic)
# print mod, Repository.objects.filter(modulo__iexact=mod).count()
# data[mod] = Repository.objects.filter(modulo__iexact=mod).count()
dicdata2['values'] = datainterior
dicdata2['key'] = "Cumulative Return"
dicdata3['values'] = data
dicdata3['color'] = "#d67777"
dicdata3['key'] = "Diagnosticos Identificados"
data3.append(dicdata3)
data2.append(dicdata2)
#-------sunburst
databurst = []
dictburst = {}
dictburst['name'] = "CHP"
childrenmodulo = []
for modulo in Repository.objects.values("modulo").distinct():
childrenmodulodic = {}
mod = str(modulo['modulo'])
childrenmodulodic['name'] = mod
childrenesp = []
for especialidade in Repository.objects.filter(modulo__iexact=mod).values("des_especialidade").distinct():
childrenespdic = {}
esp = str(especialidade['des_especialidade'])
childrenespdic['name'] = esp
childrencode = []
for code in Repository.objects.filter(modulo__iexact=mod,des_especialidade__iexact=esp).values("cod_diagnosis").distinct():
childrencodedic = {}
codee= str(code['cod_diagnosis'])
childrencodedic['name'] = 'ICD9 - '+codee
childrencodedic['size'] = Repository.objects.filter(modulo__iexact=mod,des_especialidade__iexact=esp,cod_diagnosis__iexact=codee).count()
childrencode.append(childrencodedic)
childrenespdic['children'] = childrencode
#childrenespdic['size'] = Repository.objects.filter(des_especialidade__iexact=esp).count()
childrenesp.append(childrenespdic)
childrenmodulodic['children'] = childrenesp
childrenmodulo.append(childrenmodulodic)
dictburst['children'] = childrenmodulo
databurst.append(dictburst)
# print databurst
# --------stacked area chart
datastack = []
for modulo in Repository.objects.values("modulo").distinct():
datastackdic = {}
mod = str(modulo['modulo'])
datastackdic['key'] = mod
monthsarray = []
year = timezone.localtime(timezone.now()+timedelta(hours=1)).year
month = timezone.localtime(timezone.now()+timedelta(hours=1)).month
last = timezone.localtime(timezone.now()+timedelta(hours=1)) - relativedelta(years=1)
lastyear = int(last.year)
lastmonth = int(last.month)
#i = 1
while lastmonth <= int(month) or lastyear<int(year):
date = str(lastmonth) + '/' + str(lastyear)
if (lastmonth < 12):
datef = str(lastmonth + 1) + '/' + str(lastyear)
else:
lastmonth = 01
lastyear = int(lastyear)+1
datef = str(lastmonth)+'/'+ str(lastyear)
lastmonth = 0
datainicial = datetime.strptime(date, '%m/%Y')
datainicial = timezone.make_aware(datainicial, timezone.utc)
datafinal = datetime.strptime(datef, '%m/%Y')
datafinal = timezone.make_aware(datafinal, timezone.utc)
#print "lastmonth",lastmonth,"lastyear", lastyear
#print "datainicial:",datainicial,"datafinal: ",datafinal
filtro = Repository.objects.filter(modulo__iexact=mod)
count = filtro.filter(dt_diag__gte=datainicial, dt_diag__lt=datafinal).count()
conv = datetime.strptime(date, '%m/%Y')
ms = datetime_to_ms_str(conv)
monthsarray.append([ms, count])
#i += 1
lastmonth += 1
datastackdic['values'] = monthsarray
datastack.append(datastackdic)
#print datastack
if request.user.last_login is not None:
#print(request.user.last_login)
contador_novas = Repository.objects.filter(dt_diag__lte=today_end, dt_diag__gte=today_start).count()
return render_to_response('loggedin.html',
{'user': request.user.username, 'contador': contador_novas, 'data': data, 'data2': data2,
'data3': data3,
'databurst': databurst, 'datastack':datastack})
def datetime_to_ms_str(dt):
return str(1000 * mktime(dt.timetuple()))

I think the problem is with this condition.
while lastmonth <= int(month) or lastyear<int(year):
During December, month=12, so lastmonth <= int(month) will always be True. So the loop whill always return True, even once lastyear is more that the current year.
You want to loop if the loop is in the previous year, or if the loop is in the current year and the month is not in the future. Therefore, I think you want to change it to the following:
while lastyear < year or (lastyear == year and lastmonth <= month):
To be sure that the code is working and to understand it, you need to add lots of print statements to the loops, see how lastmonth and lastyear change, and check that the loop exits when you expect it to. You also need to test it for other values of year and month so that it doesn't break next month. Ideally you want to extract this bit of the code into a separate function. It would be easier to understand the loop if it only returned a list of (month, year) integers, instead of doing lots of date formatting at the same time. Then it would be easier to add unit tests.