Python Minizinc gives sometimes a different result than Minizinc IDE - python

Depending on the input data I sometimes obtain a different result (different objective and solution) when using the Minizinc Python library or the Minizinc IDE.
My Python code is the following:
from minizinc import Instance, Model, Solver
import datetime
gecode = Solver.lookup("gecode")
model = Model()
model.add_file("./model.mzn")
model.add_file("./data.dzn", parse_data=True)
instance = Instance(gecode, model)
TIMELIMIT = datetime.timedelta(seconds=60)
result = instance.solve(timeout=TIMELIMIT)
print(result.status)
result.solution
As far as I can tell, my solver parameters are the same in the IDE, I use the same model and same data.
I can see the following command in the IDE:
Command: minizinc --json-stream --param-file-no-push /private/var/folders/hf/_75w1jh170x75y2_2vwt60lc0000gn/T/mzn_KneEzb.mpc model.mzn data.dzn
Configuration:
{
"intermediate-solutions": true,
"solver": "org.gecode.gecode#6.3.0",
"time-limit": 60000
}
In one specific input data instance, IDE finds an optimal at obj = 80, and Python finds obj = 104.
In another instance of input data, both IDE and Python find the same obj.
What can explain this different result?
Update: I am adding the MiniZinc model and data:
model.mzn
int: nNodes = nC+2*nV;
% ============== Vehicle Routing Problem ============= %
include "globals.mzn";
int: nC; int: nV; int: timeBudget;
set of int: VEHICLE = 1..nV;
set of int: CUSTOMER = 1..nC;
set of int: NODES = 1..nNodes;
set of int: START_DEPOT_NODES = nC+1..nC+nV;
set of int: END_DEPOT_NODES = nC+nV+1..nC+2*nV;
set of int: TIME = 0..timeBudget;
array[NODES] of int: serviceTime;
array[NODES, NODES] of int: distance;
% Time windows
% departure time must be between TimeWindows[client_i, 1] and TimeWindows[client_i, 2]
array[1..nC, 1..2] of int: TimeWindows;
% -------- variables ------------- %
array[NODES] of var VEHICLE: vehicle;
array[NODES] of var NODES: successor;
array[NODES] of var TIME: arrivalTime;
% -------- Start Nodes, End Nodes constraints ---------- %
constraint forall (n in START_DEPOT_NODES) % associate each start
( vehicle[n] = n-nC ); % node with a vehicle
constraint forall (n in END_DEPOT_NODES) % associate each end
( vehicle[n] = n-nC-nV ); % node with a vehicle
% -------- Successor Nodes constraints ---------- %
constraint forall (n in nC+nV+1..nC+2*nV-1) % successors of end nodes
( successor[n] = n-nV+1 ); % are start nodes
constraint successor[nC+2*nV] = nC+1;
constraint forall (n in START_DEPOT_NODES) % vehicles leave the
( arrivalTime[n] = 0 ); % depot at time zero
constraint circuit(successor); % hamiltonian circuit
constraint forall (n in CUSTOMER) % use the same vehicle
( vehicle[successor[n]] = vehicle[n] ); % along a subtour
constraint forall (n in 1..nC+nV)
( arrivalTime[n] + serviceTime[n] + distance[n,successor[n]]
<= arrivalTime[successor[n]] ); % time constraints
% -------- Time windows constraints -------------- %
constraint
forall(i in 1..nC)(
TimeWindows[i, 1] <= arrivalTime[i]
);
constraint
forall(i in 1..nC)(
TimeWindows[i, 2] >= arrivalTime[i]
);
% -------- objective ------------ %
% minimize distance (could be travel time in hours)
var int: obj = (sum (n in NODES) (distance[n,successor[n]]));
solve minimize obj; % expected overall travel time of each vehicle
% -------- output ------------ %
output ["obj = \(obj) \n"];
data.dzn
nV =5; nC = 40; timeBudget = 5000;
serviceTime = [ 6, 7, 3, 10, 2, 5, 2, 9, 7, 6, 5, 7, 7, 9, 9, 3, 7, 9, 1, 2, 5, 4, 7, 6, 3, 4, 6, 6, 2, 1, 10, 1, 5, 4, 10, 2, 10, 9, 2, 10, 5, 6, 4, 9, 9, 10, 6, 6, 6, 5];
distance = [| 0, 3, 7, 2, 10, 1, 6, 1, 10, 7, 3, 9, 5, 10, 5, 6, 2, 4, 3, 6, 5, 4, 4, 8, 7, 8, 8, 7, 8, 8, 9, 5, 2, 2, 9, 1, 2, 3, 1, 6, 6, 1, 1, 5, 3, 1, 5, 1, 4, 9
|5, 0, 7, 7, 6, 9, 1, 2, 2, 7, 10, 1, 9, 7, 5, 9, 4, 9, 8, 8, 4, 2, 10, 6, 1, 2, 7, 7, 4, 6, 5, 7, 9, 1, 10, 8, 6, 9, 1, 8, 6, 10, 7, 5, 7, 4, 10, 6, 8, 2
|7, 7, 0, 8, 10, 10, 6, 5, 9, 7, 5, 3, 2, 8, 10, 6, 10, 9, 8, 2, 1, 9, 10, 7, 4, 6, 1, 5, 8, 8, 9, 1, 2, 2, 6, 3, 2, 7, 1, 3, 6, 9, 7, 3, 6, 6, 6, 3, 3, 8
|7, 3, 6, 0, 10, 5, 8, 2, 5, 7, 1, 10, 8, 2, 5, 6, 9, 7, 1, 2, 1, 3, 3, 10, 10, 3, 9, 7, 2, 3, 4, 10, 10, 7, 6, 4, 9, 9, 8, 4, 1, 5, 6, 3, 10, 4, 5, 9, 3, 7
|5, 4, 9, 4, 0, 9, 8, 1, 1, 4, 9, 2, 10, 4, 4, 9, 7, 6, 8, 10, 5, 5, 5, 7, 8, 10, 10, 7, 3, 1, 6, 2, 4, 6, 5, 3, 10, 1, 2, 7, 3, 1, 2, 8, 1, 5, 10, 1, 5, 5
|2, 4, 4, 4, 7, 0, 8, 9, 3, 2, 8, 7, 7, 7, 5, 9, 9, 8, 3, 2, 3, 8, 3, 2, 8, 4, 1, 6, 4, 3, 2, 3, 6, 4, 2, 10, 6, 5, 5, 5, 10, 6, 4, 10, 10, 10, 8, 3, 2, 4
|5, 3, 6, 3, 7, 1, 0, 9, 10, 5, 4, 10, 4, 8, 9, 8, 6, 4, 1, 9, 2, 1, 4, 4, 6, 6, 9, 1, 9, 1, 7, 2, 1, 5, 4, 3, 6, 4, 4, 10, 4, 10, 7, 10, 4, 7, 8, 1, 3, 4
|9, 1, 5, 4, 6, 5, 10, 0, 4, 8, 10, 7, 5, 10, 7, 5, 4, 4, 10, 7, 8, 6, 2, 7, 9, 6, 2, 3, 9, 8, 3, 9, 4, 6, 5, 1, 4, 6, 7, 6, 9, 1, 9, 5, 5, 5, 5, 6, 7, 1
|5, 6, 4, 2, 9, 1, 5, 4, 0, 3, 3, 3, 6, 1, 4, 10, 1, 10, 10, 4, 8, 4, 7, 2, 9, 4, 4, 4, 4, 10, 8, 9, 9, 7, 9, 3, 9, 9, 3, 7, 1, 9, 2, 1, 3, 9, 8, 4, 1, 8
|10, 5, 8, 5, 7, 2, 1, 1, 5, 0, 7, 2, 5, 10, 5, 7, 1, 7, 7, 8, 3, 1, 3, 3, 9, 10, 1, 7, 1, 3, 4, 1, 8, 8, 4, 4, 5, 3, 3, 2, 3, 9, 4, 2, 3, 8, 5, 7, 9, 9
|9, 5, 8, 1, 2, 3, 1, 5, 10, 4, 0, 8, 10, 9, 3, 5, 1, 2, 10, 4, 9, 2, 1, 1, 10, 1, 4, 7, 7, 8, 10, 2, 6, 7, 8, 7, 4, 3, 5, 7, 7, 8, 10, 2, 3, 4, 7, 6, 6, 7
|8, 10, 3, 2, 9, 4, 2, 3, 7, 4, 8, 0, 9, 5, 1, 6, 7, 7, 9, 5, 4, 2, 3, 8, 7, 1, 2, 10, 9, 5, 4, 4, 4, 9, 3, 4, 9, 9, 6, 9, 6, 10, 9, 3, 8, 3, 9, 3, 6, 2
|1, 1, 6, 3, 6, 1, 9, 3, 9, 2, 6, 1, 0, 5, 10, 5, 1, 7, 7, 5, 3, 1, 7, 3, 7, 3, 5, 10, 1, 8, 3, 6, 2, 6, 2, 1, 2, 5, 8, 8, 10, 4, 6, 1, 10, 2, 3, 3, 1, 1
|3, 5, 3, 4, 7, 7, 10, 9, 5, 3, 5, 3, 8, 0, 5, 6, 5, 9, 8, 8, 4, 1, 6, 8, 5, 9, 7, 5, 7, 5, 5, 1, 3, 6, 7, 9, 9, 10, 2, 3, 2, 6, 8, 9, 10, 3, 2, 1, 4, 5
|6, 5, 6, 8, 6, 5, 2, 9, 5, 3, 10, 8, 4, 6, 0, 10, 2, 2, 3, 3, 7, 5, 5, 9, 4, 2, 10, 10, 4, 4, 2, 3, 4, 8, 10, 6, 1, 1, 1, 9, 8, 1, 9, 5, 6, 9, 7, 5, 7, 8
|6, 9, 8, 1, 8, 10, 7, 1, 4, 2, 8, 1, 4, 3, 2, 0, 3, 7, 4, 9, 5, 2, 10, 2, 8, 9, 3, 3, 2, 8, 9, 4, 1, 10, 2, 10, 9, 6, 10, 2, 9, 9, 9, 2, 4, 8, 7, 9, 9, 5
|8, 3, 4, 5, 5, 5, 4, 6, 6, 2, 6, 6, 5, 3, 4, 6, 0, 7, 1, 7, 1, 8, 5, 10, 8, 8, 9, 6, 10, 3, 10, 6, 4, 6, 8, 9, 9, 9, 1, 7, 3, 5, 4, 7, 3, 7, 2, 10, 10, 7
|10, 7, 1, 4, 7, 6, 2, 5, 1, 9, 7, 9, 6, 9, 6, 2, 3, 0, 5, 4, 3, 4, 2, 5, 5, 1, 1, 1, 1, 10, 4, 5, 6, 2, 2, 7, 10, 9, 4, 3, 4, 3, 9, 2, 6, 10, 9, 6, 4, 2
|2, 10, 10, 8, 5, 4, 4, 2, 1, 7, 7, 9, 2, 2, 7, 1, 7, 4, 0, 9, 5, 3, 10, 4, 10, 9, 7, 3, 4, 1, 8, 6, 1, 8, 2, 6, 1, 1, 6, 7, 3, 3, 10, 1, 10, 5, 3, 1, 5, 9
|8, 4, 2, 6, 5, 5, 6, 2, 1, 3, 10, 10, 10, 4, 3, 3, 1, 4, 6, 0, 6, 4, 3, 10, 4, 5, 3, 10, 1, 6, 6, 8, 4, 8, 4, 7, 4, 5, 9, 1, 3, 6, 1, 5, 10, 3, 10, 7, 7, 5
|3, 9, 8, 5, 2, 7, 9, 9, 10, 4, 9, 10, 10, 2, 8, 2, 9, 2, 2, 8, 0, 2, 5, 3, 9, 10, 9, 8, 2, 2, 5, 4, 7, 6, 5, 7, 6, 1, 4, 10, 6, 4, 4, 8, 4, 5, 4, 3, 10, 5
|4, 2, 5, 7, 7, 5, 9, 8, 2, 9, 5, 6, 5, 4, 4, 1, 1, 5, 8, 8, 4, 0, 2, 10, 8, 6, 8, 8, 3, 8, 2, 4, 6, 7, 1, 4, 1, 1, 6, 3, 3, 10, 9, 7, 1, 8, 10, 4, 1, 7
|1, 10, 1, 5, 4, 8, 8, 5, 5, 9, 7, 9, 1, 8, 4, 7, 8, 3, 5, 6, 2, 4, 0, 1, 10, 3, 10, 7, 8, 4, 10, 4, 10, 1, 10, 5, 1, 9, 8, 5, 6, 7, 6, 7, 9, 10, 9, 5, 8, 2
|7, 1, 8, 8, 9, 4, 9, 10, 1, 5, 6, 8, 8, 10, 9, 7, 10, 5, 3, 4, 10, 6, 5, 0, 6, 2, 6, 8, 2, 5, 5, 3, 4, 7, 6, 6, 2, 1, 1, 7, 8, 7, 9, 3, 6, 5, 2, 10, 2, 4
|3, 6, 6, 8, 9, 8, 4, 9, 2, 4, 3, 5, 8, 4, 1, 4, 9, 9, 3, 8, 2, 5, 10, 2, 0, 4, 5, 8, 5, 7, 5, 4, 4, 5, 1, 5, 2, 3, 2, 8, 4, 6, 9, 3, 8, 4, 1, 7, 5, 8
|9, 2, 7, 6, 1, 4, 9, 2, 5, 7, 2, 5, 2, 5, 5, 3, 6, 4, 8, 4, 8, 6, 10, 1, 6, 0, 6, 4, 3, 3, 4, 3, 7, 3, 3, 7, 8, 10, 7, 1, 2, 5, 4, 5, 8, 4, 7, 7, 2, 9
|2, 10, 9, 3, 1, 9, 7, 10, 8, 5, 10, 8, 4, 6, 9, 2, 3, 9, 4, 10, 9, 8, 7, 4, 6, 8, 0, 8, 7, 2, 2, 9, 7, 6, 9, 4, 3, 8, 4, 2, 8, 1, 1, 9, 4, 10, 2, 8, 8, 2
|1, 4, 4, 8, 8, 8, 3, 6, 7, 8, 1, 2, 10, 2, 2, 5, 4, 4, 8, 8, 2, 8, 5, 5, 2, 9, 9, 0, 4, 6, 10, 8, 3, 5, 2, 8, 4, 7, 2, 3, 4, 6, 3, 9, 6, 3, 8, 10, 3, 2
|9, 10, 1, 9, 8, 9, 6, 3, 3, 8, 9, 8, 7, 6, 6, 7, 10, 8, 5, 6, 6, 4, 2, 6, 9, 2, 10, 10, 0, 5, 9, 4, 3, 10, 9, 8, 3, 1, 5, 8, 1, 9, 7, 5, 4, 6, 10, 10, 7, 2
|9, 4, 6, 2, 2, 2, 2, 6, 6, 5, 5, 1, 6, 5, 6, 4, 7, 6, 4, 9, 2, 10, 9, 10, 2, 4, 1, 9, 1, 0, 5, 9, 3, 8, 10, 7, 4, 3, 10, 1, 8, 1, 7, 10, 5, 6, 1, 10, 10, 3
|4, 10, 3, 7, 3, 7, 2, 9, 10, 2, 2, 4, 3, 9, 5, 5, 9, 8, 3, 5, 2, 9, 7, 1, 4, 6, 2, 5, 4, 6, 0, 6, 1, 4, 5, 10, 8, 3, 2, 8, 7, 6, 10, 2, 5, 1, 7, 4, 10, 3
|6, 8, 3, 6, 2, 3, 5, 1, 10, 6, 8, 4, 6, 5, 7, 1, 6, 7, 1, 3, 7, 6, 10, 5, 5, 5, 10, 7, 2, 10, 9, 0, 1, 9, 2, 2, 8, 3, 6, 7, 3, 10, 5, 5, 7, 4, 6, 7, 7, 10
|3, 1, 5, 9, 6, 8, 5, 8, 4, 9, 3, 3, 6, 6, 7, 6, 1, 8, 7, 10, 2, 2, 8, 7, 4, 6, 3, 2, 6, 3, 4, 9, 0, 2, 7, 2, 7, 3, 3, 4, 3, 9, 3, 2, 4, 10, 1, 7, 2, 7
|10, 10, 4, 3, 5, 1, 7, 10, 6, 3, 5, 10, 3, 9, 7, 2, 5, 10, 4, 3, 10, 1, 7, 9, 1, 1, 5, 4, 3, 3, 7, 2, 5, 0, 10, 10, 1, 9, 2, 3, 6, 6, 7, 1, 9, 6, 5, 1, 1, 10
|5, 8, 7, 3, 2, 1, 5, 1, 1, 4, 9, 4, 6, 3, 9, 10, 1, 3, 4, 1, 9, 2, 10, 6, 10, 7, 6, 6, 10, 7, 5, 3, 2, 8, 0, 3, 3, 5, 8, 5, 2, 6, 2, 10, 5, 4, 3, 7, 1, 7
|10, 7, 7, 3, 7, 3, 2, 3, 2, 5, 7, 8, 1, 8, 10, 1, 1, 2, 7, 2, 6, 10, 9, 7, 10, 1, 7, 8, 9, 5, 3, 1, 10, 3, 5, 0, 6, 3, 5, 9, 4, 7, 3, 7, 7, 7, 1, 6, 9, 5
|8, 7, 8, 7, 10, 1, 3, 5, 4, 1, 8, 10, 8, 1, 7, 5, 1, 8, 5, 9, 3, 10, 9, 4, 7, 4, 2, 9, 1, 5, 8, 6, 8, 10, 1, 7, 0, 8, 8, 8, 1, 1, 3, 1, 1, 4, 9, 3, 8, 6
|10, 4, 1, 4, 3, 8, 1, 10, 3, 9, 4, 3, 5, 10, 5, 10, 10, 2, 6, 10, 10, 1, 1, 4, 7, 7, 2, 1, 4, 8, 2, 9, 4, 7, 1, 7, 6, 0, 7, 10, 2, 1, 8, 5, 2, 7, 6, 7, 1, 9
|10, 6, 5, 8, 7, 6, 9, 5, 6, 8, 9, 9, 9, 1, 3, 4, 7, 9, 5, 7, 6, 1, 1, 5, 1, 1, 5, 10, 3, 6, 9, 3, 4, 4, 9, 10, 9, 1, 0, 10, 7, 5, 4, 10, 8, 6, 2, 10, 4, 5
|5, 9, 1, 7, 9, 9, 5, 9, 5, 4, 10, 5, 8, 2, 7, 1, 5, 8, 4, 7, 1, 7, 7, 5, 2, 4, 9, 5, 7, 5, 8, 4, 1, 7, 7, 8, 4, 7, 4, 0, 4, 1, 2, 1, 5, 7, 9, 1, 10, 2
|9, 5, 7, 7, 6, 7, 10, 7, 1, 3, 2, 8, 4, 1, 8, 2, 6, 2, 10, 3, 5, 10, 6, 4, 10, 7, 7, 5, 10, 2, 10, 6, 1, 7, 10, 9, 2, 8, 3, 5, 0, 3, 10, 4, 4, 9, 9, 5, 10, 6
|8, 9, 3, 5, 10, 8, 3, 9, 1, 10, 1, 9, 7, 7, 3, 1, 10, 9, 1, 3, 5, 10, 2, 8, 10, 5, 4, 10, 9, 4, 7, 5, 2, 7, 10, 1, 8, 5, 1, 4, 4, 0, 2, 4, 6, 4, 8, 10, 9, 3
|5, 2, 1, 2, 9, 8, 8, 5, 4, 8, 1, 2, 3, 9, 2, 5, 9, 5, 6, 9, 1, 3, 2, 4, 1, 9, 4, 6, 1, 5, 9, 6, 1, 6, 8, 8, 8, 6, 7, 9, 8, 7, 0, 10, 8, 3, 3, 4, 2, 10
|1, 4, 8, 7, 8, 10, 4, 7, 7, 4, 2, 1, 3, 4, 1, 3, 5, 10, 3, 5, 10, 6, 5, 7, 10, 3, 6, 2, 5, 2, 8, 7, 2, 10, 4, 7, 7, 7, 5, 4, 9, 2, 5, 0, 3, 1, 3, 7, 3, 6
|6, 10, 4, 6, 7, 8, 1, 4, 9, 10, 5, 1, 3, 7, 5, 5, 1, 9, 2, 1, 8, 8, 3, 1, 6, 2, 9, 6, 9, 9, 8, 3, 8, 2, 5, 8, 3, 9, 3, 4, 10, 10, 7, 3, 0, 9, 6, 5, 3, 4
|4, 1, 1, 4, 8, 2, 3, 5, 4, 4, 5, 2, 6, 6, 2, 4, 2, 10, 9, 7, 3, 6, 9, 6, 10, 10, 8, 1, 10, 8, 5, 3, 8, 10, 3, 9, 8, 3, 8, 10, 4, 9, 2, 5, 3, 0, 5, 9, 5, 4
|2, 9, 7, 10, 9, 1, 2, 9, 3, 8, 1, 4, 4, 10, 9, 1, 4, 4, 2, 8, 1, 9, 9, 7, 8, 7, 5, 2, 1, 6, 10, 5, 1, 8, 10, 1, 4, 4, 7, 1, 7, 3, 3, 7, 4, 10, 0, 1, 2, 5
|5, 9, 4, 6, 5, 5, 7, 5, 9, 5, 5, 9, 8, 2, 5, 5, 2, 9, 7, 4, 5, 5, 8, 5, 2, 1, 2, 7, 7, 2, 9, 10, 4, 2, 9, 6, 9, 4, 8, 4, 10, 8, 2, 9, 10, 10, 9, 0, 3, 5
|3, 10, 8, 7, 5, 5, 9, 2, 5, 8, 3, 8, 1, 1, 7, 4, 8, 3, 1, 4, 7, 4, 3, 9, 1, 8, 5, 9, 9, 7, 2, 6, 6, 10, 1, 6, 7, 3, 8, 1, 6, 10, 10, 1, 1, 5, 8, 2, 0, 2
|6, 6, 8, 8, 8, 10, 8, 10, 8, 3, 4, 5, 8, 4, 10, 6, 8, 10, 2, 8, 10, 7, 10, 3, 9, 9, 5, 9, 5, 8, 8, 10, 1, 1, 8, 4, 2, 2, 9, 10, 4, 5, 4, 1, 4, 5, 3, 3, 2, 0
|];
TimeWindows = [| 7, 504
|10, 582
|6, 552
|3, 531
|8, 538
|7, 520
|7, 514
|7, 585
|10, 599
|5, 585
|1, 549
|7, 531
|3, 526
|1, 553
|8, 562
|3, 575
|6, 534
|1, 574
|5, 554
|6, 577
|7, 532
|6, 605
|9, 531
|8, 509
|9, 529
|2, 532
|2, 559
|4, 584
|6, 535
|8, 574
|9, 580
|7, 571
|7, 511
|7, 596
|5, 607
|4, 536
|5, 593
|1, 575
|2, 565
|9, 525
|];


The key of this seems to be the timeout. With a higher timeout both approaches should always give the same result. But that doesn't really explain why MiniZincIDE and MiniZinc-Python give different results, but it could be just some fluke.
I tested the model with the Chuffed solver (+ free search flag) and it took 3.4s:
...
----------
obj = 73
----------
==========
Google OR-tools took about the same time (with 12 threads it took 0.9s).
As mentioned in
Placing array of rectangles inside a given area using Minizinc , one might have to experiment with different solvers/search strategies and variants of the model to get faster times.
For this model it seems to suffice to simply change the solver (if that is a acceptable action).

Related

Append arrays of different dimensions to get a single array

l have three vectors (numpy arrays), vector_1, vector_2, vector_3
as follow :
Dimension(vector1)=(200,2048)
Dimension(vector2)=(200,8192)
Dimension(vector3)=(200,32768)
l would like to append these vectors to get vector_4 :
Dimension(vector4)= (200,2048+8192+32768)= (200, 43008)
Add respectively vector1 then vector2 then vector3
l tries the following :
vector4=numpy.concatenate((vector1,vector2,vector3),axis=0)
ValueError: all the input array dimensions except for the concatenation axis must match exactly
and
vector4=numpy.append(vector4,[vector1,vector2,vectors3],axis=0)
TypeError: append() missing 1 required positional argument: 'values'

I believe you are looking for numpy.hstack.
>>> import numpy as np
>>> a = np.arange(4).reshape(2,2)
>>> b = np.arange(6).reshape(2,3)
>>> c = np.arange(8).reshape(2,4)
>>> a
array([[0, 1],
[2, 3]])
>>> b
array([[0, 1, 2],
[3, 4, 5]])
>>> c
array([[0, 1, 2, 3],
[4, 5, 6, 7]])
>>> np.hstack((a,b,c))
array([[0, 1, 0, 1, 2, 0, 1, 2, 3],
[2, 3, 3, 4, 5, 4, 5, 6, 7]])

The error message is pretty much telling you exactly what is the problem:
ValueError: all the input array dimensions except for the concatenation axis must match exactly
But you are doing the opposite, the concatenation axis dimensions match exactly but the others don't. Consider:
In [3]: arr1 = np.random.randint(0,10,(20, 5))
In [4]: arr2 = np.random.randint(0,10,(20, 3))
In [5]: arr3 = np.random.randint(0,10,(20, 11))
Note the dimensions. Just give it the correct axis. So use the second rather than the first:
In [8]: arr1.shape, arr2.shape, arr3.shape
Out[8]: ((20, 5), (20, 3), (20, 11))
In [9]: np.concatenate((arr1, arr2, arr3), axis=1)
Out[9]:
array([[3, 1, 4, 7, 3, 6, 1, 1, 6, 7, 4, 6, 8, 6, 2, 8, 2, 5, 0],
[4, 2, 2, 1, 7, 8, 0, 7, 2, 2, 3, 9, 8, 0, 7, 3, 5, 9, 6],
[2, 8, 9, 8, 5, 3, 5, 8, 5, 2, 4, 1, 2, 0, 3, 2, 9, 1, 0],
[6, 7, 3, 5, 6, 8, 3, 8, 4, 8, 1, 5, 4, 4, 6, 4, 0, 3, 4],
[3, 5, 8, 8, 7, 7, 4, 8, 7, 3, 8, 7, 0, 2, 8, 9, 1, 9, 0],
[5, 4, 8, 3, 7, 8, 3, 2, 7, 8, 2, 4, 8, 0, 6, 9, 2, 0, 3],
[0, 0, 1, 8, 6, 4, 4, 4, 2, 8, 4, 1, 4, 1, 3, 1, 5, 5, 1],
[1, 6, 3, 3, 9, 2, 3, 4, 9, 2, 6, 1, 4, 1, 5, 6, 0, 1, 9],
[4, 5, 4, 7, 1, 4, 0, 8, 8, 1, 6, 0, 4, 6, 3, 1, 2, 5, 2],
[6, 4, 3, 2, 9, 4, 1, 7, 7, 0, 0, 5, 9, 3, 7, 4, 5, 6, 1],
[7, 7, 0, 4, 1, 9, 9, 1, 0, 1, 8, 3, 6, 0, 5, 1, 4, 0, 7],
[7, 9, 0, 4, 0, 5, 5, 9, 8, 9, 9, 7, 8, 8, 2, 6, 2, 3, 1],
[4, 1, 6, 5, 4, 5, 6, 7, 9, 2, 5, 8, 6, 6, 6, 8, 2, 3, 1],
[7, 7, 8, 5, 0, 8, 5, 6, 4, 4, 3, 5, 9, 8, 7, 9, 8, 8, 1],
[3, 9, 3, 6, 3, 2, 2, 4, 0, 1, 0, 4, 3, 0, 1, 3, 4, 1, 3],
[5, 1, 9, 7, 1, 8, 3, 9, 4, 7, 6, 7, 4, 7, 0, 1, 2, 8, 7],
[6, 3, 8, 0, 6, 2, 1, 8, 1, 0, 0, 3, 7, 2, 1, 5, 7, 0, 7],
[5, 4, 7, 5, 5, 8, 3, 2, 6, 1, 0, 4, 6, 9, 7, 3, 9, 2, 5],
[1, 4, 8, 5, 7, 2, 0, 2, 6, 2, 6, 5, 5, 4, 6, 1, 8, 8, 1],
[4, 4, 5, 6, 2, 6, 0, 5, 1, 8, 4, 5, 8, 9, 2, 1, 0, 4, 2]])
In [10]: np.concatenate((arr1, arr2, arr3), axis=1).shape
Out[10]: (20, 19)

python: FOR loop, Iterating assignment not assigning correct value

I am making a list, with a list nested inside. This second list I need to iterate through, and assign a value to a position. So I wrote this
matrix=[]
list=[0,1,2,3,4,5,6,7,8,9]
def board():
for x in range(10):
for y in range(10):
matrix.append([[x,y],1]) #A grid is made, with a spare element
for i in range(100):
matrix[i][1]=list #The grid is two elements, co-ords...
#and a list of attributes
board()
for i in range(100):
matrix[i][2][0]=i #The first attribute,(in 2nd of2elements),
print(matrix) ##SHOULD be its own position
All variations of this code result in the same thing:
Error or the number "99" being placed in EVERY INSTANCE of that
location. That is to say my final FOR loop seems to be assigning "99"
NOT i...
I have tried several variations of this (generating the numbers inside the FOR loop was my preference, alas) I have spent about 6 hours staring at this.
All my logic tells me that the computer should KNOW that I want the first element, of that second element, to be a count 0,1,2...
I know this can't be because of a limit on the language, so tell me what I am not seeing.

matrix[i][1]=list
When you do this the same array is being assigned in the matrix array. Hence if you change one item, all the items get changed. Instead assign a copy of the list like this:
matrix[i][1] = list[:]
As pointed out by #Bakuriu , in python 3 you can use list.copy() method. In Python 2 you can use the copy module to copy containers with l2 = copy.copy(l1)
Also, you should change the name of the variable 'list' as 'list' is an inbuilt type in python.

You should replace matrix[i][2][0]=i assignement as it raises an IndexError to:
for i in range(100):
matrix[i][1][0]=i #The first attribute,(in 2nd of2elements),
print(matrix) ##SHOULD be its own position

Lists are faster in for loop, but here You have solution with dicts:
from pprint import pprint as ppr
matrix={}
list=[0,1,2,3,4,5,6,7,8,9]
def board():
for x in range(10):
for y in range(10):
matrix[(x,y)]=1
for k in matrix.keys():
matrix[k]=list
board()
ppr(matrix)
for k in matrix.iterkeys():
v=matrix[k][:]
v[2]=k[0]*10+k[1]
matrix[k]=v
ppr(matrix)
Input:
{(0, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(1, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(2, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(3, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(4, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(5, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(6, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(7, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(8, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 0): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 1): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 3): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 4): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 5): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 6): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 7): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 8): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(9, 9): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}
Output:
{(0, 0): [0, 1, 0, 3, 4, 5, 6, 7, 8, 9],
(0, 1): [0, 1, 1, 3, 4, 5, 6, 7, 8, 9],
(0, 2): [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
(0, 3): [0, 1, 3, 3, 4, 5, 6, 7, 8, 9],
(0, 4): [0, 1, 4, 3, 4, 5, 6, 7, 8, 9],
(0, 5): [0, 1, 5, 3, 4, 5, 6, 7, 8, 9],
(0, 6): [0, 1, 6, 3, 4, 5, 6, 7, 8, 9],
(0, 7): [0, 1, 7, 3, 4, 5, 6, 7, 8, 9],
(0, 8): [0, 1, 8, 3, 4, 5, 6, 7, 8, 9],
(0, 9): [0, 1, 9, 3, 4, 5, 6, 7, 8, 9],
(1, 0): [0, 1, 10, 3, 4, 5, 6, 7, 8, 9],
(1, 1): [0, 1, 11, 3, 4, 5, 6, 7, 8, 9],
(1, 2): [0, 1, 12, 3, 4, 5, 6, 7, 8, 9],
(1, 3): [0, 1, 13, 3, 4, 5, 6, 7, 8, 9],
(1, 4): [0, 1, 14, 3, 4, 5, 6, 7, 8, 9],
(1, 5): [0, 1, 15, 3, 4, 5, 6, 7, 8, 9],
(1, 6): [0, 1, 16, 3, 4, 5, 6, 7, 8, 9],
(1, 7): [0, 1, 17, 3, 4, 5, 6, 7, 8, 9],
(1, 8): [0, 1, 18, 3, 4, 5, 6, 7, 8, 9],
(1, 9): [0, 1, 19, 3, 4, 5, 6, 7, 8, 9],
(2, 0): [0, 1, 20, 3, 4, 5, 6, 7, 8, 9],
(2, 1): [0, 1, 21, 3, 4, 5, 6, 7, 8, 9],
(2, 2): [0, 1, 22, 3, 4, 5, 6, 7, 8, 9],
(2, 3): [0, 1, 23, 3, 4, 5, 6, 7, 8, 9],
(2, 4): [0, 1, 24, 3, 4, 5, 6, 7, 8, 9],
(2, 5): [0, 1, 25, 3, 4, 5, 6, 7, 8, 9],
(2, 6): [0, 1, 26, 3, 4, 5, 6, 7, 8, 9],
(2, 7): [0, 1, 27, 3, 4, 5, 6, 7, 8, 9],
(2, 8): [0, 1, 28, 3, 4, 5, 6, 7, 8, 9],
(2, 9): [0, 1, 29, 3, 4, 5, 6, 7, 8, 9],
(3, 0): [0, 1, 30, 3, 4, 5, 6, 7, 8, 9],
(3, 1): [0, 1, 31, 3, 4, 5, 6, 7, 8, 9],
(3, 2): [0, 1, 32, 3, 4, 5, 6, 7, 8, 9],
(3, 3): [0, 1, 33, 3, 4, 5, 6, 7, 8, 9],
(3, 4): [0, 1, 34, 3, 4, 5, 6, 7, 8, 9],
(3, 5): [0, 1, 35, 3, 4, 5, 6, 7, 8, 9],
(3, 6): [0, 1, 36, 3, 4, 5, 6, 7, 8, 9],
(3, 7): [0, 1, 37, 3, 4, 5, 6, 7, 8, 9],
(3, 8): [0, 1, 38, 3, 4, 5, 6, 7, 8, 9],
(3, 9): [0, 1, 39, 3, 4, 5, 6, 7, 8, 9],
(4, 0): [0, 1, 40, 3, 4, 5, 6, 7, 8, 9],
(4, 1): [0, 1, 41, 3, 4, 5, 6, 7, 8, 9],
(4, 2): [0, 1, 42, 3, 4, 5, 6, 7, 8, 9],
(4, 3): [0, 1, 43, 3, 4, 5, 6, 7, 8, 9],
(4, 4): [0, 1, 44, 3, 4, 5, 6, 7, 8, 9],
(4, 5): [0, 1, 45, 3, 4, 5, 6, 7, 8, 9],
(4, 6): [0, 1, 46, 3, 4, 5, 6, 7, 8, 9],
(4, 7): [0, 1, 47, 3, 4, 5, 6, 7, 8, 9],
(4, 8): [0, 1, 48, 3, 4, 5, 6, 7, 8, 9],
(4, 9): [0, 1, 49, 3, 4, 5, 6, 7, 8, 9],
(5, 0): [0, 1, 50, 3, 4, 5, 6, 7, 8, 9],
(5, 1): [0, 1, 51, 3, 4, 5, 6, 7, 8, 9],
(5, 2): [0, 1, 52, 3, 4, 5, 6, 7, 8, 9],
(5, 3): [0, 1, 53, 3, 4, 5, 6, 7, 8, 9],
(5, 4): [0, 1, 54, 3, 4, 5, 6, 7, 8, 9],
(5, 5): [0, 1, 55, 3, 4, 5, 6, 7, 8, 9],
(5, 6): [0, 1, 56, 3, 4, 5, 6, 7, 8, 9],
(5, 7): [0, 1, 57, 3, 4, 5, 6, 7, 8, 9],
(5, 8): [0, 1, 58, 3, 4, 5, 6, 7, 8, 9],
(5, 9): [0, 1, 59, 3, 4, 5, 6, 7, 8, 9],
(6, 0): [0, 1, 60, 3, 4, 5, 6, 7, 8, 9],
(6, 1): [0, 1, 61, 3, 4, 5, 6, 7, 8, 9],
(6, 2): [0, 1, 62, 3, 4, 5, 6, 7, 8, 9],
(6, 3): [0, 1, 63, 3, 4, 5, 6, 7, 8, 9],
(6, 4): [0, 1, 64, 3, 4, 5, 6, 7, 8, 9],
(6, 5): [0, 1, 65, 3, 4, 5, 6, 7, 8, 9],
(6, 6): [0, 1, 66, 3, 4, 5, 6, 7, 8, 9],
(6, 7): [0, 1, 67, 3, 4, 5, 6, 7, 8, 9],
(6, 8): [0, 1, 68, 3, 4, 5, 6, 7, 8, 9],
(6, 9): [0, 1, 69, 3, 4, 5, 6, 7, 8, 9],
(7, 0): [0, 1, 70, 3, 4, 5, 6, 7, 8, 9],
(7, 1): [0, 1, 71, 3, 4, 5, 6, 7, 8, 9],
(7, 2): [0, 1, 72, 3, 4, 5, 6, 7, 8, 9],
(7, 3): [0, 1, 73, 3, 4, 5, 6, 7, 8, 9],
(7, 4): [0, 1, 74, 3, 4, 5, 6, 7, 8, 9],
(7, 5): [0, 1, 75, 3, 4, 5, 6, 7, 8, 9],
(7, 6): [0, 1, 76, 3, 4, 5, 6, 7, 8, 9],
(7, 7): [0, 1, 77, 3, 4, 5, 6, 7, 8, 9],
(7, 8): [0, 1, 78, 3, 4, 5, 6, 7, 8, 9],
(7, 9): [0, 1, 79, 3, 4, 5, 6, 7, 8, 9],
(8, 0): [0, 1, 80, 3, 4, 5, 6, 7, 8, 9],
(8, 1): [0, 1, 81, 3, 4, 5, 6, 7, 8, 9],
(8, 2): [0, 1, 82, 3, 4, 5, 6, 7, 8, 9],
(8, 3): [0, 1, 83, 3, 4, 5, 6, 7, 8, 9],
(8, 4): [0, 1, 84, 3, 4, 5, 6, 7, 8, 9],
(8, 5): [0, 1, 85, 3, 4, 5, 6, 7, 8, 9],
(8, 6): [0, 1, 86, 3, 4, 5, 6, 7, 8, 9],
(8, 7): [0, 1, 87, 3, 4, 5, 6, 7, 8, 9],
(8, 8): [0, 1, 88, 3, 4, 5, 6, 7, 8, 9],
(8, 9): [0, 1, 89, 3, 4, 5, 6, 7, 8, 9],
(9, 0): [0, 1, 90, 3, 4, 5, 6, 7, 8, 9],
(9, 1): [0, 1, 91, 3, 4, 5, 6, 7, 8, 9],
(9, 2): [0, 1, 92, 3, 4, 5, 6, 7, 8, 9],
(9, 3): [0, 1, 93, 3, 4, 5, 6, 7, 8, 9],
(9, 4): [0, 1, 94, 3, 4, 5, 6, 7, 8, 9],
(9, 5): [0, 1, 95, 3, 4, 5, 6, 7, 8, 9],
(9, 6): [0, 1, 96, 3, 4, 5, 6, 7, 8, 9],
(9, 7): [0, 1, 97, 3, 4, 5, 6, 7, 8, 9],
(9, 8): [0, 1, 98, 3, 4, 5, 6, 7, 8, 9],
(9, 9): [0, 1, 99, 3, 4, 5, 6, 7, 8, 9]}
You could use numpy here, with fast indexing (faster is cython memoryview so look at it too):
matrix=np.zeros((100,10),dtype=int)
list=[0,1,2,3,4,5,6,7,8,9]
def board(matrix):
for x in range(10):
for y in range(10):
matrix[x*10+y]=1 #but here You populate whole sub-array of shape (10) with 1, so it will be: [1,1,1,1...,1]
print matrix
for k in range(matrix.shape[0]):
matrix[k]=list
board(matrix)
ppr(matrix)
for k in range(matrix.shape[0]):
matrix[k,2]=k
ppr(matrix)

How can iterate throught a dictionary in a list in python

This is an output from traversing through different folders in a directory:
r= [{'s.data': [22, 10, 21, 2, 3, 3, 4, 4, 4], 'd.data': [1, 5, 3,67,8,4, 9, 0, 2], 'q.data': [2, 3, 4, 5, 6, 3, 4, 2, 32], 't.data': [32, 21, 9, 2, 13, 5, 4, 9, 7], 'k.data': [3, 7, 2, 5, 68, 90, 23, 11, 22]}, {'s.data': [9, 8, 7, 6, 5, 4, 3, 2, 1], 'd.data': [2, 6, 3, 8, 9, 0, 3, 2, 5], 'q.data': [1, 2, 3, 4, 5, 6, 7, 8, 9], 't.data': [0, 8, 9, 7, 5, 6, 4, 2, 3], 'k.data': [4, 3, 2, 6, 7, 8, 9, 0, 1]}, {'s.data': [4, 3, 6, 4, 7, 2, 8, 1, 9], 'd.data': [6, 4, 3, 5, 6, 7, 8, 9, 0], 'q.data': [23, 2, 3, 6, 8, 4, 5, 0, 8], 't.data': [2, 1, 3, 4, 5, 11, 2, 6, 7], 'k.data': [1, 0, 2, 9, 3, 8, 4, 7, 4]}, {'s.data': [1, 9, 7, 0, 2, 4, 2, 3, 5], 'd.data': [4, 3, 7, 9, 2, 3, 0, 9, 10], 'q.data': [0, 23, 4, 2, 4, 6, 8, 3, 0], 't.data': [4, 1, 7, 4, 0, 8, 9, 9, 0], 'k.data': [9, 2, 6, 1, 2, 3, 5, 6, 9]}, {'s.data': [22, 10, 21, 2, 3, 3, 4, 4, 4], 'd.data': [5, 7, 4, 5, 8, 9, 0, 2, 3], 'q.data': [3, 2, 4, 5, 6, 7, 8, 9, 1], 't.data': [2, 3, 1, 4, 5, 6, 7, 8, 0], 'k.data': [3, 2, 5, 6, 7, 4, 5, 6, 2]}]
How can I print only the values like the following without the keys as follows:
[[22, 10, 21, 2, 3, 3, 4, 4, 4],
[9, 8, 7, 6, 5, 4, 3, 2, 1],
[4, 3, 6, 4, 7, 2, 8, 1, 9],
[1, 9, 7, 0, 2, 4, 2, 3, 5]]
Any help will be appreciated!

Something like this should do the trick, if you don't need a specific ordering across the s.data, k.data, etc keys.
print [x.values() for x in r]
Alternativelym if you care only about s.data or any other specific key, you can do
k = 's.data'
print [x[k] for x in r]

You apparently want only the values for the key 's.data' in each of the dictionaries. You can do that as follows:
[d['s.data'] for d in r]

Combining multiple numpy arrays into one

I have this O/P as shown in the below pic
My O/P consists of around 100+ numpy arrays like the one shown above. I'm trying to combine all these 100+ numpy arrays into an single numpy array for further data processing. Any ideas on how to get this done???? I'm using python V3.4

You can use hstack and vstack:
In [29]: hstack((arange(10) for _ in range(10)))
Out[29]:
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8,
9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1,
2, 3, 4, 5, 6, 7, 8, 9])
In [30]: vstack((arange(10) for _ in range(10)))
Out[30]:
array([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]])

Numpy Routine(s) to create a regular grid inside a 2d array

I am trying to write a function that would create a regular grid of 5 pixels by 5 pixels inside a 2d array. I was hoping some combination of numpy.arange and numpy.repeat might do it, but so far I haven't had any luck because numpy.repeat will just repeat along the same row.
Here is an example:
Let's say I want a 5x5 grid inside a 2d array of shape (20, 15). It should look like:
array([[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 9, 9, 9, 9, 9,10,10,10,10,10,11,11,11,11,11],
[ 9, 9, 9, 9, 9,10,10,10,10,10,11,11,11,11,11],
[ 9, 9, 9, 9, 9,10,10,10,10,10,11,11,11,11,11],
[ 9, 9, 9, 9, 9,10,10,10,10,10,11,11,11,11,11],
[ 9, 9, 9, 9, 9,10,10,10,10,10,11,11,11,11,11]])
I realize I could simply use a loop and slicing to accomplish this, but I could be applying this to very large arrays and I worry that the performance of that would be too slow or impractical.
Can anyone recommend a method to accomplish this?
Thanks in advance.
UPDATE:
All the answers provided seem to work well. Can anyone tell me which will be the most efficient to use for large arrays? By large array I mean it could be 100000 x 100000 or more with 15 x 15 grid cell sizes.

Broadcasting is the answer here:
m, n, d = 20, 15, 5
arr = np.empty((m, n), dtype=np.int)
arr_view = arr.reshape(m // d, d, n // d, d)
vals = np.arange(m // d * n // d).reshape(m // d, 1, n // d, 1)
arr_view[:] = vals
>>> arr
array([[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8],
[ 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11],
[ 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11],
[ 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11],
[ 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11],
[ 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11]])

Similar to Jaime's answer:
np.repeat(np.arange(0, 10, 3), 4)[..., None] + np.repeat(np.arange(3), 5)[None, ...]

kron will do this expansion (as Brionius also suggested in the comments):
xi, xj, ni, nj = 5, 5, 4, 3
r = np.kron(np.arange(ni*nj).reshape((ni,nj)), np.ones((xi, xj)))
Although I haven't tested it, I assume it's less efficient than the broadcasting approach, but a bit more concise and easier to understand (I hope). It's likely less efficient because: 1) it requires the array of ones, 2) it does xi*xj multiplications by 1, and 3) it does a bunch of concats.

Categories