I have to perform a large number of such calculations:
X.dot(Y).dot(Xt)
X = 1 x n matrix
Y = symmetric n x n matrix, each element one of 5 values (e.g. 0, 0.25, 0.5, 0.75, 1)
Xt = n x 1 matrix, transpose of X, i.e. X[np.newaxis].T
X and Y are dense. The problem I have is for large n, I cannot store and use matrix Y due to memory issues. I am limited to using one machine, so distributed calculations are not an option.
It occurred to me that Y has 2 features which theoretically can reduce the amount of memory required to store Y:
Elements of Y are covered by a small list of values.
Y is symmetric.
How can I implement this in practice? I have looked up storage of symmetric matrices, but as far as I am aware all numpy matrix multiplications require "unpacking" the symmetry to produce a regular n x n matrix.
I understand numpy is designed for in-memory calculations, so I'm open to alternative python-based solutions not reliant on numpy. I'm also open to sacrificing speed for memory-efficiency.
UPDATE: I found using a format that crams 3 matrix elements into one byte is actually quite fast. In the example below the speed penalty is less than 2x compared to direct multiplication using # while the space saving is more than 20x.
>>> Y = np.random.randint(0, 5, (3000, 3000), dtype = np.int8)
>>> i, j = np.triu_indices(3000, 1)
>>> Y[i, j] = Y[j, i]
>>> values = np.array([0.3, 0.5, 0.6, 0.9, 2.0])
>>> Ycmp = (np.reshape(Y, (1000, 3, 3000)) * np.array([25, 5, 1], dtype=np.int8)[None, :, None]).sum(axis=1, dtype=np.int8)
>>>
>>> full = values[Y]
>>> x # full # x
1972379.8153972814
>>>
>>> vtable = values[np.transpose(np.unravel_index(np.arange(125), (5,5,5)))]
>>> np.dot(np.concatenate([(vtable * np.bincount(row, x, minlength=125)[:, None]).sum(axis=0) for row in Ycmp]), x)
1972379.8153972814
>>>
>>> timeit('x # full # x', globals=globals(), number=100)
0.7130507210385986
>>> timeit('np.dot(np.concatenate([(vtable * np.bincount(row, x, minlength=125)[:, None]).sum(axis=0) for row in Ycmp]), x)', globals=globals(), number=100)
1.3755558349657804
The solutions below are slower and less memory efficient. I'll leave them merely for reference.
If you can afford half a byte per matrix element, then you can use np.bincount like so:
>>> Y = np.random.randint(0, 5, (1000, 1000), dtype = np.int8)
>>> i, j = np.triu_indices(1000, 1)
>>> Y[i, j] = Y[j, i]
>>> values = np.array([0.3, 0.5, 0.6, 0.9, 2.0])
>>> full = values[Y]
>>> # full would correspond to your original matrix,
>>> # Y is the 'compressed' version
>>>
>>> x = np.random.random((1000,))
>>>
>>> # direct method for reference
>>> x # full # x
217515.13954751115
>>>
>>> # memory saving version
>>> np.dot([(values * np.bincount(row, x)).sum() for row in Y], x)
217515.13954751118
>>>
>>> # to save another almost 50% exploit symmetry
>>> upper = Y[i, j]
>>> diag = np.diagonal(Y)
>>>
>>> boundaries = np.r_[0, np.cumsum(np.arange(999, 0, -1))]
>>> (values*np.bincount(diag, x*x)).sum() + 2 * np.dot([(values*np.bincount(upper[boundaries[i]:boundaries[i+1]], x[i+1:],minlength=5)).sum() for i in range(999)], x[:-1])
217515.13954751115
Each row of Y, if represented as a numpy.array of datatype int as suggested in #PaulPanzer's answer, can be compressed to occupy less memory: In fact, you can store 27 elements with 64 bit, because 64 / log2(5) = 27.56...
First, compress:
import numpy as np
row = np.random.randint(5, size=100)
# pad with zeros to length that is multiple of 27
if len(row)%27:
row_pad = np.append(row, np.zeros(27 - len(row)%27, dtype=int))
else:
row_pad = row
row_compr = []
y_compr = 0
for i, y in enumerate(row_pad):
if i > 0 and i % 27 == 0:
row_compr.append(y_compr)
y_compr = 0
y_compr *= 5
y_compr += y
# append last
row_compr.append(y_compr)
row_compr = np.array(row_compr, dtype=np.int64)
Then, decompress:
row_decompr = []
for y_compr in row_compr:
y_block = np.zeros(shape=27, dtype=np.uint8)
for i in range(27):
y_block[26-i] = y_compr % 5
y_compr = int(y_compr // 5)
row_decompr.append(y_block)
row_decompr = np.array(row_decompr).flatten()[:len(row)]
The decompressed array coincides with the original row of Y:
assert np.allclose(row, row_decompr)
Related
I have two numpy arrays (4x4 each). I would like to concatenate them to a tensor of (4x4x2) in which the first 'sheet' is the first array, second 'sheet' is the second array, etc. However, when I try np.stack the output of d[1] is not showing the correct values of the first matrix.
import numpy as np
x = array([[ 3.38286851e-02, -6.11905173e-05, -9.08147798e-03,
-2.46860166e-02],
[-6.11905173e-05, 1.74237508e-03, -4.52140165e-04,
-1.22904439e-03],
[-9.08147798e-03, -4.52140165e-04, 1.91939979e-01,
-1.82406361e-01],
[-2.46860166e-02, -1.22904439e-03, -1.82406361e-01,
2.08321422e-01]])
print(np.shape(x)) # 4 x 4
y = array([[ 6.76573701e-02, -1.22381035e-04, -1.81629560e-02,
-4.93720331e-02],
[-1.22381035e-04, 3.48475015e-03, -9.04280330e-04,
-2.45808879e-03],
[-1.81629560e-02, -9.04280330e-04, 3.83879959e-01,
-3.64812722e-01],
[-4.93720331e-02, -2.45808879e-03, -3.64812722e-01,
4.16642844e-01]])
print(np.shape(y)) # 4 x 4
d = np.dstack((x,y))
np.shape(d) # indeed it is 4,4,2... but if I do d[1] then it is not the first x matrix.
d[1] # should be y
If you do np.dstack((x, y)), which is the same as the more explicit np.stack((x, y), axis=-1), you are concatenating along the last, not the first axis (i.e., the one with size 2):
(x == d[..., 0]).all()
(y == d[..., 1]).all()
Ellipsis (...) is a python object that means ": as many times as necessary" when used in an index. For a 3D array, you can equivalently access the leaves as
d[:, :, 0]
d[:, :, 1]
If you want to access the leaves along the first axis, your array must be (2, 4, 4):
d = np.stack((x, y), axis=0)
(x == d[0]).all()
(y == d[1]).all()
Use np.stack instead of np.dstack:
>>> d = np.stack([y, x])
>>> np.all(d[0] == y)
True
>>> np.all(d[1] == x)
True
>>> d.shape
(2, 4, 4)
How do I (efficiently) sample zero values from a scipy.sparse.coo_matrix?
>>> import numpy as np
>>> from scipy.sparse import coo_matrix
>>> # create sparse array
>>> X = np.array([[1., 0.], [2., 1.], [0., 0.]])
>>> X_sparse = coo_matrix(X)
>>> # randomly sample 0's from X_sparse, retrieving as [(row, col), (row_col)]
>>> def sample_zeros(sp_arr, n, replacement=False):
>>> # ???
>>> return negs
>>> zero_indices = sample_zeros(X_sparse, n=3, replacement=False)
>>> print(zero_indices)
[(0, 1), (2, 0), (2, 1)]
Efficiency is important here, since I will be doing this in an iterator that feeds a neural network.
Since you know the shape of X, you could use np.random.choice to generate
random (row, col) locations in X:
h, w = X.shape
rows = np.random.choice(h, size=n)
cols = np.random.choice(w, size=n)
The main difficulty is how to check if a (row, col) is a non-zero location in X.
Here's a way to do that: Make a new sparse X which equals 1 wherever X is nonzero.
Next, create a new sparse matrix, Y, with non-zero values at the random locations generated above. Then subtract:
Y = Y - X.multiply(Y)
This sparse matrix Y will be zero wherever X is nonzero.
So if we've managed to generate enough nonzero values in Y, then we can use their (row, col) locations as the return value for sample_negs:
import unittest
import sys
import numpy as np
import scipy.sparse as sparse
def sample_negs(X, n=3, replace=False):
N = np.prod(X.shape)
m = N - X.size
if n == 0:
result = []
elif (n < 0) or (not replace and m < n) or (replace and m == 0):
raise ValueError("{n} samples from {m} locations do not exist"
.format(n=n, m=m))
elif n/m > 0.5:
# Y (in the else clause, below) would be pretty dense so there would be no point
# trying to use sparse techniques. So let's use hpaulj's idea
# (https://stackoverflow.com/a/53577267/190597) instead.
import warnings
warnings.filterwarnings("ignore", category=sparse.SparseEfficiencyWarning)
Y = sparse.coo_matrix(X == 0)
rows = Y.row
cols = Y.col
idx = np.random.choice(len(rows), size=n, replace=replace)
result = list(zip(rows[idx], cols[idx]))
else:
X_row, X_col = X.row, X.col
X_data = np.ones(X.size)
X = sparse.coo_matrix((X_data, (X_row, X_col)), shape=X.shape)
h, w = X.shape
Y = sparse.coo_matrix(X.shape)
Y_size = 0
while Y_size < n:
m = n - Y.size
Y_data = np.concatenate([Y.data, np.ones(m)])
Y_row = np.concatenate([Y.row, np.random.choice(h, size=m)])
Y_col = np.concatenate([Y.col, np.random.choice(w, size=m)])
Y = sparse.coo_matrix((Y_data, (Y_row, Y_col)), shape=X.shape)
# Remove values in Y where X is nonzero
# This also consolidates (row, col) duplicates
Y = sparse.coo_matrix(Y - X.multiply(Y))
if replace:
Y_size = Y.data.sum()
else:
Y_size = Y.size
if replace:
rows = np.repeat(Y.row, Y.data.astype(int))
cols = np.repeat(Y.col, Y.data.astype(int))
idx = np.random.choice(rows.size, size=n, replace=False)
result = list(zip(rows[idx], cols[idx]))
else:
rows = Y.row
cols = Y.col
idx = np.random.choice(rows.size, size=n, replace=False)
result = list(zip(rows[idx], cols[idx]))
return result
class Test(unittest.TestCase):
def setUp(self):
import warnings
warnings.filterwarnings("ignore", category=sparse.SparseEfficiencyWarning)
self.ncols, self.nrows = 100, 100
self.X = sparse.random(self.ncols, self.nrows, density=0.05, format='coo')
Y = sparse.coo_matrix(self.X == 0)
self.expected = set(zip(Y.row, Y.col))
def test_n_too_large(self):
self.assertRaises(ValueError, sample_negs, self.X, n=100*100+1, replace=False)
X_dense = sparse.coo_matrix(np.ones((4,2)))
self.assertRaises(ValueError, sample_negs, X_dense, n=1, replace=True)
def test_no_replacement(self):
for m in range(100):
negative_list = sample_negs(self.X, n=m, replace=False)
negative_set = set(negative_list)
self.assertEqual(len(negative_list), m)
self.assertLessEqual(negative_set, self.expected)
def test_no_repeats_when_replace_is_false(self):
negative_list = sample_negs(self.X, n=10, replace=False)
self.assertEqual(len(negative_list), len(set(negative_list)))
def test_dense_replacement(self):
N = self.ncols * self.nrows
m = N - self.X.size
for i in [-1, 0, 1]:
negative_list = sample_negs(self.X, n=m+i, replace=True)
negative_set = set(negative_list)
self.assertEqual(len(negative_list), m+i)
self.assertLessEqual(negative_set, self.expected)
def test_sparse_replacement(self):
for m in range(100):
negative_list = sample_negs(self.X, n=m, replace=True)
negative_set = set(negative_list)
self.assertEqual(len(negative_list), m)
self.assertLessEqual(negative_set, self.expected)
if __name__ == '__main__':
sys.argv.insert(1,'--verbose')
unittest.main(argv = sys.argv)
Since sample_negs is rather complicated, I've included some unit tests
to hopefully verify reasonable behavior.
I don't think there's an efficient way that takes advantage of the sparse matrix structure:
In [197]: >>> X = np.array([[1., 0.], [2., 1.], [0., 0.]])
...: >>> X_sparse = sparse.coo_matrix(X)
In [198]: X_sparse
Out[198]:
<3x2 sparse matrix of type '<class 'numpy.float64'>'
with 3 stored elements in COOrdinate format>
In [199]: print(X_sparse)
(0, 0) 1.0
(1, 0) 2.0
(1, 1) 1.0
With the dense array you could do something like:
In [204]: zeros = np.argwhere(X==0)
In [205]: zeros
Out[205]:
array([[0, 1],
[2, 0],
[2, 1]])
In [206]: idx=np.random.choice(3,3, replace=False)
In [207]: idx
Out[207]: array([0, 2, 1])
In [208]: zeros[idx,:]
Out[208]:
array([[0, 1],
[2, 1],
[2, 0]])
We could ask for all 0s of the sparse matrix:
In [209]: X_sparse==0
/usr/local/lib/python3.6/dist-packages/scipy/sparse/compressed.py:214: SparseEfficiencyWarning: Comparing a sparse matrix with 0 using == is inefficient, try using != instead.
", try using != instead.", SparseEfficiencyWarning)
Out[209]:
<3x2 sparse matrix of type '<class 'numpy.bool_'>'
with 3 stored elements in Compressed Sparse Row format>
In [210]: print(_)
(0, 1) True
(2, 0) True
(2, 1) True
I have an array of 5 values, consisting of 4 values and one index. I sort and split the array along the index. This leads me to splits of matrices with different lengths. From here on I want to calculate the mean, variance of the fourth values and covariance of the first 3 values for every split. My current approach works with a for loop, which I would like to replace by matrix operations, but I am struggeling with the different sizes of my matrices.
import numpy as np
A = np.random.rand(10,5)
A[:,-1] = np.random.randint(4, size=10)
sorted_A = A[np.argsort(A[:,4])]
splits = np.split(sorted_A, np.where(np.diff(sorted_A[:,4]))[0]+1)
My current for loop looks like this:
result = np.zeros((len(splits), 5))
for idx, values in enumerate(splits):
if(len(values))>0:
result[idx, 0] = np.mean(values[:,3])
result[idx, 1] = np.var(values[:,3])
result[idx, 2:5] = np.cov(values[:,0:3].transpose(), ddof=0).diagonal()
else:
result[idx, 0] = values[:,3]
I tried to work with masked arrays without success, since I couldn't load the matrices into the masked arrays in a proper form. Maybe someone knows how to do this or has a different suggestion.
You can use np.add.reduceat as follows:
>>> idx = np.concatenate([[0], np.where(np.diff(sorted_A[:,4]))[0]+1, [A.shape[0]]])
>>> result2 = np.empty((idx.size-1, 5))
>>> result2[:, 0] = np.add.reduceat(sorted_A[:, 3], idx[:-1]) / np.diff(idx)
>>> result2[:, 1] = np.add.reduceat(sorted_A[:, 3]**2, idx[:-1]) / np.diff(idx) - result2[:, 0]**2
>>> result2[:, 2:5] = np.add.reduceat(sorted_A[:, :3]**2, idx[:-1], axis=0) / np.diff(idx)[:, None]
>>> result2[:, 2:5] -= (np.add.reduceat(sorted_A[:, :3], idx[:-1], axis=0) / np.diff(idx)[:, None])**2
>>>
>>> np.allclose(result, result2)
True
Note that the diagonal of the covariance matrix are just the variances which simplifies this vectorization quite a bit.
I can use numpy's vectorize function to create an array of objects of some arbitrary class:
import numpy as np
class Body:
"""
Simple class to represent a point mass in 2D space, more to
play with numpy than anything else...
"""
def __init__(self, position, mass, velocity):
self.position = position
self.mass = mass
self.velocity = velocity
def __repr__(self):
return "m = {} p = {} v = {}".format(self.mass,
self.position, self.velocity)
if __name__ == '__main__':
positions = np.array([0 + 0j, 1 + 1j, 2 + 0j])
masses = np.array([2, 5, 1])
velocities = np.array([0 + 0j, 0 + 1j, 1 + 0j])
vBody = np.vectorize(Body)
points = vBody(positions, masses, velocities)
Now, if I wanted to retrieve a vector containing (say) the velocities from the points array, I could just use an ordinary Python list comprehension
v = [p.velocity for p in points]
But is there a numpy-thonic way to do it? On large arrays would this be more efficient than using a list comprehension?
So, I would encourage you not to use numpy arrays with an object dtype. However, what you have here is essentially a struct, so you could use numpy to your advantage using a structured array. So, first, create a dtype:
>>> import numpy as np
>>> bodytype = np.dtype([('position', np.complex), ('mass', np.float), ('velocity', np.complex)])
Then, initialize your body array:
>>> bodyarray = np.zeros((len(positions),), dtype=bodytype)
>>> bodyarray
array([(0j, 0.0, 0j), (0j, 0.0, 0j), (0j, 0.0, 0j)],
dtype=[('position', '<c16'), ('mass', '<f8'), ('velocity', '<c16')])
Now, you can set your values easily:
>>> positions = np.array([0 + 0j, 1 + 1j, 2 + 0j])
>>> masses = np.array([2, 5, 1])
>>> velocities = np.array([0 + 0j, 0 + 1j, 1 + 0j])
>>> bodyarray['position'] = positions
>>> bodyarray['mass'] = masses
>>> bodyarray['velocity'] = velocities
And now you have an array of "bodies" that can take full advantage of numpy as well as letting you access "attributes" like this:
>>> bodyarray
array([(0j, 2.0, 0j), ((1+1j), 5.0, 1j), ((2+0j), 1.0, (1+0j))],
dtype=[('position', '<c16'), ('mass', '<f8'), ('velocity', '<c16')])
>>> bodyarray['mass']
array([ 2., 5., 1.])
>>> bodyarray['velocity']
array([ 0.+0.j, 0.+1.j, 1.+0.j])
>>> bodyarray['position']
array([ 0.+0.j, 1.+1.j, 2.+0.j])
>>>
Note here,
>>> bodyarray.shape
(3,)
The straight forward list comprehension approach to creating points:
In [285]: [Body(p,m,v) for p,m,v in zip(positions, masses,velocities)]
Out[285]: [m = 2 p = 0j v = 0j, m = 5 p = (1+1j) v = 1j, m = 1 p = (2+0j) v = (1+0j)]
In [286]: timeit [Body(p,m,v) for p,m,v in zip(positions, masses,velocities)]
100000 loops, best of 3: 6.74 µs per loop
For this purpose, creating an array of objects, the frompyfunc is faster than np.vectorize (though you should use otypes with vectorize).
In [287]: vBody = np.frompyfunc(Body,3,1)
In [288]: vBody(positions, masses, velocities)
Out[288]:
array([m = 2 p = 0j v = 0j, m = 5 p = (1+1j) v = 1j,
m = 1 p = (2+0j) v = (1+0j)], dtype=object)
vectorize is slower than the comprehension, but this frompyfunc version is competitive
In [289]: timeit vBody(positions, masses, velocities)
The slowest run took 12.26 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 8.56 µs per loop
vectorize/frompyfunc adds some useful functionality with broadcasting. For example by using ix_, I can generate a cartesian product of your 3 inputs, and 3d set of points, not just 3:
In [290]: points = vBody(*np.ix_(positions, masses, velocities))
In [291]: points.shape
Out[291]: (3, 3, 3)
In [292]: points
Out[292]:
array([[[m = 2 p = 0j v = 0j, m = 2 p = 0j v = 1j, m = 2 p = 0j v = (1+0j)],
....
[m = 1 p = (2+0j) v = 0j, m = 1 p = (2+0j) v = 1j,
m = 1 p = (2+0j) v = (1+0j)]]], dtype=object)
In [293]:
In short, a 1d object array has few advantages compared to a list; it's only when you need to organize the objects in 2 or more dimensions that these arrays have advantages.
As for accessing attributes, you have either use list comprehension, or the equivalent vectorize operations.
[x.position for x in points.ravel()]
Out[294]:
[0j,
0j,
0j,
...
(2+0j),
(2+0j)]
In [295]: vpos = np.frompyfunc(lambda x:x.position,1,1)
In [296]: vpos(points)
Out[296]:
array([[[0j, 0j, 0j],
[0j, 0j, 0j],
...
[(2+0j), (2+0j), (2+0j)],
[(2+0j), (2+0j), (2+0j)]]], dtype=object)
In Tracking Python 2.7.x object attributes at class level to quickly construct numpy array
explores some alternative ways of storing/accessing object attributes.
I'd like to add two numpy arrays of different shapes, but without broadcasting, rather the "missing" values are treated as zeros. Probably easiest with an example like
[1, 2, 3] + [2] -> [3, 2, 3]
or
[1, 2, 3] + [[2], [1]] -> [[3, 2, 3], [1, 0, 0]]
I do not know the shapes in advance.
I'm messing around with the output of np.shape for each, trying to find the smallest shape which holds both of them, embedding each in a zero-ed array of that shape and then adding them. But it seems rather a lot of work, is there an easier way?
Thanks in advance!
edit: by "a lot of work" I meant "a lot of work for me" rather than for the machine, I seek elegance rather than efficiency: my effort getting the smallest shape holding them both is
def pad(a, b) :
sa, sb = map(np.shape, [a, b])
N = np.max([len(sa),len(sb)])
sap, sbp = map(lambda x : x + (1,)*(N-len(x)), [sa, sb])
sp = np.amax( np.array([ tuple(sap), tuple(sbp) ]), 1)
not pretty :-/
I'm messing around with the output of np.shape for each, trying to find the smallest shape which holds both of them, embedding each in a zero-ed array of that shape and then adding them. But it seems rather a lot of work, is there an easier way?
Getting the np.shape is trivial, finding the smallest shape that holds both is very easy, and of course adding is trivial, so the only "a lot of work" part is the "embedding each in a zero-ed array of that shape".
And yes, you can eliminate that, by just calling the resize method (or the resize function, if you want to make copies instead of changing them in-place). As the docs explain:
Enlarging an array: … missing entries are filled with zeros
For example, if you know the dimensionality statically:
>>> a1 = np.array([[1, 2, 3], [4, 5, 6]])
>>> a2 = np.array([[2], [2]])
>>> shape = [max(a.shape[axis] for a in (a1, a2)) for axis in range(2)]
>>> a1.resize(shape)
>>> a2.resize(shape)
>>> print(a1 + a2)
array([[3, 4, 3],
[4, 5, 6]])
This is the best I could come up with:
import numpy as np
def magic_add(*args):
n = max(a.ndim for a in args)
args = [a.reshape((n - a.ndim)*(1,) + a.shape) for a in args]
shape = np.max([a.shape for a in args], 0)
result = np.zeros(shape)
for a in args:
idx = tuple(slice(i) for i in a.shape)
result[idx] += a
return result
You can clean up the for loop a little if you know how many dimensions you expect on result, something like:
for a in args:
i, j = a.shape
result[:i, :j] += a
You may try my solution - for dimension 1 arrays you have to expand your arrays to
dimension 2 (as shown in the example below), before passing it to the function.
import numpy as np
import timeit
matrix1 = np.array([[0,10],
[1,20],
[2,30]])
matrix2 = np.array([[0,10],
[1,20],
[2,30],
[3,40]])
matrix3 = np.arange(0,0,dtype=int) # empty numpy-array
matrix3.shape = (0,2) # reshape to 0 rows
matrix4 = np.array([[0,10,100,1000],
[1,20,200,2000]])
matrix5 = np.arange(0,4000,1)
matrix5 = np.reshape(matrix5,(4,1000))
matrix6 = np.arange(0.0,4000,0.5)
matrix6 = np.reshape(matrix6,(20,400))
matrix1 = np.array([1,2,3])
matrix1 = np.expand_dims(matrix1, axis=0)
matrix2 = np.array([2,1])
matrix2 = np.expand_dims(matrix2, axis=0)
def add_2d_matrices(m1, m2, pos=(0,0), filler=None):
"""
Add two 2d matrices of different sizes or shapes,
offset by xy coordinates, whereat x is "from left to right" (=axis:1)
and y is "from top to bottom" (=axis:0)
Parameterse:
- m1: first matrix
- m2: second matrix
- pos: tuple (x,y) containing coordinates for m2 offset,
- filler: gaps are filled with the value of filler (or zeros)
Returns:
- 2d array (float):
containing filler-values, m1-values, m2-values
or the sum of m1,m2 (at overlapping areas)
Author:
Reinhard Daemon, Austria
"""
# determine shape of final array:
_m1 = np.copy(m1)
_m2 = np.copy(m2)
x,y = pos
y1,x1 = _m1.shape
y2,x2 = _m2.shape
xmax = max(x1, x2+x)
ymax = max(y1, y2+y)
# fill-up _m1 array with zeros:
y1,x1 = _m1.shape
diff = xmax - x1
_z = np.zeros((y1,diff))
_m1 = np.hstack((_m1,_z))
y1,x1 = _m1.shape
diff = ymax - y1
_z = np.zeros((diff,x1))
_m1 = np.vstack((_m1,_z))
# shift _m2 array by 'pos' and fill-up with zeros:
y2,x2 = _m2.shape
_z = np.zeros((y2,x))
_m2 = np.hstack((_z,_m2))
y2,x2 = _m2.shape
diff = xmax - x2
_z = np.zeros((y2,diff))
_m2 = np.hstack((_m2,_z))
y2,x2 = _m2.shape
_z = np.zeros((y,x2))
_m2 = np.vstack((_z,_m2))
y2,x2 = _m2.shape
diff = ymax - y2
_z = np.zeros((diff,x2))
_m2 = np.vstack((_m2,_z))
# add the 2 arrays:
_m3 = _m1 + _m2
# find and fill the "unused" positions within the summed array:
if filler not in (None,0,0.0):
y1,x1 = m1.shape
y2,x2 = m2.shape
x1min = 0
x1max = x1-1
y1min = 0
y1max = y1-1
x2min = x
x2max = x + x2-1
y2min = y
y2max = y + y2-1
for xx in range(xmax):
for yy in range(ymax):
if x1min <= xx <= x1max and y1min <= yy <= y1max:
continue
if x2min <= xx <= x2max and y2min <= yy <= y2max:
continue
_m3[yy,xx] = filler
return(_m3)
t1 = timeit.Timer("add_2d_matrices(matrix5, matrix6, pos=(1,1), filler=111.111)", \
"from __main__ import add_2d_matrices,matrix5,matrix6")
print("ran:",t1.timeit(number=10), "milliseconds")
print("\n\n")
my_res = add_2d_matrices(matrix1, matrix2, pos=(1,1), filler=99.99)
print(my_res)