We Keep Coding

Append zeros to array

I want to create an array that initially has these values:
[[[0,0,0,0]], [[0,0,0,0]]]
Then appending zeros to the next row:
[[[0,0,0,0], [0,0,0,0]], [[0,0,0,0], [0,0,0,0]]]
and again similarly appending zeros to the next row in a loop:
[[[0,0,0,0], [0,0,0,0], ...], [[0,0,0,0], [0,0,0,0], ...]]
How can I do this with python?
I tried
import numpy, pandas
x = numpy.array([[[0, 0, 0, 0]], [[0, 0, 0, 0]]])
numpy.append(x[0], [[0, 0, 0, 0]], axis=0)
print(x)

What you're looking for might be numpy.insert(arr, obj, values, axis=None) to "Insert values along the given axis before the given indices". In your case, in the array A (arr) you want to insert zeros (values) along axis 1 (axis=1). If you want to insert the element at the beginning, specify the position 0 (obj), but if you want it at the end use -1:
>>> A = np.array([[[0,0,0,0]], [[0,0,0,0]]])
>>> A.shape
(2, 1, 4)
>>> B = np.insert(A, 1, 0, axis=1)
>>> B.shape
(2, 2, 4)

In [160]: x = numpy.array([[[0, 0, 0, 0]], [[0, 0, 0, 0]]])
In [161]: x
Out[161]:
array([[[0, 0, 0, 0]],
[[0, 0, 0, 0]]])
In [162]: x.shape
Out[162]: (2, 1, 4)
Let's be clear what you start with - note the shape!
Your append attempt:
In [163]: np.append(x[0], [[0, 0, 0, 0]], axis=0)
Out[163]:
array([[0, 0, 0, 0],
[0, 0, 0, 0]])
In [164]: _.shape
Out[164]: (2, 4)
It makes a NEW array - x[0] is (1,4) array, and you join another (1,4) to make a (2,4). np.append when used like this is just np.concatenate. Read, and reread, the docs if necessary.
It did not change x. np.append is NOT a list append clone.
In [165]: x
Out[165]:
array([[[0, 0, 0, 0]],
[[0, 0, 0, 0]]])
If you must work incrementally, use list and list append. Arrays have a fixed size, so 'growing' them requires making a new array.

What's difference between np.zeros(img) and np.copy(img)*0?

What is the difference between these three:
np.zeros(img), np.zeros_like(img), and np.copy(img)*0?
When should I use each of them?

zeros is the basic operation, creating an array with a given shape and dtype:
In [313]: arr = np.zeros((3,4),int)
In [314]: arr
Out[314]:
array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
zeros_like is an alternative input to zeros, where you provide an array, and it copies its shape and dtype (but not values):
In [315]: bar = np.zeros_like(arr)
In [316]: bar
Out[316]:
array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
copy*0 also works, but generally I wouldn't recommend it. I did use N2 *=0 recently in while testing some code to reset an array.
In [317]: car = arr.copy()*0
In [318]: car
Out[318]:
array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])

Despite they can reach a similar effect of creating a zero NumPy array of the shape you intended. Their performances are significantly different.
As shown in the screenshot below, np.zeros_like(x) is slightly slower than np.zeros(x.shape) while np.copy(x)*0 is the slowest of all.
For example, if you increase the array size, you can really see the difference here:

Numpy - Matrix multiplication to return ndarray, not sum

All, I have an application that requires returning a numpy ndarray, rather than a simple sum, when multiplying two matrices; e.g.:
import numpy as np
x = np.array([[1, 1, 0], [0, 1, 1]])
y = np.array([[1, 0, 0, 1], [1, 0, 1, 0], [0, 0, 0, 0]])
w = x # y
>>> array([[2, 0, 1, 1],
[1, 0, 1, 0]])
However, the requirement is to return an ndarray (in this case..):
array([[[1,1,0], [0,0,0], [0,1,0], [1,0,0]],
[[0,1,0], [0,0,0], [0,1,0], [0,0,0]]])
Note that the matrix multiplication operation may be repeated; the output will be used as the left-side matrix of ndarrays for the next matrix multiplication operation, which would yield a higher-order ndarray after the second matrix multiplication operation, etc..
Any way to achieve this? I've looked at overloading __add__, and __radd__ by subclassing np.ndarray as discussed here, but mostly got dimension incompatibility errors.
Ideas?
Update:
Addressing #Divakar's answer E.g., for chained operation, adding
z = np.array([[1, 1, 0], [0, 0, 0], [1, 0, 0], [0, 1, 0]])
s1 = x[...,None] * y
s2 = s1[...,None] * z
results in an undesired output.
I suspect the issue starts with s1, which in the case above returns s1.shape = (2,3,4). It should be (2,4,3) since [2x3][3x4] = [2x4], but we're not really summing here, just return an array of length 3.
Similarly, s2.shape should be (2,3,4,3), which [incidentally] it is, but with undesired output (it's not 'wrong', just not what we're looking for).
To elaborate, s1*z should be [2x4][4x3] = [2x3] matrix. Each element of the matrix is itself an ndarray, of [4x3] since we have 4 rows in z to multiply the elements in s1, and each element in s1 is itself 3 elements long (again, we're not arithmetically adding elements, but return ndarrays with the extended dimension being the row count in the R-matrix of the operation.
Ultimately, the desired output would be:
s2 = array([[[[1, 1, 0],
[0, 0, 0],
[0, 1, 0],
[0, 0, 0]],
[[1, 1, 0],
[0, 0, 0],
[0, 0, 0],
[1, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]],
[[[0, 1, 0],
[0, 0, 0],
[0, 1, 0],
[0, 0, 0]],
[[0, 1, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]]])

Extend them to 3D and leverage broadcasting -
x[:,None] * y.T
Or with np.einsum -
np.einsum('ij,jk->ikj',x,y)
Going by OP's comment and the quote from the question :
... matrix multiplication operation may be repeated; the output will
be used as the left-side matrix of ndarrays for the next matrix
multiplication operation, which would yield a higher-order ndarray
after the second matrix multiplication operation, etc..
It seems, we need to do something along these lines -
s1 = x[...,None] * y
s2 = s1[...,None] * z # and so on.
Though, the order of the axes would be different in this case, but it seems to be the simplest way to extend the solution to a generic number of incoming 2D arrays.
Following the edits in the question, seems like you are placing the incoming arrays from the first axis onwards for element-wise multiplication. So, if I got that right, you can swap axes to get the correct order, like so -
s1c = (x[...,None] * y).swapaxes(1,-1)
s2c = (s1c.swapaxes(1,-1)[...,None] * z).swapaxes(1,-1) # and so on.
If you are only interested in the final output, swap axes only at the final stage and skip those in the intermediate ones.

Numpy: increment elements of an array given the indices required to increment

I am trying to turn a second order tensor into a binary third order tensor. Given a second order tensor as a m x n numpy array: A, I need to take each element value: x, in A and replace it with a vector: v, with dimensions equal to the maximum value of A, but with a value of 1 incremented at the index of v corresponding to the value x (i.e. v[x] = 1). I have been following this question: Increment given indices in a matrix, which addresses producing an array with increments at indices given by 2 dimensional coordinates. I have been reading the answers and trying to use np.ravel_multi_index() and np.bincount() to do the same but with 3 dimensional coordinates, however I keep on getting a ValueError: "invalid entry in coordinates array". This is what I have been using:
def expand_to_tensor_3(array):
(x, y) = array.shape
(a, b) = np.indices((x, y))
a = a.reshape(x*y)
b = b.reshape(x*y)
tensor_3 = np.bincount(np.ravel_multi_index((a, b, array.reshape(x*y)), (x, y, np.amax(array))))
return tensor_3
If you know what is wrong here or know an even better method to accomplish my goal, both would be really helpful, thanks.

You can use (A[:,:,np.newaxis] == np.arange(A.max()+1)).astype(int).
Here's a demonstration:
In [52]: A
Out[52]:
array([[2, 0, 0, 2],
[3, 1, 2, 3],
[3, 2, 1, 0]])
In [53]: B = (A[:,:,np.newaxis] == np.arange(A.max()+1)).astype(int)
In [54]: B
Out[54]:
array([[[0, 0, 1, 0],
[1, 0, 0, 0],
[1, 0, 0, 0],
[0, 0, 1, 0]],
[[0, 0, 0, 1],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]],
[[0, 0, 0, 1],
[0, 0, 1, 0],
[0, 1, 0, 0],
[1, 0, 0, 0]]])
Check a few individual elements of A:
In [55]: A[0,0]
Out[55]: 2
In [56]: B[0,0,:]
Out[56]: array([0, 0, 1, 0])
In [57]: A[1,3]
Out[57]: 3
In [58]: B[1,3,:]
Out[58]: array([0, 0, 0, 1])
The expression A[:,:,np.newaxis] == np.arange(A.max()+1) uses broadcasting to compare each element of A to np.arange(A.max()+1). For a single value, this looks like:
In [63]: 3 == np.arange(A.max()+1)
Out[63]: array([False, False, False, True], dtype=bool)
In [64]: (3 == np.arange(A.max()+1)).astype(int)
Out[64]: array([0, 0, 0, 1])
A[:,:,np.newaxis] is a three-dimensional view of A with shape (3,4,1). The extra dimension is added so that the comparison to np.arange(A.max()+1) will broadcast to each element, giving a result with shape (3, 4, A.max()+1).
With a trivial change, this will work for an n-dimensional array. Indexing a numpy array with the ellipsis ... means "all the other dimensions". So
(A[..., np.newaxis] == np.arange(A.max()+1)).astype(int)
converts an n-dimensional array to an (n+1)-dimensional array, where the last dimension is the binary indicator of the integer in A. Here's an example with a one-dimensional array:
In [6]: a = np.array([3, 4, 0, 1])
In [7]: (a[...,np.newaxis] == np.arange(a.max()+1)).astype(int)
Out[7]:
array([[0, 0, 0, 1, 0],
[0, 0, 0, 0, 1],
[1, 0, 0, 0, 0],
[0, 1, 0, 0, 0]])

You can make it work this way:
tensor_3 = np.bincount(np.ravel_multi_index((a, b, array.reshape(x*y)),
(x, y, np.amax(array) + 1)))
The difference is that I add 1 to the amax() result, because ravel_multi_index() expects that the indexes are all strictly less than the dimensions, not less-or-equal.
I'm not 100% sure if this is what you wanted; another way to make the code run is to specify mode='clip' or mode='wrap' in ravel_multi_index(), which does something a bit different and I'm guessing is less correct. But you can try it.

matlab find() for nonzero element in python

I have a sparse matrix (numpy.array) and I would like to have the index of the nonzero elements in it.
In Matlab I would write:
[i, j] = find(CM)
and in Python what should I do?
I have tried numpy.nonzero (but I don't know how to take the indices from that) and flatnonzero (but it's not convenient for me, I need both the row and column index).
Thanks in advance!

Assuming that by "sparse matrix" you don't actually mean a scipy.sparse matrix, but merely a numpy.ndarray with relatively few nonzero entries, then I think nonzero is exactly what you're looking for. Starting from an array:
>>> a = (np.random.random((5,5)) < 0.10)*1
>>> a
array([[0, 0, 0, 0, 0],
[0, 0, 0, 0, 1],
[0, 0, 1, 0, 0],
[1, 0, 0, 0, 0],
[0, 0, 0, 0, 0]])
nonzero returns the indices (here x and y) where the nonzero entries live:
>>> a.nonzero()
(array([1, 2, 3]), array([4, 2, 0]))
We can assign these to i and j:
>>> i, j = a.nonzero()
We can also use them to index back into a, which should give us only 1s:
>>> a[i,j]
array([1, 1, 1])
We can even modify a using these indices:
>>> a[i,j] = 2
>>> a
array([[0, 0, 0, 0, 0],
[0, 0, 0, 0, 2],
[0, 0, 2, 0, 0],
[2, 0, 0, 0, 0],
[0, 0, 0, 0, 0]])
If you want a combined array from the indices, you can do that too:
>>> np.array(a.nonzero()).T
array([[1, 4],
[2, 2],
[3, 0]])
(there are lots of ways to do this reshaping; I chose one almost at random.)

This goes slightly beyond what you as and I only mention it since I once faced a similar problem. If you want the indices to access some other array there is some very simple sytax:
import numpy as np
array = np.random.randint(0, 2, size=(3, 3))
data = np.random.random(size=(3, 3))
Now array looks something like
>>> print array
array([[0, 1, 0],
[1, 0, 1],
[1, 1, 0]])
while data could be
>>> print data
array([[ 0.92824816, 0.43605604, 0.16627849],
[ 0.00301434, 0.94342538, 0.95297402],
[ 0.32665135, 0.03504204, 0.86902492]])
Then if we want the elements of data which are zero:
>>> print data[array==0]
array([ 0.92824816, 0.16627849, 0.94342538, 0.86902492])
Which is nice and simple.