A trick when you want to flatten a matrix X of shape (a,b,c,d) to a matrix X_flatten of shape (b ∗∗ c ∗∗ d, a) is to use:
X_flatten = X.reshape(X.shape[0], -1).T
I read this trick in coursera DL course, how does this work? Where did -1 come from and what does it mean?
X.shape[0] returns the first dimension of your original array:
X = np.random.rand(4, 4, 4, 4)
print(X.shape)
results in
(4, 4, 4, 4)
and therefore
X.shape[0]
returns
4
Using the reshape command, you can omit one of the target matrix dimensions by using -1 as a placeholder,
because one of the dimensions can be inferred by numpy.
I.e. by supplying the 4 from X.shape[0], numpy knows what the remaining first dimension must be for the array to contain all your values.
In the example
new_X = X.reshape(X.shape[0], -1).T
print(new_X.shape)
it is
(64, 4)
which would be equivalent to calling
new_X = X.reshape(X.shape[0], 64).T
print(new_X.shape)
The .T function just transposes the array resulting from the reshape command.
We can do the same in, Using the basic concepts of python
nested_list=[10,20,[30,40,[50]],[80,[10,[20]],90],60]
flat_list=[]
def unpack(list1):
for item in list1:
try:
len(item)
unpack(item)
except:
flat_list.append(item)
unpack(nested_list)
print (flat_list)
Related
I want to get dot product of two arrays along the batch dimension. np.dot gave a super weird result. Let suppose I have a batch of size 2. So what would be the proper way to get the results?
X = np.random.randn(2,3,4)
X_t = np.transpose(X,axes=[0,2,1]) # shape now is [2,4,3]
np.matmul(X,X_t) # shape is [2,3,3]
np.dot(X,X_t) # shape is [2,3,2,3] SUPER Weird
np.einsum('ijk,ikl->ijl',X,X_t) # Dimension as [2,3,3] Same as Matmul()
What is the correct way of matrix multiplication for conditions like these?
Use # operator. It reduces the first (0th) dimention.
Matmul for other dims.
import numpy as np
x = np.random.randn(2, 3, 4)
x_t = np.transpose(x, axes=[0, 2, 1]) # shape now is [2,4,3]
wrong = np.dot(x, x_t) # shape is [2,3,2,3] SUPER Weird
res = x # x_t
print(res.shape)
print(wrong.shape)
out:
(2, 3, 3)
(2, 3, 2, 3)
This question already has answers here:
Transposing a 1D NumPy array
(15 answers)
Closed 3 years ago.
Let's consider a as an 1D row/horizontal array:
import numpy as np
N = 10
a = np.arange(N) # array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
a.shape # (10,)
now I want to have b a 1D column/vertical array transposed of a:
b = a.transpose() # array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
b.shape # (10,)
but the .transpose() method returns an identical ndarray whith the exact same shape!
What I expected to see was
np.array([[0], [1], [2], [3], [4], [5], [6], [7], [8], [9]])
which can be achieved by
c = a.reshape(a.shape[0], 1) # or c = a; c.shape = (c.shape[0], 1)
c.shape # (10, 1)
and to my surprise, it has a shape of (10, 1) instead of (1, 10).
In Octave/Scilab I could do:
N = 10
b = 0:(N-1)
a = b'
size(b) % ans = 1 10
size(a) % ans = 10 1
I understand that numpy ndarrays are not matrices (as discussed here), but the behavior of the numpy's transpose function just doesn't make sense to me! I would appreciate it if you could help me understand how this behavior makes sense and what am I missing here.
P.S. So what I have understood so far is that b = a.transpose() is the equivalent of b = a; b.shape = b.shape[::-1] which if you had a "2D array" of (N, 1) would return a (1, N) shaped array, as you would expect from a transpose operator. However, numpy seems to treat the "1D array" of (N,) as a 0D scalar. I think they should have named this method something else, as this is very misleading/confusing IMHO.
To understand the numpy array better, you should take a look at this review paper: The NumPy array: a structure for efficient numerical computation
In short, numpy ndarrays have this attribute called the stride, which is
the number of bytes to skip in memory to proceed to the next element.
For a (10, 10) array of bytes, for example, the strides may be (10,
1), in other words: proceed one byte to get to the next column and ten
bytes to locate the next row.
For your ndarray a, a.stride = (8,), which shows that it is only 1 dimensional, and that to get to the next element on this single dimension, you need to advance 8 bytes in memory (each int is 64-bit).
Strides are useful for representing transposes:
By modifying strides, for example, an array can be transposed or
reshaped at zero cost (no memory needs to be copied).
So if there was a 2-dimensional ndarray, say b = np.ones((3,5)) for example, then b.strides = (40, 8), while b.transpose().strides = (8, 40). So as you see a transposed 2D-ndarray is simply the exact same array, whose strides have been reordered. And since your 1D ndarray has only 1 dimension, swapping the the values of its strides (i.e. taking its transpose), doesn't do anything.
As you already mentioned that numpy array are not matrix. The defination of transpose function is like below
Permute the dimensions of an array.
Which means that numpy's transpose method will move data from one dimension to another. As 1D array has only one dimension there is no other dimension to move the data t0. So you need add a dimension before transpose has any effect. This behavior make sense also to be consistent with higher dimensional array (3D, 4D ...) array.
There is a clean way to achive what you want
N = 10
a = np.arange(N)
a[ :, np.newaxis]
I am a newbie in python. I have a question about the dimension of array.
I have (10,192,192,1) array which type is (class 'numpy.ndarray').
I would like to divid this array to 10 separated array like 10 * (1,192,192,1). but I always got (192,192,1) array when I separate.
How can I get separated arrays as a same dimension type of original one?
below is my code.
b = np.ndarray((a.shape[0],a.shape[1],a.shape[2],a.shape[3]))
print(b.shape) # (10,192,192,1)
for i in range(a.shape[0]):
b[i] = a[i]
print(b[i].shape) # (192,192,1), but I want to get (1,192,192,1)
you can reshape it using np.array()
b = np.zeros((192,192,1))
print(b.shape) #(192, 192, 1)
print(np.array([b]).shape) #(1, 192, 192, 1)
Simply use the numpy .reshape() function:
b[i].reshape((1,192,192,1))
Check out the docs here
For example:
>>> x = np.zeros((13,24))
>>> x.shape
(13,24)
>>> x.resize((1,13,24)).shape
(1,13,24)
Python numpy array 'size' confuses me a lot
a = np.array([1,2,3])
a.size = (3, )
------------------------
b = np.array([[2,1,3,5],
[2,2,5,1],
[3,6,99,5]])
b.size = (3,4)
'b' makes sense since it has 3 rows and 4 columns in each
But how is 'a' size = (3, ) ? Shouldn't it be (1,3) since its 1 row and 3 columns?
You should resist the urge to think of numpy arrays as having rows and columns, but instead consider them as having dimensions and shape. This is an important point which differentiates np.array and np.matrix:
x = np.array([1, 2, 3])
print(x.ndim, x.shape) # 1 (3,)
y = np.matrix([1, 2, 3])
print(y.ndim, y.shape) # 2 (1, 3)
An n-D array can only use n integer(s) to represent its shape. Therefore, a 1-D array only uses 1 integer to specify its shape.
In practice, combining calculations between 1-D and 2-D arrays is not a problem for numpy, and syntactically clean since # matrix operation was introduced in Python 3.5. Therefore, there is rarely a need to resort to np.matrix in order to satisfy the urge to see expected row and column counts.
In the rare instances where 2 dimensions are required, you can still use numpy.array with some manipulation:
a = np.array([1, 2, 3])[:, None] # equivalent to np.array([[1], [2], [3]])
print(a.ndim, a.shape) # 2 (3, 1)
b = np.array([[1, 2, 3]]) # equivalent to np.array([1, 2, 3])[:, None].T
print(b.ndim, b.shape) # 2 (1, 3)
No, a numpy.ndarray with shape (1, 3) would look like:
np.array([[1,2,3]])
Think about how the shape corresponds to indexing:
arr[0, ...] #First row
I still have three more options, namely:
arr[0,0]
arr[0,1]
arr[0,2]
Try doing that with a 1 dimensional array
I think you meant ndarray.shape. In that case, there's no need for confusion. Quoting the documentation from ndarray.shape:
Tuple of array dimensions.
ndarray.shape simply returns a shape tuple.
In [21]: a.shape
Out[21]: (3,)
This simply means that a is an 1D array with 3 entries.
If the shape tuple returns it as (1,3) then a would become a 2D array. For that you need to use:
In [23]: a = a[np.newaxis, :]
In [24]: a.shape
Out[24]: (1, 3)
Since array b is 2D, the shape tuple has two entries.
In [22]: b.shape
Out[22]: (3, 4)
I have simple array like this
x = np.array([1,2,3,4])
In [3]: x.shape
Out[3]: (4,)
But I don't want shape to return (4,), but (4,1). How can I achieve this?
Generally in Numpy you would declare a matrix or vector using two square brackets. It's common misconception to use single square brackets for single dimensional matrix or vector.
Here is an example:
a = np.array([[1,2,3,4], [5,6,7,8]])
a.shape # (2,4) -> Multi-Dimensional Matrix
In similar way if I want single dimensional matrix then just remove the data not the outer square bracket.
a = np.array([[1,2,3,4]])
a.shape # (1,4) -> Row Matrix
b = np.array([[1], [2], [3], [4]])
b.shape # (4, 1) -> Column Matrix
When you use single square brackets, it's likely to give some odd dimensions.
Always enclose your data within another square bracket for such single dimensional matrix (like you are entering the data for multi-dimensional matrix) without data for those extra dimensions.
Also: You could also always reshape
x = np.array([1,2,3,4])
x = x.reshape(4,1)
x.shape # (4,1)
One Line:
x = np.array([1,2,3,4]).reshape(4,1)
x.shape # (4,1)
If you want a column vector use
x2 = x[:, np.newaxis]
x2.shape # (4, 1)
Alternatively, you could reshape the array yourself:
arr1 = np.array([1,2,3,4])
print arr1.shape
# (4,)
arr2 = arr1.reshape((4,1))
print arr2.shape
# (4, 1)
You could of course reshape the array when you create it:
arr1 = np.array([1,2,3,4]).reshape((4,1))
If you want to change the array in place as suggested by #FHTMitchell in the comments:
arr1.resize((4, 1))
Below achieves what you want. However, I strongly suggest you look at why exactly you need shape to return (4, 1). Most matrix-type operations are possible without this explicit casting.
x = np.array([1,2,3,4])
y = np.matrix(x)
z = y.T
x.shape # (4,)
y.shape # (1, 4)
z.shape # (4, 1)
You can use zip to transpose at python (non-numpy) level:
>>> a = [1, 2, 3, 4]
>>>
>>> *zip(a),
((1,), (2,), (3,), (4,))
>>>
>>> import numpy as np
>>> np.array([*zip(a)])
array([[1],
[2],
[3],
[4]])
Please note that while this is convenient in terms of key strokes it is a bit wasteful given that a tuple object has to be constructed for every list element whereas reshaping an array comes essentially for free. So do not use this on long lists.