This question already has answers here:
Efficient Numpy 2D array construction from 1D array
(7 answers)
Closed 6 years ago.
I have an array like
[1,3,4,5,6,2,1,,,,]
now, I want to change it to
[[1,3,4],[3,4,5],[4,5,6],[5,6,2],,,,]
How can I achieve this using numpy? Is there any function to do so? And, using loop is not an option.
np.lib.stride_tricks.as_strided method will does that.
Here strides is (4,4) for 32 bit int. If you want more flexible code, I have commented stride parameter in the code. shape parameter determines output array dimensions.
> import numpy as np
> A = [1,2,3,4,5,6]
> n = 3 # output matrix has 3 columns
> m = len(A) - (n-1) # calculate number of output matrix rows using input matrix length
> # strides_param = np.array(A, dtype=np.int32).strides * 2
> np.lib.stride_tricks.as_strided(A, shape=(m,n), strides=(4,4))
array([[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6]])
Using list comprehension instead of a library but why make it complicated. (Do you consider this a loop?)
x = [1,3,4,5,6,2,1]
y = [x[n:n+3] for n in range(len(x)-2)]
Result is:
[[1, 3, 4], [3, 4, 5], [4, 5, 6], [5, 6, 2], [6, 2, 1]]
Related
Hello I'm new in python I couldn't solve my problem. Suppose I have a list (a), this list has many matricies which is the same shape. I want to get one matrix that result of mean of each elements.
here is the list and its elements:
a[0]=[1 2 3]
a[1]=[3 4 5]
a[2]=[6 7 8]
Here is the desired matrix:
mean=[10/3 13/3 16/3]
Mean of each element of a list of matrices
Actually, this answer is good for me but it's for the R, not python. Sorry if I made a mistake while asking a question.
Using Python list comprehension
a = [[1, 2, 3],
[3, 4, 5],
[6, 7, 8]]
mean = [sum(row)/len(row) for row in zip(*a)] # use zip(*a) to transpose matrix
# since sum along columns
# by summing rows of transposed a
# [3.3333333333333335, 4.333333333333333, 5.333333333333333]
Here is a pure python solution that would work with any matrice dimension:
matrice = [
[1, 2, 3],
[3, 4, 5],
[6, 7, 8]
]
def mean_mat(mat):
dim_axis_0 = mat.__len__()
mean = [0 for i in range(dim_axis_0)]
for vector in mat:
for i, value in enumerate(vector):
mean[i] += (value / dim_axis_0)
return mean
print(mean_mat(matrice))
>>> [3.333333333333333, 4.333333333333334, 5.333333333333334]
However, as user1740577 pointed out, you should checkout the Numpy library.
try this:
import numpy as np
a= [[1,2,3],[3,4,5],[6,7,8]]
np.mean(a, axis=0)
# array([3.33333333, 4.33333333, 5.33333333])
This question already has an answer here:
how to reshape an N length vector to a 3x(N/3) matrix in numpy using reshape
(1 answer)
Closed 2 years ago.
I have an array: [1, 2, 3, 4, 5, 6]. I would like to use the numpy.reshape() function so that I end up with this array:
[[1, 4],
[2, 5],
[3, 6]
]
I'm not sure how to do this. I keep ending up with this, which is not what I want:
[[1, 2],
[3, 4],
[5, 6]
]
These do the same thing:
In [57]: np.reshape([1,2,3,4,5,6], (3,2), order='F')
Out[57]:
array([[1, 4],
[2, 5],
[3, 6]])
In [58]: np.reshape([1,2,3,4,5,6], (2,3)).T
Out[58]:
array([[1, 4],
[2, 5],
[3, 6]])
Normally values are 'read' across the rows in Python/numpy. This is call row-major or 'C' order. Read down is 'F', for FORTRAN, and is common in MATLAB, which has Fortran roots.
If you take the 'F' order, make a new copy and string it out, you'll get a different order:
In [59]: np.reshape([1,2,3,4,5,6], (3,2), order='F').copy().ravel()
Out[59]: array([1, 4, 2, 5, 3, 6])
You can set the order in np.reshape, in your case you can use 'F'. See docs for details
>>> arr
array([1, 2, 3, 4, 5, 6])
>>> arr.reshape(-1, 2, order = 'F')
array([[1, 4],
[2, 5],
[3, 6]])
The reason that you are getting that particular result is that arrays are normally allocates in C order. That means that reshaping by itself is not sufficient. You have to tell numpy to change the order of the axes when it steps along the array. Any number of operations will allow you to do that:
Set the axis order to F. F is for Fortran, which, like MATLAB, conventionally uses column-major order:
a.reshape(2, 3, order='F')
Swap the axes after reshaping:
np.swapaxes(a.reshape(2, 3), 0, 1)
Transpose the result:
a.reshape(2, 3).T
Roll the second axis forward:
np.rollaxis(a.reshape(2, 3), 1)
Notice that all but the first case require you to reshape to the transpose.
You can even manually arrange the data
np.stack((a[:3], a[3:]), axis=1)
Note that this will make many unnecessary copies. If you want the data copied, just do
a.reshape(2, 3, order='F').copy()
I'm trying to speed up a process, I think this might be possible using numpy's apply_along_axis. The problem is that not all my axis have the same length.
When I do:
a = np.array([[1, 2, 3],
[2, 3, 4],
[4, 5, 6]])
b = np.apply_along_axis(sum, 1, a)
print(b)
This works fine. But I would like to do something similar to (please note that the first row has 4 elements and the rest have 3):
a = np.array([[1, 2, 3, 4],
[2, 3, 4],
[4, 5, 6]])
b = np.apply_along_axis(sum, 1, a)
print(b)
But this fails because:
numpy.AxisError: axis 1 is out of bounds for array of dimension 1
I've looked around and the only 'solution' I've found is to add zeros to make all the arrays the same length, which would probably defeat the purpose of performance improvement.
Is there any way to use numpy_apply_along_axis on a non-regular shaped numpy array?
You can transform your initial array of iterable-objects to ndarray by padding them with zeros in a vectorized manner:
import numpy as np
a = np.array([[1, 2, 3, 4],
[2, 3, 4],
[4, 5, 6]])
max_len = len(max(a, key = lambda x: len(x))) # max length of iterable-objects contained in array
cust_func = np.vectorize(pyfunc=lambda x: np.pad(array=x,
pad_width=(0,max_len),
mode='constant',
constant_values=(0,0))[:max_len], otypes=[list])
a_pad = np.stack(cust_func(a))
output:
array([[1, 2, 3, 4],
[2, 3, 4, 0],
[4, 5, 6, 0]])
It depends.
Do you know the size of the vectors before or are you appending to a list?
see e.g. http://stackoverflow.com/a/58085045/7919597
You could for example pad the arrays
import numpy as np
a1 = [1, 2, 3, 4]
a2 = [2, 3, 4, np.nan] # pad with nan
a3 = [4, 5, 6, np.nan] # pad with nan
b = np.stack([a1, a2, a3], axis=0)
print(b)
# you can apply the normal numpy operations on
# arrays with nan, they usually just result in a nan
# in a resulting array
c = np.diff(b, axis=-1)
print(c)
Afterwards you can apply a moving window on each row over the columns.
Have a look at https://stackoverflow.com/a/22621523/7919597 which is only 1d, but can give you an idea of how it could work.
It is possible to use a 2d array with only one row as kernel (shape e.g. (1, 3)) with scipy.signal.convolve2d and use the idea above.
This is a workaround to get a "row-wise 1D convolution":
from scipy import signal
krnl = np.array([[0, 1, 0]])
d = signal.convolve2d(c, krnl, mode='same')
print(d)
This question already has answers here:
numpy subtract every row of matrix by vector
(3 answers)
numpy subtract/add 1d array from 2d array
(2 answers)
Closed 5 years ago.
I have two numpy array a and b
a=np.array([[1,2,3],[4,5,6],[7,8,9]])
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
b = np.array([1,2,3])
array([1, 2, 3])
I would like to substract to each row of a the correspondent element of b (ie. to the first row of a, the first element of b, etc)
so that c is
array([[0, 1, 2],
[2, 3, 4],
[4, 5, 6]])
Is there a python command to do this?
Is there a python command to do this?
Yes, the - operator.
In addition you need to make b into a column vector so that broadcasting can do the rest for you:
a - b[:, np.newaxis]
# array([[0, 1, 2],
# [2, 3, 4],
# [4, 5, 6]])
yup! You just need to make b a column vector first
a - b[:, np.newaxis]
Reshape b into a column vector, then subtract:
a - b.reshape(3, 1)
b isn't altered in place, but the result of the reshape method call will be the column vector:
array([[1],
[2],
[3]])
Allowing the "shape" of the subtraction you wanted. A little more general reshape operation would be:
b.reshape(b.size, 1)
Taking however many elements b has, and molding them into an N x 1 vector.
Update: A quick benchmark shows kazemakase's answer, using b[:, np.newaxis] as the reshaping strategy, to be ~7% faster. For small vectors, those few extra fractions of a µs won't matter. But for large vectors or inner loops, prefer his approach. It's a less-general reshape, but more performant for this use.
It seems that I can't convert a list with non-fixed length elements to tensor.For example, I get a list like [[1,2,3],[4,5],[1,4,6,7]],and I want to convert it to a tensor by tf.convert_to_tensor, and It doesn't work and throw a ValueError: Argument must be a dense tensor: [[1, 2, 3], [4, 5], [1, 4, 6, 7]] - got shape [3], but wanted [3, 3].I don't want to pad or crop the elements for some reasons, is there any method to solve it?
Thanks in advance!
Tensorflow (as far as I know) currently does not support Tensors with different lengths along a dimension.
Depending on your goal, you could pad your list with zeros (inspired by this question) and then convert to a tensor. For example using numpy:
>>> import numpy as np
>>> x = np.array([[1,2,3],[4,5],[1,4,6,7]])
>>> max_length = max(len(row) for row in x)
>>> x_padded = np.array([row + [0] * (max_length - len(row)) for row in x])
>>> x_padded
array([[1, 2, 3, 0],
[4, 5, 0, 0],
[1, 4, 6, 7]])
>>> x_tensor = tf.convert_to_tensor(x_padded)