Is there a short-code efficient way of "glueing" two arrays together such that if the arrays differ in length then the glued product must be such that the values from the longer are filled between values from the smaller untill the the new product has the same length as sum of the length of the two arrays? Or: Is there a way to create an array where x = [a j j j b j j j ], that is to say, take array that has values [a b], create a new one by filling 3 js between each element of that array to get : [a j j j b]
There is the obvious way of doing this by a loop since I know the size of the product beforehand but I suspect there must be a more "numpyic" solution at hand.
It is easy to do when both arrays I want to "glue" are of the same size and the product is [a j b j c j], ie every other as can be seen in this:
np.append(np.zeros((10,1)),np.ones((10,1)),1).reshape(2*10)
array([ 0., 1., 0., 1., 0., 1., 0., 1., 0., 1., 0., 1., 0.,
1., 0., 1., 0., 1., 0., 1.])
but you cannot do
np.append(np.zeros((10,1)),np.ones((20,1)),1).reshape(20+10)
I apologize if the question isn't clear enough, please do tell which parts I can clarify, my English is broken.
Assuming that len(A) == n and len(B) == N and Nis a multiple of n, ie there is some integer m such that N = m*n, and you can do:
import numpy as np
A = np.zeros(10)
B = np.ones(20)
n = len(A)
C = np.concatenate([A.reshape(n, 1), B.reshape(n, -1)], axis=1)
C = C.ravel()
This is pretty much what you have in the question, but the trick is to reshape B to be (n, m) instead of (N, 1) ie (10, 2) instead of (20, 1) in this case. The -1 in reshape is short hand for "whatever will make it work" it's a lazy way of doing B.reshape(n, len(B)//n).
Based on your question it seems like the array B might just be homogenous array, (ie all(B == j)), in which case you could just do:
import numpy as np
A = np.zeros(10)
j = 1.
C = np.zeros([10, 3])
C[:, 0] = A
C[:, 1:] = j
C = C.ravel()
Related
Hello I m trying to create a tensor that will have inside N matrices of n by n size. I tried to initialize it with
Q=torch.zeros(N, (n,n))
but i get the following error
zeros(): argument 'size' must be tuple of ints, but found element of type tuple at pos 2
Also I want to fill it later with random matrices with integer values and I will turn it to semidefinte so I thought of the following
for i in range(0,N):
Q[i]=torch.randint(0,10,(n,n))
Q = Q*Q.t()
Is it correct? Is there any other faster way with a build in command?
N matrices of n x n size is equivalent to three dimensional tensor of shape [N, n, n]. You can do it like so:
import torch
N = 32
n = 10
tensor = torch.randint(0, 10, size=(N, n, n))
No need to fill it with zeros to begin with, you can create it directly.
You can also iterate over 0 dimension similar to what you did:
for i in range(0, N):
tensor[i] = tensor[i] * tensor[i].T
See #Dishin H Goyani answer for faster approach with permutation.
Here you supposed to pass N, n, n to get N matrices of n by n size. As #Szymon already explain in his answer
Q = torch.randint(0, 10, size=(N, n, n))
For Later part you can use torch.Tensor.permute to transpose internal tensors
Q = Q * Q.permute(0, 2, 1)
Use torch.empty to create uninitialized tensor (it's faster then torch.zeros) torch.empty
Q = torch.empty(N, n, n)
Initialize it:
for i in range(0, N):
Q[i] = torch.randint(0, 10, (n, n))
use .permute as #Dishin H Goyani has proposed.
You can use * operator on iterables like tuples to pass it as positional arguments.
Here sample code:
>>> import torch
>>> N = 2
>>> n = 3
>>> Q = torch.zeros(N, *(n, n))
>>> Q
tensor([[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]]])
Is there a faster / inbuilt way to generate identity matrices with arbitrary shape in the first dimensions and an identity in the last m dimensions?
import numpy as np
base_shape = (10, 11, 12)
n_dim = 4
# m = 2
frames2d = np.zeros(base_shape + (n_dim, n_dim))
for i in range(n_dim):
frames2d[..., i, i] = 1
# m = 3
frames3d = np.zeros(base_shape + (n_dim, n_dim, n_dim))
for i in range(n_dim):
frames3d[..., i, i, i] = 1
Approach #1
We can leverage np.einsum for a diagonal view inspired by this post and hence assign 1s there for our desired output. So, for say the m=3 case, after initializing with zeros, we can simply do -
diag_view = np.einsum('...iii->...i',frames3d)
diag_view[:] = 1
Generalizing to include those input params, it would be -
def ndeye_einsum(base_shape, n_dim, m):
out = np.zeros(list(base_shape) + [n_dim]*m)
diag_view = np.einsum('...'+'i'*m+'->...i',out)
diag_view[:] = 1
return out
So, to reproduce those same arrays, it would be -
frames2d = ndeye_einsum(base_shape, n_dim, m=2)
frames3d = ndeye_einsum(base_shape, n_dim, m=3)
Approach #2
Again, from the same linked post, we can also reshape to 2D and assign into step-sized sliced array along the cols, like so -
def ndeye_reshape(base_shape, n_dim, m):
N = (n_dim**np.arange(m)).sum()
out = np.zeros(list(base_shape) + [n_dim]*m)
out.reshape(-1,n_dim**m)[:,::N] = 1
return out
This again works on a view and hence should be equally efficient as approach #1.
Approach #3
Another way would be to use integer-based indexing. So, for example for assigning into frames3d in one-go, it would be -
I = np.arange(n_dim)
frames3d[..., I, I, I] = 1
Generalizing that becomes -
def ndeye_ellipsis_indexer(base_shape, n_dim, m):
I = np.arange(n_dim)
indexer = tuple([Ellipsis]+[I]*m)
out = np.zeros(list(base_shape) + [n_dim]*m)
out[indexer] = 1
return out
Extending to higher-dims with view
The dims along base_shape are basically replications of elements from the last m dims. As such, we can get those higher dims as a higher-dim array view with np.broadcast_to. We will create basically a m-dim identity array and then broadcast-view into higher dims. This would be applicable across all three approaches posted earlier. To demonstrate, how to use it on the einsum based solution, we would have -
# Create m-dim "trailing-base" array, basically a m-dim identity array
def ndeye_einsum_trailingbase(n_dim, m):
out = np.zeros([n_dim]*m)
diag_view = np.einsum('i'*m+'->...i',out)
diag_view[:] = 1
return out
def ndeye_einsum_view(base_shape, n_dim, m):
trail_base = ndeye_einsum_trailingbase(n_dim, m)
return np.broadcast_to(trail_base, list(base_shape) + [n_dim]*m)
Thus, again we would have, e.g. -
frames3d = ndeye_einsum_view(base_shape, n_dim, m=3)
This would be a view into a m-dim array and hence efficient both on memory and performance.
One approach to have an identity matrix along the last two dimensions of the array, is to use np.broadcast_to and specifying the resulting shape the ndarray should have (this does not generalize to higher dimensions):
base_shape = (10, 11, 12)
n_dim = 4
frame2d = np.broadcast_to(np.eye(n_dim), a.shape+(n_dim,)*2)
print(frame2d.shape)
# (10, 11, 12, 4, 4)
print(frame2d)
array([[[[[1., 0., 0., 0.],
[0., 1., 0., 0.],
[0., 0., 1., 0.],
[0., 0., 0., 1.]],
[[1., 0., 0., 0.],
[0., 1., 0., 0.],
[0., 0., 1., 0.],
[0., 0., 0., 1.]],
...
suppose I have this 2d array A:
[[0,0,0,0],
[0,0,0,0],
[0,0,0,0],
[0,0,0,4]]
and I want to sum B:
[[1,2,3]
[4,5,6]
[7,8,9]]
centered on A[0][0] so the result would be:
array_sum(A,B,0,0) =
[[5,6,0,4],
[8,9,0,0],
[0,0,0,0],
[2,0,0,5]]
I was thinking that I should make a function that compares if its on a boundary and then adjust the index for that:
def array_sum(A,B,i,f):
...
if i == 0 and j == 0:
A[-1][-1] = A[-1][-1]+B[0][0]
...
else:
A[i-1][j-1] = A[i][j]+B[0][0]
A[i][j] = A[i][j]+B[1][1]
A[i+1][j+1] = A[i][j]+B[2][2]
...
but I don't know if there is a better way of doing that, I was reading about broadcasting or maybe using convolute for that, but I'm not sure if there is a better way to do that.
Assuming B.shape is all odd numbers, you can use np.indices, manipulate them to point where you want, and use np.add.at
def array_sum(A, B, loc = (0, 0)):
A_ = A.copy()
ix = np.indices(B.shape)
new_loc = np.array(loc) - np.array(B.shape) // 2
new_ix = np.mod(ix + new_loc[:, None, None],
np.array(A.shape)[:, None, None])
np.add.at(A_, tuple(new_ix), B)
return A_
Testing:
array_sum(A, B)
Out:
array([[ 5., 6., 0., 4.],
[ 8., 9., 0., 7.],
[ 0., 0., 0., 0.],
[ 2., 3., 0., 5.]])
As a rule of thumb slice indexing is faster (~2x) than fancy indexing. This appears to be true even for the small example in OP. Downside: the code is slightly more complicated.
import numpy as np
from numpy import s_ as _
from itertools import product, starmap
def wrapsl1d(N, n, c):
# check in 1D whether a patch of size n centered at c in a vector
# of length N fits or has to be wrapped around
# return appropriate slice objects for both vector and patch
assert n <= N
l = (c - n//2) % N
h = l + n
# return list of pairs (index into A, index into patch)
# 2 pairs if we wrap around, otherwise 1 pair
return [_[l:h, :]] if h <= N else [_[l:, :N-l], _[:h-N, n+N-h:]]
def use_slices(A, patch, center=(0, 0)):
slAptch = product(*map(wrapsl1d, A.shape, patch.shape, center))
# the product now has elements [(idx0A, idx0ptch), (idx1A, idx1ptch)]
# transpose them:
slAptch = starmap(zip, slAptch)
out = A.copy()
for sa, sp in slAptch:
out[sa] += patch[sp]
return out
I have np.arrays C, R and S of shapes (?, d), (?, n) and (?, d) respectively; where d<=n and the question mark represents any number of matching dimensions. Now I would like to do the following assignment (this is of course not proper python code, but it works if ? is just a single number):
for i in range(?):
R[i][S[i]]=C[i]
That is: I want for each tuple i of indices (within the bounds specified by ?) to take the corresponding array R[i] in R and assign d many positions (the ones specified by S[i]) to be the values in the array C[i].
What is the pythonic way to do this?
Example:
setup
import numpy as np
m,n,d= 2,7,4
R=np.zeros((m,n))
C=np.arange(d*m).reshape((m,d))
S=np.array([[0,2,4,6],[3,4,5,6]])
this works:
for i in range(m):
R[i][S[i]]=C[i]
this does not work:
R[S]=C
Your 2D example can be done as follows:
R[np.arange(m)[:, None], S] = C
# array([[ 0., 0., 1., 0., 2., 0., 3.],
# [ 0., 0., 0., 4., 5., 6., 7.]])
The 3D case would be similar:
i, j, k = R.shape
i, j, k = np.ogrid[i, j, k]
R[i, j, S] = C
In ND one could write:
idx = np.ogrid[tuple(map(slice, R.shape))]
idx[-1] = S
R[idx] = C
Here's the problem. Let's say I have a matrix A =
array([[ 1., 0., 2.],
[ 0., 0., 2.],
[ 0., -1., 3.]])
and a vector of indices p = array([0, 2, 1]). I want to turn a 3x3 matrix A to an array of length 3 (call it v) where v[j] = A[j, p[j]] for j = 0, 1, 2. I can do it the following way:
v = map(lambda (row, idx): row[idx], zip(A, p))
So for the above matrix A and a vector of indices p I expect to get array([1, 2, -1]) (ie 0th element of row 0, 2nd element of row 1, 1st element of row 2).
But can I achieve the same result by using native numpy (ie without explicitly zipping and then mapping)? Thanks.
I don't think that such a functionality exists. To achieve what you want, I can think of two easy ways. You could do:
np.diag(A[:, p])
Here the array p is applied as a column index for every row such that on the diagonal you will have the elements that you are looking for.
As an alternative you can avoid to produce a lot of unnecessary entries by using:
A[np.arange(A.shape[0]), p]