Given a 3d tenzor, say:
batch x sentence length x embedding dim
a = torch.rand((10, 1000, 96))
and an array(or tensor) of actual lengths for each sentence
lengths = torch .randint(1000,(10,))
outputs tensor([ 370., 502., 652., 859., 545., 964., 566., 576.,1000., 803.])
How to fill tensor ‘a’ with zeros after certain index along dimension 1 (sentence length) according to tensor ‘lengths’ ?
I want smth like that :
a[ : , lengths : , : ] = 0
One way of doing it (slow if batch size is big enough):
for i_batch in range(10):
a[ i_batch , lengths[i_batch ] : , : ] = 0
You can do it using a binary mask.
Using lengths as column-indices to mask we indicate where each sequence ends (note that we make mask longer than a.size(1) to allow for sequences with full length).
Using cumsum() we set all entries in mask after the seq len to 1.
mask = torch.zeros(a.shape[0], a.shape[1] + 1, dtype=a.dtype, device=a.device)
mask[(torch.arange(a.shape[0]), lengths)] = 1
mask = mask.cumsum(dim=1)[:, :-1] # remove the superfluous column
a = a * (1. - mask[..., None]) # use mask to zero after each column
For a.shape = (10, 5, 96), and lengths = [1, 2, 1, 1, 3, 0, 4, 4, 1, 3].
Assigning 1 to respective lengths at each row, mask looks like:
mask =
tensor([[0., 1., 0., 0., 0., 0.],
[0., 0., 1., 0., 0., 0.],
[0., 1., 0., 0., 0., 0.],
[0., 1., 0., 0., 0., 0.],
[0., 0., 0., 1., 0., 0.],
[1., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 1., 0.],
[0., 0., 0., 0., 1., 0.],
[0., 1., 0., 0., 0., 0.],
[0., 0., 0., 1., 0., 0.]])
After cumsum you get
mask =
tensor([[0., 1., 1., 1., 1.],
[0., 0., 1., 1., 1.],
[0., 1., 1., 1., 1.],
[0., 1., 1., 1., 1.],
[0., 0., 0., 1., 1.],
[1., 1., 1., 1., 1.],
[0., 0., 0., 0., 1.],
[0., 0., 0., 0., 1.],
[0., 1., 1., 1., 1.],
[0., 0., 0., 1., 1.]])
Note that it exactly has zeros where the valid sequence entries are and ones beyond the lengths of the sequences. Taking 1 - mask gives you exactly what you want.
Enjoy ;)
Related
Given the below tensor:
tensor = torch.Tensor([[1., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 1., 0., 0.],
[0., 0., 0., 0., 1.],
[1., 0., 0., 0., 0.],
[1., 0., 0., 0., 0.],
[0., 0., 0., 1., 0.],
[0., 0., 0., 0., 1.]])
and below is the tensor containing the indices:
indices = torch.tensor([2, 6, 7, 5, 4, 0, 3, 1])
How can I sort tensor using the values inside of indices?
Trying with sorted gives the error:
TypeError: 'Tensor' object is not callable`.
While numpy.sort gives:
ValueError: Cannot specify order when the array has no fields.`
You can use the indices like this:
tensor = torch.Tensor([[1., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 1., 0., 0.],
[0., 0., 0., 0., 1.],
[1., 0., 0., 0., 0.],
[1., 0., 0., 0., 0.],
[0., 0., 0., 1., 0.],
[0., 0., 0., 0., 1.]])
indices = torch.tensor([2, 6, 7, 5, 4, 0, 3, 1])
sorted_tensor = tensor[indices]
print(sorted_tensor)
# output
tensor([[0., 0., 1., 0., 0.],
[0., 0., 0., 1., 0.],
[0., 0., 0., 0., 1.],
[1., 0., 0., 0., 0.],
[1., 0., 0., 0., 0.],
[1., 0., 0., 0., 0.],
[0., 0., 0., 0., 1.],
[0., 1., 0., 0., 0.]])
i am able to get the checkerboard pattern, the + pattern and the one with 1's on the border but i am not able to figure this out. Can somebody help?
If you're sticking with whole dimensions, then as #Péter Leéh pointed out:
>>> np.eye(n) + np.fliplr(np.eye(n))
array([[1., 0., 0., 1.],
[0., 1., 1., 0.],
[0., 1., 1., 0.],
[1., 0., 0., 1.]])
will suffice, np.fliplr(x) (horizontal flip) is identical to np.flip(x, axis=1).
However if n is odd, then you will have to replace the center element with a 1. e.g. n=5:
>>> x = np.eye(n) + np.fliplr(np.eye(n))
>>> x[n//2, n//2] = 1
array([[1., 0., 0., 0., 1.],
[0., 1., 0., 1., 0.],
[0., 0., 1., 0., 0.],
[0., 1., 0., 1., 0.],
[1., 0., 0., 0., 1.]])
I have a problem with labels for segmentation, the label can have this value: 0, 200, 210, 220, 230, 240. I use this code:
mask = tf.keras.utils.to_categorical(y, 241)
The code work, but i want map the mask with only 6 classes, is this possible?
mask = tf.keras.utils.to_categorical(y,6)
On solution is that first replace your list with ordered indexes and then make it categorical. Because to_categorical expects indices for your list.
Here is the example code if you have limited categories:
y = [0, 200,210,0,240,230,200,0,210,220,240,0]
replacements = {
0: 0,
200: 1,
210: 2,
220: 3,
230: 4,
240: 5,
}
y = [replacements.get(x, x) for x in y]
y = tf.keras.utils.to_categorical(y)
Or you can use a simpler way like this:
from sklearn.preprocessing import LabelEncoder
y = tf.keras.utils.to_categorical(LabelEncoder().fit_transform(y))
If you print y:
array([[1., 0., 0., 0., 0., 0.],
[0., 1., 0., 0., 0., 0.],
[0., 0., 1., 0., 0., 0.],
[1., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 1.],
[0., 0., 0., 0., 1., 0.],
[0., 1., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0., 0.],
[0., 0., 0., 1., 0., 0.],
[0., 0., 0., 0., 0., 1.],
[1., 0., 0., 0., 0., 0.]], dtype=float32)
I have an array, and use the to_categorical function in keras:
labels = np.array([1,7,7,1,7])
keras.utils.to_categorical(labels)
I get this response:
array([[0., 1., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 1.],
[0., 0., 0., 0., 0., 0., 0., 1.],
[0., 1., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 1.]], dtype=float32)
How can I get only two columns? One for the 1 and one for the 7.
This is a possible way, but not a very good one:
labels = np.delete(labels, np.s_[0:1], axis=1)
np.delete(labels, np.s_[1:6], axis=1)
that gives:
array([[1., 0.],
[0., 1.],
[0., 1.],
[1., 0.],
[0., 1.]], dtype=float32)
Is there a better way to achieve this? Preferably by some "hidden" function in Keras utils or similar?
IIUC, you can just index your array by any column that has a value:
cat = keras.utils.to_categorical(labels)
>>> cat
array([[0., 1., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 1.],
[0., 0., 0., 0., 0., 0., 0., 1.],
[0., 1., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 1.]])
# Select column if it has at least one value:
>>> cat[:,cat.any(0)]
array([[1., 0.],
[0., 1.],
[0., 1.],
[1., 0.],
[0., 1.]])
You could also use pandas:
import pandas as pd
cat = pd.get_dummies(labels).values
>>> cat
array([[1, 0],
[0, 1],
[0, 1],
[1, 0],
[0, 1]], dtype=uint8)
Use np.unique with return_inverse flag -
# Get unique IDs mapped to each group of elements
In [73]: unql, idx = np.unique(labels, return_inverse=True)
# Perform outer comparison for idx against range of unique groups
In [74]: (idx[:,None] == np.arange(len(unql))).astype(float)
Out[74]:
array([[1., 0.],
[0., 1.],
[0., 1.],
[1., 0.],
[0., 1.]])
Alternatively with direct usage of unique labels -
In [96]: (labels[:,None] == np.unique(labels)).astype(float)
Out[96]:
array([[1., 0.],
[0., 1.],
[0., 1.],
[1., 0.],
[0., 1.]])
So lets say I have a (4,10) array initialized to zeros, and I have an input array in the form [2,7,0,3]. The input array will modify the zeros matrix to look like this:
[[0,0,1,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,1,0,0],
[1,0,0,0,0,0,0,0,0,0],
[0,0,0,1,0,0,0,0,0,0]]
I know I can do that by looping through the input target and indexing the matrix array with something like matrix[i][target in input target], but I tried to do it without a loop doing something like:
matrix[:, input_target] = 1, but that sets me the entire matrix to all 1.
Apparently the way to do it is:
matrix[range(input_target.shape[0]), input_target], the question is why this works and not using the colon ?
Thanks!
You only wish to update one column for each row. Therefore, with advanced indexing you must explicitly provide those row identifiers:
A = np.zeros((4, 10))
A[np.arange(A.shape[0]), [2, 7, 0, 3]] = 1
Result:
array([[ 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.],
[ 1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.]])
Using a colon for the row indexer will tell NumPy to update all rows for the specified columns:
A[:, [2, 7, 0, 3]] = 1
array([[ 1., 0., 1., 1., 0., 0., 0., 1., 0., 0.],
[ 1., 0., 1., 1., 0., 0., 0., 1., 0., 0.],
[ 1., 0., 1., 1., 0., 0., 0., 1., 0., 0.],
[ 1., 0., 1., 1., 0., 0., 0., 1., 0., 0.]])