I have two 1D tensors. One is a vector of predictions, second is vector of labels. I am trying to write a loop that checks the element-wise difference between vectors. If such diff is spotted, I want to do another operation, for simplicity let's say I want to print ("Diff spotted"). So far I came up with this but I got an error: Expected object of scalar type Byte but got scalar type Float for argument #2 'other'. I would appreciate help here. Maybe there is some more efficient way to do it, without loop.
for i in enumerate(t1):
if t1[i] != t2[i]:
print("Diff spotted")
You can use the eq() function in pytorch to check if to tensors are the same element-wise. For every index of an element which is the same as the labels element you get a True:
for label in predictions.round().eq(labels):
for element in label:
if element == False:
print("Diff spotted!")
Related
I am incremently sampling a batch of size torch.Size([n, 8]).
I also have a list valid_indices of length n which contains tuples of indices that are valid for each entry in the batch.
For instance valid_indices[0] may look like this: (0,1,3,4,5,7) , which suggests that indices 2 and 6 should be excluded from the first entry in batch along dim 1.
Particularly I need to exclude these values for when I use torch.max(batch, dim=1, keepdim=True).
Indices to be excluded (if any) may differ from entry to entry within the batch.
Any ideas? Thanks in advance.
I assume that you are getting the good old
IndexError: too many indices for tensor of dimension 1
error when you use your tuple indices directly on the tensor.
At least that was the error that I was able to reproduce when I execute the following line
t[0][valid_idx0]
Where t is a random tensor with size (10,8) and valid_idx0 is a tuple with 4 elements.
However, same line works just fine when you convert your tuple to a list as following
t[0][list(valid_idx0)]
>>> tensor([0.1847, 0.1028, 0.7130, 0.5093])
But when it comes to applying these indices to 2D tensors, things get a bit different, since we need to preserve the structure of our tensor for batch processing.
Therefore, it would be reasonable to convert our indices to mask arrays.
Let's say we have a list of tuples valid_indices at hand. First thing will be converting it to a list of lists.
valid_idx_list = [list(tup) for tup in valid_indices]
Second thing will be converting them to mask arrays.
masks = np.zeros((t.size()))
for i, indices in enumerate(valid_idx_list):
masks[i][indices] = 1
Done. Now we can apply our mask and use the torch.max on the masked tensor.
torch.max(t*masks)
Kindly see the colab notebook that I've used to reproduce the problem.
https://colab.research.google.com/drive/1BhKKgxk3gRwUjM8ilmiqgFvo0sfXMGiK?usp=sharing
I will preface by saying that I am very new to python and Stack Overflow. I am attempting to do an operation on a bunch of different arrays. An example of one of these arrays would be
s2 = np.array( ['e','e','e','k','k','k','e','e','e','k','k','k','e','e','e','k',\
'k','k','e','e','e','k','k','k','e','e','e','k','k','k','e','e',\
'e','k','k','k','e','e','e','k','k','k','e','e','e','k','k','k','e','k'])
I am attempting to do a summation using nested for loops, but whenever I use the statement
for i in range(s2):
I get the error described above
TypeError: only integer scalar arrays can be converted to a scalar index
I am wondering how I would be able to tell python that I am looking specifically in this array, even though it is an array of strings, and it seems to want an integer array. Again, newbie here, so any help is appreciated
for i in range(s2):
range needs to have a number or len like range(len(s2)):
you can iterate this list like:
for item in s2:
print(item)
or
for x in range(len(s2)):
print(s2[x])
If you are trying to iterate over it in some other rather than one item at a time, let me know
I defined a two-dimensional array as follows:
predict_result = np.zeros((69,1000))
In the loop, I was trying to inject a predicted one-dimensional array into it.
for ij in range(0,1000):
# # some code to generate Ypredict
predict_result[:,ij]=Ypredict
Ypredict is always the shape of (69,1). However, running the program gives the following error
predict_result[:,ij]=Ypredict ValueError: could not broadcast input
array from shape (69,1) into shape (69)
How can I correct this error?
Either change the (69,1) array to (69,), or make the receiving slot (69,1)
ravel is one on several ways of flattening the 2d array:
predict_result[:,ij]=Ypredict.ravel()
Index with a list rather than a scalar works the other way:
predict_result[:,[ij]]=Ypredict
You could also use broadcasting to put the smaller array into the larger without a loop - as noted in the other answer:
(69,1000) <= (69,1) works
The 2 basic rules of broadcasting:
a size 1 dimension can be added at the start to match the number of dimensions
all size 1 dimensions can be changed to match to same dimension of the other array(s).
(69,) cannot be changed to match (69,1). But (69,1) can be changed to match (69,1000).
You don't need for-loop:
predict_result[:] = Ypredict
Or you can create the result by repeat:
np.repeat(Ypredict, 1000, axis=1)
I have two lists,
list_a
list_b
whose shape are [10,50] and [40,50]
and I'm trying to combine them into one [50,50] array, starting with the following code (edited for readability)
array_a=np.array(list_a)
array_b=np.array(list_b)
array_c=np.concatenate(array_a,array_b)
But it keeps giving me an error that says
"TypeError: only length-1 arrays can be converted to Python scalars"
What's the issue here, and how can I fix it? This error isn't very helpful...
np.concatenate expects a tuple as argument, i.e. it should be
array_c=np.concatenate((array_a,array_b))
The first argument is a tuple of an arbitrary number of arrays, the second argument (in your case array_b) tells concatenate along which axis it should operate.
The issue here is that np.concatenate expects an iterable sequence of array-like objects for the first argument. Here it just takes array_a as the first argument. It is taking array_b as the second argument, which specifies which array axis to concatenate along. As this argument needs to be integer-like, it is attempting to convert array_b to an integer, but failing as it contains more than one item. Hence this error message.
To solve it, you need to wrap your two arrays in an iterable such as a tuple, like this:
cc=np.concatenate((array_a,array_b))
This results in both arrays being passed as the first argument to the function. (Edit: Wrapping in a list also works, i.e. concatenate([array_a,array_b]). Haven't tried other forms).
In your example, this will work, as the second argument defaults to 0, which means the arrays can have a different length in the first dimension only (the zeroth-indexed dimension). For you, these lengths are 10 and 40, and the other dimension is 50 for both. If your array dimensions were reversed, so they were now [50,10] and [50,40], you would need to set the axis to the second dimension (index 1) like so:
cc=np.concatenate((array_a,array_b),1)
I have this MATLAB code that I need to translate to python, however there is an issue in creating a new column in the firings array. In MATLAB, the code creates an n*2 matrix that is initially empty and I want to be able to do the same in python. Using NumPy, I created fired = np.where(v >= 30). However python creates a tuple rather than an array so it throws an error:
TypeError: unsupported operand type(s) for +: 'int' and 'tuple'
This is the code I have in MATLAB that I would like converted into Python
firings=[];
firings=[firings; t+0*fired, fired];
Help is appreciated! Thanks!
np.where generates a two-element tuple if the array is 1D in nature. For the 1D case, you would need to access the first element of the result of np.where only:
fired = np.where(v >= 30)[0]
You can then go ahead and concatenate the matrices. Also a suggestion provided by user #Divakar would be to use np.flatnonzero which would equivalently find the non-zero values in a NumPy array and flattened into a 1D array for less headaches:
fired = np.flatnonzero(v >= 30)
Take note that the logic to concatenate would not work if there were no matches found in fired. You will need to take this into account when you look at your concatenating logic. The convenient thing with MATLAB is that you're able to concatenate empty matrices and the result is no effect (obviously).
Also note that there is no conception of a row vector or column vector in NumPy. It is simply a 1D array. If you want to specifically force the array to be a column vector as you have it, you need to introduce a singleton axis in the second dimension for you to do this. Note that this only works provided that np.where gave you matched results. After, you can use np.vstack and np.hstack to vertically and horizontally concatenate arrays to help you do what you ask. What you have to do first is create a blank 2D array, then do what we just covered:
firings = np.array([[]]) # Create blank 2D array
# Some code here...
# ...
# ...
# fired = find(v >= 30); % From MATLAB
fired = np.where(v >= 30)[0]
# or you can use...
# fired = np.flatnonzero(v >= 30)
if np.size(fired) != 0:
fired = fired[:, None] # Introduce singleton axis
# Update firings with two column vectors
# firings = [firings; t + 0 * fired, fired]; % From MATLAB
firings = np.vstack([firings, np.hstack([t + 0*fired, fired])])
Here np.size finds the total number of elements in the NumPy array. If the result of np.where generated no results, the number of elements in fired should be 0. Therefore the if statement only executes if we have found at least one element in v subject to v >= 30.
If you use numpy, you can define an ndarray:
import numpy as np
firings=np.ndarray(shape=(1,2)
firings[0][0:]=(1.,2.)
firings=np.append(firings,[[3.,4.]],axis=0)