We Keep Coding

Pytorch get "reduced" tensor by indices

I have a tensor a = torch.arange(6).reshape(2,3), and another tensor b=(torch.rand(a.size())> 0.5).int().nonzero().
I want to create a new tensor that contains only values from a of the indices that are indicated by b.
For example:
a = torch.arange(6).reshape(2,3) # tensor([[0, 1, 2],
# [3, 4, 5]])
b = (torch.rand(a.size())> 0.5).int().nonzero() # tensor([[0, 1],
# [0, 2],
# [1, 0],
# [1, 1]])
The desired output is:
tensor([1,2,3,4])
I know that I can iterate over the values of b and access those values in a as indices but I wanted to know if there is a better Pytorch way to to this (using tensor operations only).
** The shape of the output tensor doesn't really matter, I just need to have a tensor with only the values indicated by b.

If I understand you correctly, you can do:
a[b[:,0], b[:,1]]
This will produce a 1D tensor with the values at the indices specified by b. Note that the output might not be the same from run to run of the program since the indices are selected nondeterministically.
If you don't know the number of dimension in advance, you'll need to use map() to generate the desired slices:
a[tuple(map(lambda x: b[:,x], range(a.dim())))]

concatenate a column vector to the end of a matrix

I was trying to add a column vector at the end of a matrix as follows :
import numpy as np
datas=[[1,2],[3,4]]
temp=[1,2]
datas=np.array(datas)
temp=np.transpose(np.array(temp))
np.append(datas,temp,axis=1)
But I'm getting dimension mismatch error?
How do I do this properly then?

you need to add one dimension to temp so that both the array have same dimension
import numpy as np
datas=[[1,2],[3,4]]
temp=[1,2]
datas=np.array(datas)
temp=np.array(temp)[:, np.newaxis] ## this adds new dimension
np.append(datas,temp,axis=1)
you can also do it using concatenate function like below. It will perform better if you are concatenating more than two arrays. Here you create python list ls in a loop and then concatenate them
ls = [datas,temp]
np.concatenate(ls, axis=1)

Would recommend you just use np.expand_dims() and then np.hstack()
datas=[[1,2],[3,4]]
temp=[1,2]
#Expand the dims of temp
temp = np.expand_dims(temp,1)
#Stack horizontally
np.hstack((datas, temp))
array([[1, 2, 1],
[3, 4, 2]])

Expected 2D array, got scalar array instead

I don't know why the error occurs, and I've tried to reshape but no luck.
Thanks for answering.
X_test = np.append(X_test, scaler.transform(working_data.iloc[-1][0]))
And here is the error message I receive.
ValueError: Expected 2D array, got scalar array instead:
array=9583.994119999077.
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.
And the full code is here:https://activewizards.com/blog/bitcoin-price-forecasting-with-deep-learning-algorithms/
Really appreciate your help.

It is a common error that happens while working with arrays in Python. So let's say you have an array
a = [[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]
shape of this array will be (3,3). Now if you access a[-1,0] you will get 7, which will have no shape as it's scalar. If you want to append this to an array say [[2, 3, 4]] this will not work, as latter is a 2-D array and for append to work, you need arrays of same dimension. So if you call reshape like this: a[-1,0].reshape(-1,1) then you will get an array of shape (1,1), that is instead of 7 you will get [[7]]. Another way to bypass this would be to access the value like this: a[-1,:1], this essentially does the same thing as a[-1,0] but it indicates the array that you want the resulting array as 2-D.
Another thing to note will be to check the axis you are trying to append to, as I do not know the shape of X_test, I can not say you want to add a new row, or a new column, so keep that in mind in case that causes an error later.

About Theano.tensor

Recently I use theano to create a gragh whitch is used for identifying flowers, however, the output of theano's inner function seem not be the type that I expect, for example:
a = numpy.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
sum = theano.tensor.sum(a, axis = 1)
sum_array = numpy.asarray(sum, dtype = numpy.float32)
I don't know why it doesn't work, simply I just want to creat an array to store the sum-result.
It just a simple example, in my project, I use the function "conv2d" and create an output after convolving the images, but I can't get the information of the output like the shape:
conv_out = conv2d(input, filter_shape, image_shape, ...)
output = theano.tensor.tanh(con_out, bias.dimshuffle('x','0','x','x'))
How can I change the 'output' into a 4D matrix and conveniently get its shape and other information?

Theano is different from regular python in that what you are creating are symbolic functions.
You need call Theano.function() for the symbolic function to be compiled. Then you need to call the resulting function with parameters.

Convert 1D array into numpy matrix

I have a simple, one dimensional Python array with random numbers. What I want to do is convert it into a numpy Matrix of a specific shape. My current attempt looks like this:
randomWeights = []
for i in range(80):
randomWeights.append(random.uniform(-1, 1))
W = np.mat(randomWeights)
W.reshape(8,10)
Unfortunately it always creates a matrix of the form:
[[random1, random2, random3, ...]]
So only the first element of one dimension gets used and the reshape command has no effect. Is there a way to convert the 1D array to a matrix so that the first x items will be row 1 of the matrix, the next x items will be row 2 and so on?
Basically this would be the intended shape:
[[1, 2, 3, 4, 5, 6, 7, 8],
[9, 10, 11, ... , 16],
[..., 800]]
I suppose I can always build a new matrix in the desired form manually by parsing through the input array. But I'd like to know if there is a simpler, more eleganz solution with built-in functions I'm not seeing. If I have to build those matrices manually I'll have a ton of extra work in other areas of the code since all my source data comes in simple 1D arrays but will be computed as matrices.

reshape() doesn't reshape in place, you need to assign the result:
>>> W = W.reshape(8,10)
>>> W.shape
(8,10)

You can use W.resize(), ndarray.resize()