I am really confused about why do I need to use ravel() before fitting the data to SGDRegressor.
This is the code:
from sklearn.linear_model import SGDRegressor
sgd_reg = SGDRegressor(max_iter = 1000, tol = 1e-3, penalty = None, eta0= 0.1)
sgd_reg.fit(X, y.ravel())
These are the shape of X and y:
>>> X.shape
(100, 1)
>>> y.shape
(100, 1)
>>> y.ravel().shape
(100,)
Think of y as a two-dimensional matrix, although it has only one column. But the fit method expects y to be a flat array. That's why you have to use ravel, to convert the 2d to a 1d array.
It's common in machine learning papers and textbooks to write y as a matrix, because it can simplify the notation when matrices are multiplied. But you could also write it as a simple one-dimensional vector. You could say it makes no difference, because it really only has one dimension in either case, but mathematically and in the Python implementation, the matrix and the vector are two different objects.
Related
I have a problem. I get a task.
Create LinearRegression X to Y.
fit() a to reshape X and Y vectors new shape: (-1, 1).
This is part of my code
tuple1 = tuple(zip(X,Y))
np.reshape(tuple1, (-1, 1))
reg = LinearRegression().fit(tuple1)
I don't understand the question. The problem is the three last lines in my code. So first I should merge X and Y into a tuple to make reshape? But then I must use linear regression so I need X and Y which are not merged. I don't get it.
As the method fit() accepts properly shaped arrays, ...
The way it is defined, X is a 1D vector (X.shape gives (5,))
as scikit-learn fit() methods in general expect an array of vectors
So X is a problem, because that's not an array of vectors, but just a 1D vector.
reshape X and Y vectors by using the method reshape() and passing to it a tuple with a new shape: (-1, 1)
X.reshape(-1, 1).shape gives (5, 1), which is what we need. I see where you got confused: The "tuple" refers to the arguments of the reshape function (literally the tuple (-1, 1)), not to the result of the transformation.
Perform the reshaping on site (in the function call), keep the original vectors as they are.
Reshape in the function call: reg = LinearRegression().fit(X.reshape(-1, 1), Y), i.e. don't mess with the variables beforehand.
Note: X can stay the way it is, because that's ok as a 1D vector (only one dependent variable); so "you will have to reshape X and Y vectors" is not correct.
In linear algebra, vectors are normalized when they are divided by their norm, that is, the squared sum of their components.
Yet, sklearn.preprocessing.normalize method does not accept vectors, only matrices of at least two columns:
"ValueError: Expected 2D array, got 1D array instead"
Why?
normalize works on a data set, not a vector. You have the wrong definition of "normalize" for this function. It works on individual vectors. If you give it a 2D array of a single column (shape of [N, 1]), you can get your vector normalized in the "normal" fashion.
According to the documentation for sklearn.preprocessing.normalize, the parameter x is the data to normalize, element by element, and has the shape [n_samples, n_features]. The function normalize perform this operation on a single array-like dataset, either using the L1 or L2 norms.
I have numpy array 'test' of dimension (100, 100, 16, 16) which gives me a different 16x16 array for points on a 100x100 grid.
I also have some eigenvalues and vectors where vals has the dimension (100, 100, 16) and vecs (100, 100, 16, 16) where vecs[x, y, :, i] would be the ith eigenvector of the matrix at the point (x, y) corresponding to the ith eigenvalue vals[x, y, i].
Now I want to take the first eigenvector of the array at ALL points on the grid, do a matrix product with the test matrix and then do a scalar product of the resulting vector with all the other eigenvectors of the array at all points on the grid and sum them.
The resulting array should have the dimension (100, 100). After this I would like to take the 2nd eigenvector of the array, matrix multiply it with test and then take the scalar product of the result with all the eigenvectors that is not the 2nd and so on so that in the end I have 16 (100, 100) or rather a (100, 100, 16) array. I only succeded sofar with a lot of for loops which I would like to avoid, but using tensordot gives me the wrong dimension and I don't see how to pick the axis which is vectorized along for the np.dot function.
I heard that einsum might be suitable to this task, but everything that doesn't rely on the python loops is fine by me.
import numpy as np
from numpy import linalg as la
test = np.arange(16*16*100*100).reshape((100, 100, 16, 16))
vals, vecs = la.eig(test + 1)
np.tensordot(vecs, test, axes=[2, 3]).shape
>>> (10, 10, 16, 10, 10, 16)
EDIT: Ok, so I used np.einsum to get a correct intermediate result.
np.einsum('ijkl, ijkm -> ijlm', vecs, test)
But in the next step I want to do the scalarproduct only with all the other entries of vec. Can I implement maybe some inverse Kronecker delta in this einsum formalism? Or should I switch back to the usual numpy now?
Ok, I played around and with np.einsum I found a way to do what is described above. A nice feature of einsum is that if you repeat doubly occuring indices in the 'output' (so right of the '->'-thing) you can have element-wise multiplication along some and contraction along some other axes (something that you don't have in handwritten tensor algebra notation).
result = np.einsum('ijkl, ijlm -> ijkm', np.einsum('ijkl, ijkm -> ijlm', vecs, test), vecs)
This nearly does the trick. Now only the diagonal terms have to be taken out. We could do this by just substracting the diagonal terms like this:
result = result - result * np.eye(np.shape(test)[-1])[None, None, ...]
I have a matrix that I initialized with numpy.random.uniform like so:
W = np.random.uniform(-1, 1, (V,N))
In my case, V = 10000 and N = 50, x is a positive integer
When I multiply W by a one hot vector x_vec of dimension V X 1, like W.T.dot(x_vec), I get a column vector with a shape of (50,1). When I try to get the same vector by indexing W, as in W[x].T or W[x,:].T I get shape (50,).
Can anyone explain to me why these two expression return different shapes and if it's possible to return a (50,1) matrix (vector) with the indexing method. The vector of shape (50,) is problematic because it doesn't behave the same way as the (50,1) vector when I multiply it with other matrices, but I'd like to use indexing to speed things up a little.
*Sorry in advance if this question should be in a place like Cross Validated instead of Stack Exchange
They are different operations. matrix (in the maths sense) times matrix gives matrix, some of your matrices just happen to have width 1.
Indexing with an integer scalar eats the dimension you are indexing into. Once you are down to a single dimension, .T does nothing because it doesn't have enough axes to shuffle.
If you want to go from (50,) to (50, 1) shape-wise, the recipe is indexing with None like so v[:, None]. In your case you have at least two one-line options:
W[x, :][:, None] # or W[x][:, None] or
W[x:x+1, :].T # or W[x:x+1].T
The second-line option preserves the first dimension of W by requesting a subrange of length one. The first option can be contracted into a single indexing operation - thanks to #hpaulj for pointing this out - which gives the arguably most readable option:
W[x, :, None]
The first index (scalar integer x) consumes the first dimension of W, the second dimension (unaffected by :) becomes the first and None creates a new dimension on the right.
Consider the following simple example:
X = numpy.zeros([10, 4]) # 2D array
x = numpy.arange(0,10) # 1D array
X[:,0] = x # WORKS
X[:,0:1] = x # returns ERROR:
# ValueError: could not broadcast input array from shape (10) into shape (10,1)
X[:,0:1] = (x.reshape(-1, 1)) # WORKS
Can someone explain why numpy has vectors of shape (N,) rather than (N,1) ?
What is the best way to do the casting from 1D array into 2D array?
Why do I need this?
Because I have a code which inserts result x into a 2D array X and the size of x changes from time to time so I have X[:, idx1:idx2] = x which works if x is 2D too but not if x is 1D.
Do you really need to be able to handle both 1D and 2D inputs with the same function? If you know the input is going to be 1D, use
X[:, i] = x
If you know the input is going to be 2D, use
X[:, start:end] = x
If you don't know the input dimensions, I recommend switching between one line or the other with an if, though there might be some indexing trick I'm not aware of that would handle both identically.
Your x has shape (N,) rather than shape (N, 1) (or (1, N)) because numpy isn't built for just matrix math. ndarrays are n-dimensional; they support efficient, consistent vectorized operations for any non-negative number of dimensions (including 0). While this may occasionally make matrix operations a bit less concise (especially in the case of dot for matrix multiplication), it produces more generally applicable code for when your data is naturally 1-dimensional or 3-, 4-, or n-dimensional.
I think you have the answer already included in your question. Numpy allows the arrays be of any dimensionality (while afaik Matlab prefers two dimensions where possible), so you need to be correct with this (and always distinguish between (n,) and (n,1)). By giving one number as one of the indices (like 0 in 3rd row), you reduce the dimensionality by one. By giving a range as one of the indices (like 0:1 in 4th row), you don't reduce the dimensionality.
Line 3 makes perfect sense for me and I would assign to the 2-D array this way.
Here are two tricks that make the code a little shorter.
X = numpy.zeros([10, 4]) # 2D array
x = numpy.arange(0,10) # 1D array
X.T[:1, :] = x
X[:, 2:3] = x[:, None]