I'd like to cast a numpy ndarray object of shape (n,) into one having shape (n, 1). The best I've come up with is to roll my own _to_col function:
def _to_col(a):
return a.reshape((a.size, 1))
But it is hard for me to believe that such a ubiquitous operation is not already built into numpy's syntax. I figure that I just have not been able to hit upon the right Google search to find it.
I'd use the following:
a[:,np.newaxis]
An alternative (but perhaps slightly less clear) way to write the same thing is:
a[:,None]
All of the above (including your version) are constant-time operations.
np.expand_dims is my favorite when I want to add arbitrary axis.
None or np.newaxis is good for code that doesn't need to have flexible axis. (aix's answer)
>>> np.expand_dims(np.arange(5), 0).shape
(1, 5)
>>> np.expand_dims(np.arange(5), 1).shape
(5, 1)
example usage: demean an array by any given axis
>>> x = np.random.randn(4,5)
>>> x - x.mean(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: shape mismatch: objects cannot be broadcast to a single shape
>>> ax = 1
>>> x - np.expand_dims(x.mean(ax), ax)
array([[-0.04152658, 0.4229244 , -0.91990969, 0.91270622, -0.37419434],
[ 0.60757566, 1.09020783, -0.87167478, -0.22299015, -0.60311856],
[ 0.60015774, -0.12358954, 0.33523495, -1.1414706 , 0.32966745],
[-1.91919832, 0.28125008, -0.30916116, 1.85416974, 0.09293965]])
>>> ax = 0
>>> x - np.expand_dims(x.mean(ax), ax)
array([[ 0.15469413, 0.01319904, -0.47055919, 0.57007525, -0.22754506],
[ 0.70385617, 0.58054228, -0.52226447, -0.66556131, -0.55640947],
[ 1.05009459, -0.27959876, 1.03830159, -1.23038543, 0.73003287],
[-1.90864489, -0.31414256, -0.04547794, 1.32587149, 0.05392166]])
Related
I have to write a python function where i need to compute the vector
For A is n by n and xn is n by 1
r_n = Axn - (xn^TAxn)xn
Im using numpy but .T doesn't work on vectors and when I just do
r_n = A#xn - (xn#A#xn)#xn but xn#A#xn gives me a scaler.
I've tried changing the A with the xn but nothing seems to work.
Making a 3x1 numpy array like this...
import numpy as np
a = np.array([1, 2, 3])
...and then attempting to take its transpose like this...
a_transpose = a.T
...will, confusingly, return this:
# [1 2 3]
If you want to define a (column) vector whose transpose you can meaningfully take, and get a row vector in return, you need to define it like this:
a = np.reshape(np.array([1, 2, 3]), (3, 1))
print(a)
# [[1]
# [2]
# [3]]
a_transpose = a.T
print(a_transpose)
# [[1 2 3]]
If you want to define a 1 x n array whose transpose you can take to get an n x 1 array, you can do it like this:
a = np.array([[1, 2, 3]])
and then get its transpose by calling a.T.
If A is (n,n) and xn is (n,1):
A#xn - (xn#A#xn)#xn
(n,n)#(n,1) - ((n,1)#(n,n)#(n,1)) # (n,1)
(n,1) error (1 does not match n)
If xn#A#xn gives scalar that's because xn is (n,) shape; as per np.matmul docs that's a 2d with two 1d arrays
(n,)#(n,n)#(n,) => (n,)#(n,) -> scalar
I think you want
(1,n) # (n,n) # (n,1) => (1,1)
Come to think of it that (1,1) array should be same single values as the scalar.
Sample calculation; 1st with the (n,) shape:
In [6]: A = np.arange(1,10).reshape(3,3); x = np.arange(1,4)
In [7]: A#x
Out[7]: array([14, 32, 50]) # (3,3)#(3,)=>(3,)
In [8]: x#A#x # scalar
Out[8]: 228
In [9]: (x#A#x)#x
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[9], line 1
----> 1 (x#A#x)#x
ValueError: matmul: Input operand 0 does not have enough dimensions (has 0, gufunc core with signature (n?,k),(k,m?)->(n?,m?) requires 1)
matmul does not like to work with scalars. But we can use np.dot instead, or simply multiply:
In [10]: (x#A#x)*x
Out[10]: array([228, 456, 684]) # (3,)
In [11]: A#x - (x#A#x)*x
Out[11]: array([-214, -424, -634])
Change the array to (3,1):
In [12]: xn = x[:,None]; xn.shape
Out[12]: (3, 1)
In [13]: A#xn - (xn.T#A#xn)*xn
Out[13]:
array([[-214],
[-424],
[-634]]) # same numbers but in (3,1) shape
Say I have a 1-D array dims:
dims = np.array((1,2,3,4))
I want to create a n-th order normally distributed tensor where n is the size of the dims and dims[i] is the size of the i-th dimension.
I tried to do
A = np.random.randn(dims)
But this doesn't work. I could do
A = np.random.randn(1,2,3,4)
which would work but n can be large and n can be random in itself. How can I read in a array of the size of the dimensions in this case?
Use unpacking with an asterisk:
np.random.randn(*dims)
Unpacking is standard Python when the signature is randn(d0, d1, ..., dn)
In [174]: A = np.random.randn(*dims)
In [175]: A.shape
Out[175]: (1, 2, 3, 4)
randn docs suggests standard_normal which takes a tuple (or array which can be treated as a tuple):
In [176]: B = np.random.standard_normal(dims)
In [177]: B.shape
Out[177]: (1, 2, 3, 4)
In fact the docs, say new code should use this:
In [180]: rgn = np.random.default_rng()
In [181]: rgn.randn
Traceback (most recent call last):
File "<ipython-input-181-b8e8c46209d0>", line 1, in <module>
rgn.randn
AttributeError: 'numpy.random._generator.Generator' object has no attribute 'randn'
In [182]: rgn.standard_normal(dims).shape
Out[182]: (1, 2, 3, 4)
I am new to programming.
I am trying to run this code
from numpy import *
x = empty((2, 2), int)
x = append(x, array([1, 2]), axis=0)
x = append(x, array([3, 5]), axis=0)
print(x)
But i get this error
Traceback (most recent call last):
File "/home/samip/PycharmProjects/MyCode/test.py", line 3, in <module>
x = append(x, array([1, 2]), axis=0)
File "<__array_function__ internals>", line 5, in append
File "/usr/lib/python3/dist-packages/numpy/lib/function_base.py", line 4700, in append
return concatenate((arr, values), axis=axis)
File "<__array_function__ internals>", line 5, in concatenate
ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 1 has 1 dimension(s)
I suspect you are trying to replicate this working list code:
In [56]: x = []
In [57]: x.append([1,2])
In [58]: x
Out[58]: [[1, 2]]
In [59]: np.array(x)
Out[59]: array([[1, 2]])
But with arrays:
In [53]: x = np.empty((2,2),int)
In [54]: x
Out[54]:
array([[73096208, 10273248],
[ 2, -1]])
Despite the name, the np.empty array is NOT a close of the empty list. It has 4 elements, the shape that you specified.
In [55]: np.append(x, np.array([1,2]), axis=0)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-55-64dd8e7900e3> in <module>
----> 1 np.append(x, np.array([1,2]), axis=0)
<__array_function__ internals> in append(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/numpy/lib/function_base.py in append(arr, values, axis)
4691 values = ravel(values)
4692 axis = arr.ndim-1
-> 4693 return concatenate((arr, values), axis=axis)
4694
4695
<__array_function__ internals> in concatenate(*args, **kwargs)
ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 1 has 1 dimension(s)
Note that np.append has passed the task on to np.concatenate. With the axis parameter, that's all this append does. It is NOT a list append clone.
np.concatenate demands consistency in the dimensions of its inputs. One is (2,2), the other (2,). Mismatched dimensions.
np.append is a dangerous function, and not that useful even when used correctly. np.concatenate (and the various stack) functions are useful. But you need to pay attention to shapes. And don't use them iteratively. List append is more efficient for that.
When you got this error, did you look up the np.append, np.empty (and np.concatenate) functions? Read and understand the docs? In the long run SO questions aren't a substitute for reading the documentation.
You can create empty list by []. In order to add new item use append. For add other list use extend.
x = [1, 2, 3]
x.append(4)
x.extend([5, 6])
print(x)
# [1, 2, 3, 4, 5, 6]
The issue is that the line x = empty((2, 2), int) is creating a 2D array.
Later when you try to append array([1, 2]) you get an error because it is a 1D array.
You can try the code below.
from numpy import *
x = empty((2, 2), int)
x = append(x,[1,2])
print(x)
as you can see in the error your two arrays must match the same shape, x.shape returns (2,2), and array([1,2]).shape returns (2,) so what you have to do is
x = np.append(x, np.array([1,2]).reshape((1,2)), axis=0)
Printing x returns :
array([[1.966937e-316, 4.031792e-313],
[0.000000e+000, 4.940656e-324],
[1.000000e+000, 2.000000e+000]])
I am using the LSTM tutorial for Theano (http://deeplearning.net/tutorial/lstm.html). In the lstm.py (http://deeplearning.net/tutorial/code/lstm.py) file, I don't understand the following line:
c = m_[:, None] * c + (1. - m_)[:, None] * c_
What does m_[:, None] mean? In this case m_ is the theano vector while c is a matrix.
This question has been asked and answered on the Theano mailing list, but is actually about the basics of numpy indexing.
Here are the question and answer
https://groups.google.com/forum/#!topic/theano-users/jq92vNtkYUI
For completeness, here is another explanation: slicing with None adds an axis to your array, see the relevant numpy documentation, because it behaves the same in both numpy and Theano:
http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#numpy.newaxis
Note that np.newaxis is None:
import numpy as np
a = np.arange(30).reshape(5, 6)
print a.shape # yields (5, 6)
print a[np.newaxis, :, :].shape # yields (1, 5, 6)
print a[:, np.newaxis, :].shape # yields (5, 1, 6)
print a[:, :, np.newaxis].shape # yields (5, 6, 1)
Typically this is used to adjust shapes to be able to broadcast to higher dimensions. E.g. tiling 7 times in the middle axis can be achieved as
b = a[:, np.newaxis] * np.ones((1, 7, 1))
print b.shape # yields (5, 7, 6), 7 copies of a along the second axis
I think the Theano vector's __getitem__ method expects a tuple as an argument! like this:
class Vect (object):
def __init__(self,data):
self.data=list(data)
def __getitem__(self,key):
return self.data[key[0]:key[1]+1]
a=Vect('hello')
print a[0,2]
Here print a[0,2] when a is an ordinary list will raise an exception:
>>> a=list('hello')
>>> a[0,2]
Traceback (most recent call last):
File "<string>", line 1, in <module>
TypeError: list indices must be integers, not tuple
But here the __getitem__ method is different and it accepts a tuple as an argument.
You can pass the : sign to __getitem__ like this as : means slice:
class Vect (object):
def __init__(self,data):
self.data=list(data)
def __getitem__(self,key):
return self.data[0:key[1]+1]+list(key[0].indices(key[1]))
a=Vect('hello')
print a[:,2]
Speaking about None, it can be used when indexing in plain Python as well:
>>> 'hello'[None:None]
'hello'
I'm planning on plotting y^n vs x for different values of n. Here is my sample code:
import numpy as np
x=np.range(1,5)
y=np.range(2,9,2)
exponent=np.linspace(1,8,50)
z=y**exponent
With this, I got the following error:
ValueError: operands could not be broadcast together with shapes (4) (5)
My idea is that for each value of n, I will get an array where that array contains the new values of y that is now raised to n. For instance:
y1= [] #an array where y**1
y2= [] #an array where y**1.5
y3= [] #an array where y**2
etc. I don't know if how I can get that 50 arrays for y**n and is there an easier way to do it? Thank you.
You can use "broadcasting" (explained here in the docs) and create a new axis:
z = y**exponent[:,np.newaxis]
In other words, instead of
>>> y = np.arange(2,9,2)
>>> exponent = np.linspace(1, 8, 50)
>>> z = y**exponent
Traceback (most recent call last):
File "<ipython-input-40-2fe7ff9626ed>", line 1, in <module>
z = y**exponent
ValueError: operands could not be broadcast together with shapes (4,) (50,)
You can use array[:,np.newaxis] (or array[:,None], the same thing, but newaxis is more explicit about your intent) to give the array an extra dimension of size 1:
>>> exponent.shape
(50,)
>>> exponent[:,np.newaxis].shape
(50, 1)
and so
>>> z = y**exponent[:,np.newaxis]
>>> z.shape
(50, 4)
>>> z[0]
array([ 2., 4., 6., 8.])
>>> z[1]
array([ 2.20817903, 4.87605462, 7.75025005, 10.76720154])
>>> z[0]**exponent[1]
array([ 2.20817903, 4.87605462, 7.75025005, 10.76720154])