Related
I have a requirement where I have 2 2D numpy arrays, and I would like to combine them in a specific manner:
x = [[0, 1, 2],
[3, 4, 5],
[6, 7, 8]]
| | |
0 1 2
y = [[10, 11, 12],
[13, 14, 15],
[16, 17, 18]]
| | |
3 4 5
x op y = [ 0 3 1 4 2 5 ] (in terms of the columns)
In other words,
The combination of x and y should look something like this:
[[ 0., 10., 1., 11., 2., 12.],
[ 3., 13., 4., 14., 5., 15.],
[ 6., 16., 7., 17., 8., 18.]]
Where I alternately combine the columns of each individual array to form the final 2D array. I have come up with one way of doing so, but it is rather ugly. Here's my code:
x = np.arange(9).reshape(3, 3)
y = np.arange(start=10, stop=19).reshape(3, 3)
>>> a = np.zeros((6, 3)) # create a 2D array where num_rows(a) = num_cols(x) + num_cols(y)
>>> a[: : 2] = x.T
>>> a[1: : 2] = y.T
>>> a.T
array([[ 0., 10., 1., 11., 2., 12.],
[ 3., 13., 4., 14., 5., 15.],
[ 6., 16., 7., 17., 8., 18.]])
As you can see, this is a very ugly sequence of operations. Furthermore, things become even more cumbersome in higher dimensions. For example, if you have x and y to be [3 x 3 x 3], then this operation has to be repeated in each dimension. So I'd probably have to tackle this with a loop.
Is there a simpler way around this?
Thanks.
In [524]: x=np.arange(9).reshape(3,3)
In [525]: y=np.arange(10,19).reshape(3,3)
This doesn't look at all ugly to me (one liners are over rated):
In [526]: a = np.zeros((3,6),int)
....
In [528]: a[:,::2]=x
In [529]: a[:,1::2]=y
In [530]: a
Out[530]:
array([[ 0, 10, 1, 11, 2, 12],
[ 3, 13, 4, 14, 5, 15],
[ 6, 16, 7, 17, 8, 18]])
still if you want a one liner, this might do:
In [535]: np.stack((x.T,y.T),axis=1).reshape(6,3).T
Out[535]:
array([[ 0, 10, 1, 11, 2, 12],
[ 3, 13, 4, 14, 5, 15],
[ 6, 16, 7, 17, 8, 18]])
The idea on this last was to combine the arrays on a new dimension, and reshape is some way other. I found it by trial and error.
and with another trial:
In [539]: np.stack((x,y),2).reshape(3,6)
Out[539]:
array([[ 0, 10, 1, 11, 2, 12],
[ 3, 13, 4, 14, 5, 15],
[ 6, 16, 7, 17, 8, 18]])
Here is a compact way to write it with a loop, it might be generalizable to higher dimension arrays with a little work:
x = np.array([[0,1,2], [3,4,5], [6,7,8]])
y = np.array([[10,11,12], [13,14,15], [16,17,18]])
z = np.zeros((3,6))
for i in xrange(3):
z[i] = np.vstack((x.T[i],y.T[i])).reshape((-1,),order='F')
Task:
As an example, we have 3 input vectors:
foo = [1, 2, 3, 4, 5, 6]
bar = [50, 60, 70, 80, 90, 100]
spam = [-10, -20, -30, -40, -50, -60]
Also, we have 4 output vectors that have linear dependency from input vectors:
foofoo = [1, 1, 2, 2, 3, 3]
barbar = [4, 4, 5, 5, 6, 6]
spamspam = [7, 7, 8, 8, 9, 9]
hamham = [10, 10, 11, 11, 12, 12]
How to use Linear Regression at this data in Python?
You can use OLS (Ordinary Least Squares model) as done here:
#imports
import numpy as np
import statsmodels.api as sm
#generate the input matrix
X=[foo,bar,spam]
#turn it into a numpy array
X = np.array(X).T
#add a constant column
X=sm.add_constant(X)
This gives the input matrix X:
array([[ 1., 1., 50., -10.],
[ 1., 2., 60., -20.],
[ 1., 3., 70., -30.],
[ 1., 4., 80., -40.],
[ 1., 5., 90., -50.],
[ 1., 6., 100., -60.]])
And now you can fit each desired output vector:
resFoo = sm.OLS(endog=foofoo, exog=X).fit()
resBar = sm.OLS(endog=barbar, exog=X).fit()
resSpam = sm.OLS(endog=spamspam, exog=X).fit()
resham = sm.OLS(endog=hamham, exog=X).fit()
The result gives you the coefficients (for the constant, and the three columns foo, bar, and spam):
>>> resFoo.params
array([-0.00063323, 0.0035345 , 0.01001583, -0.035345 ])
You can now check it with the input:
>>> np.matrix(X)*np.matrix(resFoo.params).T
matrix([[ 0.85714286],
[ 1.31428571],
[ 1.77142857],
[ 2.22857143],
[ 2.68571429],
[ 3.14285714]])
Which is close to the desired output of foofoo.
See this question for different ways to do the regression: Multiple linear regression in Python
I have a Numpy ndarray matrix of float values and I need to select spesific rows where certain columns have values satisfying certain criteria. For example lets say I have the following numpy matrix:
matrix = np.ndarray([4, 5])
matrix[0,:] = range(1,6)
matrix[1,:] = range(6,11)
matrix[2,:] = range(11,16)
matrix[3,:] = range(16,21)
Lets say I want to select rows from the matrix where the first column's value is between 1 and 6 and the value of second column is between 2-7.
How can I get the row-indexes of the matrix where these conditions are satisfied? What about if I want to delete the rows that satisfy the conditional criterion?
For a numpy based solution, you can use numpy.where and then get the row indexes from it and then use it for indexing you matrix. Example -
matrix[np.where((1 <= matrix[:,0]) & (matrix[:,0] <= 6)
& (2 <= matrix[:,1]) & (matrix[:,1] <= 7))]
Demo -
In [169]: matrix
Out[169]:
array([[ 1., 2., 3., 4., 5.],
[ 6., 7., 8., 9., 10.],
[ 11., 12., 13., 14., 15.],
[ 16., 17., 18., 19., 20.]])
In [170]: matrix[np.where((1 <= matrix[:,0]) & (matrix[:,0] <= 6)
.....: & (2 <= matrix[:,1]) & (matrix[:,1] <= 7))]
Out[170]:
array([[ 1., 2., 3., 4., 5.],
[ 6., 7., 8., 9., 10.]])
Another method , as indicated in the comments would be to use boolean masks, Example -
mask = ((1 <= matrix[:,0]) & (matrix[:,0] <= 6)
& (2 <= matrix[:,1]) & (matrix[:,1] <= 7))
matrix[mask,:]
Demo -
In [41]: matrix
Out[41]:
array([[ 1., 2., 3., 4., 5.],
[ 6., 7., 8., 9., 10.],
[ 11., 12., 13., 14., 15.],
[ 16., 17., 18., 19., 20.]])
In [42]: mask = ((1 <= matrix[:,0]) & (matrix[:,0] <= 6)
....: & (2 <= matrix[:,1]) & (matrix[:,1] <= 7))
In [43]:
In [43]: matrix[mask,:]
Out[43]:
array([[ 1., 2., 3., 4., 5.],
[ 6., 7., 8., 9., 10.]])
You can get the indices with :
rows = np.logical_and(0 < matrix[:, 0], < matrix[:, 0] < 6 ) * np.logical_and(1 < matrix[:, 1], matrix[:, 1] < 7)
Then newMatrix = np.delete(matrix, rows, axis = 0)
You mentioned MATLAB. Here's the equivalent to the accepted answer using Octave
octave:17> ma=reshape(1:20,5,4)
ma =
1 6 11 16
2 7 12 17
3 8 13 18
4 9 14 19
5 10 15 20
octave:18> mask=(1<=ma(1,:))&(ma(1,:)<=6)&(2<=ma(2,:))&(ma(2,:)<=7)
mask =
1 1 0 0
octave:19> ma(:,mask)
ans =
1 6
2 7
3 8
4 9
5 10
The accepted answer without where is:
In [592]: mask=(1 <= matrix[:,0]) & (matrix[:,0] <= 6) &(2 <= matrix[:,1]) & (matrix[:,1] <= 7)
In [593]: matrix[mask,:]
Out[593]:
array([[ 1., 2., 3., 4., 5.],
[ 6., 7., 8., 9., 10.]])
I switched rows and columns in the Octave version because that is its natural way of generating the same numbers (MATLAB/Octave use the equivalent of numpys 'F' order - see below).
The other changes are 0 v 1 start index, and () v []. Otherwise the two notations are similar.
A simpler way to generate the matrix in numpy:
In [594]: np.arange(1,21).reshape(4,5)
Out[594]:
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]])
Or with the MATLAB layout:
In [595]: np.arange(1,21).reshape(5,4,order='F')
Out[595]:
array([[ 1, 6, 11, 16],
[ 2, 7, 12, 17],
[ 3, 8, 13, 18],
[ 4, 9, 14, 19],
[ 5, 10, 15, 20]])
Get row indices:
row_indices = [x for x in range(4) if matrix[x][0] in range(1,7) and matrix[x][1] in range(2,8)]
Delete rows:
indices = [x for x in range(4) if not( matrix[x][0] in range(1,7) and matrix[x][1] in range(2,8))]
new_matrix = matrix[indices]
I'm trying to resize numpy array, but it seems that the resize works by first flattening the array, then getting first X*Y elem and putting them in the new shape. What I want to do instead is to cut the array at coord 3,3, not rearrange it. Similar thing happens when I try to upsize it say to 7,7 ... instead of "rearranging" I want to fill the new cols and rows with zeros and keep the data as it is.
Is there a way to do that ?
> a = np.zeros((5,5))
> a.flat = range(25)
> a
array(
[[ 0., 1., 2., 3., 4.],
[ 5., 6., 7., 8., 9.],
[ 10., 11., 12., 13., 14.],
[ 15., 16., 17., 18., 19.],
[ 20., 21., 22., 23., 24.]])
> a.resize((3,3),refcheck=False)
> a
array(
[[ 0., 1., 2.],
[ 3., 4., 5.],
[ 6., 7., 8.]])
thank you ...
Upsizing to 7x7 goes like this
upsized = np.zeros([7, 7])
upsized[:5, :5] = a
I believe you want to use numpy's slicing syntax instead of resize. resize works by first raveling the array and working with a 1D view.
>>> a = np.arange(25).reshape(5,5)
>>> a
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
>>> a[:3,:3]
array([[ 0, 1, 2],
[ 5, 6, 7],
[10, 11, 12]])
What you are doing here is taking a view of the numpy array. For example to update the original array by slicing:
>>> a[:3,:3] = 0
>>> a
array([[ 0, 0, 0, 3, 4],
[ 0, 0, 0, 8, 9],
[ 0, 0, 0, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
An excellent guide on numpy's slicing syntax can be found here.
Upsizing (or padding) only works by making a copy of the data. You start with an array of zeros and fill in appropriately
upsized = np.zeros([7, 7])
upsized[:5, :5] = a
I have a numpy_array. Something like [ a b c ].
And then I want to concatenate it with another NumPy array (just like we create a list of lists). How do we create a NumPy array containing NumPy arrays?
I tried to do the following without any luck
>>> M = np.array([])
>>> M
array([], dtype=float64)
>>> M.append(a,axis=0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'numpy.ndarray' object has no attribute 'append'
>>> a
array([1, 2, 3])
In [1]: import numpy as np
In [2]: a = np.array([[1, 2, 3], [4, 5, 6]])
In [3]: b = np.array([[9, 8, 7], [6, 5, 4]])
In [4]: np.concatenate((a, b))
Out[4]:
array([[1, 2, 3],
[4, 5, 6],
[9, 8, 7],
[6, 5, 4]])
or this:
In [1]: a = np.array([1, 2, 3])
In [2]: b = np.array([4, 5, 6])
In [3]: np.vstack((a, b))
Out[3]:
array([[1, 2, 3],
[4, 5, 6]])
Well, the error message says it all: NumPy arrays do not have an append() method. There's a free function numpy.append() however:
numpy.append(M, a)
This will create a new array instead of mutating M in place. Note that using numpy.append() involves copying both arrays. You will get better performing code if you use fixed-sized NumPy arrays.
You may use numpy.append()...
import numpy
B = numpy.array([3])
A = numpy.array([1, 2, 2])
B = numpy.append( B , A )
print B
> [3 1 2 2]
This will not create two separate arrays but will append two arrays into a single dimensional array.
Sven said it all, just be very cautious because of automatic type adjustments when append is called.
In [2]: import numpy as np
In [3]: a = np.array([1,2,3])
In [4]: b = np.array([1.,2.,3.])
In [5]: c = np.array(['a','b','c'])
In [6]: np.append(a,b)
Out[6]: array([ 1., 2., 3., 1., 2., 3.])
In [7]: a.dtype
Out[7]: dtype('int64')
In [8]: np.append(a,c)
Out[8]:
array(['1', '2', '3', 'a', 'b', 'c'],
dtype='|S1')
As you see based on the contents the dtype went from int64 to float32, and then to S1
I found this link while looking for something slightly different, how to start appending array objects to an empty numpy array, but tried all the solutions on this page to no avail.
Then I found this question and answer: How to add a new row to an empty numpy array
The gist here:
The way to "start" the array that you want is:
arr = np.empty((0,3), int)
Then you can use concatenate to add rows like so:
arr = np.concatenate( ( arr, [[x, y, z]] ) , axis=0)
See also https://docs.scipy.org/doc/numpy/reference/generated/numpy.concatenate.html
Actually one can always create an ordinary list of numpy arrays and convert it later.
In [1]: import numpy as np
In [2]: a = np.array([[1,2],[3,4]])
In [3]: b = np.array([[1,2],[3,4]])
In [4]: l = [a]
In [5]: l.append(b)
In [6]: l = np.array(l)
In [7]: l.shape
Out[7]: (2, 2, 2)
In [8]: l
Out[8]:
array([[[1, 2],
[3, 4]],
[[1, 2],
[3, 4]]])
I had the same issue, and I couldn't comment on #Sven Marnach answer (not enough rep, gosh I remember when Stackoverflow first started...) anyway.
Adding a list of random numbers to a 10 X 10 matrix.
myNpArray = np.zeros([1, 10])
for x in range(1,11,1):
randomList = [list(np.random.randint(99, size=10))]
myNpArray = np.vstack((myNpArray, randomList))
myNpArray = myNpArray[1:]
Using np.zeros() an array is created with 1 x 10 zeros.
array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])
Then a list of 10 random numbers is created using np.random and assigned to randomList.
The loop stacks it 10 high. We just have to remember to remove the first empty entry.
myNpArray
array([[31., 10., 19., 78., 95., 58., 3., 47., 30., 56.],
[51., 97., 5., 80., 28., 76., 92., 50., 22., 93.],
[64., 79., 7., 12., 68., 13., 59., 96., 32., 34.],
[44., 22., 46., 56., 73., 42., 62., 4., 62., 83.],
[91., 28., 54., 69., 60., 95., 5., 13., 60., 88.],
[71., 90., 76., 53., 13., 53., 31., 3., 96., 57.],
[33., 87., 81., 7., 53., 46., 5., 8., 20., 71.],
[46., 71., 14., 66., 68., 65., 68., 32., 9., 30.],
[ 1., 35., 96., 92., 72., 52., 88., 86., 94., 88.],
[13., 36., 43., 45., 90., 17., 38., 1., 41., 33.]])
So in a function:
def array_matrix(random_range, array_size):
myNpArray = np.zeros([1, array_size])
for x in range(1, array_size + 1, 1):
randomList = [list(np.random.randint(random_range, size=array_size))]
myNpArray = np.vstack((myNpArray, randomList))
return myNpArray[1:]
a 7 x 7 array using random numbers 0 - 1000
array_matrix(1000, 7)
array([[621., 377., 931., 180., 964., 885., 723.],
[298., 382., 148., 952., 430., 333., 956.],
[398., 596., 732., 422., 656., 348., 470.],
[735., 251., 314., 182., 966., 261., 523.],
[373., 616., 389., 90., 884., 957., 826.],
[587., 963., 66., 154., 111., 529., 945.],
[950., 413., 539., 860., 634., 195., 915.]])
Try this code :
import numpy as np
a1 = np.array([])
n = int(input(""))
for i in range(0,n):
a = int(input(""))
a1 = np.append(a, a1)
a = 0
print(a1)
Also you can use array instead of "a"
If I understand your question, here's one way. Say you have:
a = [4.1, 6.21, 1.0]
so here's some code...
def array_in_array(scalarlist):
return [(x,) for x in scalarlist]
Which leads to:
In [72]: a = [4.1, 6.21, 1.0]
In [73]: a
Out[73]: [4.1, 6.21, 1.0]
In [74]: def array_in_array(scalarlist):
....: return [(x,) for x in scalarlist]
....:
In [75]: b = array_in_array(a)
In [76]: b
Out[76]: [(4.1,), (6.21,), (1.0,)]
As you want to concatenate along an existing axis (row wise), np.vstack or np.concatenate will work for you.
For a detailed list of concatenation operations, refer to the official docs.
This is for people working with numpy's ndarrays. The function numpy.concatenate() does work as well.
>>a = np.random.randint(0,9, size=(10,1,5,4))
>>a.shape
(10, 1, 5, 4)
>>b = np.random.randint(0,9, size=(15,1,5,4))
>>b.shape
(15, 1, 5, 4)
>>X = np.concatenate((a, b))
>>X.shape
(25, 1, 5, 4)
Much the same way as vstack()
>>Y = np.vstack((a,b))
>>Y.shape
(25, 1, 5, 4)
There is a handfull of method to stack arrays together, depending on the direction of the stack.
You may consider np.stack() (doc), np.vstack() (doc) and np.hstack() (doc) for example.