Data = [day(1) day(2)...day(N)...day(2N)..day(K-N)...day(K)]
I am looking to create a numpy array with two arrays, N and K with shapes (120,) and (300,). The array needs to be of the form:
x1 = [day(1) day(2) day (3)...day(N)]
x2 = [day(2) day(3)...day(N) day(N+1)]
xN = [day(N) day(N+1) day(N+2)...day(2N)]
xK-N = [day(K-N) day(K-N+1)...day(K)]
X is basically of shape (K-N)xN, with the above x1,x2,...xK-N as rows. I have tried using iloc for getting two arrays N and K with the same shapes. Good till then. But, when I try to merge the arrays using X = np.array([np.concatenate((N[i:], K[:i] )) for i in range(len(N)]), I am getting an NxN array in the form of an overlap array only, and not in the desired format.
Is this what you are trying to produce (with simpler data)?
In [253]: N,K=10,15
In [254]: data = np.arange(K)+10
In [255]: data
Out[255]: array([10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24])
In [256]: np.array([data[np.arange(N)+i] for i in range(K-N+1)])
Out[256]:
array([[10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
[11, 12, 13, 14, 15, 16, 17, 18, 19, 20],
[12, 13, 14, 15, 16, 17, 18, 19, 20, 21],
[13, 14, 15, 16, 17, 18, 19, 20, 21, 22],
[14, 15, 16, 17, 18, 19, 20, 21, 22, 23],
[15, 16, 17, 18, 19, 20, 21, 22, 23, 24]])
There's another way of generating this, using advanced ideas about strides:
np.lib.stride_tricks.as_strided(data, shape=(K-N+1,N), strides=(4,4))
In the first case, all values in the new array are copies of the original. The strided case is actually a view. So any changes to data appear in the 2d array. And without data copying, the 2nd is also faster. I can try to explain it if you are interested.
Warren suggests using hankel. That's a short function, which in our case does essentially:
a, b = np.ogrid[0:K-N+1, 0:N]
data[a+b]
a+b is an array like:
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[ 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
[ 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
[ 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
[ 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]])
In this example case it is just a bit better than the list comprehension solution, but I expect it will be a lot better for much larger cases.
It is probably not worth adding a dependence on scipy for the following, but if you are already using scipy in your code, you could use the function scipy.linalg.hankel:
In [75]: from scipy.linalg import hankel
In [76]: K = 16
In [77]: x = np.arange(K)
In [78]: x
Out[78]: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])
In [79]: N = 8
In [80]: hankel(x[:K-N+1], x[K-N:])
Out[80]:
array([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 2, 3, 4, 5, 6, 7, 8, 9],
[ 3, 4, 5, 6, 7, 8, 9, 10],
[ 4, 5, 6, 7, 8, 9, 10, 11],
[ 5, 6, 7, 8, 9, 10, 11, 12],
[ 6, 7, 8, 9, 10, 11, 12, 13],
[ 7, 8, 9, 10, 11, 12, 13, 14],
[ 8, 9, 10, 11, 12, 13, 14, 15]])
Related
i have this array that i generated using the default_rng:
import numpy as np
from numpy.random import default_rng
rng = default_rng(seed=10)
rng = rng.integers(1,20,(5,10))
rng
>>>array([[15, 19, 6, 4, 16, 16, 10, 3, 16, 10],
[ 3, 3, 8, 14, 8, 16, 1, 9, 10, 19],
[ 5, 16, 2, 7, 15, 11, 18, 15, 18, 16],
[ 3, 18, 17, 3, 19, 15, 6, 3, 8, 18],
[15, 5, 10, 17, 13, 6, 3, 19, 5, 10]], dtype=int64)
I want to add 10 zeros in this matrix using the generator with seed=5.
I thought to create a new array with dimessions [5,10] and to put 10 zeros inside and the rest to be one and then mutliply the two arrays but i have to use the generator so i can't do this.
Try with np.random.choice to choose the index, then set the values at those indexes to 0:
np.random.seed(10)
idx = np.random.choice(np.arange(5*10), size=5, replace=False)
rng.ravel()[idx] = 0
Output:
array([[15, 19, 6, 4, 16, 16, 10, 3, 16, 10],
[ 3, 3, 8, 14, 8, 16, 1, 9, 10, 19],
[ 5, 16, 2, 0, 15, 11, 18, 15, 18, 16],
[ 3, 18, 17, 3, 19, 15, 6, 0, 8, 18],
[15, 5, 0, 17, 0, 6, 3, 0, 5, 10]])
Of course
idx = np.random.choice(rng.ravel(), 10, replace= False)
print(idx)
rng.ravel()[idx] = 0
rng
Output
[10 17 3 6 15 15 15 16 15 15]
array([[15, 19, 6, 0, 16, 16, 0, 3, 16, 10],
[ 0, 3, 8, 14, 8, 0, 0, 0, 10, 19],
[ 5, 16, 2, 7, 15, 11, 18, 15, 18, 16],
[ 3, 18, 17, 3, 19, 15, 6, 3, 8, 18],
[15, 5, 10, 17, 13, 6, 3, 19, 5, 10]], dtype=int64)
So instead of take 10 zeros i take only 6 becaus of 15 appears five times in my idx.
In Jupyter notebooks, or in IPython, long lists are displayed one element per line. How do I display them on a single line? I don't mind if the line wraps.
In the following example, I'd like the 3rd list to be shown as a "row", not as a "column".
In [1]: [list(range(n)) for n in range(10,40,10)]
Out[1]:
[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
[0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29]]
The output I am looking for is the following or similar:
[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]]
My goal is to make the output easier to read for humans.
I would simply use
A = [list(range(n)) for n in range(10,40,10)]
for i in A:
print(i)
I have an array like matrix using numpy like this.
import numpy as np
a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16], [17, 18, 19, 20]])
the desired array is like this:
array([[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8],])
description: first and second arrays move to the end of the matrix.
I tried something with changing a to a list and used append and del functions and then convert it to a numpy array but it could not be something good to write in python.
is there any function to replace an array position in a larger array-like matrix in numpy?
Function that takes the number of rotations
In [5]: a
Out[5]:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20]])
In [14]: def rotate(n):
...: n = n%len(a)
...: return np.concatenate([a[n:], a[:n]])
In [13]: rotate(2)
Out[13]:
array([[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8]])
What if you give n more than the length of the array? It's handled - n = n%len(a)
In [16]: rotate(9)
Out[16]:
array([[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
Another solution given in comments is roll() method.
In [6]: a
Out[6]:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20]])
In [7]: def rotate(n):
...: n = n % len(a)
...: return np.roll(a,-n,axis=0)
...:
In [8]: rotate(8)
Out[8]:
array([[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
In [9]: rotate(2)
Out[9]:
array([[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8]])
This is so easy if you use this simple line of code. no function and other things are needed.
simply use numpy.roll. see explanations here.
# Assume your matrix is named a.
>>> a
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20]])
>>> np.roll(a,-(n % len(a)),axis=0)
array([[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8]])
Given a numpy array
import numpy as np
a = np.arange(4*7).reshape([4, 7])
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27]])
I can apply slicing to swap the second and third columns by:
a[:, [0, 2, 1, 3, 4, 5, 6]]
array([[ 0, 2, 1, 3, 4, 5, 6],
[ 7, 9, 8, 10, 11, 12, 13],
[14, 16, 15, 17, 18, 19, 20],
[21, 23, 22, 24, 25, 26, 27]])
But, can I use slices to swap the second and third columns for all rows but the first one? The expected output would be:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 9, 8, 10, 11, 12, 13],
[14, 16, 15, 17, 18, 19, 20],
[21, 23, 22, 24, 25, 26, 27]])
For in-situ edit, we can use flipping after slicing out the two columns -
a[1:,1:3] = a[1:,2:0:-1]
Sample run -
In [556]: a = np.arange(4*7).reshape([4, 7])
In [557]: a
Out[557]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27]])
In [559]: a[1:,1:3] = a[1:,2:0:-1]
In [560]: a
Out[560]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 9, 8, 10, 11, 12, 13],
[14, 16, 15, 17, 18, 19, 20],
[21, 23, 22, 24, 25, 26, 27]])
For columns that are two-step apart, use a stepsize of 2 to assign (LHS) and -2 to select (RHS). Hence, for column IDs 1 & 3 -
In [577]: a = np.arange(4*7).reshape([4, 7])
In [578]: a
Out[578]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27]])
In [579]: a[1:,1:4:2] = a[1:,3:0:-2]
In [580]: a
Out[580]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 10, 9, 8, 11, 12, 13],
[14, 17, 16, 15, 18, 19, 20],
[21, 24, 23, 22, 25, 26, 27]])
Another method would be with explicit column numbered indexing -
a[1:,[1,2]] = a[1:,[2,1]]
Note that this creates a copy with a[1:,[2,1]] and as such would be less memory efficient than the first method.
I would like to select every nth group of n columns in a numpy array. It means that I want the first n columns, not the n next columns, the n next columns, not the n next columns etc.
For example, with the following array and n=2:
import numpy as np
arr = np.array([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15, 16, 17, 18, 19, 20]])
I would like to get:
[[1, 2, 5, 6, 9, 10],
[11, 12, 15, 16, 19, 20]]
And with n=3:
[[1, 2, 3, 7, 8, 9],
[11, 12, 13, 17, 18, 19]]
With n=1 we can simply use the syntax arr[:,::2], but is there something similar for n>1?
You can use modulus to create ramps starting from 0 until 2n and then select the first n from each such ramp. Thus, for each ramp, we would have first n as True and rest as False, to give us a boolean array covering the entire length of the array. Then, we simply use boolean indexing along the columns to select the valid columns for the final output. Thus, the implementation would look something like this -
arr[:,np.mod(np.arange(arr.shape[-1]),2*n)<n]
Step by step code runs to give a better idea -
In [43]: arr
Out[43]:
array([[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15, 16, 17, 18, 19, 20]])
In [44]: n = 3
In [45]: np.mod(np.arange(arr.shape[-1]),2*n)
Out[45]: array([0, 1, 2, 3, 4, 5, 0, 1, 2, 3])
In [46]: np.mod(np.arange(arr.shape[-1]),2*n)<n
Out[46]: array([ True,True,True,False,False,False,True,True,True,False])
In [47]: arr[:,np.mod(np.arange(arr.shape[-1]),2*n)<n]
Out[47]:
array([[ 1, 2, 3, 7, 8, 9],
[11, 12, 13, 17, 18, 19]])
Sample runs across various n -
In [29]: arr
Out[29]:
array([[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15, 16, 17, 18, 19, 20]])
In [30]: n = 1
In [31]: arr[:,np.mod(np.arange(arr.shape[-1]),2*n)<n]
Out[31]:
array([[ 1, 3, 5, 7, 9],
[11, 13, 15, 17, 19]])
In [32]: n = 2
In [33]: arr[:,np.mod(np.arange(arr.shape[-1]),2*n)<n]
Out[33]:
array([[ 1, 2, 5, 6, 9, 10],
[11, 12, 15, 16, 19, 20]])
In [34]: n = 3
In [35]: arr[:,np.mod(np.arange(arr.shape[-1]),2*n)<n]
Out[35]:
array([[ 1, 2, 3, 7, 8, 9],
[11, 12, 13, 17, 18, 19]])