Related
Generation of a list of many lists each with different ranges
Isc_act = [0.1, 0.2, 0.3]
I_cel = []
a = []
for i in range(0,len(Isc_act)):
a = np.arange(0, Isc_act[i], 0.1*Isc_act[i])
I_cel[i].append(a)
print(I_cel)
Output is:
IndexError: list index out of range
My code is giving error. But, I want to get I_cel = [[0,0.01,..,0.1],[0,0.02,0.04,...,0.2],[0, 0.03, 0.06,...,0.3]]. Hence, the 'nested list' I_cel has three lists and each list has 10 values.
The simplest fix to your code, probably what you were intending to do:
Isc_act = [0.1, 0.2, 0.3]
I_cel = []
for i in range(0,len(Isc_act)):
a = np.arange(0, Isc_act[i], 0.1*Isc_act[i])
I_cel.append(a)
print(I_cel)
Note that the endpoint will be one step less than you wanted! For example row zero, you have to pick two of the below:
Steps of size 0.01
Start point 0.0 and end point 0.1
10 elements total
You can not have all three.
More numpythonic approach:
>>> Isc_act = [0.1, 0.2, 0.3]
>>> (np.linspace(0, 1, 11).reshape(11,1) # [Isc_act]).T
array([[0. , 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1 ],
[0. , 0.02, 0.04, 0.06, 0.08, 0.1 , 0.12, 0.14, 0.16, 0.18, 0.2 ],
[0. , 0.03, 0.06, 0.09, 0.12, 0.15, 0.18, 0.21, 0.24, 0.27, 0.3 ]])
linspace gives better control of the end point when dealing with floats:
In [84]: [np.linspace(0,x,11) for x in [.1,.2,.3]]
Out[84]:
[array([0. , 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1 ]),
array([0. , 0.02, 0.04, 0.06, 0.08, 0.1 , 0.12, 0.14, 0.16, 0.18, 0.2 ]),
array([0. , 0.03, 0.06, 0.09, 0.12, 0.15, 0.18, 0.21, 0.24, 0.27, 0.3 ])]
Or we could scale just one array (arange with integers is predictable):
In [86]: np.array([.1,.2,.3])[:,None]*np.arange(0,11)
Out[86]:
array([[0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. ],
[0. , 0.2, 0.4, 0.6, 0.8, 1. , 1.2, 1.4, 1.6, 1.8, 2. ],
[0. , 0.3, 0.6, 0.9, 1.2, 1.5, 1.8, 2.1, 2.4, 2.7, 3. ]])
Case 1 (solved): Array A has shape (say) (300,50). Array B is an indices array with the shape (300,5), such that B[i,j] indicate for the row i the index of another row to "concate" next to the row i. The end result is an array C with the shape (300,5,50), such that C[i,j,:] = A[B[i,j],:]. This can be done by calling A[B,:].
Here is small script example for case 1:
import numpy as np
## A is the data array
A = np.arange(20).reshape((5,4))
## B indicate for each row which rows to pull together
B = np.array([[0,2],[1,2],[2,0],[3,4],[4,1]])
A[B,:] #The desired result
Case 2 (unsolved): Same problem, only now A is shaped (100,300,50). If B is the indicies matrix shaped (100,300,5), the end result would be an array C with the shape (100,300,5,50) such that C[i,j,k,:] = A[i,B[i,j,k],:]. A[B,:] doesn't work anymore, because it result with a shape (100,300,5,300,50), due to broadcasting.
How should I approach this with indexing?
One approach would be reshaping to 2D keeping the number of columns intact and then indexing into the first axis with the flattened B indices and finally reshaping back to the desired one.
Thus, the implementation would be -
A.reshape(-1,A.shape[-1])[B.ravel()].reshape(100,300,5,50)
Those reshaping being merely views into the arrays, should be quite efficient.
This solves both cases. Here's a sample run for the case #1 -
1) Inputs :
In [667]: A = np.random.rand(3,4)
...: B = np.random.randint(0,3,(3,5))
...:
2) Original method :
In [668]: A[B,:]
Out[668]:
array([[[ 0.1 , 0.91, 0.1 , 0.98],
[ 0.1 , 0.91, 0.1 , 0.98],
[ 0.1 , 0.91, 0.1 , 0.98],
[ 0.45, 0.16, 0.02, 0.02],
[ 0.1 , 0.91, 0.1 , 0.98]],
[[ 0.45, 0.16, 0.02, 0.02],
[ 0.48, 0.6 , 0.96, 0.21],
[ 0.48, 0.6 , 0.96, 0.21],
[ 0.1 , 0.91, 0.1 , 0.98],
[ 0.45, 0.16, 0.02, 0.02]],
[[ 0.48, 0.6 , 0.96, 0.21],
[ 0.45, 0.16, 0.02, 0.02],
[ 0.48, 0.6 , 0.96, 0.21],
[ 0.45, 0.16, 0.02, 0.02],
[ 0.45, 0.16, 0.02, 0.02]]])
3) Proposed method :
In [669]: A.reshape(-1,A.shape[-1])[B.ravel()].reshape(3,5,4)
Out[669]:
array([[[ 0.1 , 0.91, 0.1 , 0.98],
[ 0.1 , 0.91, 0.1 , 0.98],
[ 0.1 , 0.91, 0.1 , 0.98],
[ 0.45, 0.16, 0.02, 0.02],
[ 0.1 , 0.91, 0.1 , 0.98]],
[[ 0.45, 0.16, 0.02, 0.02],
[ 0.48, 0.6 , 0.96, 0.21],
[ 0.48, 0.6 , 0.96, 0.21],
[ 0.1 , 0.91, 0.1 , 0.98],
[ 0.45, 0.16, 0.02, 0.02]],
[[ 0.48, 0.6 , 0.96, 0.21],
[ 0.45, 0.16, 0.02, 0.02],
[ 0.48, 0.6 , 0.96, 0.21],
[ 0.45, 0.16, 0.02, 0.02],
[ 0.45, 0.16, 0.02, 0.02]]])
I have numpy arrays of shape (600,600,3), where the values are [-1.0, 1.0]. I would like to expand the array to (600,600,6), where the original values are split into the amounts above and below 0. Some examples (1,1,3) arrays, where th function foo() does the trick:
>>> a = [-0.5, 0.2, 0.9]
>>> foo(a)
[0.0, 0.5, 0.2, 0.0, 0.9, 0.0] # [positive component, negative component, ...]
>>> b = [1.0, 0.0, -0.3] # notice the behavior of 0.0
>>> foo(b)
[1.0, 0.0, 0.0, 0.0, 0.0, 0.3]
Use slicing to assign the min/max to different parts of the output array
In [33]: a = np.around(np.random.random((2,2,3))-0.5, 1)
In [34]: a
Out[34]:
array([[[-0.1, 0.3, 0.3],
[ 0.3, -0.2, -0.1]],
[[-0. , -0.2, 0.3],
[-0.1, -0. , 0.1]]])
In [35]: out = np.zeros((2,2,6))
In [36]: out[:,:,::2] = np.maximum(a, 0)
In [37]: out[:,:,1::2] = np.maximum(-a, 0)
In [38]: out
Out[38]:
array([[[ 0. , 0.1, 0.3, 0. , 0.3, 0. ],
[ 0.3, 0. , 0. , 0.2, 0. , 0.1]],
[[-0. , 0. , 0. , 0.2, 0.3, 0. ],
[ 0. , 0.1, -0. , 0. , 0.1, 0. ]]])
I am trying to generate a .wav file in python using Numpy. I have voltages ranging between 0-5V and I need to normalize them between -1 and 1 to use them in a .wav file.
I have seen this website which uses numpy to generate a wav file but the algorithm used to normalize is no long available.
Can anyone explain how I would go about generating these values in Python on my Raspberry Pi.
isn't this just a simple calculation? Divide by half the maximum value and minus 1:
In [12]: data=np.linspace(0,5,21)
In [13]: data
Out[13]:
array([ 0. , 0.25, 0.5 , 0.75, 1. , 1.25, 1.5 , 1.75, 2. ,
2.25, 2.5 , 2.75, 3. , 3.25, 3.5 , 3.75, 4. , 4.25,
4.5 , 4.75, 5. ])
In [14]: data/2.5-1.
Out[14]:
array([-1. , -0.9, -0.8, -0.7, -0.6, -0.5, -0.4, -0.3, -0.2, -0.1, 0. ,
0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. ])
The following function should do what you want, irrespective of the range of the input data, i.e., it works also if you have negative values.
import numpy as np
def my_norm(a):
ratio = 2/(np.max(a)-np.min(a))
#as you want your data to be between -1 and 1, everything should be scaled to 2,
#if your desired min and max are other values, replace 2 with your_max - your_min
shift = (np.max(a)+np.min(a))/2
#now you need to shift the center to the middle, this is not the average of the values.
return (a - shift)*ratio
my_norm(data)
You can use the fit_transform method in sklearn.preprocessing.StandardScaler. This method will remove the mean from your data and scale your array to unit variance (-1,1)
from sklearn.preprocessing import StandardScaler
data = np.asarray([[0, 0, 0],
[1, 1, 1],
[2,1, 3]])
data = StandardScaler().fit_transform(data)
And if you print out data, you will now have:
[[-1.22474487 -1.41421356 -1.06904497]
[ 0. 0.70710678 -0.26726124]
[ 1.22474487 0.70710678 1.33630621]]
I have a matrix
A = np.array([[0.2, 0.4, 0.6],
[0.5, 0.5, 0.5],
[0.6, 0.4, 0.2]])
I want a new matrix, where the value of the entry in row i and column j is the product of all the entries of the ith row of A, except for the cell of that row in the jth column.
array([[ 0.24, 0.12, 0.08],
[ 0.25, 0.25, 0.25],
[ 0.08, 0.12, 0.24]])
The solution that first occurred to me was
np.repeat(np.prod(A, 1, keepdims = True), 3, axis = 1) / A
But this only works so long as no entries have values zero.
Any thoughts? Thank you!
Edit: I have developed
B = np.zeros((3, 3))
for i in range(3):
for j in range(3):
B[i, j] = np.prod(i, A[[x for x in range(3) if x != j]])
but surely there is a more elegant way to accomplish this, which makes use of numpy's efficient C backend instead of inefficient python loops?
If you're willing to tolerate a single loop:
B = np.empty_like(A)
for col in range(A.shape[1]):
B[:,col] = np.prod(np.delete(A, col, 1), 1)
That computes what you need, a single column at a time. It is not as efficient as theoretically possible because np.delete() creates a copy; if you care a lot about memory allocation, use a mask instead:
B = np.empty_like(A)
mask = np.ones(A.shape[1], dtype=bool)
for col in range(A.shape[1]):
mask[col] = False
B[:,col] = np.prod(A[:,mask], 1)
mask[col] = True
A variation on your solution using repeat, uses [:,None].
np.prod(A,axis=1)[:,None]/A
My 1st stab at handling 0s is:
In [21]: B
array([[ 0.2, 0.4, 0.6],
[ 0. , 0.5, 0.5],
[ 0.6, 0.4, 0.2]])
In [22]: np.prod(B,axis=1)[:,None]/(B+np.where(B==0,1,0))
array([[ 0.24, 0.12, 0.08],
[ 0. , 0. , 0. ],
[ 0.08, 0.12, 0.24]])
But as the comment pointed out; the [0,1] cell should be 0.25.
This corrects that problem, but now has problems when there are multiple 0s in a row.
In [30]: I=B==0
In [31]: B1=B+np.where(I,1,0)
In [32]: B2=np.prod(B1,axis=1)[:,None]/B1
In [33]: B3=np.prod(B,axis=1)[:,None]/B1
In [34]: np.where(I,B2,B3)
Out[34]:
array([[ 0.24, 0.12, 0.08],
[ 0.25, 0. , 0. ],
[ 0.08, 0.12, 0.24]])
In [55]: C
array([[ 0.2, 0.4, 0.6],
[ 0. , 0.5, 0. ],
[ 0.6, 0.4, 0.2]])
In [64]: np.where(I,sum1[:,None],sum[:,None])/C1
array([[ 0.24, 0.12, 0.08],
[ 0.5 , 0. , 0.5 ],
[ 0.08, 0.12, 0.24]])
Blaz Bratanic's epsilon approach is the best non iterative solution (so far):
In [74]: np.prod(C+eps,axis=1)[:,None]/(C+eps)
A different solution iterating over the columns:
def paulj(A):
P = np.ones_like(A)
for i in range(1,A.shape[1]):
P *= np.roll(A, i, axis=1)
return P
In [130]: paulj(A)
array([[ 0.24, 0.12, 0.08],
[ 0.25, 0.25, 0.25],
[ 0.08, 0.12, 0.24]])
In [131]: paulj(B)
array([[ 0.24, 0.12, 0.08],
[ 0.25, 0. , 0. ],
[ 0.08, 0.12, 0.24]])
In [132]: paulj(C)
array([[ 0.24, 0.12, 0.08],
[ 0. , 0. , 0. ],
[ 0.08, 0.12, 0.24]])
I tried some timings on a large matrix
In [13]: A=np.random.randint(0,100,(1000,1000))*0.01
In [14]: timeit paulj(A)
1 loops, best of 3: 23.2 s per loop
In [15]: timeit blaz(A)
10 loops, best of 3: 80.7 ms per loop
In [16]: timeit zwinck1(A)
1 loops, best of 3: 15.3 s per loop
In [17]: timeit zwinck2(A)
1 loops, best of 3: 65.3 s per loop
The epsilon approximation is probably the best speed we can expect, but has some rounding issues. Having to iterate over many columns hurts the speed. I'm not sure why the np.prod(A[:,mask], 1) approach is slowest.
eeclo https://stackoverflow.com/a/22441825/901925 suggested using as_strided. Here's what I think he has in mind (adapted from an overlapping block question, https://stackoverflow.com/a/8070716/901925)
def strided(A):
h,w = A.shape
A2 = np.hstack([A,A])
x,y = A2.strides
strides = (y,x,y)
shape = (w, h, w-1)
blocks = np.lib.stride_tricks.as_strided(A2[:,1:], shape=shape, strides=strides)
P = blocks.prod(2).T # faster to prod on last dim
# alt: shape = (w-1, h, w), and P=blocks.prod(0)
return P
Timing for the (1000,1000) array is quite an improvement over the column iterations, though still much slower than the epsilon approach.
In [153]: timeit strided(A)
1 loops, best of 3: 2.51 s per loop
Another indexing approach, while relatively straight forward, is slower, and produces memory errors sooner.
def foo(A):
h,w = A.shape
I = (np.arange(w)[:,None]+np.arange(1,w))
I1 = np.array(I)%w
P = A[:,I1].prod(2)
return P
Im on the run, so I do not have time to work out this solution; but what id do is create a contiguous circular view over the last axis, by means of concatenating the array to itself along the last axis, and then use np.lib.index_tricks.as_strided to select the appropriate elements to take an np.prod over. No python loops, no numerical approximation.
edit: here you go:
import numpy as np
A = np.array([[0.2, 0.4, 0.6],
[0.5, 0.5, 0.5],
[0.5, 0.0, 0.5],
[0.6, 0.4, 0.2]])
B = np.concatenate((A,A),axis=1)
C = np.lib.index_tricks.as_strided(
B,
A.shape +A.shape[1:],
B.strides+B.strides[1:])
D = np.prod(C[...,1:], axis=-1)
print D
Note: this method is not ideal, as it is O(n^3). See my other posted solution, which is O(n^2)
If you are willing to tolerate small error you could use the solution you first proposed.
A += 1e-10
np.around(np.repeat(np.prod(A, 1, keepdims = True), 3, axis = 1) / A, 9)
Here is an O(n^2) method without python loops or numerical approximation:
def double_cumprod(A):
B = np.empty((A.shape[0],A.shape[1]+1),A.dtype)
B[:,0] = 1
B[:,1:] = A
L = np.cumprod(B, axis=1)
B[:,1:] = A[:,::-1]
R = np.cumprod(B, axis=1)[:,::-1]
return L[:,:-1] * R[:,1:]
Note: it appears to be about twice as slow as the numerical approximation method, which is in line with expectation.