Related
I wanna make a function that takes an array as its first parameter takes an arbitrary sized and shaped arr array and overwrites all its values that are in the given [a,b] interval to be equal to c. The a, b, c numbers are given to the function as parameters.like input and output below
arr = np.array([[[5., 2., -5.], [4., 3., 1.]]])
overwrite_interval(arr, -2., 2., 100.) -> ndarray([[[5., 100., -5.], [4., 3., 100.]]])
def overwrite_interval(arr , a , b , c):
for i in arr[:,:]:
arr[a,b] = c
arr = np.array([[[5., 2., -5.], [4., 3., 1.]]])
assert overwrite_interval(arr, -2., 2., 100.) #-> ndarray([[[5., 100., -5.], [4., 3., 100.]]])
I think the way you've worded your question doesn't line up with the example you've given. Firstly, the example array you've given is 3D, not 2D. You can do
>>> arr.shape
(1,2,3)
>>> arr.ndim
3
Presumably this is a mistake, and you want your array to be 2D, so you would do
arr = np.array([[5., 2., -5.], [4., 3., 1.]])
instead.
Secondly, if a and b are values that, if an element is between then to set that element to value c rather than a and b being indexes, then the np.where function is great for this.
def overwrite_interval(arr , a , b , c):
inds = np.where((arr >= a) * (arr <= b))
arr[inds] = c
return arr
np.where returns a tuple, so sometimes it can be easier to work with boolean arrays directly. In which case, the function would look like this
def overwrite_interval(arr , a , b , c):
inds = (arr >= a) * (arr <= b)
arr[inds] = c
return arr
Does this work for you, and is this your intended meaning? Note that the solution I've provided would work as is if you still meant for the initial array to be a 3D array.
This is one of the first things I try to code in python (and any programming language) and my first question here, so I hope I provide everything neccessary to help me.
I have upper triangular matrix and I need to solve system of equations Wx=y, where W (3x3 matrix) and y (vector) are given. I cannot use numpy.linalg functions, so I try to implement this, but backwards of course.
After several failed attempts, I limited my task to 3x3 matrix. Without loop, code looks like this:
x[0,2]=y[2]/W[2,2]
x[0,1]=(y[1]-W[1,2]*x[0,2])/W[1,1]
x[0,0]=(y[0]-W[0,2]*x[0,2]-W[0,1]*x[0,1])/W[0,0]
Now, every new sum contains more elements, which are schematic, but nevertheless need to be defined somehow. I suppose there must be sum function in numpy, but not linalg, which does such things, but I cannot find it.
My newest, partial "attempt" begins with something like this:
n=3
for k in range(n):
for i in range(n-k-1):
x[0,n-k-1]=y[n-k-1]/W[n-k-1,n-k-1]
Which, of course, contains only first element of each sum.
I would be thankful for any assistance.
Example I am working on:
y=np.array([ 0.80064077, 2.64300842, -0.74912957])
W=np.array([[6.244998,2.88230677,-5.44435723],[0.,2.94827198,2.26990852],[0.,0.,0.45441135]]
n=W.shape[1]
x=np.zeros((1,n), dtype=np.float)
Proper solution should look like:
[-2.30857143 2.16571429 -1.64857143]
Here's one approach to use generic n and with one-loop -
def one_loop(y, W, n):
out = np.zeros((1,n))
for i in range(n-1,-1,-1):
sums = (W[i,i+1:]*out[0,i+1:]).sum()
out[0,i] = (y[i] - sums)/W[i,i]
return out
For performance, we can replace that sum-reduction step with a dot-product. Thus, sums could be alternatively computed like so -
sums = W[i,i+1:].dot(x[0,i+1:])
Sample runs
1) n = 3 :
In [149]: y
Out[149]: array([ 5., 8., 7.])
In [150]: W
Out[150]:
array([[ 6., 6., 2.],
[ 3., 3., 3.],
[ 4., 8., 5.]])
In [151]: x = np.zeros((1,3))
...: x[0,2]=y[2]/W[2,2]
...: x[0,1]=(y[1]-W[1,2]*x[0,2])/W[1,1]
...: x[0,0]=(y[0]-W[0,2]*x[0,2]-W[0,1]*x[0,1])/W[0,0]
...:
In [152]: x
Out[152]: array([[-0.9 , 1.26666667, 1.4 ]])
In [154]: one_loop(y, W, n=3)
Out[154]: array([[-0.9 , 1.26666667, 1.4 ]])
2) n = 4 :
In [156]: y
Out[156]: array([ 5., 8., 7., 6.])
In [157]: W
Out[157]:
array([[ 6., 2., 3., 3.],
[ 3., 4., 8., 5.],
[ 8., 6., 6., 4.],
[ 8., 4., 2., 2.]])
In [158]: x = np.zeros((1,4))
...: x[0,3]=y[3]/W[3,3]
...: x[0,2]=(y[2]-W[2,3]*x[0,3])/W[2,2]
...: x[0,1]=(y[1]-W[1,3]*x[0,3]-W[1,2]*x[0,2])/W[1,1]
...: x[0,0]=(y[0]-W[0,3]*x[0,3]-W[0,2]*x[0,2]-W[0,1]*x[0,1])/W[0,0]
...:
In [159]: x
Out[159]: array([[-0.22222222, -0.08333333, -0.83333333, 3. ]])
In [160]: one_loop(y, W, n=4)
Out[160]: array([[-0.22222222, -0.08333333, -0.83333333, 3. ]])
One more take (now updated to the state-of-the-art provided by Divakar in another answer):
import numpy as np
y=np.array([ 0.80064077, 2.64300842, -0.74912957])
W=np.array([[6.244998,2.88230677,-5.44435723],[0.,2.94827198,2.26990852],[0.,0.,0.45441135]])
n=W.shape[1]
x=np.zeros((1,n), dtype=np.float)
for i in range(n-1, -1, -1):
x[0,i] = (y[i]-W[i,i+1:].dot(x[0,i+1:]))/W[i,i]
print(x)
gives:
[[-2.30857143 2.16571429 -1.64857143]]
My take
n=3
for k in range(n):
print("s=y[%d]"% (n-k-1))
s = y[n-k-1]
for i in range(0,k):
print("s - W[%d,%d]*x[0,%d]" % (n-k-1, n-i-1, n-i-1))
s = s - W[n-k-1,n-i-1]*x[0,n-i-1]
print("x[0,%d] = s/W[%d,%d]" % (n-k-1,n-k-1,n-k-1))
x[0,n-k-1] = s/W[n-k-1,n-k-1]
print(x)
and without print statements
n=3
for k in range(n):
s = y[n-k-1]
for i in range(0,k):
s = s - W[n-k-1,n-i-1]*x[0,n-i-1]
x[0,n-k-1] = s/W[n-k-1,n-k-1]
print(x)
Output
s=y[2]
x[0,2] = s/W[2,2]
s=y[1]
s - W[1,2]*x[0,2]
x[0,1] = s/W[1,1]
s=y[0]
s - W[0,2]*x[0,2]
s - W[0,1]*x[0,1]
x[0,0] = s/W[0,0]
[[-2.30857143 2.16571429 -1.64857143]]
I am trying to compute the coefficients of the kth Chebyshev polynomial. Let's just set k to 5 for this. So far, I have the following:
a = (0,0,0,0,0,1) #selects the 5th Chebyshev polynomial
p = numpy.polynomial.chebyshev.Chebyshev(a) #type here is Chebyshev
cpoly = numpy.polynomial.chebyshev.cheb2poly(p) #trying to convert to Poly
print cpoly.all_coeffs()
After the second line runs, I have an object of type Chebyshev, as expected. However, the third line fails to convert to a type Poly, and converts to type numpy.ndarray. Thus, I get an error saying that ndarray has no attribute all_coeffs.
Anyone know how to fix this?
#cel has the right idea in the comments - you need to pass the coefficients of the Chebyshev polynomial to cheb2poly, not the object itself:
import numpy as np
cheb = np.polynomial.chebyshev.Chebyshev((0,0,0,0,0,1))
coef = np.polynomial.chebyshev.cheb2poly(cheb.coef)
print(coef)
# [ 0., 5., 0., -20., 0., 16.]
i.e. 16x5 - 20x3 + 5x. You can confirm that these are the correct coefficients here.
To turn these coefficients into a Polynomial object, you just need to pass the array to the Polynomial constructor:
poly = np.polynomial.Polynomial(coef)
In [1]: import numpy.polynomial
In [2]: p = numpy.polynomial.Chebyshev.basis(5)
In [3]: p
Out[3]: Chebyshev([ 0., 0., 0., 0., 0., 1.], [-1., 1.], [-1., 1.])
In [4]: p.convert(kind=numpy.polynomial.Polynomial)
Out[4]: Polynomial([ 0., 5., 0., -20., 0., 16.], [-1., 1.], [-1., 1.])
I have a large 2D array that I would like to declare once, and change occasionnaly only some values depending on a parameter, without traversing the whole array.
To build this array, I have subclassed the numpy ndarray class with dtype=object and assign to the elements I want to change a function e.g. :
def f(parameter):
return parameter**2
for i in range(np.shape(A)[0]):
A[i,i]=f
for j in range(np.shape(A)[0]):
A[i,j]=1.
I have then overridden the __getitem__ method so that it returns the evaluation of the function with given parameter if it is callable, otherwise return the value itself.
def __getitem__(self, key):
value = super(numpy.ndarray, self).__getitem__(key)
if callable(value):
return value(*self.args)
else:
return value
where self.args were previously given to the instance of myclass.
However, I need to work with float arrays at the end, and I can't simply convert this array into a dtype=float array with this technique. I also tried to use numpy views, which does not work either for dtype=object.
Do you have any better alternative ? Should I override the view method rather than getitem ?
Edit I will maybe have to use Cython in the future, so if you have a solution involving e.g. C pointers, I am interested.
In this case, it does not make sens to bind a transformation function, to every index of your array.
Instead, a more efficient approach would be to define a transformation, as a function, together with a subset of the array it applies to. Here is a basic implementation,
import numpy as np
class LazyEvaluation(object):
def __init__(self):
self.transforms = []
def add_transform(self, function, selection=slice(None), args={}):
self.transforms.append( (function, selection, args))
def __call__(self, x):
y = x.copy()
for function, selection, args in self.transforms:
y[selection] = function(y[selection], **args)
return y
that can be used as follows:
x = np.ones((6, 6))*2
le = LazyEvaluation()
le.add_transform(lambda x: 0, [[3], [0]]) # equivalent to x[3,0]
le.add_transform(lambda x: x**2, (slice(4), slice(4,6))) # equivalent to x[4,4:6]
le.add_transform(lambda x: -1, np.diag_indices(x.shape[0], x.ndim), ) # setting the diagonal
result = le(x)
print(result)
which prints,
array([[-1., 2., 2., 2., 4., 4.],
[ 2., -1., 2., 2., 4., 4.],
[ 2., 2., -1., 2., 4., 4.],
[ 0., 2., 2., -1., 4., 4.],
[ 2., 2., 2., 2., -1., 2.],
[ 2., 2., 2., 2., 2., -1.]])
This way you can easily support all advanced Numpy indexing (element by element access, slicing, fancy indexing etc.), while at the same time keeping your data in an array with a native data type (float, int, etc) which is much more efficient than using dtype='object'.
In pure Python you can grow matrices column by column pretty easily:
data = []
for i in something:
newColumn = getColumnDataAsList(i)
data.append(newColumn)
NumPy's array doesn't have the append function. The hstack function doesn't work on zero sized arrays, thus the following won't work:
data = numpy.array([])
for i in something:
newColumn = getColumnDataAsNumpyArray(i)
data = numpy.hstack((data, newColumn)) # ValueError: arrays must have same number of dimensions
So, my options are either to remove the initalization iside the loop with appropriate condition:
data = None
for i in something:
newColumn = getColumnDataAsNumpyArray(i)
if data is None:
data = newColumn
else:
data = numpy.hstack((data, newColumn)) # works
... or to use a Python list and convert is later to array:
data = []
for i in something:
newColumn = getColumnDataAsNumpyArray(i)
data.append(newColumn)
data = numpy.array(data)
Both variants seem a little bit awkward to be. Are there nicer solutions?
NumPy actually does have an append function, which it seems might do what you want, e.g.,
import numpy as NP
my_data = NP.random.random_integers(0, 9, 9).reshape(3, 3)
new_col = NP.array((5, 5, 5)).reshape(3, 1)
res = NP.append(my_data, new_col, axis=1)
your second snippet (hstack) will work if you add another line, e.g.,
my_data = NP.random.random_integers(0, 9, 16).reshape(4, 4)
# the line to add--does not depend on array dimensions
new_col = NP.zeros_like(my_data[:,-1]).reshape(-1, 1)
res = NP.hstack((my_data, new_col))
hstack gives the same result as concatenate((my_data, new_col), axis=1), i'm not sure how they compare performance-wise.
While that's the most direct answer to your question, i should mention that looping through a data source to populate a target via append, while just fine in python, is not idiomatic NumPy. Here's why:
initializing a NumPy array is relatively expensive, and with this conventional python pattern, you incur that cost, more or less, at each loop iteration (i.e., each append to a NumPy array is roughly like initializing a new array with a different size).
For that reason, the common pattern in NumPy for iterative addition of columns to a 2D array is to initialize an empty target array once(or pre-allocate a single 2D NumPy array having all of the empty columns) the successively populate those empty columns by setting the desired column-wise offset (index)--much easier to show than to explain:
>>> # initialize your skeleton array using 'empty' for lowest-memory footprint
>>> M = NP.empty(shape=(10, 5), dtype=float)
>>> # create a small function to mimic step-wise populating this empty 2D array:
>>> fnx = lambda v : NP.random.randint(0, 10, v)
populate NumPy array as in the OP, except each iteration just re-sets the values of M at successive column-wise offsets
>>> for index, itm in enumerate(range(5)):
M[:,index] = fnx(10)
>>> M
array([[ 1., 7., 0., 8., 7.],
[ 9., 0., 6., 9., 4.],
[ 2., 3., 6., 3., 4.],
[ 3., 4., 1., 0., 5.],
[ 2., 3., 5., 3., 0.],
[ 4., 6., 5., 6., 2.],
[ 0., 6., 1., 6., 8.],
[ 3., 8., 0., 8., 0.],
[ 5., 2., 5., 0., 1.],
[ 0., 6., 5., 9., 1.]])
of course if you don't known in advance what size your array should be
just create one much bigger than you need and trim the 'unused' portions
when you finish populating it
>>> M[:3,:3]
array([[ 9., 3., 1.],
[ 9., 6., 8.],
[ 9., 7., 5.]])
Usually you don't keep resizing a NumPy array when you create it. What don't you like about your third solution? If it's a very large matrix/array, then it might be worth allocating the array before you start assigning its values:
x = len(something)
y = getColumnDataAsNumpyArray.someLengthProperty
data = numpy.zeros( (x,y) )
for i in something:
data[i] = getColumnDataAsNumpyArray(i)
The hstack can work on zero sized arrays:
import numpy as np
N = 5
M = 15
a = np.ndarray(shape = (N, 0))
for i in range(M):
b = np.random.rand(N, 1)
a = np.hstack((a, b))
Generally it is expensive to keep reallocating the NumPy array - so your third solution is really the best performance wise.
However I think hstack will do what you want - the cue is in the error message,
ValueError: arrays must have same number of dimensions`
I'm guessing that newColumn has two dimensions (rather than a 1D vector), so you need data to also have two dimensions..., for example, data = np.array([[]]) - or alternatively make newColumn a 1D vector (generally if things are 1D it is better to keep them 1D in NumPy, so broadcasting, etc. work better). in which case use np.squeeze(newColumn) and hstack or vstack should work with your original definition of the data.