Related
I am trying to calculate a matrix from an array that is inputted.
I would like to be able to input
a = [0,1,2]
in python and would like to reshape it with Numpy such that the result is that the array is in the form of x_i^j at row i and column j,
so for example
the input is:
a = [0,1,2]
and the output should be
[[1,0,0],
[1,1,1],
[1,2,4]]
and I have used the following code
xij = np.matrix([np.power(xi,j) for j in x for xi in x]).reshape(3,3)
[[ 1, 2, 3],
[ 1, 4, 9],
[ 1, 8, 27]]
I assume I'm using the wrong formula for Numpy,
please could you assist me in this to solve the problem.
Thanks in advance
You need to use a range(len(a)) to get the exponents and the correct order of for loops
a = [0,1,2]
xij = np.matrix([np.power(xi,j) for xi in a for j in range(len(a))]).reshape(3,3)
# matrix([[1, 0, 0],
# [1, 1, 1],
# [1, 2, 4]])
With array broadcasting:
In [823]: np.array([0,1,2])**np.arange(3)[:,None]
Out[823]:
array([[1, 1, 1],
[0, 1, 2],
[0, 1, 4]])
In [825]: np.array([1,2,3])**np.arange(1,4)[:,None]
Out[825]:
array([[ 1, 2, 3],
[ 1, 4, 9],
[ 1, 8, 27]])
indexes = np.array([[0,1,3],[1,2,4 ]])
data = np.random.rand(2,5)
Now, i would like an array of shape (2,3), where
result[0] = data[0,indexes[0]]
result[1] = data[1,indexes[1]]
What would be the proper way to achieve this? A numpy way that yould generalize to bigger arrays (perhaps even higher dimensional).
Please note the difference to questions like this, where the array of indexes contains tuples. This is not what I am asking.
Edit
A more general formulation of the question would be:
data.shape == (s0, s1, .., sn)
indexes.shape == (s0, s1, ..., sn-1, K)
so, they have all dimension but the last equal
Than
result[i, j, ..., k] = data[i, j,...,k, indexes[i, j, ..., k]]
where
len([i, j, ..., k]) == len(data)-1 == len(indexes) - 1
Here are NumPy and TensorFlow solutions:
import numpy as np
import tensorflow as tf
def gather_index_np(data, index):
data = np.asarray(data)
index = np.asarray(index)
# Make open grid of all but last dimension indices
grid = np.ogrid[tuple(slice(s) for s in index.shape[:-1])]
# Add extra dimension in grid
grid = [g[..., np.newaxis] for g in grid]
# Complete index
index_full = tuple(grid + [index])
# Index data to get result
result = data[index_full]
return result
def gather_index_tf(data, index):
data = tf.convert_to_tensor(data)
index = tf.convert_to_tensor(index)
index_shape = tf.shape(index)
d = index.shape.ndims
# Make grid of all dimension indices
grid = tf.meshgrid(*(tf.range(index_shape[i]) for i in range(d)), indexing='ij')
# Complete index
index_full = tf.stack(grid[:-1] + [index], axis=-1)
# Index data to get result
result = tf.gather_nd(data, index_full)
return result
Example:
import numpy as np
import tensorflow as tf
data = np.arange(10).reshape((2, 5))
index = np.array([[0, 1, 3], [1, 2, 4]])
print(gather_index_np(data, index))
# [[0 1 3]
# [6 7 9]]
with tf.Session() as sess:
print(sess.run(gather_index_tf(data, index)))
# [[0 1 3]
# [6 7 9]]
numpy has take_along_axis which does what you describe plus it also lets you choose the axis.
Example:
>>> a = np.arange(24).reshape(2,3,4)
>>> i = np.random.randint(0,4,(2,3,5))
>>> i
array([[[3, 3, 0, 1, 3],
[3, 1, 0, 3, 3],
[3, 2, 0, 3, 3]],
[[2, 3, 0, 0, 0],
[1, 1, 3, 1, 2],
[1, 3, 0, 0, 2]]])
>>> np.take_along_axis(a, i, -1)
array([[[ 3, 3, 0, 1, 3],
[ 7, 5, 4, 7, 7],
[11, 10, 8, 11, 11]],
[[14, 15, 12, 12, 12],
[17, 17, 19, 17, 18],
[21, 23, 20, 20, 22]]])
I have the following function, that applies the histogram intersection kernel for 2 arrays:
def histogram_intersection_kernel(X, Y):
k = np.array([])
for x_i,y_i in zip(X,Y):
k = np.append(k,np.minimum(x_i,y_i))
return np.sum(k)
now, lets say I have the following matrix "mat":
[[1,0,0,2,3],
[2,3,4,0,1],
[3,3,5,0,1]]
I would like to find an efficient way to get the matrix that is the result of applying "histogram_intersection_kernel" to all of the combinations of rows in mat. In this example it would be:
[[6,2,2],
[6,10,10],
[2,10,12]]
Extend dimensions to 3D and leverage broadcasting -
np.minimum(a[:,None,:],a[None,:,:]).sum(axis=2)
Or simply -
np.minimum(a[:,None],a).sum(2)
Sample run -
In [248]: a
Out[248]:
array([[1, 0, 0, 2, 3],
[2, 3, 4, 0, 1],
[3, 3, 5, 0, 1]])
In [249]: np.minimum(a[:,None],a).sum(2)
Out[249]:
array([[ 6, 2, 2],
[ 2, 10, 10],
[ 2, 10, 12]])
I randomly generate a matrix. Let's assume for simplicity that it is in the following form np.shape(A) = (2,4):
import numpy as np
A:
matrix([[ 1, 2, 3, 4],
[ 3, 4, 10, 8]])
Then, I estimate the following expression:
import numpy as np
K = 3
I = 4
C0 = np.sum(np.maximum(A[-1] - K, 0)) / I
The question is how do I input the following restriction: if any number of a column in the matrix A is less than or equal to (<=) K (3), then change the last number of that column to zero? So basically, my matrix should transform to this:
A:
matrix([[ 1, 2, 3, 4],
[ 0, 0, 0, 8]])
This is one way.
A[-1][np.any(A <= 3, axis=0)] = 0
# matrix([[1, 2, 3, 4],
# [0, 0, 0, 8]])
A[-1][np.any((A > 2) & (A <= 3), axis=0)] = 0
# matrix([[1, 2, 3, 4],
# [0, 4, 0, 8]])
Let's say I have a matrix like this:
import numpy as np
a = np.array([[1, 2, 3], [89, 43, 2], [12, -3, 4], [-2, 4, 7]])
array([[ 1, 2, 3],
[89, 43, 2],
[12, -3, 4],
[-2, 4, 7]])
and a vector that looks like this:
b = np.array([1, 2, 3])
If I now want to do a elementwise multiplication I can simply do
c = a * b
and obtain
array([[ 1, 4, 9],
[89, 86, 6],
[12, -6, 12],
[-2, 8, 21]])
My question is: How can I do this kind of multiplication only for certain rows in my matrix? I currently do it like this:
E = a.copy()
# ignore these rows
ignInd = [1, 3]
for ind in xrange(a.shape[0]):
if ind not in ignInd:
E[ind, :] = a[ind, :] * b
The matrix E looks as desired (the rows 1 and 3 are the same as in a):
array([[ 1, 4, 9],
[89, 43, 2],
[12, -6, 12],
[-2, 4, 7]])
Can someone come up with a smarter solution than this?
It seems like you could just do the multiplication and then put back the original data where you want to ignore ...
>>> import numpy as np
>>> a = np.array([[1,2,3],[89,43,2],[12, -3, 4], [-2, 4, 7]])
>>> b = np.array([1,2,3])
>>> c = a * b
>>> ignInd = [1,3]
>>> c[ignInd, :]
array([[89, 86, 6],
[-2, 8, 21]])
>>> c[ignInd, :] = a[ignInd, :]
>>> c
array([[ 1, 4, 9],
[89, 43, 2],
[12, -6, 12],
[-2, 4, 7]])
You can index an NumPy array directly with another NumPy array. In your case, you have the indexes of rows you want to ignore, so you can build an array of indexes to include from this:
In [21]: ignInd = [1,3] #ignore these rows
In [22]: ind = np.array([i for i in range(a.shape[0]) if i not in ignInd])
In [23]: E2 = a.copy()
In [24]: E2[ind,:] = a[ind,:]*b
In [25]: E2
Out[25]:
array([[ 1, 4, 9],
[89, 43, 2],
[12, -6, 12],
[-2, 4, 7]])
EDIT: as #DSM comments, for large arrays it would be more efficient to build the index array using NumPy's vectorized methods, viz. ind = np.setdiff1d(np.arange(len(a)), ignInd) instead of the list comprehension used above.
You can use boolean indexing with np.in1d to select the rows excluded in the give indices list. The implementation would look something like this -
E = a.copy()
mask = ~np.in1d(np.arange(a.shape[0]),ignInd,)
E[mask] = a[mask]*b
For the 1st row in a
a[0] = a[0] * b
..and so on for the other rows.