In the package scipy there is the function to define a binary structure (such as a taxicab (2,1) or a chessboard (2,2)).
import numpy
from scipy import ndimage
a = numpy.zeros((6,6), dtype=numpy.int)
a[1:5, 1:5] = 1;a[3,3] = 0 ; a[2,2] = 2
s = ndimage.generate_binary_structure(2,2) # Binary structure
#.... Calculate Sum of
result_array = numpy.zeros_like(a)
What i want is to iterate over all cells of this array with the given structure s. Then i want to append a function to the current cell value indexed in a empty array (example function sum), which uses the values of all cells in the binary structure.
For example:
array([[0, 0, 0, 0, 0, 0],
[0, 1, 1, 1, 1, 0],
[0, 1, 2, 1, 1, 0],
[0, 1, 1, 0, 1, 0],
[0, 1, 1, 1, 1, 0],
[0, 0, 0, 0, 0, 0]])
# The array a. The value in cell 1,2 is currently one. Given the structure s and an example function such as sum the value in the resulting array (result_array) becomes 7 (or 6 if the current cell value is excluded).
Someone got an idea?
For the particular case of sums, you could use ndimage.convolve:
In [42]: import numpy as np
In [43]: a = np.zeros((6,6), dtype=np.int)
a[1:5, 1:5] = 1;
a[3,3] = 0;
a[2,2] = 2
In [48]: s = ndimage.generate_binary_structure(2,2) # Binary structure
In [49]: ndimage.convolve(a,s)
Out[49]:
array([[1, 2, 3, 3, 2, 1],
[2, 5, 7, 7, 4, 2],
[3, 7, 9, 9, 5, 3],
[3, 7, 9, 9, 5, 3],
[2, 4, 5, 5, 3, 2],
[1, 2, 3, 3, 2, 1]])
For the particular case of products, you could use the fact that log(a*b) = log(a)+log(b) to convert the problem back to one involving sums. For example, if we wanted to "product-convolve" b:
b = a[1:-1, 1:-1]
print(b)
# [[1 1 1 1]
# [1 2 1 1]
# [1 1 0 1]
# [1 1 1 1]]
we could compute:
print(np.exp(ndimage.convolve(np.log(b), s, mode = 'constant')))
# [[ 2. 2. 2. 1.]
# [ 2. 0. 0. 0.]
# [ 2. 0. 0. 0.]
# [ 1. 0. 0. 0.]]
The situation becomes more complicated if b includes negative values:
b[0,1] = -1
print(b)
# [[ 1 -1 1 1]
# [ 1 2 1 1]
# [ 1 1 0 1]
# [ 1 1 1 1]]
but not impossible:
logb = np.log(b.astype('complex'))
real, imag = logb.real, logb.imag
print(np.real_if_close(
np.exp(
sum(j * ndimage.convolve(x, s, mode = 'constant')
for x,j in zip((real, imag),(1,1j))))))
# [[-2. -2. -2. 1.]
# [-2. -0. -0. 0.]
# [ 2. 0. 0. 0.]
# [ 1. 0. 0. 0.]]
It's easier if you use a 2-deep wall of zeroes:
In [11]: a0
Out[11]:
array([[ 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 1., 1., 1., 1., 0., 0.],
[ 0., 0., 1., 2., 1., 1., 0., 0.],
[ 0., 0., 1., 1., 0., 1., 0., 0.],
[ 0., 0., 1., 1., 1., 1., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0.]])
In [12]: b0 = zeros_like(a0)
In [13]: for i in range(1,len(a0)-1):
....: for j in range(1,len(a0)-1):
....: b0[i,j] = sum(a0[i-1:i+2, j-1:j+2] * s)
This enables you to multiply the two sub-matrices together and sum, as desired. (You could also do something more elaborate here...)
In [14]: b0
Out[14]:
array([[ 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 1., 2., 3., 3., 2., 1., 0.],
[ 0., 2., 5., 7., 7., 4., 2., 0.],
[ 0., 3., 7., 9., 9., 5., 3., 0.],
[ 0., 3., 7., 9., 9., 5., 3., 0.],
[ 0., 2., 4., 5., 5., 3., 2., 0.],
[ 0., 1., 2., 3., 3., 2., 1., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0.]])
In [15]: b0[1:len(b0)-1, 1:len(b0)-1]
Out[15]:
array([[ 1., 2., 3., 3., 2., 1.],
[ 2., 5., 7., 7., 4., 2.],
[ 3., 7., 9., 9., 5., 3.],
[ 3., 7., 9., 9., 5., 3.],
[ 2., 4., 5., 5., 3., 2.],
[ 1., 2., 3., 3., 2., 1.]])
Related
So I made a question some days ago (here is the question) basically what I'm trying to do is given a 3D array, and a list of coordinates, add values to those coordinates until shortest path changes, the user Mohammed Kashif help me with my question but that was with a 2D list of coordinates, now what I'm trying to do is doing the same but with a 3D list of coordinates, the first list of coordinate with the first 5 arrays and the second list of coordinates with the next 5 arrays. This is what I try:
(This is just an example, my list of coordinates, and the 3D array can have more elements)
import numpy as np
import networkx as nx
from copy import deepcopy
arr = np.array ([[[ 0., 303., 43., 26., 20.],
[ 0., 0., 0., 0., 0.],
[ 0., 21., 0., 8., 0.],
[ 0., 47., 5., 0., 4.],
[ 0., 35., 0., 1., 0.]],
[[ 0., 356., 40., 32., 49.],
[ 0., 0., 0., 0., 0.],
[ 0., 36., 0., 1., 0.],
[ 0., 25., 1., 0., 1.],
[ 0., 40., 0., 3., 0.]],
[[ 0., 372., 27., 34., 44.],
[ 0., 0., 0., 0., 0.],
[ 0., 37., 0., 3., 0.],
[ 0., 41., 8., 0., 1.],
[ 0., 34., 0., 6., 0.]],
[[ 0., 300., 46., 37., 46.],
[ 0., 0., 0., 0., 0.],
[ 0., 40., 0., 1., 0.],
[ 0., 48., 2., 0., 5.],
[ 0., 43., 0., 2., 0.]],
[[ 0., 321., 42., 22., 22.],
[ 0., 0., 0., 0., 0.],
[ 0., 42., 0., 3., 0.],
[ 0., 20., 3., 0., 5.],
[ 0., 20., 0., 9., 0.]],
[[ 0., 319., 48., 21., 39.],
[ 0., 0., 0., 0., 0.],
[ 0., 29., 0., 0., 1.],
[ 0., 38., 0., 0., 7.],
[ 0., 40., 1., 5., 0.]],
[[ 0., 374., 46., 25., 28.],
[ 0., 0., 0., 0., 0.],
[ 0., 25., 0., 0., 2.],
[ 0., 44., 0., 0., 6.],
[ 0., 44., 2., 9., 0.]],
[[ 0., 341., 34., 21., 49.],
[ 0., 0., 0., 0., 0.],
[ 0., 27., 0., 0., 9.],
[ 0., 25., 0., 0., 8.],
[ 0., 49., 1., 1., 0.]],
[[ 0., 310., 30., 44., 47.],
[ 0., 0., 0., 0., 0.],
[ 0., 34., 0., 0., 2.],
[ 0., 21., 0., 0., 8.],
[ 0., 37., 9., 8., 0.]],
[[ 0., 321., 27., 44., 31.],
[ 0., 0., 0., 0., 0.],
[ 0., 21., 0., 0., 5.],
[ 0., 41., 0., 0., 1.],
[ 0., 41., 1., 5., 0.]]])
n = 5 #Number of array for each path list and coordinates list
graphs = []
pathaux = []
for i in arr:
graphs.append(nx.from_numpy_array(i, create_using = nx.DiGraph)) #Create graphs from numpy array
for graph in graphs:
pathaux.append(nx.shortest_path(graph, 0, 1, weight = 'weight')) #Find the shortest path
path = [pathaux[i: i+n] for i in range(0, len(pathaux), n)] #Creates a 3D list "separating" for each 5 arrays
print(path)
#path = [[[0, 4, 3, 2, 1], [0, 3, 1], [0, 2, 1], [0, 3, 2, 1], [0, 3, 1]], [[0, 3, 4, 2, 1], [0, 4, 2, 1], [0, 3, 1], [0, 2, 4, 3, 1], [0, 2, 1]]]
#path[0] = [[0, 4, 3, 2, 1], [0, 3, 1], [0, 2, 1], [0, 3, 2, 1], [0, 3, 1]] #Shortest path for the first 5 arrays
#path[1] = [[0, 3, 4, 2, 1], [0, 4, 2, 1], [0, 3, 1], [0, 2, 4, 3, 1], [0, 2, 1]] #Shortest path for the next 5 arrays
#Here are the 3D list of coordinates, the first two pair is for the first 5 array, the next four pairs are for the other 5 arrays
coordinates = [[[4, 3], [3, 2]], [[3, 4], [4, 2], [2, 4], [4, 3]]]
for i in coordinates:
for x,y in i:
# Make deepcopies of path and arr
# For the first iteration, set newpath = path
new_path = deepcopy(pathaux)
temp_arr = deepcopy(arr)
# Set counter for each coordinate to zero
cnt = 0
# Iterate till a change in path is observed
while pathaux == new_path:
# Add 1 to x,y
temp_arr[:, x, y] = temp_arr[:, x, y] + 1
# Increment the counter
cnt += 1
# Reconstruct the graph and shortest path
temp_graph = []
new_path = []
for i in temp_arr:
temp_graph.append(nx.from_numpy_array(i, create_using = nx.DiGraph))
for graph in temp_graph:
new_path.append(nx.shortest_path(graph, 0, 1, weight = 'weight'))
#If we are out of the loop, this means that
# the shortest path has changed. Print the details.
print("For coordinates X={} and Y={} the change is at {}".format(x, y, cnt))
Here is the output for this code
#For the first 5 arrays
For coordinates X=4 and Y=3 the change is at 3
For coordinates X=3 and Y=2 the change is at 1
#For the next 5 arrays
For coordinates X=3 and Y=4 the change is at 1
For coordinates X=4 and Y=2 the change is at 1
For coordinates X=2 and Y=4 the change is at 1
For coordinates X=4 and Y=3 the change is at 3
As you can see the coordinates are right, but the values of change are wrong, if I do it manually, adding values to the coordinates until my list changes this is my real output:
#For the first 5 arrays
For coordinates X=4 and Y=3 the change is at 5
For coordinates X=3 and Y=2 the change is at 6
#For the next 5 arrays
For coordinates X=3 and Y=4 the change is at 1
For coordinates X=4 and Y=2 the change is at 1
For coordinates X=2 and Y=4 the change is at 3
For coordinates X=4 and Y=3 the change is at 3
I thought about maybe reshape arr from a 3D array to a 4D array with shape (2, 5, 5, 5) but I believe I have to do a double for-loop so its more slow, I know it's a long question, but I don't know how to make the corresponding first 5 arrays with the first shortest path and the first list of coordinates work together and do the same for the next five arrays with the corresponding shortest path and coordinates. Again, sorry for the long question, any help will be appreciated, thank you!
You should consider partitions separately. Apply the following changes to your existing code, and it should work:
...
arr_partition = [arr[i: i+n] for i in range(0, len(arr), n)]
j = 0
for i in coordinates:
print("The tree is", j)
for x, y in i:
new_path = deepcopy(path[j])
temp_arr = deepcopy(arr_partition[j])
cnt = 0
while path[j] == new_path:
...
print()
j += 1
...
Output:
The tree is 0
For coordinates X=4 and Y=3 the change is at 5
For coordinates X=3 and Y=2 the change is at 6
The tree is 1
For coordinates X=3 and Y=4 the change is at 1
For coordinates X=4 and Y=2 the change is at 1
For coordinates X=2 and Y=4 the change is at 3
For coordinates X=4 and Y=3 the change is at 3
I have three two dimensional Numpy arrays x, w, d and want to create a fourth one called a. w and d define only the shape of a with d.shape + w.shape. I want to have x in the entries of a with a zeros elsewhere.
Specifically, I want a loop-free version of this code:
a = np.zeros(d.shape + w.shape)
for j in range(d.shape[1]):
a[:,j,:,j] = x
For example, given:
x = np.array([
[2, 3],
[1, 1],
[8,10],
[0, 1]
])
w = np.array([
[ 0, 1, 1],
[-1,-2, 1]
])
d = np.matmul(x,w)
I want a to be
array([[[[ 2., 0., 0.],
[ 3., 0., 0.]],
[[ 0., 2., 0.],
[ 0., 3., 0.]],
[[ 0., 0., 2.],
[ 0., 0., 3.]]],
[[[ 1., 0., 0.],
[ 1., 0., 0.]],
[[ 0., 1., 0.],
[ 0., 1., 0.]],
[[ 0., 0., 1.],
[ 0., 0., 1.]]],
[[[ 8., 0., 0.],
[10., 0., 0.]],
[[ 0., 8., 0.],
[ 0., 10., 0.]],
[[ 0., 0., 8.],
[ 0., 0., 10.]]],
[[[ 0., 0., 0.],
[ 1., 0., 0.]],
[[ 0., 0., 0.],
[ 0., 1., 0.]],
[[ 0., 0., 0.],
[ 0., 0., 1.]]]])
This answer inspired the following solution:
# shape a: (4, 3, 2, 3)
# shape x: (4, 2)
a = np.zeros(d.shape + w.shape)
a[:, np.arange(a.shape[1]), :, np.arange(a.shape[3])] = x
It uses Numpy's broadcasting (see here or here) im combination with Advanced Indexing to enlarge x to fit the slicing.
I happen to have an even simpler solution: a = np.tensordot(x, np.identity(3), axes = 0).swapaxes(1,2)
The size of the identity matrix will be decided by the number of times you wish to repeat the elements of x.
In order to compute confusion matrix (not the accuracy) loop over the predicted and true labels may be needed. How to perform that in a numpy manner, if next code does not give needed result?
>> a = np.zeros((5, 5))
>> indices = np.array([
[0, 0],
[2, 2],
[4, 4],
[0, 0],
[2, 2],
[4, 4],
])
np.add.at(a, indices, 1)
>> a
>> array([
[4., 4., 4., 4., 4.],
[0., 0., 0., 0., 0.],
[4., 4., 4., 4., 4.],
[0., 0., 0., 0., 0.],
[4., 4., 4., 4., 4.]
])
# Wanted
>> array([
[2., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 2., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 2.]
])
Docs say If first operand has multiple dimensions, indices can be a tuple of array like index objects or slice objects.
Using next tupling wanted result is reached.
np.add.at(a, (indices[:, 0], indices[:, 1]), 1)
I'm still playing around with tensorflow and been trying to use the gather_nd op, but the return value is not in the shape/format I want...
Input Tensor: - shape: (2, 7, 4)
array([[[ 0., 0., 1., 2.],
[ 0., 0., 2., 2.],
[ 0., 0., 3., 3.],
[ 0., 0., 4., 3.],
[ 0., 0., 5., 4.],
[ 0., 0., 6., 4.],
[ 0., 0., 7., 5.]],
[[ 1., 1., 0., 2.],
[ 1., 2., 0., 2.],
[ 1., 3., 0., 3.],
[ 1., 4., 0., 3.],
[ 1., 5., 0., 4.],
[ 1., 6., 0., 5.],
[ 1., 7., 0., 5.]]], dtype=float32)
Indices returned by tf.where op: - shape: (3, 2)
array([[0, 0],
[0, 1],
[1, 0]])
tf.gather results: (shape = [3, 4])
array([[ 0., 0., 1., 2.],
[ 0., 0., 2., 2.],
[ 1., 1., 0., 2.]], dtype=float32)
desired results: = (2, sparse, 4)
array([[[ 0., 0., 1., 2.],
[ 0., 0., 2., 2.]],
[[ 1., 1., 0., 2.]]], dtype=float32)
What's the best way to achieve this, keeping in mind that tf.where = dynamic shapes and no guarantees of shape consistency across the 2nd dimension (axis=1)?
NB: Ignore this question - See my answer
I think its a Tensorflow version problem. In my version (1.2.1), I get the exact desired output from your inputs. However, I also tried the following code according to the older version.
import tensorflow as tf
indices = [[[0, 0, 0], [0, 0, 1], [0, 0, 2], [0, 0, 3]],
[[0, 1, 0], [0, 1, 1], [0, 1, 2], [0, 1, 3]],
[[1, 0, 0], [1, 0, 1], [1, 0, 2], [1, 0, 3]]]
params = [[[ 0., 0., 1., 2.],
[ 0., 0., 2., 2.],
[ 0., 0., 3., 3.],
[ 0., 0., 4., 3.],
[ 0., 0., 5., 4.],
[ 0., 0., 6., 4.],
[ 0., 0., 7., 5.]],
[[ 1., 1., 0., 2.],
[ 1., 2., 0., 2.],
[ 1., 3., 0., 3.],
[ 1., 4., 0., 3.],
[ 1., 5., 0., 4.],
[ 1., 6., 0., 5.],
[ 1., 7., 0., 5.]]]
output = tf.gather_nd(params, indices)
with tf.Session()as sess:
print (sess.run(output))
Hope this helps.
I realized the idiocy of my question.
# of tuples when 1st dim is 0 != # of tuples when 1st dim is 1
I'm not sure that what I'm asking is feasible ...
>>> x1
array([[ 0., -1., 2.],
[ 3., -4., 2.],
[ -2., 1., -8.]])
>>> x3
array([[ 0., -5., 2.],
[ 3., 0., -3.],
[ 3., 2., 8.]])
I need two matricies to be output: S and T, such that X is the sum of all positive values in X and Y, and T is the sum of all negative values in X and Y.
For example:
S = array([ [ 0., 0., 4.],
[ 6., 0., 2.],
[ 3., 3., 8.]])
T = array([ [ 0., -6., 0.],
[ 0., -4., -3.],
[ -2., 0., -8.]])
I am using Python 2.6.7.
You can use np.clip() to selectively add
In [140]: x1.clip(min=0) + x3.clip(min=0)
Out[140]:
array([[ 0., 0., 4.],
[ 6., 0., 2.],
[ 3., 3., 8.]])
In [141]: x1.clip(max=0) + x3.clip(max=0)
Out[141]:
array([[ 0., -6., 0.],
[ 0., -4., -3.],
[-2., 0., -8.]])
As well as clip you can do this by multiplying by boolean arrays:
>>> x1 * (x1 > 0) + x3 * (x3 > 0)
array([[ 0., -0., 4.],
[ 6., 0., 2.],
[ 3., 3., 8.]])
>>> x1 * (x1 <= 0) + x3 * (x3 <= 0)
array([[ 0., -6., 0.],
[ 0., -4., -3.],
[-2., 0., -8.]])
>>>