Related
I have a numpy array that is built with coded sections. The sections come in 2x2 blocks. I have a large dictionary of what those 2x2 blocks should be replaced with. How do I replace those 2x2 codes with values in the dictionary.
info_dict = {
5: np.array([[1, 0], [1, 0]], "int"),
6: np.array([[1, 0], [0, 1]], "int"),
7: np.array([[0, 1], [1, 0]], "int"),
8: np.array([[1, 1], [0, 0]], "int"),
}
print(np.array([[5, 5, 8, 8], [5, 5, 8, 8], [6, 6, 7, 7], [6, 6, 7, 7]]))
print(np.array([[1, 0, 1, 1], [1, 0, 0, 0], [1, 0, 0, 1], [0, 1, 1, 0]]))
# before (coded)
[[5 5 8 8]
[5 5 8 8]
[6 6 7 7]
[6 6 7 7]]
# after (final matrix)
[[1 0 1 1]
[1 0 0 0]
[1 0 0 1]
[0 1 1 0]]
For reference
#5
[[1 0]
[1 0]]
#6
[[1 0]
[0 1]]
#7
[[0 1]
[1 0]]
#8
[[1 1]
[0 0]]
One way to do it:
import numpy as np
info_dict = {
5: np.array([[1, 0], [1, 0]], "int"),
6: np.array([[1, 0], [0, 1]], "int"),
7: np.array([[0, 1], [1, 0]], "int"),
8: np.array([[1, 1], [0, 0]], "int"),
}
a = np.array([[5, 5, 8, 8], [5, 5, 8, 8], [6, 6, 7, 7], [6, 6, 7, 7]])
np.block([[info_dict[b] for b in r] for r in a[::2, ::2]])
It gives:
[[1 0 1 1]
[1 0 0 0]
[1 0 0 1]
[0 1 1 0]]
Assuming your dictionary has a small number of integers in it, and your squares are N to a side, you info_dict as follows:
mapping = np.zeros((max(info_dict) + 1, N, N), dtype=int)
for channel, value in info_dict.items():
mapping[channel] = value
If you can store the dictionary in this format to begin with, that's even better. Now you can use simple index with some rearrangement to get the result:
after = mapping[before[::N, ::N]].transpose(0, 2, 1, 3).reshape(before.shape)
So, I would like to stack couple 2d arrays to vector so it would look like this:
[[[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]]
[[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]]
[[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]]]
I can make smth like this:
import numpy as np
a = np.zeros((5, 5), dtype=int)
b = np.zeros((5, 5), dtype=int)
c = np.stack((a, b), 0)
print(c)
To get this:
[[[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]]
[[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]]]
But I cant figure out how to add third 2d array to such vector or how to create such vector of 2d arrays iteratively in a loop. Append, stack, concat just dont keep the needed shape
So, any suggestions?
Thank you!
Conclusion:
Thanks to Tom and Mozway we've got two answers
Tom's:
data_x_train = x_train[np.where((y_train==0) | (y_train==1))
Mozway's:
out = np.empty((0,5,5))
while condition:
# get new array
a = XXX
out = np.r_[out, a[None]]
out
Assuming the following arrays:
a = np.ones((5, 5), dtype=int)
b = np.ones((5, 5), dtype=int)*2
c = np.ones((5, 5), dtype=int)*3
You can stack all at once using:
np.stack((a, b, c), 0)
If you really need to add the arrays iteratively, you can use np.r_:
out = a[None]
for i in (b,c):
out = np.r_[out, i[None]]
output:
array([[[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1]],
[[2, 2, 2, 2, 2],
[2, 2, 2, 2, 2],
[2, 2, 2, 2, 2],
[2, 2, 2, 2, 2],
[2, 2, 2, 2, 2]],
[[3, 3, 3, 3, 3],
[3, 3, 3, 3, 3],
[3, 3, 3, 3, 3],
[3, 3, 3, 3, 3],
[3, 3, 3, 3, 3]]])
edit: if you do not know the arrays in advance
out = np.empty((0,5,5))
while condition:
# get new array
a = XXX
out = np.r_[out, a[None]]
out
Do you mean something like:
np.tile(a, (3, 1, 1))
array([[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]])
Edit:
Do you mean something like:
test = np.tile(a, (3000, 1, 1))
filtered_subset = test[[1, 10, 100], :, :]
array([[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]])
I have a requirement where I want to convert a 2D matrix to 3D by separating 3 unique values across 3 dimensions.
For Example:
convert
A = [1 2 3 3
1 1 2 1
3 2 2 3
1 3 3 2]
to
A = [[1 0 0 0
1 1 0 1
0 0 0 0
1 0 0 0]
[0 1 0 0
0 0 1 0
0 1 1 0
0 0 0 1]
[0 0 1 1
0 0 0 0
1 0 0 1
0 1 1 0]]
Pardon me if the syntax of matrix representation is not correct.
Use broadcasting with outer-equality for a vectorized solution -
# Input array
In [8]: A
Out[8]:
array([[1, 2, 3, 3],
[1, 1, 2, 1],
[3, 2, 2, 3],
[1, 3, 3, 2]])
In [11]: np.equal.outer(np.unique(A),A).view('i1')
Out[11]:
array([[[1, 0, 0, 0],
[1, 1, 0, 1],
[0, 0, 0, 0],
[1, 0, 0, 0]],
[[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 1, 1, 0],
[0, 0, 0, 1]],
[[0, 0, 1, 1],
[0, 0, 0, 0],
[1, 0, 0, 1],
[0, 1, 1, 0]]], dtype=int8)
To use the explicit dimension-extension + comparison, it would be :
(A == np.unique(A)[:,None,None]).view('i1')
You can use np.unique and take advantage of boolean arrays and cast them to int using numpy.ndarray.astype.
import numpy as np
a=np.array([[1, 2, 3, 3], [1, 1, 2, 1], [3, 2, 2, 3], [1, 3, 3, 2]])
[a==i.astype(int) for i in np.unique(a)]
Output:
[array([[1, 0, 0, 0],
[1, 1, 0, 1],
[0, 0, 0, 0],
[1, 0, 0, 0]]),
array([[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 1, 1, 0],
[0, 0, 0, 1]]),
array([[0, 0, 1, 1],
[0, 0, 0, 0],
[1, 0, 0, 1],
[0, 1, 1, 0]])]
EDIT: Ch3steR's answer is better
A = np.array([[1,2,3,3], [1,1,2,1], [3,2,2,3], [1,3,3,2]])
unique_values = np.unique(A)
B = np.array([np.zeros_like(A) for i in range(len(unique_values))])
for idx, value in enumerate(unique_values):
B[idx][A == value] = 1
I am trying to re-implement the below function (written in numpy) using Tensorflow 1.9.0.
def lateral_inhibition2(conv_spikes,SpikesPerNeuronAllowed):
vbn = np.where(SpikesPerNeuronAllowed==0)
conv_spikes[vbn[0],vbn[1],:]=0
return conv_spikes
conv_spikes is a binary tensor of rank 3 and SpikesPerNeuronAllowed is tensor of rank 2. conv_spikes is a variable that indicates if a neuron in a specific location has spiked if the location contains 1 and a 0 indicates that neuron in that location hasn't spiked. SpikesPerNeuronAllowed variable indicates if all the neurons in a X-Y location along the Z axis are allowed to spike or not. A 1 in SpikesPerNeuronAllowed indicates that neurons at the corresponding X-Y location in conv_spikes and along the Z axis are allowed to spike. A 0 indicates that neurons at the corresponding X-Y location in conv_spikes and along the Z axis are not allowed to spike.
conv_spikes2 = (np.random.rand(5,5,3)>=0.5).astype(np.int16)
temp2 = np.random.choice([0, 1], size=(25,), p=[3./4, 1./4])
SpikesPerNeuronAllowed2 = temp2.reshape(5,5)
print(conv_spikes2[:,:,0])
print
print(conv_spikes2[:,:,1])
print
print(conv_spikes2[:,:,2])
print
print(SpikesPerNeuronAllowed2)
produces the following output
##First slice of conv_spikes across Z-axis
[[0 0 1 1 1]
[1 0 0 1 1]
[1 0 1 1 0]
[0 1 0 1 1]
[0 1 0 0 0]]
##Second slice of conv_spikes across Z-axis
[[0 0 1 0 0]
[0 0 1 0 1]
[0 0 1 1 1]
[0 0 0 1 0]
[1 1 1 1 1]]
##Third slice of conv_spikes across Z-axis
[[0 1 1 0 0]
[0 0 1 0 0]
[0 1 1 0 0]
[0 0 0 1 0]
[1 0 1 1 1]]
##SpikesPerNeuronAllowed2
[[0 0 0 0 1]
[0 0 0 0 0]
[0 0 0 0 0]
[1 1 0 0 0]
[0 0 0 1 0]]
Now, when the function is called
conv_spikes2 = lateral_inhibition2(conv_spikes2,SpikesPerNeuronAllowed2)
print(conv_spikes2[:,:,0])
print
print(conv_spikes2[:,:,1])
print
print(conv_spikes2[:,:,2])
produces the following output
##First slice of conv_spikes across Z-axis
[[0 0 0 0 1]
[0 0 0 0 0]
[0 0 0 0 0]
[0 1 0 0 0]
[0 0 0 0 0]]
##Second slice of conv_spikes across Z-axis
[[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 1 0]]
##Third slice of conv_spikes across Z-axis
[[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 1 0]]
I tried to repeat the same in Tensorflow as belows
conv_spikes_tf = tf.Variable((np.random.rand(5,5,3)>=0.5).astype(np.int16))
a_placeholder = tf.placeholder(tf.float32,shape=(5,5))
b_placeholder = tf.placeholder(tf.float32)
inter2 = tf.where(tf.equal(a_placeholder,b_placeholder))
output= sess.run(inter2,feed_dict{a_placeholder:SpikesPerNeuronAllowed2,b_placeholder:0})
print(output)
produces the below output
[[0 0]
[0 1]
[0 2]
[0 3]
[1 0]
[1 1]
[1 2]
[1 3]
[1 4]
[2 0]
[2 1]
[2 2]
[2 3]
[2 4]
[3 2]
[3 3]
[3 4]
[4 0]
[4 1]
[4 2]
[4 4]]
I try to update conv_spikes_tf with the below code results in an error, I tried going through the manual for scatter_nd_update but I don't think I understood very well.
update = tf.scatter_nd_update(conv_spikes_tf, output, np.zeros(output.shape[0]))
sess.run(update)
ValueError: The inner 1 dimensions of input.shape=[5,5,3] must match the inner 1 dimensions of updates.shape=[21,2]: Dimension 0 in both shapes must be equal, but are 3 and 2. Shapes are [3] and [2]. for 'ScatterNdUpdate_8' (op: 'ScatterNdUpdate') with input shapes: [5,5,3], [21,2], [21,2].
I don't understand the error message, specifically what is inner 1 dimensions mean and how can I achieve the above numpy functionality with tensorflow?
The last dim of updates in tf.scatter_nd_update should be 3, which is equal to the last dim of ref.
update = tf.scatter_nd_update(conv_spikes_tf, output, np.zeros(output.shape[0], 3))
If I understand correctly, you want to apply SpikesPerNeuronAllowed2(mask) to conv_spikes. A easier way is to reshape conv_spikes to (3,5,5) and multiply SpikesPerNeuronAllowed2.
I use a constant example to show the result. You can change it to tf.Variable as well.
conv = (np.random.rand(3,5,5)>=0.5).astype(np.int32)
tmp = np.random.choice([0, 1], size=(25,), p=[3./4, 1./4])
mask = tmp.reshape(5,5)
# array([[[1, 1, 0, 0, 0],
# [0, 1, 0, 0, 1],
# [0, 1, 0, 0, 1],
# [1, 0, 0, 0, 1],
# [1, 0, 0, 1, 0]],
# [[1, 0, 0, 0, 1],
# [1, 0, 1, 1, 1],
# [0, 0, 1, 0, 1],
# [0, 0, 0, 1, 1],
# [0, 0, 0, 1, 1]],
# [[0, 0, 0, 1, 0],
# [0, 1, 1, 0, 1],
# [0, 1, 1, 0, 1],
# [1, 1, 1, 1, 0],
# [1, 1, 1, 0, 1]]], dtype=int32)
# array([[0, 0, 0, 1, 1],
# [0, 0, 0, 1, 0],
# [0, 0, 0, 0, 0],
# [0, 1, 0, 1, 0],
# [0, 0, 1, 0, 1]])
tf_conv = tf.constant(conv, dtype=tf.int32)
tf_mask = tf.constant(mask, dtype=tf.int32)
res = tf_conv * tf_mask
sess = tf.InteractiveSession()
sess.run(res)
# array([[[0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0]],
# [[0, 0, 0, 0, 1],
# [0, 0, 0, 1, 0],
# [0, 0, 0, 0, 0],
# [0, 0, 0, 1, 0],
# [0, 0, 0, 0, 1]],
# [[0, 0, 0, 1, 0],
# [0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0],
# [0, 1, 0, 1, 0],
# [0, 0, 1, 0, 1]]], dtype=int32)
I want to create matrixes that contain range of integers like
[[range(1, 3), 0, 0],
[range(5, 7), range(0, 2), 0],
[0, range(1, 3), 0]]
Answer that I expected is
[[[1 0 0], [5 0 0], [0, 1, 0]]
[[2 0 0], [5 0 0], [0, 1, 0]]
[[1 0 0], [6 0 0], [0, 1, 0]]
[[2 0 0], [6 0 0], [0, 1, 0]]
[[1 0 0], [5 0 0], [0, 2, 0]]
[[2 0 0], [5 0 0], [0, 2, 0]]
[[1 0 0], [6 0 0], [0, 2, 0]]
[[2 0 0], [6 0 0], [0, 2, 0]]]
rather than range object inside matrix.
I can achieve my goal with using loops
def foo(p1_range, p2_range, p3_range)
for p1 in range(p1_range):
for p2 in range(p2_range):
for p3 in range(p3_range):
yield [[p1 0 0], [p2 0 0], [0, p3, 0]]
Is there any pythonic way to solve this?
Note: I normaly need 7 parameters in my main code... It looks awful with this way.
You can use itertools.product to reduce indentation levels and eliminate the nested loops:
from itertools import product
def foo(p1_range, p2_range, p3_range):
for p1, p2, p3 in product(p1_range, p2_range, p3_range):
yield [[p1, 0, 0], [p2, 0, 0], [0, p3, 0]]