Model is predicting only zeroes with one 1 after using softmax activation - python

I'm using Tensorflow 1.15 and Keras 2.3.1
My Keras model with such code:
model = keras.Sequential()
model.add(L.InputLayer(batch_input_shape= (None, 768)))
model.add(L.Dense(input_shape = (None,768), activation='relu', units = 256))
model.add(Dropout(0.25))
model.add(L.Dense(input_shape = (None,256), activation='relu', units = 128))
model.add(Dropout(0.25))
model.add(L.Dense(input_shape=(None,128), activation='softmax', units = len(tensor_val_cat[0])))
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy', precision, recall, f1])
Is predicting this for every sample, while I am using softmax activation in the last Dense layer:
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)
But when I am changing last activation from softmax to reLU, it becomes much better:
array([ 22.15598 , 11.37785 , 4.0738773, 82.95521 , 0. ,
0. , 0. , 18.631542 , 0. , 5.5782046,
0. , 4.9239693, 0. , 16.169178 , 0. ,
6.9248867, 8.373851 , 8.884083 , 9.291541 , 0. ,
0. , 7.2818174, 0. , 0. , 6.06402 ,
12.253789 , 5.980045 , 6.226884 , 8.512158 , 7.8921595,
13.949636 , 7.111844 , 0. , 30.935558 , 0. ,
18.647184 , 0. , 8.561647 , 0. , 10.515696 ,
10.424169 , 9.935862 , 9.651736 , 0. , 0. ,
8.253851 , 26.623386 , 21.973936 , 17.13469 , 0. ,
18.480967 , 0. , 4.4326754, 9.894943 , 0. ,
20.798775 , 3.174389 , 6.4419975, 19.948698 , 5.1431727,
7.7155123, 20.939203 , 0. , 17.534533 , 5.989612 ,
10.36599 , 8.2254505, 0. , 13.186766 , 11.333556 ,
14.58358 , 12.923845 , 0. , 3.8745575, 6.7094135,
24.11962 , 0. , 9.189409 , 0. , 0. ,
0. , 6.585493 , 24.013597 , 14.495815 , 0. ,
6.1411734, 49.32834 , 25.741114 , 11.39783 , 12.999501 ,
16.318121 , 0. , 15.020708 , 0. , 11.767337 ,
260.82144 , 28.074688 , 5.5710874, 8.066694 , 11.450023 ,
0. , 7.5654263, 17.537619 , 0. , 18.570484 ,
3.9268994, 9.846983 , 54.34089 , 6.3504252, 0. ,
12.469551 , 11.262119 , 81.61232 , 0. , 8.149481 ,
0. , 10.952655 , 18.77889 , 45.96867 , 25.36038 ,
0. , 0. , 0. , 10.455843 , 0. ,
39.179607 , 0. , 6.2759237, 18.627535 , 0. ,
3.2353485, 0. , 0. , 15.829307 , 11.45084 ,
6.2554755, 90.0754 , 15.144176 , 0. , 15.053744 ,
0. , 7.800592 , 11.731603 , 0. , 0. ,
0. , 8.28636 , 0. , 0. , 0. ,
53.148182 , 0. , 5.1440187, 0. , 63.715683 ,
32.42369 , 0. , 18.376951 , 13.3018875, 0. ,
0. , 0. , 3.497302 , 3.9504528, 7.3174777,
0. , 6.7174683, 11.641777 , 5.5688257, 0. ,
0. , 11.408187 , 9.692002 , 19.378729 , 0. ,
11.024126 , 11.887186 , 5.125866 , 0. , 0. ,
15.08582 , 0. , 33.485287 , 23.441591 , 10.062655 ,
0. , 8.631276 , 0. , 13.220199 , 0. ,
5.132385 , 0. , 0. , 10.135473 , 13.213521 ,
0. , 12.063043 , 12.336954 , 7.578767 , 0. ,
4.895108 , 27.40385 , 6.0983057, 2.5386598, 24.318102 ,
14.043896 , 0. , 10.418289 , 5.761671 , 17.748152 ,
5.8953934, 0. , 0. , 0. , 9.80458 ,
0. , 20.902594 , 31.065573 , 13.040765 , 8.419372 ,
0. , 11.227969 , 0. , 5.400335 , 0. ,
0. , 5.159615 , 11.675414 , 0. , 0. ,
8.022585 , 12.005584 , 4.038471 , 6.218788 , 10.806794 ,
15.874523 , 11.118925 , 0. , 7.7477336, 0. ,
0. , 7.1560225, 0. , 8.95097 , 0. ,
12.333616 , 11.627193 , 5.4876437, 25.533556 , 5.655847 ,
0. , 14.226913 , 5.2383256, 0. , 5.673967 ,
0. , 5.9973536, 0. , 12.03303 , 6.4873867,
3.6789234, 4.8231483, 0. , 21.4692 , 0. ,
0. , 0. , 0. , 12.463381 , 0. ,
0. , 0. , 10.32079 , 19.267895 , 6.7940717,
3.1542706, 0. , 0. , 0. , 10.24344 ,
11.1568 , 3.5449536, 22.421698 , 0. , 0. ,
7.1022887, 9.147839 , 0. , 0. , 14.933947 ,
6.061274 , 6.5337996, 5.5994735, 0. , 2.0698977,
25.453054 , 0. , 6.533775 , 3.5728502, 13.730647 ],
dtype=float32)
I would like too use softmax activation, so please, can you help me with fixing the stated problem?


softmax squashes all the input values into range [0, 1] and they sum up to 1 which is exactly what you see. So you can treat them as probabilities.
ReLU on the other hand maps the input values to range [0, +∞] and they need not sum up to 1.
As you see the argmax of both the softmax and ReLU is the same (having max values of 1 and 260.82144 respectively. So the problem is really not in the activation function.

Related

Python: Fill out edges of binary array

I'm using the following code to generate an array based on coordinates of edges:
verts = np.array(list(itertools.product((0,2), (0,2))))
arr = np.zeros((5, 5))
arr[tuple(verts.T)] = 1
plt.imshow(arr)
which gives me
or, as a numeric array:
[[1., 0., 1., 0., 0.],
[0., 0., 0., 0., 0.],
[1., 0., 1., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]]
Now, I would like to fill out the spaces in between the corners (ie. yellow squares):
so that I get the following array:
[[1., 1., 1., 0., 0.],
[1., 1., 1., 0., 0.],
[1., 1., 1., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]]

Replace (0,2) using range(0,3) (3 as ranges are inclusive-exclusive) that is
import itertools
import numpy as np
verts = np.array(list(itertools.product(range(0,3), range(0,3))))
arr = np.zeros((5, 5))
arr[tuple(verts.T)] = 1
print(arr)
output
[[1. 1. 1. 0. 0.]
[1. 1. 1. 0. 0.]
[1. 1. 1. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]]

String to List of Arrays in python

I'm working on a ML model, and I have to convert a list of arrays (the weights of the layers) to a string, to send it over MQTT. Then I have to convert it to a list of arrays again, and thats where I don't know how to solve it. The initial look of the list is like this:
Initial list from model.get_weights()
:
[array([[ 0.05541647, -0.00467741, 0.06709623, ..., -0.06240537,
-0.05044469, -0.06255569],
[ 0.05793238, -0.04376897, -0.03331734, ..., 0.04109375,
-0.05561347, -0.05630576],
[ 0.03568218, 0.00916858, 0.02733664, ..., 0.04085189,
0.07445424, 0.05173937],
...,
[ 0.00326935, 0.05949181, -0.02493389, ..., 0.01619817,
0.02883349, -0.00364999],
[ 0.05162556, -0.07704586, -0.00726594, ..., 0.03567791,
0.06234651, 0.05147751],
[-0.04587721, 0.06365172, -0.06174358, ..., -0.07004303,
-0.00196535, -0.05049317]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32), array([[ 0.11118214, -0.00349338, 0.18680657, ..., 0.01847704,
-0.03098661, -0.04094526],
[-0.06314829, 0.00289522, -0.11807185, ..., -0.10976926,
-0.12070866, 0.19067971],
[-0.05408052, -0.02283411, 0.16553403, ..., -0.12856016,
0.00681128, -0.05486405],
...,
[-0.12182648, -0.03314751, 0.04840027, ..., 0.13398318,
-0.092302 , 0.13001741],
[ 0.01030177, 0.14168383, -0.18688273, ..., -0.17727108,
-0.1098071 , -0.12000293],
[ 0.03310342, 0.17201088, -0.08573408, ..., 0.15494372,
-0.16848558, 0.12254588]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)][array([[-0.07572845, 0.07035964, -0.00726507, ..., -0.01283053,
-0.02842413, 0.02551443],
[-0.00741241, -0.02386538, 0.00442091, ..., 0.0693512 ,
0.02695736, -0.07246653],
[ 0.06941632, -0.01986459, 0.02596217, ..., 0.04713184,
0.03926247, 0.07958693],
...,
[ 0.04515444, -0.02030407, -0.00393321, ..., 0.025347 ,
-0.01182116, 0.04929114],
[-0.06743087, 0.02246762, 0.0225632 , ..., 0.03987813,
-0.00048529, 0.00320805],
[ 0.07628443, -0.06414777, 0.04115602, ..., -0.03207976,
-0.01118261, 0.00946496]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32), array([[ 0.19875868, -0.0724885 , -0.15991165, ..., -0.04141769,
0.11540116, 0.1246707 ],
[ 0.03422281, 0.09608312, 0.18289839, ..., 0.20248671,
-0.05454096, -0.11580068],
[ 0.12459688, 0.17984338, 0.02630243, ..., -0.20585045,
-0.08128738, 0.08814187],
...,
[ 0.07335795, -0.02979451, 0.18084474, ..., 0.10529856,
-0.01682918, 0.09111448],
[-0.04859972, 0.00864089, 0.12390362, ..., 0.17152672,
-0.00713953, 0.06918244],
[ 0.07703741, 0.08441998, 0.07430147, ..., 0.08184789,
-0.17301415, -0.11319483]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)][array([[ 0.0367789 , 0.00915425, -0.02733853, ..., -0.02040792,
-0.03245208, 0.05279592],
[-0.07986325, -0.0093028 , 0.04690679, ..., -0.03594837,
-0.03365551, 0.04181867],
[-0.01529652, -0.04739384, -0.04961624, ..., 0.03608193,
-0.02728439, 0.03388698],
...,
[ 0.06456115, -0.06791718, 0.02804885, ..., -0.02433868,
-0.06182578, -0.01848171],
[ 0.02070352, -0.03081129, -0.06013838, ..., 0.00220076,
-0.05257946, 0.04429463],
[-0.00666717, -0.05574629, -0.03431721, ..., 0.07651306,
0.02397371, -0.06563253]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32), array([[-0.03700976, 0.03013754, -0.10353263, ..., -0.02945483,
0.0997458 , -0.00535272],
[-0.09297995, -0.00978217, -0.15470384, ..., 0.18909012,
-0.02411154, 0.03662926],
[-0.14865722, 0.13019712, -0.16894627, ..., 0.02009523,
0.18213274, -0.0228352 ],
...,
[-0.01553613, 0.09343223, 0.08486612, ..., -0.05365789,
0.01778294, -0.16807753],
[-0.18208605, 0.04372226, 0.00357029, ..., -0.19741432,
-0.05363443, 0.02788939],
[ 0.08774336, -0.01484367, 0.20057438, ..., -0.14653617,
-0.01546355, 0.05677335]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)][array([[-0.06932048, 0.04931927, 0.02986243, ..., -0.00124229,
-0.04131682, 0.04874287],
[ 0.02503149, -0.01789933, 0.01456298, ..., -0.07483141,
-0.00834411, 0.06528252],
[-0.07246303, -0.05168567, -0.07982197, ..., 0.03553585,
-0.07355539, 0.0455386 ],
...,
[-0.03427464, -0.05049596, 0.04526667, ..., 0.0540349 ,
-0.07729132, 0.02335045],
[ 0.00899633, 0.02592985, -0.06459068, ..., -0.06000284,
-0.06346118, 0.00611115],
[ 0.05585308, -0.00852666, -0.01165473, ..., -0.07250661,
-0.07178727, 0.04963235]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32), array([[ 0.1062278 , 0.07988457, -0.20682454, ..., 0.0976506 ,
-0.0116874 , -0.06627488],
[ 0.02052386, -0.20188682, -0.15016697, ..., 0.15503861,
0.04030807, 0.17274798],
[-0.0675576 , 0.09332336, -0.1745064 , ..., 0.07768513,
-0.04787958, 0.06289487],
...,
[-0.20753261, 0.06955643, -0.19981481, ..., -0.01403984,
0.04701854, -0.20236667],
[ 0.11430956, 0.02020629, 0.03855045, ..., -0.05780427,
0.0012497 , -0.12894002],
[ 0.1534607 , -0.18565604, 0.13524099, ..., -0.184562 ,
-0.06643088, 0.08209728]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)]
but after convert it to a String it looks like this
List after convert it to String, weightsStr = ''.join(str(e) for e in weights)
:
[[ 0.05541647 -0.00467741 0.06709623 ... -0.06240537 -0.05044469
-0.06255569]
[ 0.05793238 -0.04376897 -0.03331734 ... 0.04109375 -0.05561347
-0.05630576]
[ 0.03568218 0.00916858 0.02733664 ... 0.04085189 0.07445424
0.05173937]
...
[ 0.00326935 0.05949181 -0.02493389 ... 0.01619817 0.02883349
-0.00364999]
[ 0.05162556 -0.07704586 -0.00726594 ... 0.03567791 0.06234651
0.05147751]
[-0.04587721 0.06365172 -0.06174358 ... -0.07004303 -0.00196535
-0.05049317]][0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0.][[ 0.11118214 -0.00349338 0.18680657 ... 0.01847704 -0.03098661
-0.04094526]
[-0.06314829 0.00289522 -0.11807185 ... -0.10976926 -0.12070866
0.19067971]
[-0.05408052 -0.02283411 0.16553403 ... -0.12856016 0.00681128
-0.05486405]
...
[-0.12182648 -0.03314751 0.04840027 ... 0.13398318 -0.092302
0.13001741]
[ 0.01030177 0.14168383 -0.18688273 ... -0.17727108 -0.1098071
-0.12000293]
[ 0.03310342 0.17201088 -0.08573408 ... 0.15494372 -0.16848558
0.12254588]][0. 0. 0. 0. 0. 0. 0. 0. 0. 0.][[-0.07572845 0.07035964 -0.00726507 ... -0.01283053 -0.02842413
0.02551443]
[-0.00741241 -0.02386538 0.00442091 ... 0.0693512 0.02695736
-0.07246653]
[ 0.06941632 -0.01986459 0.02596217 ... 0.04713184 0.03926247
0.07958693]
...
[ 0.04515444 -0.02030407 -0.00393321 ... 0.025347 -0.01182116
0.04929114]
[-0.06743087 0.02246762 0.0225632 ... 0.03987813 -0.00048529
0.00320805]
[ 0.07628443 -0.06414777 0.04115602 ... -0.03207976 -0.01118261
0.00946496]][0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0.][[ 0.19875868 -0.0724885 -0.15991165 ... -0.04141769 0.11540116
0.1246707 ]
[ 0.03422281 0.09608312 0.18289839 ... 0.20248671 -0.05454096
-0.11580068]
[ 0.12459688 0.17984338 0.02630243 ... -0.20585045 -0.08128738
0.08814187]
...
[ 0.07335795 -0.02979451 0.18084474 ... 0.10529856 -0.01682918
0.09111448]
[-0.04859972 0.00864089 0.12390362 ... 0.17152672 -0.00713953
0.06918244]
[ 0.07703741 0.08441998 0.07430147 ... 0.08184789 -0.17301415
-0.11319483]][0. 0. 0. 0. 0. 0. 0. 0. 0. 0.][[ 0.0367789 0.00915425 -0.02733853 ... -0.02040792 -0.03245208
0.05279592]
[-0.07986325 -0.0093028 0.04690679 ... -0.03594837 -0.03365551
0.04181867]
[-0.01529652 -0.04739384 -0.04961624 ... 0.03608193 -0.02728439
0.03388698]
...
[ 0.06456115 -0.06791718 0.02804885 ... -0.02433868 -0.06182578
-0.01848171]
[ 0.02070352 -0.03081129 -0.06013838 ... 0.00220076 -0.05257946
0.04429463]
[-0.00666717 -0.05574629 -0.03431721 ... 0.07651306 0.02397371
-0.06563253]][0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0.][[-0.03700976 0.03013754 -0.10353263 ... -0.02945483 0.0997458
-0.00535272]
[-0.09297995 -0.00978217 -0.15470384 ... 0.18909012 -0.02411154
0.03662926]
[-0.14865722 0.13019712 -0.16894627 ... 0.02009523 0.18213274
-0.0228352 ]
...
[-0.01553613 0.09343223 0.08486612 ... -0.05365789 0.01778294
-0.16807753]
[-0.18208605 0.04372226 0.00357029 ... -0.19741432 -0.05363443
0.02788939]
[ 0.08774336 -0.01484367 0.20057438 ... -0.14653617 -0.01546355
0.05677335]][0. 0. 0. 0. 0. 0. 0. 0. 0. 0.][[-0.06932048 0.04931927 0.02986243 ... -0.00124229 -0.04131682
0.04874287]
[ 0.02503149 -0.01789933 0.01456298 ... -0.07483141 -0.00834411
0.06528252]
[-0.07246303 -0.05168567 -0.07982197 ... 0.03553585 -0.07355539
0.0455386 ]
...
[-0.03427464 -0.05049596 0.04526667 ... 0.0540349 -0.07729132
0.02335045]
[ 0.00899633 0.02592985 -0.06459068 ... -0.06000284 -0.06346118
0.00611115]
[ 0.05585308 -0.00852666 -0.01165473 ... -0.07250661 -0.07178727
0.04963235]][0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0.][[ 0.1062278 0.07988457 -0.20682454 ... 0.0976506 -0.0116874
-0.06627488]
[ 0.02052386 -0.20188682 -0.15016697 ... 0.15503861 0.04030807
0.17274798]
[-0.0675576 0.09332336 -0.1745064 ... 0.07768513 -0.04787958
0.06289487]
...
[-0.20753261 0.06955643 -0.19981481 ... -0.01403984 0.04701854
-0.20236667]
[ 0.11430956 0.02020629 0.03855045 ... -0.05780427 0.0012497
-0.12894002]
[ 0.1534607 -0.18565604 0.13524099 ... -0.184562 -0.06643088
0.08209728]][0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
I'm not really sure how to return it to his original form.
Thank you!!

As far as I see, you have numpy arrays there. In this case, I think, the best approach would be to use numpy load and save methods.
https://numpy.org/doc/stable/reference/generated/numpy.save.html
https://numpy.org/doc/stable/reference/generated/numpy.load.html
Or if you need a human readable string representation array2string and fromstring
https://numpy.org/doc/stable/reference/generated/numpy.array2string.html
https://numpy.org/doc/stable/reference/generated/numpy.fromstring.html

First check with the dimension of the array that it is 2d,3d, or multi-dimension and then according to that create properly for loops to iterate through all the item of array list and use manual type casting to convert all values to string.
Example for 1D array to convert int & float to string
arr = [1,2,5,6,4,8,1.55,6.33,2.22,3.03,-0.222,-52222]
print(arr)
for i in range(len(arr)):
arr[i] = str(arr[i])
print(arr)
Output :
[1,2,5,6,4,8,1.55,6.33,2.22,3.03,-0.222,-52222]
['1', '2', '5', '6', '4', '8', '1.55', '6.33', '2.22', '3.03', '-0.222', '-52222']

Accessing matrix in Python by an array of indices [duplicate]

This question already has answers here:
Numpy extract submatrix
(6 answers)
How to create a sub-matrix in numpy
(2 answers)
Closed 2 years ago.
I'm relatively new to Python and would appreciate your help!
Suppose I have two square matrices - one large M and one smaller K - and an integer array of indices ind, not necessarily sorted. The length of ind matches the dimensions of K. In Octave/MATLAB, I can easily do this:
M(ind, ind) = K
This will distribute all the components of K to those positions of M that correspond to indices ind. This is often used in Finite Element computations.
Is there a way to do the same thing just as elegantly in Python? You may assume my M and K are NumPy arrays that were constructed via the operations:
M = np.zeros((12, 12))
K = np.zeros((6, 6))
I did some work on these matrices and filled them with data. My ind array is a NumPy array as well.
However, when I do something like
M[ind, ind] = K
I get shape mismatch as an error. Plugging ind.tolist() instead of ind into M won't change anything.
Thanks for any advice!

You are looking for this:
M[np.ix_(ind,ind)] = K
example:
M = np.zeros((12, 12))
K = np.ones((6, 6))
ind = np.arange(6)
output:
[[1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0.]
[1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0.]
[1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0.]
[1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0.]
[1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0.]
[1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]

If you are using numpy arrays, It can be donde directly using this syntax:
import numpy as np
M = np.zeros((12, 12))
K = np.ones((6, 6))
M[:6, :6] = K
# This is equivalent to:
# M[0:6, 0:6] = K
M will then look like this:
array([[1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0.],
[1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0.],
[1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0.],
[1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0.],
[1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0.],
[1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])
Here you have more information about slice indexing in python
https://www.pythoncentral.io/how-to-slice-listsarrays-and-tuples-in-python/

How does the scipy distance_transform_edt function work?

https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.ndimage.morphology.distance_transform_edt.html
I'm having trouble understanding how the Euclidean distance transform function works in Scipy. From what I understand, it is different than the Matlab function (bwdist). As an example, for the input:
[[ 0. 0. 0. 0. 0.]
[ 0. 1. 0. 0. 0.]
[ 0. 0. 0. 0. 0.]
[ 0. 0. 0. 1. 0.]
[ 0. 0. 0. 0. 0.]]
The scipy.ndimage.distance_transform_edt function returns the same array:
[[ 0. 0. 0. 0. 0.]
[ 0. 1. 0. 0. 0.]
[ 0. 0. 0. 0. 0.]
[ 0. 0. 0. 1. 0.]
[ 0. 0. 0. 0. 0.]]
But the matlab function returns this:
1.4142 1.0000 1.4142 2.2361 3.1623
1.0000 0 1.0000 2.0000 2.2361
1.4142 1.0000 1.4142 1.0000 1.4142
2.2361 2.0000 1.0000 0 1.0000
3.1623 2.2361 1.4142 1.0000 1.4142
which makes more sense, as it is returning the "distance" to the nearest one.

It is not clear from the docstring, but distance_transform_edt computes the distance from non-zero (i.e. non-background) points to the nearest zero (i.e. background) point.
For example:
In [42]: x
Out[42]:
array([[0, 0, 0, 0, 0, 1, 1, 1],
[0, 1, 1, 1, 0, 1, 1, 1],
[0, 1, 1, 1, 0, 1, 1, 1],
[0, 0, 1, 1, 0, 0, 0, 1]])
In [43]: np.set_printoptions(precision=3) # Easier to read the result with fewer digits.
In [44]: distance_transform_edt(x)
Out[44]:
array([[ 0. , 0. , 0. , 0. , 0. , 1. , 2. , 3. ],
[ 0. , 1. , 1. , 1. , 0. , 1. , 2. , 2.236],
[ 0. , 1. , 1.414, 1. , 0. , 1. , 1. , 1.414],
[ 0. , 0. , 1. , 1. , 0. , 0. , 0. , 1. ]])
You can get the equivalent of Matlab's bwdist(a) by applying distance_transform_edt() to np.logical_not(a) (i.e. invert the foreground and background):
In [71]: a
Out[71]:
array([[ 0., 0., 0., 0., 0.],
[ 0., 1., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 1., 0.],
[ 0., 0., 0., 0., 0.]])
In [72]: distance_transform_edt(np.logical_not(a))
Out[72]:
array([[ 1.414, 1. , 1.414, 2.236, 3.162],
[ 1. , 0. , 1. , 2. , 2.236],
[ 1.414, 1. , 1.414, 1. , 1.414],
[ 2.236, 2. , 1. , 0. , 1. ],
[ 3.162, 2.236, 1.414, 1. , 1.414]])

Warren has already explained how distance_transform_edt works.
In your case,you could change sampling units along x and y
ndimage.distance_transform_edt(a)
array([[ 0., 0., 0., 0., 0.],
[ 0., 1., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 1., 0.],
[ 0., 0., 0., 0., 0.]])
But
>>> ndimage.distance_transform_edt(a, sampling=[2,2])
array([[ 0., 0., 0., 0., 0.],
[ 0., 2., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 2., 0.],
[ 0., 0., 0., 0., 0.]])
Or
ndimage.distance_transform_edt(a, sampling=[3,3])
array([[ 0., 0., 0., 0., 0.],
[ 0., 3., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 3., 0.],
[ 0., 0., 0., 0., 0.]])

Python indirect indexing

This might be a super easy question if you know how to do it, but I just can't figure out the syntax:
I have an array of 5x10 zeros: y1 = np.zeros((5,10)) and an array 5x1 of index: index=np.array([2,3,2,5,6]). For each row of y1, I would like to set 1 at the column given by the index. The result would look like
array([[ 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 1., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.]])
Anyone can help please :-) ?

You can do multi-dimensional array indexing with array[index_1, index_2]. For your problem:
y1[range(y1.shape[0]), index] = 1
range(y1.shape[0]) generates the array [0,1,...,n-1], where n is the number of rows in y1. This array is your row index, and index is your column index.

Just use an enumerate()
import numpy as np
y1 = np.zeros((5,10))
index=np.array([2,3,2,5,6])
for i,item in enumerate(y1):
item[index[i]] = 1
print(y1)
# [[ 0. 0. 1. 0. 0. 0. 0. 0. 0. 0.]
# [ 0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]
# [ 0. 0. 1. 0. 0. 0. 0. 0. 0. 0.]
# [ 0. 0. 0. 0. 0. 1. 0. 0. 0. 0.]
# [ 0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]]
Does this do what you want?

Categories