Why are there so many zeros in my wav reading? - python
I'm trying to make a speech recognition algorithm. I've a wav file containing +-20 minutes of speech. I've read it into a numpy array, each chunck of 1024 values is a row. As somehow not all chunks, provided by the wave module's file's readframes method, are of the same length, some rows are padded with zeros using the numpy.padd function in order to make the array have a homogenous shape. These paddings are only appended behind the array, and thus cannot cause the following.
I've noticed that there are columns in the array that are only containing zeros. These columns are always occuring in pairs and are always seperated by two columns containing normal values. The sixth column seems to be an exception on this pattern: It does also contain ones sometimes. Are these columns real recorded speach or are they put in between by my computer for some reason? This is important to know, as I don't wan't my algorithm to train on computer generated values. Should I delete those values or is it better to keep them? The array does not have to be playable as audio anymore.
Here's a sample out of the array. I can't attach the complete array, as it is way to big for that. A more complete version can be found here.
[78, 1, 0, 0, 79, 1, 0, 0, 12, 1, 0, 0, 185, 0, 0, 0, 177, 0, 0, 0, 28, 1, 0, 0, 245, 1, 0, 0, 38, 3, 0, 0, 106, 4, 0, 0, 81, 5, 0, 0, 148, 5, 0, 0, 74, 5, 0, 0, 168, 4, 0, 0, 229, 3, 0, 0, 83, 3, 0, 0, 31, 3, 0, 0, 26, 3, 0, 0, 33, 3, 0, 0, 40, 3, 0, 0, 22, 3, 0, 0, 246, 2, 0, 0, 211, 2, 0, 0, 136, 2, 0, 0, 240, 1, 0, 0, 247, 0, 0, 0, 176, 255, 0, 0, 97, 254, 0, 0, 69, 253, 0, 0, 131, 252, 0, 0, 54, 252, 0, 0, 81, 252, 0, 0, 188, 252, 0, 0, 79, 253, 0, 0, 207, 253, 0, 0, 48, 254, 0, 0, 97, 254, 0, 0, 73, 254, 0, 0, 6, 254, 0, 0, 175, 253, 0, 0, 90, 253, 0, 0, 58, 253, 0, 0, 73, 253, 0, 0, 101, 253, 0, 0, 132, 253, 0, 0, 147, 253, 0, 0, 164, 253, 0, 0, 199, 253, 0, 0, 224, 253, 0, 0, 8, 254, 0, 0, 97, 254, 0, 0, 228, 254, 0, 0, 163, 255, 0, 0, 138, 0, 0, 0, 89, 1, 0, 0, 251, 1, 0, 0, 45, 2, 0, 0, 157, 1, 0, 0, 95, 0, 0, 0, 161, 254, 0, 0, 185, 252, 0, 0, 56, 251, 0, 0, 134, 250, 0, 0, 226, 250, 0, 0, 51, 252, 0, 0, 246, 253, 0, 0, 175, 255, 0, 0, 208, 0, 0, 0, 240, 0, 0, 0, 96, 0, 0, 0, 124, 255, 0, 0, 119, 254, 0, 0, 223, 253, 0, 0, 243, 253, 0, 0, 120, 254, 0, 0, 77, 255, 0, 0, 254, 255, 0, 0, 253, 255, 0, 0, 63, 255, 0, 0, 224, 253, 0, 0, 61, 252, 0, 0, 27, 251, 0, 0, 35, 251, 0, 0, 180, 252, 0, 0, 154, 255, 0, 0, 1, 3, 0, 0, 6, 6, 0, 0, 189, 7, 0, 0, 116, 7, 0, 0, 111, 5, 0, 0, 118, 2, 0, 0, 116, 255, 0, 0, 133, 253, 0, 0, 55, 253, 0, 0, 124, 254, 0, 0, 17, 1, 0, 0, 19, 4, 0, 0, 87, 6, 0, 0, 42, 7, 0, 0, 53, 6, 0, 0, 195, 3, 0, 0]
[240, 0, 0, 0, 235, 254, 0, 0, 155, 254, 0, 0, 87, 0, 0, 0, 80, 3, 0, 0, 25, 6, 0, 0, 137, 7, 0, 0, 253, 6, 0, 0, 146, 4, 0, 0, 57, 1, 0, 0, 35, 254, 0, 0, 73, 252, 0, 0, 56, 252, 0, 0, 185, 253, 0, 0, 251, 255, 0, 0, 60, 2, 0, 0, 204, 3, 0, 0, 18, 4, 0, 0, 15, 3, 0, 0, 21, 1, 0, 0, 137, 254, 0, 0, 95, 252, 0, 0, 99, 251, 0, 0, 137, 251, 0, 0, 210, 252, 0, 0, 46, 255, 0, 0, 161, 1, 0, 0, 40, 3, 0, 0, 102, 3, 0, 0, 95, 2, 0, 0, 127, 0, 0, 0, 92, 254, 0, 0, 160, 252, 0, 0, 18, 252, 0, 0, 216, 252, 0, 0, 133, 254, 0, 0, 181, 0, 0, 0, 136, 2, 0, 0, 248, 2, 0, 0, 213, 1, 0, 0, 130, 255, 0, 0, 202, 252, 0, 0, 199, 250, 0, 0, 241, 249, 0, 0, 56, 250, 0, 0, 121, 251, 0, 0, 15, 253, 0, 0, 72, 254, 0, 0, 235, 254, 0, 0, 188, 254, 0, 0, 180, 253, 0, 0, 61, 252, 0, 0, 191, 250, 0, 0, 167, 249, 0, 0, 108, 249, 0, 0, 33, 250, 0, 0, 144, 251, 0, 0, 106, 253, 0, 0, 20, 255, 0, 0, 227, 255, 0, 0, 169, 255, 0, 0, 173, 254, 0, 0, 82, 253, 0, 0, 3, 252, 0, 0, 69, 251, 0, 0, 115, 251, 0, 0, 146, 252, 0, 0, 109, 254, 0, 0, 137, 0, 0, 0, 64, 2, 0, 0, 28, 3, 0, 0, 241, 2, 0, 0, 233, 1, 0, 0, 148, 0, 0, 0, 143, 255, 0, 0, 60, 255, 0, 0, 183, 255, 0, 0, 185, 0, 0, 0, 203, 1, 0, 0, 149, 2, 0, 0, 208, 2, 0, 0, 97, 2, 0, 0, 127, 1, 0, 0, 103, 0, 0, 0, 66, 255, 0, 0, 74, 254, 0, 0, 172, 253, 0, 0, 121, 253, 0, 0, 158, 253, 0, 0, 218, 253, 0, 0, 242, 253, 0, 0, 207, 253, 0, 0, 97, 253, 0, 0, 191, 252, 0, 0, 91, 252, 0, 0, 144, 252, 0, 0, 82, 253, 0, 0, 100, 254, 0, 0, 122, 255, 0, 0, 34, 0, 0, 0]
[1, 0, 0, 0, 65, 255, 0, 0, 89, 254, 0, 0, 151, 253, 0, 0, 47, 253, 0, 0, 69, 253, 0, 0, 221, 253, 0, 0, 237, 254, 0, 0, 76, 0, 0, 0, 166, 1, 0, 0, 187, 2, 0, 0, 133, 3, 0, 0, 3, 4, 0, 0, 86, 4, 0, 0, 179, 4, 0, 0, 47, 5, 0, 0, 198, 5, 0, 0, 96, 6, 0, 0, 197, 6, 0, 0, 210, 6, 0, 0, 145, 6, 0, 0, 37, 6, 0, 0, 217, 5, 0, 0, 226, 5, 0, 0, 73, 6, 0, 0, 35, 7, 0, 0, 86, 8, 0, 0, 105, 9, 0, 0, 11, 10, 0, 0, 252, 9, 0, 0, 222, 8, 0, 0, 224, 6, 0, 0, 151, 4, 0, 0, 73, 2, 0, 0, 111, 0, 0, 0, 152, 255, 0, 0, 125, 255, 0, 0, 130, 255, 0, 0, 105, 255, 0, 0, 214, 254, 0, 0, 135, 253, 0, 0, 223, 251, 0, 0, 108, 250, 0, 0, 144, 249, 0, 0, 117, 249, 0, 0, 255, 249, 0, 0, 244, 250, 0, 0, 239, 251, 0, 0, 101, 252, 0, 0, 38, 252, 0, 0, 99, 251, 0, 0, 105, 250, 0, 0, 195, 249, 0, 0, 239, 249, 0, 0, 4, 251, 0, 0, 216, 252, 0, 0, 218, 254, 0, 0, 61, 0, 0, 0, 161, 0, 0, 0, 19, 0, 0, 0, 227, 254, 0, 0, 190, 253, 0, 0, 69, 253, 0, 0, 179, 253, 0, 0, 209, 254, 0, 0, 21, 0, 0, 0, 250, 0, 0, 0, 35, 1, 0, 0, 76, 0, 0, 0, 162, 254, 0, 0, 204, 252, 0, 0, 104, 251, 0, 0, 239, 250, 0, 0, 177, 251, 0, 0, 127, 253, 0, 0, 162, 255, 0, 0, 42, 1, 0, 0, 95, 1, 0, 0, 246, 255, 0, 0, 23, 253, 0, 0, 154, 249, 0, 0, 220, 246, 0, 0, 235, 245, 0, 0, 32, 247, 0, 0, 33, 250, 0, 0, 214, 253, 0, 0, 203, 0, 0, 0, 227, 1, 0, 0, 234, 0, 0, 0, 137, 254, 0, 0, 202, 251, 0, 0, 251, 249, 0, 0, 42, 250, 0, 0, 77, 252, 0, 0, 154, 255, 0, 0, 24, 3, 0, 0, 162, 5, 0, 0, 107, 6, 0, 0, 139, 5, 0, 0, 154, 3, 0, 0]
It appeared that the file contained two different sound channels. The second sound channel was not recorded, as I have only one microphone. This caused this channel to be defaulted to all zeros. Because .wav files use two bits to record each channel, this resulted in two places being filled with zeros. Then the wave module combined all the different channels into one chunck, each containing 2 correct bits and 2 incorrect (0) bits and combined 100 of these frames, my chunck size, into one array and thus deliver the array given.
I could perfectly remove the zeros out of the array without any audio quality loss. However, I also had to reduce the amount of channels to 1, as I just deleted the second channel.
Related
I am facing ValueError: Shapes (1, 14) and (1, 139, 14) are incompatible
import keras
from keras_self_attention import SeqSelfAttention
inputs = keras.layers.Input(shape=(None,))
embd = keras.layers.Embedding(vocab_size,
300, weights=[embedding_matrix],
trainable=False, mask_zero=True, name='Encoder-Word-Embedding')(inputs)
lstm = keras.layers.Bidirectional(keras.layers.LSTM(units=150,
return_sequences=True))(embd)
att = SeqSelfAttention(attention_type=SeqSelfAttention.ATTENTION_TYPE_MUL,
kernel_regularizer=keras.regularizers.l2(1e-4),
bias_regularizer=keras.regularizers.l1(1e-4),
attention_regularizer_weight=1e-4,
name='Attention')(lstm)
dense = keras.layers.Dense(units=max_seq_len_y, name='Dense')(att)
model = keras.models.Model(inputs=inputs, outputs=[dense])
model.compile(
optimizer='adam',
loss={'Dense': 'sparse_categorical_crossentropy'},
metrics={'Dense': 'categorical_accuracy'},
)
model.summary()
history = model.fit(x=X, y=y, batch_size=1, epochs=32)
X.shape, y.shape
=> ((52, 139), (52, 14))
X[0]
array([293, 40, 294, 129, 75, 130, 129, 131, 295, 296, 132,
297, 298,
2, 299, 34, 12, 76, 300, 27, 301, 15, 1, 302, 133, 4,
77, 303, 3, 134, 304, 78, 34, 305, 11, 306, 307, 4, 1,
132, 135, 22, 10, 308, 11, 136, 4, 1, 309, 50, 4, 310,
11, 78, 311, 312, 3, 77, 1, 313, 130, 10, 137, 11, 12,
109, 7, 314, 315, 7, 1, 76, 316, 4, 317, 318, 34, 138,
319, 139, 320, 3, 77, 321, 79, 322, 4, 1, 323, 324, 4,
1, 325, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0])
y[0]
array([1040, 1041, 2, 1042, 0, 0, 0, 0, 0,
0, 0,
0, 0, 0])
Here X,y represent the sequences converted using keras tokenizer library. My aim is to learn title of the paragraph.
Try to add activation='softmax' in your last Dense layer.
How do you apply a maximum filter that ignores ties?
I would like to transform an image in order to identify local maxima. The desired output is an array that has the same shape as the input image, but only has binary values. "1" indicates a local maximum in a local 3x3 area, and 0 indicates that the pixel is not a local maximum. The approach I have taken is to use scipy.ndimage.filters.maximum_filter. This technique works, however in the case that there are two pixels that are maximum in the 3x3 area, both are given the value 1. import numpy as np import scipy.ndimage.filters as filters #here i have the top left corner of the famous image "Lena" image = np.array( [[142, 142, 141, 140, 142, 136, 143, 140, 144], [142, 142, 141, 140, 142, 136, 143, 140, 144], [142, 141, 141, 140, 142, 136, 143, 140, 144], [142, 142, 141, 140, 142, 136, 143, 140, 144], [142, 141, 141, 140, 142, 136, 143, 140, 144], [144, 144, 137, 135, 140, 140, 138, 140, 140], [140, 140, 143, 138, 139, 143, 139, 137, 140], [138, 138, 135, 136, 138, 139, 136, 138, 138], [135, 136, 138, 137, 139, 139, 138, 137, 143]]) neighborhood = np.ones((3, 3), dtype = bool) filtered = (filters.maximum_filter(image, footprint=neighborhood) == image)* 1 """ Here is what the result looks like: """ filtered = np.array( [[1, 1, 0, 0, 1, 0, 1, 0, 1], [1, 1, 0, 0, 1, 0, 1, 0, 1], [1, 0, 0, 0, 1, 0, 1, 0, 1], [1, 1, 0, 0, 1, 0, 1, 0, 1], [0, 0, 0, 0, 1, 0, 1, 0, 1], [1, 1, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 1, 0, 0, 1], [0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 1, 0, 1, 1, 0, 0, 1]]) If you look at the 'filtered' variable in the code snippet, observe that this filter picked up 5 local maxima in the top 3x3 corner. However if you look at the original image, there are a lot of 'ties' - that is, that the value 142 was not higher than the other pixels, but equal to the second highest. I am also aware that the way that you handle border pixels is important, but my question still remains. What should I do so that ties are awarded '0' and not '1'? For example pixel [1, 1] I would like to be 0.
Maximum filters are non-linear filters. You cannot use convolution to implement non-linear filters. Please read https://en.wikipedia.org/wiki/Convolution The maximum of (144,144,144) is 144. That's because of the mathematical definition of "maximum". If you don't like that you should not use a maximum filter. Please read https://en.wikipedia.org/wiki/Maxima_and_minima I suggest you implement your own filter that deals with that requirement.
I solved my issue by applying a convolution with a 3x3 kernel to the maxima that I identified. Using this kernel creates an array where all the maxima that are 'tied' have a value greater than zero. Thereafter I just removed them. import scipy.ndimage as nd kernel = np.ones((3, 3)) new_maxima = nd.convolve(filtered, kernel) new_maxima = np.where(new_maxima > 1, 0, new_maxima) new_maxima = new_maxima * filtered result = np.array( [[0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 1, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0]]) You can see that all the 'tied' maxima have been removed.
How to write a classification algorithm in tensorflow using keras in python?
I have a training set of 2 images which has 64 features and a label attached to them i.e. matched/not matched. How can I feed this data in a neural network using keras? My data is as follows: [ [ [ 239, 1, 255, 255, 255, 255, 2, 0, 130, 3, 1, 101, 22, 154, 0, 240, 30, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 71, 150, 212 ], [ 239, 1, 255, 255, 255, 255, 2, 0, 130, 3, 1, 101, 22, 154, 0, 240, 30, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 71, 150, 212 ], "true" ], [ [ 239, 1, 255, 255, 255, 255, 2, 0, 130, 3, 1, 81, 28, 138, 0, 241, 254, 128, 6, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 0, 128, 2, 128, 2, 192, 6, 224, 6, 224, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 62 ], [ 239, 1, 255, 255, 255, 255, 2, 0, 130, 3, 1, 81, 28, 138, 0, 241, 254, 128, 6, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 0, 128, 2, 128, 2, 192, 6, 224, 6, 224, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 62 ], "true" ], .... ] I want to train neural network so that after training if I provide it 2 array of 64 features then it should able to tell whether they matched or not?
Since you kind of extracted the futures already, I'd suggest by just going with some dense layers and convert the "true" and "false" to a 1 and 0 respectively, and just use a sigmoid on the final dense layer.
Try to experiment with something simple first, see how it goes and continue from there on, need more help, just ask
EDIT
def generator(batch_size=10, nr_features=126):
feed_data = np.zeros((batch_size, nr_features))
labels = np.zeros(batch_size)
i = 0
for entry in data:
if entry.pop(-1) == "true":
labels[i] = 1
else:
labels[i] = 0
feed_data[i, :] = np.array(entry).flatten()
i += 1
if not (i % batch_size):
i = 0
yield feed_data, labels
model = keras.Sequential()
model.add(keras.layers.Dense(126, input_dim=126))
model.add(keras.layers.Dense(20))
model.add(keras.layers.Dense(1))
model.add(keras.layers.Activation('sigmoid'))
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
for d, l in generator():
model.train_on_batch(d, l)
So what happens,
the data in the generator is your full data, I pop out the true/false, convert it to 1/0 and put it into the label array, I concat all features as a feature vecotr of 126. So feed_data.shape = (10, 126) and labels.shape = (10).
I feed that to a simple fully connected network, one that ends up with a sigmoid. Sigmoid is useful for probablilty, so in this case the output will be the probability that a feature vecotr is true. and I just feed the data.
Simple example, is not the full code but should get you started, I tested it, and it runs for me, I did not train anything yet though, that's something for you, good luck!
Oh, and questions, ask away
Python: Generate Private Key from an Array?
I am new to python as well as Cryptography. I want to generate the PrivateKey from a variable. After looking for so many options I only get to know that I can generate the key pair from Random function only. Here is my variable a = [[[3, 1, 85, 33, 0, 0, 255, 254, 255, 254, 255, 254, 255, 254, 255, 6, 248, 0, 240, 0, 224, 0, 192, 0, 192, 0, 128, 0, 128, 0, 128, 0, 128, 0, 128, 0, 128, 0, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 108, 181, 11, 254, 92, 186, 205, 94, 27, 192, 36, 126, 26, 164, 13, 159, 114, 59, 162, 95, 50, 189, 78, 95, 99, 24, 140, 124, 72, 165, 204, 156, 114, 40, 99, 28, 57, 25, 141, 122, 107, 39, 226, 122, 24, 169, 35, 186, 93, 39, 162, 242, 24, 185, 229, 120, 38, 169, 205, 57, 31, 184, 141, 153, 78, 152, 12, 86, 92, 152, 13, 22, 77, 155, 226, 182, 66, 163, 163, 16, 89, 157, 227, 180, 67, 160, 204, 142, 101, 165, 226, 244, 45, 167, 99, 20, 117, 157, 76, 44, 38, 181, 225, 44, 107, 32, 75, 179, 110, 32, 163, 16, 38, 165, 101, 48, 32, 43, 14, 76, 38, 50, 204, 205, 50, 163, 227, 172, 38, 46, 98, 108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [3, 1, 101, 34, 0, 0, 255, 254, 255, 14, 248, 2, 240, 2, 224, 0, 192, 0, 192, 0, 128, 0, 128, 0, 128, 0, 128, 0, 192, 0, 192, 0, 224, 0, 224, 0, 240, 0, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 36, 17, 229, 30, 78, 19, 140, 30, 99, 21, 226, 190, 113, 22, 97, 190, 30, 152, 36, 94, 115, 162, 75, 222, 21, 35, 228, 126, 119, 168, 97, 222, 99, 40, 140, 222, 32, 52, 14, 222, 108, 53, 204, 94, 67, 55, 78, 222, 76, 193, 100, 62, 57, 172, 142, 31, 92, 181, 77, 191, 47, 55, 101, 31, 119, 184, 75, 63, 75, 11, 22, 92, 108, 15, 227, 156, 44, 157, 34, 28, 25, 158, 13, 220, 32, 33, 228, 92, 35, 165, 141, 220, 44, 152, 140, 189, 100, 142, 139, 218, 35, 156, 205, 154, 32, 169, 228, 90, 95, 11, 140, 18, 94, 141, 226, 146, 40, 170, 14, 82, 38, 172, 100, 210, 51, 149, 33, 208, 70, 13, 162, 215, 44, 147, 228, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]] So my questions are, Is it possible to generate the PrivateKey from a sequence/data If so, any suggestions that how can I proceed with that And last but not the least, which algorithm should I use (RSA/ECC)
No, it's not possible to generate a private key from just a sequence/data.
Keys are generated using the random number generator(RNG) or pseudo-random number generator(PRNG) that uses a system entropy seed, making it even more harder for an attacker to guess.
Take the case of RSA, as you mentioned you can either create a private key from a random function or have more control over the private key generation by setting up the variables manually. Here you can find the way to generate a private key with some variable that you can manually decide using the 'Crypto.PublicKey.RSA.construct()' function of the 'PyCryptodome' package. For this, you should have a good understanding of RSA.
It is possible to generate a key from the user input in the symmetric cryptography. 'Scrypt' and 'Argon2' are two packages that help you achieve it. The input has to be converted to binary first. The input can be a string or numbers. Here is a simple example using the 'Scrypt' package. You can find more details here.
import scrypt, secrets
password = b'not a number'
salt = secrets.token_bytes(32)
scrypt_key = scrypt.hash(password, salt, N=16384, r=8, p=1, 32)
print('Salt: ', salt)
print('Key: ', scrypt_key)
Output:
Salt: b'\xdbS\x1e\xa2\x81e\xd3\x948p\xc3lmk\xd6\x8b\xb94\x1c\xd5A/\xa5gZ\xb1\xc2\x15\x99\x9d\xc8\xb8'
Key: b'J\xc1\xc1"\xfd\x05\xfb\x14J\x96\xea\xe3\x1d\xa6\xbb\x01\xf7sj\x87\xf9\x18%\x00YK\x1f\xe8\xc8\x8d\xff%'
The algorithm depends on your application. Although, nowadays the ECC cryptography is considered more secure than the RSA cryptography.
Dividing integer with numpy.array returns array with zeros
I seem to have some problems dividing a integer with an numpy.array.. Doing this seem to return a array of zeroes, instead of decimals values.. Example: >>> import numpy as np >>> length = 257 >>> length = 256 >>> x = np.array(range(0,length) ... ) >>> x array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255]) >>> x*1/length array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]) >>> Why is it returning zeros?
That's because it is doing an integer division. Try this x*1/256.0
Because the array consists out of integers, and in python-2.x / is defined as integer division which round down the result, so 99/100 is considered to be 0. You can first cast to a float: x.astype(float)*1/length A float is a floating point and such datastructures - putting it bold - support a numbers with decimal dot.