Related
I have a task which is to extract and analyse a payload. the payload is stored in the array below:
byte[] source = new byte[449]
{
252, 72, 131, 228, 240, 232, 204,
0, 0, 0, 65, 81, 65, 80,
82, 81, 72, 49, 210, 86,
101, 72, 139, 82, 96, 72, 139,
82, 24, 72, 139, 82, 32,
72, 139, 114, 80, 77, 49, 201, 72,
15, 183, 74, 74, 72, 49, 192, 172,
60, 97, 124, 2, 44, 32,
65, 193, 201, 13, 65, 1, 193, 226,
237, 82, 72, 139,82, 32, 139, 66,
60, 72, 1, 208, 102, 129, 120, 24, 11,
2,65, 81, 15, 133, 114, 0, 0, 0,
139, 128, 136, 0, 0, 0, 72, 133,
192, 116, 103, 72, 1, 208, 80, 139,
72, 24, 68, 139, 64, 32, 73, 1, 208,
227, 86, 72, 255, 201, 65, 139, 52, 136,
72, 1, 214, 77, 49, 201, 72, 49,
192, 172, 65, 193, 201, 13, 65, 1,
193, 56, 224, 117, 241, 76, 3, 76,
36, 8, 69, 57, 209, 117, 216, 88, 68,
139, 64, 36, 73, 1, 208, 102, 65, 139,
12, 72, 68, 139, 64, 28, 73, 1, 208,
65, 139, 4, 136, 65, 88, 65, 88,
72, 1, 208, 94, 89, 90, 65, 88, 65,
89, 65, 90, 72, 131, 236, 32, 65,
82, 255, 224, 88, 65, 89, 90, 72, 139, 18,
233, 75, 255, 255, 255, 93, 73,
190, 119, 115, 50, 95, 51, 50, 0, 0,
65, 86, 73, 137, 230, 72, 129,
236, 160, 1, 0, 0, 73, 137, 229, 73,
188, 2, 0, 1, 187, 51, 161, 134, 90,
65, 84, 73, 137, 228, 76, 137, 241,
65, 186, 76, 119, 38, 7, 255, 213, 76,
137, 234, 104, 1, 1, 0, 0, 89,
65, 186, 41, 128, 107, 0, 255, 213, 106, 10,
65, 94, 80, 80, 77, 49, 201,77, 49, 192, 72,
255, 192, 72, 137, 194,
72, 255, 192, 72, 137, 193, 65,
186, 234, 15, 223, 224, 255, 213, 72, 137,
199, 106, 16, 65, 88, 76, 137, 226, 72,
137, 249, 65, 186, 153, 165, 116, 97,
255, 213, 133, 192, 116, 12, 73, 255,
206, 117, 229, 104, 240, 181, 162, 86,
255, 213, 72, 131, 236, 16, 72, 137,
226, 77, 49, 201, 106, 4, 65, 88,
72, 137, 249, 65, 186, 2, 217, 200, 95,
255, 213, 72, 131, 196, 32, 94, 137,
246, 106, 64, 65, 89, 104, 0,
16, 0, 0, 65, 88, 72, 137, 242,
72, 49, 201, 65, 186, 88, 164, 83,
229, 255, 213, 72, 137, 195, 73, 137,
199, 77, 49, 201, 73, 137, 240,
72, 137, 218, 72, 137, 249, 65, 186, 2,
217, 200, 95, 255, 213, 72,
1, 195, 72, 41, 198, 72, 133, 246, 117,
225, 65, 255, 231,
};
can anyone help with this? like how to convert it e.g writing a python script or if there are any online converters (I looked but I couldn't find anything).
I need it in actual bytes format so I can analyse it and understand what the payload does exactly.
note - I do not mean the data type byte
thanks
This is an unusual question, as you're already creating a Java bytes object, but if you want a Python bytes object, you can use bytes.
From the documentation:
bytes(iterable_of_ints) -> bytes
bytes(string, encoding[, errors]) -> bytes
bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer
bytes(int) -> bytes object of size given by the parameter initialized with null bytes
bytes() -> empty bytes object
Construct an immutable array of bytes from:
- an iterable yielding integers in range(256)
- a text string encoded using the specified encoding
- any object implementing the buffer API.
- an integer
In this case, we're interested in bytes(iterable_of_ints), which we pass a Python list of your arguments.
source=[
252, 72, 131, 228, 240, 232, 204,
0, 0, 0, 65, 81, 65, 80,
82, 81, 72, 49, 210, 86,
101, 72, 139, 82, 96, 72, 139,
82, 24, 72, 139, 82, 32,
72, 139, 114, 80, 77, 49, 201, 72,
15, 183, 74, 74, 72, 49, 192, 172,
60, 97, 124, 2, 44, 32,
65, 193, 201, 13, 65, 1, 193, 226,
237, 82, 72, 139,82, 32, 139, 66,
60, 72, 1, 208, 102, 129, 120, 24, 11,
2,65, 81, 15, 133, 114, 0, 0, 0,
139, 128, 136, 0, 0, 0, 72, 133,
192, 116, 103, 72, 1, 208, 80, 139,
72, 24, 68, 139, 64, 32, 73, 1, 208,
227, 86, 72, 255, 201, 65, 139, 52, 136,
72, 1, 214, 77, 49, 201, 72, 49,
192, 172, 65, 193, 201, 13, 65, 1,
193, 56, 224, 117, 241, 76, 3, 76,
36, 8, 69, 57, 209, 117, 216, 88, 68,
139, 64, 36, 73, 1, 208, 102, 65, 139,
12, 72, 68, 139, 64, 28, 73, 1, 208,
65, 139, 4, 136, 65, 88, 65, 88,
72, 1, 208, 94, 89, 90, 65, 88, 65,
89, 65, 90, 72, 131, 236, 32, 65,
82, 255, 224, 88, 65, 89, 90, 72, 139, 18,
233, 75, 255, 255, 255, 93, 73,
190, 119, 115, 50, 95, 51, 50, 0, 0,
65, 86, 73, 137, 230, 72, 129,
236, 160, 1, 0, 0, 73, 137, 229, 73,
188, 2, 0, 1, 187, 51, 161, 134, 90,
65, 84, 73, 137, 228, 76, 137, 241,
65, 186, 76, 119, 38, 7, 255, 213, 76,
137, 234, 104, 1, 1, 0, 0, 89,
65, 186, 41, 128, 107, 0, 255, 213, 106, 10,
65, 94, 80, 80, 77, 49, 201,77, 49, 192, 72,
255, 192, 72, 137, 194,
72, 255, 192, 72, 137, 193, 65,
186, 234, 15, 223, 224, 255, 213, 72, 137,
199, 106, 16, 65, 88, 76, 137, 226, 72,
137, 249, 65, 186, 153, 165, 116, 97,
255, 213, 133, 192, 116, 12, 73, 255,
206, 117, 229, 104, 240, 181, 162, 86,
255, 213, 72, 131, 236, 16, 72, 137,
226, 77, 49, 201, 106, 4, 65, 88,
72, 137, 249, 65, 186, 2, 217, 200, 95,
255, 213, 72, 131, 196, 32, 94, 137,
246, 106, 64, 65, 89, 104, 0,
16, 0, 0, 65, 88, 72, 137, 242,
72, 49, 201, 65, 186, 88, 164, 83,
229, 255, 213, 72, 137, 195, 73, 137,
199, 77, 49, 201, 73, 137, 240,
72, 137, 218, 72, 137, 249, 65, 186, 2,
217, 200, 95, 255, 213, 72,
1, 195, 72, 41, 198, 72, 133, 246, 117,
225, 65, 255, 231,
]
my_bytes=bytes(source)
print(my_bytes)
```
Presumably you want to take a number represented by a decimal string and convert in to a sting of 8 0's and 1's (i.e. have leading 0's)
The way to do this is
a='100'
format(int(a),'0=8b')
'01100100'
If you want to apply it to every string in a list you can use map
a=['0','1','2']
list(map(lambda k: format(int(k),'0=8b'),a))
['00000000', '00000001', '00000010']
I have applied contouring to the image and generated the feature vector. So my feture vector contains [Number_of_countours, position of each countours]
For eg, if number of countours are 17 then my feature vector would look like
[17, 74, 185, 97, 194, 118, 176, 144, 181, 146, 166, 156, 176, 39,
154, 77, 172, 129, 133, 168, 142, 58, 132, 82, 140, 37, 127, 55, 141,
117, 105, 151, 112, 58, 99, 86, 108, 24, 99, 53, 117, 154, 98, 195,
120, 61, 72, 78, 82, 122, 64, 155, 78, 84, 55, 105, 65, 172, 53, 178,
71, 115, 53, 131, 64, 186, 35, 208, 90]
For eg, if the number of countor is 14 then feature vector is
[14, 123, 182, 193, 224, 80, 181, 110, 189, 137, 161, 167, 173, 53,
155, 98, 170, 70, 142, 90, 152, 185, 136, 201, 146, 147, 125, 173,
132, 175, 120, 191, 138, 63, 119, 90, 129, 208, 98, 224, 156, 133, 98,
167, 106, 92, 52, 113, 63, 136, 49, 159, 61, 185, 48, 194, 75]
Goal:
I want to find the similarity measure between this vector so that the size of the vector remains same (for Bag of Features)
Can someone help me how to get the similarity between different feature vectors?
I have a nested numpy array (dtype=object), it contains 333 arrays that increase consistently from size 52x1 to size 52x333
I would like to effectively extract and concatenate these arrays so that I have a single 52x55611 array
I imagine this may be straightforward but my attempts using numpy.reshape have been unsuccesful
If you want to stack them along the second axis, you can use numpy.hstack.
list_of_arrays = [ array_1, ..., array_n] #all these arrays have same shape[0]
big_array = np.hstack( list_of_arrays)
if I have understood you correctly, you could use numpy.concatenate.
>>> import numpy as np
>>> a = np.array([range(52)])
>>> b = np.array([range(52,104), range(104, 156)])
>>> np.concatenate((a,b))
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,
26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38,
39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51],
[ 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77,
78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90,
91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103],
[104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129,
130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142,
143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155]])
>>>
.mat files can be loaded into Python with:
import scipy.io
matdata1 = scipy.io.loadmat('file1.mat')
matdata2 = scipy.io.loadmat('file2.mat')
the files can then be piped and save the mat files as text by calling the following Python function:
def mat2txt(matdata):
for k, v in matdata.items(): #Python 3 specific
if isinstance(v,dict):
myprint(v)
else:
print (k,v)
The two .mat files that are being compared are of the same structure and type with different values.
I would like to able to identify the different values in human readable format, and not just their location.
I have tried:
diff matdata1.txt matdata2.txt
diff matdata1.txt matdata2.txt | grep "<" | sed 's/^<//g'
grep -v -F -x -f matdata1.txt matdata2.txt
which do not point to specific differences in values, and they are not within Python. I hoped to store the .mat files as .txt to be able to create a static state to compare the files data at different dates relative to itself and other files, as well as, affording the opportunity to store in git for future comparisons.
A toy example of the resulting data files are:
matdata1.txt
b [[([[(array([[0]], dtype=uint8),)]],)]]
a [[([[(array([[0]], dtype=uint8),)]],)]]
c [[ ([[(array([[ ([[122, 139, 156, 173, 190, 207, 224, 1, 18, 35, 52, 69, 86, 103, 120], [138, 155, 172, 189, 206, 223, 15, 17, 34, 51, 68, 85, 102, 119, 121], [154, 171, 188, 205, 222, 14, 16, 33, 50, 67, 84, 101, 118, 135, 137], [170, 187, 204, 221, 13, 30, 32, 49, 66, 83, 100, 117, 134, 136, 153], [186, 203, 220, 12, 29, 31, 48, 65, 82, 99, 116, 133, 150, 152, 169], [202, 22, 11, 28, 45, 47, 64, 81, 98, 115, 132, 149, 151, 168, 185], [218, 10, 27, 33, 46, 63, 80, 97, 114, 131, 148, 165, 167, 184, 201], [9, 26, 43, 60, 62, 11, 96, 113, 130, 147, 164, 166, 183, 200, 217], [25, 42, 59, 61, 78, 95, 112, 99, 146, 163, 180, 182, 199, 216, 8], [41, 58, 75, 77, 94, 111, 128, 145, 162, 179, 181, 198, 215, 7, 24], [57, 74, 76, 93, 110, 127, 144, 161, 178, 195, 197, 214, 6, 23, 40], [73, 90, 92, 109, 126, 143, 160, 177, 194, 196, 213, 5, 22, 39, 56], [89, 91, 108, 125, 142, 159, 176, 193, 210, 212, 4, 21, 38, 55, 72], [105, 107, 124, 141, 158, 175, 192, 209, 211, 3, 20, 37, 54, 71, 88], [106, 123, 140, 157, 174, 191, 208, 225, 2, 19, 36, 53, 70, 87, 104]],)]],
dtype=[('c', 'O')]),)]],)]]
__header__ b'MATLAB 5.0 MAT-file, Platform: PCWIN64, Created on: Mon Jun 27 20:55:29 2016'
d [[1]]
__globals__ []
f ['string']
__version__ 1.0
e [[2]]
matdata2.txt
e [[2]]
d [[1]]
__globals__ []
f ['string']
__header__ b'MATLAB 5.0 MAT-file, Platform: PCWIN64, Created on: Mon Jun 27 20:54:48 2016'
c [[ ([[(array([[ ([[122, 139, 156, 173, 190, 207, 224, 1, 18, 35, 52, 69, 86, 103, 120], [138, 155, 172, 189, 206, 223, 15, 17, 34, 51, 68, 85, 102, 119, 121], [154, 171, 188, 205, 222, 14, 16, 33, 50, 67, 84, 101, 118, 135, 137], [170, 187, 204, 221, 13, 30, 32, 49, 66, 83, 100, 117, 134, 136, 153], [186, 203, 220, 12, 29, 31, 48, 65, 82, 99, 116, 133, 150, 152, 169], [202, 219, 11, 28, 45, 47, 64, 81, 98, 115, 132, 149, 151, 168, 185], [218, 10, 27, 44, 46, 63, 80, 97, 114, 131, 148, 165, 167, 184, 201], [9, 26, 43, 60, 62, 79, 96, 113, 130, 147, 164, 166, 183, 200, 217], [25, 42, 59, 61, 78, 95, 112, 129, 146, 163, 180, 182, 199, 216, 8], [41, 58, 75, 77, 94, 111, 128, 145, 162, 179, 181, 198, 215, 7, 24], [57, 74, 76, 93, 110, 127, 144, 161, 178, 195, 197, 214, 6, 23, 40], [73, 90, 92, 109, 126, 143, 160, 177, 194, 196, 213, 5, 22, 39, 56], [89, 91, 108, 125, 142, 159, 176, 193, 210, 212, 4, 21, 38, 55, 72], [105, 107, 124, 141, 158, 175, 192, 209, 211, 3, 20, 37, 54, 71, 88], [106, 123, 140, 157, 174, 191, 208, 225, 2, 19, 36, 53, 70, 87, 104]],)]],
dtype=[('c', 'O')]),)]],)]]
a [[([[(array([[1]], dtype=uint8),)]],)]]
b [[([[(array([[0]], dtype=uint8),)]],)]]
__version__ 1.0
Suppose I have made a large list of numbers, and I want to make another one which I will add, pairwise, with the first list.
Here's the first list, A:
[109, 77, 57, 34, 94, 68, 96, 72, 39, 67, 49, 71, 121, 89, 61, 84, 45, 40, 104, 68, 54, 60, 68, 62, 91, 45, 41, 118, 44, 35, 53, 86, 41, 63, 111, 112, 54, 34, 52, 72, 111, 113, 47, 91, 107, 114, 105, 91, 57, 86, 32, 109, 84, 85, 114, 48, 105, 109, 68, 57, 78, 111, 64, 55, 97, 85, 40, 100, 74, 34, 94, 78, 57, 77, 94, 46, 95, 60, 42, 44, 68, 89, 113, 66, 112, 60, 40, 110, 89, 105, 113, 90, 73, 44, 39, 55, 108, 110, 64, 108]
And here's B:
[35, 106, 55, 61, 81, 109, 82, 85, 71, 55, 59, 38, 112, 92, 59, 37, 46, 55, 89, 63, 73, 119, 70, 76, 100, 49, 117, 77, 37, 62, 65, 115, 93, 34, 107, 102, 91, 58, 82, 119, 75, 117, 34, 112, 121, 58, 79, 69, 68, 72, 110, 43, 111, 51, 102, 39, 52, 62, 75, 118, 62, 46, 74, 77, 82, 81, 36, 87, 80, 56, 47, 41, 92, 102, 101, 66, 109, 108, 97, 49, 72, 74, 93, 114, 55, 116, 66, 93, 56, 56, 93, 99, 96, 115, 93, 111, 57, 105, 35, 99]
How might I generate the arithmetic addition logic, processing each pairwise value one by one (A[0] and B[0], through A[99], B[99]) and producing the list C (A[0] + B[0] through A[99]+ B[99])?
result = [(x + y) for x, y in itertools.izip(A, B)]
Or:
result = map(operator.add, itertools.izip(A, B))
Here is two possible options:
Use list comprehension.
Use NumPy.
I will be using shortened versions of your lists for convenience, and the element-wise sum will go into c.
List comprehension
a = [109, 77, 57, 34, 94, 68, 96]
b = [35, 106, 55, 61, 81, 109, 82]
c = [a_el + b_el for a_el,b_el in zip(a, b)]
NumPy
import numpy as np
a = np.array([109, 77, 57, 34, 94, 68, 96])
b = np.array([35, 106, 55, 61, 81, 109, 82])
c = a + b
With a list comprehension:
C = [A[i]+ B[i] for i in range(len(A))]
And even safer:
C = [A[i]+ B[i] for i in range(len(A)) if len(A) == len(B)]