error while receiving data and converting it into f64 - python

I am trying to send data over a TCP connection from rust to python, however while receiving the data in python I am getting the following error when trying to convert it from bytes to f64.
Traceback (most recent call last):
File "server.py", line 36, in <module>
[x] = struct.unpack('f', data)
struct.error: unpack requires a buffer of 4 bytes
I am using the follwoing method to convert the data from bytes to f64,
[x] = struct.unpack('f', data)
print(x)
my data looks like this, which I am sending over a tcp
x: 0.011399809271097183 (f64 from rust)
and getting something like
b'?t=*\x00\x00\x00\x00?s\xbd\xc6\x80\x00\x00\x00?q\xd5s\x00\x00\x00\x00?|\xae\x85\x80\x00\x00\x00?e\xb5\xc3\x00\x00\x00\x00?yp;\x80\x00\x00\x00?p\x7f\x98\x00\x00\x00\x00?hG|\x00\x00\x00\x00?o\x8d&\x00\x00\x00\x00?cv[\x00\x00\x00\x00?s\xdf\x97\x80\x00\x00\x00?{\x0e\xde\x80\x00\x00\x00?n\xec\xbf\x00\x00\x00\x00?n\xd8E\x00\x00\x00\x00?y+\xdd\x80\x00\x00\x00?r\xd90\x80\x00\x00\x00?r\xc2\x89\x00\x00\x00\x00?q\xc2i\x00\x00\x00\x00?kq"\x00\x00\x00\x00?t5\xec\x80\x00\x00\x00?|\xaak\x80\x00\x00\x00?z\x10\x9d\x00\x00\x00\x00?o\xeb\xde\x00\x00\x00\x01?m6\xfc\x00\x00\x00\x00'

It appears 24 8-byte floats (double type in C) were sent in big-endian format. At least the data appears as something reasonable, although none of them match the single value listed in the question:
>>> data = b'?t=*\x00\x00\x00\x00?s\xbd\xc6\x80\x00\x00\x00?q\xd5s\x00\x00\x00\x00?|\xae\x85\x80\x00\x00\x00?e\xb5\xc3\x00\x00\x00\x00?yp;\x80\x00\x00\x00?p\x7f\x98\x00\x00\x00\x00?hG|\x00\x00\x00\x00?o\x8d&\x00\x00\x00\x00?cv[\x00\x00\x00\x00?s\xdf\x97\x80\x00\x00\x00?{\x0e\xde\x80\x00\x00\x00?n\xec\xbf\x00\x00\x00\x00?n\xd8E\x00\x00\x00\x00?y+\xdd\x80\x00\x00\x00?r\xd90\x80\x00\x00\x00?r\xc2\x89\x00\x00\x00\x00?q\xc2i\x00\x00\x00\x00?kq"\x00\x00\x00\x00?t5\xec\x80\x00\x00\x00?|\xaak\x80\x00\x00\x00?z\x10\x9d\x00\x00\x00\x00?o\xeb\xde\x00\x00\x00\x01?m6\xfc\x00\x00\x00\x00'
>>> s
b'?t=*\x00\x00\x00\x00?s\xbd\xc6\x80\x00\x00\x00?q\xd5s\x00\x00\x00\x00?|\xae\x85\x80\x00\x00\x00?e\xb5\xc3\x00\x00\x00\x00?yp;\x80\x00\x00\x00?p\x7f\x98\x00\x00\x00\x00?hG|\x00\x00\x00\x00?o\x8d&\x00\x00\x00\x00?cv[\x00\x00\x00\x00?s\xdf\x97\x80\x00\x00\x00?{\x0e\xde\x80\x00\x00\x00?n\xec\xbf\x00\x00\x00\x00?n\xd8E\x00\x00\x00\x00?y+\xdd\x80\x00\x00\x00?r\xd90\x80\x00\x00\x00?r\xc2\x89\x00\x00\x00\x00?q\xc2i\x00\x00\x00\x00?kq"\x00\x00\x00\x00?t5\xec\x80\x00\x00\x00?|\xaak\x80\x00\x00\x00?z\x10\x9d\x00\x00\x00\x00?o\xeb\xde\x00\x00\x00\x01?m6\xfc\x00\x00\x00\x00'
>>> len(data)
192
>>> len(s)/8
24.0
>>> import struct
>>> struct.unpack('>24d',s)
(0.004941143095493317, 0.004819655790925026, 0.004353951662778854, 0.007002374157309532, 0.0026501473039388657, 0.0062105488032102585, 0.00402793288230896, 0.0029637739062309265, 0.0038514845073223114, 0.0023757722228765488, 0.004851905629038811, 0.006605977192521095, 0.0037749987095594406, 0.003765234723687172, 0.006145348772406578, 0.004601659253239632, 0.004580054432153702, 0.004335794597864151, 0.003349844366312027, 0.0049342382699251175, 0.006998462602496147, 0.0063634999096393585, 0.003896649926900864, 0.003566257655620575)

Related

How to decode a byte array to string array in Python

Hello I used encryption algorithm called fernet to encrypt values in the database. I want to retrieve those values decrypted. The array which is being returned is a byte array, but I would like to decode it. It doesn't seem to be working. I am getting this error:
Traceback (most recent call last):
File "C:\Users\d\OneDrive\Bureaublad\Assignment8\CDMS.py", line 51, in <module>
get_clients()
File "C:\Users\d\OneDrive\Bureaublad\Assignment8\CDMS.py", line 41, in get_clients
print(arr[i].decode())
TypeError: list indices must be integers or slices, not bytes
This is the array which im trying to decode:
[(b'gAAAAABg5EBDnm6LZdHNzzjnDlFcA80SeS1s-wXhKx_opwOm0vSNsfcxheZC4abQ-qvifmxxbCT8gGy2O1dz-76gx3_ALVmeZQ==', b'gAAAAABg5EBD1k2kpmYaAI0PNfjHondg5mB9RwxoIxVqAkfZcAeYrK-H4oqyYjV9WcHFN0CF7Xf0dN2rAXKrkAq-myyfNUplSg==', b'gAAAAABg5EBDJrUQazdiAdf4yejxIwzZHSln-hAcYLkh27-7HO5gE8XjHZjUz3Y2-lHoDIINSAzUt10J93eZ2EeCNUGA1mVrgw==', b'gAAAAABg5EBDp5znqWT87IMASAjDuhMtB7uMV2rezOEQzy_O4ka8R8Dn0fCkZhajVBIQ3S0-MQiM2Mq0qS8JbKCD_rZhbaFPcA==', b'gAAAAABg5EBDZ3YdqhKFRZWQeKP3VUvaVWp-pmSub2aZtQGIpirZ6Pi9zplxdqCindAOHJIQrlcjAr_FG_vnH8U89_bx7FjX2Q==', b'gAAAAABg5EBDjIcwfpyMYXxoEK9XZjJEDyFM3sWfpM61hgAIOf4cNTj1etS1nCMsLIZtKIrF5wSRkmzP_tiXyDolisVkjr6uuA=='), ('gAAAAABg5EDjNQV_tNHuRRkZvdLDfnI20iGZ0Cpzv--1Xct0brLjCOlqnt0ZOU0AYe-tbQ390yIF2B-xlDGuP1-isAtgkWtGTQ==', 'gAAAAABg5EDjC84Zs7HPm1zPe9O2sWNX_HSf3pZeyivmqaPC-VYu5qely818-srsNOtVYtNoGCineThtqQuEC7vjteYV3sHIjA==', 'gAAAAABg5EDj-rUfVVvzObgt4HcgUr7UczgKTrdQyim-ZIY9AeFFGpreZeUS0G9ojkqX_o1AeDyUkcZ4mKIVwoK1dSOINPcp1Q==', 'gAAAAABg5EDjRwttFLM-AhzcFPsKX_cZv9BXnZd__wUH2fRudsFCTwwGnQxhWgxNjNN42zUs_ig6iVmLOJVZRJWno5LzRNQuVQ==', 'gAAAAABg5EDjyAbkEspirSsfj9tzwk5nwoR1rTP9QLjFSOO43Ju3eA1sBEiyTzn_yqpN8O67efYH1F7cEIrK9ne8D9zIRlg_RA==', 'gAAAAABg5EDj0Cgif60GqnxTu-_sd5gnFZvgvNyUZLoyh_QDRtq0ZHihaWiWv9ck9WlI1dKPKeg4nddFHep4lhb0YslRg7ZMhA=='), ('gAAAAABg5EOIDTUoHx9go4zl2J_WEcIcTKlCICNDGm241Izle6lH4BekWq6xH8i5IkhaSZPz82ADlQSyVAhhmCtu8FPN4sixOg==', 'gAAAAABg5EOIwgZcgSDSI0M4y9-faMjWBXtyx-GsJHmVV6P8QEru6wWMqPu0KFRFJB8CscVA8PtazieIcjcFqnpP_GTsG7Zebw==', 'gAAAAABg5EOIUa-sJK-76B45cAMZ-Vpog6KouDdo7p56Tmyq4GwF4Cw5ANALF-xckR52gpN1B6_YCY6V5x642lyKMLbzZh3O6A==', 'gAAAAABg5EOIm6UpODzgfb9sxJujrNtaPRDJS_GjXnS9jNHAb4wN3aQljZAKARboQiRvG1WYkVLCetaNN8jOSgF15tWgYmugNQ==', 'gAAAAABg5EOIOQ9mB2ONC-0HCXER4xsCyBxRRcDVRoe6WICyQJvS1ZpdgI2UjASL4yYvRjoLxyRm03Ka0o4iuPoTYIOk7JOnIA==', 'gAAAAABg5EOIt6wbW5vS2PrZjeu7RhBioL-U_c63nXZLJRH1gajaaV6kzbsIHLSFYcawXiwZ75sUnQTFj5JPtMlTFj8TRo80nw=='), ('gAAAAABg5EQXGsA0VtAientcwIOpL6igK7fan8lYGKlAbyEpEg4sH4JoHmOmYN4vTfZxXiS_E1Jn_me-L_xpbqgtC9z2ihPnWQ==', 'gAAAAABg5EQXDij9NMYOva_CTBXsPZMT8rGiEPZnHOuFbjIHHX2Wi46DOFMtEHy6O1MP7HimMJsPRkXgIrb7MGr610clFlhjRg==', 'gAAAAABg5EQXupxCcXLmtqjskNWx9TQVaF-ZHovEhv1g6qNA8zDkjY_4rA2Utk8sFkaaJH86pBuyIFmtGZ1i47zr0FufPneJcQ==', 'gAAAAABg5EQX6qWDg3eBdr4Q2YT6n5Zgy6UwLQFrB-5C-lXjoIu8NjK14zh87B8K_IAhjGN9VRJLskQMN8BtoCo4ZopoS1AqhQ==', 'gAAAAABg5EQXBUIZLqjgVMHauf279CbtMzam4yIaZHSO6-cOC8-0t_VTjgG7mpjcujMgTLeOFwBF_mwIHRpG8HubGMUnjbnQ4A==', 'gAAAAABg5EQXNCIO8OmZA5vsULGgagHsTE-KM5LeEOMjVJ4SzkAtev1DsbYQmST98FmHOUwolR2WbDw2pK11J9C292Jvil_nnQ=='), ('gAAAAABg5EQjygSazmI5VnieVQG1cAsvgsm_u7tHE9X9CevMXqIEcWbe1SOK5wRDza9LXXn_NbCaffJXrq4Dz9BcZh96mh9aqQ==', 'gAAAAABg5EQjEhshfY6M7Yqp4Nm4EE1-Cz7B6P8m7F7jA99qk3D3U3W4ejWz576t5Qg1uBYWclY6NI_SouQ2cGH7PXEdjv_4Gw==', 'gAAAAABg5EQjKzUAlURxaoeAiDAAiVNIvgKE3r8U0fwFQWKtUWSTTAe0RugxV8_QYDqOndxVBxhrRsWDTP1p2Kzbn_do9T7Zfg==', 'gAAAAABg5EQj8UYXm2nqqsrN2Xh5y7L0h1uezTzL-WFZxekurC_Z4VcZfuOVKE5nFlFWfHu6x0OUer1j9N1bIZPVvMY_6_NsSg==', 'gAAAAABg5EQj1_sXfsbRtX24ZnY2rjoorebkZp1010W4wtssa2T6ZJxrzQjHr1ELYbB77Mtm_w_euihEoXigkhZhQXkkTysC6Q==', 'gAAAAABg5EQjeHuoLttV3wRj9DgIVseVvdMmYmWX3-D-UIckrjrizKM6N6OSpq98rPkk9DCutqgW9Jup2KD4bshhCtr8ipO1-g=='), ('gAAAAABg5EVhxeWhA7-oBZPT93nSe3p6vvxqcysxOjdyGyVNQdBwdtM2heHrhNnNSes8yang0Cx9vgBIsU7Yjrfdgz8tOWpK_A==', 'gAAAAABg5EVhdtXop2FNWsZSndOxQ3iEuJQyfI1flL0swLpcJKKTjBlEZhWpwoB3XTO2STYRVrimyEljZs751aVgveF0-rpEpA==', 'gAAAAABg5EVhyWQRkpFOd6Y-_OYqDJX5FVfpYXmd_2SKMZd-EPFYqOu5s47C7lw23zs4eHFVeEF0tSnSpr58rW6hWnxlOnjhnQ==', 'gAAAAABg5EVhCmgEjitk1cutyI5Nn5qYFsh0LRd3-uBJekPrZFoJb6S7hODtwNP-cQoV8R6u5eRLdUOeqffwCKIbBPYA9_lyTQ==', 'gAAAAABg5EVhKZggbtmupc2FqzfbSREmsws-MFYTEX-kGwfw-JJmBOh_7RJnTa-23HWJQtfChSWWuXzsxULJ5vI289c0BVIqrw==', 'gAAAAABg5EVhy8aowZUiuX6M2IPW35XFZNBueXDd_JFal39ViFmg6avpbkvs0mhVzpqVvXjNcMmr3QbNeXf3daLCDa7_KdCmmg==')]
This is the code where im trying to decode it:
def get_clients():
arr = []
retrieved_vals = ""
c.execute("SELECT * FROM client")
arr = c.fetchall()
i = 0
for i in arr[i]:
print(arr[i].decode())
What can i do?
Arr is a tuple within a list with encoded strings.
so your for needs to look like this: for i in arr[0]: print(i.decode())

Python struct.unpack byte length issues

I have the following code:
msg = b'0,[\x00\x01\x86\xec\x96N'
print(struct.unpack("<"+"I",msg))
however everytime i try to do this it says
struct.error: unpack requires a buffer of 4 bytes
What i tried to do is the following
times = int(len(msg)/4)
struct.unpack("<"+"I" * times,msg)
but it doesnt always work, i think on uneven numbers, how can i get the correct size so i dont encounter these issues?
struct.unpack requires that the length of the buffer being consumed is exactly the size of the format. [1]
Use struct.unpack_from instead, which requires that the length of the buffer being consumed is at least the size of the format. [2]
>>> msg = b'0,[\x00\x01\x86\xec\x96N'
>>> import struct
>>> print(struct.unpack("<"+"I", msg))
Traceback (most recent call last):
File "<input>", line 1, in <module>
struct.error: unpack requires a buffer of 4 bytes
>>> print(struct.unpack_from("<"+"I", msg))
(5975088,)
Additional bytes will be ignored by unpack_from
[1] https://docs.python.org/3/library/struct.html#struct.unpack
[2] https://docs.python.org/3/library/struct.html#struct.unpack_from

Convert Python's binascii.crc_hqx() back to ascii

I'm using the standard Python3 lib binascii, and specifically the crc_hqx() function
binascii.crc_hqx(data, value)
Compute a 16-bit CRC value of data, starting with value as the initial CRC, and return the result. This uses the CRC-CCITT polynomial x16 + x12 + x5 + 1, often represented as 0x1021. This CRC is used in the binhex4 format.
I'm able to convert to CRC with this code:
import binascii
t = 'abcd'
z = binascii.crc_hqx(t.encode('ascii'), 0)
print(t,z)
which, as expected, prints the line
abcd 43062
But how do I convert back to ASCII?
I've tried variations with the a2b_hqx() function
binascii.a2b_hqx(string)
Convert binhex4 formatted ASCII data to binary, without doing RLE-decompression. The string should contain a complete number of binary bytes, or (in case of the last portion of the binhex4 data) have the remaining bits zero.
The simplest version would be:
y = binascii.a2b_hqx(str(z))
But I've also tried variations with bytearray() and str.encode(), etc.
For this code:
import binascii
t = 'abcd'
z = binascii.crc_hqx(t.encode('ascii'), 0)
print(t,z)
y = binascii.a2b_hqx(str(z))
the Traceback:
abcd 43062
Traceback (most recent call last):
File "test.py", line 5, in <module>
y = binascii.a2b_hqx(str(z))
binascii.Incomplete: String has incomplete number of bytes
And with this code:
y = binascii.a2b_hqx(bytearray(z))
This Traceback:
binascii.Error: Illegal char
What is generated is a checksum and not possible to convert back to ascii.

Error in packing 128 byte structure using Struct in python

I want to pack 128 byte of different data types. The structure as follows
4 bytes - 0x12345678,
2 bytes - 0x1234,
120 bytes - 0x00 (repeats 120 times),
2 byte - 0x99 ,
I tried with below code but fails
struct.pack('<LH120BH',0x12345678,0x1234,0x00,0x99 )
gives error
Traceback (most recent call last):
File "<pyshell#10>", line 1, in <module>
struct.pack('<LH120BH',0x12345678,0x1234,0x00,0x99 )
struct.error: pack expected 123 items for packing (got 4)
pls help me. Thanks in advane
You may need to pack 0x00 in to an array if you want it to repeat 120 times and unpack it when call struct.pack, maybe something like this:
struct.pack('<LH120BH',0x12345678,0x1234,*[0x00] * 120,0x99)

XOR'ing bytes in a memory dump to check for existance of a key

Reading in a binary file and XORing bytes to find out if there is an AES 128 key present.
with open("FakeMemDump1.bin","rb") as memorydump:
mem = memorydump.read(memorybytes)
for i in mem:
w = mem[i]^mem[i+15]
x = mem[i+4]^mem[i+19]
y = mem[i+8]^mem[i+23]
z = mem[i+12]^mem[i+27]
if w==mem[i+16] and x==mem[i+20] and y==mem[i+24] and z==mem[i+28]:
keySched = mem[i-1:i+175]
print ("Key Schedule : "%keySched)
Traceback (most recent call last):
File "AES128Keyfind.py", line 8, in <module>
w = mem[i]^mem[i+15]
TypeError: string indices must be integers, not str
Above is the error I am receiving, is the byte being called passed in as a string even though the file is read as a binary file?

Categories