Reading a python binary file with a C# BinaryReader - python

I need to export some data like integers, floats etc. to a binary file with python. Afterwards, I have to read the file with C# again but it doesnt work for me.
I tried several ways of writing a binary file with python and it works as long as I read it with python as well:
a = 3
b = 5
with open('test.tcd', 'wb') as file:
file.write(bytes(a))
file.write(bytes(b))
or writing it like this:
import pickle as p
with open('test.tcd', 'wb') as file:
p.dump([a, b], file)
Currently I am reading the file in C# like this:
static void LoadFile(String path)
{
BinaryReader br = new BinaryReader(new FileStream(path, FileMode.Open));
int a = br.ReadInt32();
int b = br.ReadInt32();
System.Diagnostics.Debug.WriteLine(a);
System.Diagnostics.Debug.WriteLine(b);
br.Close();
}
Unfortunately the output isnt 3 and 5, instead my output is just zero. How do i read or write the binary file properly?

In Python, you have to write your integers with 4 bytes each. Read more here: struct.pack
a = 3
b = 5
with open('test.tcd', 'wb') as file:
f.write(struct.pack("<i", 3))
f.write(struct.pack("<i", 5))
Your C# code should work now.

It's possible python is not writing data in the same format that C# expects. You may need to swap byte endianess or do something else. You could read the raw bytes instead and use BitConverter to see if that fixes it.
Another option is to specify the endianess explicitly in python, I think big endian is the default binary reader format for C#.
an_int = 5
a_bytes_big = an_int.to_bytes(2, 'big')
print(a_bytes_big)
Output
b'\x00\x05'
a_bytes_little = an_int.to_bytes(2, 'little')
print(a_bytes_little)
Output
b'\x05\x00'

Related

MATLAB vs. Python Binary File Read

I have a MATLAB application that reads a .bin file and parses through the data. I am trying to convert this script from MATLAB to Python but am seeing discrepancies in the values being read.
The read function utilized in the MATLAB script is:
fname = 'file.bin';
f=fopen(fname);
data = fread(f, 100);
fclose(f);
The Python conversion I attempted is: (edited)
fname = 'file.bin'
with open(fname, mode='rb') as f:
data= list(f.read(100))
I would then print a side-by-side comparison of the read bytes with their index and found discrepancies between the two. I have confirmed that the values read in Python are correct by executing $ hexdump -n 100 -C file.bin and by viewing the file's contents on the application HexEdit.
I would appreciate any insight into the source of discrepancies between the two programs and how I may be able to resolve it.
Note: I am trying to only utilize built-in Python libraries to resolve this issue.
Solution: Utilizing incorrect file path/structure between programming languages. Implementing #juanpa.arrivillaga's suggestion cleanly reproduced the MATLAB results.
An exact translation of the MATLAB code, using NumPy, would be:
data = np.frombuffer(f.read(100), dtype=np.uint8).astype(np.float64)
python automatically transforms single bytes into unsigned integers, as done by matlab, so you just need to do the following.
fname = 'file.bin'
with open(fname, mode='rb') as f:
bytes_arr = f.read(100)
# Conversion for visual comparison purposes
data = [x for x in bytes_arr]
print(data)
also welcome to python, bytes is a built-in type, so please don't override the built-in bytes type ... or you'll run into unexpected problems.
Edit: as pointed by #juanpa.arrivillaga you could use the faster
fname = 'file.bin'
with open(fname, mode='rb') as f:
bytes_arr = f.read(100)
# Conversion for visual comparison purposes
data = list(bytes_arr)

How can I produce a similar output from file reading in python but in php

I have a program I am converting from python to PHP, basically it starts by reading a file and continues on to unpack the data. During the conversion I've realised that unpacking formats are different and tested this with the python output, though PHP does not produce the same output.
Is it possible to produce a similar output from the fopen and fread function as the python open function output?
Currently I have the python producing this byte string: b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
and I would like the same in PHP output but all I seem to get is a repeating amount of this symbol: � (Null bytes)
any ideas on how I could produce a byte string like the one in python but in php? Is there a way to use the python open function in PHP or reproduce the function so it can be used in PHP?
This is my basic php code:
$file = fopen($filename, "rb");
$contents = fread($file, filesize($filename));
This is my basic python code for reading the file:
f = open(filename, 'rb')
f.read()

How to print binary file output as Base 2 (in bits)?

I have a bin file which contains binary data stored in bytes.
When trying to read them in python, the output is something like this \xb5D\xbe"jSUk\xe75\x18}#\'%\x89oRqR\xfb\xe9\xe9\
How can I print file contents as Base 2 binary ?
For example 10000000 01000000 11000000 , etc
Here is an example reading 8 bytes at a time and formatting them in the way that you describe.
Note that you probably already have system utilities that will do a similar task, for example the od program on Unix-like systems.
with open("your_binary_file", "rb") as f:
while True:
data = f.read(8)
if not data:
break
print(" ".join(f"{byte:08b}" for byte in data))

Why am I only writing 28,672 bits to this file?

I have been working on a project where it is necessary to program a binary file, of a certain kind, to a AT28C256 chip. The specifics are not important beyond the fact that the file needs to be 32,768 bytes in size (exactly).
I have some "minimal problem" code here:
o = open("images.bin", "wb")
c = 0
for i in range(256):
for j in range(128):
c += 1
o.write(chr(0).encode('utf-8'))
print(c)
This, to me, would appear to write 32,768 bytes to a file (the split into i,j is necessary because I need to write an image to the device) as 128*256 = 32768. And the output of c is 32768!
But the file it creates is 28672 bytes long! The fact that this is 7000 in hex has not passed me by but I'm not sure why this is happening. Any ideas?
You should call o.close() to flush the write buffer and close the file properly.

Decompressing with PyLZMA

I compress my files using this script
import pylzma, struct
i = open(path+fileName,'rb')
o = open(path+zipName+'.zip','wb')
data = i.read()
c = pylzma.compressfile(data, eos=1)
result = c.read(5)
result += struct.pack('<Q', len(data))
o.write(result + c.read())
i.close()
o.close()
I use this method as shown in the PyLZMA documentation because it allows my files to be readable by 7zip or lzma.exe. Decompression using 7zip works fine but it does not work when I use PyLZMA. I use this:
i = open(path+name+'.zip', 'rb')
o = open(path+name, 'wb')
data = i.read()
u = pylzma.decompress(data)
o.write(u)
It stops on pylzma.decompress and I receive the following error:
TypeError: Error while decompressing: 1
If I'm reading the documentation correctly (I'm having trouble installing PyLZMA so I am unable to verify), compress() outputs a string that decompress() can handle.
However, in order to make the compressed string compatible with other utilities, it is necessary to insert the 8-byte length in between the first 5 bytes and the rest of the compressed data.
Thus, if you want to decompress the data using PyLZMA, I suspect you will need to manually remove that 8-byte length field (quickest way would probably be to open the input file, read 5 bytes, skip 8, then read the remainder of the file).

Categories