We Keep Coding

Advice to go from MATLAB .mat file with structures to a numpy array?

I am trying to read in a large structure from a .mat file into a jupyter notebook. I am a little new to python so I'm not sure why my solution isn't working.
The structure from MATLAB(2020) I am reading in is structured like this
pose.frames.ind
Where there are 44 frames and in each frame 63 ind. I am reading it into a jupyter notebook like this with mat4py. I am trying to adapt someone else's code so after I read it in I need to convert it to a tensor so it can go into another function.
from mat4py import loadmat
val = loadmat('pose.mat')
pose_body = val['pose']['frames']['ind'][0]
pose_body = np.asarray(pose_body)
pose_body = torch.FloatTensor(pose_body).to(comp_device)
When I feed pose_body = np.zeros([1,63]) into the line that changes it to a torch tensor, the code works fine. However, when I try to feed it this array I imported something goes wrong and I get the error of
RuntimeError: invalid argument 0: Tensors must have same number of dimensions: got 2 and 1 at C:/w/1/s/tmp_conda_3.7_055306/conda/conda-bld/pytorch_1556690124416/work/aten/src\THC/generic/THCTensorMath.cu:62
Is there an easier/better way to convert the data from matlab in the format I need? I am a little unfamiliar with python, and it seems like they're the same type of array. From doing type(pose_body), and tf.size(pose_body) I know that both numpy arrays have a shape of 63, a dtype of int32, and are of class "numpy.ndarray".

Converting float32 to bit-equivalent int32

The Pillow module in Python insists on opening a 32-bit/pixel TIFF file I have as if the pixels were of type float32, whereas I believe the correct interpretation is unsigned int32. If I go ahead and load the data into a 640x512 array of type float32, how can I retype it as uint32 while preserving the underlying binary representation?
In Fortran and C, it's easy to have pointers or data structures of different type pointing to the same block of physical memory so that the raw memory contents can be easily be interpreted according to whatever type I want. Is there an equivalent procedure in Python?
Sample follows (note that I have no information about compression etc.; the file in question was extracted by a commercial software program from a proprietary file format):
from PIL import Image
infile = "20181016_071207_367_R.tif"
im = Image.open(infile)
data = np.array(im.getdata())
print(data)
[ -9.99117374 -10.36103535 -9.80696869 ... -18.41988373 -18.35027885
-18.69905663]

Assuming you have im.mode originally equal to F, you can force Pillow to re-load the same data under a different mode (an very unusual desire indeed) in a somewhat hackish way like that:
imnew = im.convert(mode='I')
imnew.frombytes(im.tobytes())
More generally (outside the context of PIL), whenever you encounter the need to deal with raw memory representation in Python, you should usually rely on numpy or Python's built-in memoryview class with the struct module.
Here is an example of reinterpreting an array of numpy float32 as int32:
a = np.array([1.0, 2.0, 3.0], dtype='float32')
a_as_int32 = a.view('int32')
Here is an example of doing the same using memoryview:
# Create a memory buffer
b = bytearray(4*3)
# Write three floats
struct.pack_into('fff', b, 0, *[1.0, 2.0, 3.0])
# View the same memory as three ints
mem_as_ints = memoryview(b).cast('I')

The answer, in this case, is that Pillow is loading the image with the correct type (float 32) as specified in the image exported from the thermal camera. There is no need to cast the image to integer, and doing so would cause an incorrect result.

np.fromfile with count=-1 adds unexpected zeros

I am trying to use np.fromfile in order to read a binary file that I have written with Fortran using direct access. However if I set count=-1, instead of max_items, np.fromfile returns a larger array than expected; adding zeros to the vector I've written in binary.
Fortran test code:
program testing
implicit none
integer*4::i
open(1,access='DIRECT', recl=20, file='mytest', form='unformatted',convert='big_endian')
write(1,rec=1) (i,i=1,20)
close(1)
end program
How I am using np.fromfile:
import numpy as np
f=open('mytest','rb')
f.seek(0)
x=np.fromfile(f,'>i4',count=20)
print len(x),x
so if I use it like this it returns exactly my [1,...,20] np array, but setting count=-1 returns [1,...,20,0,0,0,0,0] with a size of 1600.
I am using a little endian machine (shouldn't affect anything) and I am compiling the Fortran code with ifort.
I am just curious about the reason this happens, to avoid any surprises in the future.

Read a float binary file into 2D arrays in python and matlab

I have some binary input files (extension ".bin") that describe a 2D field of ocean depth and which are all negative float numbers. I have been able to load them in matlab as follows:
f = fopen(filename,'r','b');
data = reshape(fread(f,'float32'),[128 64]);
This matlab code gives me double values between 0 and -5200. However, when I try to the same in Python, I strangely get values between 0 and 1e-37. The Python code is:
f = open(filename, 'rb')
data = np.fromfile(f, np.float32)
data.shape = (64,128)
The strange thing is that there is a mask value of 0 for land which shows up in the right places in the (64,128) array in both cases. It seems to just be the magnitude and sign of the numpy.float32 values that are off.
What am I doing wrong in the Python code?

numpy.fromfile isn't platform independant, especially the "byte-order" is mentioned in the documentation:
Do not rely on the combination of tofile and fromfile for data storage, as the binary files generated are are not platform independent. In particular, no byte-order or data-type information is saved.
You could try:
data = np.fromfile(f, '>f4') # big-endian float32
and:
data = np.fromfile(f, '<f4') # little-endian float32
and check which one (big endian or little endian) gives the correct values.

Base on your matlab fopen, the file is in big endian ('b'). But your python code does not take care of the endianness.

struct.error: unpack requires a string argument of length 4 - audio file

I am a very beginner in programming and I use Ubuntu.
But now I am trying to perform sound analysis with Python.
In the following code I used wav package to open the wav file and the struct to convert the information:
from wav import *
from struct import *
fp = wave.open(sound.wav, "rb")
total_num_samps = fp.getnframes()
num_fft = (total_num_samps / 512) - 2 #for a fft lenght of 512
for i in range(num_fft):
tempb = fp.readframes(512);
tempb2 = struct.unpack('f', tempb)
print (tempb2)
So in terminal the message that appears is:
struct.error: unpack requires a string argument of length 4
Please, can someone help me to solve this? Someone have a suggestion of other strategy to interpret the sound file?

The format string provided to struct has to tell it exactly the format of the second argument. For example, "there are one hundred and three unsigned shorts". The way you've written it, the format string says "there is exactly one float". But then you provide it a string with way more data than that, and it barfs.
So issue one is that you need to specify the exact number of packed c types in your byte string. In this case, 512 (the number of frames) times the number of channels (likely 2, but your code doesn't take this into account).
The second issue is that your .wav file simply doesn't contain floats. If it's 8-bit, it contains unsigned chars, if it's 16 bit it contains signed shorts, etc. You can check the actual sample width for your .wav by doing fp.getsampwidth().
So then: let's assume you have 512 frames of two-channel 16 bit audio; you would write the call to struct as something like:
channels = fp.getnchannels()
...
tempb = fp.readframes(512);
tempb2 = struct.unpack('{}h'.format(512*channels), tempb)

Using SciPy, you could load the .wav file into a NumPy array using:
import scipy.io.wavfile as wavfile
sample_rate, data = wavfile.read(FILENAME)
NumPy/SciPy will also be useful for computing the FFT.
Tips:
On Ubuntu, you can install NumPy/SciPy with
sudo apt-get install python-scipy
This will install NumPy as well, since NumPy is a dependency of SciPy.
Avoid using * imports such as from struct import *. This copies
names from the struct namespace into the current module's global
namespace. Although it saves you a bit of typing, you pay an awful
price later when the script becomes more complex and you lose
track of where variables are coming from (or worse, the imported variables
mask the value of other variables with the same name).