I'm reading 16-bit integers from a piece of hardware over the serial port.
Using Python, how can I get the LSB and MSB right, and make Python understand that it is a 16 bit signed integer I'm fiddling with, and not just two bytes of data?
Try using the struct module:
import struct
# read 2 bytes from hardware as a string
s = hardware.readbytes(2)
# h means signed short
# < means "little-endian, standard size (16 bit)"
# > means "big-endian, standard size (16 bit)"
value = struct.unpack("<h", s) # hardware returns little-endian
value = struct.unpack(">h", s) # hardware returns big-endian
Related
import struct
x = struct.pack('i', 10)
print(x)
result:
b'\n\x00\x00\x00'
if i want to convert b'\n\x00\x00\x00' to and integer it works:
bytes_to_convert = b'\n\x00\x00\x00'
bytes_converter = int.from_bytes(bytes_to_convert, byteorder='little', signed=False)
print(bytes_converter)
result:
10
everything's working perfectly but when it comes to bytes() function :
print(bytes('10', 'utf-16'))
if i want to convert it to an integer:
i get this value : 206161706751
Where is the difference between using bytes() and using struct to pack data into bytes i used i in struct means 4 bytes and bytes is using 4 bytes what is the difference ?
10 is a number ("ten"), which is stored in your computer as 00000000 00001010
'10' is a string, consisting of symbols 1 and 0. It is stored as 00110000 00110001.
From the computer's standpoint, there's nothing in common between these two pieces of data. Yes, the string '10' happens to be a name for the number ten (in our language, at least), but this connection only exists in our brain. The computer is not aware of it.
This question already has answers here:
Convert bytes to int?
(7 answers)
Closed 8 months ago.
I receive a 32-bit number over the serial line, using num = ser.read(4). Checking the value of num in the shell returns something like a very unreadable b'\xcbu,\x0c'.
I can check against the ASCII table to find the values of "u" and ",", and determine that the hex value of the received number is actually equal to "cb 75 2c 0c", or in the format that Python outputs, it's b'\xcb\x75\x2c\x0c'. I can also type the value into a calculator and convert it to decimal (or run int(0xcb752c0c) in Python), which returns 3413453836.
How can I do this conversion from a binary string literal to an integer in Python?
I found two alternatives to solve this problem.
Using the int.from_bytes(bytes, byteorder, *, signed=False) method
Using the struct.unpack(format, buffer) from the builtin struct module
Using int.from_bytes
Starting from Python 3.2, you can use int.from_bytes.
Second argument, byteorder, specifies endianness of your bytestring. It can be either 'big' or 'little'. You can also use sys.byteorder to get your host machine's native byteorder.
from the docs:
The byteorder argument determines the byte order used to represent the integer. If byteorder is "big", the most significant byte is at the beginning of the byte array. If byteorder is "little", the most significant byte is at the end of the byte array. To request the native byte order of the host system, use sys.byteorder as the byte order value.
int.from_bytes(bytes, byteorder, *, signed=False)
Code applying in your case:
>>> import sys
>>> int.from_bytes(b'\x11', byteorder=sys.byteorder)
17
>>> bin(int.from_bytes(b'\x11', byteorder=sys.byteorder))
'0b10001'
Here is the official demonstrative code from the docs:
>>> int.from_bytes(b'\x00\x10', byteorder='big')
16
>>> int.from_bytes(b'\x00\x10', byteorder='little')
4096
>>> int.from_bytes(b'\xfc\x00', byteorder='big', signed=True)
-1024
>>> int.from_bytes(b'\xfc\x00', byteorder='big', signed=False)
64512
>>> int.from_bytes([255, 0, 0], byteorder='big')
16711680
Using the struct.unpack method
The function you need to achieve your goal is struct.unpack.
To understand where you can use it, we need to understand the parameters to give and their impact on the result.
struct.unpack(format, buffer)
Unpack from the buffer buffer (presumably packed by pack(format, ...)) according to the format string format. The result is a tuple even if it contains exactly one item. The buffer’s size in bytes must match the size required by the format, as reflected by calcsize().
The buffer is the bytes that we have to give and the format is how the bytestring should be read.
The information will be split into a string, format characters, that can be endianness, ctype, bytesize, ecc..
from the docs:
Format characters have the following meaning; the conversion between C and Python values should be obvious given their types. The ‘Standard size’ column refers to the size of the packed value in bytes when using standard size; that is, when the format string starts with one of '<', '>', '!' or '='. When using native size, the size of the packed value is platform-dependent.
This table represents the format characters currently avaiable in Python 3.10.6:
Format
C-Type
Standard Size
x
pad byte
c
char
1
b
signed char
1
B
uchar
1
?
bool
1
h
short
2
H
ushort
2
i
int
4
I
uint
4
l
long
4
L
ulong
4
q
long long
8
Q
unsigned long long
8
n
ssize_t
N
unsigned ssize_t
f
float
d
double
s
char[]
p
char[]
P
void*
and here is a table to use it to correct byte order:
Character
Byte order
Size
Alignment
#
native
native
Native
=
native
standard
None
<
little-endian
standard
None
>
big-endian
standard
None
!
network (= big-endian)
standard
None
Examples
Here is an example how you can use it:
>>> import struct
>>> format_chars = '<i' #4 bytes, endianness is 'little'
>>> struct.unpack(format_chars,b"f|cs")
(1935899750,)
check the builtin struct module.
https://docs.python.org/3/library/struct.html
in your case, it should probably be something like:
import struct
struct.unpack(">I", b'\xcb\x75\x2c\x0c')
but it depends on Endianness and signed/unsigned, so do read the entire doc.
I am faced with a problem in Python and I think I don't understand how signed numbers are handled in Python. My logic works in Java where everything is signed so need some help in Python.
I have some bytes that are coded in HEX and I need to decode them and interpret them to numbers. The protocol are defined.
Say the input may look like:
raw = '016402570389FFCF008F1205DB2206CA'
And I decode like this:
bin_bytes = binascii.a2b_hex(raw)
lsb = bin_bytes[5] & 0xff
msb = bin_bytes[6] << 8
aNumber = int(lsb | msb)
print(" X: " + str(aNumber / 4000.0))
After dividing by 4000.0, X can be in a range of -0.000025 to +0.25.
This logic works when X is in positive range. When X is expected
to be negative, I am getting back a positive number.
I think I am not handling "msb" correctly when it is a signed number.
How should I handlehandle negative signed number in
Python?
Any tips much appreciated.
You can use Python's struct module to convert the byte string to integers. It takes care of endianness and sign extension for you. I guess you are trying to interpret this 16-byte string as 8 2-byte signed integers, in big-endian byte order. The format string for this is '>8h. The > character tells Python to interpret the string as big endian, 8 means 8 of the following data type, and h means signed short integers.
import struct
nums = struct.unpack('>8h', bin_bytes)
Now nums is a tuple of integers that you can process further.
I'm not quite sure if your data is little or big endian. If it is little-endian, you can use < to indicate that in the struct.unpack format string.
I need to send exactly 24 bits over an ethernet connection, and the program on the other end expects an unsigned int in some cases and a signed int in others (C types). I want to use the struct class, but it doesn't have a type with 3 bytes built in (like uint24_t).
Similar questions to this have been asked, but the answer always involves sending 4 bytes and padding the data packet with zeros. I cannot do this, however, since I am not writing the program which is receiving the data, and it expects exactly 24 bits.
I am very new at this type of programming, so help is appreciated!
Using the struct module, create a string that contains exactly three 8-bit bytes.
import struct
# 24 bits: 01010101 10101010 11110000
byte1 = 0x55
byte2 = 0xaa
byte3 = 0xf0
data = struct.pack("BBB", byte1, byte2, byte3)
Depending on how you get the bits to send, you can define the string directly:
data = '\x55\xaa\xf0'
I am currently using an Arduino that's outputting some integers (int) through Serial (using pySerial) to a Python script that I'm writing for the Arduino to communicate with X-Plane, a flight simulation program.
I managed to separate the original into two bytes so that I could send it over to the script, but I'm having a little trouble reconstructing the original integer.
I tried using basic bitwise operators (<<, >> etc.) as I would have done in a C++like program, but it does not seem to be working.
I suspect it has to do with data types. I may be using integers with bytes in the same operations, but I can't really tell which type each variable holds, since you don't really declare variables in Python, as far as I know (I'm very new to Python).
self.pot=self.myline[2]<<8
self.pot|=self.myline[3]
You can use the struct module to convert between integers and representation as bytes. In your case, to convert from a Python integer to two bytes and back, you'd use:
>>> import struct
>>> struct.pack('>H', 12345)
'09'
>>> struct.unpack('>H', '09')
(12345,)
The first argument to struct.pack and struct.unpack represent how you want you data to be formatted. Here, I ask for it to be in big-ending mode by using the > prefix (you can use < for little-endian, or = for native) and then I say there is a single unsigned short (16-bits integer) represented by the H.
Other possibilities are b for a signed byte, B for an unsigned byte, h for a signed short (16-bits), i for a signed 32-bits integer, I for an unsigned 32-bits integer. You can get the complete list by looking at the documentation of the struct module.
For example, using Big Endian encoding:
int.from_bytes(my_bytes, byteorder='big')
What you have seems basically like it should work, assuming the data stored in myline has the high byte first:
myline = [0, 1, 2, 3]
pot = myline[2]<<8 | myline[3]
print 'pot: {:d}, 0x{:04x}'.format(pot, pot) # outputs "pot: 515, 0x0203"
Otherwise, if it's low-byte first you'd need to do the opposite way:
myline = [0, 1, 2, 3]
pot = myline[3]<<8 | myline[2]
print 'pot: {:d}, 0x{:04x}'.format(pot, pot) # outputs "pot: 770, 0x0302"
This totally works:
long = 500
first = long & 0xff #244
second = long >> 8 #1
result = (second << 8) + first #500
If you are not sure of types in 'myline' please check Stack Overflow question How to determine the variable type in Python?.
To convert a byte or char to the number it represents, use ord(). Here's a simple round trip from an int to bytes and back:
>>> number = 3**9
>>> hibyte = chr(number / 256)
>>> lobyte = chr(number % 256)
>>> hibyte, lobyte
('L', '\xe3')
>>> print number == (ord(hibyte) << 8) + ord(lobyte)
True
If your myline variable is string or bytestring, you can use the formula in the last line above. If it somehow is a list of integers, then of course you don't need ord.