We Keep Coding

Convert binary to signed, little endian 16bit integer in Python

Trying to a convert a binary list into a signed 16bit little endian integer
input_data = [['1100110111111011','1101111011111111','0010101000000011'],['1100111111111011','1101100111111111','0010110100000011']]
Desired Output =[[-1074, -34, 810],[-1703, -39, 813]]
This is what I've got so far. It's been adapted from: Hex string to signed int in Python 3.2?,
Conversion from HEX to SIGNED DEC in python
results = []
for i in input_data:
hex_convert = [hex(int(x,2)) for x in i]
convert = [int(y[4:6] + y[2:4], 16) for y in hex_convert]
results.append(convert)
print (results)
output: [[64461, 65502, 810], [64463, 65497, 813]]
This is works fine, but the above are unsigned integers. I need signed integers capable of handling negative values. I then tried a different approach:
results_2 = []
for i in input_data:
hex_convert = [hex(int(x,2)) for x in i]
to_bytes = [bytes(j, 'utf-8') for j in hex_convert]
split_bits = [int(k, 16) for k in to_bytes]
convert_2 = [int.from_bytes(b, byteorder = 'little', signed = True) for b in to_bytes]
results_2.append(convert_2)
print (results_2)
Output: [[108191910426672, 112589973780528, 56282882144304], [108191943981104, 112589235583024, 56282932475952]]
This result is even more wild than the first. I know my approach is wrong, and it doesn't help that i've never been able to get my head around binary conversion etc, but I feel i'm on the right path with:
(b, byteorder = 'little', signed = True)
but can't work out where i'm wrong. Any help explaining this concept would be greatly appreciated.

This result is even more wild than the first. I know my approach is wrong... but can't work out where i'm wrong.
The problem is in the conversion to bytes. Let's look at it a step at a time:
int(x, 2)
Fine; we treat the string as a base-2 representation of the integer value, and get that integer. Only problem is it's a) unsigned and b) big-endian.
hex(int(x,2))
What this does is create a string representation of the integer, in base 16, with a 0x prefix. Notably, there are two text characters per byte that we want. This is already heading is down the wrong path.
You might have thought of using hexadecimal because you've seen \xAB style escapes inside string representations. This is a completely different thing. The string '\xAB' contains one character. The string '0xAB' contains four.
From there, everything else is still nonsense. Converting to bytes with a text encoding just means that the text character 0 for example is replaced with the byte value 48 (since in UTF-8 it's encoded with a single byte with that value). For this data we get the same results with UTF-8 that we would by assuming plain ASCII (since UTF-8 is "ASCII transparent" and there are no non-ASCII characters in the text).
So how do we do it?
We want to convert the integer from the first step into the bytes used to represent it. Just as there is a .from_bytes class method allowing us to create an integer from underlying bytes, there is an instance method allowing us to get the bytes that would represent the integer.
So, we use .to_bytes, specifying the length, signedness and endianness that was assumed when we created the int from the binary string - that gives us bytes that correspond to that string. Then, we re-create the integer from those bytes, except now specifying the proper signedness and endianness. The reason that .to_bytes makes us specify a length is because the integer doesn't have a particular length - there are a minimum number of bytes required to represent it, but you could use as many more as you like. (This is especially important if you want to handle signed values, since it will do sign-extension automatically.)
Thus:
for i in input_data:
values = [int(x,2) for x in i]
as_bytes = [x.to_bytes(2, byteorder='big', signed=False) for x in values]
reinterpreted = [int.from_bytes(x, byteorder='little', signed=True) for x in as_bytes]
results_2.append(reinterpreted)
But let's improve the organization of the code a bit. I will first make a function to handle a single integer value, and then we can use comprehensions to process the list. In fact, we can use nested comprehensions for the nested list.
def as_signed_little(binary_str):
# This time, taking advantage of positional args and default values.
as_bytes = int(binary_str, 2).to_bytes(2, 'big')
return int.from_bytes(as_bytes, 'little', signed=True)
# And now we can do:
results_2 = [[as_signed_little(x) for x in i] for i in input_data]

Python convert Hex string to Signed Integer

I have a hex string, for instance: 0xb69958096aff3148
And I want to convert this to a signed integer like: -5289099489896877752
In Python, if I use the int() function on above hex number, it returns me a positive value as shown below:
>>> int(0xb69958096aff3148)
13157644583812673864L
However, if I use the "Hex" to "Dec" feature on Windows Calculator, I get the value as: -5289099489896877752
And I need the above signed representation.
For 32-bit numbers, as I understand, we could do the following:
struct.unpack('>i', s)
How can I do it for 64-bit integers?
Thanks.

If you want to convert it to 64-bit signed integer then you can still use struct and pack it as unsigned integer ('Q'), then unpack as signed ('q'):
>>> struct.unpack('<q', struct.pack('<Q', int('0xb69958096aff3148', 16)))
(-5289099489896877752,)

I would recommend the bitstring package available through conda or pip.
from bitstring import BitArray
b = BitArray('0xb69958096aff3148')
b.int
# returns
-5289099489896877752
Want the unsigned int?:
b.uint
# returns:
13157644583812673864

You could do a 64-bit version of this, for example:
def signed_int(h):
x = int(h, 16)
if x > 0x7FFFFFFFFFFFFFFF:
x -= 0x10000000000000000
return x
print(signed_int('0xb69958096aff3148'))
Output
-5289099489896877752

Converting bytes to signed numbers in Python

I am faced with a problem in Python and I think I don't understand how signed numbers are handled in Python. My logic works in Java where everything is signed so need some help in Python.
I have some bytes that are coded in HEX and I need to decode them and interpret them to numbers. The protocol are defined.
Say the input may look like:
raw = '016402570389FFCF008F1205DB2206CA'
And I decode like this:
bin_bytes = binascii.a2b_hex(raw)
lsb = bin_bytes[5] & 0xff
msb = bin_bytes[6] << 8
aNumber = int(lsb | msb)
print(" X: " + str(aNumber / 4000.0))
After dividing by 4000.0, X can be in a range of -0.000025 to +0.25.
This logic works when X is in positive range. When X is expected
to be negative, I am getting back a positive number.
I think I am not handling "msb" correctly when it is a signed number.
How should I handlehandle negative signed number in
Python?
Any tips much appreciated.

You can use Python's struct module to convert the byte string to integers. It takes care of endianness and sign extension for you. I guess you are trying to interpret this 16-byte string as 8 2-byte signed integers, in big-endian byte order. The format string for this is '>8h. The > character tells Python to interpret the string as big endian, 8 means 8 of the following data type, and h means signed short integers.
import struct
nums = struct.unpack('>8h', bin_bytes)
Now nums is a tuple of integers that you can process further.
I'm not quite sure if your data is little or big endian. If it is little-endian, you can use < to indicate that in the struct.unpack format string.

How can I convert two bytes of an integer back into an integer in Python?

I am currently using an Arduino that's outputting some integers (int) through Serial (using pySerial) to a Python script that I'm writing for the Arduino to communicate with X-Plane, a flight simulation program.
I managed to separate the original into two bytes so that I could send it over to the script, but I'm having a little trouble reconstructing the original integer.
I tried using basic bitwise operators (<<, >> etc.) as I would have done in a C++like program, but it does not seem to be working.
I suspect it has to do with data types. I may be using integers with bytes in the same operations, but I can't really tell which type each variable holds, since you don't really declare variables in Python, as far as I know (I'm very new to Python).
self.pot=self.myline[2]<<8
self.pot|=self.myline[3]

You can use the struct module to convert between integers and representation as bytes. In your case, to convert from a Python integer to two bytes and back, you'd use:
>>> import struct
>>> struct.pack('>H', 12345)
'09'
>>> struct.unpack('>H', '09')
(12345,)
The first argument to struct.pack and struct.unpack represent how you want you data to be formatted. Here, I ask for it to be in big-ending mode by using the > prefix (you can use < for little-endian, or = for native) and then I say there is a single unsigned short (16-bits integer) represented by the H.
Other possibilities are b for a signed byte, B for an unsigned byte, h for a signed short (16-bits), i for a signed 32-bits integer, I for an unsigned 32-bits integer. You can get the complete list by looking at the documentation of the struct module.

For example, using Big Endian encoding:
int.from_bytes(my_bytes, byteorder='big')

What you have seems basically like it should work, assuming the data stored in myline has the high byte first:
myline = [0, 1, 2, 3]
pot = myline[2]<<8 | myline[3]
print 'pot: {:d}, 0x{:04x}'.format(pot, pot) # outputs "pot: 515, 0x0203"
Otherwise, if it's low-byte first you'd need to do the opposite way:
myline = [0, 1, 2, 3]
pot = myline[3]<<8 | myline[2]
print 'pot: {:d}, 0x{:04x}'.format(pot, pot) # outputs "pot: 770, 0x0302"

This totally works:
long = 500
first = long & 0xff #244
second = long >> 8 #1
result = (second << 8) + first #500
If you are not sure of types in 'myline' please check Stack Overflow question How to determine the variable type in Python?.

To convert a byte or char to the number it represents, use ord(). Here's a simple round trip from an int to bytes and back:
>>> number = 3**9
>>> hibyte = chr(number / 256)
>>> lobyte = chr(number % 256)
>>> hibyte, lobyte
('L', '\xe3')
>>> print number == (ord(hibyte) << 8) + ord(lobyte)
True
If your myline variable is string or bytestring, you can use the formula in the last line above. If it somehow is a list of integers, then of course you don't need ord.

Typecasting in Python

I need to convert strings in Python to other types such as unsigned and signed 8, 16, 32, and 64 bit ints, doubles, floats, and strings.
How can I do this?

You can convert a string to a 32-bit signed integer with the int function:
string = "1234"
i = int(string) # i is a 32-bit integer
If the string does not represent an integer, you'll get a ValueError exception. Note, however, that if the string does represent an integer, but that integer does not fit into a 32-bit signed int, then you'll actually get an object of type long instead.
You can then convert it to other widths and signednesses with some simple math:
s8 = (i + 2**7) % 2**8 - 2**7 # convert to signed 8-bit
u8 = i % 2**8 # convert to unsigned 8-bit
s16 = (i + 2**15) % 2**16 - 2**15 # convert to signed 16-bit
u16 = i % 2**16 # convert to unsigned 16-bit
s32 = (i + 2**31) % 2**32 - 2**31 # convert to signed 32-bit
u32 = i % 2**32 # convert to unsigned 32-bit
s64 = (i + 2**63) % 2**64 - 2**63 # convert to signed 64-bit
u64 = i % 2**64 # convert to unsigned 64-bit
You can convert strings to floating point with the float function:
f = float("3.14159")
Python floats are what other languages refer to as double, i.e. they are 64-bits. There are no 32-bit floats in Python.

Python only has a single int type. To convert a string to an int, use int() like this:
>>> str = '123'
>>> num = int(str)
>>> num
123
Edit: Also to convert to float, use float() in the exact same way.

The following types -- for the most part -- don't exist in Python in the first place. In Python, strings are converted to ints, longs or floats, because that's all there is.
You're asking for conversions that aren't relevant to Python in the first place. Here's the list of types you asked for and their Python equivalent.
unsigned and signed int 8 bits, int
unsigned and signed int 16 bits, int
unsigned and signed int 32 bits, unsigned: long, signed int
unsigned and signed int 64 bits, long
double, float
float, float
string, this is what you had to begin with
I don't know what the following are, so I don't know a Python equivalent.
unsigned and signed 8 bit,
unsigned and signed 16 bit,
unsigned and signed 32 bit,
unsigned and signed 64 bit.
You already have all the conversions that matter: int(), long() and float().

I don't think this can necessarily be answered well without more information. As others have said, there are only int and long for integers in python -- the language doesn't adhere to the bit-width and signedness archetypes of lower-level programming languages.
If you're operating completely within python, then you're probably asking the wrong question. There's likely a better way to do what you need.
If you are interoperating with, for instance, C code, or over the network, then there are ways to do this, and it looks like the answer to your previous posting covered that avenue pretty handily.

I just now had a problem where I had a value passed as a 16 bit signed twos complement number from modbus.
I needed to convert this to a signed number.
I ended up writing this, which seems to work fine.
# convert a 32 bit (prob) integer as though it was
# a 16 bit 2's complement signed one
def conv_s16(i):
if (i & 0x8000):
s16 = -(((~i) & 0xFFFF) + 1)
else:
s16 = i
return s16