I just learned Python (3.x) and I am stuck with HEX String conversion to Float. I have this HEX String values:
'0x22354942F31AFA42CE6A494311518A43082CAF437C6BD4C35F78FA433BF10F442A5222448D3D3544200749C438295C4468AF6E4406B4804450518A4423B0934450E99CC4'
And I want to turn it into float.
I have tried to use this code:
bs=bytes.fromhex(row[2:])
fmt = '<' + ('H' * (len(bs) // 2))
res=struct.unpack(fmt, bs)
and it gives me the result of 13602.0,16969.0,6899.0,17146.0,27342.0,17225.0,20753.0,17290.0,11272.0,17327.0,27516.0,50132.0,30815.0,17402.0,61755.0,17423.0,21034.0,17442.0,15757.0,17461.0,1824.0,50249.0,10552.0,17500.0,44904.0,17518.0,46086.0,17536.0,20816.0,17546.0,45091.0,17555.0,59728.0,50332.0
After checking it, I found out that the code that what I currently have is float in base 16, while I need it in base 32 (or maybe not because I am not sure what base/format), with expected float results as 50.3018875, 125.052635,201.4172,276.633331,350.344,424.839722,500.9404,575.7692,649.2838,724.961731,804.1113,880.644043,954.7407,1029.62573,106.541,1181.50427,1255.291 the values which I got from this Calculator Converter.
What should I change in the coding to get the expected results?
Thank you.
Let's break things down here, because you seem to be confused a bit with all of the juggling of representations. You have some hexadecimal string (that's base 16 encoding) of some binary data. That's your 0x22354942F31AFA42CE6A494311.... You correctly identified that you can convert this from its encoded form to python bytes with bytes.fromhex:
hex_encoded = '0x22354942F31AFA42CE6A494311518A43082CAF437C6BD4C35F78FA433BF10F442A5222448D3D3544200749C438295C4468AF6E4406B4804450518A4423B0934450E99CC4'
binary_data = bytes.fromhex(hex_encoded[2:]) # we do 2: to remove the leading '0x'
At this point, unless we know how binary_data was constructed we can't do anything. But we can take some guesses. You know the first few numbers are floating points: 50.3018875, 125.052635, 201.4172, .... Typically floats are encoded using the IEEE 754 standard. This provides 3 different encodings of a floating point number: binary16 (16 bits), float (32 bits), and double (64 bits). You can see these in the struct documentation, they are format codes 'e', 'f', and 'd', respectively. We can try each to see which of (if any) your binary data is encoded as. By trial and error, we discover your data was encoded as 32-bit floats, so you can decode them with:
FLOAT = 'f'
fmt = '<' + FLOAT * (len(binary_data) // struct.calcsize(FLOAT))
numbers = struct.unpack(fmt, binary_data)
print(numbers)
Why did what you tried not work? Well you used the format code 'H' which is for an unsigned short. This is an integer, which is why you were getting back numbers with no fractional part!
Related
Trying to a convert a binary list into a signed 16bit little endian integer
input_data = [['1100110111111011','1101111011111111','0010101000000011'],['1100111111111011','1101100111111111','0010110100000011']]
Desired Output =[[-1074, -34, 810],[-1703, -39, 813]]
This is what I've got so far. It's been adapted from: Hex string to signed int in Python 3.2?,
Conversion from HEX to SIGNED DEC in python
results = []
for i in input_data:
hex_convert = [hex(int(x,2)) for x in i]
convert = [int(y[4:6] + y[2:4], 16) for y in hex_convert]
results.append(convert)
print (results)
output: [[64461, 65502, 810], [64463, 65497, 813]]
This is works fine, but the above are unsigned integers. I need signed integers capable of handling negative values. I then tried a different approach:
results_2 = []
for i in input_data:
hex_convert = [hex(int(x,2)) for x in i]
to_bytes = [bytes(j, 'utf-8') for j in hex_convert]
split_bits = [int(k, 16) for k in to_bytes]
convert_2 = [int.from_bytes(b, byteorder = 'little', signed = True) for b in to_bytes]
results_2.append(convert_2)
print (results_2)
Output: [[108191910426672, 112589973780528, 56282882144304], [108191943981104, 112589235583024, 56282932475952]]
This result is even more wild than the first. I know my approach is wrong, and it doesn't help that i've never been able to get my head around binary conversion etc, but I feel i'm on the right path with:
(b, byteorder = 'little', signed = True)
but can't work out where i'm wrong. Any help explaining this concept would be greatly appreciated.
This result is even more wild than the first. I know my approach is wrong... but can't work out where i'm wrong.
The problem is in the conversion to bytes. Let's look at it a step at a time:
int(x, 2)
Fine; we treat the string as a base-2 representation of the integer value, and get that integer. Only problem is it's a) unsigned and b) big-endian.
hex(int(x,2))
What this does is create a string representation of the integer, in base 16, with a 0x prefix. Notably, there are two text characters per byte that we want. This is already heading is down the wrong path.
You might have thought of using hexadecimal because you've seen \xAB style escapes inside string representations. This is a completely different thing. The string '\xAB' contains one character. The string '0xAB' contains four.
From there, everything else is still nonsense. Converting to bytes with a text encoding just means that the text character 0 for example is replaced with the byte value 48 (since in UTF-8 it's encoded with a single byte with that value). For this data we get the same results with UTF-8 that we would by assuming plain ASCII (since UTF-8 is "ASCII transparent" and there are no non-ASCII characters in the text).
So how do we do it?
We want to convert the integer from the first step into the bytes used to represent it. Just as there is a .from_bytes class method allowing us to create an integer from underlying bytes, there is an instance method allowing us to get the bytes that would represent the integer.
So, we use .to_bytes, specifying the length, signedness and endianness that was assumed when we created the int from the binary string - that gives us bytes that correspond to that string. Then, we re-create the integer from those bytes, except now specifying the proper signedness and endianness. The reason that .to_bytes makes us specify a length is because the integer doesn't have a particular length - there are a minimum number of bytes required to represent it, but you could use as many more as you like. (This is especially important if you want to handle signed values, since it will do sign-extension automatically.)
Thus:
for i in input_data:
values = [int(x,2) for x in i]
as_bytes = [x.to_bytes(2, byteorder='big', signed=False) for x in values]
reinterpreted = [int.from_bytes(x, byteorder='little', signed=True) for x in as_bytes]
results_2.append(reinterpreted)
But let's improve the organization of the code a bit. I will first make a function to handle a single integer value, and then we can use comprehensions to process the list. In fact, we can use nested comprehensions for the nested list.
def as_signed_little(binary_str):
# This time, taking advantage of positional args and default values.
as_bytes = int(binary_str, 2).to_bytes(2, 'big')
return int.from_bytes(as_bytes, 'little', signed=True)
# And now we can do:
results_2 = [[as_signed_little(x) for x in i] for i in input_data]
I am faced with a problem in Python and I think I don't understand how signed numbers are handled in Python. My logic works in Java where everything is signed so need some help in Python.
I have some bytes that are coded in HEX and I need to decode them and interpret them to numbers. The protocol are defined.
Say the input may look like:
raw = '016402570389FFCF008F1205DB2206CA'
And I decode like this:
bin_bytes = binascii.a2b_hex(raw)
lsb = bin_bytes[5] & 0xff
msb = bin_bytes[6] << 8
aNumber = int(lsb | msb)
print(" X: " + str(aNumber / 4000.0))
After dividing by 4000.0, X can be in a range of -0.000025 to +0.25.
This logic works when X is in positive range. When X is expected
to be negative, I am getting back a positive number.
I think I am not handling "msb" correctly when it is a signed number.
How should I handlehandle negative signed number in
Python?
Any tips much appreciated.
You can use Python's struct module to convert the byte string to integers. It takes care of endianness and sign extension for you. I guess you are trying to interpret this 16-byte string as 8 2-byte signed integers, in big-endian byte order. The format string for this is '>8h. The > character tells Python to interpret the string as big endian, 8 means 8 of the following data type, and h means signed short integers.
import struct
nums = struct.unpack('>8h', bin_bytes)
Now nums is a tuple of integers that you can process further.
I'm not quite sure if your data is little or big endian. If it is little-endian, you can use < to indicate that in the struct.unpack format string.
I want to convert this hex string '8436d4ccd436d3333' to IEEE floating point. I've try to do this with struct.unpack but it's requires a string argument of length 4.
struct.unpack('>f', binascii.unhexlify('8436d999a436e0000'))
I'm using this website to verify if my conversion attempts are correct : https://gregstoll.dyndns.org/~gregstoll/floattohex/ but I can't find a way to do this.
Thanks for any help
At a guess, each hex string contains two single-precision floating-point values, not one, and the initial 8 is part of the whatever message protocol is being used, and not a part of either of those floats. With that guess, I get some plausible-looking numbers:
>>> struct.unpack('>ff', binascii.unhexlify('436d4ccd436d3333'))
(237.3000030517578, 237.1999969482422)
>>> struct.unpack('>ff', binascii.unhexlify('436d999a436e0000'))
(237.60000610351562, 238.0)
And to reinforce the plausibility, here's what I get by encoding the corresponding 1-digit-past-the-decimal-point values:
>>> binascii.hexlify(struct.pack('>ff', 237.3, 237.2))
b'436d4ccd436d3333'
>>> binascii.hexlify(struct.pack('>ff', 237.6, 238.0))
b'436d999a436e0000'
I am reading data from a binary file, it contains floating point data of which I want only first 6 digits after decimal point but its printing a pretty long string.
self.dataArray.append(struct.unpack("f", buf)[0])
I tried with this
self.dataArray.append(struct.unpack(".6f", buf)[0])
But it didn't worked.
Thanks in advance
a float isnt a string and a string isnt a float.
all a float is, is a number of bytes interpreted as both a whole number part and a fractional part
the_float = struct.unpack("f", buf)[0]
print "The Float String %0.6f"%(the_float)
How to convert floating point number to base-16 numbers, 8 hexadecimal digits per 32-bit FLP number in python?
eg : input = 1.2717441261e+20 output wanted : 3403244E
If you want the byte values of the IEEE-754 representation, the struct module can do this:
>>> import struct
>>> f = 1.2717441261e+20
>>> struct.pack('f', f)
'\xc9\x9c\xdc`'
This is a string version of the bytes, which can then be converted into a string representation of the hex values:
>>> struct.pack('f', f).encode('hex')
'c99cdc60'
And, if you want it as a hex integer, parse it as such:
>>> s = struct.pack('f', f).encode('hex')
>>> int(s, 16)
3382500448
To display the integer as hex:
>>> hex(int(s, 16))
'0xc99cdc60'
Note that this does not match the hex value in your question -- if your value is the correct one you want, please update the question to say how it is derived.
There are several possible ways to do so, but none of them leads to the result you wanted.
You can code this float value into its IEEE binary representation. This leads indeed to a 32 bit number (if you do it with single precision). But it leads to different results, no matter which endianness I suppose:
import struct
struct.pack("<f", 1.2717441261e+20).encode("hex")
# -> 'c99cdc60'
struct.pack(">f", 1.2717441261e+20).encode("hex")
# -> '60dc9cc9'
struct.unpack("<f", "3403244E".decode("hex"))
# -> (687918336.0,)
struct.unpack(">f", "3403244E".decode("hex"))
# -> (1.2213533295835077e-07,)
As the other one didn't fit result-wise, I'll take the other answers and include them here:
float.hex(1.2717441261e+20)
# -> '0x1.b939919e12808p+66'
Has nothing to do with 3403244E as well, so maybe you want to clarify what exactly you mean.
There are surely other ways to do this conversation, but unless you specify which method you want, no one is likely to be able to help you.
There is something wrong with your expected output :
import struct
input = 1.2717441261e+20
buf = struct.pack(">f", input)
print ''.join("%x" % ord(c) for c in struct.unpack(">4c", buf) )
Output :
60dc9cc9
Try float.hex(input) if input is already a float.
Try float.hex(input). This should convert a number into a string representing the number in base 16, and works with floats, unlike hex(). The string will begin with 0x however, and will contain 13 digits after the decimal point, so I can't help you with the 8 digits part.
Source: http://docs.python.org/2/library/stdtypes.html#float.hex