I am trying to convert a floating number to Real48 (Borland Pascal) in python. I have found this method written i C# (I think)
static byte[] DoubleToReal48(double d)
{
byte[] r = new byte[6];
long bits = BitConverter.DoubleToInt64Bits(d);
bool negative = ((bits >> 63) & 1) != 0;
long exponent = ((bits >> 52) & 0x7FF) - 1023;
long mantissa = bits & 0xFFFFFFFFFFFFFL;
long raw = (negative ? 1 : 0);
raw = (raw << 39) | (mantissa >> 13);
raw = (raw << 8) | ((exponent + 129) & 0xFF);
for (int k = 0; k < 6; k++)
{
r[k] = (byte)(raw & 0xFF);
raw >>= 8;
}
return r;
}
I don't know C#, so I'm having a hard time translating it to python. Can anyone help me? Or perhaps there is a better way?
I have found this but it seems to only convert Real48 to double, not the other way around as I need.
Let me try to translate the C# to python
1.- Convert the bits from a double number to a number (unsigned long long).
long bits = BitConverter.DoubleToInt64Bits(d);
translates to
struct.unpack("Q", struct.pack("d", python_double) )[0]
it uses the struct module from standard library.
2.- Extract each field from the double representation
bool negative = ((bits >> 63) & 1) != 0;
long exponent = ((bits >> 52) & 0x7FF) - 1023;
long mantissa = bits & 0xFFFFFFFFFFFFFL;
That can translated alsmot verbatim, since the same operators exist in python.
negative = ((bits >> 63) & 1) # Integer 1 for negative, 0 for positive
exponent = ((bits >> 52) & 0x7FF) - 1023
mantissa = bits & 0xFFFFFFFFFFFFF
3.- Pack again the number into 48 bits
long raw = (negative ? 1 : 0);
raw = (raw << 39) | (mantissa >> 13);
raw = (raw << 8) | ((exponent + 129) & 0xFF);
Same as number 2,
raw = negative
raw = (raw << 39) | (mantissa >> 13)
raw = (raw << 8) | ((exponent + 129) & 0xFF)
4.- Convert the number to a string of 6 bytes
for (int k = 0; k < 6; k++)
{
r[k] = (byte)(raw & 0xFF);
raw >>= 8;
}
Use struck as in step 1,
data = struck.pack('Q', raw)
data = data[:6] # Drop the 16 MSB
NOTE: I'm dropping the last 2 chars because struct uses the processors' endianness by default (x86 in my case), but that can be controled with the first argument.
TIP: If you want to convert a number to a binary representation you could use '{:064b}'.format(number). This will pad it with zeros to fill 64 chars.
Related
Lot of solutions suggest using xor with right shift, as described here
https://www.geeksforgeeks.org/finding-the-parity-of-a-number-efficiently/
def findParity(x):
x = x ^ (x >> 16);
x = x ^ (x >> 8);
x = x ^ (x >> 4);
x = x ^ (x >> 2);
x = x ^ (x >> 1);
return x & 1;
But they assume 32 or 64 bit or some 2^n bits integer. In python integer could have any number of bits. For instance i = 7, has only 3 bits.
i = 7
print(len(bin(i)) - 2)
Any suggestions on how to calculate parity using xor and right shift for arbitrary number of bits ?
You can use a loop to dynamically change the length of the parity check:
def parity(num):
length = math.ceil(math.log2(math.ceil(math.log2(num)+1)))
for i in range(length-1, -1, -1):
print(2**i)
num^=(num >> (2**i))
return num&1
You will need to use log twice because you first have to find the length of the number then you need log that many operations.
I am trying to port this snippet of code to python from C. The outputs are different even though it's the same code.
This is the C version of the code which works:
int main(void)
{
uint8_t pac[] = {0x033,0x55,0x22,0x65,0x76};
uint8_t len = 5;
uint8_t chan = 0x64;
btLeWhiten(pac, len, chan);
for(int i = 0;i<=len;i++)
{
printf("Whiten %02d \r\n",pac[i]);
}
while(1)
{
}
return 0;
}
void btLeWhiten(uint8_t* data, uint8_t len, uint8_t whitenCoeff)
{
uint8_t m;
while(len--){
for(m = 1; m; m <<= 1){
if(whitenCoeff & 0x80){
whitenCoeff ^= 0x11;
(*data) ^= m;
}
whitenCoeff <<= 1;
}
data++;
}
}
What I currently have in Python is:
def whiten(data, len, whitenCoeff):
idx = len
while(idx > 0):
m = 0x01
for i in range(0,8):
if(whitenCoeff & 0x80):
whitenCoeff ^= 0x11
data[len - idx -1 ] ^= m
whitenCoeff <<= 1
m <<= 0x01
idx = idx - 1
pac = [0x33,0x55,0x22,0x65,0x76]
len = 5
chan = 0x64
def main():
whiten(pac,5,chan)
print pac
if __name__=="__main__":
main()
The problem i see is that whitenCoeff always remain 8 bits in the C snippet but it gets larger than 8 bits in Python on each loop pass.
You've got a few more problems.
whitenCoeff <<= 1; is outside of the if block in your C code, but it's inside of the if block in your Python code.
data[len - idx -1 ] ^= m wasn't translated correctly, it works backwards from the C code.
This code produces the same output as your C code:
def whiten(data, whitenCoeff):
for index in range(len(data)):
for i in range(8):
if (whitenCoeff & 0x80):
whitenCoeff ^= 0x11
data[index] ^= (1 << i)
whitenCoeff = (whitenCoeff << 1) & 0xff
return data
if __name__=="__main__":
print whiten([0x33,0x55,0x22,0x65,0x76], 0x64)
In C you are writing data from 0 to len-1 but in Python you are writing data from -1 to len-2. Remove the -1 from this line:
data[len - idx -1 ] ^= m
like this
data[len - idx] ^= m
you also need to put this line outside the if:
whitenCoeff <<= 1
whitenCoeff <<= 1 in C becomes 0 after a while because it is a 8-bit data.
In python, there's no such limit, so you have to write:
whitenCoeff = (whitenCoeff<<1) & 0xFF
to mask higher bits out.
(don't forget to check vz0 remark on array boundary)
plus there was an indentation issue.
rewritten code which gives same result:
def whiten(data, whitenCoeff):
idx = len(data)
while(idx > 0):
m = 0x01
for i in range(0,8):
if(whitenCoeff & 0x80):
whitenCoeff ^= 0x11
data[-idx] ^= m
whitenCoeff = (whitenCoeff<<1) & 0xFF
m <<= 0x01
idx = idx - 1
pac = [0x33,0x55,0x22,0x65,0x76]
chan = 0x64
def main():
whiten(pac,chan)
print(pac)
if __name__=="__main__":
main()
Slightly off-topic: Note that the C version already has problems:
for(int i = 0;i<=len;i++)
should be
for(int i = 0;i<len;i++)
I solved it by anding the python code with 0xFF. That keeps the variable from increasing beyond 8 bits.
Your code in C does not appear to work as intended since it displays one more value than is available in pac. Correcting for this should cause 5 values to be displayed instead of 6 values. To copy the logic from C over to Python, the following was written in an attempt to duplicate the results:
#! /usr/bin/env python3
def main():
pac = bytearray(b'\x33\x55\x22\x65\x76')
chan = 0x64
bt_le_whiten(pac, chan)
print('\n'.join(map('Whiten {:02}'.format, pac)))
def bt_le_whiten(data, whiten_coeff):
for offset in range(len(data)):
m = 1
while m & 0xFF:
if whiten_coeff & 0x80:
whiten_coeff ^= 0x11
data[offset] ^= m
whiten_coeff <<= 1
whiten_coeff &= 0xFF
m <<= 1
if __name__ == '__main__':
main()
To simulate 8-bit unsigned integers, the snippet & 0xFF is used in several places to truncate numbers to the proper size. The bytearray data type is used to store pac since that appears to be the most appropriate storage method in this case. The code still needs documentation to properly understand it.
Using an existing C example algorithm, I want to generate the correct CRC32 hash for a string in python. However, I am receiving incorrect results. I mask the result of every operation and attempt to copy the original algorithm's logic. The C code was provided by the same website which has a webpage string hash checking tool, so it is likely to be correct.
Below is a complete Python file including C code in its comments which it attempts to mimic. All pertinent information is in the file.
P_32 = 0xEDB88320
init = 0xffffffff
_ran = True
tab32 = []
def mask32(n):
return n & 0xffffffff
def mask8(n):
return n & 0x000000ff
def mask1(n):
return n & 0x00000001
def init32():
for i in range(256):
crc = mask32(i)
for j in range(8):
if (mask1(crc) == 1):
crc = mask32(mask32(crc >> 1) ^ P_32)
else:
crc = mask32(crc >> 1)
tab32.append(crc)
global _ran
_ran = False
def update32(crc, char):
char = mask8(char)
t = crc ^ char
crc = mask32(mask32(crc >> 8) ^ tab32[mask8(t)])
return crc
def run(string):
if _ran:
init32()
crc = init
for c in string:
crc = update32(crc, ord(c))
print(hex(crc)[2:].upper())
check0 = "The CRC32 of this string is 4A1C449B"
check1 = "123456789" # CBF43926
run(check0) # Produces B5E3BB64
run(check1) # Produces 340BC6D9
# Check CRC-32 on http://www.lammertbies.nl/comm/info/crc-calculation.html#intr
"""
/* http://www.lammertbies.nl/download/lib_crc.zip */
#define P_32 0xEDB88320L
static int crc_tab32_init = FALSE;
static unsigned long crc_tab32[256];
/*******************************************************************\
* *
* unsigned long update_crc_32( unsigned long crc, char c ); *
* *
* The function update_crc_32 calculates a new CRC-32 value *
* based on the previous value of the CRC and the next byte *
* of the data to be checked. *
* *
\*******************************************************************/
unsigned long update_crc_32( unsigned long crc, char c ) {
unsigned long tmp, long_c;
long_c = 0x000000ffL & (unsigned long) c;
if ( ! crc_tab32_init ) init_crc32_tab();
tmp = crc ^ long_c;
crc = (crc >> 8) ^ crc_tab32[ tmp & 0xff ];
return crc;
} /* update_crc_32 */
/*******************************************************************\
* *
* static void init_crc32_tab( void ); *
* *
* The function init_crc32_tab() is used to fill the array *
* for calculation of the CRC-32 with values. *
* *
\*******************************************************************/
static void init_crc32_tab( void ) {
int i, j;
unsigned long crc;
for (i=0; i<256; i++) {
crc = (unsigned long) i;
for (j=0; j<8; j++) {
if ( crc & 0x00000001L ) crc = ( crc >> 1 ) ^ P_32;
else crc = crc >> 1;
}
crc_tab32[i] = crc;
}
crc_tab32_init = TRUE;
} /* init_crc32_tab */
"""
There's just one thing that's wrong with the current implementation and the fix is actually just one line of code to the end of your run function which is:
crc = crc ^ init
Which if added to your run function look like this:
def run(string):
if _ran:
init32()
crc = init
for c in string:
crc = update32(crc, ord(c))
crc = crc ^ init
print(hex(crc)[2:].upper())
This will give you the correct results you are expecting.The reason that this is necessary is after you are done updating the CRC32, the finalization of it is XORing it with the 0xFFFFFFFF. Since you only had the init table and update functions and not the finalize, you were one step off from the actual crc.
Another C implimentation that is a little more straightforward is this one it's a little bit easier to see the whole process. The only thing slightly obsure is the init poly ~0x0 is the same (0xFFFFFFFF).
I need to convert this CRC32 algorithm to python (using 3.3), but I am a python noob. I tried the built in binascii.crc32(), but the CRC was incorrect. Apparently, STMicro does the CRC32 a bit different. I found an algorithm that works, now I just need it to be in python.
//****************************************************************************
DWORD Crc32Fast(DWORD Crc, DWORD Data)
{
static const DWORD CrcTable[16] = { // Nibble lookup table for 0x04C11DB7 polynomial
0x00000000,0x04C11DB7,0x09823B6E,0x0D4326D9,0x130476DC,0x17C56B6B,0x1A864DB2,0x1E475005,
0x2608EDB8,0x22C9F00F,0x2F8AD6D6,0x2B4BCB61,0x350C9B64,0x31CD86D3,0x3C8EA00A,0x384FBDBD };
Crc = Crc ^ Data; // Apply all 32-bits
// Process 32-bits, 4 at a time, or 8 rounds
Crc = (Crc << 4) ^ CrcTable[Crc >> 28]; // Assumes 32-bit reg, masking index to 4-bits
Crc = (Crc << 4) ^ CrcTable[Crc >> 28]; // 0x04C11DB7 Polynomial used in STM32
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
return(Crc);
}
//****************************************************************************
DWORD Crc32FastBlock(DWORD Crc, DWORD Size, DWORD *Buffer) // 32-bit units
{
while(Size--)
Crc = Crc32Fast(Crc, *Buffer++);
return(Crc);
}
If my understanding is not mistaken, this should be the code that you want:
CRC_TABLE = (0x00000000, 0x04C11DB7, 0x09823B6E, 0x0D4326D9,
0x130476DC, 0x17C56B6B, 0x1A864DB2, 0x1E475005,
0x2608EDB8, 0x22C9F00F, 0x2F8AD6D6, 0x2B4BCB61,
0x350C9B64, 0x31CD86D3, 0x3C8EA00A, 0x384FBDBD)
def dword(value):
return value & 0xFFFFFFFF
def crc32_fast(crc, data):
crc, data = dword(crc), dword(data)
crc ^= data
for _ in range(8):
crc = dword(crc << 4) ^ CRC_TABLE[crc >> 28]
return crc
def crc32_fast_block(crc, buffer):
for data in buffer:
crc = crc32_fast(crc, data)
return crc
def crc32_fast_bytes(crc, bytes_data, byteorder='big'):
if len(bytes_data) & 3:
raise ValueError('bytes_data length must be multiple of four')
for index in range(0, len(bytes_data), 4):
data = int.from_bytes(bytes_data[index:index+4], byteorder)
crc = crc32_fast(crc, data)
return crc
The function crc32_fast_block expects an initial crc value and an iterable of numbers to run the algorithm on. crc32_fast_bytes is almost the same but expects a bytes value with a length being a multiple of four.
I need script that will calculate crc32 with the same output for both Python and C.
I'm using right now zlib.crc32, but for C there is no such library and we are writing it on our own basing on Wikipedia. But it doesn't return the same value.
This is our code of C script (copied from wikipedia, based on RFC):
unsigned int crc32( unsigned char *message, unsigned int n )
{
//int i, crc;
unsigned int crc;
unsigned int i;
unsigned int byte, c;
const unsigned int g0 = 0xEDB88320, g1 = g0>>1,
g2 = g0>>2, g3 = g0>>3, g4 = g0>>4, g5 = g0>>5,
g6 = (g0>>6)^g0, g7 = ((g0>>6)^g0)>>1;
i = 0;
crc = 0xFFFFFFFF;
//while ((byte = message[i]) != 0)
while( i != n)
{
byte = message[i]; // Get next byte.
// byte = FrmReadByte( i ); // Get next byte.
crc = crc ^ byte;
c = ((crc<<31>>31) & g7) ^ ((crc<<30>>31) & g6) ^
((crc<<29>>31) & g5) ^ ((crc<<28>>31) & g4) ^
((crc<<27>>31) & g3) ^ ((crc<<26>>31) & g2) ^
((crc<<25>>31) & g1) ^ ((crc<<24>>31) & g0);
crc = ((unsigned)crc >> 8) ^ c;
i = i + 1;
}
return ~crc;
}
EDIT:
We've got only 4KB of ram memory, program itself doesn't live there. Taking 1KB of memory by crc32 script is probably too much and wouldn't fit there.
Thanks for pointing out that ZLIB library exists for C as well.
I'm using right now zlib.crc32, but for C there is no such library
Um, yes, there is. It's called zlib. zlib is written in C, and it's what Python is using! Hence the name of the class.
You can use the crc32() function in zlib. That implementation is a fair bit faster than others you might find. Read zlib.h for the interface information.
You can compile zlib yourself, or it may already be installed on your system.
Update:
I now see your comment (which should be edited into the question since it is critical to getting the right answer) that you have extremely limited memory. Then you can use this:
static uint32_t crc32(uint32_t crc, unsigned char *buf, size_t len)
{
int k;
crc = ~crc;
while (len--) {
crc ^= *buf++;
for (k = 0; k < 8; k++)
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
}
return ~crc;
}
The crc is initially set to zero.
The use of ~ will give the correct result, since the uint32_t type in stdint.h is assured to be 32 bits.
If you can afford a little more code space, then unrolling the loop will likely speed it up (if the compiler doesn't already do this):
static uint32_t crc32(uint32_t crc, unsigned char *buf, size_t len)
{
crc = ~crc;
while (len--) {
crc ^= *buf++;
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
}
return ~crc;
}
You said that you only have 4 KBytes of "memory". Is that just working memory for the program, or does the program have to live there as well? If you have more space in flash for example for the code, then the table can be precomputed and stored with the code. A table-driven CRC will be much faster. The zlib code provides table-driven CRCs that do one byte at a time and four-bytes at a time, requiring respectively a 1Kbyte or 4Kbyte table.
Update 2:
Since you answered in a comment that the 4KBytes are just working memory, then you should use a table-driven CRC. You can simply use the crc32() function in zlib's crc32.c and the table in crc32.h with BYFOUR undefined.
C:
UInt32
crc32(UInt32 crc, UInt8 *p, SInt len)
{
crc = ~crc;
while (--len >= 0) {
crc = crc ^ *p++;
for (SInt i = 8; --i >= 0;) {
crc = (crc >> 1) ^ (0xedb88320 & -(crc & 1));
}
}
return ~crc;
}
void
crc_unitTest(void)
{
UInt8 b1[] = { 0, 0, 0, 0, 0, 0, 0, 0 };
UInt8 b2[] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
UInt8 b3[] = { 0xff, 0, 0xff, 0, 0xff, 0, 0xff, 0 };
assert(crc32(0, b1, 8) == 0x6522df69);
assert(crc32(0, b2, 10) == 0x456cd746);
assert(crc32(0, b3, 8) == 0xea8c89c0);
}
Python:
def crc32(crc, p, len):
crc = 0xffffffff & ~crc
for i in range(len):
crc = crc ^ p[i]
for j in range(8):
crc = (crc >> 1) ^ (0xedb88320 & -(crc & 1))
return 0xffffffff & ~crc
def unitTest():
b1 = [ 0, 0, 0, 0, 0, 0, 0, 0 ]
b2 = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ]
b3 = [ 0xff, 0, 0xff, 0, 0xff, 0, 0xff, 0 ]
assert(crc32(0, b1, 8) == 0x6522df69)
assert(crc32(0, b2, 10) == 0x456cd746)
assert(crc32(0, b3, 8) == 0xea8c89c0)
As you need a C implementation that doesn't use a lookup table (which most implementations do) with a matching Python equivalent you could use ZIP's CRC32 as suggested by Mark Ransom ( binascii.crc32 ) and the matching, tableless implementation I borrowed here
/* Calculating ZIP CRC-32 in 'C'
=============================
Reference model for the translated code */
#define poly 0xEDB88320
/* Some compilers need
#define poly 0xEDB88320uL
*/
/* On entry, addr=>start of data
num = length of data
crc = incoming CRC */
int crc32(char *addr, int num, int crc)
{
int i;
for (; num>0; num--) /* Step through bytes in memory */
{
crc = crc ^ *addr++; /* Fetch byte from memory, XOR into CRC */
for (i=0; i<8; i++) /* Prepare to rotate 8 bits */
{
if (crc & 1) /* b0 is set... */
crc = (crc >> 1) ^ poly; /* rotate and XOR with ZIP polynomic */
else /* b0 is clear... */
crc >>= 1; /* just rotate */
/* Some compilers need:
crc &= 0xFFFFFFFF;
*/
} /* Loop for 8 bits */
} /* Loop until num=0 */
return(crc); /* Return updated CRC */
}
EDIT: as several people pointed out there are issues with the above code, the one below matches Wikipedia's (see http://ideone.com/pWLVSo ) and Python's ( http://ideone.com/SvYuyE - 1277644989==0x4c2750bd). This code comes from this page where other implementations and possible improvements over the basic version I copied
const uint32_t Polynomial = 0xEDB88320;
uint32_t crc32_bitwise(const void* data, size_t length, uint32_t previousCrc32 = 0) {
uint32_t crc = ~previousCrc32; // same as previousCrc32 ^ 0xFFFFFFFF
unsigned char* current = (unsigned char*) data;
while (length--) {
crc ^= *current++;
for (unsigned int j = 0; j < 8; j++) {
if (crc & 1)
crc = (crc >> 1) ^ Polynomial;
else
crc = crc >> 1;
}
}
return ~crc; // same as crc ^ 0xFFFFFFFF
}