We Keep Coding

8-bit CRC calcuation in micropython

I'm trying to implement and use an 8-bit CRC in micropython, to be used together with an ADC (ADS1235 Texas Instruments).
I've tried for some time now to rewrite existing programs (mainly implemented in C) and code this function from the bottom up but to no avail.
The functions bellow are the closest I could find to what I'm seeking. The CRC I'm using has the polynomial 0x07.
Functions taken from PM 2Ring comment "1
def crc_16_CCITT(msg):
poly = 0x8408
crc = 0xffff
for byte in msg:
for _ in range(8):
if (byte ^ crc) & 1:
crc = (crc >> 1) ^ poly
else:
crc >>= 1
byte >>= 1
return crc ^ 0xffff
I've tried to use PM 2Ring table based implementation but that doesn't work either
def make_crc_table():
poly = 0x8408
table = []
for byte in range(256):
crc = 0
for bit in range(8):
if (byte ^ crc) & 1:
crc = (crc >> 1) ^ poly
else:
crc >>= 1
byte >>= 1
table.append(crc)
return table
table = make_crc_table()
def crc_16_fast(msg):
crc = 0xffff
for byte in msg:
crc = table[(byte ^ crc) & 0xff] ^ (crc >> 8)
return crc ^ 0xffff
My modifications to the first function can be seen bellow:
def crc_8_CCITT(msg):
poly = 0x07
crc = 0x00
for byte in msg:
for _ in range(8):
if (byte ^ crc) & 1:
crc = (crc >> 1) ^ poly
else:
crc >>= 1
byte >>= 1
return crc ^ 0x55

For the CRC it appears that you are trying to implement, which is a reflected CRC, you need to reflect the polynomial. You need poly = 0xe0.
The code can be simplified some. The for loop over the message can be:
for byte in msg:
crc ^= byte
for _ in range(8):
if crc & 1:
crc = (crc >> 1) ^ poly
else:
crc >>= 1
As to whether that is the CRC you actually need, I have no idea. Is the CRC you need reflected? Is the initial value zero? Where did the 0x55 come from? Do you have a specification of the CRC that you are not revealing to us? Do you have any example data for which you know the correct CRC value?
Update:
Based on the comment below, the OP needs to implement CRC-8/I-432-1. It is not a reflected CRC, so the shifts are up, not down, has polynomial 0x07, an initial value of zero, and a final exclusive or of 0x55. The implementation for that would be:
def crc8_itu(msg):
crc = 0
for byte in msg:
crc ^= byte
for _ in range(8):
crc = (crc << 1) ^ 7 if crc & 0x80 else crc << 1
crc &= 0xff
return crc ^ 0x55

Turns out that the CRC had other parameters than I expected.
Reading the C-code section in the TI Application Report
I see that the initial CRC-value is 0xFF and the final XOR-value is 0x00.
Using an online calculator with this setup,
I'm able to get the same CRC-result as the ADS1235.
Rewriting #Mark Adler's code:
def crc8_8_atm(msg):
crc = 0xFF
for byte in msg:
crc ^= byte
for _ in range(8):
crc = (crc << 1) ^ 0x07 if crc & 0x80 else crc << 1
crc &= 0xff
return crc ^ 0x00
Gives me the expected result.
Big thanks to #Mark Adler

Improve Python code/ STM CRC32 function

I have function to calculate STM crc32 of bytes array. The problem is that for 5120 byte array it takes 20ms which is definitely too long. There is an option to improve speed of this code, it can max takes about 5ms??
code :
def crc32_stm(bytes_arr):
length = len(bytes_arr)
crc = 0xffffffff
k = 0
while length >= 4:
v = ((bytes_arr[k] << 24) & 0xFF000000) | ((bytes_arr[k+1] << 16) & 0xFF0000) | \
((bytes_arr[k+2] << 8) & 0xFF00) | (bytes_arr[k+3] & 0xFF)
crc = ((crc << 8) & 0xffffffff) ^ custom_crc_table[0xFF & ((crc >> 24) ^ v)]
crc = ((crc << 8) & 0xffffffff) ^ custom_crc_table[0xFF & ((crc >> 24) ^ (v >> 8))]
crc = ((crc << 8) & 0xffffffff) ^ custom_crc_table[0xFF & ((crc >> 24) ^ (v >> 16))]
crc = ((crc << 8) & 0xffffffff) ^ custom_crc_table[0xFF & ((crc >> 24) ^ (v >> 24))]
k += 4
length -= 4
if length > 0:
v = 0
for i in range(length):
v |= (bytes_arr[k+i] << 24-i*8)
if length == 1:
v &= 0xFF000000
elif length == 2:
v &= 0xFFFF0000
elif length == 3:
v &= 0xFFFFFF00
crc = (( crc << 8 ) & 0xffffffff) ^ custom_crc_table[0xFF & ( (crc >> 24) ^ (v ) )];
crc = (( crc << 8 ) & 0xffffffff) ^ custom_crc_table[0xFF & ( (crc >> 24) ^ (v >> 8) )];
crc = (( crc << 8 ) & 0xffffffff) ^ custom_crc_table[0xFF & ( (crc >> 24) ^ (v >> 16) )];
crc = (( crc << 8 ) & 0xffffffff) ^ custom_crc_table[0xFF & ( (crc >> 24) ^ (v >> 24) )];
global stmCrc
stmCrc = crc
return crc

You can use crcmod to generate C code that you compile and then call from Python. That will be orders of magnitude faster than the Python code.

C function to Python (different results)

I am trying to port this snippet of code to python from C. The outputs are different even though it's the same code.
This is the C version of the code which works:
int main(void)
{
uint8_t pac[] = {0x033,0x55,0x22,0x65,0x76};
uint8_t len = 5;
uint8_t chan = 0x64;
btLeWhiten(pac, len, chan);
for(int i = 0;i<=len;i++)
{
printf("Whiten %02d \r\n",pac[i]);
}
while(1)
{
}
return 0;
}
void btLeWhiten(uint8_t* data, uint8_t len, uint8_t whitenCoeff)
{
uint8_t m;
while(len--){
for(m = 1; m; m <<= 1){
if(whitenCoeff & 0x80){
whitenCoeff ^= 0x11;
(*data) ^= m;
}
whitenCoeff <<= 1;
}
data++;
}
}
What I currently have in Python is:
def whiten(data, len, whitenCoeff):
idx = len
while(idx > 0):
m = 0x01
for i in range(0,8):
if(whitenCoeff & 0x80):
whitenCoeff ^= 0x11
data[len - idx -1 ] ^= m
whitenCoeff <<= 1
m <<= 0x01
idx = idx - 1
pac = [0x33,0x55,0x22,0x65,0x76]
len = 5
chan = 0x64
def main():
whiten(pac,5,chan)
print pac
if __name__=="__main__":
main()
The problem i see is that whitenCoeff always remain 8 bits in the C snippet but it gets larger than 8 bits in Python on each loop pass.

You've got a few more problems.
whitenCoeff <<= 1; is outside of the if block in your C code, but it's inside of the if block in your Python code.
data[len - idx -1 ] ^= m wasn't translated correctly, it works backwards from the C code.
This code produces the same output as your C code:
def whiten(data, whitenCoeff):
for index in range(len(data)):
for i in range(8):
if (whitenCoeff & 0x80):
whitenCoeff ^= 0x11
data[index] ^= (1 << i)
whitenCoeff = (whitenCoeff << 1) & 0xff
return data
if __name__=="__main__":
print whiten([0x33,0x55,0x22,0x65,0x76], 0x64)

In C you are writing data from 0 to len-1 but in Python you are writing data from -1 to len-2. Remove the -1 from this line:
data[len - idx -1 ] ^= m
like this
data[len - idx] ^= m
you also need to put this line outside the if:
whitenCoeff <<= 1

whitenCoeff <<= 1 in C becomes 0 after a while because it is a 8-bit data.
In python, there's no such limit, so you have to write:
whitenCoeff = (whitenCoeff<<1) & 0xFF
to mask higher bits out.
(don't forget to check vz0 remark on array boundary)
plus there was an indentation issue.
rewritten code which gives same result:
def whiten(data, whitenCoeff):
idx = len(data)
while(idx > 0):
m = 0x01
for i in range(0,8):
if(whitenCoeff & 0x80):
whitenCoeff ^= 0x11
data[-idx] ^= m
whitenCoeff = (whitenCoeff<<1) & 0xFF
m <<= 0x01
idx = idx - 1
pac = [0x33,0x55,0x22,0x65,0x76]
chan = 0x64
def main():
whiten(pac,chan)
print(pac)
if __name__=="__main__":
main()
Slightly off-topic: Note that the C version already has problems:
for(int i = 0;i<=len;i++)
should be
for(int i = 0;i<len;i++)

I solved it by anding the python code with 0xFF. That keeps the variable from increasing beyond 8 bits.

Your code in C does not appear to work as intended since it displays one more value than is available in pac. Correcting for this should cause 5 values to be displayed instead of 6 values. To copy the logic from C over to Python, the following was written in an attempt to duplicate the results:
#! /usr/bin/env python3
def main():
pac = bytearray(b'\x33\x55\x22\x65\x76')
chan = 0x64
bt_le_whiten(pac, chan)
print('\n'.join(map('Whiten {:02}'.format, pac)))
def bt_le_whiten(data, whiten_coeff):
for offset in range(len(data)):
m = 1
while m & 0xFF:
if whiten_coeff & 0x80:
whiten_coeff ^= 0x11
data[offset] ^= m
whiten_coeff <<= 1
whiten_coeff &= 0xFF
m <<= 1
if __name__ == '__main__':
main()
To simulate 8-bit unsigned integers, the snippet & 0xFF is used in several places to truncate numbers to the proper size. The bytearray data type is used to store pac since that appears to be the most appropriate storage method in this case. The code still needs documentation to properly understand it.

CRC32 hash of python string

Using an existing C example algorithm, I want to generate the correct CRC32 hash for a string in python. However, I am receiving incorrect results. I mask the result of every operation and attempt to copy the original algorithm's logic. The C code was provided by the same website which has a webpage string hash checking tool, so it is likely to be correct.
Below is a complete Python file including C code in its comments which it attempts to mimic. All pertinent information is in the file.
P_32 = 0xEDB88320
init = 0xffffffff
_ran = True
tab32 = []
def mask32(n):
return n & 0xffffffff
def mask8(n):
return n & 0x000000ff
def mask1(n):
return n & 0x00000001
def init32():
for i in range(256):
crc = mask32(i)
for j in range(8):
if (mask1(crc) == 1):
crc = mask32(mask32(crc >> 1) ^ P_32)
else:
crc = mask32(crc >> 1)
tab32.append(crc)
global _ran
_ran = False
def update32(crc, char):
char = mask8(char)
t = crc ^ char
crc = mask32(mask32(crc >> 8) ^ tab32[mask8(t)])
return crc
def run(string):
if _ran:
init32()
crc = init
for c in string:
crc = update32(crc, ord(c))
print(hex(crc)[2:].upper())
check0 = "The CRC32 of this string is 4A1C449B"
check1 = "123456789" # CBF43926
run(check0) # Produces B5E3BB64
run(check1) # Produces 340BC6D9
# Check CRC-32 on http://www.lammertbies.nl/comm/info/crc-calculation.html#intr
"""
/* http://www.lammertbies.nl/download/lib_crc.zip */
#define P_32 0xEDB88320L
static int crc_tab32_init = FALSE;
static unsigned long crc_tab32[256];
/*******************************************************************\
* *
* unsigned long update_crc_32( unsigned long crc, char c ); *
* *
* The function update_crc_32 calculates a new CRC-32 value *
* based on the previous value of the CRC and the next byte *
* of the data to be checked. *
* *
\*******************************************************************/
unsigned long update_crc_32( unsigned long crc, char c ) {
unsigned long tmp, long_c;
long_c = 0x000000ffL & (unsigned long) c;
if ( ! crc_tab32_init ) init_crc32_tab();
tmp = crc ^ long_c;
crc = (crc >> 8) ^ crc_tab32[ tmp & 0xff ];
return crc;
} /* update_crc_32 */
/*******************************************************************\
* *
* static void init_crc32_tab( void ); *
* *
* The function init_crc32_tab() is used to fill the array *
* for calculation of the CRC-32 with values. *
* *
\*******************************************************************/
static void init_crc32_tab( void ) {
int i, j;
unsigned long crc;
for (i=0; i<256; i++) {
crc = (unsigned long) i;
for (j=0; j<8; j++) {
if ( crc & 0x00000001L ) crc = ( crc >> 1 ) ^ P_32;
else crc = crc >> 1;
}
crc_tab32[i] = crc;
}
crc_tab32_init = TRUE;
} /* init_crc32_tab */
"""

There's just one thing that's wrong with the current implementation and the fix is actually just one line of code to the end of your run function which is:
crc = crc ^ init
Which if added to your run function look like this:
def run(string):
if _ran:
init32()
crc = init
for c in string:
crc = update32(crc, ord(c))
crc = crc ^ init
print(hex(crc)[2:].upper())
This will give you the correct results you are expecting.The reason that this is necessary is after you are done updating the CRC32, the finalization of it is XORing it with the 0xFFFFFFFF. Since you only had the init table and update functions and not the finalize, you were one step off from the actual crc.
Another C implimentation that is a little more straightforward is this one it's a little bit easier to see the whole process. The only thing slightly obsure is the init poly ~0x0 is the same (0xFFFFFFFF).

Convert this CRC32 algorithm to Python 3.3

I need to convert this CRC32 algorithm to python (using 3.3), but I am a python noob. I tried the built in binascii.crc32(), but the CRC was incorrect. Apparently, STMicro does the CRC32 a bit different. I found an algorithm that works, now I just need it to be in python.
//****************************************************************************
DWORD Crc32Fast(DWORD Crc, DWORD Data)
{
static const DWORD CrcTable[16] = { // Nibble lookup table for 0x04C11DB7 polynomial
0x00000000,0x04C11DB7,0x09823B6E,0x0D4326D9,0x130476DC,0x17C56B6B,0x1A864DB2,0x1E475005,
0x2608EDB8,0x22C9F00F,0x2F8AD6D6,0x2B4BCB61,0x350C9B64,0x31CD86D3,0x3C8EA00A,0x384FBDBD };
Crc = Crc ^ Data; // Apply all 32-bits
// Process 32-bits, 4 at a time, or 8 rounds
Crc = (Crc << 4) ^ CrcTable[Crc >> 28]; // Assumes 32-bit reg, masking index to 4-bits
Crc = (Crc << 4) ^ CrcTable[Crc >> 28]; // 0x04C11DB7 Polynomial used in STM32
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
Crc = (Crc << 4) ^ CrcTable[Crc >> 28];
return(Crc);
}
//****************************************************************************
DWORD Crc32FastBlock(DWORD Crc, DWORD Size, DWORD *Buffer) // 32-bit units
{
while(Size--)
Crc = Crc32Fast(Crc, *Buffer++);
return(Crc);
}

If my understanding is not mistaken, this should be the code that you want:
CRC_TABLE = (0x00000000, 0x04C11DB7, 0x09823B6E, 0x0D4326D9,
0x130476DC, 0x17C56B6B, 0x1A864DB2, 0x1E475005,
0x2608EDB8, 0x22C9F00F, 0x2F8AD6D6, 0x2B4BCB61,
0x350C9B64, 0x31CD86D3, 0x3C8EA00A, 0x384FBDBD)
def dword(value):
return value & 0xFFFFFFFF
def crc32_fast(crc, data):
crc, data = dword(crc), dword(data)
crc ^= data
for _ in range(8):
crc = dword(crc << 4) ^ CRC_TABLE[crc >> 28]
return crc
def crc32_fast_block(crc, buffer):
for data in buffer:
crc = crc32_fast(crc, data)
return crc
def crc32_fast_bytes(crc, bytes_data, byteorder='big'):
if len(bytes_data) & 3:
raise ValueError('bytes_data length must be multiple of four')
for index in range(0, len(bytes_data), 4):
data = int.from_bytes(bytes_data[index:index+4], byteorder)
crc = crc32_fast(crc, data)
return crc
The function crc32_fast_block expects an initial crc value and an iterable of numbers to run the algorithm on. crc32_fast_bytes is almost the same but expects a bytes value with a length being a multiple of four.