I would like to have a bit trick implementation of 64 bit popcount in Python. I tried to copy this code as follows:
def popcount_test(y):
y -= ((y >> 1) & 0x5555555555555555)
y = (y & 0x3333333333333333) + (y >> 2 & 0x3333333333333333)
return ((y + (y >> 4)) & 0xf0f0f0f0f0f0f0f) * 0x101010101010101 >> 56
Unfortunately it is not right. We can see this by doing popcount on the int 1234.
popcount_test(1234)
261
bin(1234).count('1')
5
What is the correct bit trick implement in Python?
Further tests can be carried out with:
import random
num = random.randint(0, 2**64-1)
print(popcount_test(num), bin(num).count('1'))
The problem is that the C version expects the multiply result to only produce the low-order 64 bits, but Python uses extended precision integers so you're getting the whole thing. You can fix it by masking the result to 8 bits after shifting it:
def popcount_test(y):
y -= ((y >> 1) & 0x5555555555555555)
y = (y & 0x3333333333333333) + (y >> 2 & 0x3333333333333333)
return (((y + (y >> 4)) & 0xf0f0f0f0f0f0f0f) * 0x101010101010101 >> 56) & 0xff
This produces the following:
>>> popcount_test(1234)
5
>>>
To make the solution obvious, I'm adding it here, but the credit goes to #TimPeters and #Heap-Overflow
def popcount_test(y):
y -= ((y >> 1) & 0x5555555555555555)
y = (y & 0x3333333333333333) + (y >> 2 & 0x3333333333333333)
return ((((y + (y >> 4)) & 0xf0f0f0f0f0f0f0f) * 0x101010101010101) >> 56) & 0xff
This is how Python does it in Modules/mathmodule.c:
static unsigned long
count_set_bits(unsigned long n)
{
unsigned long count = 0;
while (n != 0) {
++count;
n &= n - 1; /* clear least significant bit */
}
return count;
}
Related
Lot of solutions suggest using xor with right shift, as described here
https://www.geeksforgeeks.org/finding-the-parity-of-a-number-efficiently/
def findParity(x):
x = x ^ (x >> 16);
x = x ^ (x >> 8);
x = x ^ (x >> 4);
x = x ^ (x >> 2);
x = x ^ (x >> 1);
return x & 1;
But they assume 32 or 64 bit or some 2^n bits integer. In python integer could have any number of bits. For instance i = 7, has only 3 bits.
i = 7
print(len(bin(i)) - 2)
Any suggestions on how to calculate parity using xor and right shift for arbitrary number of bits ?
You can use a loop to dynamically change the length of the parity check:
def parity(num):
length = math.ceil(math.log2(math.ceil(math.log2(num)+1)))
for i in range(length-1, -1, -1):
print(2**i)
num^=(num >> (2**i))
return num&1
You will need to use log twice because you first have to find the length of the number then you need log that many operations.
I wrote this bitwise Karatsuba multiplication algorithm. It does not use strings or math.pow. It's just divide-and-conquer-recursion, bitwise operations and addition:
def karatsuba(x,y):
n = max(x.bit_length(), y.bit_length())
if n < 2:
return x&y
# split in O(1)
n = (n + 1) >> 1
b = x >> n;
a = x - (b << n);
d = y >> n;
c = y - (d << n);
ac = karatsuba(a, c);
bd = karatsuba(b, d);
abcd = karatsuba(a+b, c+d);
return ac + ((abcd - ac - bd) << n) + (bd << (n << 1));
print(karatsuba(23,24))
print(karatsuba(-29,31))
# 552
# 381
It works absolutly fine with positive numbers, but obviously -29*31 is not equal 381.
What's the easiest way to fix the problem?
My first idea was to make the number positive with (~(-29)+1) = 29, store wheather it was negative or not in a boolean and handle that boolean in my return statement, but is there a better (maybe bitwise) solution?
Thanks in advance
The issue is with your exit case, in particular x&y returns the wrong value for negative numbers:
-1 & 1 == 1 # Needs to return -1
So you can fix this with testing for it or or just returning:
if n < 2:
return x*y
E.g.:
In []:
print(karatsuba(-29,31))
Out[]:
-899
I'm trying to cast a big negative value inside a Cython class to an uint64_t type variable. But i keep getting this error:
OverflowError: can't convert negative value to unsigned long
cdef uint64_t temp2 = <uint64_t>(temp - bitReversal(current_pos))
The number i get from temp - bitReversal(current_pos) is -1152831344652320768 and if i hardcode it it works. For now i build a really ugly hack converting the negative number to the corresponding unsigned one but it is as expected really slow.
Thanks abarnert that worked.
This Line made it work:
cdef uint64_t temp2 = <uint64_t>(temp - <uint64_t>bitReversal(current_pos))
But it is really strange because both variables are of type uint64_t.
def bitReversal(uint64_t x):
x = (((x & 0xaaaaaaaaaaaaaaaa) >> 1) | ((x & 0x5555555555555555) << 1))
x = (((x & 0xcccccccccccccccc) >> 2) | ((x & 0x3333333333333333) << 2))
x = (((x & 0xf0f0f0f0f0f0f0f0) >> 4) | ((x & 0x0f0f0f0f0f0f0f0f) << 4))
x = (((x & 0xff00ff00ff00ff00) >> 8) | ((x & 0x00ff00ff00ff00ff) << 8))
x = (((x & 0xffff0000ffff0000) >> 16) | ((x & 0x0000ffff0000ffff) << 16))
cdef uint64_t result = <uint64_t>((x >> 32) | (x << 32))
return result
Is there a more efficient way of performing the following calculation? It works fine, but something tells me that x &= (1 << 8) - 1 ^ 1 << 3 can be written to avoid some calculations and increase speed.
def unset_mask(width, index):
return (1 << width) - 1 ^ 1 << index
x = 0b11111111
x &= unset_mask(8, 3)
assert x == 0b11110111
Actually, you don't need to state the width. Bigints behave the right way when you do this:
>>> bin(255 & ~(1 << 3))
'0b11110111'
>>> bin(65535 & ~(1 << 3))
'0b1111111111110111'
>>> bin(75557863725914323419135 & ~(1 << 3))
'0b1111111111111111111111111111111111111111111111111111111111111111111111110111'
It's because negative numbers have an "infinite" string of ones preceding them. So when you complement a positive number (which starts with an "infinte" string of zeros), you get a negative number (-(x + 1) to be exact). Just don't trust the bin representation of negative numbers; it doesn't reflect the actual bits in memory.
So you would rewrite unset_mask like so:
def unset_mask(index):
return ~(1 << index)
x = 0b11111111
x &= unset_mask(3)
print x == 0b11110111 # prints True
You can use this to clear a bit in x:
x &= ~(1 << index)
This will unset the bit:
x ^= 1 << 3 & x
In a function:
def unset_bit(x, n):
return 1 << n & x ^ x
Given a Python integer which is within the size of 4 bits, how does one transform it – with bitwise arithmetic instead of string processing – into an integer within the size of 4 bytes, for which each bit in the original corresponds to a byte which is the bit repeated 8 times?
For example: 0b1011 should become 0b11111111000000001111111111111111
With apologies to ncoghlan:
expanded_bits = [
0b00000000000000000000000000000000,
0b00000000000000000000000011111111,
0b00000000000000001111111100000000,
0b00000000000000001111111111111111,
0b00000000111111110000000000000000,
0b00000000111111110000000011111111,
0b00000000111111111111111100000000,
0b00000000111111111111111111111111,
0b11111111000000000000000000000000,
0b11111111000000000000000011111111,
0b11111111000000001111111100000000,
0b11111111000000001111111111111111,
0b11111111111111110000000000000000,
0b11111111111111110000000011111111,
0b11111111111111111111111100000000,
0b11111111111111111111111111111111,
]
Then just index this list with the nibble you want to transform:
>>> bin(expanded_bits[0b1011])
"0b11111111000000001111111111111111"
I'd just do a loop:
x = 0b1011
y = 0
for i in range(4):
if x & (1 << i):
y |= (255 << (i * 8))
print "%x" % y
The following recursive solution uses only addition, left/right shift operators and bitwise & operator with integers:
def xform_rec(n):
if n == 0:
return 0
else:
if 0 == n & 0b1:
return xform_rec(n >> 1) << 8
else:
return 0b11111111 + (xform_rec(n >> 1) << 8)
Or, as a one-liner:
def xform_rec(n):
return 0 if n == 0 else (0 if 0 == n & 0b1 else 0b11111111) + (xform_rec(n >> 1) << 8)
Examples:
>>> print bin(xform_rec(0b1011))
0b11111111000000001111111111111111
>>> print bin(xform_rec(0b0000))
0b0
>>> print bin(xform_rec(0b1111))
0b11111111111111111111111111111111)