I am doing some calculations and I was wondering how you can invert a byte in python.
For example:
0b11111111 should be flipped to 0b00000000.
I have tried turning into a string and flipping it then but I cant somehow turn it back into a Integer.
Also, I want to add that the ~ does not work, as it simply just makes the same number but negative
bin(0b11010111 ^ 0b11111111) // bit subtraction
# Result
# bin(0b11010111 ^ 0b11111111)
#'0b101000'
# for each bit subtraction with 1 occur
To explain why ~ "dit not work": you want to invert all bits in a byte (8-bits), but the 0b.... value is not a byte, but an int which has more bits than just 8.
~var inverts all bits in an integer. You can clear all bits except the lowest 8 with a mask to get the result you expect:
MASK8 = 0b11111111 # or 255 decimal
v = 0b11101110
inv = ~v & MASK8 # 0b10001
Related
im a beginner with python and want to make a program that converts a hex RGB value to a 15 bit RGB one (5 bits for every color) i heard that it can be done by bitshifts but i don´t get how i also didn´t find anything helpful on the internet can someone please help me
If you're doing this manually (say, for homework), think of the problem like this:
If you have a six character hex representation of a color ("7FD87F" for instance), it's made up of the RGB components R: 7F, G: D8, B: 7F.
Two hexadecimal digits can encode 256 different states (16^2 = 256), so each of these components is 8 bits in size (256 = 2^8).
You want to transform your value into a color space where each component is represented in 5 bits. The way to do this is to throw away the three least significant bits from each of the components. For example:
0b10101010 => 0b10101
As you correctly mentioned, you would do this via bit-shifting. You'll then need to recombine the components. As a hint, here's how I would recombine the original 8-bit components into a single 24-bit representation:
(R << 16) + (G << 8) + (B << 0)
# or just B since a shift by zero is equivalent to
# multiplying by 2^0 is
# multiplying by 1 is
# the multiplicative identity
So the sketch of your algorithm, assuming you are starting with a hex string and not an integer:
Split the hex string into individual color components
Convert the string representations to numeric representations
Bit shift the components
Recombine the 5-bit components into a 15-bit representation
Additionally, steps 1 and 2 are interchangible with some bit-masking and a little more bit-shifting.
Say I have a number, 18573628, where each digit represents some kind of flag, and I want to check if the value of the fourth flag is set to 7 or not (which it is).
I do not want to use indexing. I want to in some way and with a flag mask, such as this:
00070000
I would normally use np.logical_and() or something like that, but that will consider any positive value to be True. How can I and while considering the value of a digit? For example, preforming the operation with
flags = 18573628
and
mask = 00070000
would yield 00010000
though trying a different mask, such as
mask = 00040000
would yield 00000000
What you can do is
if (x // 10**n % 10) == y:
...
to check if the n-th digit of x (counting from right) is equal to y
You have to use divide and modulo for a decimal mask:
flags = 18573628
mask = 10000
if (flags / mask) % 10 == 7:
do_something
You can convert the input number into an array of digit numbers and then simply indexing into that array with that specific index or indices would give us those digit(s). For doing that conversion, we can use np.fromstring, like so -
In [87]: nums = np.fromstring(str(18573628),dtype=np.uint8)-48
In [88]: nums
Out[88]: array([1, 8, 5, 7, 3, 6, 2, 8], dtype=uint8)
In [89]: nums[3] == 7
Out[89]: True
Say I have a number, 18573628, where each digit represents some kind of flag, and I want to check if the value of the fourth flag is set to 7
Firstly, bitwise operations like & are bit-wise, which is to say they operate on base-2 digits. They don't operate naturally on digits of any other base, although bases which are themselves powers of 2 work out ok.
To stick with bit-wise operations
You need to know how many values each flag can take, to figure out how many bits each flag needs to encode.
If you want to allow each flag the values zero to nine, you need four bits. However, in this scheme, your number won't behave like a normal integer (storing a base-10 digit in each 4-bit group is called Binary Coded Decimal).
The reason it won't behave like a normal integer is that flag values 1,2,3 will be stored as 1 * 16**2 + 2*16 + 3 instead of the 1 * 10**2 + 2*10 + 3 you'd normally expect. So you'd need to write some code to support this use. However, extracting flag n (counting from zero at the right) just becomes
def bcdFlagValue(bcd, flagnum):
if flagnum == 0:
return bcd & 0x0F;
return 0x0F & (bcd >> ((flagnum-1) * 4))
If you actually need a different range of values for each flag, you need to choose the correct number of bits, and adjust the shift and mask values appropriately.
In either case, you'll need a helper function if you want to print your flags as the base-10 number you showed.
To use normal base 10 numbers
You need to use division and modulo (as 6502 showed), because base-10 numbers don't fit evenly into base-2 bits, so simple bit operations don't work
Note
The BCD approach saves space at the cost of complexity, effort and some speed - from subsequent comments, it's probably simpler to just use the string of digit characters directly unless you really need to save 4 bits per digit.
if flags and mask are hexadecimal values, you can do:
flags = int("18573628", 16)
mask = int("00070000", 16)
result = flags & mask
print(hex(result))
=> '0x70000'
Without dealing with the particulars of your case (the SDSS data, which should be documented in the product specification), let's look at some options.
First, you need to to know if it is to be read in big-endian or little-endian order (is the first bit to the right or to the left). Then you need to know the size of each flag. For a series of yes-no parameters, it could simply be 1 bit (0 or 1). For up to four options, it could be two bits (00, 01, 10, 11), etc. It is also possible that some combinations are reserved for future expansion, don't currently have meaning, and should not be expected to occur in the data. I've also seen instances where the flag size varies, so first n bits mean refer to parameter x, next n bits refer to parameter y, etc.
There is a good explanation of the concept as part of Landsat-8 satellite imagery:
http://landsat.usgs.gov/qualityband.php
To read the values, you convert the base 10 integer to binary, and traverse it in the specified chunks, converting back to int to obtain the parameter values according to your product specification.
I have a 64-bit hex number inputting into my script
0x0000040800000000. I want to take this number and extract bits 39:32.
How is this possible? I have been parsing individual parts of a string and have ended up in a mess.
I was initially converting it into binary and parsing out sections of the string from
command_register = "".join(["{0:04b}".format(int(c,16)) for c in str(command_register)])
You simply need to first convert your hex string into an integer and then use normal maths to extract the bits.
Bit numbering is usually done from the least significant bit, i.e. the furthest right when displayed in binary is bit 0. So to extract bits 39:32 (8 consecutive bits), you would simply need a mask of 0xFF00000000. Simply AND your number and shift the result 32 bits to the right.
Using your hex value and extracting bits 39 to 32 would give you a value of 0x08. The following script shows you how:
hex_string = "0x0000040800000000"
number = int(hex_string, 16) # Convert to an integer
mask_39_to_32 = 0xFF00000000 # Suitable mask to extract the bits with
print(f"As hex: 0x{number:X}")
print()
print("Bits 39-32: xxxxxxxx")
print(f" As binary: {bin(number)[2:]:0>64s}")
print(f" Mask: {bin(mask_39_to_32)[2:]:0>64s}")
print(f"AND result: {bin(number & mask_39_to_32)[2:]:0>64s}")
print(f" Shifted: {bin((number & mask_39_to_32) >> 32)[2:]:0>64s}")
print(f" As an int: {(number & mask_39_to_32) >> 32}")
Which displays the following output:
As hex: 0x40800000000
Bits 39-32: xxxxxxxx
As binary: 0000000000000000000001000000100000000000000000000000000000000000
Mask: 0000000000000000000000001111111100000000000000000000000000000000
AND result: 0000000000000000000000000000100000000000000000000000000000000000
Shifted: 0000000000000000000000000000000000000000000000000000000000001000
As an int: 8
The mask needed for 47 to 40 would be:
Bits 47-40: xxxxxxxx
As binary: 0000000000000000111111110000000000000000000000000000000000000000
As hex: 0xFF0000000000
The use of hexadecimal simply makes it less verbose, and clearer once you get used to it. Groups of 8 bits for masks always end up as 'FF'.
The Wikipedia article on bitwise operations should help you to understand the process.
In order to test building an Xor operation with more basic building blocks (using Nand, Or, and And in my case) I need to be able to do a Not operation. The built-in not only seems to do this with single bits. If I do:
x = 0b1100
x = not x
I should get 0b0011 but instead I just get 0b0. What am I doing wrong? Or is Python just missing this basic functionality?
I know that Python has a built-in Xor function but I've been using Python to test things for an HDL project/course where I need to build an Xor gate. I wanted to test this in Python but I can't without an equivalent to a Not gate.
The problem with using ~ in Python, is that it works with signed integers. This is also the only way that really makes sense unless you limit yourself to a particular number of bits. It will work ok with bitwise math, but it can make it hard to interpret the intermediate results.
For 4 bit logic, you should just subtract from 0b1111
0b1111 - 0b1100 # == 0b0011
For 8 bit logic, subtract from 0b11111111 etc.
The general form is
def bit_not(n, numbits=8):
return (1 << numbits) - 1 - n
Another way to achieve this, is to assign a mask like this (should be all 1's):
mask = 0b1111
Then xor it with your number like this:
number = 0b1100
mask = 0b1111
print(bin(number ^ mask))
You can refer the xor truth table to know why it works.
Python bitwise ~ operator invert all bits of integer but we can't see native result because all integers in Python has signed representation.
Indirectly we can examine that:
>>> a = 65
>>> a ^ ~a
-1
Or the same:
>>> a + ~a
-1
Ther result -1 means all bits are set. But the minus sign ahead don't allow us to directly examine this fact:
>>> bin(-1)
'-0b1'
The solution is simple: we must use unsigned integers.
First way is to import numpy or ctypes modules wich both support unsigned integers. But numpy more simplest using than ctypes (at least for me):
import numpy as np
a = np.uint8(0b1100)
y = ~x
Check result:
>>> bin(x)
'0b1100'
>>> bin(y)
'0b11110011'
And finally check:
>>> x + y
255
Unsigned integer '255' for 8-bits integers (bytes) mean the same as '-1' becouse has all bits set to 1. Make sure:
>>> np.uint8(-1)
255
And another simplest solution, not quite right, but if you want to include additional modules, you can invert all bits with XOR operation, where second argument has all bits are set to 1:
a = 0b1100
b = a ^ 0xFF
This operation will also drop most significant bit of signed integer and we can see result like this:
>>> print('{:>08b}'.format(a))
00001100
>>> print('{:>08b}'.format(b))
11110011
Finally solution contains one more operation and therefore is not optimal:
>>> b = ~a & 0xFF
>>> print('{:>08b}'.format(b))
11110011
Try this, it's called the bitwise complement operator:
~0b1100
The answers here collectively have great nuggets in each one, but all do not scale well with depending on edge cases.
Rather than fix upon an 8-bit mask or requiring the programmer to change how many bits are in the mask, simply create a mask based on input via bit_length():
def bit_not(num):
return num ^ ((1 << num.bit_length()) - 1)
string of binary can be used to preserve the left 0s, since we know that:
bin(0b000101) # '0b101'
bin(0b101) # '0b101'
This function will return string format of the NOT of input number
def not_bitwise(n):
'''
n: input string of binary number (positive or negative)
return: binary number (string format)
'''
head, tail = n.split('b')
not_bin = head+'b'+tail.replace('0','a').replace('1','0').replace('a','1')
return not_bin
Example:
In[266]: not_bitwise('0b0001101')
Out[266]: '0b1110010'
In[267]: int(not_bitwise('0b0001101'), 2)
Out[267]: 114
In[268]: not_bitwise('-0b1010101')
Out[268]: '-0b0101010'
In[269]: int(not_bitwise('-0b1010101'), 2)
Out[269]: -42
The general form given by John La Rooy, can be simplified in this way (python == 2.7 and >=3.1):
def bit_not(n):
return (1 << n.bit_length()) - 1 - n
I use python 2.6
>>> hex(-199703103)
'-0xbe73a3f'
>>> hex(199703103)
'0xbe73a3f'
Positive and negative value are the same?
When I use calc, the value is FFFFFFFFF418C5C1.
Python's integers can grow arbitrarily large. In order to compute the raw two's-complement the way you want it, you would need to specify the desired bit width. Your example shows -199703103 in 64-bit two's complement, but it just as well could have been 32-bit or 128-bit, resulting in a different number of 0xf's at the start.
hex() doesn't do that. I suggest the following as an alternative:
def tohex(val, nbits):
return hex((val + (1 << nbits)) % (1 << nbits))
print tohex(-199703103, 64)
print tohex(199703103, 64)
This prints out:
0xfffffffff418c5c1L
0xbe73a3fL
Because Python integers are arbitrarily large, you have to mask the values to limit conversion to the number of bits you want for your 2s complement representation.
>>> hex(-199703103 & (2**32-1)) # 32-bit
'0xf418c5c1L'
>>> hex(-199703103 & (2**64-1)) # 64-bit
'0xfffffffff418c5c1L'
Python displays the simple case of hex(-199703103) as a negative hex value (-0xbe73a3f) because the 2s complement representation would have an infinite number of Fs in front of it for an arbitrary precision number. The mask value (2**32-1 == 0xFFFFFFFF) limits this:
FFF...FFFFFFFFFFFFFFFFFFFFFFFFF418c5c1
& FFFFFFFF
--------------------------------------
F418c5c1
Adding to Marks answer, if you want a different output format, use
'{:X}'.format(-199703103 & (2**32-1))
For those who want leading zeros for positive numbers, try this:
val = 42
nbits = 16
'{:04X}'.format(val & ((1 << nbits)-1))
Thanks #tm1, for the inspiration!