I wish to implement a 2d bit map class in Python. The class would have the following requirements:
Allow the creating of arbitrarily sized 2d bitmaps. i.e. to create an 8 x 8 bitmap (8 bytes), something like:
bitmap = Bitmap(8,8)
provide an API to access the bits in this 2d map as boolean or even integer values, i.e.:
if bitmap[1, 2] or bitmap.get(0, 1)
Able to retrieve the data as packed Binary data. Essentially it would be each row of the bit map concatenated and returned as Binary data. It may be padded to the nearest byte or something similar.
bitmap.data()
Be able to create new maps from the binary data retrieved:
new_bitmap = Bitmap(8, 8, bitmap.data())
I know Python is able to perform binary operations, but I'd like some suggestions as how best to use them to implement this class.
Bit-Packing numpy ( SciPY ) arrays does what you are looking for.
The example shows 4x3 bit (Boolean) array packed into 4 8-bit bytes. unpackbits unpacks uint8 arrays into a Boolean output array that you can use in computations.
>>> a = np.array([[[1,0,1],
... [0,1,0]],
... [[1,1,0],
... [0,0,1]]])
>>> b = np.packbits(a,axis=-1)
>>> b
array([[[160],[64]],[[192],[32]]], dtype=uint8)
If you need 1-bit pixel images, PIL is the place to look.
No need to create this yourself.
Use the very good Python Imaging Library (PIL)
Related
I have a Vector Grid Data from a Deformable Registration Grid Sequence whos type is binary.
i'm Trying to convert this data to a list of, i think, signed floating point value elements. but can find the function that allows me perform this operation. Let me show you a piece of the information.
b' dZ=\x00\x90\xb3=\x00\x18\x89\xbd \xe9}=\x00\xc0\xd6=\x00\xa0\xa5\xbd\xe0]\x93=\x00\x10\xfd=\x00\xa8\xc4\xbd\xc0\x8e\xa9=
...
\x95\xf9\xbb\xbc\x00\x80\x06=\xc6\x88(=\xa9\xcb\x82\xbc\x00#\xa6<A\xce\xc6<\xc5\xd5\x19\xbc\x00\x00\x0e<k\xba\x17<\x02\x07i\xbb'
i'll appreciate your help
Vector Grid Data consists of triplets of 4 byte floating point values. Try
from struct import unpack
data = b"..."
values = unpack(f"<{len(data) / 4}f", data)
I have a bytearray which I want to convert to a numpy array of int16 to perform FFT operations on. The bytearray is coming out of a UDP socket so first I convert two consecutive bytes into an int16 using struct.unpack, and then convert it a numpy array using np.asarray.
Current approach, however, is too slow. Original bytearray is of length 1e6 bytes, so each of the mentioned steps (struct.unpack and np.asarray) takes 20 ms and with a total of 40ms. This is a relatively long frame time for my applications so I need it a bit shortened.
Currently, I'm doing this:
temp1 = self.data_buffer[0:FRAME_LEN_B]
self.temp_list = np.asarray(struct.unpack('h' * (len(temp1) // 2), temp1))
You can try np.frombuffer. This can wrap any object supporting the buffer protocol, which bytearray explicitly does, into an array:
arr = np.frombuffer(self.data_buffer, dtype=np.int16, size=FRAME_LEN_B // 2)
You can manipulate the array however you want after that: slice, reshape, transpose, etc.
If your native byte order is opposite to what you have coming in from the network, you can swap the interpretation order without having to swap the data in-place:
dt = np.dtype(np.int16)
dt.newbyteorder('>')
arr = np.frombuffer(self.data_buffer, dtype=dt, size=FRAME_LEN_B // 2)
If the order is non-native, operations on the array may take longer, as the data will have to be swapped every time on the fly. You can therefore change the byte order in-place ahead of time if that is the case:
arr.byteswap(inplace=True)
This will overwrite the contents of the original packet. If you want to make a separate copy, just set inplace=False, which is the default.
I have several int16 streams in strings and I want them sum together (without overflow) and return it as an int16 string. Background is mixing several wave files into one stream.
decodeddata1 = numpy.fromstring(data, numpy.int16)
decodeddata2 = numpy.fromstring(data2, numpy.int16)
newdata = decodeddata1 + decodeddata2
return newdata.tostring()
Is there a way doing this with numpy or is there another library?
Processing each single value in python is too slow and results in stutter.
The most important thing is performance, since this code is used in a callback method feeding the audio.
#edit:
test input data:
a = np.int16([20000,20000,-20000,-20000])
b = np.int16([10000,20000,-10000,-20000])
print a + b --> [ 30000 -25536 -30000 25536]
but I want to keep the maximum levels:
[ 30000 40000 -30000 -40000]
The obvious consequence of mixing two signals together with a dynamic range of -32768<x<32767 is a resulting signal of with range of -65537<x<65536 - which requires 17 bits to represent it.
To avoid clipping, you will need to gain-scale the inputs - the obvious way is to divide the sum (or both of the inputs) by 2.
numpy looks as thought it should be quite fast for this - at least faster than python's builtin variable-size integer type. If the additional arithmetic is a performance concern, you should consider your choice of language.
I have an array of Cartesian coordinates
xy = np.array([[0,0], [2,3], [3,4], [2,5], [5,2]])
which I want to convert into an array of complex numbers representing the same:
c = np.array([0, 2+3j, 3+4j, 2+5j, 5+2j])
My current solution is this:
c = np.sum(xy * [1,1j], axis=1)
This works but seems crude to me, and probably there is a nicer version with some built-in magic using np.complex() or similar, but the only way I found to use this was
c = np.array(list(map(lambda c: np.complex(*c), xy)))
This doesn't look like an improvement.
Can anybody point me to a better solution, maybe using one of the many numpy functions I don't know by heart (is there a numpy.cartesian_to_complex() working on arrays I haven't found yet?), or maybe using some implicit conversion when applying a clever combination of operators?
Recognize that complex128 is just a pair of floats. You can then do this using a "view" which is free, after converting the dtype from int to float (which I'm guessing your real code might already do):
xy.astype(float).view(np.complex128)
The astype() converts the integers to floats, which requires construction of a new array, but once that's done the view() is "free" in terms of runtime.
The above gives you shape=(n,1); you can np.squeeze() it to remove the extra dimension. This is also just a view operation, so takes basically no time.
How about
c=xy[:,0]+1j*xy[:,1]
xy[:,0] will give an array of all elements in the 0th column of xy and xy[:,1] will give that of the 1st column.
Multiply xy[:,1] with 1j to make it imaginary and then add the result with xy[:,0].
I would like to create numpy.ndarray objects that hold complex integer values in them. NumPy does have complex support built-in, but for floating-point formats (float and double) only; I can create an ndarray with dtype='cfloat', for example, but there is no analogous dtype='cint16'. I would like to be able to create arrays that hold complex values represented using either 8- or 16-bit integers.
I found this mailing list post from 2007 where someone inquired about such support. The only workaround they recommended involved defining a new dtype that holds pairs of integers. This seems to represent each array element as a tuple of two values, but it's not clear what other work would need to be done in order to make the resulting data type work seamlessly with arithmetic functions.
I also considered another approach based on registration of user-defined types with NumPy. I don't have a problem with going to the C API to set this up if it will work well. However, the documentation for the type descriptor strucure seems to suggest that the type's kind field only supports signed/unsigned integer, floating-point, and complex floating-point numeric types. It's not clear that I would be able to get anywhere trying to define a complex integer type.
What are some recommendations for an approach that may work?
Whatever scheme I select, it must be amenable to wrapping of existing complex integer buffers without performing a copy. That is, I would like to be able to use PyArray_SimpleNewFromData() to expose the buffer to Python without having to make a copy of the buffer first. The buffer would be in interleaved real/imaginary format already, and would either be an array of int8_t or int16_t.
I also deal with lots of complex integer data, generally basebanded data.
I use
dtype = np.dtype([('re', np.int16), ('im', np.int16)])
It's not perfect, but it adequately describes the data. I use it for loading into memory without doubling the size of the data. It also has the advantage of being able to load and store transparently with HDF5.
DATATYPE H5T_COMPOUND {
H5T_STD_I16LE "re";
H5T_STD_I16LE "im";
}
Using it is straightforward and is just different.
x = np.zeros((3,3),dtype)
x[0,0]['re'] = 1
x[0,0]['im'] = 2
x
>> array([[(1, 2), (0, 0), (0, 0)],
>> [(0, 0), (0, 0), (0, 0)],
>> [(0, 0), (0, 0), (0, 0)]],
>> dtype=[('re', '<i2'), ('im', '<i2')])
To do math with it, I convert to a native complex float type. The obvious approach doesn't work, but it's also not that hard.
y = x.astype(np.complex64) # doesn't work, only gets the real part
y = x['re'] + 1.j*x['im'] # works, but slow and big
y = x.view(np.int16).astype(np.float32).view(np.complex64)
y
>> array([[ 1.+2.j, 0.+0.j, 0.+0.j],
>> [ 0.+0.j, 0.+0.j, 0.+0.j],
>> [ 0.+0.j, 0.+0.j, 0.+0.j]], dtype=complex64)
This last conversion approach was inspired by an answer to What's the fastest way to convert an interleaved NumPy integer array to complex64?
Consider using matrices of the form [[a,-b],[b,a]] as a stand-in for the complex numbers.
Ordinary multiplication and addition of matrices corresponds to addition an multiplication of complex numbers (this subring of the collection of 2x2 matrices is isomorphic to the complex numbers).
I think Python can handle integer matrix algebra.
Python, and hence NumPy, does support complex numbers. If you want complex integers, just use np.round or ignore the decimal part.
For example,
import numpy as np
# Create 100 complex numbers in a 1D array
a = 100*np.random.sample(100) + (100*np.random.sample(100)*1j)
# Reshape to a 2D array
np.round(a)
a.reshape(10, 10)
# Get the real and imaginary parts of a couple of x/y points as integers
print int(a[1:2].real)
print int(a[3:4].imag)