I have a bunch of images (300 images of 400 X 400 pixels) with filenames like:
001.bmp
002.bmp
003.bmp
...
First, I tried reading one of them, e.g. using imread I get a (400L, 400L, 3L) matrix, the problem is the 3L (I think is RBG format), so the question here is: how can I read them and get a (400L, 400L, 1L) matrix that I need to proccess them?
Second, I tried to read the 300 images using a loop like the following:
data = np.zeros((400,400,300))
for i in range(300):
data[:,:,i] = imread('{0}.bmp'.format(i))
but it doesn't work, very probably my code is wrong. Actually doing this, I want to concatenate each (300) image data (400 X 400) into a matrix of (400 X 400 X 300).
When trying to use:
data[:,:,i] = imread('{0}.bmp'.format(i))
search for '1.bmp' and not '001.bmp', but due to the list go from 000 to 299, I got a problem with that and I cant write '00{0}.bmp'.format(i) to complete the filename, because for two- and three digits numbers I got '0012.bmp' or '00123.bmp'
Well, after hours, I got to do this
arrays = []
for number in range(0, 299):
numstr = str(number).zfill(3)
fname = numstr + '.bmp'
a = imread(fname, flatten=1)
arrays.append(a)
data = np.array(arrays)
This code its work well. Thankyou, for give me clues!
First, you are right that the last dimension are the color channels. I assume you want a grayscale image, which you can get with:
data = imread(fname, flatten=1)
That comes from the imread documentation here.
Second, your issue with the loop can be due to a couple of things. First, I don't see indentation in the code in your post, so make sure that is there on the loop body in the code that you are actually trying to run. Second, the code has a ".txt" extension. Are you sure you don't actually want ".bmp"?
Related
I am trying to read data from this png image, and then place the image length at the start of the data, and pad it a given number of spaces defined by my header variable. However, once I do that, the image length increases drastically for a reason beyond my knowledge. Please can someone inform me of what is happening? I must be missing something since I am still fairly new to this field.
HEADER = 10
PATH = os.path.abspath("penguin.png")
print(PATH)
with open(PATH,"rb") as f:
imgbin = f.read()
print(len(imgbin))
imgbin = f"{len(imgbin):<{HEADER}}"+str(imgbin)
print(len(imgbin))
when I first print the length of the data, I get a length of 163287, and on the second print, I get a length of 463797
This is because you are changing the data from binary string to a string when you load the image to when you pass it through str:
len(imgbin), len(str(imgbin))
>>> (189255, 545639)
(note I use a different image so the numbers are different). You can solve this issue by adding a binary string to the start like so:
with open(PATH,"rb") as f:
imgbin = f.read()
imgbin = f"{len(imgbin):<{HEADER}}".encode('utf-8')+imgbin
print(len(imgbin))
>>> 189245
>>> 189255
You can find out more about binary strings here.
For reference it is worth noting that png images are uint-8 in type (i.e. 0-255). It is possible to manipulate them as binary strings because they can be utf-8 (i.e. the same size). However, it might be worth using something like numpy where you have uint-8 as a data type so as to avoid this.
I'm looping through a very large dataframe(11361 x 22679) and converting the values of each row to a pixel image using pyplot. So in the end I should have 11361 images with 151 x 151 pixels (I add 0's to the end to make it square).
allDF is a list of 33 DataFrames that correspond to the 33 subirectories in newFileNames the images need to save to.
I've tried deleting each DataFrame and image at the end of each iteration.
I've tried converting the float values to int.
I've tried gc.collect() at the end of each iteration (even though I know it's redundant)
I've taken measures not to store any additional values by always referencing the original data.
The only thing that helps is if I process one frame at a time. It still slows down, but because there are less iterations it's not as slow. So, I think the inner loop or one of the functions is the issue.
def shape_pixels(imglist):
for i in range(122):
imglist.append(0.0)
imgarr = np.array(imglist).reshape((151,151))
imgarr.reshape((151,151))
return imgarr
def create_rbg_image(subpath,imgarr,imgname):
# create/save image
img = plt.imshow(imgarr, cmap=rgbmap)
plt.axis('off')
plt.savefig(dirpath+subpath+imgname,
transparent=True,
bbox_inches=0,pad_inches=0)
for i in range(len(allDF)):
for j in range(len(allDF[i])):
fname = allDF[i]['File Name'].iloc[j][0:36]
newlist = allDF[i].iloc[j][1:].tolist()
newarr = shape_pixels(allDF[i].iloc[j][1:].tolist())
create_rbg_image(newFileNames[i]+'\\',shape_pixels(allDF[i].iloc[j][1:].tolist()),allDF[i]['File Name'].iloc[j][0:36])
I'd like to be able to run the code for the entire dataset and just come back to it when it's done, but I ran it overnight and got less than 1/3 of the way through. If it continues to slow down I'll never be done.
The first minute generates over 150 images The second generates 80. Then 48, 32, 27, and so on.. eventually it takes several minutes to create just one.
I don
plot.close('all') helped significantly, but I switched to using PIL and hexadec values, This was significantly more efficient and I was able to generate all 11k+ images in under 20 minutes
I am new to Tkinter (and Python) and I would like to find the most efficient way to format RGB values into a string so it can be used with the PhotoImage.put() function.
Let's say I have a Numpy rank 3 array in which the RGB values are stored, the 3rd dimension having a length of 3 for red, green and blue respectively. The most intuitive way to proceed would be:
for i in range(0, n_pixels_x):
for j in range(0, n_pixels_y):
hexcode = "#%02x%02x%02x" % (array[i,j,0], array[i,j,1], array[i,j,2])
img.put(hexcode, (j,i))
Unfortunately, this is way too slow for large images.
As described in the PhotoImage Wiki, it is possible to pass one large string to put() so the function is called only once. Then, I need to efficiently convert my array into such a string, which should be formatted like this (for a 4x2 image):
"{#ff0000 #ff0000 #ff0000 #ff0000} {#ff0000 #ff0000 #ff0000 #ff0000}"
Again, this could easily be done with nested for loops, but I would like to avoid them for efficiency reasons. Is there any way to use join() in order to do what I want?
If needed, I can store the content of my array differently, the only constraint being that I should be able to modify the color values easily.
Edit: After working on this a bit, I found a way to format my values approximately 10 times faster than by using nested loops. Here is the commented piece of code:
# 1. Create RGB array
array = np.zeros((n_pixels_x*n_pixels_y, 3))
array = np.asarray(array, dtype = "uint32")
array[1,:] = [0, 100, 255]
# 2. Create a format string
fmt_str = "{" + " ".join(["#%06x"]*n_pixels_x) + "}"
fmt_str = " ".join([fmt_str]*n_pixels_y)
# 3. Convert RGB values to hex
array_hex = (array[:,0]<<16) + (array[:,1]<<8) + array[:,2]
# 4. Format array
img_str = fmt_str % tuple(array_hex)
For a 640x480 array, steps 3 and 4 take ~0.1s to execute on my laptop (evaluated with timeit.default_timer()). Using nested loops, it takes between 0.9s and 1.0s.
I would still like to reduce the computation time, but I'm not sure if any improvement is still possible at this point.
I was able to find another way to format my array, and this really seems to be the quickest solution. The solution is to simply use Image and ImageTk to generate an image object directly from the array:
array = np.zeros((height, width, 3), 'uint8')
imageObject = Image.fromarray(array)
img = ImageTk.PhotoImage(image = imageObject, mode = 'RGB'))
This takes approximately 0.02s to run, which is good enough for my needs, and there is no need to use the put() function.
I actually found this answer from another question: How do I convert a numpy array to (and display) an image?
I am looking for a fast way to save a grayscale image as a 4-bit png with python. The images that I have to save are quite big, so saving them takes quite some time.
Suppose my image is stored in a numpy-array (dtype=8-bit). With PyPng I can do:
import png
data = map(lambda x: map(int, x/17), data)
png.from_array(data, 'L;4').save(filename)
This will save a proper 4-bit png. With Pillow, I can do:
import PIL.Image as Image
im = Image.fromarray(data)
im.save(filename)
The second approach (Pillow) is about 10 times as fast as the first one (even without the conversation), however the images are 8-bit pngs. I tried adding the lines
im = im.point(lambda i: i/17) # convert values
im.mode = 'L;4'
but then I get *** SystemError: unknown raw mode, even though the Mode 'L;4' is specified in https://github.com/python-pillow/Pillow/blob/master/PIL/PngImagePlugin.py
Does anyone know how to save 4-bit pngs with Pillow or is there another fast way to do it?
Pillow doesn't support 4-bits grayscale. However, if, like me, you just want to convert the 8-bit image to a 4-bit bytestring, you can.
Just dividing by 17 isn't enough, because each pixel will still be output as 1 byte. You need to pair each subsequent nibble with its neighbor nibble to get a full byte.
For that you could use something like this:
def convert_8bit_to_4bit(bytestring):
fourbit = []
for i in range(0,len(bytestring),2):
first_nibble = int(bytestring[i] / 17)
second_nibble = int(bytestring[i+1] / 17)
fourbit += [ first_nibble << 4 | second_nibble ]
fourbit = bytes(fourbit)
return fourbit
Dependent on how your other application will handle the order of the nibbles you might have to switch 'first_nibble' and 'second_nibble' with each other
I'm looking to cut up image data into regularly sized screen blocks. Currently the method I've been using is this:
def getScreenBlocksFastNew(bmpstr):
pixelData = array.array('c')
step = imgWidth * 4
pixelCoord = (blockY * blockSizeY * imgWidth +
blockSizeX * blockX)* 4
for y in range(blockSizeY):
pixelData.extend( bmpstr[pixelCoord : pixelCoord + blockSizeX * 4] )
pixelCoord += step
return pixelData
bmpstr is a string of the raw pixel data, stored as one byte per RGBA value. (I also have the option of using a tuple of ints. They seem to take about the same amount of time for each). This creates an array of a block of pixels, depicted by setting blockX, blockY and blockSizeX, blockSizeY. Currently blockSizeX = blockSizeY = 22, which is the optimal size screen block for what I am doing.
My problem is that this process takes .0045 seconds per 5 executions, and extrapolating that out to the 2000+ screen blocks to fill the picture resolution requires about 1.7 seconds per picture, which is far too slow.
I am looking to make this process faster, but I'm not sure what the proper algorithm will be. I am looking to have my pixelData array pre-created so I don't have to reinstantiate it every time. However this leaves me with a question: what is the fastest way to copy the pixel RGBA values from bmpstr to an array, without using extend or append? Do I need to set each value individually? That can't be the most efficient way.
For example, how can I copy values bmpstr[0:100] into pixelData[0:100] without using extend or setting each value individually?