We Keep Coding

How to get only possible non overlapping blocks from images of any given size and show them in Python

I have images of varying resolutions, and I would like to extract non-overlapping blocks from these images.
However, because the images have not fixed size and my block size is big (64x64), I would like to get only non-overlapping blocks that could be found in an image. If the block exceeds the image borders, I don't want to get them.
I tried the view_as_blocks function from scikit-image as below:
from skimage.util import view_as_blocks
for elem in listOfFiles:
# Reading image
print("Reading image "+elem)
img = cv2.imread(elem)
print(img.shape) #for example, one image is (2059, 2059, 3)
Blocks = view_as_blocks(img, block_shape=(64, 64, 3))
The code returns the following error:
ValueError: 'block_shape' is not compatible with 'arr_in'
I also tried the Patch Extractor from scikit-learn, as follows:
from sklearn.feature_extraction import image
import cv2
import numpy
for elem in listOfFiles:
# Reading image
print("Reading image "+elem)
img = cv2.imread(elem)
print(img.shape)
pe = image.PatchExtractor(patch_size=(64,64))
pe_fit = pe.fit(img)
pe_trans = pe.transform(img)
print('Patches shape: {}'.format(pe_trans.shape))
The error that returns to me is the following:
ValueError: negative dimensions are not allowed
the function image.extract_patches_2d from sklearns runs perfectly, but unfortunatelly it works only for overlapping blocks as you can see here.
These functions also don't help me because I also want to show the image with these blocks selected, so I also need another matrix with coordinates of such blocks and show the selected blocks.
Is that possible to that in Python?

Since you don't care about the incomplete blocks at the edges, you can manually check the number of blocks along each dimension, and crop your image to that shape:
from skimage.util import view_as_blocks
for elem in listOfFiles:
# Reading image
print("Reading image "+elem)
img = cv2.imread(elem)
print(img.shape) #for example, one image is (2059, 2059, 3)
block_shape = np.array((64, 64, 3))
nblocks = np.array(img.shape) // block_shape # integer division
crop_r, crop_c, crop_ch = nblocks * block_shape
cropped_img = img[:crop_r, :crop_c, :crop_ch]
Blocks = view_as_blocks(cropped_img, block_shape=(64, 64, 3))

Change shape, upscale, image array in Neural Net

I have an array of image pixel values that I would like to upscale for input into my neural network. It is an array of shape (28000, 48, 48, 1). These are normalized image pixel values and would like to upscale these to a higher resolution for input into my CNN. The arrays look like this...
array([[[[-0.6098866 ],
[-0.4592209 ],
[-0.40325198],
...,
[-0.7694696 ],
[-0.90518403],
[-0.95160526]],
[[-0.66049284],
[-0.68162924],
[-0.694159 ],
Both my X_train and y_train image arrays have shape of (28000,48,48,1). I would like to upscale or resize these 28000 image arrays to size 75x75. Please help. Should I convert arrays back to non-normalized arrays or images and then maybe use cv2 to upscale? How would I do this?

One easy way to resize images is using the Python module PIL (Python Image Library), which you can install with pip install pillow. Example below to demonstrate resizing a single image:
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt
# Open image
panda_pil = Image.open("panda.jpg")
print(np.array(panda_pil).shape)
# (613, 696, 3)
panda_pil_resized = panda_pil.resize((75, 75))
print(np.array(panda_pil_resized).shape)
# (75, 75, 3)
plt.imshow(np.array(panda_pil_resized))
plt.show()
You can download the panda image as follows:
import urllib.request
panda_fname = "panda.jpg"
panda_url = "https://upload.wikimedia.org/wikipedia/commons/f/fe/Giant_Panda_in_Beijing_Zoo_1.JPG"
urllib.request.urlretrieve(panda_url, panda_fname)
To resize all 28000 images, one approach would be to do this as a preprocessing step in a for-loop, and save the images to a numpy array.
Edit: You can loop through your original 28000x2304 image array and upscale each image individually in a for-loop. To get the PIL.Image object from a np.ndarray object, you can use Pil.Image.from_array, as shown below (I have just generated a random array of Gaussian noise but it should work the same with your images):
import numpy as np
from PIL import Image
from time import perf_counter
old_width, old_height = 48, 48
new_width, new_height = 75, 75
num_images = 28000
old_image_array = np.random.normal(size=[num_images, old_width*old_height])
new_image_array = np.empty(shape=[num_images, new_width*new_height])
print("Starting conversion...")
t0 = perf_counter()
# Loop over each image individually
for i in range(num_images):
# Get the ith image and reshape
old_image = old_image_array[i].reshape(old_width, old_height)
# Convert to PIL.Image
old_image_pil = Image.fromarray(old_image)
# Upscale resolution
new_image_pil = old_image_pil.resize((new_width, new_height))
# Convert to numpy array
new_image = np.array(new_image_pil)
# Reshape and store in new image array
new_image_array[i] = new_image.reshape(new_width*new_height)
t1 = perf_counter()
print("Time taken = {:.3f} s".format(t1 - t0))
print(old_image_array.shape, new_image_array.shape)
Console output:
Starting conversion...
Time taken = 2.771 s
(28000, 2304) (28000, 5625)
There may well be a more efficient way of doing this, but this method is simple, and uses tools which are useful to know about if you don't know about them already (PIL is a good module for manipulating images, see this blog post if you want to learn more about PIL).

PIL TypeError: Cannot handle this data type

I have an image stored in a numpy array that I want to convert to PIL.Image in order to perform an interpolation only available with PIL.
When trying to convert it through Image.fromarray() it raises the following error:
TypeError: Cannot handle this data type
I have read the answers here and here but they do not seem to help in my situation.
What I'm trying to run:
from PIL import Image
x # a numpy array representing an image, shape: (256, 256, 3)
Image.fromarray(x)

tl;dr
Does x contain uint values in [0, 255]? If not and especially if x ranges from 0 to 1, that is the reason for the error.
Explanation
Most image libraries (e.g. matplotlib, opencv, scikit-image) have two ways of representing images:
as uint with values ranging from 0 to 255.
as float with values ranging from 0 to 1.
The latter is more convenient when performing operations between images and thus is more popular in the field of Computer Vision.
However PIL seems to not support it for RGB images.
If you take a look here
it seems that when you try to read an image from an array, if the array has a shape of (height, width, 3) it automatically assumes it's an RGB image and expects it to have a dtype of uint8!
In your case, however, you have an RBG image with float values from 0 to 1.
Solution
You can fix it by converting your image to the format expected by PIL:
im = Image.fromarray((x * 255).astype(np.uint8))

I solved it different way.
Problem Situation:
When working with gray image or binary image, if the numpy array shape is (height, width, 1), this error will be raised also.
For example, a 32 by 32 pixel gray image (value 0 to 255)
np_img = np.random.randint(low=0, high=255, size=(32, 32, 1), dtype=np.uint8)
# np_img.shape == (32, 32, 1)
pil_img = Image.fromarray(np_img)
will raise TypeError: Cannot handle this data type: (1, 1, 1), |u1
Solution:
If the image shape is like (32, 32, 1), reduce dimension into (32, 32)
np_img = np.squeeze(np_img, axis=2) # axis=2 is channel dimension
pil_img = Image.fromarray(np_img)
This time it works!!
Additionally, please make sure the dtype is uint8(for gray) or bool(for binary).

In my case it was only because I forgotted to add the "RGB" arg in the "fromarray" func.
pil_img = Image.fromarray(np_img, 'RGB')

I found a different issue for the same error in my case. The image I used was in RGBA format, so before using fromarray() function just convert it to RGB using the convert() function and it will work perfectly fine.
image_file = Image.open(image_file)
image_file = image_file.convert('RGB')
P.S.: Posting this solution as an initial step, before converting the image to np.

In my case file format of the images was changed to png to jpg. It worked well when I corrected the image format of the error images.

According to the scikit-image's documentation, you can convert an image to unsigned byte format, with values in [0, 255] using img_as_ubyte:
from skimage import img_as_ubyte
from PIL import Image
new_image = Image.fromarray(img_as_ubyte(image))

Cannot reshape array using imageio. Why?

I'm trying to mimic what this line of code does, using imageio :
img_array = scipy.misc.imread('/Users/user/Desktop/IMG_5.png', flatten=True)
img_data = 255.0 - img_array.reshape(784)`
However when using imageio I get:
img = imageio.imread('/Users/user/Desktop/IMG_5.png')
img.flatten()
Output: Image([212, 211, 209, ..., 192, 190, 191], dtype=uint8)
img.reshape(1, 784)
ValueError: cannot reshape array of size 2352 into shape (1,784)
Can someone explain what is going on here, why is my image size 2352? I resized the image to 28x28 pixels before importing it.

I know this question already has an accepted answer, however, it implies to use skimage library instead of imageio as the question (and scipy) suggest. So here it goes.
According to imageio's doc on translating from scipy, you should change flatten argument by as_gray argument.
So this line:
img_array = scipy.misc.imread('/Users/user/Desktop/IMG_5.png', flatten=True)
should gives you same result as this:
img_array = imageio.imread('/Users/user/Desktop/IMG_5.png', as_gray=True)
It worked for me. If it didn't work for you, perhaps there is another problem. Providing an image as an example might help.

An RGB image has three channels, so 784 pixels three times is 2352. Shouldn't you save the results of img.flatten() in a variable? img_flat = img.flatten(). If you do this you should get the three color layers flatten to one gray-scale layer, then you can reshape it.
Edit: It's probably going to be easier to just use skimage in the same fashion you used the deprecated scipy:
from skimage import transform,io
# read in grey-scale
grey = io.imread('your_image.png', as_grey=True)
# resize to 28x28
small_grey = transform.resize(grey, (28,28), mode='symmetric', preserve_range=True)
# reshape to (1,784)
reshape_img = small_grey.reshape(1, 784)

To get image from cifat10-dataset

I am trying to get images from cifar10-dataset. When i rebuild image from array,
i see 9 same images in one picture, i don't know what is the problem.
When i load image from data, single_img shape (3072,). After that, i reshape
my single_img varible (32, 32, 3). I don't know where is the problem.
Here my code;
import cPickle
from PIL import Image
import numpy as np
f = open("/home/leo/Downloads/cifar-10-batches-py/data_batch_1", "rb")
tupled_data= cPickle.load(f)
f.close()
img = tupled_data['data']
single_img = np.array(img[0])
single_img_reshaped = single_img.reshape(32, 32 ,3)
j = Image.fromarray(single_img_reshaped)
j.save("/home/leo/Desktop/blabla.bmp")
Example image;

Be sure to be careful about the format of the pixel array of the image..
[R....G....B]
So you just change its format to
[[[R,G,B],....,[R,G,B]]
[[R,G,B],....,[R,G,B]]
[[R,G,B],....,[R,G,B]]]
But
single_img_reshaped = single_img.reshape(32, 32 ,3)
don't do it like before.