We Keep Coding

How do I reshape an image to NxNx3 blocks and perform operations on their channels separately

I am trying to get a better understanding of numpy reshaping and transpose operations so that I can perform tasks on each local area of a color image (as opposed to the image as a whole). I can do these by creating slices and looping over slices, but I would prefer not having to create python loops. I have come up with some examples that should help me understand the parts that I have been having trouble with. I ordered them from easiest to most difficult. The last one is ultimately the one that I want to solve.
img = np.random.randint(low=0, high=256, size=(6,6,3), dtype=np.uint8)
img_mean = np.mean(img) #mean of the whole image, one value.
channel_means = np.mean(img, axis=(0,1)) #mean of each channel, three values.
binarized_img = np.where(img > img_mean, np.uint8(255), np.uint8(0)) #all values changed to either 0 or 255. Shape of image remains 5,5,3.
binarized_channels = #I would like to be able to do the same as above, but by using a different mean for each channel and without using python loops.
three_by_three_block_means = #I want to reshape the array into four 3x3x3 blocks and get each block's mean (should be 4 different means).
three_by_three_block_channel_means = #Same as above, but this time I want the mean of each channel of each block (should be 12 different means).
#I also want to be able to change the block's size arbitrarily, i.e. from 3x3x3 blocks to 2x2x3 blocks when needed.
binarized_blocks = #same as binarized_img, but done separately for each block based on their means instead of the mean of the whole image.
binarized_block_channels = #same as binarized_blocks, but done separately for each channel in each block.
If someone could show me how to complete these examples using only numpy (no python loops), I could learn from them and use them to accomplish the (similar) tasks that I frequently have trouble with.

The solution to your problem are Strided Convolutions, use scipy.signal.convolve to compute the block means.
from scipy import signal
img = np.random.randint(low=0, high=256, size=(6,6,3), dtype=np.uint8)
img_mean = np.mean(img) #mean of the whole image, one value.
channel_means = np.mean(img, axis=(0,1)) #mean of each channel, three values.
binarized_img = np.where(img > img_mean, np.uint8(255), np.uint8(0)) #all values changed to either 0 or 255. Shape of image remains 5,5,3.
I would like to be able to do the same as above, but by using a
different mean for each channel and without using python loops.
binarized_channels = np.where(img > channels_mean, np.uint8(0),np.uint8(255))
I want to reshape the array into four 3x3x3 blocks and get each
block's mean (should be 4 different means).
Define a mean kernel (all ones divided by the sum of the kernel) of arbitrary shape, and perform a valid convolution of the image. Since scipy does not offer a stride argument we have to do this manually with [::s,::s].
s = 3
kernel = np.ones((s,s,s))/s**3
three_by_three_block_means = signal.convolve(img, kernel, 'valid')[::s,::s] # shape: (2, 2, 1)
Same as above, but this time I want the mean of each channel of each
block (should be 12 different means).
kernel = np.ones(s,s,1)/s**2
three_by_three_block_channel_means = np.concolve(img, kernel, 'valid')[::s,::s] # shape: (2, 2, 3)
I also want to be able to change the block's size arbitrarily, i.e.
from 3x3x3 blocks to 2x2x3 blocks when needed.
Simply change the size of the kernel.
Same as binarized_img, but done separately for each block based on
their means instead of the mean of the whole image.
binarized_blocks = np.where(three_by_three_block_means > img_mean,np.uint8(0),np.uint8(255))
Same as binarized_blocks, but done separately for each channel in each
block.
binarized_block_channels = np.where(three_by_three_block_channel_means > channel_means, np.uint8(0), np.uint8(255))
Hope that solves your problem. Let me know if something is unclear.

Median filter produces unexpected result on FITS file

This is based on a couple of other questions that haven't quite been answered, so I've started a new post. I'm working on finding the median of a masked array in 50-pixel patches. The image and the mask are both 901x877 telescope images.
import numpy as np
import matplotlib.pyplot as plt
from astropy.io import fits
# Use the fits files as input image and mask
hdulist = fits.open('xbulge-w1.fits')
w1data = hdulist[0].data
hdulist3 = fits.open('xbulge-mask.fits')
mask = 1 - hdulist3[0].data
w1masked = np.ma.array(w1data, mask = mask)
# Use general arrays as input image and mask
#w1data = np.arange(790177).reshape(901,877)
#w1masked = np.ma.masked_inside(w1data, 30000, 60000)
side = 50
w, h = w1data.shape
width_index = np.array(range(w//side)) * side
height_index = np.array(range(h//side)) * side
def assign_patch(patch, median, side):
"""Break this loop out to prevent 4 nested 'for' loops"""
for j in range(side):
for i in range(side):
patch[i,j] = median
return patch
for width in width_index:
for height in height_index:
patch = w1masked[width:width+side, height:height+side]
median = np.median(patch)
assign_patch(patch, median, side)
plt.imshow(w1masked)
plt.show()
The problem is, when I use the general arrays as input image and mask (the commented out section), it works fine, but when I use the FITS files, it produces 'side'-sized patches on the output image. I can't figure out what's going on with this.

I don't know how your FITS files look like but there are several things standing out:
np.median doesn't take the mask into account. In fact in recent NumPy releases this (correctly) prints a Warning if attempted. You should be using np.ma.median instead. If you would update your NumPy you'll likely see this:
UserWarning: Warning: 'partition' will ignore the 'mask' of the MaskedArray.
The assign_patch function is unnecessary when you know that you can use slice assignment:
w1masked[width:width+side, height:height+side] = median
# instead of "assign_patch(patch, median, side)"
That's also much faster than doing a double loop to replace each value.
I assume that the issue is in fact because you use np.median instead of np.ma.median. There are lots of values a masked pixel could have including nan, 0, inf, ... so if these are taken into account (when they should be ignored) could produce any kind of problems, especially if the median starts returning nans or similar.
More generally if you really wanted a median filter you can't just calculate the median of a patch and replace all values in the patch with that median. You should be using a median filter that takes the mask into account. Unfortunately I've never seen such a filter implemented in any wide-spread Python package. But if you have numba you could checkout a (very experimental!) package of mine numbamisc which contains a median_filter that takes masks into account.

Fastest way to find difference between image pixel and palette colour in Python

I am working on some code for converting an image to the palette of the NES. My current code is somewhat successful, but very very slow.
I am doing it by using Pythagoras' theorem. I'm using the RGB colour values as coordinates in 3D space and doing it that way. The colour in the palette with the smallest distance from the pixel's RGB is the colour that gets used.
class image_filter():
def load(self,path):
self.i = Image.open(path)
self.i = self.i.convert("RGB")
self.pix = self.i.load()
def colour_filter(self,colours=NES):
start = time.time()
for y in range(self.i.size[1]):
for x in range(self.i.size[0]):
pixel = list(self.pix[x,y])
distances = []
for colour in colours:
distance = ((colour[0]-pixel[0])**2)+((colour[1]-pixel[1])**2)+((colour[2]-pixel[2])**2)
distances.append(distance)
pixel = colours[distances.index(sorted(distances,key=lambda x:x)[0])]
self.pix[x,y] = tuple(pixel)
print "Took "+str(time.time()-start)+" seconds."
f = image_filter()
f.load("C:\\path\\to\\image.png")
f.colour_filter()
f.i.save("C:\\path\\to\\new\\image.png")
Using the list:
NES = [(124,124,124),(0,0,252),
(0,0,188),(68,40,188),
(148,0,132),(168,0,32),
(168,16,0),(136,20,0),
(80,48,0),(0,120,0),
(0,104,0),(0,88,0),
(0,64,88),(0,0,0),
(188,188,188),(0,120,248),
(0,88,248),(104,68,252),
(216,0,204),(228,0,88),
(248,56,0),(228,92,16),
(172,124,0),(0,184,0),
(0,168,0),(0,168,68),
(0,136,136),(248,248,248),
(60,188,252),(104,136,252),
(152,120,248),
(248,120,248),(248,88,152),
(248,120,88),(252,160,68),
(184,248,24),(88,216,84),
(88,248,152),(0,232,216),
(120,120,120),(252,252,252),(164,228,252),
(184,184,248),(216,184,248),
(248,184,248),(248,164,192),
(240,208,176),(252,224,168),
(248,216,120),(216,248,120),
(184,248,184),(184,248,216),
(0,252,252),(216,216,216)]
This produces the following Input:
and Output:
This takes between 14 and 20 seconds, which is much too long for its intended application. Does anyone know of any ways to greatly speed this up?
As an idea, I was thinking it may be possible to use numpy arrays for this; however I am not at all familiar enough with numpy arrays to be able to pull it off.
If possible, I would also like to try avoiding using scipy -- I know that, at least under Windows, it can be a pain to install and would prefer to steer clear.

Approach #1 : We could use Scipy's cdist to get the euclidean distances and then look for the min distance arg and thus select the appropriate colour.
Thus, with NumPy arrays as the inputs, we would have an implementation like so -
from scipy.spatial.distance import cdist
out = colours[cdist(pix.reshape(-1,3),colours).argmin(1)].reshape(pix.shape)
Approach #2 : Here's another approach with broadcasting and np.einsum -
subs = pix - colours[:,None,None]
out = colours[np.einsum('ijkl,ijkl->ijk',subs,subs).argmin(0)]
Interfacing between PIL/lists and NumPy arrays
To accept images read through PIL, use :
pix = np.asarray(Image.open('input_filename'))
To Use colours as array :
colours = np.asarray(NES)
# .... Use one of the listed approaches and get out as output array
To output the image :
i = Image.fromarray(out.astype('uint8'),'RGB')
i.save("output_filename")
Sample input, output using given colour palette NES -

Numpy image slicing returning black patches/ wrong values

The end goal is to take an image and slice it up into samples that I save. The problem is that my slices are randomly returning black/ incorrect patches. Bellow is a small sample program.
import scipy.ndimage as ndimage
import scipy.misc as misc
import numpy as np
image32 = misc.imread("work0.png")
patches = np.zeros((36, 8, 8))
for i in range(4):
for j in range(4):
patches[i*4 + j] = image32[i:i+8,j:j+8]
misc.imsave("{0}{1}.png".format(i,j), patches[i*4 + j])
An example of my image would be:
Patch of 0,0 of 8x8 patch yields:

Two things:
You are initializing your patch matrix to be the wrong data type. By default, numpy will make patches matrix a np.float64 type and if you use this with saving, you won't get the results you would expect. Specifically, if you consult Mr. F's answer, there is actually some scaling performed on floating-point images where the minimum and maximum values of the image get scaled to black and white respectively and so if you have an image that is completely uniform in background, both the minimum and maximum will be the same and will get visualized to black. As such, the best thing is to respect the original image's data type, namely setting the dtype of your patches matrix to np.uint8.
Judging from your for loop indexing, you want to extract out 8 x 8 patches that are non-overlapping. This means that if you have a 32 x 32 image with 8 x 8 patches, you have 16 patches in total arranged in a 4 x 4 grid.
Therefore, you need to change the patches statement so that it has 16 in the first dimension, not 36. In addition, you'll have to adjust the way you're indexing into your image to extract out the 8 x 8 patches because right now, the patches are overlapping. Specifically, you want to make the image patch indexing go from 8*i to 8*(i+1) for the rows and 8*j to 8*(j+1) for the columns. If you substitute sample values of i and j yourself, you'll see that we get unique 8 x 8 patches for each grid in your image.
With both of the above things I noted, the modified code should be:
import scipy.ndimage as ndimage
import scipy.misc as misc
import numpy as np
image32 = misc.imread('work0.png')
patches = np.zeros((16,8,8), dtype=np.uint8) # Change
for i in range(4):
for j in range(4):
patches[i*4 + j] = image32[8*i:8*(i+1),8*j:8*(j+1)] # Change
misc.imsave("{0}{1}.png".format(i,j), patches[i*4 + j])
When I do this and take a look at the output images, I get what I expect.
To be absolutely sure, let's plot the segments using matplotlib. You've conveniently saved all of the patches in patches so it shouldn't be a problem showing what we need. However, I'll place some code in comments so that you can read in the images that were saved from disk with your above code so you can verify that it still works, regardless of looking at patches or the images on disk:
import matplotlib.pyplot as plt
plt.figure()
for i in range(4):
for j in range(4):
plt.subplot(4, 4, 4*i + j + 1)
img = patches[4*i + j]
# or you can do this:
# img = misc.imread('{0}{1}.png'.format(i,j))
img = np.dstack([img, img, img])
plt.imshow(img)
plt.show()
The weird thing about matplotlib.pyplot.imshow is that if you have an image that is single channel (such as your case) that has the same intensity all around, it gets visualized to black no matter what the colour map is, much like what we experienced with imsave. Therefore, I had to artificially make this a RGB image but with all of the channels to be the same so this gets visualized as grayscale before we show the image.
We get:

According to this answer the issue is that imsave normalizes the data so that the computed minimum is defined as black (and, if there is a distinct maximum, that is defined as white).
This led me to go digging as to why the suggested use of uint8 did work to create the desired output. As it turns out, in the source there is a function called bytescale that gets called internally.
Actually, imsave itself is a very thin wrapper around toimage followed by save (from the image object). Inside of toimage if mode is None (which it is by default), that's when bytescale gets invoked.
It turns out that bytescale has an if statement that checks for the uint8 data type, and if the data is in that format, it returns the data unaltered. But if not, then the data is scaled according to a max and min transformation (where 0 and 255 are the default low and high pixel values to compare to).
This is the full snippet of code linked above:
if data.dtype == uint8:
return data
if high < low:
raise ValueError("`high` should be larger than `low`.")
if cmin is None:
cmin = data.min()
if cmax is None:
cmax = data.max()
cscale = cmax - cmin
if cscale < 0:
raise ValueError("`cmax` should be larger than `cmin`.")
elif cscale == 0:
cscale = 1
scale = float(high - low) / cscale
bytedata = (data * 1.0 - cmin) * scale + 0.4999
bytedata[bytedata > high] = high
bytedata[bytedata < 0] = 0
return cast[uint8](bytedata) + cast[uint8](low)
For the blocks of your data that are all 255, cscale will be 0, which will be checked for and changed to 1. Then the line
bytedata = (data * 1.0 - cmin) * scale + 0.4999
will result in the whole image block having the float value of 0.4999, thus set explicitly to 0 in the next chunk of code (when casted to uint8 from float) as for example:
In [102]: np.cast[np.uint8](0.4999)
Out[102]: array(0, dtype=uint8)
You can see in the body of bytescale that there are only two possible ways to return: either your data is type uint8 and it's returned as-is, or else it goes through this kind of silly scaling process. So in the end, it is indeed correct, and good practice, to be using uint8 for the pieces of your code that specifically load from or save to an image format via these functions.
So this cascade of stuff is why you were getting all zeros in the outputted image file and why the other suggestion of using dtype=np.uint8 actually helps you. It's not because you need to avoid floating point data for images, just because of this bizarre convention to check and scale data on the part of imsave.

Numpy manipulating array of True values dependent on x/y index

So I have an array (it's large - 2048x2048), and I would like to do some element wise operations dependent on where they are. I'm very confused how to do this (I was told not to use for loops, and when I tried that my IDE froze and it was going really slow).
Onto the question:
h = aperatureimage
h[:,:] = 0
indices = np.where(aperatureimage>1)
for True in h:
h[index] = np.exp(1j*k*z)*np.exp(1j*k*(x**2+y**2)/(2*z))/(1j*wave*z)
So I have an index, which is (I'm assuming here) essentially a 'cropped' version of my larger aperatureimage array. *Note: Aperature image is a grayscale image converted to an array, it has a shape or text on it, and I would like to find all the 'white' regions of the aperature and perform my operation.
How can I access the individual x/y values of index which will allow me to perform my exponential operation? When I try index[:,None], leads to the program spitting out 'ValueError: broadcast dimensions too large'. I also get array is not broadcastable to correct shape. Any help would be appreciated!
One more clarification: x and y are the only values I would like to change (essentially the points in my array where there is white, z, k, and whatever else are defined previously).
EDIT:
I'm not sure the code I posted above is correct, it returns two empty arrays. When I do this though
index = (aperatureimage==1)
print len(index)
Actually, nothing I've done so far works correctly. I have a 2048x2048 image with a 128x128 white square in the middle of it. I would like to convert this image to an array, look through all the values and determine the index values (x,y) where the array is not black (I only have white/black, bilevel image didn't work for me). I would then like to take all the values (x,y) where the array is not 0, and multiply them by the h[index] value listed above.
I can post more information if necessary. If you can't tell, I'm stuck.
EDIT2: Here's some code that might help - I think I have the problem above solved (I can now access members of the array and perform operations on them). But - for some reason the Fx values in my for loop never increase, it loops Fy forever....
import sys, os
from scipy.signal import *
import numpy as np
import Image, ImageDraw, ImageFont, ImageOps, ImageEnhance, ImageColor
def createImage(aperature, type):
imsize = aperature*8
middle = imsize/2
im = Image.new("L", (imsize,imsize))
draw = ImageDraw.Draw(im)
box = ((middle-aperature/2, middle-aperature/2), (middle+aperature/2, middle+aperature/2))
import sys, os
from scipy.signal import *
import numpy as np
import Image, ImageDraw, ImageFont, ImageOps, ImageEnhance, ImageColor
def createImage(aperature, type):
imsize = aperature*8 #Add 0 padding to make it nice
middle = imsize/2 # The middle (physical 0) of our image will be the imagesize/2
im = Image.new("L", (imsize,imsize)) #Make a grayscale image with imsize*imsize pixels
draw = ImageDraw.Draw(im) #Create a new draw method
box = ((middle-aperature/2, middle-aperature/2), (middle+aperature/2, middle+aperature/2)) #Bounding box for aperature
if type == 'Rectangle':
draw.rectangle(box, fill = 'white') #Draw rectangle in the box and color it white
del draw
return im, middle
def Diffraction(aperaturediameter = 1, type = 'Rectangle', z = 2000000, wave = .001):
# Constants
deltaF = 1/8 # Image will be 8mm wide
z = 1/3.
wave = 0.001
k = 2*pi/wave
# Now let's get to work
aperature = aperaturediameter * 128 # Aperaturediameter (in mm) to some pixels
im, middle = createImage(aperature, type) #Create an image depending on type of aperature
aperaturearray = np.array(im) # Turn image into numpy array
# Fourier Transform of Aperature
Ta = np.fft.fftshift(np.fft.fft2(aperaturearray))/(len(aperaturearray))
# Transforming and calculating of Transfer Function Method
H = aperaturearray.copy() # Copy image so H (transfer function) has the same dimensions as aperaturearray
H[:,:] = 0 # Set H to 0
U = aperaturearray.copy()
U[:,:] = 0
index = np.nonzero(aperaturearray) # Find nonzero elements of aperaturearray
H[index[0],index[1]] = np.exp(1j*k*z)*np.exp(-1j*k*wave*z*((index[0]-middle)**2+(index[1]-middle)**2)) # Free space transfer for ap array
Utfm = abs(np.fft.fftshift(np.fft.ifft2(Ta*H))) # Compute intensity at distance z
# Fourier Integral Method
apindex = np.nonzero(aperaturearray)
U[index[0],index[1]] = aperaturearray[index[0],index[1]] * np.exp(1j*k*((index[0]-middle)**2+(index[1]-middle)**2)/(2*z))
Ufim = abs(np.fft.fftshift(np.fft.fft2(U))/len(U))
# Save image
fim = Image.fromarray(np.uint8(Ufim))
fim.save("PATH\Fim.jpg")
ftfm = Image.fromarray(np.uint8(Utfm))
ftfm.save("PATH\FTFM.jpg")
print "that may have worked..."
return
if __name__ == '__main__':
Diffraction()
You'll need numpy, scipy, and PIL to work with this code.
When I run this, it goes through the code, but there is no data in them (everything is black). Now I have a real problem here as I don't entirely understand the math I'm doing (this is for HW), and I don't have a firm grasp on Python.
U[index[0],index[1]] = aperaturearray[index[0],index[1]] * np.exp(1j*k*((index[0]-middle)**2+(index[1]-middle)**2)/(2*z))
Should that line work for performing elementwise calculations on my array?

Could you perhaps post a minimal, yet complete, example? One that we can copy/paste and run ourselves?
In the meantime, in the first two lines of your current example:
h = aperatureimage
h[:,:] = 0
you set both 'aperatureimage' and 'h' to 0. That's probably not what you intended. You might want to consider:
h = aperatureimage.copy()
This generates a copy of aperatureimage while your code simply points h to the same array as aperatureimage. So changing one changes the other.
Be aware, copying very large arrays might cost you more memory then you would prefer.
What I think you are trying to do is this:
import numpy as np
N = 2048
M = 64
a = np.zeros((N, N))
a[N/2-M:N/2+M,N/2-M:N/2+M]=1
x,y = np.meshgrid(np.linspace(0, 1, N), np.linspace(0, 1, N))
b = a.copy()
indices = np.where(a>0)
b[indices] = np.exp(x[indices]**2+y[indices]**2)
Or something similar. This, in any case, sets some values in 'b' based on the x/y coordinates where 'a' is bigger than 0. Try visualizing it with imshow. Good luck!
Concerning the edit
You should normalize your output so it fits in the 8 bit integer. Currently, one of your arrays has a maximum value much larger than 255 and one has a maximum much smaller. Try this instead:
fim = Image.fromarray(np.uint8(255*Ufim/np.amax(Ufim)))
fim.save("PATH\Fim.jpg")
ftfm = Image.fromarray(np.uint8(255*Utfm/np.amax(Utfm)))
ftfm.save("PATH\FTFM.jpg")
Also consider np.zeros_like() instead of copying and clearing H and U.
Finally, I personally very much like working with ipython when developing something like this. If you put the code in your Diffraction function in the top level of your script (in place of 'if __ name __ &c.'), then you can access the variables directly from ipython. A quick command like np.amax(Utfm) would show you that there are indeed values!=0. imshow() is always nice to look at matrices.