Related
I'm new to Python, and I'm trying to deconstruct image bands as arrays of numbers by applying the Singular Value Decomposition (SVD) to them and then putting them back together with matplotlib.image and the Image module from PIL. An SVD may also be written as a sum of dyads s1u1v1T + ... + sKuKvKT, and the point in decomposing it in this way is that a near-perfect approximation of the image can be made from just a few of those dyads, so less data is required.
There must be something wrong with the calculation, though because result_r, result_g, and result_b look like this when converted to Images, and new_image looks like this.
For an example of what this should look like, here are the first dyads of the layers of this image. The image that I'm using (April23.jpg) is this.
import matplotlib.image as image
import numpy.linalg as la
import numpy as np
from PIL import Image
def getcolumn(j, m):
col = []
for i in range(len(m)):
col.append(m[i][j])
return col
def extractCols(U):
Ucols = []
for j in range(len(U[0])):
Ucols.append(getcolumn(j, U))
return np.asarray(Ucols)
def vectorMultiply(u, v):
matrix = []
for i in range(len(u)):
newVec = []
for j in range(len(v)):
newVec.append(u[i] * v[j])
matrix.append(newVec)
return np.asarray(matrix)
im = Image.open('C:/Users/<user>/Desktop/img/April23.jpg')
im.load()
sim = Image.Image.split(im)
rsim = sim[0].save("rsim.jpg") # image bands as images
gsim = sim[1].save("gsim.jpg")
bsim = sim[2].save("bsim.jpg")
# image bands as arrays of numbers
arsim = image.imread('C:/Users/<user>/Desktop/img/rsim.jpg')
agsim = image.imread('C:/Users/<user>/Desktop/img/gsim.jpg')
absim = image.imread('C:/Users/<user>/Desktop/img/bsim.jpg')
ur, sr, vhr = la.svd(arsim, False) # SVD on each band
ug, sg, vhg = la.svd(agsim, False)
ub, sb, vhb = la.svd(absim, False)
urcols = extractCols(ur)
ugcols = extractCols(ug)
ubcols = extractCols(ub)
# calculating the first dyads
result_r = np.multiply(sr[0], vectorMultiply(urcols[0], vhr[0]))
result_g = np.multiply(sg[0], vectorMultiply(ugcols[0], vhg[0]))
result_b = np.multiply(sb[0], vectorMultiply(ubcols[0], vhb[0]))
r = Image.fromarray(result_r, "L")
g = Image.fromarray(result_g, "L")
b = Image.fromarray(result_b, "L")
new_image = Image.merge("RGB", (r, g, b))
What am I missing, here? It seems to be something with the calculations. I figured for a matrix one would have to extract the columns, say the column [1, 2, 3] from a matrix [[1,...], [2,...], [3,...]], since each element of the matrix is a row. So, I wrote extractCols() for that. numpy's matrix add and multiply seem to be fine. I wrote vectorMultiply because np.dot(), np.multiply(), and np.matmul() didn't seem to realize that u was a column and kept saying the dimensions didn't match up. I tested it and it seemed to do what I wanted it to. I was also thinking that maybe the "rows" of U are actually the columns already and don't need to be extracted, but that didn't work either. I've also tried not using np.asarray() without any luck.
Any advice is appreciated.
I am translating code from MATLAB to python but cannot perfectly replicate the results of MATLAB's imresize3. My input is a 101x101x101 array. First four inputs ([0,0:3,0] or (1,1:4,1)) are: 0.3819 0.4033 0.4336 0.2767. The data input for both languages is identical.
sampleQDNormSmall = imresize3(sampleQDNorm,0.5);
This results in a 51x51x51 array where the first four values (1,1:4,1) for example are: 0.3443 0.2646 0.2700 0.2835
Now I've tried two different pieces of code in python to replicate these results:
from skimage.transform import resize
from skimage.transform import rescale
sampleQDNormSmall = resize(sampleQDNorm,(0.5*sampleQDNorm.shape[0],0.5*sampleQDNorm.shape[1],0.5*sampleQDNorm.shape[2]),order=3,anti_aliasing=True);
sampleQDNormSmall1=rescale(sampleQDNorm,0.5,order=3,anti_aliasing=True)
The first one gives a 51x51x51 array that has the first four values [0,0:3,0] of: 0.3452 0.2669 0.2774 0.3099. Which is very close but not exactly the same numerical outputs. I don't know enough about the optional arguments to know might get me a better result.
The second one gives a 50x50x50 array that has the first four values [0,0:3,0] of: 0.3422 0.2623 0.2810 0.3006. This is a different output array size and also doesn't reproduce the same numerical outputs as the MATLAB code or the other python function
I don't know enough about the optional arguments to know might get me a better result. I know for this type of array, MATLAB's default is cubic interpolation which is why I am using order 3 in python. The default for anti-aliasing in both is true. I have a two bigger arrays that I am having the same issues with: a (873x873x873) array and a bool (873x873x873) array.
The MATLAB code I'm using is considered the "correct answer" for the work I am doing so I am trying to replicate the results as accurately as possible into python. Please let me know what I can try in python to reproduce the correct data.
sampleQDNorm is roughly random decimals between 0 and 1 for [0:100,0:100,0:100] and is padded with zeros on sides [:,:,101],[:,101,:],[101,:,:]
Getting the exact same result as MATLAB imresize3 is challenging.
One reason is that MATLAB enables Antialiasing filter by default, and I can't seem to find the equivalent Python implementation.
The closet existing Python alternatives are described in this post.
scipy.ndimage.zoom supports 3D resizing.
It could be that skimage.transform.resize gives closer result, but none are identical to MATLAB result.
Reimplementing imresize3:
Looking at the MATLAB implementation of imresize3 (MATLAB source code), it is apparent that MATLAB implementation "simply" uses resize along each axis:
Resize (by half) along the vertical axis.
Resize the above result (by half) along the horizontal axis.
Resize the above result (by half) along the depth axis.
Here is a MATLAB codes sample that demonstrates the implementation (using cubic interpolation):
I1 = imread('peppers.png');
I2 = imresize(imread('autumn.tif'), [size(I1, 1), size(I1, 2)]);
I3 = imresize(imread('football.jpg'), [size(I1, 1), size(I1, 2)]);
I4 = imresize(imread('cameraman.tif'), [size(I1, 1), size(I1, 2)]);
I = cat(3, I1, I2, I3, I4);
J = imresize3(I, 0.5, 'cubic', 'Antialiasing', false);
imwrite(I1, '/Tmp/I1.png');
imwrite(I2, '/Tmp/I2.png');
imwrite(I3, '/Tmp/I3.png');
imwrite(I4, '/Tmp/I4.png');
imwrite(J(:,:,1), '/Tmp/J1.png');
imwrite(J(:,:,2), '/Tmp/J2.png');
imwrite(J(:,:,3), '/Tmp/J3.png');
imwrite(J(:,:,4), '/Tmp/J4.png');
imwrite(J(:,:,5), '/Tmp/J5.png');
K = cubicResize3(I, 0.5);
max_abs_diff = max(abs(double(J(:)) - double(K(:))));
disp(['max_abs_diff = ', num2str(max_abs_diff)])
function B = cubicResize3(A, scale)
order = [1 2 3];
B = A;
for k = 1:numel(order)
dim = order(k);
B = cubicResizeAlongDim(B, dim, scale);
end
end
function out = cubicResizeAlongDim(in, dim, scale)
% If dim is 3, permute the input matrix so that the third dimension
% becomes the first dimension. This way, we can resize along the
% third dimensions as though we were resizing along the first dimension.
isThirdDimResize = (dim == 3);
if isThirdDimResize
in = permute(in, [3 2 1]);
dim = 1;
end
if dim == 1
out_rows = round(size(in, 1)*scale);
out_cols = size(in, 2);
else % dim == 2
out_rows = size(in, 1);
out_cols = round(size(in,2)*scale);
end
out = zeros(out_rows, out_cols, size(in, 3), class(in)); % Allocate array for storing the output.
for i = 1:size(in, 3)
% Resize each color plane separately
out(:, :, i) = imresize(in(:, :, i), [out_rows, out_cols], 'bicubic', 'Antialiasing', false);
end
% Permute back so that the original dimensions are restored if we were
% resizing along the third dimension.
if isThirdDimResize
out = permute(out, [3 2 1]);
end
end
The result is max_abs_diff = 0, meaning that cubicResize3 and imresize3 gave the same output.
Note:
The above implementation stores images in Tmp folder to be used a input for testing Python implementation.
Here is a Python implementation using OpenCV:
import numpy as np
import cv2
#from scipy.ndimage import zoom
def cubic_resize_along_dim(inp, dim, scale):
""" Implementation is based on MATLAB source code of resizeAlongDim function """
# If dim is 3, permute the input matrix so that the third dimension
# becomes the first dimension. This way, we can resize along the
# third dimensions as though we were resizing along the first dimension.
is_third_dim_resize = (dim == 2)
if is_third_dim_resize:
inp = np.swapaxes(inp, 2, 0).copy() # in = permute(in, [3 2 1])
dim = 0
if dim == 0:
out_rows = int(np.round(inp.shape[0]*scale)) # out_rows = round(size(in, 1)*scale);
out_cols = inp.shape[1] # out_cols = size(in, 2);
else: # dim == 1
out_rows = inp.shape[0] # out_rows = size(in, 1);
out_cols = int(np.round(inp.shape[1]*scale)) # out_cols = round(size(in,2)*scale);
out = np.zeros((out_rows, out_cols, inp.shape[2]), inp.dtype) # out = zeros(out_rows, out_cols, size(in, 3), class(in)); % Allocate array for storing the output.
for i in range(inp.shape[2]):
# Resize each color plane separately
out[:, :, i] = cv2.resize(inp[:, :, i], (out_cols, out_rows), interpolation=cv2.INTER_CUBIC) # out(:, :, i) = imresize(inp(:, :, i), [out_rows, out_cols], 'bicubic', 'Antialiasing', false);
# Permute back so that the original dimensions are restored if we were
# resizing along the third dimension.
if is_third_dim_resize:
out = np.swapaxes(out, 2, 0) # out = permute(out, [3 2 1]);
return out
def cubic_resize3(a, scale):
b = a.copy()
for k in range(3):
b = cubic_resize_along_dim(b, k, scale)
return b
# Build 3D input image (10 channels with resolution 512x384).
i1 = cv2.cvtColor(cv2.imread('/Tmp/I1.png', cv2.IMREAD_UNCHANGED), cv2.COLOR_BGR2RGB)
i2 = cv2.cvtColor(cv2.imread('/Tmp/I2.png', cv2.IMREAD_UNCHANGED), cv2.COLOR_BGR2RGB)
i3 = cv2.cvtColor(cv2.imread('/Tmp/I3.png', cv2.IMREAD_UNCHANGED), cv2.COLOR_BGR2RGB)
i4 = cv2.imread('/Tmp/I4.png', cv2.IMREAD_UNCHANGED)
im = np.dstack((i1, i2, i3, i4)) # Stack arrays along the third axis
# Read and adjust MATLAB output (out_mat is used as reference for testing).
# out_mat is the result of J = imresize3(I, 0.5, 'cubic', 'Antialiasing', false);
j1 = cv2.imread('/Tmp/J1.png', cv2.IMREAD_UNCHANGED)
j2 = cv2.imread('/Tmp/J2.png', cv2.IMREAD_UNCHANGED)
j3 = cv2.imread('/Tmp/J3.png', cv2.IMREAD_UNCHANGED)
j4 = cv2.imread('/Tmp/J4.png', cv2.IMREAD_UNCHANGED)
j5 = cv2.imread('/Tmp/J5.png', cv2.IMREAD_UNCHANGED)
out_mat = np.dstack((j1, j2, j3, j4, j5)) # Stack arrays along the third axis
#out_py = zoom(im, 0.5, order=3, mode='reflect')
# Execute 3D resize in Python
out_py = cubic_resize3(im, 0.5)
abs_diff = np.absolute(out_mat.astype(np.int16) - out_py.astype(np.int16))
print(f'max_abs_diff = {abs_diff.max()}')
The Python implementation reads the input files stored by MATLAB (and convert from BGR to RGB when required).
The implementation compares the result of cubic_resize3 with the MATLAB output of imresize3.
The maximum difference is 12 (not zero).
Apparently cv2.resize and MATLAB imresize gives slightly different results.
Update:
Replacing:
out[:, :, i] = cv2.resize(inp[:, :, i], (out_cols, out_rows), interpolation=cv2.INTER_CUBIC)
with:
out[:, :, i] = transform.resize(inp[:, :, i], (out_rows, out_cols), order=3, mode='edge', anti_aliasing=False, preserve_range=True)
Reduces the maximum difference to 4.
I'm trying to calibrate a fisheye camera using OpenCV 3.0.0 python bindings (with an asymmetric circle grid), but I have problems to format the object and image point arrays correctly. My current source looks like this:
import cv2
import glob
import numpy as np
def main():
circle_diameter = 4.5
circle_radius = circle_diameter/2.0
pattern_width = 4
pattern_height = 11
num_points = pattern_width*pattern_height
images = glob.glob('*.bmp')
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
imgpoints = []
objpoints = []
obj = []
for i in range(pattern_height):
for j in range(pattern_width):
obj.append((
float(2*j + i % 2)*circle_radius,
float(i*circle_radius),
0
))
for name in images:
image = cv2.imread(name)
grayimage = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
retval, centers = cv2.findCirclesGrid(grayimage, (pattern_width, pattern_height), flags=(cv2.CALIB_CB_ASYMMETRIC_GRID + cv2.CALIB_CB_CLUSTERING))
imgpoints_tmp = np.zeros((num_points, 2))
if retval:
for i in range(num_points):
imgpoints_tmp[i, 0] = centers[i, 0, 0]
imgpoints_tmp[i, 1] = centers[i, 0, 1]
imgpoints.append(imgpoints_tmp)
objpoints.append(obj)
# Convertion to numpy array
imgpoints = np.array(imgpoints, dtype=np.float32)
objpoints = np.array(objpoints, dtype=np.float32)
K, D = cv2.fisheye.calibrate(objpoints, imgpoints, image_size=(1280, 800), K=None, D=None)
if __name__ == '__main__':
main()
The error message is:
OpenCV Error: Assertion failed (objectPoints.type() == CV_32FC3 || objectPoints.type() == CV_64FC3) in cv::fisheye::calibrate
objpoints has shape (31,44,3).
So objpoints array needs to be formatted in a different way, but I'm not able to achieve the correct layout. Maybe someone can help here?
In the sample of OpenCV (Camera Calibration) they set the objp to objp2 = np.zeros((8*9,3), np.float32)
However, in omnidirectional camera or fisheye camera, it should be:
objp = np.zeros((1,8*9,3), np.float32)
Idea is from here Calibrate fisheye lens using OpenCV — part 1
The correct layout of objpoints is a list of numpy arrays with len(objpoints) = "number of pictures" and each entry beeing a numpy array.
Please have a look at the official help. OpenCV documentation talks about "vectors", which is equivalent of a list or numpy.array. In this instance a "vector of vectors" can be interpreted as a list of numpy.arrays.
The data type is correct, but the shape is not. The expected shape of objpoints supposed to be (n_observations, 1, n_corners_per_observation, 3). Therefore, the code in your case should be:
imgpoints = np.array(imgpoints, dtype=np.float32).reshape(
-1,
1,
pattern_width * pattern_height,
3
)
or more general:
imgpoints = np.array(imgpoints, dtype=np.float32).reshape(
n_observations,
1,
n_corners_per_observation,
3
)
The error message is slightly misleading.
Didn't find a satisfying answer here so I messed around and eventually got this chunk to work:
calibration_flags = cv2.fisheye.CALIB_RECOMPUTE_EXTRINSIC + cv2.fisheye.CALIB_CHECK_COND + cv2.fisheye.CALIB_FIX_SKEW
# lists with each element a [1,n_points,_] array of type float32
obj_points = [np.random.rand(1,10,3).astype(np.float32)]
fisheye_points = [np.random.rand(1,10,2).astype(np.float32)]
# initialize empty variables of correct size and type, where total_num_points is summed across all arrays in each above list
rvecs = [np.zeros((1, 1, 3), dtype=np.float32) for i in range(total_num_points)]
tvecs = [np.zeros((1, 1, 3), dtype=np.float32) for i in range(total_num_points)]
D = np.zeros([4,1]).astype(np.float32)
K = np.zeros([3,3]).astype(np.float32)
outputs = cv2.fisheye.calibrate(gt_points,fisheye_points,(1920,1080),K,D,rvecs,tvecs)
I have a matrix with values from 0 to 1000, I can easily scale the values to the 0~255 range, and that is a greyscale picture if I show the matrix in opencv from Python.
The question is, how do I convert the Matrix {Dimensions = (m, n)} to a 3-layer matrix array {Dimensions = (m, n, 3)}?
This is, how to convert a greyscale picture to a colour picture?
I have made this function but it is not working
import matplotlib.pyplot as plt
from itertools import product
def convertPicturetoColor(self, image, cmap=plt.get_cmap('rainbow')):
'''
Converts a greyscale [0~255] picture to a color picture
'''
a, b = np.shape(image)
m = np.zeros((a, b, 3))
for i, j in product(xrange(a), xrange(b)):
m[i,j,:] = np.array(cmap(image[i,j]))[0:3]
return m
>>> help(cv2.applyColorMap)
Help on built-in function applyColorMap:
applyColorMap(...)
applyColorMap(src, colormap[, dst]) -> dst
and here are the map enums:
COLORMAP_AUTUMN = 0
COLORMAP_BONE = 1
COLORMAP_COOL = 8
COLORMAP_HOT = 11
COLORMAP_HSV = 9
COLORMAP_JET = 2
COLORMAP_OCEAN = 5
COLORMAP_PINK = 10
COLORMAP_RAINBOW = 4
COLORMAP_SPRING = 7
COLORMAP_SUMMER = 6
COLORMAP_WINTER = 3
so, simply:
dst = cv2.applyColorMap(src, cv2.COLORMAP_RAINBOW)
I will want to plot some images using Opencv, and for this I would like to glue images together.
Imagine I have 4 pictures. The best way would be to glue them in a 2x2 image matrix.
a = img; a.shape == (48, 48)
b = img; b.shape == (48, 48)
c = img; c.shape == (48, 48)
d = img; d.shape == (48, 48)
I now use the np.reshape which takes a list such as [a,b,c,d], and then I manually put the dimensions to get the following:
np.reshape([a,b,c,d], (a.shape*2, a.shape*2)).shape == (96, 96)
The issue starts when I have 3 pictures. I kind of figured that I can take the square root of the length of the list and then the ceiling value which will yield the square matrix dimension of 2 (np.ceil(sqrt(len([a,b,c]))) == 2). I would then have to add a white image with the dimension of the first element to the list and there we go. But I imagine there must be an easier way to accomplish this for plotting, most likely already defined somewhere.
So, how to easily combine any amount of square matrices into one big square matrix?
EDIT:
I came up with the following:
def plotimgs(ls):
shp = ls[0].shape[0] # the image's dimension
dim = np.ceil(sqrt(len(ls))) # the amount of pictures per row AND column
emptyimg = (ls[1]*0 + 1)*255 # used to add to the list to allow square matrix
for i in range(int(dim*dim - len(ls))):
ls.append(emptyimg)
enddim = int(shp*dim) # enddim by enddim is the final matrix dimension
# Convert to 600x600 in the end to resize the pictures to fit the screen
newimg = cv2.resize(np.reshape(ls, (enddim, enddim)), (600, 600))
cv2.imshow("frame", newimg)
cv2.waitKey(10)
plotimgs([a,b,d])
Somehow, even though the dimensions are okay, it actually clones some pictures more:
When I give 4 pictures, I get 8 pictures.
When I give 9 pictures, I get 27 pictures.
When I give 16 pictures, I get 64 pictures.
So in fact rather than squared, I get to the third power of images somehow. Though, e.g.
plotimg([a]*9) gives a picture with dimensions of 44*3 x 44*3 = 144x144 which should be correct for 9 images?
Here's a snippet that I use for doing this sort of thing:
import numpy as np
def montage(imgarray, nrows=None, border=5, border_val=np.nan):
"""
Returns an array of regularly spaced images in a regular grid, separated
by a border
imgarray:
3D array of 2D images (n_images, rows, cols)
nrows:
the number of rows of images in the output array. if
unspecified, nrows = ceil(sqrt(n_images))
border:
the border size separating images (px)
border_val:
the value of the border regions of the output array (np.nan
renders as transparent with imshow)
"""
dims = (imgarray.shape[0], imgarray.shape[1]+2*border,
imgarray.shape[2] + 2*border)
X = np.ones(dims, dtype=imgarray.dtype) * border_val
X[:,border:-border,border:-border] = imgarray
# array dims should be [imageno,r,c]
count, m, n = X.shape
if nrows != None:
mm = nrows
nn = int(np.ceil(count/nrows))
else:
mm = int(np.ceil(np.sqrt(count)))
nn = mm
M = np.ones((nn * n, mm * m)) * np.nan
image_id = 0
for j in xrange(mm):
for k in xrange(nn):
if image_id >= count:
break
sliceM, sliceN = j * m, k * n
img = X[image_id,:, :].T
M[sliceN:(sliceN + n), sliceM:(sliceM + m)] = img
image_id += 1
return np.flipud(np.rot90(M))
Example:
from scipy.misc import lena
from matplotlib import pyplot as plt
img = lena().astype(np.float32)
img -= img.min()
img /= img.max()
imgarray = np.sin(np.linspace(0, 2*np.pi, 25)[:, None, None] + img)
m = montage(imgarray)
plt.imshow(m, cmap=plt.cm.jet)
Reusing chunks from How do you split a list into evenly sized chunks? :
def chunks(l, n):
""" Yield successive n-sized chunks from l.
"""
for i in xrange(0, len(l), n):
yield l[i:i+n]
Rewriting your function:
def plotimgs(ls):
shp = ls[0].shape[0] # the image's dimension
dim = int(np.ceil(sqrt(len(ls)))) # the amount of pictures per row AND column
emptyimg = (ls[1]*0 + 1)*255 # used to add to the list to allow square matrix
ls.extend((dim **2 - ls) * [emptyimg]) # filling the list with missing images
newimg = np.concatenate([np.concatenate(c, axis=0) for c in chunks(ls, dim)], axis=1)
cv2.imshow("frame", newimg)
cv2.waitKey(10)
plotimgs([a,b,d])