Related
After a long time of researching and asking questions, I have made my prototype code that makes a collage of a list of photos given as a list of strs.
It resizes the images according to the positions of the images in the list, then randomly rotates the images and randomly arrange them in a minimum bounding area.
It uses cv2, numpy, PIL and rpack, to be honest I have absolutely no idea how these libraries work, why my code is working, I just know how to make them work, I only know how to put them together.
So here is my code:
import cv2
import numpy as np
import random
import rpack
from fractions import Fraction
from math import prod
from pathlib import Path
from PIL import Image
from typing import Tuple
folder = 'D:/test/'
images = [
'Mass Effect.jpg',
'Dragon Age Origins.jpg',
'Life Is Strange.jpg',
'Star Wars KOTOR.jpg',
'Dragon Age 2.jpg',
'Choice of Robots.jpg',
'Perfect Match.png',
'Jade Empire.jpg',
"Serafina's Saga.jpg",
'Rising Angels Reborn.jpg',
'Across The Void.png',
"Heart's Blight.png",
'The Gray Wolf And The Little Lamb.jpg',
'Night of the Lesbian Vampires.png',
'Tethered.png',
'Contract Demon.jpg',
"Yuki's 4P.png"
]
def resize_guide(image_size: Tuple[int, int], unit_shape: Tuple[int, int], target_ratio: float) -> Tuple[int, int]:
aspect_ratio = Fraction(*image_size).limit_denominator()
horizontal = aspect_ratio.numerator
vertical = aspect_ratio.denominator
target_area = prod(unit_shape) * target_ratio
unit_length = (target_area/(horizontal*vertical))**.5
return (int(horizontal*unit_length), int(vertical*unit_length))
images = [cv2.imread(folder+name) for name in images]
size_hint = [i**.75 for i in range(1, len(images)+1)][::-1]
resized_images = []
for image, hint in zip(images, size_hint):
height, width = image.shape[:2]
guide = resize_guide((width, height), (640,360), hint)
resized = cv2.resize(image, guide, interpolation = cv2.INTER_AREA)
resized_images.append(resized)
def make_border(image, value, border=16):
return cv2.copyMakeBorder(
image,
top=border,
bottom=border,
left=border,
right=border,
borderType=cv2.BORDER_CONSTANT,
value=value
)
def rotate_image(image, angle):
h, w = image.shape[:2]
cX, cY = (w // 2, h // 2)
M = cv2.getRotationMatrix2D((cX, cY), -angle, 1.0)
cos = np.abs(M[0, 0])
sin = np.abs(M[0, 1])
nW = int((h * sin) + (w * cos))
nH = int((h * cos) + (w * sin))
M[0, 2] += (nW / 2) - cX
M[1, 2] += (nH / 2) - cY
return cv2.warpAffine(image, M, (nW, nH))
rotated_images = []
sizes = []
for image in resized_images:
image = make_border(image, (255, 255, 255))
rotated = rotate_image(image, random.randrange(-15, 16))
image = make_border(image, (0,0,0))
rotated_images.append(rotated)
height, width = rotated.shape[:2]
sizes.append((width, height))
shapes = [(x, y, w, h) for (x, y), (w, h) in zip(rpack.pack(sizes), sizes)]
rightmost = sorted(shapes, key=lambda x: -x[0] - x[2])[0]
bound_width = rightmost[0] + rightmost[2]
downmost = sorted(shapes, key=lambda x: -x[1] - x[3])[0]
bound_height = downmost[1] + downmost[3]
collage = np.zeros([bound_height, bound_width, 3],dtype=np.uint8)
for image, (x, y, w, h) in zip(rotated_images, shapes):
collage[y:y+h, x:x+w] = image
collage = Image.fromarray(collage, 'RGB')
collage.save('D:/collages/' + random.randbytes(4).hex() + '.png')
Because the output is way too large (over 20 MiB) it can't fit here, I have uploaded it to Google Drive: https://drive.google.com/file/d/16w4wsC_od4dh4QI7BYj8MM2gMngbSLV1/view?usp=sharing
So far the results seem promising, the only complaint I have is that the colors look very strange, I swear the original images have normal colors.
Can someone please tell me what I did wrong?
OK so while executing the code, the interpreter complained a lot about:
libpng warning: iCCP: known incorrect sRGB profile
I used cracked Adobe Photoshop CS6 to edit the images, is this the source of the problem or is it something else?
You are doing everything just fine. The only mistake you are making is while storing the image. Just remove the last two lines and add the following line.
cv2.imwrite('D:/collages/' + random.randbytes(4).hex() + '.png', collage)
I got output like below after stitching result of 24 stitched images to next 25th image. Before that stitching was good.
Is anyone aware of why/when output of stitching comes like this? What are the possibilities of output coming like that? What may be the reason of that?
Stitching code is following standard stitching steps like finding keypoints, descriptors then matching points, calculating homography and then warping of images. But I am not understanding why that output is coming.
Core part of stitching is like below:
detector = cv2.SIFT_create(400)
# find the keypoints and descriptors with SIFT
gray1 = cv2.cvtColor(image1,cv2.COLOR_BGR2GRAY)
ret1, mask1 = cv2.threshold(gray1,1,255,cv2.THRESH_BINARY)
kp1, descriptors1 = detector.detectAndCompute(gray1,mask1)
gray2 = cv2.cvtColor(image2,cv2.COLOR_BGR2GRAY)
ret2, mask2 = cv2.threshold(gray2,1,255,cv2.THRESH_BINARY)
kp2, descriptors2 = detector.detectAndCompute(gray2,mask2)
keypoints1Im = cv2.drawKeypoints(image1, kp1, outImage = cv2.DRAW_MATCHES_FLAGS_DEFAULT, color=(0,0,255))
keypoints2Im = cv2.drawKeypoints(image2, kp2, outImage = cv2.DRAW_MATCHES_FLAGS_DEFAULT, color=(0,0,255))
# BFMatcher with default params
matcher = cv2.BFMatcher()
matches = matcher.knnMatch(descriptors2,descriptors1, k=2)
# Apply ratio test
good = []
for m, n in matches:
if m.distance < 0.75 * n.distance:
good.append(m)
print (str(len(good)) + " Matches were Found")
if len(good) <= 10:
return image1
matches = copy.copy(good)
matchDrawing = util.drawMatches(gray2,kp2,gray1,kp1,matches)
#Aligning the images
src_pts = np.float32([ kp2[m.queryIdx].pt for m in matches ]).reshape(-1,1,2)
dst_pts = np.float32([ kp1[m.trainIdx].pt for m in matches ]).reshape(-1,1,2)
H = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC,5.0)[0]
h1,w1 = image1.shape[:2]
h2,w2 = image2.shape[:2]
pts1 = np.float32([[0,0],[0,h1],[w1,h1],[w1,0]]).reshape(-1,1,2)
pts2 = np.float32([[0,0],[0,h2],[w2,h2],[w2,0]]).reshape(-1,1,2)
pts2_ = cv2.perspectiveTransform(pts2, H)
pts = np.concatenate((pts1, pts2_), axis=0)
# print("pts:", pts)
[xmin, ymin] = np.int32(pts.min(axis=0).ravel() - 0.5)
[xmax, ymax] = np.int32(pts.max(axis=0).ravel() + 0.5)
t = [-xmin,-ymin]
Ht = np.array([[1,0,t[0]],[0,1,t[1]],[0,0,1]]) # translate
result = cv2.warpPerspective(image2, Ht.dot(H), (xmax-xmin, ymax-ymin))
resizedB = np.zeros((result.shape[0], result.shape[1], 3), np.uint8)
resizedB[t[1]:t[1]+h1,t[0]:w1+t[0]] = image1
# Now create a mask of logo and create its inverse mask also
img2gray = cv2.cvtColor(result,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 0, 255, cv2.THRESH_BINARY)
kernel = np.ones((5,5),np.uint8)
k1 = (kernel == 1).astype('uint8')
mask = cv2.erode(mask, k1, borderType=cv2.BORDER_CONSTANT)
mask_inv = cv2.bitwise_not(mask)
difference = cv2.bitwise_or(resizedB, resizedB, mask=mask_inv)
result2 = cv2.bitwise_and(result, result, mask=mask)
result = cv2.add(result2, difference)
Edit:
This image shows match drawing while stitching 25 to result until 24 images:
And before that match drawing:
I have total 97 images to stitch. If I stitch 24 and 25 image separately they stitches properly. If I start stitching from 23rd image onwards then also stitching is good but it gives me problem when I stitches starting from 1st image. I am not able to understand the problem.
Result after stitching 23rd image:
Result after stitching 24th image:
Result after stitching 25th image is as above which went wrong.
Strange Observation: If I stitch 23,24,25 images seperately with same code it gets stitches. If I stitch images after 23 till 97 , it gets stitches. But somehow if I stitch images from 1st, it breaks while stitching 25th image. I am not understanding why this happens.
I have tried different combination like different keypoint detection, extraction methods, matching methods, different homography calculations, different warping code but those combinations didn't work. Something is missing or wrong in the steps combination code. I am not able to figure it out.
Sorry for this long question. As I am completely new to this I am not able to explain and get the things properly. Thanks for your help and guidance.
Stitched result of 23,24,25 images separately with SAME code:
With different code (gives black lines in between stitching), if I stitched 97 images then 25th goes up in stitching and stitches as shown below (right corner point):
Firstly, I was not able to recreate your problem and solve it as the images were too big for my system to process. However, I had faced the same problem in my Panorama Stitching project, so I am sharing the reason behind it and my approach to solving my problem. Hope this helps you too.
Here's what my problem looked like when I stitched 4 images together just like you did.
As you can see, the 4th image was getting distorted a lot which must not happen. The same thing happened with you but on a greater level.
Now, here's the output when I stitched 8 images after some image pre-processing.
After some pre-processing on the input images, I was able to stitch 8 images together perfectly without any distortion.
To understand the exact reason behind this kind of distortion, watch this video by Joseph Redmon between 50:26 - 1:07:23.
As suggested in the video, we'll first have to project the images onto a cylinder and then unroll them and then stitch these unrolled images together.
Below is the initial input image(left) and the image after projection and unrolling onto a cylinder(right).
For your problem, as you are using satellite images, I guess projection onto a sphere would work better than the cylinder however you'll have to give it a try.
Sharing below my code for projecting the image onto a cylinder and unrolling it for reference. The mathematics used behind it is the same as given in the video.
def Convert_xy(x, y):
global center, f
xt = ( f * np.tan( (x - center[0]) / f ) ) + center[0]
yt = ( (y - center[1]) / np.cos( (x - center[0]) / f ) ) + center[1]
return xt, yt
def ProjectOntoCylinder(InitialImage):
global w, h, center, f
h, w = InitialImage.shape[:2]
center = [w // 2, h // 2]
f = 1100 # 1100 field; 1000 Sun; 1500 Rainier; 1050 Helens
# Creating a blank transformed image
TransformedImage = np.zeros(InitialImage.shape, dtype=np.uint8)
# Storing all coordinates of the transformed image in 2 arrays (x and y coordinates)
AllCoordinates_of_ti = np.array([np.array([i, j]) for i in range(w) for j in range(h)])
ti_x = AllCoordinates_of_ti[:, 0]
ti_y = AllCoordinates_of_ti[:, 1]
# Finding corresponding coordinates of the transformed image in the initial image
ii_x, ii_y = Convert_xy(ti_x, ti_y)
# Rounding off the coordinate values to get exact pixel values (top-left corner)
ii_tl_x = ii_x.astype(int)
ii_tl_y = ii_y.astype(int)
# Finding transformed image points whose corresponding
# initial image points lies inside the initial image
GoodIndices = (ii_tl_x >= 0) * (ii_tl_x <= (w-2)) * \
(ii_tl_y >= 0) * (ii_tl_y <= (h-2))
# Removing all the outside points from everywhere
ti_x = ti_x[GoodIndices]
ti_y = ti_y[GoodIndices]
ii_x = ii_x[GoodIndices]
ii_y = ii_y[GoodIndices]
ii_tl_x = ii_tl_x[GoodIndices]
ii_tl_y = ii_tl_y[GoodIndices]
# Bilinear interpolation
dx = ii_x - ii_tl_x
dy = ii_y - ii_tl_y
weight_tl = (1.0 - dx) * (1.0 - dy)
weight_tr = (dx) * (1.0 - dy)
weight_bl = (1.0 - dx) * (dy)
weight_br = (dx) * (dy)
TransformedImage[ti_y, ti_x, :] = ( weight_tl[:, None] * InitialImage[ii_tl_y, ii_tl_x, :] ) + \
( weight_tr[:, None] * InitialImage[ii_tl_y, ii_tl_x + 1, :] ) + \
( weight_bl[:, None] * InitialImage[ii_tl_y + 1, ii_tl_x, :] ) + \
( weight_br[:, None] * InitialImage[ii_tl_y + 1, ii_tl_x + 1, :] )
# Getting x coorinate to remove black region from right and left in the transformed image
min_x = min(ti_x)
# Cropping out the black region from both sides (using symmetricity)
TransformedImage = TransformedImage[:, min_x : -min_x, :]
return TransformedImage, ti_x-min_x, ti_y
You just have to call the function ProjectOntoCylinder and pass it an image to get the resultant image and the coordinates of white pixels in the mask image. Use the code below to call this function and get the mask image.
# Applying Cylindrical projection on Image
Image_Cyl, mask_x, mask_y = ProjectOntoCylinder(Image)
# Getting Image Mask
Image_Mask = np.zeros(Image_Cyl.shape, dtype=np.uint8)
Image_Mask[mask_y, mask_x, :] = 255
Here are links to my project and its detailed documentation for reference:
Part 1:
Source Code,
Documentation
Part 2:
Source Code,
Documentation
This is similar to nkint's question from September 11, 2013. Link is here:
how to get all undistorted image with opencv
I'm a new user, so I didn't have enough reputation/clout to comment on the OP.
I have tried to emulate the code andrewmkeller posted, using Python instead of C++, with some minor changes based on Josh Bosch's response. The result is the following:
#!/usr/bin/env python
import cv2
import numpy as np
def loadUndistortedImage(fileName):
# load image
image = cv2.imread(fileName)
#print(image)
# set distortion coeff and intrinsic camera matrix (focal length, centerpoint offset, x-y skew)
cameraMatrix = np.array([[894.96803896,0,470.38713516],[0,901.32629374,922.41232898], [0,0,1]])
distCoeffs = np.array([[-0.340671222,0.110426603,-.000867987573,0.000189669273,-0.0160049526]])
# setup enlargement and offset for new image
y_shift = 60 #experiment with
x_shift = 70 #experiment with
imageShape = image.shape #image.size
print(imageShape)
imageSize = (int(imageShape[0])+2*y_shift, int(imageShape[1])+2*x_shift, 3)
print(imageSize)
# create a new camera matrix with the principal point offest according to the offset above
newCameraMatrix, validPixROI = cv2.getOptimalNewCameraMatrix(cameraMatrix, distCoeffs, imageSize,
1)
#newCameraMatrix = cv2.getDefaultNewCameraMatrix(cameraMatrix, imageSize, True) # imageSize, True
# create undistortion maps
R = np.array([[1,0,0],[0,1,0],[0,0,1]])
map1, map2 = cv2.initUndistortRectifyMap(cameraMatrix, distCoeffs, R, newCameraMatrix, imageSize,
cv2.CV_16SC2)
# remap
outputImage = cv2.remap(image, map1, map2, INTER_LINEAR)
#save output image as file with "FIX" appened to name - only works with .jpg files at the moment
index = filename.find('.jpg')
fixed_filename = filename[:index] +'_undistorted'+fileName[index:]
cv2.imwrite(fixed_filename, outputImage)
cv2.imshow('fix_img',outputImage)
cv2.waitKey(0)
return
#Undistort the images, then save the restored images
loadUndistortedImage('./calib/WIN_20200626_11_29_16_Pro.jpg')
This seemed good to me, but then problems came up when trying to use cv2.getOptimalNewCameraMatrix or cv2.getDefaultNewCameraMatrix and cv2.initUndistortRectifyMap. I kept getting told that 'the argument takes exactly 2 arguments (3 given)' even though I am putting the parameters as specified in their documentation here:
https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html
https://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html
I can remove the error from "...getDefault..." if I remove the optional params, but I'd rather not do that.
Stacktrace:
Traceback (most recent call last):
File ".\main.py", line 46, in <module>
loadUndistortedImage('./<image file name>.jpg')
File ".\main.py", line 27, in loadUndistortedImage
newCameraMatrix, validPixROI = cv2.getOptimalNewCameraMatrix(cameraMatrix, distCoeffs, imageSize, 1)
TypeError: function takes exactly 2 arguments (3 given)
I don't have enough reputation to comment, but you could try:
newcameramatrix, _ = cv2.getOptimalCameraMatrix(
camera_matrix, dist_coeffs, (width, height), 1, (width, height)
)
According to this, that's how the function should be called.
Now, instead of getting the undistorted image with cv2.initUndistortRectifyMap, you could just do:
undistorted_image = cv2.undistort(
image, camera_matrix, dist_coeffs, None, newcameramatrix
)
cv2.imshow("undistorted", undistorted_image)
Following up to my comment on Sebastian Liendo's answer, and also thanks to a Finnish responder on Github (whose Issues are not for these sort of general questions, I learned), here is 1) the updated documentation for the python functions, and 2) the heart of my revised code which does a decent job of getting around the cropping. (Don't do what I did in the question and post the ENTIRE code, just the part essential to your question.)
https://docs.opencv.org/4.3.0/d9/d0c/group_calib3d.html#ga7a6c4e032c97f03ba747966e6ad862b1
#load image
image = cv2.imread(fileName)
#images = glob.glob(pathName + '*.jpg') #loop within a specified directory
#for fileName in images:
#image = cv2.imread(fileName)
#set camera parameters
height, width = image.shape[:2]
cameraMatrix = np.array([[894.96803896,0,470.38713516],[0,901.32629374,922.41232898], [0,0,1]])
distCoeffs = np.array([[-0.340671222,0.110426603,-.000867987573,0.000189669273,-0.0160049526]])
#create new camera matrix
newCameraMatrix, validPixROI = cv2.getOptimalNewCameraMatrix(cameraMatrix, distCoeffs,(width, height), 1, (width, height))
#undistort
outputImage = cv2.undistort(image, cameraMatrix, distCoeffs, None, newCameraMatrix)
#crop, modified
x, y, w, h = validPixROI #(211, 991, 547, 755)
outputImage = outputImage[y-200:y+h+200, x-40:x+w+80] #fudge factor to minimize cropping
THE ONE CAVEAT: this code still crops a bit of the outer-trim of the original capture, but not by much. Minimizing that cropping is the reason for the fudge factor I put in the ouputImage = outputImage[...] line.
I came across the following code:
import cv2
image = cv2.imread('xyz.jpg')
(h, w) = image.shape[:2]
# compute the center of the image
(cX, cY) = (w / 2, h / 2)
region = image[0:cY, 0:cX]
How can we read this part image[0:cY, 0:cX]? I tried printing out the result, but got an array printed out that couldn't help me understand the idea.
So, for instance, let's say that the center of the image was (50,50), how would region be defined in this case?
Thanks.
I'm trying to write a Python function to crop, rotate and resize faces. It is for a facial recognition application.
I pass the coordinates of the eyes to the function and the function processes the image (rotate it so the plane of the eyes is parallel to the horizontal axis of the image and scale/crop/resize it).
The problem is that the faces are not rotating at all. They are only being cropped.
The following function is modified to return both the rotated image and a copy of the image done before the rotation. They are identical.
def CropFace(image, eye_left=(0,0), eye_right=(0,0), offset_pct=(0.25,0.25), dest_sz = (250,250)):
offset_h = math.floor(float(offset_pct[0])*dest_sz[0])
offset_v = math.floor(float(offset_pct[1])*dest_sz[1])
eye_direction = (eye_right[0] - eye_left[0], eye_right[1] - eye_left[1])
rotation = -math.atan2(float(eye_direction[1]), float(eye_direction[0]))
dist = Distance(eye_left, eye_right)
reference = dest_sz[0] - 2.0*offset_h
scale = float(dist) / float(reference)
sz = image.shape
if len(sz) > 2: sz = sz[:2]
print rotation
image2 = image.copy()
mat = cv2.getRotationMatrix2D(eye_left, rotation, 1.0)
result = cv2.warpAffine(image, mat, sz, flags = cv2.INTER_CUBIC)
crop_xy = (eye_left[0] - scale*offset_h, eye_left[1] - scale*offset_v)
crop_size = (dest_sz[0]*scale, dest_sz[1]*scale)
result = result[int(crop_xy[1]):int(crop_xy[1]+crop_size[1]), int(crop_xy[0]):int(crop_xy[0]+crop_size[0])]
image2 = image2[int(crop_xy[1]):int(crop_xy[1]+crop_size[1]), int(crop_xy[0]):int(crop_xy[0]+crop_size[0])]
return (result, image2)
The problem is that for
cv2.getRotationMatrix2D(center, angle, scale)
the angle argument is in degrees (opencv documentation)
while Python,
angle = math.atan2(y, x)
returns the angle in radians. (Python documentation)
So the angle specified by rotation was in radians when OpenCV was expecting degrees.