Reversing the translation of the Pillow Image Library Rotation - python

I'm a bit of newbie at this, trying to rotate an image in Python Pillow without changing the position of the centre of the rotated image. OR by the pillow rotate looks of things... returning the centre back to it's original spin location.
In Pillow (Image.py) there is a function that rotates an image. This function is as follows:-
def rotate(
self,
angle,
resample=NEAREST,
expand=0,
center=None,
translate=None,
fillcolor=None,
):
"""
Returns a rotated copy of this image. This method returns a
copy of this image, rotated the given number of degrees counter
clockwise around its centre.
:param angle: In degrees counter clockwise.
:param resample: An optional resampling filter. This can be
one of :py:attr:`PIL.Image.NEAREST` (use nearest neighbour),
:py:attr:`PIL.Image.BILINEAR` (linear interpolation in a 2x2
environment), or :py:attr:`PIL.Image.BICUBIC`
(cubic spline interpolation in a 4x4 environment).
If omitted, or if the image has mode "1" or "P", it is
set to :py:attr:`PIL.Image.NEAREST`. See :ref:`concept-filters`.
:param expand: Optional expansion flag. If true, expands the output
image to make it large enough to hold the entire rotated image.
If false or omitted, make the output image the same size as the
input image. Note that the expand flag assumes rotation around
the center and no translation.
:param center: Optional center of rotation (a 2-tuple). Origin is
the upper left corner. Default is the center of the image.
:param translate: An optional post-rotate translation (a 2-tuple).
:param fillcolor: An optional color for area outside the rotated image.
:returns: An :py:class:`~PIL.Image.Image` object.
"""
angle = angle % 360.0
# Fast paths regardless of filter, as long as we're not
# translating or changing the center.
if not (center or translate):
if angle == 0:
return self.copy()
if angle == 180:
return self.transpose(ROTATE_180)
if angle == 90 and expand:
return self.transpose(ROTATE_90)
if angle == 270 and expand:
return self.transpose(ROTATE_270)
# Calculate the affine matrix. Note that this is the reverse
# transformation (from destination image to source) because we
# want to interpolate the (discrete) destination pixel from
# the local area around the (floating) source pixel.
# The matrix we actually want (note that it operates from the right):
# (1, 0, tx) (1, 0, cx) ( cos a, sin a, 0) (1, 0, -cx)
# (0, 1, ty) * (0, 1, cy) * (-sin a, cos a, 0) * (0, 1, -cy)
# (0, 0, 1) (0, 0, 1) ( 0, 0, 1) (0, 0, 1)
# The reverse matrix is thus:
# (1, 0, cx) ( cos -a, sin -a, 0) (1, 0, -cx) (1, 0, -tx)
# (0, 1, cy) * (-sin -a, cos -a, 0) * (0, 1, -cy) * (0, 1, -ty)
# (0, 0, 1) ( 0, 0, 1) (0, 0, 1) (0, 0, 1)
# In any case, the final translation may be updated at the end to
# compensate for the expand flag.
w, h = self.size
if translate is None:
post_trans = (0, 0)
else:
post_trans = translate
if center is None:
# FIXME These should be rounded to ints?
rotn_center = (w / 2.0, h / 2.0)
else:
rotn_center = center
angle = -math.radians(angle)
matrix = [
round(math.cos(angle), 15),
round(math.sin(angle), 15),
0.0,
round(-math.sin(angle), 15),
round(math.cos(angle), 15),
0.0,
]
def transform(x, y, matrix):
(a, b, c, d, e, f) = matrix
return a * x + b * y + c, d * x + e * y + f
matrix[2], matrix[5] = transform(
-rotn_center[0] - post_trans[0], -rotn_center[1] - post_trans[1], matrix
)
matrix[2] += rotn_center[0]
matrix[5] += rotn_center[1]
if expand:
# calculate output size
xx = []
yy = []
for x, y in ((0, 0), (w, 0), (w, h), (0, h)):
x, y = transform(x, y, matrix)
xx.append(x)
yy.append(y)
nw = math.ceil(max(xx)) - math.floor(min(xx))
nh = math.ceil(max(yy)) - math.floor(min(yy))
# We multiply a translation matrix from the right. Because of its
# special form, this is the same as taking the image of the
# translation vector as new translation vector.
matrix[2], matrix[5] = transform(-(nw - w) / 2.0, -(nh - h) / 2.0, matrix)
w, h = nw, nh
return self.transform((w, h), AFFINE, matrix, resample, fillcolor=fillcolor)
This function also applies some translation (position shift) in order to keep the rotated images corners inside the image. The part of the code that applies the translation is this line
matrix[2], matrix[5] = transform(-(nw - w) / 2.0, -(nh - h) / 2.0, matrix)
what I would like to do is extract the values of matrix[2] and matrix[5] so that I can reverse this translation, when the rotation is called in moviepy.
To achieve something like this...
import moviepy.editor as mped
image_clip = mped.ImageClip("image.jpg", duration=3)
rotated_image = image_clip.rotate(20).set_position((pillow_rotate_x.
(-matrix[2]),pillow_rotate_y.(-matrix[5]))
So that it undoes the pillow translation, and returns the image centre to the place it was originally rotated at.
I was wondering how this could be achieved with the least code repetition ?
For example with the following code:-
import moviepy.editor as mped
import sys
import numpy as np
print("Python Version", sys.version)
baboon = mped.ImageClip("baboon.png", duration=3)
colour_clip = mped.ColorClip(size=[500, 50], color=np.array([250, 90, 0]).astype(np.uint8), duration=3) # important to use .astype(np.uint8)
cameraman = mped.ImageClip("cameraman.jpg", duration=3)
print("baboon_size", baboon.size)
print("colour_clip size", colour_clip.size)
print("cameraman size", cameraman.size)
rot_trans_col_clip = colour_clip.add_mask().rotate(20)
rot_trans_cameraman = cameraman.add_mask().rotate(20)
stacked_clips = mped.CompositeVideoClip([baboon, rot_trans_col_clip, rot_trans_cameraman])
stacked_clips.write_videofile('rotated_imagery_on_baboon.mp4', fps=5)
With the above code you can layer up some different types of content and rotate them.
The two input image files of the baboon and cameraman can be downloaded here:-
https://drive.google.com/file/d/17_s1IunwIAy1npJrsLRicieTG4NZYV4o/view?usp=sharing
https://drive.google.com/file/d/1G5YbApGX035-9mJtuz9GNgLr6jGywk-Z/view?usp=sharing
With the Translation code below (that is inside the pillow image.py file)
matrix[2], matrix[5] = transform(-(nw - w) / 2.0, -(nh - h) / 2.0, matrix)
the affect it has on the images is illustrated here:-
https://drive.google.com/file/d/1d_prYqb-fqizFcV0MD0rMXOIny2L0KW5/view?usp=sharing
You can see here that the centres of the two rotated images have been moved, so that their corners are still in view (not cropped).
Without the Pillow Translation code inside the pillow rotation function it looks like this:-
https://drive.google.com/file/d/17POoZcuk9QAxJrnwD2LFsYd--SXdR9JA/view?usp=sharing
You can see here that although the corners are a bit cropped, the centres of the images have not moved.
This is the outcome that I want. However, Pillow rotate applies a translation at the end.
Interestingly if you set expand=False on the pillow rotate:-
rot_trans_cameraman = cameraman.add_mask().rotate(20, unit='deg', expand=False)
you get this:-
https://drive.google.com/open?id=1QEzJN3NlWK_sjxPLGC_BNs2xfxxfhAIH
which has the same centre points. So it seems that without the expand flag set to false, the centre points move, however with it set to false, all the corners are cropped in symmetry.
The reason that this would be useful, is so that if you pass an angle to pillow rotate the outcome is deterministic, rather than also incorporating a translation that is dependent on the size of the images.
So my question is, how to restore the rotation centre locations ?

The answer to this question is here:-
https://github.com/python-pillow/Pillow/issues/4556
The new reference is after the expansion and centre reposition. This can then be used in the global system to reposition the element.

Related

how to apply a affine transformation to a single dot in PIL?

In PIL I have rotated a rectangle by an angle and pasted it back to the background image.
I'd like to know what will be the new coordinates of a specific corner.
function(old_coord) => new coordinates
I have read in the documentation it mentioned the center of rotation by default is the center of the image.
Here is my code:
import PIL
from PIL import Image, ImageDraw
import os
background = Image.open('background.jpg')
rectangle = Image.open("rectangle.png")
my_angle = 30
rectangle_rotate = Image.Image.rotate(rectangle, angle=my_angle, resample=Image.BICUBIC, expand=True)
# box: 2-tuple giving the upper left corner
px = int(background.size[0] / 2)
py = int(background.size[1] / 2)
background.paste(im=rectangle_rotate,
box=(px, py),
mask=rectangle_rotate)
# The new position I'm getting is wrong, how come?????
pos_xNew = px * math.cos(math.radians(my_angle)) + py * math.sin(math.radians(my_angle))
pos_yNew = -px * math.sin(math.radians(my_angle)) + py * math.cos(math.radians(my_angle))
print('pos_xNew:', pos_xNew)
print('pos_yNew:', pos_yNew)
draw_img_pts = ImageDraw.Draw(background)
r = 10
# Drawing a simple small circle circle for visualization
draw_img_pts.ellipse((pos_xNew - r, pos_yNew - r, pos_xNew + r, pos_yNew + r), fill='red')
background.save('example_with_roatation.png')
how can I find the new coordinates value? I keep getting wrong value.
Background image (input):
Rectangle image (input):
The Output I got with zero rotation as expected:
Output I got after 30 degree rotation:
Comments inline
import math
import os
import numpy as np # many operations are more concise in matrix form
import PIL
from PIL import Image, ImageDraw
def get_rotation_matrix(angle):
""" For background, https://en.wikipedia.org/wiki/Rotation_matrix
rotation is clockwise in traditional descartes, and counterclockwise,
if y goes down (as in picture coordinates)
"""
return np.array([
[np.cos(angle), -np.sin(angle)],
[np.sin(angle), np.cos(angle)]])
def rotate(points, pivot, angle):
""" Get coordinates of points rotated by a given angle counterclocwise
Args:
points (np.array): point coordinates shaped (n, 2)
pivot (np.array): [x, y] coordinates of rotation center
angle (float): counterclockwise rotation angle in radians
Returns:
np.array of new coordinates shaped (n, 2)
"""
relative_points = points - pivot
return relative_points.dot(get_rotation_matrix(angle)) + pivot
background = Image.open('background.jpg')
rectangle = Image.open("rectangle.png")
my_angle_deg = 30
my_angle = math.radians(my_angle_deg)
rsize_x, rsize_y = rectangle.size
# to get shift introduced by rotation+clipping we'll need to rotate all four corners
# starting from top-right corners, counter-clockwise
rectangle_corners = np.array([
[rsize_x, 0], # top-right
[0, 0], # top-left
[0, rsize_y], # bottom-left
[rsize_x, rsize_y] # bottom-right
])
# rectangle_corners now are:
# array([[262, 0],
# [ 0, 0],
# [ 0, 67],
# [262, 67]])
rotated_corners = rotate(rectangle_corners, rectangle_corners[0], my_angle)
# as a result of rotation, one of the corners might end up left from 0,
# e.g. if the rectangle is really tall and rotated 90 degrees right
# or, leftmost corner is no longer at 0, so part of the canvas is clipped
shift_introduced_by_rotation_clip = rotated_corners.min(axis=0)
rotated_shifted_corners = rotated_corners - shift_introduced_by_rotation_clip
# finally, demo
# this part is just copied from the question
rectangle_rotate = Image.Image.rotate(rectangle, angle=my_angle_deg, resample=Image.BICUBIC, expand=True)
# box: 2-tuple giving the upper left corner
px = int(background.size[0] / 2)
py = int(background.size[1] / 2)
background.paste(im=rectangle_rotate,
box=(px, py),
mask=rectangle_rotate)
# let's see if dots land right after these translations:
draw_img_pts = ImageDraw.Draw(background)
r = 10
for point in rotated_shifted_corners:
pos_xNew, pos_yNew = point + [px, py]
draw_img_pts.ellipse((pos_xNew - r, pos_yNew - r, pos_xNew + r, pos_yNew + r), fill='red')

Fast and Robust Image Stitching Algorithm for many images in Python?

I have a stationary camera which takes photos rapidly of the continuosly moving product but in a fixed position just of the same angle (translation perspective). I need to stitch all images into a panoramic picture. I've tried by using the class Stitcher. It worked, but it took a long time to compute.
I also tried to use another method by using the SIFT detector, FNNbasedMatcher, finding Homography and then warping the images. This method works fine if I only use two images. For multiple images it still doesn't stitch them properly. Does anyone know the best and fastest image stitching algorithm for this case?
This is my code which uses the Stitcher class.
import time
import cv2
import os
import numpy as np
import sys
def main():
# read input images
imgs = []
path = 'pics_rotated/'
i = 0
for (root, dirs, files) in os.walk(path):
images = [f for f in files]
print(images)
for i in range(0,len(images)):
curImg = cv2.imread(path + images[i])
imgs.append(curImg)
stitcher = cv2.Stitcher.create(mode= 0)
status ,result = stitcher.stitch(imgs)
if status != cv2.Stitcher_OK:
print("Can't stitch images, error code = %d" % status)
sys.exit(-1)
cv2.imwrite("imagesout/output.jpg", result)
cv2.waitKey(0)
if __name__ == '__main__':
start = time.time()
main()
end = time.time()
print("Time --->>>>>", end - start)
cv2.destroyAllWindows()enter code here
Briefing
Although OpenCV Stitcher class provides lots of methods and options to perform stitching, I find it hard to use it because of the complexity.
Therefore, I will try to provide the minimum and fastest way to perform stitching.
In case you are wondering more sophisticated approachs such as exposure compensation, I highly recommend looking at the detailed sample code.
As a side note, I will be grateful if someone can convert the following functions to use Stitcher class.
Introduction
In order to combine multiple images into the same perspective, the following operations are needed:
Detect and match features.
Compute homography (perspective transform between frames).
Warp one image onto the other perspective.
Combine the base and warped images while keeping track of the shift in origin.
Given the combination pattern, stitch multiple images.
Feature detection and matching
What are features?
They are distinguishable parts, like corners of a square, that are preserved across images.
There are different algorithms proposed for obtaining these characteristic points, like Harris, ORB, SIFT, SURF, etc.
See cv::Feature2d for the full list.
I will use SIFT because it is accurate and sufficiently fast.
A feature consists of a KeyPoint, which is the location in the image, and a descriptor, which is a set of numbers (e.g. a 128-D vector) that represents the properties of the feature.
After finding distinct points in images, we need to match the corresponding point pairs.
See cv::DescriptionMatcher.
I will use Flann-based descriptor matcher.
First, we initialize the descriptor and matcher classes.
descriptor = cv.SIFT.create()
matcher = cv.DescriptorMatcher.create(cv.DescriptorMatcher.FLANNBASED)
Then, we find the features in each image.
(kps, desc) = descriptor.detectAndCompute(image, mask=None)
Now we find the corresponding point pairs.
if (desc1 is not None and desc2 is not None and len(desc1) >=2 and len(desc2) >= 2):
rawMatch = matcher->knnMatch(desc2, desc1, k=2)
matches = []
# ensure the distance is within a certain ratio of each other (i.e. Lowe's ratio test)
ratio = 0.75
for m in rawMatch:
if len(m) == 2 and m[0].distance < m[1].distance * ratio:
matches.append((m[0].trainIdx, m[0].queryIdx))
Homography computation
Homography is the perspective transformation from one view to another.
The parallel lines in one view may not be parallel in another, like a road to sunset.
We need to have at least 4 corresponding point pairs.
The more means redundant data that have to be decomposed or eliminated.
Homography matrix that transforms the point in the initial view to its warped position.
It is a 3x3 matrix that is computed by Direct Linear Transform algorithm.
There are 8 DoF and the last element in the matrix is 1.
[pt2] = H * [pt1]
Now that we have corresponding point matches, we compute the homography.
The method we use to handle redundant data is RANSAC, which randomly selects 4 point pairs and uses the best fitting result.
See cv::findHomography for more options.
if len(matches) > 4:
(H, status) = cv.findHomography(pts1, pts2, cv.RANSAC)
Warping to perspective
By computing homography, we know which point in the source image corresponds to which point in the destination image.
In order not to lose information from the source image, we need to pad the destination image by the amount that the transformed point falls to negative regions.
At the same time, we need to keep track of the shift amount of the origin for stitching multiple images.
Auxilary functions
# find the ROI of a transformation result
def warpRect(rect, H):
x, y, w, h = rect
corners = [[x, y], [x, y + h - 1], [x + w - 1, y], [x + w - 1, y + h - 1]]
extremum = cv.transform(corners, H)
minx, miny = np.min(extremum[:,0]), np.min(extremum[:,1])
maxx, maxy = np.max(extremum[:,0]), np.max(extremum[:,1])
xo = int(np.floor(minx))
yo = int(np.floor(miny))
wo = int(np.ceil(maxx - minx))
ho = int(np.ceil(maxy - miny))
outrect = (xo, yo, wo, ho)
return outrect
# homography matrix is translated to fit in the screen
def coverH(rect, H):
# obtain bounding box of the result
x, y, _, _ = warpRect(rect, H)
# shift amount to the first quadrant
xpos = int(-x if x < 0 else 0)
ypos = int(-y if y < 0 else 0)
# correct the homography matrix so that no point is thrown out
T = np.array([[1, 0, xpos], [0, 1, ypos], [0, 0, 1]])
H_corr = T.dot(H)
return (H_corr, (xpos, ypos))
# pad image to cover ROI, return the shift amount of origin
def addBorder(img, rect):
x, y, w, h = rect
tl = (x, y)
br = (x + w, y + h)
top = int(-tl[1] if tl[1] < 0 else 0)
bottom = int(br[1] - img.shape[0] if br[1] > img.shape[0] else 0)
left = int(-tl[0] if tl[0] < 0 else 0)
right = int(br[0] - img.shape[1] if br[0] > img.shape[1] else 0)
img = cv.copyMakeBorder(img, top, bottom, left, right, cv.BORDER_CONSTANT, value=[0, 0, 0])
orig = (left, top)
return img, orig
def size2rect(size):
return (0, 0, size[1], size[0])
Warping function
def warpImage(img, H):
# tweak the homography matrix to move the result to the first quadrant
H_cover, pos = coverH(size2rect(img.shape), H)
# find the bounding box of the output
x, y, w, h = warpRect(size2rect(img.shape), H_cover)
width, height = x + w, y + h
# warp the image using the corrected homography matrix
warped = cv.warpPerspective(img, H_corr, (width, height))
# make the external boundary solid black, useful for masking
warped = np.ascontiguousarray(warped, dtype=np.uint8)
gray = cv.cvtColor(warped, cv.COLOR_RGB2GRAY)
_, bw = cv.threshold(gray, 1, 255, cv.THRESH_BINARY)
# https://stackoverflow.com/a/55806272/12447766
major = cv.__version__.split('.')[0]
if major == '3':
_, cnts, _ = cv.findContours(bw, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_NONE)
else:
cnts, _ = cv.findContours(bw, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_NONE)
warped = cv.drawContours(warped, cnts, 0, [0, 0, 0], lineType=cv.LINE_4)
return (warped, pos)
Combining warped and destination images
This is the step where image enhancement such as exposure compensation becomes involved.
In order to keep things simple, we will use mean value blending.
The easiest solution would be overriding the existing data in the destination image but averaging operation is not a burden for us.
# only the non-zero pixels are weighted to the average
def mean_blend(img1, img2):
assert(img1.shape == img2.shape)
locs1 = np.where(cv.cvtColor(img1, cv.COLOR_RGB2GRAY) != 0)
blended1 = np.copy(img2)
blended1[locs1[0], locs1[1]] = img1[locs1[0], locs1[1]]
locs2 = np.where(cv.cvtColor(img2, cv.COLOR_RGB2GRAY) != 0)
blended2 = np.copy(img1)
blended2[locs2[0], locs2[1]] = img2[locs2[0], locs2[1]]
blended = cv.addWeighted(blended1, 0.5, blended2, 0.5, 0)
return blended
def warpPano(prevPano, img, H, orig):
# correct homography matrix
T = np.array([[1, 0, -orig[0]], [0, 1, -orig[1]], [0, 0, 1]])
H_corr = H.dot(T)
# warp the image and obtain shift amount of origin
result, pos = warpImage(prevPano, H_corr)
xpos, ypos = pos
# zero pad the result
rect = (xpos, ypos, img.shape[1], img.shape[0])
result, _ = addBorder(result, rect)
# mean value blending
idx = np.s_[ypos : ypos + img.shape[0], xpos : xpos + img.shape[1]]
result[idx] = mean_blend(result[idx], img)
# crop extra paddings
x, y, w, h = cv.boundingRect(cv.cvtColor(result, cv.COLOR_RGB2GRAY))
result = result[y : y + h, x : x + w]
# return the resulting image with shift amount
return (result, (xpos - x, ypos - y))
Stitching multiple images given combination pattern
# base image is the last image in each iteration
def blend_multiple_images(images, homographies):
N = len(images)
assert(N >= 2)
assert(len(homographies) == N - 1)
pano = np.copy(images[0])
pos = (0, 0)
for i in range(N - 1):
img = images[i + 1]
# get homography matrix
H = homographies[i]
# warp pano onto image
pano, pos = warpPano(pano, img, H, pos)
return (pano, pos)
The method above warps the previously combined image, called pano, onto the next image subsequently.
A pattern, however, may have conjunction points for the best stitching view.
For example
1 2 3
4 5 6
The best pattern to combine these images is
1 -> 2 <- 3
|
V
4 -> 5 <- 6
Therefore, we need one last function to combine 1 & 2 with 2 & 3, or 1235 with 456 at node 5.
from operator import sub
# no warping here, useful for combining two different stitched images
# the image at given origin coordinates must be the same
def patchPano(img1, img2, orig1=(0,0), orig2=(0,0)):
# bottom right points
br1 = (img1.shape[1] - 1, img1.shape[0] - 1)
br2 = (img2.shape[1] - 1, img2.shape[0] - 1)
# distance from orig to br
diag2 = tuple(map(sub, br2, orig2))
# possible pano corner coordinates based on img1
extremum = np.array([(0, 0), br1,
tuple(map(sum, zip(orig1, diag2))),
tuple(map(sub, orig1, orig2))])
bb = cv.boundingRect(extremum)
# patch img1 to img2
pano, shift = addBorder(img1, bb)
orig = tuple(map(sum, zip(orig1, shift)))
idx = np.s_[orig[1] : orig[1] + img2.shape[0] - orig2[1],
orig[0] : orig[0] + img2.shape[1] - orig2[0]]
subImg = img2[orig2[1] : img2.shape[0], orig2[0] : img2.shape[1]]
pano[idx] = mean_blend(pano[idx], subImg)
return (pano, orig)
For a quick demo, you can run the Python code in GitHub.
If you want to use the above methods in C++, you can have a look at Stitch library.
Any PR or edit to this post is welcome.
As an alternative to the last step that #Burak gave, this is the way I used as I had the number of images for each of the rows (chunks), the multiStitching being nothing but a function to stitch images horizontally:
def stitchingImagesHV(img_list, size):
"""
As our multi stitching algorithm works on the horizontal line, we will hack
it to use also the vertical stitching by rotating each row "stitch_img" and
apply the same technique, and after that, the final result is rotated back to the
original direction.
"""
# Generate row chunks of "size" length from image list
chunks = [img_list[i:i + size] for i in range(0, len(img_list), size)]
list_rotated_images = []
for i in range(len(chunks)):
stitch_img = multiStitching(chunks[i])
stitch_img_rotated = cv2.rotate(stitch_img, cv2.ROTATE_90_COUNTERCLOCKWISE)
list_rotated_images.append(stitch_img_rotated.astype('uint8'))
stitch_img2 = multiStitching(list_rotated_images)
return cv2.rotate(stitch_img2, cv2.ROTATE_90_CLOCKWISE)

Python3: Resize rectangular image to a different kind of rectangle, keeping ratio and fill background with black

I have a very similar question to this: Resize rectangular image to square, keeping ratio and fill background with black, but I would like to resize to a nonsquare image and center the image either horizontally or vertically if needed.
Here are some examples of desired outputs. I made this image entirely with Paint, so the images might not actually be perfectly centered, but centering is what I'd like to achieve:
I tried the following code that I edited from the question linked:
def fix_size(fn, desired_w=256, desired_h=256, fill_color=(0, 0, 0, 255)):
"""Edited from https://stackoverflow.com/questions/44231209/resize-rectangular-image-to-square-keeping-ratio-and-fill-background-with-black"""
im = Image.open(fn)
x, y = im.size
#size = max(min_size, x, y)
w = max(desired_w, x)
h = max(desired_h, y)
new_im = Image.new('RGBA', (w, h), fill_color)
new_im.paste(im, ((w - x) // 2, (h - y) // 2))
return new_im.resize((desired_w, desired_h))
That doesn't work however as it still stretches some images into square shaped ones (at least the image b in the example. What comes to big images, it seems to rotate them instead!
The problem lies in your incorrect calculation of the image size:
w = max(desired_w, x)
h = max(desired_h, y)
You're simply taking the maximum of dimension independently - without taking into account the aspect ratio of the image. Imagine if your input is a square 1000x1000 image. You would end up creating a black 1000x1000 image, pasting the original image over it, and then resizing it to 244x138. To get the correct result, you would have to create a 1768x1000 image instead of a 1000x1000 image.
Here's the updated code that takes the aspect ratio into account:
def fix_size(fn, desired_w=256, desired_h=256, fill_color=(0, 0, 0, 255)):
"""Edited from https://stackoverflow.com/questions/44231209/resize-rectangular-image-to-square-keeping-ratio-and-fill-background-with-black"""
im = Image.open(fn)
x, y = im.size
ratio = x / y
desired_ratio = desired_w / desired_h
w = max(desired_w, x)
h = int(w / desired_ratio)
if h < y:
h = y
w = int(h * desired_ratio)
new_im = Image.new('RGBA', (w, h), fill_color)
new_im.paste(im, ((w - x) // 2, (h - y) // 2))
return new_im.resize((desired_w, desired_h))

Harris corner detection and localization in OpenCV with Python

I'm using the following code to try to detect corners of polylines in order to 'measure' the lines. The code is based on a snippet I found somewhere on SO and is based on cv2.cornerHarris():
cornerimg = cv2.cornerHarris( gray, # src
2, # blockSize
3, # ksize / aperture
0.04 # k
# dst
# borderType
)
# ?
cornerimg = cv2.normalize( cornerimg, # src
None, # dst
0, # alpha
255, # beta
cv2.NORM_MINMAX, # norm type
cv2.CV_32FC1, # dtype
None # mask
)
# ?
cornerimg = cv2.convertScaleAbs( cornerimg )
cornershow = cornerimg.copy()
# iterate over pixels to get corner positions
w, h = gray.shape
for y in range(0, h):
for x in range (0, w):
#harris = cv2.cv.Get2D( cv2.cv.fromarray(cornerimg), y, x)
#if harris[0] > 10e-06:
if cornerimg[x,y] > 64:
print("corner at ", x, y)
cv2.circle( cornershow, # dest
(x,y), # pos
4, # radius
(115,0,25) # color
)
cv2.imshow('harris cornerimg', cornershow)
The original code results in white spots at the corner location and the level seems to be an indicator of "corneryness".
The snippet (updated to use cv2) iterates over the resulting image and scans for values lager than 10e-06 for some reason, I have replaced this with a comparison of what I think should be the brightness in the image.
However, the circles drawn at those locations are nowhere near the actual hot-spots found in the normalized harris output.
What am I doing wrong?
Alternatively, cv2.goodFeaturesToTrack() can be set to use Harris (useHarrisDetector=True) but my attempt to use it does not result in what cornerHarris appears to detect properly:
cv2.goodFeaturesToTrack( blurred, # img
500, # maxCorners
0.03, # qualityLevel
10, # minDistance
None, # corners,
None, # mask,
2, # blockSize,
useHarrisDetector=True, # useHarrisDetector,
k=0.04 # k
)
What would be the equivalent function call to cv2.cornerHarris()?
The output seems to be transposed, swapping x and y indices on a square image fixes it (circles are on corner maxima).
try below:
cv2.circle( cornershow, # dest
(y,x), # pos
4, # radius
(115,0,25) # color
)

How can I draw a bezier curve using Python's PIL?

I'm using Python's Imaging Library and I would like to draw some bezier curves.
I guess I could calculate pixel by pixel but I'm hoping there is something simpler.
def make_bezier(xys):
# xys should be a sequence of 2-tuples (Bezier control points)
n = len(xys)
combinations = pascal_row(n-1)
def bezier(ts):
# This uses the generalized formula for bezier curves
# http://en.wikipedia.org/wiki/B%C3%A9zier_curve#Generalization
result = []
for t in ts:
tpowers = (t**i for i in range(n))
upowers = reversed([(1-t)**i for i in range(n)])
coefs = [c*a*b for c, a, b in zip(combinations, tpowers, upowers)]
result.append(
tuple(sum([coef*p for coef, p in zip(coefs, ps)]) for ps in zip(*xys)))
return result
return bezier
def pascal_row(n, memo={}):
# This returns the nth row of Pascal's Triangle
if n in memo:
return memo[n]
result = [1]
x, numerator = 1, n
for denominator in range(1, n//2+1):
# print(numerator,denominator,x)
x *= numerator
x /= denominator
result.append(x)
numerator -= 1
if n&1 == 0:
# n is even
result.extend(reversed(result[:-1]))
else:
result.extend(reversed(result))
memo[n] = result
return result
This, for example, draws a heart:
from PIL import Image
from PIL import ImageDraw
if __name__ == '__main__':
im = Image.new('RGBA', (100, 100), (0, 0, 0, 0))
draw = ImageDraw.Draw(im)
ts = [t/100.0 for t in range(101)]
xys = [(50, 100), (80, 80), (100, 50)]
bezier = make_bezier(xys)
points = bezier(ts)
xys = [(100, 50), (100, 0), (50, 0), (50, 35)]
bezier = make_bezier(xys)
points.extend(bezier(ts))
xys = [(50, 35), (50, 0), (0, 0), (0, 50)]
bezier = make_bezier(xys)
points.extend(bezier(ts))
xys = [(0, 50), (20, 80), (50, 100)]
bezier = make_bezier(xys)
points.extend(bezier(ts))
draw.polygon(points, fill = 'red')
im.save('out.png')
A bezier curve isn't that hard to draw yourself. Given three points A, B, C you require three linear interpolations in order to draw the curve. We use the scalar t as the parameter for the linear interpolation:
P0 = A * t + (1 - t) * B
P1 = B * t + (1 - t) * C
This interpolates between two edges we've created, edge AB and edge BC. The only thing we now have to do to calculate the point we have to draw is interpolate between P0 and P1 using the same t like so:
Pfinal = P0 * t + (1 - t) * P1
There are a couple of things that need to be done before we actually draw the curve. First off we have will walk some dt (delta t) and we need to be aware that 0 <= t <= 1. As you might be able to imagine, this will not give us a smooth curve, instead it yields only a discrete set of positions at which to plot. The easiest way to solve this is to simply draw a line between the current point and the previous point.
You can use the aggdraw on top of PIL, bezier curves are supported.
EDIT:
I made an example only to discover there is a bug in the Path class regarding curveto :(
Here is the example anyway:
from PIL import Image
import aggdraw
img = Image.new("RGB", (200, 200), "white")
canvas = aggdraw.Draw(img)
pen = aggdraw.Pen("black")
path = aggdraw.Path()
path.moveto(0, 0)
path.curveto(0, 60, 40, 100, 100, 100)
canvas.path(path.coords(), path, pen)
canvas.flush()
img.save("curve.png", "PNG")
img.show()
This should fix the bug if you're up for recompiling the module...
Although bezier curveto paths don't work with Aggdraw, as mentioned by #ToniRuža, there is another way to do this in Aggdraw. The benefit of using Aggdraw instead of PIL or your own bezier functions is that Aggdraw will antialias the image making it look smoother (see pic at bottom).
Aggdraw Symbols
Instead of using the aggdraw.Path() class to draw, you can use the aggdraw.Symbol(pathstring) class which is basically the same except you write the path as a string. According to the Aggdraw docs the way to write your path as a string is to use SVG path syntax (see: http://www.w3.org/TR/SVG/paths.html). Basically, each addition (node) to the path normally starts with
a letter representing the drawing action (uppercase for absolute path, lowercase for relative path), followed by (no spaces in between)
the x coordinate (precede by a minus sign if it is a negative number or direction)
a comma
the y coordinate (precede by a minus sign if it is a negative number or direction)
In your pathstring just separate your multiple nodes with a space. Once you have created your symbol, just remember to draw it by passing it as one of the arguments to draw.symbol(args).
Bezier Curves in Aggdraw Symbols
Specifically for cubic bezier curves you write the letter "C" or "c" followed by 6 numbers (3 sets of xy coordinates x1,y1,x2,y2,x3,y3 with commas in between the numbers but not between the first number and the letter). According the docs there are also other bezier versions by using the letter "S (smooth cubic bezier), Q (quadratic bezier), T (smooth quadratic bezier)". Here is a complete example code (requires PIL and aggdraw):
print "initializing script"
# imports
from PIL import Image
import aggdraw
# setup
img = Image.new("RGBA", (1000,1000)) # last part is image dimensions
draw = aggdraw.Draw(img)
outline = aggdraw.Pen("black", 5) # 5 is the outlinewidth in pixels
fill = aggdraw.Brush("yellow")
# the pathstring:
#m for starting point
#c for bezier curves
#z for closing up the path, optional
#(all lowercase letters for relative path)
pathstring = " m0,0 c300,300,700,600,300,900 z"
# create symbol
symbol = aggdraw.Symbol(pathstring)
# draw and save it
xy = (20,20) # xy position to place symbol
draw.symbol(xy, symbol, outline, fill)
draw.flush()
img.save("testbeziercurves.png") # this image gets saved to same folder as the script
print "finished drawing and saved!"
And the output is a smooth-looking curved bezier figure:
I found a simpler way creating a bezier curve (without aggraw and without complex functions).
import math
from PIL import Image
from PIL import ImageDraw
image = Image.new('RGB',(1190,841),'white')
draw = ImageDraw.Draw(image)
curve_smoothness = 100
#First, select start and end of curve (pixels)
curve_start = [(167,688)]
curve_end = [(678,128)]
#Second, split the path into segments
curve = []
for i in range(1,curve_smoothness,1):
split = (curve_end[0][0] - curve_start[0][0])/curve_smoothness
x = curve_start[0][0] + split * i
curve.append((x, -7 * math.pow(10,-7) * math.pow(x,3) - 0.0011 * math.pow(x,2) + 0.235 * x + 682.68))
#Third, edit any other corners of polygon
other =[(1026,721), (167,688)]
#Finally, combine all parts of polygon into one list
xys = curve_start + curve + curve_end + other #putting all parts of the polygon together
draw.polygon(xys, fill = None, outline = 256)
image.show()

Categories