We Keep Coding

How to make a single image using several images?

I have these images and there is a shadow in all images. I target is making a single image of a car without shadow by using these three images:
Finally, how can I get this kind of image as shown below:
Any kind of help or suggestions are appreciated.
EDITED
According to the comments, I used np.maximum and achieved easily to my target:
import cv2
import numpy as np
img_1 = cv2.imread('1.png', cv2.COLOR_BGR2RGB)
img_2 = cv2.imread('2.png', cv2.COLOR_BGR2RGB)
img = np.maximum(img_1, img_2)
cv2.imshow('img1', img_1)
cv2.imshow('img2', img_2)
cv2.imshow('img', img)
cv2.waitKey(0)

Here's a possible solution. The overall idea is to compute the location of the shadows, produce a binary mask identifying the location of the shadows and use this information to copy pixels from all the cropped sub-images.
Let's see the code. The first problem is to locate the three images. I used the black box to segment and crop each car, like this:
# Imports:
import cv2
import numpy as np
# image path
path = "D://opencvImages//"
fileName = "qRLI7.png"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Get the HSV image:
hsvImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2HSV)
# Get the grayscale image:
grayImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
showImage("grayImage", grayImage)
# Threshold via Otsu:
_, binaryImage = cv2.threshold(grayImage, 5, 255, cv2.THRESH_BINARY_INV)
cv2.imshow("binaryImage", binaryImage)
cv2.waitKey(0)
The previous bit uses the grayscale version of the image and applies a fixed binarization using a threshold of 5. I also pre-compute the HSV version of the original image. The result of the thresholding is this:
I'm trying to get the black rectangles and use them to crop each car. Let's get the contours and filter them by area, as the black rectangles on the binary image have the biggest area:
for i, c in enumerate(currentContour):
# Get the contour's bounding rectangle:
boundRect = cv2.boundingRect(c)
# Get the dimensions of the bounding rect:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Get the area:
blobArea = rectWidth * rectHeight
minArea = 20000
if blobArea > minArea:
# Deep local copies:
hsvImage = hsvImage.copy()
localImage = inputImage.copy()
# Get the S channel from the HSV image:
(H, S, V) = cv2.split(hsvImage)
# Crop image:
croppedImage = V[rectY:rectY + rectHeight, rectX:rectX + rectWidth]
localImage = localImage[rectY:rectY + rectHeight, rectX:rectX + rectWidth]
_, binaryMask = cv2.threshold(croppedImage, 0, 255, cv2.THRESH_OTSU + cv2.THRESH_BINARY_INV)
After filtering each contour to get the biggest one, I need to locate the position of the shadow. The shadow is mostly visible in the HSV color space, particularly, in the V channel. I cropped two versions of the image: The original BGR image, now cropped, and the V cropped channel of the HSV image. This is the binary mask that results from applying an automatic thresholding on the S channel :
To locate the shadow I only need the starting x coordinate and its width, because the shadow is uniform across every cropped image. Its height is equal to each cropped image's height. I reduced the V image to a row, using the SUM mode. This will sum each pixel across all columns. The biggest values will correspond to the position of the shadow:
# Image reduction:
reducedImg = cv2.reduce(binaryMask, 0, cv2.REDUCE_SUM, dtype=cv2.CV_32S)
# Normalize image:
max = np.max(reducedImg)
reducedImg = reducedImg / max
# Clip the values to [0,255]
reducedImg = np.clip((255 * reducedImg), 0, 255)
# Convert the mat type from float to uint8:
reducedImg = reducedImg.astype("uint8")
_, shadowMask = cv2.threshold(reducedImg, 250, 255, cv2.THRESH_BINARY)
The reduced image is just a row:
The white pixels denote the largest values. The location of the shadow is drawn like a horizontal line with the largest area, that is, the most contiguous white pixels. I process this row by getting contours and filtering, again, to the largest area:
# Get the biggest rectangle:
subContour, _ = cv2.findContours(shadowMask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for j, s in enumerate(subContour):
# Get the contour's bounding rectangle:
boundRect = cv2.boundingRect(s)
# Get the dimensions of the bounding rect:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Get the area:
blobArea = rectWidth * rectHeight
minArea = 30
if blobArea > minArea:
# Get image dimensions:
(imageHeight, imageWidth) = localImage.shape[:2]
# Set an empty array, this will be the binary mask
shadowMask = np.zeros((imageHeight, imageWidth, 3), np.uint8)
color = (255, 255, 255)
cv2.rectangle(shadowMask, (int(rectX), int(0)),
(int(rectX + rectWidth), int(0 + imageHeight)), color, -1)
# Invert mask:
shadowMask = 255 - shadowMask
# Store mask and cropped image:
shadowRois.append((shadowMask.copy(), localImage.copy()))
Alright, with that information I create a mask, where the only thing drawn in white is the location of the mask. I store this mask and the original BGR crop in the shadowRois list.
What follows is a possible method to use this information and create a full image. The idea is that I use the information of each mask to copy all the non-masked pixels. I accumulate this information on a buffer, initially an empty image, like this:
# Prepare image buffer:
buffer = np.zeros((100, 100, 3), np.uint8)
# Loop through cropped images and produce the final image:
for r in range(len(shadowRois)):
# Get data from the list:
(mask, img) = shadowRois[r]
# Get image dimensions:
(imageHeight, imageWidth) = img.shape[:2]
# Resize the buffer:
newSize = (imageWidth, imageHeight)
buffer = cv2.resize(buffer, newSize, interpolation=cv2.INTER_AREA)
# Get the image mask:
temp = cv2.bitwise_and(img, mask)
# Set info in buffer, substitute the black pixels
# for the new data:
buffer = np.where(temp == (0, 0, 0), buffer, temp)
cv2.imshow("Composite Image", buffer)
cv2.waitKey(0)
The result is this:

Object Detection with OpenCV-Python

I am trying to detect all of the overlapping circle/ellipses shapes in this image all of which have digits on them. I have tried different types of image processing techniques using OpenCV, however I cannot detect the shapes that overlap the tree. I have tried erosion and dilation however it has not helped.
Any pointers on how to go about this would be great. I have attached my code below
original = frame.copy()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (3, 3), 0)
canny = cv2.Canny(blurred, 120, 255, 1)
kernel = np.ones((5, 5), np.uint8)
dilate = cv2.dilate(canny, kernel, iterations=1)
# Find contours
cnts = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
image_number = 0
for c in cnts:
x, y, w, h = cv2.boundingRect(c)
cv2.rectangle(frame, (x, y), (x + w, y + h), (36, 255, 12), 2)
ROI = original[y:y + h, x:x + w]
cv2.imwrite("ROI_{}.png".format(image_number), ROI)
image_number += 1
cv2.imshow('canny', canny)
cv2.imshow('image', frame)
cv2.waitKey(0)

Here's a possible solution. I'm assuming that the target blobs (the saucer-like things) are always labeled - that is, they always have a white number inside them. The idea is to create a digits mask, because their size and color seem to be constant. I use the digits as guide to obtain sample pixels of the ellipses. Then, I convert these BGR pixels to HSV, create a binary mask and use that info to threshold and locate the ellipses. Let's check out the code:
# imports:
import cv2
import numpy as np
# image path
path = "D://opencvImages//"
fileName = "4dzfr.png"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Deep copy for results:
inputImageCopy = inputImage.copy()
# Convert RGB to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Get binary image via Otsu:
binaryImage = np.where(grayscaleImage >= 200, 255, 0)
# The above operation converted the image to 32-bit float,
# convert back to 8-bit uint
binaryImage = binaryImage.astype(np.uint8)
The first step is to make a mask of the digits. I also created a deep copy of the BGR image. The digits are close to white (That is, an intensity close to 255). I use 200 as threshold and obtain this result:
Now, let's locate these contours on this binary mask. I'm filtering based on aspect ratio, as the digits have a distinct aspect ratio close to 0.70. I'm also filtering contours based on hierarchy - as I'm only interested on external contours (the ones that do not have children). That's because I really don't need contours like the "holes" inside the digit 8:
# Find the contours on the binary image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
# Store the sampled pixels here:
sampledPixels = []
# Look for the outer bounding boxes (no children):
for i, c in enumerate(contours):
# Get the contour bounding rectangle:
boundRect = cv2.boundingRect(c)
# Get the dimensions of the bounding rect:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Compute the aspect ratio:
aspectRatio = rectWidth / rectHeight
# Create the filtering threshold value:
delta = abs(0.7 - aspectRatio)
epsilon = 0.1
# Get the hierarchy:
currentHierarchy = hierarchy[0][i][3]
# Prepare the list of sampling points (One for the ellipse, one for the circle):
samplingPoints = [ (rectX - rectWidth, rectY), (rectX, rectY - rectHeight) ]
# Look for the target contours:
if delta < epsilon and currentHierarchy == -1:
# This list will hold both sampling pixels:
pixelList = []
# Get sampling pixels from the two locations:
for s in range(2):
# Get sampling point:
sampleX = samplingPoints[s][0]
sampleY = samplingPoints[s][1]
# Get sample BGR pixel:
samplePixel = inputImageCopy[sampleY, sampleX]
# Store into temp list:
pixelList.append(samplePixel)
# convert list to tuple:
pixelList = tuple(pixelList)
# Save pixel value:
sampledPixels.append(pixelList)
Ok, there area a couple of things happening in the last snippet of code. We want to sample pixels from both the ellipse and the circle. We will use two sampling locations that are function of each digit's original position. These positions are defined in the samplingPoints tuple. For the ellipse, I'm sampling at a little before the top right position of the digit. For the circle, I'm sapling directly above the top right position - we end up with two pixels for each figure.
You'll notice I'm doing a little bit of data type juggling, converting lists to tuples. I want these pixels stored as a tuple for convenience. If I draw bounding rectangles of the digits, this would be the resulting image:
Now, let's loop through the pixel list, convert them to HSV and create a HSV mask over the original BGR image. The final bounding rectangles of the ellipses are stored in boundingRectangles, additionally I draw results on the deep copy of the original input:
# Final bounding rectangles are stored here:
boundingRectangles = []
# Loop through sampled pixels:
for p in range(len(sampledPixels)):
# Get current pixel tuple:
currentPixelTuple = sampledPixels[p]
# Prepare the HSV mask:
imageHeight, imageWidth = binaryImage.shape[:2]
hsvMask = np.zeros((imageHeight, imageWidth), np.uint8)
# Process the two sampling pixels:
for m in range(len(currentPixelTuple)):
# Get current pixel in the list:
currentPixel = currentPixelTuple[m]
# Create BGR Mat:
pixelMat = np.zeros([1, 1, 3], dtype=np.uint8)
pixelMat[0, 0] = currentPixel
# Convert the BGR pixel to HSV:
hsvPixel = cv2.cvtColor(pixelMat, cv2.COLOR_BGR2HSV)
H = hsvPixel[0][0][0]
S = hsvPixel[0][0][1]
V = hsvPixel[0][0][2]
# Create HSV range for this pixel:
rangeThreshold = 5
lowerValues = np.array([H - rangeThreshold, S - rangeThreshold, V - rangeThreshold])
upperValues = np.array([H + rangeThreshold, S + rangeThreshold, V + rangeThreshold])
# Create HSV mask:
hsvImage = cv2.cvtColor(inputImage.copy(), cv2.COLOR_BGR2HSV)
tempMask = cv2.inRange(hsvImage, lowerValues, upperValues)
hsvMask = hsvMask + tempMask
First, I create a 1 x 1 Matrix (or Numpy Array) with just a BGR pixel value - the first of two I previously sampled. In this way, I can use cv2.cvtColor to get the corresponding HSV values. Then, I create lower and upper threshold values for the HSV mask. However, the image seems synthetic, and a range-based thresholding could be reduced to a unique tuple. After that, I create the HSV mask using cv2.inRange.
This will yield the HSV mask for the ellipse. After applying the method for the circle we will end up with two HSV masks. Well, I just added the two arrays to combine both masks. At the end you will have something like this, this is the "composite" HSV mask created for the first saucer-like figure:
We can apply a little bit of morphology to join both shapes, just a little closing will do:
# Set kernel (structuring element) size:
kernelSize = 3
# Set morph operation iterations:
opIterations = 2
# Get the structuring element:
morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
# Perform closing:
hsvMask = cv2.morphologyEx(hsvMask, cv2.MORPH_CLOSE, morphKernel, None, None, opIterations,cv2.BORDER_REFLECT101)
This is the result:
Nice. Let's continue and get the bounding rectangles of all the shapes. I'm using the boundingRectangles list to store each bounding rectangle, like this:
# Process current contour:
currentContour, _ = cv2.findContours(hsvMask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for _, c in enumerate(currentContour):
# Get the contour's bounding rectangle:
boundRect = cv2.boundingRect(c)
# Get the dimensions of the bounding rect:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Store and set bounding rect:
boundingRectangles.append(boundRect)
color = (0, 0, 255)
cv2.rectangle(inputImageCopy, (int(rectX), int(rectY)),
(int(rectX + rectWidth), int(rectY + rectHeight)), color, 2)
cv2.imshow("Objects", inputImageCopy)
cv2.waitKey(0)
This is the image of the located rectangles once every sampled pixel is processed:

Reducing misses on template matching (using transparent template)

I'm trying to determine the position of puzzle piece on a puzzle image.
I have
an image of a puzzle piece (transparent png)
an image of a puzzleboard whith a white outline around the correct position
Firstly i extract the contour on image 1. and use it to draw my final template.
Then i match the final template to the puzzleboard and save some result images.
Here are some examples of the results i'm getting
https://imgur.com/a/ZYyw7tU
On some of the images, there's clearly a lot of mismatches and for some of the images, increasing the threshold will hide the correct match and some mismatches will remain.
Any tips or thoughts on optimization would be greatly appreciated!
This is my full code:
full_image = cv2.imread('puzzle_1.jpg')
piece = cv2.imread('piece_1.png', cv2.IMREAD_UNCHANGED)
partial_image = cv2.cvtColor(piece,cv2.COLOR_BGR2GRAY)
contours, hierarchy = cv2.findContours(partial_image.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
template = np.zeros((55, 55, 4), dtype=np.uint8)
cv2.drawContours(template, contours, -1, (255, 255, 255, 255),1)
hh, ww = template.shape[:2]
puzzleP = template[:,:,0:3]
alpha = template[:,:,3]
alpha = cv2.merge([alpha,alpha,alpha])
correlation = cv2.matchTemplate(full_image, puzzleP, cv2.TM_CCORR_NORMED, mask=alpha)
threshhold = 0.98
loc = np.where(correlation >= threshhold)
result = full_image.copy()
for pt in zip(*loc[::-1]):
cv2.rectangle(result, pt, (pt[0]+ww, pt[1]+hh), (0,0,255), 1)
print(pt)
cv2.imwrite('puzzle_piece.png', puzzleP)
cv2.imwrite('full_image_alpha.png', alpha)
cv2.imwrite('full_image_matches.jpg', result)
EDIT:
Here's an example of the two files (piece_1.png and puzzle_1.jpg) (this example has many mismatches)
https://imgur.com/a/nGSXcNg

This seems to work fine for me on your given image in Python/OpenCV.
Read the large and small images
Convert the small image to gray and threshold to binary
Get the largest contour of the binary image
Draw that contour as white on black background to use as the mask
Extract the BGR channels of the transparent small image as the template
Do the template matching and get the largest match location
Draw the match box on a copy of the large image
Save the results
Large image:
Small image:
import cv2
import numpy as np
# read images
full_image = cv2.imread('puzzle_1.jpg')
piece = cv2.imread('piece_1.png', cv2.IMREAD_UNCHANGED)
# convert piece to gray and threshold to binary
partial_image = cv2.cvtColor(piece,cv2.COLOR_BGR2GRAY)
partial_image = cv2.threshold(partial_image, 0, 255, cv2.THRESH_BINARY)[1]
# get largest contour from binary image
contours = cv2.findContours(partial_image.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)
# draw the contour of the piece outline as the mask
mask = np.zeros((55, 55, 3), dtype=np.uint8)
cv2.drawContours(mask, [big_contour], 0, (255,255,255), 1)
hh, ww = mask.shape[:2]
# extract the template from the BGR (no alpha) piece
template = piece[:,:,0:3]
correlation = cv2.matchTemplate(full_image, template, cv2.TM_CCORR_NORMED, mask=mask)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(correlation)
max_val_ncc = '{:.3f}'.format(max_val)
print("normalize_cross_correlation: " + max_val_ncc)
xx = max_loc[0]
yy = max_loc[1]
print('xmatchloc =',xx,'ymatchloc =',yy)
# draw template bounds and corner intersection in red onto img
result = full_image.copy()
cv2.rectangle(result, (xx, yy), (xx+ww, yy+hh), (0, 0, 255), 1)
# save results
cv2.imwrite('puzzle_template.png', template)
cv2.imwrite('puzzle_mask.png', mask)
cv2.imwrite('full_image_matches.jpg', result)
# show results
cv2.imshow('full_image', full_image)
cv2.imshow('piece', piece)
cv2.imshow('partial_image', partial_image)
cv2.imshow('mask', mask)
cv2.imshow('template', template)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
(bgr) Template:
Mask:
Largest Match Location:

Image Differencing: Ignore translational movement

I'm looking for some insight into what the best approach might be to my problem.
I'm comparing two separate images for differences, but I'm running into a problem with small translational movements.
I have a "gospel" image which is the "gold standard" per se:
gospel image
Then I have multiple different taken images to compare against.
Here's an example: example image
Here's an example difference image showing my problem: difference image
As you can see, they are quite small. The way that I am differencing the images now is by first resizing the images to 32x32, manually decreasing the contrast by 100 and then applying a blur using OpenCV.
After, I am using skimage's 'structural_integrity' function to subtract and quantify the differences between the images. The rest is purely for viewing.
import cv2
import numpy as np
from PIL import Image
from skimage.metrics import structural_similarity
def change_contrast(img, level):
img = Image.fromarray(img)
factor = (259 * (level + 255)) / (255 * (259 - level))
def contrast(c):
return 128 + factor * (c - 128)
return np.asarray(img.point(contrast))
# Open and preprocess the images
image_orig = cv2.imread(IMAGE_PATH)
image = cv2.resize(image, (32, 32))
image = change_contrast(image_orig, -100)
image = cv2.blur(image, (5, 5))
image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gospel_orig = cv2.imread(GOSPEL_PATH)
gospel = cv2.resize(gospel_orig, (32, 32))
gospel = change_contrast(gospel, -100)
gospel = cv2.blur(gospel, (5, 5))
gospel = cv2.cvtColor(gospel, cv2.COLOR_BGR2GRAY)
# Get image similarities and an output difference image
(score, diff) = structural_similarity(image, gospel, full=True)
print("Image similarity", score)
diff = (diff * 255).astype("uint8")
# Viewing stuff below
thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
filled_gospel = cv2.cvtColor(gospel, cv2.COLOR_GRAY2BGR)
for c in contours:
area = cv2.contourArea(c)
if area > 40:
x,y,w,h = cv2.boundingRect(c)
cv2.drawContours(filled_gospel, [c], 0, (0,255,0), -1)
cv2.imshow('image', image)
cv2.imshow('gospel', gospel)
cv2.imshow('diff',diff)
cv2.imshow('filled gospel',filled_gospel)
cv2.waitKey(0)
When I do the above steps, you can see some translational differences between the 'gospel' and the taken image. What would be the best way to combat this as I only want to get the differences in the black of the letter, not how well it is aligned?

Here is how I would do template matching and differencing in Python/OpenCV.
Read the reference and example images
Pad the example image to twice its dimensions with its background gray color.
Do template matching with the reference to find the best match location and match score.
Crop the padded example image with its top left corner at the match location, but the size of the reference image
Get the absolute difference image
Save results
Reference:
Example:
import cv2
import numpy as np
# read reference and convert to gray
ref = cv2.imread('reference.png')
ref_gray = cv2.cvtColor(ref, cv2.COLOR_BGR2GRAY)
hr, wr = ref_gray.shape
# read example and convert to gray
ex = cv2.imread('example.png')
ex_gray = cv2.cvtColor(ex, cv2.COLOR_BGR2GRAY)
he, we = ex_gray.shape
# pad the example to double its dimensions with gray=190
color=190
wp = we // 2
hp = he // 2
ex_gray = cv2.copyMakeBorder(ex_gray, hp,hp,wp,wp, cv2.BORDER_CONSTANT, value=color)
# do template matching
corrimg = cv2.matchTemplate(ref_gray,ex_gray,cv2.TM_CCOEFF_NORMED)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(corrimg)
max_val_corr = '{:.3f}'.format(max_val)
print("correlation: " + max_val_corr)
xx = max_loc[0]
yy = max_loc[1]
print('x_match_loc =',xx,'y_match_loc =',yy)
# crop the padded example image at top left corner of xx,yy and size hr x wr
ex_gray_crop = ex_gray[yy:yy+hr, xx:xx+wr]
# get absolute difference image
ref_grayf = ref_gray.astype(np.float32)
ex_gray_cropf = ex_gray_crop.astype(np.float32)
diff = 255 - np.abs(cv2.add(ref_gray, -ex_gray_crop))
# compute mean of diff
mean = cv2.mean(diff)[0]
print("mean of diff in range 0 to 100 =",mean)
cv2.imshow('ref_gray', ref_gray)
cv2.imshow('ex_gray', ex_gray)
cv2.imshow('ex_gray_crop', ex_gray_crop)
cv2.imshow('correlation image', corrimg)
cv2.imshow('diff', diff)
cv2.waitKey(0)
cv2.destroyAllWindows()
# save results
cv2.imwrite('reference_gray.jpg', ref_gray)
cv2.imwrite('example_gray_padded.jpg', ex_gray)
cv2.imwrite('reference_example_correlation.jpg', (255*corrimg).clip(0,255).astype(np.uint8))
cv2.imwrite('example_gray_padded_cropped.jpg', ex_gray_crop)
cv2.imwrite('reference_example_diff.jpg', diff)
Example padded:
Correlation Image showing locations of best match:
Match Results:
correlation: 0.969
x_match_loc = 10 y_match_loc = 9
mean of diff in range 0 to 100 = 1.3956887102667155
Example cropped to align with reference:
Diff image (white is where they differ):

OpenCV - Smoothing borders

I want to stitch multiple image patches to a new and mainly gray background image. The image patches contain colored elements which shall not be changed, if possible. Their shape and color is diverse. Like the new background image the borders of the image patches are also gray, just slightly different, but you can see strong borders if I just go by
ImgPatch = cv2.imread("C://...//ImagePatch.png")
NewBackground = cv2.imread("C://...//NewBackground.png")
height, width, channels = ImgPatch.shape
NewBackground[y:y+height,x:x+width] = ImgPatch
I tried cv2.seamlessClone() (docs.opencv.org) as explained in this tutorial:
www.learnopencv.com/seamless-cloning-using-opencv-python-cpp
The edges are perfectly smoothed, but unfortunately the colors of the elements are changed way too much. I know the approximate width and height of the gray border of each image patch. If i could specifically smooth that area that may be a start and lets the result look already better than what I have. I tried different masks with cv2.seamlessClone(), of which none of the tried ways workes. So unfortunately I couldn't find a correct way to blend only the border of the patches so far.
The following images visualize my problem in a very abstract way.
What I have:
Left: Background, Right: Image patch
What I want:
What I currently get by using cv2.seamlessClone():
Any help would be very much appreciated!
EDIT As I probably was not clear enough: The real images are way more complex and so unfortunately I can not get reasonable results for all image patches by using cv2.findContour... What I am looking for is a method to merge the borders, so you can not see the exact transition of patch to background anymore.

patch = cv2.imread('patch.png', cv2.IMREAD_UNCHANGED);
image = cv2.imread('image.png', cv2.IMREAD_UNCHANGED);
mask = 255 * np.ones(patch.shape, patch.dtype)
width, height, channels = image.shape
center = (height//2, width//2)
mixed_clone = cv2.seamlessClone(patch, image, mask, center, cv2.cv2.NORMAL_CLONE)

You could try to find contour in your image patch with cv2.findContour() (red spot). Then remove the background of the contour and save the image. You can finally combine the one you saved (red spot without background) with the gray background image with cv2.add(). I have combined some code I once played with and the code in OpenCV docs (for cv2.add()). Hope it helps a bit (Note the example ads the image in upper left corner - if you want elswhere you should change the code). Cheers!
Example:
import cv2
import numpy as np
from PIL import Image
img = cv2.imread('background2.png', cv2.IMREAD_UNCHANGED)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, threshold = cv2.threshold(gray, 100, 255, cv2.THRESH_BINARY_INV)
height,width = gray.shape
mask = np.zeros((height,width), np.uint8)
_, contours, hierarchy = cv2.findContours(threshold,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
cnt = max(contours, key=cv2.contourArea)
cv2.drawContours(mask,[cnt], -1, (255,255,255),thickness=-1)
masked = cv2.bitwise_and(img, img, mask=mask)
_,thresh = cv2.threshold(mask,1,255,cv2.THRESH_BINARY)
contours = cv2.findContours(thresh,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
x,y,w,h = cv2.boundingRect(contours[0])
circle = masked[y:y+h,x:x+w]
cv2.imwrite('temp.png', circle)
cv2.waitKey(0)
cv2.destroyAllWindows()
img = Image.open('temp.png')
img = img.convert("RGBA")
datas = img.getdata()
newData = []
for item in datas:
if item[0] == 0 and item[1] == 0 and item[2] == 0:
newData.append((255, 255, 255, 0))
else:
newData.append(item)
img.putdata(newData)
img.save('background3.png', "PNG")
img1 = cv2.imread('background1.png')
img2 = cv2.imread('background3.png')
rows,cols,channels = img2.shape
roi = img1[0:rows, 0:cols ]
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 110, 255, cv2.THRESH_BINARY_INV)
mask_inv = cv2.bitwise_not(mask)
img1_bg = cv2.bitwise_and(roi,roi,mask = mask_inv)
img2_fg = cv2.bitwise_and(img2,img2,mask = mask)
dst = cv2.add(img1_bg,img2_fg)
img1[0:rows, 0:cols] = dst
cv2.imshow('img',img1)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result: