I have two images, one image which contains a box and one without. There is a small vertical disparity between the two pictures since the camera was not at the same spot and was translated a bit. I want to cut out the box and replace the hole with the information from the other picture.
I want to achieve something like this (a slide from a computer vision course)
I thought about using the cv2.createBackgroundSubtractorMOG2() method, but it does not seem to work with only 2 pictures.
Simply subtracting the picture from another does not work either because of the disparity.
The course suggests using RANSAC to compute the most likely relationship between two pictures and subtract the area thaht changed a lot. But how do I actually fill in the holes?
Many thanks in advance!!
If you plant ot use only a pair of images (or only a few images), image stitching methods are better than background subtraction.
The steps are:
Calculate homography between the two images.
Warp the second image to overlap the second.
Replace the region with the human with pixels from the warped image.
This link shows a basic example of image stitching. You will need extra work if both images have humans in different places, but otherwise it should not be hard to tweak this code.
You can try this library for background subtraction issues. https://github.com/andrewssobral/bgslibrary
there is python wrappers of this tool.
Related
I always wanted to have a device that, from a live camera feed, could detect an object, create a 3D model of it, and then identify it. It would work a lot like the Scanner tool from Subnautica. Imagine my surprise when I found OpenCV, a free-to-use computer vision tool for Python!
My first step is to get the computer to recognize that there is an object at the center of the camera feed. To do this, I found a Canny() function that could detect edges and display them as white lines in a black image, which should make a complete outline of the object in the center. I also used the floodFill() function to fill in the black zone between the white lines with gray, which would show that the computer recognizes that there is an object there. My attempt is in the following image.
The red dot is the center of the live video.
The issue is that the edge lines can have holes in them due to a blur between two colors, which can range from individual pixels to entire missing lines. As a result, the gray gets out and doesn't highlight me as the only object, and instead highlights the entire wall as well. Is there a way to fill those missing pixels in or is there a better way of doing this?
Welcome to SO and the exiting world of machine vision !
What you are describing is a very classical problem in the field, and not a trivial one at all. It depends heavily on the shape and appearance of what you define as the object of interest and the overall structure, homogeneity and color of the background. Remember, the computer has no concept of what an "object" is, the only thing it 'knows' is a matrix of numbers.
In your example, you might start out with selecting the background area by color (or hue, look up HSV). Everything else is your object. This is what classical greenscreening techniques do, and it only works with (a) a homogenous background, which does not share a color with your object and (b) a single or multiple not overlapping objects.
The problem with your edge based approach is that you won't get a closed edge safely, and deciding where the inside and outside of the object is might get tricky.
Advanced ways to do this would get you into Neural Network territory, but maybe try to get the basics down first.
Here are two links to tutorials on converting color spaces and extracting contours:
https://docs.opencv.org/4.x/df/d9d/tutorial_py_colorspaces.html
https://docs.opencv.org/3.4/d4/d73/tutorial_py_contours_begin.html
If you got that figured out, look into stereo vision or 3D imaging in general, and that subnautica scanner might just become reality some day ;)
Good luck !
For an analysis, I'd like to take the a bunch of TIF images I have, and fill in the black background to create a perfect square.
I would like to keep the same general pattern of the image when I fill in the black spots, instead of just filling in the black with random bits of white and blue. My first thought for doing this is to impose some sort of symmetrical "reflection" of the image onto the black portions - the concept is detailed below.
The thing is, I'm not sure how to go about this - my first thought was to convert the image to a NumPy array and copy the individual rows of pixels over for a pseudo-reflection, but that could take a lot of time since I would be accounting for the length of the black portion in each row, and it wouldn't even be the desired result. I was wondering if there was a package or method that did something like this already, perhaps in PIL.
Any ideas are appreciated, as I am not to familiar with image processing in Python (or in general) - thank you!
EDIT: Here is a google drive link for the example image in question.
EDIT 2: Here is a google drive link for another example, this time with two "overlapping" black background areas (in other words, the actual data has a corner)
I'm trying to build a footage filter that only sends only "good" frames to the database.
Here is my current rating function:
def rateImg(img):
try:
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
except:
gray = img
edges = cv2.Canny(gray, 0, 255)
countours, _ = cv2.findContours(
edges, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
num_of_countours = len(countours)
lap = cv2.Laplacian(gray, cv2.CV_64F).var()
lap = round(lap, 2)
return [lap, num_of_countours]
First off, I use variance of Laplacian to calculate the sharpness of an image from a particular time window.
It should technically provide me a "good" frame, but that's not always the case.
The camera I have to use isn't great and sometimes glitches out like this and frames like this have the highest variance of Laplacian.
So, my current solution is to calculate the number of countours in an image and if an image crosses a particular threshold I classify it as "glitched". But with this approach the algorithm rates images with a lot of objects as "glitched".
Also, I have tried detecting squares and rectangles, but that proved to be much less effective than the countour approach.
Is there any way to detect obvious glitches in an image?
I feel like there should be, because as a human I can easily classify glitched and normal images at a glance. I just can't seem to pin-point what exactly makes them different.
Is there any way to detect obvious glitches in an image?
Yes, but probably not for complex random glitches, have a look in this similar question
In that case, you can detect if there is a large area of the image containing the same color. Photo taken from the camera would never contain the same RGB value although they look similar. However, this would be perfectly normal if the images are arts drawn on a digital devices.
As a human I can easily classify glitched and normal images at a
glance... What exactly makes them (me and a program) different. Is there any way to detect obvious glitches in an image
In fact, you can't identify a glitched image. You try to recognize the objects in it. When you see something "weird" that you don't recognize, you consider it as a glitched image. The machine can neither achieve this. You can train an AI that report images with unrecognizable "parts" as glitched but it will never be 100% accurate
Converting your image to HSV and runnign the Brightness Channel through an edge filter on ImageJ gives me this:
As you can see, the "glitched" region appears pretty uniformly brighter then the rest of the image, and should be detectable in some form. How often do you get a picture from you camera ? Depending on how much change occurs between two pictures, you might get away with subtracting the current one from the one before it to just look at changes.
You have not shown what an image with
a lot of objects
looks like, so you'd have to try if this works for those cases.
OpenCV functions needed for this workflow would be:
cvtColor() with COLOR_BGR2GRAY for color conversion (there might be faster ways to get a good greyvalue then HSV)
one of the edge detectors. Canny() or Sobel() would be the first i'd try
some greyvalue statistics. threshold() and CountNonZero() for a simple approach, which you could refine for sectors on the image or such
PS:
I feel like there should be, because as a human I can easily classify
glitched and normal images at a glance.
That's a common fallacy: Us humans (the sight-centric beings that we are) are fantastic at pattern recognition and interpolation and are rarely aware how much of that (including a lot of error correction) is happening every microsecond. A puny 2D camera can not hope to live up to that. (obligatory XKCD: https://xkcd.com/1425/)
How to go from the image on the left to the image on the right programmatically using Python (and maybe some tools, like OpenCV)?
I made this one by hand using an online tool for clipping. I am completely noob in image processing (especially in practice). I was thinking to apply some edge or contour detection to create a mask, which I will apply later on the original image to paint everything else (except the region of interest) black. But I failed miserably.
The goal is to preprocess a dataset of very similar images, in order to train a CNN binary classifier. I tried to train it by just cropping the image close to the region of interest, but the noise is so high that the CNN learned absolutely nothing.
Can someone help me do this preprocessing?
I used OpenCV's implementation of watershed algorithm to solve your problem. You can find out how to use it if you read this great tutorial, so I will not explain this into a lot of detail.
I selected four points (markers). One is located on the region that you want to extract, one is outside and the other two are within lower/upper part of the interior that does not interest you. I then created an empty integer array (the so-called marker image) and filled it with zeros. Then I assigned unique values to pixels at marker positions.
The image below shows the marker positions and marker values, drawn on the original image:
I could also select more markers within the same area (for example several markers that belong to the area you want to extract) but in that case they should all have the same values (in this case 255).
Then I used watershed. The first input is the image that you provided and the second input is the marker image (zero everywhere except at marker positions). The algorithm stores the result in the marker image; the region that interests you is marked with the value of the region marker (in this case 255):
I set all pixels that did not have the 255 value to zero. I dilated the obtained image three times with 3x3 kernel. Then I used the dilated image as a mask for the original image (i set all pixels outside the mask to zero) and this is the result i got:
You will probably need some kind of method that will find markers automatically. The difficulty of this task depends heavily on the set of the input images. In some cases, the method can be really straightforward and simple (as in the tutorial linked above) but sometimes this can be a tough nut to crack. But I can't recommend anything because I don't know how your images look like in general (you only provided one). :)
Hi I am wanting to use the python imaging library to crop images to a specific size for a website. I have a problem, these images are meant to show people's faces so I need to automatically crop based on them.
I know face detection is a difficult concept so I'm thinking of using the face.com API http://developers.face.com/tools/#faces/detect which is fine for what I want to do.
I'm just a little stuck on how I would use this data to crop a select area based on the majority of faces.
Can anybody help?
Joe
There is a library for python that have a concept of smart-cropping that among other options, can use face detection to do a smarter cropping.
It uses opencv under the hood, but you are isolated from it.
https://github.com/globocom/thumbor
If you have some rectangle that you want to excise from an image, here's what I might try first:
(optional) If the image is large, do a rough square crop centered on the face with dimensions sqrt(2) larger than the longer edge (if rectangular). Worst-case (45° rotation), it will still grab everything important.
Rotate based on the face orientation (something like rough_crop.rotate(math.degrees(math.atan(ydiff/xdiff)), trig is fun)
Do a final crop. If you did the initial crop, the face should be centered, otherwise you'll have to transform (rotate) all your old coordinates to the new image (more trig!).