We have a dataset of images with different items from the hyper markets, at this stage we want to detect those objects without using labelled data (cause that what we are trying to avoid).
For instance we have this image:
and we would like to detect those store items (bounding boxes preferred)
I've just drawn it around 3 items to convey my point but in the solution it would apply to all the items.
What we have tried:
K means clustering (but the main issue is finding k), we don't know how many items there are )
Otsu Image thresholding (but the objects have different colors, both dark and light, so the thresholding takes away one or the other) + the objects have hollow parts inside it like the bottle handles.
What we would like to have:
We would like to any leads on what to do here. This doesn't have to be just code solutions, any topics or ideas that could lead us to the solution would be equally appreciated.
Thank you again
Related
I have raw microscopy images like this:
And I want to segment the objects, as you see some of them are really close and I have a great range of intensity values.
background: 700 a.u.
fluorescent shapes: from 7000 to 32000 a.u.
To segment them I use Otsu binary segmentation from skimage package (without prior processing of the image)
thresh, imgthresh=cv2.threshold(image, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
The result is pretty good, but still fails in detecting the brightest shapes as individual objects.
I have tried a lot of things: watershed algorithm, image preprocessing (blurring), eroding , adaptive thresholding, but nothing works properly since the main problem is the difference in fluorescent values of the image.
Any smart idea on how to solve this?
Because your data have such a large range in intensity values, single histogram based methods on the whole image (e.g. Otsu) are going to have a little trouble accomplishing this task. I think that your best bet is going to be either:
threshold_multiotsu: and choose number of classes based on number of 'clusters' of intensities. Unfortunately, you will likely need to alter the number of classes on an image by image basis so this isn't super robust.
threshold_local: I know you said that you tried this but you might revisit this and alter the block_size parameter until you get something that looks reasonable. Based on your example images (and assuming a little bit about why the objects in your example images are green) it looks like that objects in close spatial proximity to one another generally have similar intensity values. Furthermore, you likely won't have to go through and alter the parameters as much as you would in option 1.
I suspect that these will be the simplest and most straight forward approaches but you could also delve into identifying the object edges using something from skimage.feature and then filling objects. Maybe something like outline here: https://scikit-image.org/docs/stable/auto_examples/features_detection/plot_blob.html. This will be a bit more involved, but these methods should be more robust with identifying objects with largely varied intensity values.
If all else fails you can try a couple of SOTA packages. The main ones that I am thinking of are https://github.com/stardist/stardist and https://github.com/MouseLand/cellpose but these seem like a bit of overkill based on your example data here.
Most neural nets for images are great a detecting objects and labeling them. They can take a picture and label some of the objects in it. -- think yolo5
Template matching, on the other hand, looks for a template that is mostly the same in a larger image. -- opencv2.templateMatching
What I hope to have is something kind of "inbetween" the two version. Given a manual entered template image, give me the Rectangles in a larger picture where this template occurs - but must be scale invariant, and transform invariant (within reason).
The opencv2 version is too strict in what it counts as matches -- 10% size change can make matches fail, slight rotations can cause it to fail. This makes it not robust enough to be useful for.
Take for instance the following (below), where we see highlighted pictures of airplanes.
This would be the ideal output.
The input would be 1 of the small green squares, ideally any 1 of them would work.
Are there things out there that can do this already?
Essentially, a opencv2.templateMatching that is more reasonably "fuzzy".
Or if I was doing this with Balls, I would use as template a picture of a ball or even baseball as a clean template, and then highlight 3 balls in the following image.
I don't need image recognition, I need image...similarity with a given template (that is better than opencv2.templateMatching cause that one is terrible)
For those interested in the future, I ultimately had to go with a full YOLOv5 network to do a custom training.
I was unable to find a "cheaper" solution.
I have an immage processing problem that im struggling to figure out a solution for the image. here is the image. Basically its a segmentation and counting problem using scikit image in python. Basically i have to write a psudeo code of how i would go about counting these "rectangle" objects in a source image that i have. The rectangles are surrounded by other different objects of different shape and size. Recently i have done a similar beginner problem to count the number of coins in an image. this one was much easier because all of the objects were of the same nature.
Could any of you help me with ideas of how to go about counting the scissors, seperating and isolating them from all of the other objects in the image. My thought process so far is to
read in image
convert to grayscale
plot a histogram
from this threshold preferqbley using otsu
remove all unwanted objects that touc border using skimage clear_border
however unlike the coins which are simple and all nearly identical i dont know how to go about isolating the rectangle objects. Is there any advanced segmentaqion techniques in skimage that could be used for this. Like i was thinking of blob but i dont think that will work here. If anyone could provide any insight please let me know i would be very grateful
It depends how general you need your solution to be. In the image you showed, the scissors are the only objects that have two holes in them. We can use the skimage.measure.regionprops property euler_number, described in the documentation as:
Euler characteristic of the set of non-zero pixels. Computed as number of connected components subtracted by number of holes (input.ndim connectivity). In 3D, number of connected components plus number of holes subtracted by number of tunnels.
So, for scissors, that will be 1-2 = -1, whereas for solid objects it's 1 and for objects with 1 hole it's 1-1 = 0. So you can say:
from skimage import measure
objects = measure.label(borders_cleared)
props_list = measure.regionprops(objects)
num_scissors = 0
for props in props_list: # one RegionProps object per region
if props.euler_number == -1:
num_scissors += 1
When the segmentation itself is easy, as in the image you showed, then my strategy would always be to find a property or combination of properties in regionprops that allows me to distinguish the objects I'm interested from others. This could be size, elongation, roundness, ... Using the extra_properties= keyword argument, you can even compute other properties defined by any function you can imagine.
I am generating images (thumbnails) from a video every 3 seconds. Now I need to discard/remove all the similar images. Is there a way I could this?
I generate thumbnails using FFMPEG. I read about various image-diff solutions like given in this SO post, but I do not want to do this manually. How and what parameters should be considered that could tell if a particular image is similar to other images present.
You can calculate the Structural Similarity Index between images and based on the score keep or discard an image. There are other measures you can use, but basically a method that returns a score. Try PIL or OpenCV
https://pillow.readthedocs.io/en/3.1.x/reference/ImageChops.html?highlight=difference
https://www.pyimagesearch.com/2017/06/19/image-difference-with-opencv-and-python/
I dont have enough reputation to comment my idea on your problem, so i will just go ahead and post it as an answer in hope of helping you.
I am quite confused about the term "similar" but since you are reffering on video frames i am going to assume that you want to avoid having "similar" frames that have been captured because of poor camera movement. If that's the case you might want to consider using salient point descriptors.
To be more specific you can detect salient points (using for instance Harris) and then use a point descriptor algorithm (such as SURF) and discard the frames that have been found to have "too many" similar points with a pre-selected frame.
Keep in mind that in order for the above process to be successful, the frames must be as sharp as possible, i guess you don't want to extract as a thubnail a blurred frame anyway. So applying a blurred images detection might be useful in your case.
I have an image containing cells. I can't provide it, but it is similar to the image used as an example here: http://blogs.mathworks.com/steve/2006/06/02/cell-segmentation/ but without the characteristic nuclei.
I have done some processing and am now left with a pretty good segmentation, but some cells are close to each other and I need to split them. Most of them consist of more or less overlapping ellipses.
I am certain that a few iterations of simple erosion will split almost all of those regions. But some of the other cells are so small, they will disappear before the others split. Therefore I need an algorithm that erodes the image, allowing region splitting, but does not delete the last pixel of a region.
I want to use watershed afterwards to segment the cells.
I guess I could implement this on my own by searching for cennected regions and then tracking that I don't lose any or something like that, but the implementation seems messy even in my head and I think there must be an easier way. So my question is basically, what's the name of this so I can google an implementation? Or if there is no off-the-shelf solution, what's an elegant way of implementing this without dozens of iterations and for loops etc.
(Language is python)
It's a classical problem, and if the overlap between cells is too important, let's say 40% or more, then there is not a good solution.
However, if the overlap is not important, here is the solution:
You start from the segmentation you have, let's call it S
You computer the ultimate eroded UE(S). It will give you the center of each cell. It will give you something like the red points on this image. In this image, they use a distance map, an ultimate eroded will be more stable. If there are still many red points per cell, then a dilation of the UE(S) will fix your problem like this example.
You invert Inv(S) or compute the voronoi diagram Voi(S) in order to have a marker in the background.
Watershed on the gradient image of S, using the UE(S) as inner marker (perfect because you have one point by cell) and Inv(S) or Voi(S) as background/outer marker.
You will get something like this example.