I am trying to build up an algorithm to detect some objects and track them over time. My input data is a tif multi-stack file, which I read as a np array. I apply a U-Net model to create a binary mask and then identify the coordinates of single objects using scipy.
Up to here everything kind of works but I just cannot get my head around the tracking. I have a dictionary where keys are the frame numbers and values are lists of tuples. Each tuple contain the coordinates of each object.
Now I have to link the objects together, which on paper seems pretty simple. I was hoping there was a function or a package to do so (ideally something similar to trackMate or M2track on ImageJ), but I cannot find anything like that. I am considering writing my own nearest neighbor tool but I'd like to know whether there is a less painful way (and also, I would like to consider also more advanced metrics).
The other option I considered is using cv2, but this would require converting the data in a format cv2 likes, which will significantly slow down the code. In addition, I would like to keep the data as close as possible to the original input, so no cv2 for me.
I solved it using trackpy.
http://soft-matter.github.io/trackpy/v0.5.0/
trackpy properly reads multistack tiff files (OpenCv can't).
Related
for my school project, I need to find images in a large dataset. I'm working with python and opencv. Until now, I've managed to find an exact match of an image in the dataset but it takes a lot of time even though I had 20 images for the test code. So, I've searched few pages of google and I've tried the code on these pages
image hashing
building an image hashing search engine
feature matching
Also, I've been thinking to search through the hashed dataset, save their paths, then find the best feature matching image among them. But most of the time, my narrowed down working area is so much different than what is my query image.
The image hashing is really great. It looks like what I need but there is a problem: I need to find an exact match, not similar photos. So, I'm asking you guys, if you have any suggestion or a piece of code might help or improve the reference code that I've linked, can you share it with me? I'd be really happy to try or research what you guys send or suggest.
opencv is probably the wrong tool for this. The algorithms there are geared towards finding similar matches, not exact ones. The general idea is to use machine learning to teach the code to recognize what a car looks like so it can detect cars in videos, even when the color or form changes (driving in the shadow, different make, etc).
I've found two approaches work well when trying to build an image database.
Use a normal hash algorithm like SHA-256 plus maybe some metadata (file or image size) to find matches
Resize the image down to 4x4 or even 2x2. Use the pixel RGB values as "hash".
The first approach is to reduce the image to a number. You can then put the number in a look up table. When searching for the image, apply the same hashing algorithm to the image you're looking for. Use the new number to look in the table. If it's there, you have a match.
Note: In all cases, hashing can produce the same number for different pictures. So you have to compare all the pixels of two pictures to make sure it's really an exact match. That's why it sometimes helps to add information like the picture size (in pixels, not file size in bytes).
The second approach allows to find pictures which very similar to the eye but in fact slightly different. Imagine cropping off a single pixel column on the left or tilting the image by 0.01°. To you, the image will be the same but for a computer, they will by totally different. The second approach tries to average small changes out. The cost here is that you will get more collisions, especially for B&W pictures.
Finding exact image matches using hash functions can be done with the undouble library (Disclaimer: I am also the author). It works using a multi-step process of pre-processing the images (grayscaling, normalizing, and scaling), computing the image hash, and the grouping of images based on a threshold value.
I need to copy images from 'Asset' folder in Windows 10 which has background images automatically downloaded. Some of these images will never be displayed and at some point deleted. To make sure I have seen all the new images before they are deleted I have created a Python script that copy these images into a different folder. To efficient I need a way to compare two images those that only the new ones are copied. All I need to do is to have a function that takes two images compare them with a simple approach to be sure that the two images are not visually identical. A simple test would be to take an image file copy it and compare the copy and the original, in which case the function should be able to tell that those are the same images.
How can I compare two images in python? I need simple and efficient way to do it. Several answers I have read are a bit complicated.
I encountered a similar problem before. I used PIL.Image.tobytes() to convert the image to a byte object, then call hash() on the byte object and compared the hash values.
Compare two images in python
Option 1:
Use ImageChops module and it contains a number of arithmetical image operations, called channel operations (“chops”). These can be used for various purposes, including special effects, image compositions, algorithmic painting, and more.
Example:
ImageChops.difference(image1, image2) ⇒ image
Returns the absolute value of the difference between the two images.
out = abs(image1 - image2)
Option 2:
Scikit-image is an image processing toolbox for SciPy.
In scikit-image, please use the compare_ssim to Compute the mean structural similarity index between two images.
References:
Python Compare Two Images
I have seen an example of loading a TIFF sequence of say a 3D tiff stack or animation. However I cannot figure out how to do the opposite, taking say, a 3D+ numpy array and saving it as a TIFF sequence. Are there are any examples of encoding this? I can read a 2D array using PIL.fromarray. It would be nice if this method had some way of loading a multi-dimensional array but a naive method call will throw an unsupported-type exception.
Presumably if one did write such a sequence they might also want to add some headers to dictate channels, time and so forth. My particular bias is being able to open such images in applications like ImageJ/FIJI or converting from TIFF to other formats. Maybe there are better ways to go about this in the first place.
I know this is very old, but in case anyone comes here looking for the answer, this does seem to have been solved nicely.
e.g.
im.save("filename.tiff", format="TIFF", save_all=True)
https://pillow.readthedocs.io/en/latest/releasenotes/3.4.0.html
I was wondering how to create a fingerprint database. If fingerprints are stored as images, how do you compare images in a database, or create an image search engine like TinEye?
I know this is a big subject, but I'm just looking for a starting point. Can this be done using Python/Django libraries and MySQL?
OpenCV comes with a sample program that does what you are looking for. It's called find_obj.py. Pull it up in your editor and change:
surf = cv2.SURF(1000)
to
surf = cv2.SURF(100)
This should find lots of "inlier" points of interest in the negative of the fingerprint scan.
You can play around with a number of the variables and eventually find the best configuration for the sort of images you're comparing. It's also fairly straightforward to alter the sample to allow you to compare a single image against an entire directory.
I should point out that this will only be effective for the sort of digitized fingerprint scans used by law enforcement.
The Python Imaging Library is probably the best library to get started on image processing with.
The library most commonly used for real time image processing (you don't need real time, but you can't go wrong with fast) is OpenCV. It has Python Bindings and built-in feature detection algorithms. See also this comparison.
For an overview of image comparison algorithms take a look at this question.
As a very simple approach you can crawl all images and compute a hash for each.
Later on, when user submits an image for a search, you compute a hash for that too and look for the same hash in your database.
However, this is really simplistic approach and will only work if searched for exact image copies. Ideally, each image should be converted to some simplified feature set (to have tolerance against different versions of the same image --- different formats, sizes, noise, etc.) used for comparison. For instance, it could be worth trying convert images (both crawled and submitted for search) to grayscale of 128x128 size and compute hash of that.
Does anyone know a way get the pixel data from a PythonMagick.Image instance without having to write it to disk first?
For instance, I can read in an image using:
import PythonMagick
im = PythonMagick.Image('image.jp2')
I would now like to be able to get the uncompressed image data so that I can use it in something else like NumPy or matplotlib, but I can't seem to find any way to do this. I would just use matplotlib or PIL directly but the image format I'm reading in is JPEG 2000 which is only supported by PythonMagick as far as I know.
Any suggestions?
Disclaimer: I don't have PythonMagick built where I am right now and am no expert, so (1) any or all of the following may be wrong, (2) it will certainly be less specific than you'd like, and (3) if someone else knows better I hope they won't be put off by seeing an answer already here. Anyway:
From a quick look at the code, it looks as if you can read pixel values one by one using the pixelColor method on the Image class. This returns a PythonMagick.Color value, from which you can extract R,G,B components. The underlying C++ library supports reading out lots of pixels at a time using Image::writePixels, which is also present in PythonMagick.Image; but I think the proper use of that method depends on other things that aren't implemented in PythonMagick. That's a pity, because I bet it would have been much much more efficient than reading one pixel at a time.
Alternatively and probably better, it looks as if you can write the contents of the image to a PythonMagick.Blob object in memory, which basically does the same as writing to a file only without the file :-). You can choose what format it should write in, just as you do when writing to a file. There seems to be something called get_blob_data for extracting the contents of a Blob. Something like this:
im = PythonMagick.Image('image.jp2')
blob = PythonMagick.Blob()
im.write(blob, "png")
data = PythonMagick.get_blob_data(blob)
The resulting data is, I think, a Python string whose bytes are the binary representation of the image. (I'm assuming you're using Python 2.x, where the string type is 8-bit. I don't know whether PythonMagick works with 3.x.) I think there are some formats that are basically raw pixel data; try "RGB". You can then extract the contents via lots of struct.unpack or whatever.