We Keep Coding

Extract Data from an Image with Python/OpenCV/Tesseract?

I'm trying to extract some contents from a cropped image. I tried pytesseract and opencv template matching but the results are very poor. OpenCV template matching sometimes fails due to poor quality of the icons and tesseract gives me a line of text with false characters.
I'm trying to grab the values like this:
0:26 83 1 1
Any thoughts or techniques?

A technique you could use would be to blur your image. From what it looks like, the image is kind of low res and blurry already, so you wouldn't need to blur the image super hard. Whenever I need to use a blur function in Opencv, I normally choose the gaussian blur, as its technique of blurring each pixel as well as each surrounding pixel is great. Once the image is blurred, I would threshold, or adaptive threshold the image. Once you have gotten this far, the image that should be shown should be mostly hard lines with little bits of short lines mixed between. Afterwards, dilate the threshold image just enough to have the bits where there are a lot of hard edges connect. Once a dilate has been performed, find the contours of that image, and sort based on their height with the image. Since I assume the position of those numbers wont change, you will only have to sort your contours based on the height of the image. Afterwards, once you have sorted your contours, just create bounding boxes over them, and read the text from there.
However, if you want to do this the quick and dirty way, you can always just manually create your own ROI's around each area you want to read and do it that way.
First Method
Gaussian blur the image
Threshold the image
Dilate the image
Find Contours
Sort Contours based on height
Create bounding boxes around relevent contours
Second Method
Manually create ROI's around the area you want to read text from

Opencv - Extracting data from in-game images

I need some help with an OpenCV project I'm working on. I'm taking images from a computer game (in this case, Fortnite), and I would like to extract different elements from them, eg. timer value, quantities of materials, health and shield etc.
Currently I perform a series of image preprocessing functions until I get a binary image, followed by locating the contours in the image and then sending those contours to a machine learning algorithm (K-Nearest-Neighbours).
I am able to succeed in a lot of cases, but there are some images where I don't manage to find some of the contours, therefore I don't find all the data.
An important thing to note is that I use the same preprocessing pipeline for all images, because I'm looking for as robust of a solution that I can manage.
I would like to know what I can do to improve the performance of my program. -
Is KNN a good model for this sort of task, or are there other models that might give me better results?
Is there any way to recognise characters without locating contours?
How can I make my preprocessing pipeline as robust as possible, given the fact that there is a lot of variance in the background across all images?
My goal is to process the images as fast as possible, starting out with a minimum of at least 2 images per second.
Thanks in advance for any help or advice you can give me!
Here is an example image before preprocessing
Here is the image after preprocessing, in this example I cannot find the contour for the 4 on the right side.

Quite simply, enlarging the image might help, since it increases the dark border of the number.
I threw together some code that does that. The result could be improved, but my point here is to show that the 4 can now be detected as a contour. To increase efficiency I only selected contours within a certain size.
Also, since it is part of the HUD, that usually means that the location on screen is always the same. If so, you can get great performance increase by only selecting the area with values (described here) - as I have done manually.
Finally, since the numbers have a consistent shape, you could try matchShapes as an alternative to kNN to recognize the numbers. I don't know how they compare in performance though, so you'll have to try that out yourself.
Result:
Code:
import numpy as np
import cv2
# load image
img = cv2.imread("fn2.JPG")
# enlarge image
img = cv2.resize(img,None,fx=4, fy=4, interpolation = cv2.INTER_CUBIC)
# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
# create mask using threshold
ret,mask = cv2.threshold(gray,200,255,cv2.THRESH_BINARY)
# find contours in mask
im, contours, hierarchy = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
# draw contour on image
for cnt in contours:
if cv2.contourArea(cnt) < 3000 and cv2.contourArea(cnt) > 200:
cv2.drawContours(img, [cnt], 0, (255,0,0), 2)
#show images
cv2.imshow("Mask", mask)
cv2.imshow("Image", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

How to convert from edges to contours in OpenCV

I have been getting images like this after edge detection:
I'd like it to connect the edges together into straight-line polygons.
I thought this could be done using findCountours with chain approximation, but that doesn't seem to be working well for me.
How can I convert an image like the one above into a simple straight-line polygons (that look like skewed triangles and trapezoids and squares)?

You need to first detect the lines and then construct the contours. You can do that using HoughLines(). There is a short tutorial here.

Blur the image, then find the contours.
If the edges are that close together, a simple blurring with something like
def blur_image(image, amount=3):
'''Blurs the image
Does not affect the original image'''
kernel = np.ones((amount, amount), np.float32) / (amount**2)
return cv2.filter2D(image, -1, kernel)
should connect all the little gaps, and you can do contour detection with that.
If you then want to convert those contours into polygons, you can look to approximate those contours as polygons. A great tutorial with code for that is here.
The basic idea behind detecting polygons is running
cv2.findContours(image, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
Which tells OpenCV to approximate the contours with straight lines, creating polygons. (I'm not sure what the cv2.RETR_EXTERNAL parameter does at this time.)

Canny Operation on Image

I want in the image below to (by using Python):
1.) Find the contours of the bones (only the sides will do)
2.) Recognize and draw all the contours.
It could look something like this:
A better contour is even good. I am not entirely sure as to how I could tackle this,
The gradient of the image is:

A initial way of approaching this would be using canny edge detection using the right threshold values and then find the contours.
import cv2
# Load the image
img = cv2.imread("/home/tribta/Desktop/feet.png")
# Find the contours
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(img,60,200)
im2, contours, hierarchy = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
hierarchy = hierarchy[0] # get the actual inner list of hierarchy descriptions
# For each contour, find the bounding rectangle and draw it
cv2.drawContours(img, contours, -1, (0,255,0), 3)
# Finally show the image
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Then you could add some biomedical processing criteria to distinguish the different contours and verify if it is really a bone.

You should apply first a threshold using cv2.threshold to clean your images from objects you don't want to, try different values of intensity to catch the borders, then apply some morphology operation such as OPENING and CLOSING with cv2.morphologyEx to clean a bit the image, fill the holes and finally applied cv2.findContours and cv2.drawContours to get the finally image of the contour of the bones.
check those command on opencv library
You will find really good examples on stackoverflow and internet, try adapting your code to those.
Hope this was a good help to you.

Contour Arrays in OpenCV

In another user's post, they discussed how they were having trouble using OpenCV's Hough Circle Transform to detect and draw over each ring on a bull target here. A proposed solution was to instead using contours to locate each circle. When I attempted the same thing, my result was exactly what I was looking for; the only problem is that the program I'm attempting to create needs a series of circles, one on each ring, so that I may then calculate the distance from the center and therefore the particular ring where a given point exists.
Following the code if the mentioned post, I have the following:
import cv2
import numpy as np
image = cv2.imread('bull.png')
image_copy = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
grayscaled_image = cv2.cvtColor(image_copy, cv2.COLOR_GRAY2BGR)
cv2.imshow("confirm", grayscaled_image)
cv2.waitKey(0)
cv2.destroyAllWindows()
_,contours,_ = cv2.findContours(255 - image_copy, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
print(contours)
cv2.drawContours(image, contours, -1, color=(0,255,0), thickness=1)
cv2.imshow("detected circles", image)
cv2.waitKey(0)
cv2.destroyAllWindows()
Which gives me the same results as shown in the linked post.
And this is great. I love the progress. However, I initially tried using the same method that the linked poster did because I liked the coordinate and radius information provided by the circles.
I found in one of OpenCV's tutorials that you can contain a contour with a circle, rectangle, or ellipse. The tutorial is rather easy to follow for an individual contour, but in my code, I have no idea what _,contours,_ means. Is it some sort of array? How do I access the individual contours? When I tried printing the variable, it gave me close to 4,000 lines. I won't post them here, of course, but they appeared to be sets of coordinates.
So my question is: How can I go about converting this grouping of contours into more manageable circles?

As I mentioned in your previous question you can use minEnclosingCircle on each contour. and you'll get circles centers and radiuses.