We Keep Coding

Corner Refine Method, CORNER_REFINE_SUBPIX, from AcUro with Python and OpenCV

Hi There
I want to increase the accuracy of the marker detection from aruco.detectMarkers. So, I want to use Corner Refine Method with CORNER_REFINE_SUBPIX, but I do not understand how it is implemented in python.
Sample code:
frame = cv.imread("test.png")
gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
para = aruco.DetectorParameters_create()
det_corners, ids, rejected = aruco.detectMarkers(gray,dictionary,parameters=para)
aruco.drawDetectedMarkers(frame,det_corners,ids)
Things I have tried:
para.cornerRefinementMethod()
para.cornerRefinementMethod(aruco.CORNER_REFINE_SUBPIX)
para.cornerRefinementMethod.CORNER_REFINE_SUBPIX
para = aruco.DetectorParameters_create(aruco.CORNER_REFINE_SUBPIX)
para = aruco.DetectorParameters_create(para.cornerRefinementMethod(aruco.CORNER_REFINE_SUBPIX))
They did not work, I’m pretty new to python ArUco so I hope that there is a simple and obvious solution.
I would also Like to implement enclosed markers like in the Documentation(Page 4). Do you happen to know if there is a way to generate these enclosed markers in python?

Concerning the first part of your question, you were pretty close: I assume your trouble is in switching and tweaking the "para" options. If so, you only need to set the corresponding values in the parameters object like
para.cornerRefinementMethod = aruco.CORNER_REFINE_SUBPIX
Note that "aruco.CORNER_REFINE_SUBPIX" is simply an integer. You can verify this by typing type(aruco.CORNER_REFINE_SUBPIX) in the console. Thus assigning values to the "para" object works like mentioned above.
You might also want to tweak the para.cornerRefinementWinSize which seems to be implemented in units of code pixels, not actual image pixel units.
Concerning the second part, you might have to write a function, that adds the boxes at the corner points, which you can get using the detectMarker function. Note that the corner points are always ordered clockwise, thus you can easily assign the correct offset values (like "up & left", "up & right" etc.).

para.cornerRefinementMethod = 1
may work.

How to use PyFFTW's wisom

I didn't see an actual example on pyfftw's documentation of how to use the 'wisdom' feature so I'm a little confused.
My code looks something like the following:
# first FFT
input = pyfftw.zeros_aligned(arraySize, dtype='complex64')
input[:] = image
fftwObj = pyfftw.builders.fft2(input, planner_effort='FFTW_EXHAUSTIVE')
imageFFT = fftwObj(input)
wisdom = pyfftw.export_wisdom()
pyfftw.import_wisdom(wisdom)
# second FFT with the same input size but different input
input = pyfftw.zeros_aligned(arraySize, dtype='complex64')
input[:] = image2
fftwObj = pyfftw.builders.fft2(input, planner_effort='FFTW_EXHAUSTIVE')
imageFFT2 = fftwObj(input)
The docs say that export_wisdom outputs a tuple of strings and that import_wisdom takes in this tuple as an argument.
When am I supposed to export the wisdom and am I supposed to save this tuple out to a file for each FFT?
When do I load it back in? Before the call to each FFT?

Basically, exporting and importing wisdom is a method to maintain state between sessions.
The wisdom is the knowledge about how best to plan an FFT. During a session, the internal "wisdom" is made up of all the plans that are made, and the wisdom that has been imported. Repeatedly importing the same wisdom file is not useful because that knowledge is already known after the first import.
You export wisdom when you want the knowledge about a particular transform plan to be used instead of having to work it out again. It need only plan for that transform once per session though.

Abaqus Python getByBoundingBox command

I have a 2D part in Abaqus with many partitions and I therefore want to select many edges with the getByBoundingBox command to create a surface set. This is the bit of code I have:
p = mdb.models['Model-1'].parts['Plate']
s = p.edges
edges = s.getByBoundingBox((0,0.02,0,0.003,0.04,0))
p.Surface(side1Edges=edges, name='r1')
But it gives me the following error: "edges = s.getByBoundingBox((0,0.02,0,0.003,0.04,0)) TypeError: arg1; found tuple, expecting float".
Any advice much appreciated.

The corners of the bounding box should be provided as 6 separate numbers and not as a single tuple. The solution is very simple, just change the leading "((" and trailing "))" to single "(" and ")". So the call looks like this s.getByBoundingBox(0,0.02,0,0.003,0.04,0).

this comes quite late but in case anyone enters and has the same doubt:
When telling to Abaqus which edge/face/element you are actually selecting, sometimes you need to specify the "ID" of that object, that's why it's asking for a float instead of a tuple in the error message. This can be solved as:
You select the edge/face/node/element you want:
edge = s.getByBoundingBox((0,0.02,0,0.003,0.04,0))
Create a intermediate variable to know the "ID" of the element:
edge_id = edge.id
You now can refenciate it in the dialog for creating surfaces:
p.Surface(side1Edges=p.edges[edge_id], name='r1')
In this case, you're telling Abaqus to select the edge with the id "edge_id" from all the edges that your part "p" has.
This happens many times and you've to be aware what Abaqus is expecting from the code. Sometimes can be the object itselft, a tuple of elements or simply a float expressed as a tupple e.g: edge = (number, )
Regards

I tried the modified code on a 2D plate with the following code
p = mdb.models['Model-1'].parts['Plate']
s = p.edges
edges=s.getByBoundingBox(0,0,0,25,25,1)
And it does not crash. But its not really clear how you can create a surface using this. You need to use a different strategy to achieve what you want. You can find create a surface using the 'pointOn' method.

Tips on performing OCR - not getting desired results

So I have the following image:
I'm trying to extract three arrays:
var a = [30,31,32,35,37,40,44];
var b = [6,7,11,15,18,21,22];
var c = [5,11,15,18,23,37,28];
I tried feeding this image into tesseract ~/Desktop/test.png out to no avail:
9 % ooenesew #
5 ‘ 904399
And here is the result from ocrad ~/Desktop/test.ppm:
o
?
28
Can any OCR experts suggest what I might try next? I'm comfortable using Python/OpenCV, but will try anything.

If your images always look like in the example, you might have to do some tidy up to remove anything that is not a number (all the black background and the circle). Then the method described in the accepted answer on the linked question might be sufficient for your needs, since it looks like you are not dealing with different fonts and sizes:
Simple Digit Recognition OCR in OpenCV-Python

How to find a template in an image using a mask (or transparency) with OpenCV and Python?

Let us assume we are looking for this template:
The corners of our template are transparent, so the background will vary, like so:
Assuming we could use the following mask with our template:
It would be very easy to find it.
What I have tried:
I have tried matchTemplate but it doesn't support masks (as far as I know), and using the alpha channel (transparency) in the template does not achieve this, as it compares the alpha channels instead of ignoring those pixels.
I have also looked into "region of interest", which I thought would be the solution, but with it you can only specify a rectangular area. I'm not even sure if it works on the template or not.
I'm sure this is possible to do by writing my own algorithm, but I was hoping this is possible via. standard OpenCV to avoid reinventing the wheel. Not to mention, it would most likely be more optimised than my own.
So, how could I do something like this with OpenCV + Python?

This could be achieved using only matchTemplate function, but a little workaround is needed.
Lets analyse the default metrics(CV_TM_SQDIFF_NORMED). According to matchTemplate documentation
the default metrics looks like this
R(x, y) = sum (I(x+x', y+y') - T(x', y'))^2
Where I is image matrix, T is template, R is result matrix. Summation is done over template coordinates x' and y',
So, lets alter this metrics by inserting weight matrix W, which has the same dimensions as
T.
Q(x, y) = sum W(x', y')*(I(x+x', y+y') - T(x', y'))^2
In this case, by setting W(x', y') = 0 you can actually make pixel be ignored. So, how to make such metrics? With simple math:
Q(x, y) = sum W(x', y')*(I(x+x', y+y') - T(x', y'))^2
= sum W(x', y')*(I(x+x', y+y')^2 - 2*I(x+x', y+y')*T(x', y') + T(x', y')^2)
= sum {W(x', y')*I(x+x', y+y')^2} - sum{W(x', y')*2*I(x+x', y+y')*T(x', y')} + sum{W(x', y')*T(x', y')^2)}
So, we divided Q metrics into tree separate sums. And all those sums could be calculated
with matchTemplate function (using CV_TM_CCORR method). Namely
sum {W(x', y')*I(x+x', y+y')^2} = matchTemplate(I^2, W, method=2)
sum{W(x', y')*2*I(x+x', y+y')*T(x', y')} = matchTemplate(I, 2*W*T, method=2)
sum{W(x', y')*T(x', y')^2)} = matchTemplate(T^2, W, method=2) = sum(W*T^2)
The last element is a constant, so, for minimisation it does not have any effect. On the other hand, it still might me useful to see if our template have perfect match (if Q is approaching to zero). Nonetheless, for last element we actually do not need matchTemplate function, since it could be calculated directly.
The final pseudocode looks like this:
result = matchTemplate(I^2, W, method=2) - matchTemplate(I, 2*W*T, method=2) + as.scalar(sum(W*T^2))
Does it really do exactly as defined? Mathematically yes.
Practically, there is some small rounding error, because matchTemplate function
works on 32-bit floating-point, but I believe it is not a big problem.
Please note, that you can extent analysis and have weighted equivalents for any metrics offered by matchTemplate.
This actually worked for me. I am sorry I don't give actual code. I am working in R, so
I don't have the code in Python. But idea is quite straightforward.
I hope this will help.

What worked for me the one time I needed this was to fill the "mask" areas with white noise. Then it gets effectively washed out of the correlation when looking for matches. Otherwise I got, as I presume you did, false matches on the masked areas.

One answer to your question is convolution. Use the template as kernel and filter the image.
The destination Mat will have dense bright areas where your template might be. You'll have to cluster the results (e.g. Mean-shift).
In that way, you'll have a very simplistic implementation of the Generalized Hough Transform or a Template-based convolution matching.

Imagemagick 7.0.3.9 now has a masked compare capability so that you can limit the template matching region. See http://www.imagemagick.org/discourse-server/viewtopic.php?f=4&t=31053
Also, I see that OpenCV 3.0 now has masked template matching. See http://docs.opencv.org/3.0.0/df/dfb/group__imgproc__object.html#ga586ebfb0a7fb604b35a23d85391329be
However, it is only for method == CV_TM_SQDIFF and method == CV_TM_CCORR_NORMED. see python opencv matchTemplate is mask feature implemented?

ImageMagick has logic for finding subimages in other images and it works quite well.
compare -verbose -dissimilarity-threshold 0.1 -subimage-search subimage bigimage
I've used it to find and blur watermarks off some products. Don't ask.
(Sometimes you have to do what you have to do..)

2021 Update: I've been trying to find a solution for transparency in templates throughout the day, and I think I finally found a way to do it. matchTemplate() has a mask parameter, which apparently works exactly like OP wants it to: ignore certain pixels from a template when searching for it in another image. And since my templates already contain transparency in them, I decided to use my template as both a template and mask parameter. Surprisingly, it worked.
I'm using JavaScript with opencv4nodejs, so the following python code snippet might be completely off, but the theory is there and I'm fairly positive it should work.
# Import OpenCV
import cv2 as cv
# Read both the image and the template
image = cv.imread("image.png", cv.IMREAD_COLOR)
template = cv.imread("template.png", cv.IMREAD_COLOR)
# Match with template as both template and mask parameter
result = cv.matchTemplate(image, template, cv.TM_CCORR_NORMED, None, template)
Here's a gist for JavaScript with opencv4nodejs if you're interested.
Now that I think about it, it seems really stupid and way too good to be true, but I've been getting good matches (0.98+) on most tests. Hope this helps!