I'm working on the optical flow tutorial of openCV using Python 2.7 with OpenCV 3.1.0 and have a question concerning the use of cv2.line(). Here is the original code with the highlighted part of interest:
import numpy as np
import cv2
cap = cv2.VideoCapture('slow.flv')
# params for ShiTomasi corner detection
feature_params = dict( maxCorners = 100,
qualityLevel = 0.3,
minDistance = 7,
blockSize = 7 )
# Parameters for lucas kanade optical flow
lk_params = dict( winSize = (15,15),
maxLevel = 2,
criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))
# Create some random colors
color = np.random.randint(0,255,(100,3))
# Take first frame and find corners in it
ret, old_frame = cap.read()
old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)
p0 = cv2.goodFeaturesToTrack(old_gray, mask = None, **feature_params)
# Create a mask image for drawing purposes
mask = np.zeros_like(old_frame)
while(1):
ret,frame = cap.read()
frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# calculate optical flow
p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)
# Select good points
good_new = p1[st==1]
good_old = p0[st==1]
################## IMPORTANT ##################
# draw the tracks
for i,(new,old) in enumerate(zip(good_new,good_old)):
a,b = new.ravel()
c,d = old.ravel()
mask = cv2.line(mask, (a,b),(c,d), color[i].tolist(), 2)
frame = cv2.circle(frame,(a,b),5,color[i].tolist(),-1)
################## IMPORTANT ##################
########### START insert code below ###########
# Mean-vector of camera movement
############ END insert code below ############
img = cv2.add(frame,mask)
cv2.imshow('frame',img)
k = cv2.waitKey(30) & 0xff
if k == 27:
break
# Now update the previous frame and previous points
old_gray = frame_gray.copy()
p0 = good_new.reshape(-1,1,2)
cv2.destroyAllWindows()
cap.release()
In my workspace the variables a, b, c and d are shown as array scalar float32. So I would assume, that they need to be converted to tuples of int in order to execute cv2.line() or cv2.circle().
When I try to add code using cv2.line() I have to use a conversion to int (see below), otherwise I receive a very clear message: TypeError: integer argument expected, got float
###################### START added code
ofvec = p1 - p0
ofvec = np.mean(ofvec, 1) # Collapse the first dimension
ofvec_cam = np.mean(ofvec,0) # mean of camera movement
height, width = old_frame.shape[:2]
x0 = np.int(width/2)
y0 = np.int(height/2)
pt_center = (x0, y0)
x = np.int( x0 - ofvec_cam[0].tolist() )
y = np.int( y0 - ofvec_cam[1].tolist() )
pt_ofvec_cam = (x, y)
frame = cv2.line(frame, pt_center, pt_ofvec_cam, [0, 0, 255], 2)
###################### END added code
Can anyone explain this difference to me? Thanks in advance and have a nice day!
AMTQ
It seems that cv2.line() treats differently two types of floats: "standard" Python floats and numpy floats. See the minimum working example using Python 2.7 with OpenCV 3.1.0:
import numpy as np, cv2
mask = np.zeros([10, 20, 3], dtype=np.uint8)
color = [0, 0, 0]
# Using Numpy
a = np.float32(12.34)
mask = cv2.line(mask, (a,a), (a,a), color)
# Using standard Python data type
b = 12.34
mask = cv2.line(mask, (b,b), (b,b), color)
In case a the command executes without a hitch, in case b we find the above mentioned error:
in <module> mask = cv2.line(mask, (b,b), (b,b), color)
TypeError: integer argument expected, got float`
Concerning the initial question I confirm that in the OpenCV tutorial the variables a, b, c and d are all numpy-floats whereas in the added code the variables x and y are standard Python floats before they are converted to numpy-ints by np.int().
Remarks
Both data types provide a method__int__() which returns the int-value of the float (see also difference between native int type and the numpy int types).
The only reference to speak of that I have found is this note concerning the method fromarray in the documentation of OpenCV 2.4.13:
Note In the new Python wrappers (cv2 module) the function is not needed, since cv2 can process Numpy arrays (and this is the only supported array type).
In the docs of OpenCV 3.1.0 the method fromarray does not exist anymore.
Related
For testing I generate a grid image as matrix and again the grid points as point array:
This represents a "distorted" camera image along with some feature points.
When I now undistort both the image and the grid points, I get the following result:
(Note that the fact that the "distorted" image is straight and the "undistorted" image is morphed is not the point, I'm just testing the undistortion functions with a straight test image.)
The grid image and the red grid points are totally misaligned now. I googled and found that some people forget to specify the "new camera matrix" parameter in undistortPoints but I didn't. The documentation also mentions a normalization but I still have the problem when I use the identity matrix as camera matrix. Also, in the central region it fits perfectly.
Why is this not identical, do I use something in a wrong way?
I use cv2 (4.1.0) in Python. Here is the code for testing:
import numpy as np
import matplotlib.pyplot as plt
import cv2
w = 401
h = 301
# helpers
#--------
def plotImageAndPoints(im, pu, pv):
plt.imshow(im, cmap="gray")
plt.scatter(pu, pv, c="red", s=16)
plt.xlim(0, w)
plt.ylim(0, h)
plt.show()
def cv2_undistortPoints(uSrc, vSrc, cameraMatrix, distCoeffs):
uvSrc = np.array([np.matrix([uSrc, vSrc]).transpose()], dtype="float32")
uvDst = cv2.undistortPoints(uvSrc, cameraMatrix, distCoeffs, None, cameraMatrix)
uDst = [uv[0] for uv in uvDst[0]]
vDst = [uv[1] for uv in uvDst[0]]
return uDst, vDst
# test data
#----------
# generate grid image
img = np.ones((h, w), dtype = "float32")
img[0::20, :] = 0
img[:, 0::20] = 0
# generate grid points
uPoints, vPoints = np.meshgrid(range(0, w, 20), range(0, h, 20), indexing='xy')
uPoints = uPoints.flatten()
vPoints = vPoints.flatten()
# see if points align with the image
plotImageAndPoints(img, uPoints, vPoints) # perfect!
# undistort both image and points individually
#---------------------------------------------
# camera matrix parameters
fx = 1
fy = 1
cx = w/2
cy = h/2
# distortion parameters
k1 = 0.00003
k2 = 0
p1 = 0
p2 = 0
# convert for opencv
mtx = np.matrix([
[fx, 0, cx],
[ 0, fy, cy],
[ 0, 0, 1]
], dtype = "float32")
dist = np.array([k1, k2, p1, p2], dtype = "float32")
# undistort image
imgUndist = cv2.undistort(img, mtx, dist)
# undistort points
uPointsUndist, vPointsUndist = cv2_undistortPoints(uPoints, vPoints, mtx, dist)
# test if they still match
plotImageAndPoints(imgUndist, uPointsUndist, vPointsUndist) # awful!
Any help appreciated!
A bit late to the party, but to help others running into this issue:
The problem is that UndistortPoints is an iterative calculation which in some cases exits before a stable solution has been reached. This can be fixed by modifying the termination criteria for the calculation, which can be done by using UndistortPointsIter. You should replace:
uvDst = cv2.undistortPoints(uvSrc, cameraMatrix, distCoeffs, None, cameraMatrix)
with:
uvDst = cv2.undistortPointsIter(uvSrc, cameraMatrix, distCoeffs, None, cameraMatrix,(cv2.TERM_CRITERIA_COUNT | cv2.TERM_CRITERIA_EPS, 40, 0.03))
Now, it tries 40 iterations to find a solution, rather than the default 5 iterations.
Here is a code to get the optical flow output from a stabilized video (no camera movement) and save it as a set of frames
import cv2 as cv
import numpy as np
# The video feed is read in as a VideoCapture object
cap = cv.VideoCapture("2_stable_video.avi")
# ret = a boolean return value from getting the frame, first_frame = the first frame in the entire video sequence
ret, first_frame = cap.read()
# Converts frame to grayscale because we only need the luminance channel for detecting edges - less computationally expensive
prev_gray = cv.cvtColor(first_frame, cv.COLOR_BGR2GRAY)
# Creates an image filled with zero intensities with the same dimensions as the frame
mask = np.zeros_like(first_frame)
# Sets image saturation to maximum
mask[..., 1] = 255
count = 0
while(cap.isOpened()):
# ret = a boolean return value from getting the frame, frame = the current frame being projected in the video
ret, frame = cap.read()
# Opens a new window and displays the input frame
cv.imshow("input", frame)
# Converts each frame to grayscale - we previously only converted the first frame to grayscale
gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
# Calculates dense optical flow by Farneback method
flow = cv.calcOpticalFlowFarneback(prev_gray, gray, None, 0.5, 3, 15, 3, 5, 1.2, 0)
# Computes the magnitude and angle of the 2D vectors
magnitude, angle = cv.cartToPolar(flow[..., 0], flow[..., 1])
# Sets image hue according to the optical flow direction
mask[..., 0] = angle * 180 / np.pi / 2
# Sets image value according to the optical flow magnitude (normalized)
mask[..., 2] = cv.normalize(magnitude, None, 0, 255, cv.NORM_MINMAX)
# Converts HSV to RGB (BGR) color representation
rgb = cv.cvtColor(mask, cv.COLOR_HSV2BGR)
# Opens a new window and displays the output frame
cv.imshow("dense optical flow", rgb[40:150,120:220])
cv.imwrite("frames_modified_2/%d.png" % count, rgb[40:150,120:220])
count +=1
# Updates previous frame
prev_gray = gray
# Frames are read by intervals of 1 millisecond. The programs breaks out of the while loop when the user presses the 'q' key
if cv.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv.destroyAllWindows()
Can someone please suggest how to quantify the difference between the frames? i.e. to estimate speed/velocity ?
Here's an example to obtain pixel magnitude translation from .bsq frames. You can modify the the code to input a video file instead. You are probably most interested in the get_translation() function. Example:
Graph displaying pixel translation from frame-to-frame
Code
import numpy as np
import argparse
import os
import cv2
from matplotlib import pyplot as plt
from matplotlib import cm
import time
import random
# Usage: python translate_analyzer.py -p <filename.bsq>
# Automatic brightness and contrast optimization with optional histogram clipping
def automatic_brightness_and_contrast(image, clip_hist_percent=25):
if len(image.shape) == 3:
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
else:
gray = image
# Calculate grayscale histogram
hist = cv2.calcHist([gray],[0],None,[256],[0,256])
hist_size = len(hist)
# Calculate cumulative distribution from the histogram
accumulator = []
accumulator.append(float(hist[0]))
for index in range(1, hist_size):
accumulator.append(accumulator[index -1] + float(hist[index]))
# Locate points to clip
maximum = accumulator[-1]
clip_hist_percent *= (maximum/100.0)
clip_hist_percent /= 2.0
# Locate left cut
minimum_gray = 0
while accumulator[minimum_gray] < clip_hist_percent:
minimum_gray += 1
# Locate right cut
maximum_gray = hist_size -1
while accumulator[maximum_gray] >= (maximum - clip_hist_percent):
maximum_gray -= 1
# Calculate alpha and beta values
alpha = 255 / (maximum_gray - minimum_gray)
beta = -minimum_gray * alpha
auto_result = cv2.convertScaleAbs(image, alpha=alpha, beta=beta)
return (auto_result, alpha, beta)
# Draw flow
def draw_flow(img, flow, step=30):
h, w = img.shape[:2]
y, x = np.mgrid[step/2:h:step, step/2:w:step].reshape(2,-1).astype(int)
fx, fy = flow[y,x].T
lines = np.vstack([x, y, x+fx, y+fy]).T.reshape(-1, 2, 2)
lines = np.int32(lines + 0.5)
vis = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
cv2.polylines(vis, lines, 1, (36, 255, 12))
for (x1, y1), (_x2, _y2) in lines:
cv2.circle(vis, (x1, y1), 2, (36, 255, 12), -1)
return vis
# Return translation value
def get_translation(img, flow, step=30):
return (np.median(flow[:,:,0].T), flow[:, :, 0].T)
# Get file path
ap = argparse.ArgumentParser()
ap.add_argument("-p", "--path", help="Path to the directory")
args = vars(ap.parse_args())
if not args['path']:
print('Usage: python translate_analyzer.py -p <directory>')
exit(1)
# Extract file name
bsq_fname = os.path.split(args['path'])[-1]
if '.bsq' not in bsq_fname:
print('ERROR: Invalid bsq file. Select correct file.')
exit(1)
width = 640
height = 512
frame_count = int(os.path.getsize(bsq_fname)/(2*height*width))
x,y,w,h = 0,0,100,512
# Simulates calibrated frames to display on video frame
data_file = np.fromfile(bsq_fname, dtype=np.uint16, count=-1)
data_file = data_file.reshape((width, height, frame_count), order='F')
data_file = np.rot90(data_file)
print(bsq_fname)
fname = bsq_fname.split()[0]
prev = data_file[:,:,0].copy()
prev //= 64
prev = automatic_brightness_and_contrast(prev)[0]
prev = prev[y:y+h, x:x+w]
translation_data = []
frame_direction = []
start = time.time()
for index in range(1, frame_count):
data = data_file[:,:,index].copy()
data //= 64
data = automatic_brightness_and_contrast(data)[0]
data = data[y:y+h, x:x+w]
flow = cv2.calcOpticalFlowFarneback(prev=prev, next=data, flow=None, pyr_scale=0.5, levels=2, winsize=80, iterations=2, poly_n=7, poly_sigma=4.5, flags=0)
translation, pixel_direction = get_translation(data, flow)
prev = data
cv2.imshow('flow', draw_flow(data, flow))
cv2.waitKey(1)
translation_data.append(translation)
frame_direction = pixel_direction
index = (index+1) % frame_count
end = time.time()
print('Time:', end - start)
plt.figure()
plt.title(bsq_fname)
plt.xlabel("Frames")
plt.ylabel("Magnitude")
plt.plot(translation_data)
plt.figure()
plt.title("Pixel Direction")
plt.xlabel("Width")
plt.ylabel("Height")
plt.imshow(frame_direction.T)
plt.colorbar(orientation='vertical')
plt.show()
Updated question:
Would anyone be able to point me in the direction of any material that could help me to plot an optical flow map in python? Ideally i want to find something that provides a similar output to the video shown here: http://study.marearts.com/2014/04/opencv-study-calcopticalflowfarneback.html . Or something with a similar functional output
I have implemented the dense optical flow algorithm (cv2.calcOpticalFlowFarneback). And from this i have been able to sample the magnitudes at specified points of the image.
The video feed that is being input is 640x480, and i have set sample points to be at every fifth pixel vertically and horizontally.
import cv2
import numpy as np
import matplotlib.pyplot as plt
cap = cv2.VideoCapture("T5.avi")
ret, frame1 = cap.read()
prvs = cv2.cvtColor(frame1, cv2.COLOR_BGR2GRAY)
hsv = np.zeros_like(frame1)
hsv[..., 1] = 255
[R,C]=prvs.shape
count=0
while (1):
ret, frame2 = cap.read()
next = cv2.cvtColor(frame2, cv2.COLOR_BGR2GRAY)
flow = cv2.calcOpticalFlowFarneback(prvs, next, None, 0.5, 3, 15, 2, 5, 1.2, 0)
mag, ang = cv2.cartToPolar(flow[..., 0], flow[..., 1])
RV=np.arange(5,480,5)
CV=np.arange(5,640,5)
# These give arrays of points to sample at increments of 5
if count==0:
count =1 #so that the following creation is only done once
[Y,X]=np.meshgrid(CV,RV)
# makes an x and y array of the points specified at sample increments
temp =mag[np.ix_(RV,CV)]
# this makes a temp array that stores the magnitude of flow at each of the sample points
motionvectors=np.array((Y[:],X[:],Y[:]+temp.real[:],X[:]+temp.imag[:]))
Ydist=motionvectors[0,:,:]- motionvectors[2,:,:]
Xdist=motionvectors[1,:,:]- motionvectors[3,:,:]
Xoriginal=X-Xdist
Yoriginal=Y-Ydist
plot2 = plt.figure()
plt.quiver(Xoriginal, Yoriginal, X, Y,
color='Teal',
headlength=7)
plt.title('Quiver Plot, Single Colour')
plt.show(plot2)
hsv[..., 0] = ang * 180 / np.pi / 2
hsv[..., 2] = cv2.normalize(mag, None, 0, 255, cv2.NORM_MINMAX)
bgr = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)
cv2.imshow('frame2', bgr)
k = cv2.waitKey(30) & 0xff
if k == 27:
break
prvs = next
cap.release()
cv2.destroyAllWindows()
I think i have calculated the original and final X,Y positions of the pixels and the distances the moved and have put these into a matplotlib quiver plot.
The result i get does not coincide with the hsv plot of the dense optical flow (which i know to be correct as it was taken from the OpenCV tutorials) and the quiver plot also only shows one frame at a time and the plot must be exited before the next one displays.
Can anyone see where i have gone wrong in my calculations and how i can make the plot update automatically with each frame?
I do not know how to change the behaviour of matplotlib quiver plots, but I'm sure it is possible.
An alternative is to create a function to draw lines on top of the original image, based on the calculated optical flow. The following code should achieve this:
def dispOpticalFlow( Image,Flow,Divisor,name ):
"Display image with a visualisation of a flow over the top. A divisor controls the density of the quiver plot."
PictureShape = np.shape(Image)
#determine number of quiver points there will be
Imax = int(PictureShape[0]/Divisor)
Jmax = int(PictureShape[1]/Divisor)
#create a blank mask, on which lines will be drawn.
mask = np.zeros_like(Image)
for i in range(1, Imax):
for j in range(1, Jmax):
X1 = (i)*Divisor
Y1 = (j)*Divisor
X2 = int(X1 + Flow[X1,Y1,1])
Y2 = int(Y1 + Flow[X1,Y1,0])
X2 = np.clip(X2, 0, PictureShape[0])
Y2 = np.clip(Y2, 0, PictureShape[1])
#add all the lines to the mask
mask = cv2.line(mask, (Y1,X1),(Y2,X2), [255, 255, 255], 1)
#superpose lines onto image
img = cv2.add(Image,mask)
#print image
cv2.imshow(name,img)
return []
This code only creates lines rather than arrows, but with some effort it could be modified to display arrows.
I am using OpenCV to compute a disparity map of a scene. I have already calibrate the stereo camera by finding the intrinsic parameters individually with cv2.calibrateCamera and then with cv2.stereoCalibrate to find the rotation matrix and the translation vector.
I copy my calibration code but I think my problem is not here:
import numpy as np
import cv2
# termination criteria
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 0.000001)
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((6*7,3), np.float32)
objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)
# Arrays to store object points and image points from all the images.
objpointsL = []
imgpointsL = []
objpointsR = []
imgpointsR = []
imgR = cv2.imread('right2.jpg',0)
# Find the chess board corners
ret, cornersR = cv2.findChessboardCorners(imgR, (7,6),None)
# If found, add object points, image points (after refining them)
if ret == True:
objpointsR.append(objp)
cv2.cornerSubPix(imgR,cornersR,(11,11),(-1,-1),criteria)
imgpointsR.append(cornersR)
imgL = cv2.imread('left3.jpg',0)
#grayL = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Find the chess board corners
ret, cornersL = cv2.findChessboardCorners(imgL, (7,6),None)
# If found, add object points, image points (after refining them)
if ret == True:
objpointsL.append(objp)
cv2.cornerSubPix(imgL,cornersL,(11,11),(-1,-1),criteria)
imgpointsL.append(cornersL)
#Intrinsic parameters
distCoeffsR = np.array([1.191755160158372399e-02, -8.585146447098485067e-03, 8.429413399383550720e-04, -6.222973871460263460e-05, -7.966310474599684957e-03])
distCoeffsL = np.array([-1.627558337813042599e-02, 2.409982163230293128e-01, 4.443126374210568282e-03, 1.288079049351137243e-03, -3.177831292965794807e-01])
cameraMatrixR = np.matrix('3.252248978261580987e+02 0 3.269955537627058106e+02;0 3.228400384496266042e+02 2.341068611530280350e+02;0 0 1')
cameraMatrixL = np.matrix('4.570360097428241488e+02 0 3.465188967298854550e+02;0 4.573286269805292363e+02 2.691439570063795372e+02;0 0 1')
retval,cameraMatrixL, distCoeffsL, cameraMatrixR, distCoeffsR, R, T, E, F = cv2.stereoCalibrate(objpointsL, imgpointsL, imgpointsR, cameraMatrixL, distCoeffsL, cameraMatrixR, distCoeffsR, (640,480))
Now I cv2.stereoRectify:
lFrame = cv2.imread('izquierda.jpg')
rFrame = cv2.imread('derecha.jpg')
w, h = lFrame.shape[:2] # both frames should be of same shape
#Perform stereorectification
R1, R2, P1, P2, Q, validPixROI1, validPixROI2 = cv2.stereoRectify(cameraMatrixL, distCoeffsL, cameraMatrixR, distCoeffsR, (w,h), R, T, cv2.CALIB_ZERO_DISPARITY,0, (0,0))
#computes undistort and rectify maps
mapxL, mapyL = cv2.initUndistortRectifyMap(cameraMatrixL, distCoeffsL, R1, P1, (w,h), cv2.CV_32FC1)
mapxR, mapyR = cv2.initUndistortRectifyMap(cameraMatrixR, distCoeffsR, R2, P2, (w,h), cv2.CV_32FC1)
dstL = cv2.remap(lFrame, mapxL, mapyL,cv2.INTER_LINEAR)
dstR = cv2.remap(rFrame, mapxR, mapyR,cv2.INTER_LINEAR)
while (True):
cv2.imshow('Left normal',lFrame)
cv2.imshow('Right normal',rFrame)
cv2.imshow('Left rectify',dstL)
cv2.imshow('Right rectify',dstR)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
And these are the rectified images:
Left rectify
Right rectify
Anyone can help me please? I am getting stuck with this...
Try to switch height and width, namely change this line of your code:
w, h = lFrame.shape[:2] # both frames should be of same shape
to this:
h, w = lFrame.shape[:2] # both frames should be of same shape
I ran into the same problem and it helped me. I think this happens because OpenCV in its methods expects the second shape of numpy array as width and the first as height.
Hope, it helps.
I was also facing the similar problem and then I compared my all matrices (obtained from python) with matrices obtained from Matlab using stereocalibration app. I found out that it is happening due to incorrect calibration. I got wrong camera distortion matrices which was causing this error. You can read my full answer here: Python 2.7/OpenCV 3.3: Error in cv2.initUndistortRectifyMap . Not showing undistort rectified images
The current algorithm uses goodFeaturestoTrack to select corner points but I want to choose my own points. Secondly, I want to save the data as to which pixels has that point moved to. How would I go about solving these two problems? The code I am currently using is :-
import numpy as np
import cv2
cap = cv2.VideoCapture('output.avi')
# params for ShiTomasi corner detection
# throw every other corners below quality level. Sort rest in descending order. Pick greatest, throw rest in min and pick N greatest
feature_params = dict( maxCorners = 1, # how many pts. to locate
qualityLevel = 0.3, # b/w 0 & 1, min. quality below which everyone is rejected
minDistance = 7, # min eucledian distance b/w corners detected
blockSize = 7 ) #
# Parameters for lucas kanade optical flow
lk_params = dict( winSize = (15,15), # size of the search window at each pyramid level
maxLevel = 2, # 0, pyramids are not used (single level), if set to 1, two levels are used, and so on
criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))
# Criteria : Termination criteria for iterative search algorithm.
# after maxcount { Criteria_Count } : no. of max iterations.
# or after { Criteria Epsilon } : search window moves by less than this epsilon
# Create some random color for the pt. chosen
color = np.random.randint(0,255,(1,3))
# Take first frame and find corners in it
ret, old_frame = cap.read() #read frame
old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY) #convert to grayscale
p0 = cv2.goodFeaturesToTrack(old_gray, mask = None, **feature_params) #use goodFeaturesToTrack to find the location of the good corner.
#cvPoint pl = new cvPoint(2,3)
# Create a mask image for drawing purposes filed with zeros
mask = np.zeros_like(old_frame)
while(1):
ret,frame = cap.read()
frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# calculate optical flow
# err kind of gives us the correlation error(matching error)
p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)
# Select good points
good_new = p1[st==1]
good_old = p0[st==1]
# draw the tracks
for i,(new,old) in enumerate(zip(good_new,good_old)):
a,b = new.ravel()
c,d = old.ravel()
mask = cv2.line(mask, (a,b),(c,d), color[i].tolist(), 2)
frame = cv2.circle(frame,(a,b),5,color[i].tolist(),-1)
img = cv2.add(frame,mask)
cv2.imshow('frame',img)
k = cv2.waitKey(30) & 0xff
if k == 27:
break
#
# # Now update the previous frame and previous points
old_gray = frame_gray.copy()
p0 = good_new.reshape(-1,1,2)
#
## release and destroy all windows.
cv2.destroyAllWindows()
cv2.release()
Just as Mark Miller mentioned, You can use any feature vector as input points.
cv2.calcOpticalFlowPyrLK will use them to find on the new image new locations of those features by comparing patches of pixels around given coordinates.
To save the new ones You need to catch the returned values of cv2.calcOpticalFlowPyrLK
A little bit about other features:
cv2.goodFeaturesToTrack (Shi - Thomasi or Harris corners) are used because the two perpendicular edges are very easy to find which gives lots of features, so even if some of them get occluded You don't end up with nothing. More advanced features like ORB have a lot more information in their descriptors that can be used to link the feature on different images than two parameters aquired in Shi - Thomasi or Harris corners, and it would be a bit of waste to use them with simple Lucas - Kanade algorithm.