We Keep Coding

Frame Extraction at Fixed FPS Value

While extracting frames from a video using OpenCV, how can I set a particular value at which the frame extraction will occur?
I have seen many available sample codes for image extraction, but they are not shown any option for the frame rate.

There are many ways of frame extraction, one is to use ffmg for frame extraction.
other is, You can try this code, but we can't use any random value that you will understand while trying at different values.
change directories ap per your system.
import math
count = 0
videoFile = "train/train.mp4"
cap = cv2.VideoCapture(videoFile)
frameRate = cap.get(5) #frame rate
x=1
while(cap.isOpened()):
frameId = cap.get(1)
ret, frame = cap.read()
if (ret != True):
break
else (frameId % math.floor(frameRate) == 0):
filename ="train/frame2/frame%d.jpg" % count;count+=1
cv2.imwrite(filename, frame)
cap.release()
print ("Done!")

Trouble retrieving information from 4d array after np.roll on one axis

I'm trying to write some code that will save a set amount of buffer frames from a video to an array and later output them once a certain criteria is met. For whatever reason my frames are being 'scrambled' when I put them through the array and coming out looking like this. This is all a little over my head so I don't really know what to look for but if anyone knows why my frames are being scrambled I would appreciate some input.
import cv2
import numpy as np
interval = 1
inputclip = 'Clip.mp4'
outputclip = 'outtest.avi'
buffer = 30
cap = cv2.VideoCapture(inputclip)
ret, frame = cap.read()
height, width, nchannels = frame.shape
fourcc = cv2.VideoWriter_fourcc(*'MJPG')
out = cv2.VideoWriter( outputclip,fourcc, 60, (width,height))
#4d array which stores the buffer frames
frameArray = np.empty((buffer, 1080,1920,3))
while(True):
#roll frameArray to the 'right' one place then save a new frame to the first index
frameArray = np.roll(frameArray, 1, axis=0)
frameArray[1] = frame
# new frame
ret, frame = cap.read()
if not ret:
break
# show video
cv2.imshow('Type "q" to close.',frame)
# output video to file
out.write( frame )
# check for keystroke
key = cv2.waitKey(interval) & 0xFF
# exit if so-commanded
if key == ord('q'):
print('received key q' )
break
#write buffer frames in the proper order
for i in range(0,buffer):
out.write(frameArray[buffer-1-i])

The default type of NumPy array is float, you should set the type of frameArray to np.uint8.
The following line:
frameArray = np.empty((buffer, 1080,1920,3))
Allocates an empty array of type float.
The following line:
frameArray[1] = frame
Places an array of type np.uint8 in index 1 of frameArray.
NumPy converts the type to float without a warning...
It's not a "normal" conversion like type casting, but "viewing" the frame as float instead.
np.roll(frameArray, 1, axis=0), rolls frameArray as array of type float.
The result is "confusing".
Solution:
Allocate frameArray as type np.uint8:
frameArray = np.empty((buffer, height, width, nchannels), np.uint8)
Apparently the type of an empty NumPy array is important...
Other issues:
Don't forget to release VideoWriter at the end.
Releasing VideoCapture is also recommended:
cap.release()
out.release()
I think frame should be placed at frameArray[0], and not at frameArray[1]:
frameArray[0] = frame
You have missed the first frame.
You may add the following line above the loop:
frameArray[0] = frame
It looks like you need to start writing to the output video file, only when frameArray buffer is full.
Here is the complete code:
(The code includes few modification, I have made for testing).
import cv2
import numpy as np
interval = 1
inputclip = 'Clip.mp4'
outputclip = 'outtest.avi'
buffer = 10
cap = cv2.VideoCapture(inputclip)
ret, frame = cap.read()
height, width, nchannels = frame.shape
fourcc = cv2.VideoWriter_fourcc(*'MJPG')
out = cv2.VideoWriter(outputclip, fourcc, 60, (width, height))
#4d array which stores the buffer frames
frameArray = np.empty((buffer, height, width, nchannels), np.uint8)
frameArray[0] = frame # Keep the first frame.
frame_counter = 0 # Count frames
while(True):
# Roll frameArray to the 'right' one place then save a new frame to the first index
frameArray = np.roll(frameArray, 1, axis=0)
frameArray[0] = frame
# New frame
ret, frame = cap.read()
if not ret:
break
# Show video
cv2.imshow('Type "q" to close.', frame)
# Output video to file, only when the buffer is full.
if frame_counter >= buffer:
out.write(frameArray[-1]) # Write the last frame in frameArray.
# Check for keystroke
key = cv2.waitKey(interval) & 0xFF
# Exit if so-commanded
if key == ord('q'):
print('received key q')
break
frame_counter += 1
# Write buffer frames in the proper order
for i in range(0, buffer):
out.write(frameArray[buffer-1-i])
# Must execute release!
cap.release()
out.release()
cv2.destroyAllWindows() # Close the window at the end.

How to crop a video (such as a zoom call) into multiple files?

I have conferance call video with different people's tiles arranged on a grid.
Example:
gallery view zoom
Can I crop every video tile to a separate file using python or nodejs?

Yes, you can achieve that using OpenCV library
Read the video in OpenCV using VideoCapture API. Note down framerate while reading.
Parse through each frame and crop the frame:
Write the frame in a video using OpenCV VideoWriter
Here is the example code using (640,480) to be the new dimensions:
cap = cv2.VideoCapture(<video_file_name>)
fps = cap.get(cv2.CAP_PROP_FPS)
out = cv2.VideoWriter('<output video file name>, -1, fps, (640,480))
while(cap.isOpened()):
ret, frame = cap.read()
crop_frame = frame[y:y+h, x:x+w]
# write the crooped frame
out.write(crop_frame)
# Release reader wand writer after parsing all frames
cap.release()
out.release()

Here's the code (tested). It works by initialising a number of video outputs, then for each frame of the input video: cropping the region of interest (roi) and assigning each to the relevent output video. You might need to make tweaks depending on input video dimensions, number of times, offsets etc.
import numpy as np
import cv2
import time
cap = cv2.VideoCapture('in.mp4')
ret, frame = cap.read()
(h, w, d) = np.shape(frame)
horiz_divisions = 5 # Number of tiles stacked horizontally
vert_divisions = 5 # Number of tiles stacked vertically
divisions = horiz_divisions*vert_divisions # Total number of tiles
seg_h = int(h/vert_divisions) # Tile height
seg_w = int(w/horiz_divisions) # Tile width
# Initialise the output videos
outvideos = [0] * divisions
for i in range(divisions):
outvideos[i] = cv2.VideoWriter('out{}.avi'.format(str(i)),cv2.VideoWriter_fourcc('M','J','P','G'), 10, (seg_w,seg_h))
# main code
while(cap.isOpened()):
ret, frame = cap.read()
if ret == True:
vid = 0 # video counter
for i in range(vert_divisions):
for j in range(horiz_divisions):
# Get the coordinates (top left corner) of the current tile
row = i * seg_h
col = j * seg_w
roi = frame[row:row+seg_h,col:col+seg_w,0:3] # Copy the region of interest
outvideos[vid].write(roi)
vid += 1
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
# Release all the objects
cap.release()
for i in range(divisions):
outvideos[i].release()
# Release everything if job is finished
cv2.destroyAllWindows()
Hope this helps!

Error message when I save my video with OpenCV

I'm having an error when I run my code, I don't understand what is happening but I think is when the program finish because I have the result what I want, that is convert an existing video to gray scale and save it.
cv2.error: OpenCV(4.1.0) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'
cap = cv2.VideoCapture('videos/output.avi')
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('results/output.avi', fourcc, 20.0, (640, 480))
while (cap.isOpened()):
_, frame = cap.read()
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
out.write(frame)
# Key events
key = cv2.waitKey(1)
if key == 27: # esc
break
cap.release()
cv2.destroyAllWindows()
Thank you!

There may be a case where at least one of the frames in your video weren't read in properly. That's why the cv2.cvtColor method is throwing an error because the frame data you are providing is empty.
You should consider using the first output of cv2.VideoCapture.read() to make sure the video frame was properly captured, then write it to file. The first output is a flag that determines if the current frame was read in successfully. Also, you'll need to handle the end where we reach the end of the video. In that case, the flag will be False so that we should exit the loop. Finally, if it's your intent to write grayscale frames, there is an optional fifth parameter in cv2.VideoWriter called isColor where we can set this to False to allow us to directly write grayscale frames. This means the call to cv2.cvtColor is no longer required.
One additional thing I'll recommend is to infer the frame width and height from the video file instead of setting it yourself. This way the input and output resolution is the same. Finally, don't forget to release the cv2.VideoWriter object when you're finished and I've added an additional check for the video file to see if it has properly opened:
import numpy as np
import cv2
import sys
cap = cv2.VideoCapture('videos/output.avi')
# Check to see if the video has properly opened
if not cap.isOpened():
print("File could not be opened")
sys.exit(1)
fourcc = cv2.VideoWriter_fourcc(*'XVID')
frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) # Get the frame width and height
frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
# Change
out = cv2.VideoWriter('results/output.avi', fourcc, 20.0, (frame_width, frame_height), isColor=False)
while True:
ret, frame = cap.read() # New
if not ret: # New
break # Get out if we don't read a frame successfully
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
out.write(frame)
# Key events
key = cv2.waitKey(1)
if key == 27: # esc
break
cap.release()
out.release() # New
cv2.destroyAllWindows()
As a minor note, you don't have any windows being displayed, so cv2.destroyAllWindows() is superfluous here. Consider removing it from your code.

So this answer has another approach(Here,you can also extract different color by changing the corresponding weights in front of the B,G, R values)
import cv2
cap = cv2.VideoCapture('videos/output.avi')
frame_width = int(cap.get(3)) # finds the frame width automatically
frame_height = int(cap.get(4)) # finds the frame height automatically
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('results/outpy.avi', cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), 10, (frame_width, frame_height))
while (cap.isOpened()): # value is true if the file is successfully opened.
ret, frame = cap.read()
if ret == True: # checks if the return value is True or False. False means file ended.
# grey = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # the grey matrix has a different shape than the frame matrix
# that's why the output files were blank
# to circumvent this RGB2GRAY, I manually added the "NORMALIZED" R,G,B values.
frame[:,:,0] = 0.114*frame[:,:,0]+0.587*frame[:,:,1]+0.299*frame[:,:,2] #multiplying normalized co-efficients to B,G,R
# for extracting red, make 0.299 as 1.0 and others as 0.0; same goes for other colours.
frame[:, :, 1]=frame[:,:,0] # making the G and R values same as the B.
frame[:, :, 2]=frame[:,:,0]
# now frame is a 3d grayscale matrix. Identical to the cv2.cvtColor method....except it is 3d
# now frame is grey scaled...R,G,B values for each pixel,holds the same number....
out.write(frame)
else:
break
cap.release()
out.release()
cv2.destroyAllWindows()

How to create array of frames from an mp4 with opencv for python

I am attempting to use opencv_python to break an mp4 file down into it's frames so I can later open them with pillow, or at least be able to use the images to run my own methods on them.
I understand that the following snippet of code gets a frame from a live video or a recorded video.
import cv2
cap = cv2.VideoCapture("myfile.mp4")
boolean, frame = cap.read()
What exactly does the read function return and how can I create an array of images which I can modify.

adapted from How to process images of a video, frame by frame, in video streaming using OpenCV and Python. Untested. However, the frames are read into a numpy array and and append to a list that is converted to a numpy array when the all the frames are read in.
import cv2
import numpy as np
images = []
cap = cv2.VideoCapture("./out.mp4")
while not cap.isOpened():
cap = cv2.VideoCapture("./out.mp4")
cv2.waitKey(1000)
print "Wait for the header"
pos_frame = cap.get(cv2.cv.CV_CAP_PROP_POS_FRAMES)
while True:
frame_ready, frame = cap.read() # get the frame
if frame_ready:
# The frame is ready and already captured
# cv2.imshow('video', frame)
# store the current frame in as a numpy array
np_frame = cv2.imread('video', frame)
images.append(np_frame)
pos_frame = cap.get(cv2.cv.CV_CAP_PROP_POS_FRAMES)
else:
# The next frame is not ready, so we try to read it again
cap.set(cv2.cv.CV_CAP_PROP_POS_FRAMES, pos_frame-1)
print "frame is not ready"
# It is better to wait for a while for the next frame to be ready
cv2.waitKey(1000)
if cv2.waitKey(10) == 27:
break
if cap.get(cv2.cv.CV_CAP_PROP_POS_FRAMES) == cap.get(cv2.cv.CV_CAP_PROP_FRAME_COUNT):
# If the number of captured frames is equal to the total number of frames,
# we stop
break
all_frames = np.array(images)

Simply use this code to get an array of frames from your video:
import cv2
import numpy as np
frames = []
video = cv2.VideoCapture("spiderino_turning.mp4")
while True:
read, frame= video.read()
if not read:
break
frames.append(frame)
frames = np.array(frames)
but regarding your question, video.read() returns two values. The first one (read in the example code) indicates if the frame is successfully read or not (i.e., True on succeeding and False on any error). The second returning value is the frame that can be empty if the read attempt is unsuccessful or a 3D array (i.e., color image) otherwise.
But why can a read attempt be unsuccessful?
If you are reading from a camera, any problem with the camera (e.g., the cable is disconnected or the camera's battery is dead) can cause an error.
If you are reading from a video, the read attempt will fail when all the frames are read, and there are no more.