Hello in using MOG2 to make a Background substrator from a base frame to a next frames.
but its showing me to much ruid
id like if there is another background substractor that can elimitate this ponts.
Also i have another problem.
When a car passes with flash lights on the flashlights is showed as white im mi image . i need to ignorate the reflexion of fleshlight in the ground.
Some one knows dow to do that ?
by cod for BGS:
backSub = cv2.createBackgroundSubtractorMOG2(history=1, varThreshold=150, detectShadows=True)
fgMask = backSub.apply(frame1)
fgMask2 = backSub.apply(actualframe)
maskedFrame = fgMask2 - fgMask
cv2.imshow("maskedFrame1 "+str(id), maskedFrame)
You can try to perform a Gaussian blur before sending the frame to backSub.apply() or experiment with the parameters for cv2.createBackgroundSubtractorMOG2(): if you need a better explanation of what they do, try this page.
This is the result from a 7x7 Gaussian blur using this video.
Code:
import cv2
import numpy as np
import sys
# read input video
cap = cv2.VideoCapture('traffic.mp4')
if (cap.isOpened()== False):
print("!!! Failed to open video")
sys.exit(-1)
# retrieve input video frame size
frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = cap.get(cv2.CAP_PROP_FPS)
print('* Input Video settings:', frame_width, 'x', frame_height, '#', fps)
# adjust output video size
frame_height = int(frame_height / 2)
print('* Output Video settings:', frame_width, 'x', frame_height, '#', fps)
# create output video
video_out = cv2.VideoWriter('traffic_out.mp4', cv2.VideoWriter_fourcc(*'MP4V'), fps, (frame_width, frame_height))
#video_out = cv2.VideoWriter('traffic_out.avi', cv2.VideoWriter_fourcc('M','J','P','G'), fps, (frame_width, frame_height), True)
# create MOG
backSub = cv2.createBackgroundSubtractorMOG2(history=5, varThreshold=60, detectShadows=True)
while (True):
# retrieve frame from the video
ret, frame = cap.read() # 3-channels
if (frame is None):
break
# resize to 50% of its original size
frame = cv2.resize(frame, None, fx=0.5, fy=0.5)
# gaussian blur helps to remove noise
blur = cv2.GaussianBlur(frame, (7,7), 0)
#cv2.imshow('frame_blur', blur)
# subtract background
fgmask = backSub.apply(blur) # single channel
#cv2.imshow('fgmask', fgmask)
# concatenate both frames horizontally and write it as output
fgmask_bgr = cv2.cvtColor(fgmask, cv2.COLOR_GRAY2BGR) # convert single channel image to 3-channels
out_frame = cv2.hconcat([blur, fgmask_bgr]) #
#print('output=', out_frame.shape) # shape=(360, 1280, 3)
cv2.imshow('output', out_frame)
video_out.write(out_frame)
# quick pause to display the windows
if (cv2.waitKey(1) == 27):
break
# release resources
cap.release()
video_out.release()
cv2.destroyAllWindows()
You can use SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity https://ieeexplore.ieee.org/document/6975239.
BackgroundSubtractionSuBSENSE bgs(/*...*/);
bgs.initialize(/*...*/);
for(/*all frames in the video*/) {
//...
bgs(input,output);
//...
}
You can find the complete implementation at
https://bitbucket.org/pierre_luc_st_charles/subsense/src/master/
Plus I don't know the scale of your work, and your requirements. But Murari Mandal composed a very informative repository on GitHub comprising list of resources related to background subtraction, which can solve the above mentioned problems.
https://github.com/murari023/awesome-background-subtraction
Related
I use the following function to obtain video frames. I either pass noise_type=None to obtain original frames or pass salt and pepper to and overlay frames with salt and pepper noise (randomly replacing some RGB pixels with (0, 0, 0) or (255, 255, 255) This is passed alongside some probability that a pixel will be replaced with a black or white pixel (e.g. prob=0.1 to replace 10% of pixels with either a black or white pixel).
Please note, I am using Python 3.7.9 and OpenCV 4.4.0. Also, as the videos are to be ultimately written alongside audio data using moviepy, they are in RGB space; so running this code and viewing the video will be in the wrong colourspace, but you should still see that the video hangs during playback.
def get_video_frames(filename, noise_type=None, prob=None):
all_frames = []
video_capture = cv2.VideoCapture()
if not video_capture.open(filename):
print('Error: Cannot open video file {}'.format(filename))
return
fps = video_capture.get(cv2.CAP_PROP_FPS)
print("fps: {}".format(fps))
while True:
has_frames, frame = video_capture.read()
if not has_frames:
video_capture.release()
break
if noise_type is None:
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
frame = cv2.resize(frame, dsize=(224, 224))
elif noise_type == 'salt and pepper':
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
frame = cv2.resize(frame, dsize=(224, 224))
row,col,ch = frame.shape
s_vs_p = 0.5
salty_x_coords = np.random.choice(frame.shape[0], np.int(np.ceil(frame.shape[0]*frame.shape[1]*prob*s_vs_p)))
salty_y_coords = np.random.choice(frame.shape[1], np.int(np.ceil(frame.shape[0]*frame.shape[1]*prob*s_vs_p)))
frame[salty_x_coords, salty_y_coords] = 255, 255, 255
peppery_x_coords = np.random.choice(frame.shape[0], np.int(np.ceil(frame.shape[0]*frame.shape[1]*prob*s_vs_p)))
peppery_y_coords = np.random.choice(frame.shape[1], np.int(np.ceil(frame.shape[0]*frame.shape[1]*prob*s_vs_p)))
frame[peppery_x_coords, peppery_y_coords] = 0, 0, 0
all_frames.append(frame)
return all_frames, fps
The issue comes with playback, it seems. I generate clean frames and display them using opencv:
frames_clean, fps = get_video_frames('C:/some_video_file.mp4')
for f in frames_clean:
cv2.imshow('clean', f)
cv2.waitKey(33)
cv2.destroyAllWindows()
Then I generate noisy frames and display them using opencv:
frames_noisy, fps = get_video_frames('C:/some_video_file.mp4', noise_type='salt and pepper', prob=0.1)
for f in frames_noisy:
cv2.imshow('noisy', f)
cv2.waitKey(33)
cv2.destroyAllWindows()
The noisy video hangs/pauses/stutters on some frames. It's really unusual as both frames_clean and frames_noisy are lists of uint8 frames of the same shape. The only difference is that the noisy frames have some different pixel values. This behaviour is also present if I create a videoclip using moviepy with these frame lists, write them to disk, and play them with VLC/ Windows Media Player. After 2 days of scouring the internet, I can't find any explanation. I would like the noisy videos I generate to play as expected with a seemingly stable display rate as per the clean video without noise. Thanks for any help!
I have conferance call video with different people's tiles arranged on a grid.
Example:
gallery view zoom
Can I crop every video tile to a separate file using python or nodejs?
Yes, you can achieve that using OpenCV library
Read the video in OpenCV using VideoCapture API. Note down framerate while reading.
Parse through each frame and crop the frame:
Write the frame in a video using OpenCV VideoWriter
Here is the example code using (640,480) to be the new dimensions:
cap = cv2.VideoCapture(<video_file_name>)
fps = cap.get(cv2.CAP_PROP_FPS)
out = cv2.VideoWriter('<output video file name>, -1, fps, (640,480))
while(cap.isOpened()):
ret, frame = cap.read()
crop_frame = frame[y:y+h, x:x+w]
# write the crooped frame
out.write(crop_frame)
# Release reader wand writer after parsing all frames
cap.release()
out.release()
Here's the code (tested). It works by initialising a number of video outputs, then for each frame of the input video: cropping the region of interest (roi) and assigning each to the relevent output video. You might need to make tweaks depending on input video dimensions, number of times, offsets etc.
import numpy as np
import cv2
import time
cap = cv2.VideoCapture('in.mp4')
ret, frame = cap.read()
(h, w, d) = np.shape(frame)
horiz_divisions = 5 # Number of tiles stacked horizontally
vert_divisions = 5 # Number of tiles stacked vertically
divisions = horiz_divisions*vert_divisions # Total number of tiles
seg_h = int(h/vert_divisions) # Tile height
seg_w = int(w/horiz_divisions) # Tile width
# Initialise the output videos
outvideos = [0] * divisions
for i in range(divisions):
outvideos[i] = cv2.VideoWriter('out{}.avi'.format(str(i)),cv2.VideoWriter_fourcc('M','J','P','G'), 10, (seg_w,seg_h))
# main code
while(cap.isOpened()):
ret, frame = cap.read()
if ret == True:
vid = 0 # video counter
for i in range(vert_divisions):
for j in range(horiz_divisions):
# Get the coordinates (top left corner) of the current tile
row = i * seg_h
col = j * seg_w
roi = frame[row:row+seg_h,col:col+seg_w,0:3] # Copy the region of interest
outvideos[vid].write(roi)
vid += 1
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
# Release all the objects
cap.release()
for i in range(divisions):
outvideos[i].release()
# Release everything if job is finished
cv2.destroyAllWindows()
Hope this helps!
I captured video using cv2.VideoCapured and display. Captured Video display on same time not saved. How I can insert image on this captured video for display on same time.
Assuming you want to add image directly to video frames at a certain x,y location without doing any color blending or image transparency. you can use the following python code:
#!/usr/bin/python3
import cv2
# load the overlay image. size should be smaller than video frame size
img = cv2.imread('logo.png')
# Get Image dimensions
img_height, img_width, _ = img.shape
# Start Capture
cap = cv2.VideoCapture(0)
# Get frame dimensions
frame_width = cap.get(cv2.CAP_PROP_FRAME_WIDTH )
frame_height = cap.get(cv2.CAP_PROP_FRAME_HEIGHT )
# Print dimensions
print('image dimensions (HxW):',img_height,"x",img_width)
print('frame dimensions (HxW):',int(frame_height),"x",int(frame_width))
# Decide X,Y location of overlay image inside video frame.
# following should be valid:
# * image dimensions must be smaller than frame dimensions
# * x+img_width <= frame_width
# * y+img_height <= frame_height
# otherwise you can resize image as part of your code if required
x = 50
y = 50
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
# add image to frame
frame[ y:y+img_height , x:x+img_width ] = img
# Display the resulting frame
cv2.imshow('frame',frame)
# Exit if ESC key is pressed
if cv2.waitKey(20) & 0xFF == 27:
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
Please give more details if my assumption was wrong.
I'm having an error when I run my code, I don't understand what is happening but I think is when the program finish because I have the result what I want, that is convert an existing video to gray scale and save it.
cv2.error: OpenCV(4.1.0) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'
cap = cv2.VideoCapture('videos/output.avi')
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('results/output.avi', fourcc, 20.0, (640, 480))
while (cap.isOpened()):
_, frame = cap.read()
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
out.write(frame)
# Key events
key = cv2.waitKey(1)
if key == 27: # esc
break
cap.release()
cv2.destroyAllWindows()
Thank you!
There may be a case where at least one of the frames in your video weren't read in properly. That's why the cv2.cvtColor method is throwing an error because the frame data you are providing is empty.
You should consider using the first output of cv2.VideoCapture.read() to make sure the video frame was properly captured, then write it to file. The first output is a flag that determines if the current frame was read in successfully. Also, you'll need to handle the end where we reach the end of the video. In that case, the flag will be False so that we should exit the loop. Finally, if it's your intent to write grayscale frames, there is an optional fifth parameter in cv2.VideoWriter called isColor where we can set this to False to allow us to directly write grayscale frames. This means the call to cv2.cvtColor is no longer required.
One additional thing I'll recommend is to infer the frame width and height from the video file instead of setting it yourself. This way the input and output resolution is the same. Finally, don't forget to release the cv2.VideoWriter object when you're finished and I've added an additional check for the video file to see if it has properly opened:
import numpy as np
import cv2
import sys
cap = cv2.VideoCapture('videos/output.avi')
# Check to see if the video has properly opened
if not cap.isOpened():
print("File could not be opened")
sys.exit(1)
fourcc = cv2.VideoWriter_fourcc(*'XVID')
frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) # Get the frame width and height
frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
# Change
out = cv2.VideoWriter('results/output.avi', fourcc, 20.0, (frame_width, frame_height), isColor=False)
while True:
ret, frame = cap.read() # New
if not ret: # New
break # Get out if we don't read a frame successfully
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
out.write(frame)
# Key events
key = cv2.waitKey(1)
if key == 27: # esc
break
cap.release()
out.release() # New
cv2.destroyAllWindows()
As a minor note, you don't have any windows being displayed, so cv2.destroyAllWindows() is superfluous here. Consider removing it from your code.
So this answer has another approach(Here,you can also extract different color by changing the corresponding weights in front of the B,G, R values)
import cv2
cap = cv2.VideoCapture('videos/output.avi')
frame_width = int(cap.get(3)) # finds the frame width automatically
frame_height = int(cap.get(4)) # finds the frame height automatically
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('results/outpy.avi', cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), 10, (frame_width, frame_height))
while (cap.isOpened()): # value is true if the file is successfully opened.
ret, frame = cap.read()
if ret == True: # checks if the return value is True or False. False means file ended.
# grey = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # the grey matrix has a different shape than the frame matrix
# that's why the output files were blank
# to circumvent this RGB2GRAY, I manually added the "NORMALIZED" R,G,B values.
frame[:,:,0] = 0.114*frame[:,:,0]+0.587*frame[:,:,1]+0.299*frame[:,:,2] #multiplying normalized co-efficients to B,G,R
# for extracting red, make 0.299 as 1.0 and others as 0.0; same goes for other colours.
frame[:, :, 1]=frame[:,:,0] # making the G and R values same as the B.
frame[:, :, 2]=frame[:,:,0]
# now frame is a 3d grayscale matrix. Identical to the cv2.cvtColor method....except it is 3d
# now frame is grey scaled...R,G,B values for each pixel,holds the same number....
out.write(frame)
else:
break
cap.release()
out.release()
cv2.destroyAllWindows()
I am working on a project which requires to do face detection on raspberry pi. I have a USB camera to do this. The frame rate was apparently very slow. So, I scaled down the capture resolution using VideoCapture.set(). This decreased the resolution to 320, 214 as I set it. This increased the capture frame rate considerably but it the feed in displayed the feed on a window on 320 X 214. I want to keep the same capture resolution but I want higher size display window. I am just a beginner to python and open cv. Please help me do it. Below is the code I wrote for just a simple camera feed.
import numpy as np
import cv2
import time
cap = cv2.VideoCapture(-1)
cap.set(3, 320) #width
cap.set(4, 216) #height
cap.set(5, 15) #frame rate
time.sleep(2)
while(cap.isOpened()):
ret, frame = cap.read()
cv2.imshow("captured video", frame)
if cv2.waitKey(33) == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
If I understand you correctly, you want the display image to be a scaled up version of the original. If so, you just need cv2.resize
display_scale = 4
height, width = frame.shape[0:2]
height_display, width_display = display_scale * height, display_scale * width
# you can choose different interpolation methods
frame_display = cv2.resize(frame, (display_width, display_height),
interpolation=cv2.INTER_CUBIC)
cv2.imshow("captured video", frame_display)