Decode/encode HEVC with OpenCV Python with Intel Media SDK backend - python

I am trying to process some FHD video on Win10 in Python. I enabled the integrated GPU in BIOS, installed latest driver through Intel driver support, installed Intel Media SDK successfully and rebooted. Then I downloaded OpenCV 4.5 with all hardware codecs from here and added to a dummy project (made sure no other OpenCV exists). When I tried to use it for decoding/encoding, I had the following errors.
In decoding, I got error like
decoder = cv.VideoCapture('myfile.hevc', cv.CAP_INTEL_MFX)
# MFX: LoadPlugin failed for codec: 1129727304
but using cv.CAP_FFMPEG worked ok.
In encoding, I got error like
writer = cv.VideoWriter('output.hevc', apiPreference = cv.CAP_INTEL_MFX, fourcc = cv.VideoWriter_fourcc('h','v','c','1'), fps = 20.0, frameSize = (640,480))
# MFX: Unsupported FourCC: hvc1 (0x31637668)
# 'hevc' or 'h265' give same error code
# write to 'output.mp4' with 'mp4v' yield a different error code 0x7634706d
While writing with ffmpeg works
writer = cv.VideoWriter('C:/testData/output.mp4', apiPreference = cv.CAP_FFMPEG , fourcc = cv.VideoWriter_fourcc('m','p','4','v'), fps = 20.0, frameSize = (640,480))
# success
Much appreciated.
[Edit 01]
is the screenshot of running mediasdk_system_analayzer_64.exe.
[Edit 02]
Even though HW decoder does not show up in the analyzer, it works after I disable all monitors but 1 as suggested here.
.\sample_multi_transcode.exe -i::h265 myfile.hevc -o::h265 output.hevc -hw
# session 0 [0000023D33FCA130] PASSED (MFX_ERR_NONE) 8.34417 sec, 226 frames, 27.085 fps
VideoCapture with cv.CAP_INTEL_MFX works now. Yet VideoWriter gives the same error.

Related

why my server terminal doesnt show error?

I am connected to my VPS (Ubuntu) via SSH.
I use Django, where I process 1000's of images via Django-imagekit ( PIL ). but for some reason, it is not generating an error it just stops/terminates there. it shows all other print commands and everything.
yes, I tried 'try and except' but it doesn't work, it just terminates at 'try' only.
CODE:
from imagekit import ImageSpec
from imagekit.processors import ResizeToFit
class Thumbnail(ImageSpec):
processors = [ResizeToFit(1200)]
format = 'WEBP'
options = {'quality': 90}
Main_image = open('path/to/image.jpg','rb')
Preview_gen = Thumbnail(source=Main_image)
Preview = Preview_gen.generate()
Error while executing the last line
( but works fine on my local machine )
Specs:
Ubuntu Linux 18.04.2
Python==3.6.9
Django==3.0.7
Pillow==8.2.0
psycopg2==2.8.6
psycopg2-binary==2.8.6
django-imagekit==4.0.2
anyone any idea?
error causing image Wallpaper, .jpg, 9600 x 5598
this is the problem for very few images

setUpNet DNN module was not built with CUDA backend; switching to CPU

I want to run my script python with GPU as u see in this photo
I used the command line: watch nvidia-smi,to show Processes of GPU, unfortunately the script python use just 41Mib of GPU capacity:
this is a part of my code :
import time
import math
import cv2
import numpy as np
labelsPath = "./coco.names"
LABELS = open(labelsPath).read().strip().split("\n")
np.random.seed(42)
weightsPath = "./yolov3.weights"
configPath = "./yolov3.cfg"
net = cv2.dnn.readNetFromDarknet(configPath, weightsPath)
ln = net.getLayerNames()
ln = [ln[i[0] - 1] for i in net.getUnconnectedOutLayers()]
FR=0
vs = cv2.VideoCapture(vid_path)
# vs = cv2.VideoCapture(0) ## USe this if you want to use webcam feed
writer = None
(W, H) = (None, None)
fl = 0
q = 0
while True:
(grabbed, frame) = vs.read()
if not grabbed:
break
if W is None or H is None:
(H, W) = frame.shape[:2]
FW=W
if(W<1075):
FW = 1075
FR = np.zeros((H+210,FW,3), np.uint8)
col = (255,255,255)
FH = H + 210
FR[:] = col
blob = cv2.dnn.blobFromImage(frame, 1 / 255.0, (416, 416),
swapRB=True, crop=False)
net.setInput(blob)
start = time.time()
layerOutputs = net.forward(ln)
end = time.time()
I tried to add this command line to force run with GPU ,
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)
then after running the script again it gives me this message and continue running the script with CPU :
[ WARN:0] global /io/opencv/modules/dnn/src/dnn.cpp (1363) setUpNet DNN module was not built with CUDA backend; switching to CPU
You'll need to manually build OpenCV to work with your GPU.
Here is a great tutorial on how to do so.
You might have to uninstall your opencv-python package using pip in case you are already having one, only then will the custom built opencv be accessible to the program.
pip3 uninstall opencv-python
Compatibility chart of cuda and cudnn:
https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html#cudnn-cuda-hardware-versions
Checking the computation capability version from:
https://en.wikipedia.org/wiki/CUDA
Which is 7.5
In GPU supported, for 7.5 computation capability, CUDA SDK 11.0 – 11.2 support for compute capability 3.5 – 8.6 (Kepler (in part), Maxwell, Pascal, Volta, Turing, Ampere):
check for your Supported NVIDIA Hardware.
In my case, I was using Tesla T4 having Turing, which is compatible with cuDNN.
so in compilation report, you can see that Cmake returns cuDNN availability as "NO":
Got the docker Image Using:
sudo docker nvidia/cuda:11.1-cudnn8-runtime-ubuntu18.04
Compiled Opencv Cuda from:
https://www.pyimagesearch.com/2020/02/03/how-to-use-opencvs-dnn-module-with-nvidia-gpus-cuda-and-cudnn/

How to correctly check if a camera is available?

I am using OpenCV to open and read from several webcams. It all works fine, but I cannot seem to find a way to know if a camera is available.
I tried this code (cam 2 does not exist):
import cv2
try:
c = cv2.VideoCapture(2)
except:
print "Cam 2 is invalid."
But this just prints a lot of errors:
VIDEOIO ERROR: V4L: index 2 is not correct!
failed to open /usr/lib64/dri/hybrid_drv_video.so
Failed to wrapper hybrid_drv_video.so
failed to open /usr/lib64/dri/hybrid_drv_video.so
Failed to wrapper hybrid_drv_video.so
GStreamer Plugin: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /builddir/build/BUILD/opencv-3.2.0/modules/videoio/src/cap_gstreamer.cpp, line 832
VIDEOIO(cvCreateCapture_GStreamer(CV_CAP_GSTREAMER_V4L2, reinterpret_cast<char *>(index))): raised OpenCV exception:
/builddir/build/BUILD/opencv-3.2.0/modules/videoio/src/cap_gstreamer.cpp:832: error: (-2) GStreamer: unable to start pipeline
in function cvCaptureFromCAM_GStreamer
OpenCV Error: Unspecified error (unicap: failed to get info for device
) in CvCapture_Unicap::initDevice, file /builddir/build/BUILD/opencv-3.2.0/modules/videoio/src/cap_unicap.cpp, line 139
VIDEOIO(cvCreateCameraCapture_Unicap(index)): raised OpenCV exception:
/builddir/build/BUILD/opencv-3.2.0/modules/videoio/src/cap_unicap.cpp:139: error: (-2) unicap: failed to get info for device
in function CvCapture_Unicap::initDevice
CvCapture_OpenNI::CvCapture_OpenNI : Failed to enumerate production trees: Can't create any node of the requested type!
<VideoCapture 0x7fa5b5de0450>
No exception is thrown. When using c.read() later, I do get False, but I would like to do this in the initialisation phase of my program.
So, how do I find out how many valid cameras I have or check if a certain number 'maps' to a valid one?
Using cv2.VideoCapture( invalid device number ) does not throw exceptions. It constructs a <VideoCapture object> containing an invalid device - if you use it you get exceptions.
Test the constructed object for None and not isOpened() to weed out invalid ones.
For me this works (1 laptop camera device):
import cv2 as cv
def testDevice(source):
cap = cv.VideoCapture(source)
if cap is None or not cap.isOpened():
print('Warning: unable to open video source: ', source)
testDevice(0) # no printout
testDevice(1) # prints message
Output with 1:
Warning: unable to open video source: 1
Example from: https://github.com/opencv/opencv_contrib/blob/master/samples/python2/video.py
lines 159ff
cap = cv.VideoCapture(source)
if 'size' in params:
w, h = map(int, params['size'].split('x'))
cap.set(cv.CAP_PROP_FRAME_WIDTH, w)
cap.set(cv.CAP_PROP_FRAME_HEIGHT, h)
if cap is None or not cap.isOpened():
print 'Warning: unable to open video source: ', source
Another solution, which is available in Linux is to use the /dev/videoX device in the VideoCapture() call. The devices are there when the cam is plugged in. Together with glob(), it is trivial to get all the cameras:
import cv2, glob
for camera in glob.glob("/dev/video?"):
c = cv2.VideoCapture(camera)
Of course a check is needed on c using isOpened(), but you are sure you only scan the available cameras.
Here is a "NOT working" solution to help you prevent tumbling over in this pitfall:
import cv2 as cv
import PySpin
print (cv.__version__)
# provided by Patrick Artner as solution to be working for other cameras than
# those of Point Grey (FLIR).
def testDevice(source):
cap = cv.VideoCapture(source)
if cap is None or not cap.isOpened():
print('Warning: unable to open video source: ', source)
# ... PySpin / Spinnaker (wrapper/SDK libary) ...
system = PySpin.System.GetInstance()
cam_list = system.GetCameras()
cam = ''
cam_num = 0
for ID, cam in enumerate(cam_list):
# Retrieve TL device nodemap
if ID == cam_num:
print ('Got cam')
cam = cam
cam.Init()
# ... CV2 again ...
for i in range(10):
testDevice(i) # no printout
You can try this code:
from __future__ import print_function
import numpy as np
import cv2
# detect all connected webcams
valid_cams = []
for i in range(8):
cap = cv2.VideoCapture(i)
if cap is None or not cap.isOpened():
print('Warning: unable to open video source: ', i)
else:
valid_cams.append(i)
caps = []
for webcam in valid_cams:
caps.append(cv2.VideoCapture(webcam))
while True:
# Capture frame-by-frame
for webcam in valid_cams:
ret, frame = caps[webcam].read()
# Display the resulting frame
cv2.imshow('webcam'+str(webcam), frame)
k = cv2.waitKey(1)
if k == ord('q') or k == 27:
break
# When everything done, release the capture
for cap in caps:
cap.release()
cv2.destroyAllWindows()

Error occuring while connecting USB camera and two arduino with raspberry pi simultaneously

I have the following code for connecting one usb camera and two arduino board with raspberry pi.
import cv2.cv as cv, time
import numpy as np
import serial,time
ser1 = serial.Serial('/dev/ttyUSB0')
ser2 = serial.Serial('/dev/ttyUSB1')
capture = cv.CaptureFromCAM(0)
while True:
try:
for i in range(10):
x = ser1.readline()
with open("test1.txt", "a") as myfile:
myfile.write(x)
img = cv.QueryFrame(capture)
print("Taking image...")
cv.SaveImage('pic{:>05}.jpg'.format(i), img)
time.sleep(2)
y = ser2.readline()
with open("test2.txt", "a") as myfile:
myfile.write(y)
cv.WaitKey(1)
except:
continue
The camera is for taking continuous image frame and storing the latest 10 images in the current folder in each 2sec(Overwriting previous data). The datas from the arduino also logging into two separate text file. While running the code i am getting error messages as shown below.
pi#raspberrypi ~ $ sudo python accident.py
libv4l2: error setting pixformat: Device or resource busy
HIGHGUI ERROR: libv4l unable to ioctl S_FMT
libv4l2: error setting pixformat: Device or resource busy
libv4l1: error setting pixformat: Device or resource busy
HIGHGUI ERROR: libv4l unable to ioctl VIDIOCSPICT
Taking image...
Taking image...
Taking image...
^Z
[4]+ Stopped
While I searched for this problem , I found that, all the USB port of rasp-pi having the same serial port.
Can anybody help me to solve this issue...

Capturing network video using opencv

I am using ffserver and ffmpeg combination to capture web camera video and transmit it through my network.
I want to capture this video using opencv and python from another computer.
I can see the video (cam1.asf) in the browser of another computer. But my opencv + python code could not capture any frame.
Code for ffserver
HTTPPort 8090
HTTPBindAddress 0.0.0.0
MaxHTTPConnections 2000
MaxClients 1000
MaxBandWidth 2000
<Feed feed1.ffm>
File ./tmp/feed1.ffm
FileMaxSize 1G
ACL allow 127.0.0.1
</Feed>
<Stream cam1.asf>
Feed feed1.ffm
Format asf
VideoCodec msmpeg4v2
VideoFrameRate 30
VideoSize vga
</Stream>
FFmpeg
$ffmpeg -f video4linux2 -i /dev/video0 192.168.1.3 /cam1.ffm
This stream can be seen in the browser
But with opencv code
import sys
import cv2.cv as cv
import numpy
video="http://http://192.168.1.3:8090/cam1.asf"
capture =cv.CaptureFromFile(video)
cv.NamedWindow('Video Stream', 1 )
while True:
# capture the current frame
frame = cv.QueryFrame(capture)
#if frame is None:
# break
#else:
#detect(frame)
cv.ShowImage('Video Stream', frame)
if cv.WaitKey(10) == 27:
print 'ESC pressed. Exiting ...'
break
I donot get any output in the stream
My aim is to work with the web camera video both at the base station (ie where the web camera is connected) and also at the network location

Categories