We Keep Coding

Extract 1 single image from rtsp-stream using ffmpeg

Im looking for a way to extract one single image from an rtsp stream. My current solution is based on doing that with opencv, but i want to optimize this. So I'm going with a solution I found here Handle large number of rtsp cameras without real-time constraint
The drawback here is that ffmpeg opens a whole videostream, so a consequence is that this is not more efficient than opencv. I want to fetch one single image with ffmpeg, but currently im not able to do that. i get error
"ValueError: cannot reshape array of size 0 into shape (160,240,3)" - So there is no image in the frame. I'm not completely convinced that my ffmpeg_cmd is correct.
import numpy as np
import subprocess as sp
import cv2
# Use public RTSP Streaming for testing:
in_stream = "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4"
# Use OpenCV for getting the video resolution.
cap = cv2.VideoCapture(in_stream)
# Get resolution of input video
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
# Release VideoCapture - it was used just for getting video resolution
cap.release()
# http://zulko.github.io/blog/2013/09/27/read-and-write-video-frames-in-python-using-ffmpeg/
FFMPEG_BIN = "ffmpeg"
ffmpeg_cmd = [FFMPEG_BIN,
'-vframes', '1',
'-rtsp_transport', 'tcp'
'-i', in_stream,'-c:v','mjpeg','-f','image2pipe', 'pipe:']
process = sp.Popen(ffmpeg_cmd, stdout=sp.PIPE)
raw_frame = process.stdout.read(width * height * 3)
frame = np.frombuffer(raw_frame, np.uint8).reshape((height, width, 3))
process.stdout.close()
process.wait()
cv2.destroyAllWindows()
If someone have a completely different solution, thats ok. I need something that is more efficient than doing it with opencv.

Python: What's the fastest way to load a video file into memory?

First some background
I am trying to write my own set of tools for video analysis, mainly for detecting render errors like flashing frames and possibly some other stuff in the future.
The (obvious) goal is to write a script, that is faster and more accurate than me watching the file in real time.
Using OpenCV, I have something that looks like this:
import cv2
vid = cv2.VideoCapture("Video/OpenCV_Testfile.mov", cv2.CAP_FFMPEG)
width = 1024
height = 576
length = vid.get(cv2.CAP_PROP_FRAME_COUNT)
for f in range(length):
blue_values = []
vid.set(cv2.CAP_PROP_POS_FRAMES, f)
is_read, frame = vid.read()
if is_read:
for row in range(height):
for col in range(width):
blue_values.append(frame[row][col][0])
print(blue_values)
vid.release()
This just prints out a list of all blue values of every frame.
- Just for simplicity (My actual script compares a few values across each frame and only saves the frame number when all are equal)
Although this works, it is not a very fast operation. (Nested loops, but most important, the read() method has to be called for every frame, which is rather slow.
I tried to use multiprocessing but basically ended up having the same crashes as described here:
how to get frames from video in parallel using cv2 & multiprocessing in python
I have a 20s long 1024x576#25fps Testfile which performs as follows:
mov, ProRes: 15s
mp4, h.264: 30s (too slow)
My machine is capable of playing back h.264 in 1920x1080#50fps with mplayer (which uses ffmpeg to decode). So, I should be able to get more out of this. Which leads me to
my Question
How can I decode a video and simply dump all pixel values into a list for further (possibly multithreaded) operations? Speed is really all that matters. Note: I'm not fixated on OpenCV. Whatever works best.
Thanks!

Generating animation from image sequence in PsychoPy?

I am new to PsychoPy, having previously worked with Pygame for several months (I switched to enable stimuli to be presented on multiple screens).
I am trying to figure out how to use PsychoPy to display an animation created using a sequence of images. I previously achieved this in Pygame by saving the entire sequence of images in a single large png file (a spritesheet) and then flipping only a fraction of that image (eg. 480 x 480 pixels) per frame, while moving onto the next equally sized section of the image in the next frame. This is roughly what my code looked like in Pygame. I would be really keen to hear if there is an equivalent way of generating animations in PsychoPy by selecting only parts of an image to be displayed with each frame. So far, googling this has not provided any answers!
gameDisplay=pygame.display.set_mode((800, 480))
sequence=pygame.image.load('C:\Users\...\image_sequence.png')
#This image contains 10 images in a row which I cycle through to get an animation
image_width=480
image_height=480
start=time.time()
frame_count=0
refresh=0
while time.time()<=start+15:
gameDisplay.blit(sequence,(160,0),(frame_count*image_width,0,image_width,image_height))
if time.time()>= start+(refresh*0.25): #Flip a new image say every 250 msec
pygame.display.update()
frame_count+=1
refresh+=1
if frame_count ==10:
frame_count=0

You could use a square aperture to restrict what's visible and then move the image. So something like this (untested, but could give you some ideas):
from psychopy import visual
win = visual.Window(units='pix') # easiest to use pixels as unit
aperture = visual.Aperture(win, shape='rect', size=(480, 480))
image = visual.ImageStim('C:\Users\...\image_sequence.png')
# Move through x positions
for x in range(10):
image.pos = [(-10.0/2*+0.5+x)*480, 0] # not sure this is right, but it should move through the x-positions
image.draw()
win.flip()
If you have the original images, I think that it would be simpler to just display the original images in sequence.
import glob
from psychopy import visual
image_names = glob.glob('C:\Users\...\*.png')
# Create psychopy objects
win = visual.Window()
image_stims = [visual.ImageStim(win, image) for image in image_names]
# Display images one by one
for image in image_stims:
image.draw()
win.flip()
# add more flips here if you want a lower frame rate
Perhaps it is even fast enough to load them during runtime without dropping frames, which would simplify the code and load memory less:
# Imports, glob, and win here
# Create an ImageStim and update the image each frame
stim = visual.ImageStim(win)
for name in image_names:
stim.image = name
stim.draw()
win.flip()

Actually, given a spritesheet you might be able to do something funky and more efficient using the GratingStim. This loads an image as a texture and then allows you to set the spatial frequncy (sf) and phase of that texture. If 1.0/sf (in both dimensions) is less than the width of the stimulus (in both dimensions) only a fraction of the texture will be shown and the phase determines which fraction that will be. It isn't designed for this purpose - it's usually used to create more than one cycle of texture not less than one - but I think it will work.

How to know total number of Frame in a file with cv2 in python

How to know total number of Frame in a file ( .avi) through Python using open cv module.
If possible what all the information (resolution, fps,duration,etc) we can get of a video file through this.

With a newer OpenCV version (I use 3.1.0) it works like this:
import cv2
cap = cv2.VideoCapture("video.mp4")
length = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
print( length )
And similar for other video properties cv2.CAP_PROP_*

import cv2
cap = cv2.VideoCapture(fn)
if not cap.isOpened():
print("could not open :",fn)
return
length = int(cap.get(cv2.cv.CV_CAP_PROP_FRAME_COUNT))
width = int(cap.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT))
fps = cap.get(cv2.cv.CV_CAP_PROP_FPS)
see here for more info.
also, all of it with a grain of salt, not all those props are mandatory, some might not be available with your capture / video codec

There are two methods to determine the number of frames in a video file
Method #1: Utilize built-in OpenCV properties to access video file meta information
which is fast and efficient but inaccurate
Method #2: Manually loop over each frame in the video file with a counter which is slow and inefficient but accurate
Method #1 is fast and relys on OpenCV's video property functionality which almost instantaneously determines the number of frames in a video file. However, there is an accuracy trade-off since it is dependent on your OpenCV and video codec versions. On the otherhand, manually counting each frame will be 100% accurate although it will be significantly slower. Here's a function that attempts to perform Method #1 by default, if it fails, it will automatically utilize method #2
def frame_count(video_path, manual=False):
def manual_count(handler):
frames = 0
while True:
status, frame = handler.read()
if not status:
break
frames += 1
return frames
cap = cv2.VideoCapture(video_path)
# Slow, inefficient but 100% accurate method
if manual:
frames = manual_count(cap)
# Fast, efficient but inaccurate method
else:
try:
frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
except:
frames = manual_count(cap)
cap.release()
return frames
Benchmarks
if __name__ == '__main__':
import timeit
import cv2
start = timeit.default_timer()
print('frames:', frame_count('fedex.mp4', manual=False))
print(timeit.default_timer() - start, '(s)')
start = timeit.default_timer()
print('frames:', frame_count('fedex.mp4', manual=True))
print(timeit.default_timer() - start, '(s)')
Method #1 results
frames: 3671
0.018054921 (s)
Method #2 results
frames: 3521
9.447095287 (s)
Note the two methods differ by 150 frames and Method #2 is significantly slower than Method #1. Therefore if you need speed but willing to sacrifice accuracy, use Method #1. In situations where you're fine with a delay but need the exact number of frames, use Method #2

Here is how it works with Python 3.6.5 (on Anaconda) and OpenCV 3.4.2.
[Note]: You need to drop the "CV_" from the "CV_CAP_PROP_xx" for any property as given on the official OpenCV website.
import cv2
cap = cv2.VideoCapture("video.mp4")
property_id = int(cv2.CAP_PROP_FRAME_COUNT)
length = int(cv2.VideoCapture.get(cap, property_id))
print( length )

Another solution that doesn't depend on the sometimes buggy CV_CAP_PROP getters is to traverse your whole video file in a loop
Increase a frame counter variable every time a valid frame is encountered and stop when an invalid one comes (end of the video file).
Gathering information about the resolution is trickier because some codecs support variable resolution (similar to VBR in audio files where the bitrate is not a constant but instead covers some predefined range).
constant resolution - you need only the first frame to determine the resolution of the whole video file in this case so traversing the full video is not required
variable resolution - you need to get the resolution of every single frame (width and height) and calculate an average to get the average resolution of the video
FPS can be calculated however here you have the same problem as with the resolution - constant (CFR) vs variable (VFR). This is more of a mutli-threading problem omho. Personally I would use a frame counter, which increased after each valid frame while at an interval of 1 second a timer (running in a background thread) would trigger saving the current counter's value and then resetting it. You can store the values in a list in order to calculate the average/constant frame rate at the end when you will also know the total number of frames the video has.
The disadvantage of this rather simplistic way of doing things is that you have to traverse the whole file, which - in case it's several hours long - will definitely be noticeable by the user. In this case you can be smart about it and do that in a background process while letting the user do something else while your application is gathering this information about the loaded video file.
The advantage is that no matter what video file you have as long as OpenCV can read from it you will get quite accurate results unlike the CV_CAP_PROP which may or may not work as you expect it to.

raspberry pi image cropping very slow

I am currently working on a project to capture and process photos on a raspberry Pi.
The photos are 6000X4000 about 2 mb, from a nikon D5200 camera.
Everything is working fine, i have made a proof of concept in Java and want to transform this to python or C depending on which language is faster on the raspberry.
No the problem is that the images need to be cropped and re-sized, this takes a very long time in the raspberry. In java the whole process of reading the image, cropping and writing the new image takes about 2 minutes.
I have also tried ImageMagick but in command-line this even takes up to 3 minutes.
With a small python script i made this is reduces to 20 seconds, but this is still a bit to long for my project.
Currently i am installing OpenCV to check if this is faster, this process takes around 4 hours so i thought in the meantime i can ask a question here.
Does anybody have any good idea's or libraries to speed up the process of cropping and re-sizing the images.
Following is the python code i used
import Image
def crop_image(input_image, output_image, start_x, start_y, width, height):
"""Pass input name image, output name image, x coordinate to start croping, y coordinate to start croping, width to crop, height to crop """
input_img = Image.open(input_image)
box = (start_x, start_y, start_x + width, start_y + height)
output_img = input_img.crop(box)
output_img.save(output_image +".jpg")
def main():
crop_image("test.jpg","output", 1000, 0, 4000, 4000)
if __name__ == '__main__': main()

First approach (without sprites)
import pyglet
#from pyglet.gl import *
image = pyglet.resource.image('test.jpg')
texture = image.get_texture()
## -- In case you plan on rendering the image, use the following gl set:
#gl.glTexParameteri(gl.GL_TEXTURE_2D, gl.GL_TEXTURE_MAG_FILTER, gl.GL_NEAREST)
texture.width = 1024
texture.height = 768
texture.get_region(256, 192,771, 576)
texture.save('wham.png') # <- To save as JPG again, install PIL
Second attempt (with sprites, unfinished)
import pyglet, time
start = time.time() #DEBUG
texture = pyglet.image.load('test.jpg')
print('Loaded image in',time.time()-start,'sec') #DEBUG
sprite = pyglet.sprite.Sprite(texture)
print('Converted to sprite in',time.time()-start,'sec') #DEBUG
print(sprite.width) #DEBUG
# Gives: 6000
sprite.scale = 0.5
print('Rescaled image in',time.time()-start,'sec') #DEBUG
print(sprite.width) #DEBUG
# Gives: 3000
Both solutions end up around 3-5 seconds on an extremely slow PC with a shitty mechanical disk running under Windows XP with.. i can't even count the number of applications running including active virus scans etc.. But note that I can't remember how to save a sprite to disk, you need to access to AbstractImage data container within the sprite to get it out.
You will be heavily limited to your disk/memory-card I/O.
My image was 16MB 6000x4000 pixels.. Which i was suprised it whent as fast as 3 seconds to load.

Have you tried jpegtran. It provides for lossless cropping of jpeg. It should be in the libjpeg-progs package. I suspect that decoding the image to crop it, then re-encoding it is too much for the SD card to take.