This question already has answers here:
Are global variables thread-safe in Flask? How do I share data between requests?
(4 answers)
Closed 4 years ago.
Trying to execute a conditional statement on the basis of value provided by html form.
The flow of the code is as follows.
Index.html gets rendered with a videostream and has a checkbox to let the user decide wether they want to run face detection or not.
On pressing the check button flask saves the value to a variable called status, and tries to pass it during the opencv_camera object. All my attempts yield self/status not defined.
Really lost please help.
camera_opencv.py
from base_camera import BaseCamera
**# THE CLASS IN QUESTION**
class Camera(BaseCamera):
status = 0
def __init__(self, state=None):
if state:
status = state
else:
status = 0
super().__init__()
video_source = 0
#staticmethod
def set_video_source(source):
Camera.video_source = source
#staticmethod
def frames():
camera = cv2.VideoCapture(Camera.video_source)
if not camera.isOpened():
raise RuntimeError('Could not start camera.')
while True:
# read current frame
_, frame = camera.read()
# frame = cv2.resize(frame, (704, 396))
if status: # CONDITIONAL STATEMENT
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.3, 5)
if len(faces) > 0:
print("SUPS")
for (x,y,w,h) in faces:
cv2.rectangle(frame,(x,y),(x+w,y+h),(255,0,0),2)
# encode as a jpeg image and return it
yield cv2.imencode('.jpg', frame)[1].tobytes()
app.py
from camera_opencv import Camera
app = Flask(__name__)
state = 0 # STATE VARIABLE
#app.route('/', methods = ['GET','POST'])
def index():
state = request.form.get('check')
# print(state)
# import pdb; pdb.set_trace()
return render_template("index.html")
def gen(camera):
"""Video streaming generator function."""
while True:
frame = camera.get_frame()
yield (b'--frame\r\n'
b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')
#app.route('/video_feed')
def video_feed():
"""Video streaming route. Put this in the src attribute of an img tag."""
return Response(gen(Camera(state=state)),
mimetype='multipart/x-mixed-replace; boundary=frame')#STATE VARIABLE BEING PASSED
By their nature, web servers are stateless: they don't know if two requests come from the same user or not so each response gets executed in its own context. That means that if you set the state from 0 to 1 in your index view and then go to your /video_feed view, the state will again be 0.
To have the proper value of state available when starting the camera, add the state as a parameter to the /video_feed view. So if you want to use face detection, navigate to the url /video_feed?state=1 and grab the state parameter from the url. If you don't want to activate face detection, navigate to /video_feed?state=0 or just /video_feed and use the default state.
Related
I'm writing a basic function that uses OpenCV to take some N number of photos with a given interval of time between them, to be run on a Raspberry Pi. Both the test and the actual function work perfectly fine when run on my mac, and the function works as intended on the Raspberry Pi. But the test does not pass. when run on the Pi. I usually receive an error stating "(assertion failed)!_img.empty() in function 'imwrite'"
How can I make this test run correctly on all devices?
Camera Control class:
import cv2
import threading
import os
import time
class CameraController():
def __init__(self, id=0):
self.id = id
self.camera = cv2.VideoCapture(id)
def turn_on(self):
if self.camera.isOpened() == False:
self.camera.open()
def capture_frame(self):
retval, image = self.camera.read()
return image
class ImageCamera(CameraController):
def image_collection(self, increment, n_photos, dir):
dir_path = os.path.join(os.path.dirname(os.path.realpath(__file__)),dir)
os.makedirs(dir_path, exist_ok=True)
for i in range(n_photos):
file_name = os.path.join(dir_path,'minimum_payload_image_{}.png'.format(i))
image = self.capture_frame()
t = threading.Thread(target = cv2.imwrite, args=(file_name,image))
t.start()
time.sleep(increment)
t.join()
test classes:
from CameraController import CameraController, ImageCamera
import numpy as np
import os
class TestCameraController:
def test_is_working(self):
cam = CameraController(0)
success = cam.camera.grab()
assert success == True
def test_capture_frame(self):
cam = CameraController(0)
image = cam.capture_frame()
assert isinstance(image, np.ndarray)
class TestImageCamera:
def test_take_photos(self,tmp_path):
cam = ImageCamera(0)
n_photos = 5
increment = 1
d = tmp_path / "sub"
cam.image_collection(increment,n_photos,d)
assert len(os.listdir(d)) == n_photos
I've tried changing the OpenCV API in the VideoCapture constructor. The only time I've had a successful test with this was when I ran the test by itself but I haven't been able to recreate it.
videoPlayerThreading is my own made library to basically making 2 class with each using threading to get and show frames. objDect is also my own library to basically return frame after object detection. I got EOFError : Ran out of Input error and from the traceback I think it is caused by the multiprocessing itself hence I don't post my library because it is so long. Can anyone help me with what is wrong with it? Thank you
from multiprocessing import Process
import sys
import videoPlayerThreading as vpt
from objDect import objDect as od
def main(videoSource):
obd = od( videoSources = videoSource )
getFrame = vpt.getFrames(videoSource).start()
showFrame = vpt.showFrames(videoSource).start()
while True:
frame = getFrame.frame
frame=Process(target=obd.predictYolo, args=(frame,)).start()
showFrame.frame = frame
if getFrame.doVideo == False or showFrame.doVideo == False:
getFrame.stop()
showFrame.stop()
sys.exit()
if __name__=="__main__":
main(0)
Edit :
Here is the show frames and get frames class it basically only get and show frame using threading.
class getFrames():
def __init__(self,
videoSource:Union[int,str]=0):
self.stream = self.videoInit(videoSource)
self.hasFrame, self.frame = self.stream.read()
self.doVideo = True
def videoInit(self,
videoSource:Union[int,str]):
try:
cap = cv2.VideoCapture(videoSource)
except Exception as e:
raise Exception(f"Video source error: {e}")
return cap
def start(self):
Thread(target=self.getFrames, args=()).start()
return self
def getFrames(self):
while self.doVideo:
if not self.hasFrame:
self.stop()
else:
(self.hasFrame, self.frame) = self.stream.read()
def stop(self):
self.doVideo = False
self.stream.release()
class showFrames():
def __init__(self,
frame:cv2=None):
self.frame = frame
self.doVideo = True
def start(self):
Thread(target=self.showFrame, args=()).start()
return self
def showFrame(self):
while self.doVideo:
cv2.imshow("Video", self.frame)
if cv2.waitKey(1) == ord("q"):
self.doVideo = False
def stop(self):
self.doVideo = False
The best I can understand your program logic you need something like the following. Generator function read_frames (which may or may not need correction), reads the frames one by one yielding each frame. The main process creates a multiprocessing pool and passes each input frame to the multiprocessing pool to be processed by obd.predictYolo and sets vpt.frame with the returned frame. This continues until either there are no more frames to process or showFrame.doVideo is False. In short, I have done away with your getFrames class, which is useless here.
I do not have OpenCV installed and do not really know the package nor do I have your video file, so consider this a starting point for your further investigation.
from multiprocessing.pool import Pool
import sys
import videoPlayerThreading as vpt
from objDect import objDect as od
def read_frames(videoSource:Union[int,str]=0):
try:
stream = cv2.VideoCapture(videoSource)
except Exception as e:
raise Exception(f"Video source error: {e}")
while True:
hasFrame, frame = stream.read()
if not hasFrame:
break
yield frame
def main(videoSource):
obd = od( videoSources = videoSource )
showFrame = vpt.showFrames(videoSource).start()
with Pool() as pool:
for frame in pool.imap(obd.predictYolo, read_frames(videoSource)):
showFrame.frame = frame
if showFrame.doVideo is False:
showFrame.stop()
break
if __name__=="__main__":
main(0)
I'm struggling to understand why I cant get a "LIVE" feed from my IP camera.
It appears that there is a buffer and it causes the frames to build up if not being read - and as each iteration of my code takes some time there is a backlog and it ends up being almost slow mo to whats actually happening.
I found the below code which triggers a thread to do the reading of the camera on a loop to try and avoid this. But now i'm getting a "LIVE" feed for around 5 frames and then it stalls and shows the same image for another few.
##camera class - this stops the RTSP feed getting caught in the buffer
class Camera:
def __init__(self, rtsp_link):
#init last ready and last frame
self.last_frame = None
self.last_ready = None
self.lock = Lock()
#set capture decive
capture = cv2.VideoCapture(rtsp_link,apiPreference=cv2.CAP_FFMPEG)
#set thread to clear buffer
thread = threading.Thread(target=self.rtsp_cam_buffer, args=(capture,), name="rtsp_read_thread")
thread.daemon = True
thread.start()
#delay start of next step to avoid errors
time.sleep(2)
def rtsp_cam_buffer(self, capture):
#loop forever
while True:
with self.lock:
capture.grab()
self.last_ready, self.last_frame = capture.retrieve()
def getFrame(self):
#get last frame
if (self.last_ready is not None) and (self.last_frame is not None):
return self.last_frame.copy())
else:
return None
Whats the correct thing to do in this situation? Is there a way round this?
OR
Should I use something like gstreamer or ffmpeg to get the camera feed? If so which is better and why? Any advice or pages to give me some python examples of getting it working? I couldn't find loads about that made sense to me.
thanks
After searching online through multiple resources the suggestion for using threads to remove frames from the buffer came up ALOT. And although it seemed to work for a while it caused me issues with duplicate frames being displayed for some reason that I could not work out.
I then tried to build opencv from source with gstreamer support but even once it was compiled correctly it still didn't seem to like interfacing with gstreamer correctly.
Eventually I thought the best bet was to go back down the threading approach but again couldnt get it working. So I gave multiprocessing a shot.
I wrote the below class to handle the camera connection:
import cv2
import time
import multiprocessing as mp
class Camera():
def __init__(self,rtsp_url):
#load pipe for data transmittion to the process
self.parent_conn, child_conn = mp.Pipe()
#load process
self.p = mp.Process(target=self.update, args=(child_conn,rtsp_url))
#start process
self.p.daemon = True
self.p.start()
def end(self):
#send closure request to process
self.parent_conn.send(2)
def update(self,conn,rtsp_url):
#load cam into seperate process
print("Cam Loading...")
cap = cv2.VideoCapture(rtsp_url,cv2.CAP_FFMPEG)
print("Cam Loaded...")
run = True
while run:
#grab frames from the buffer
cap.grab()
#recieve input data
rec_dat = conn.recv()
if rec_dat == 1:
#if frame requested
ret,frame = cap.read()
conn.send(frame)
elif rec_dat ==2:
#if close requested
cap.release()
run = False
print("Camera Connection Closed")
conn.close()
def get_frame(self,resize=None):
###used to grab frames from the cam connection process
##[resize] param : % of size reduction or increase i.e 0.65 for 35% reduction or 1.5 for a 50% increase
#send request
self.parent_conn.send(1)
frame = self.parent_conn.recv()
#reset request
self.parent_conn.send(0)
#resize if needed
if resize == None:
return frame
else:
return self.rescale_frame(frame,resize)
def rescale_frame(self,frame, percent=65):
return cv2.resize(frame,None,fx=percent,fy=percent)
Displaying the frames can be done as below
cam = Camera("rtsp://admin:[somepassword]#192.168.0.40/h264Preview_01_main")
print(f"Camera is alive?: {cam.p.is_alive()}")
while(1):
frame = cam.get_frame(0.65)
cv2.imshow("Feed",frame)
key = cv2.waitKey(1)
if key == 13: #13 is the Enter Key
break
cv2.destroyAllWindows()
cam.end()
This solution has resolved all my issues of buffer lag and also repeated frames. #
Hopefully it will help anyone else in the same situation.
Lewis's solution was helpful to reduce the lag so far but there was still some lag in my case, and I have found this gist, which is a bit faster:
import os
import sys
import time
import threading
import numpy as np
import cv2 as cv
# also acts (partly) like a cv.VideoCapture
class FreshestFrame(threading.Thread):
def __init__(self, capture, name='FreshestFrame'):
self.capture = capture
assert self.capture.isOpened()
# this lets the read() method block until there's a new frame
self.cond = threading.Condition()
# this allows us to stop the thread gracefully
self.running = False
# keeping the newest frame around
self.frame = None
# passing a sequence number allows read() to NOT block
# if the currently available one is exactly the one you ask for
self.latestnum = 0
# this is just for demo purposes
self.callback = None
super().__init__(name=name)
self.start()
def start(self):
self.running = True
super().start()
def release(self, timeout=None):
self.running = False
self.join(timeout=timeout)
self.capture.release()
def run(self):
counter = 0
while self.running:
# block for fresh frame
(rv, img) = self.capture.read()
assert rv
counter += 1
# publish the frame
with self.cond: # lock the condition for this operation
self.frame = img if rv else None
self.latestnum = counter
self.cond.notify_all()
if self.callback:
self.callback(img)
def read(self, wait=True, seqnumber=None, timeout=None):
# with no arguments (wait=True), it always blocks for a fresh frame
# with wait=False it returns the current frame immediately (polling)
# with a seqnumber, it blocks until that frame is available (or no wait at all)
# with timeout argument, may return an earlier frame;
# may even be (0,None) if nothing received yet
with self.cond:
if wait:
if seqnumber is None:
seqnumber = self.latestnum+1
if seqnumber < 1:
seqnumber = 1
rv = self.cond.wait_for(lambda: self.latestnum >= seqnumber, timeout=timeout)
if not rv:
return (self.latestnum, self.frame)
return (self.latestnum, self.frame)
And then you use it like:
# open some camera
cap = cv.VideoCapture('rtsp://URL')
cap.set(cv.CAP_PROP_FPS, 30)
# wrap it
fresh = FreshestFrame(cap)
Use fresh to deal with the open camera
I am currently working on my final year college project and I am stuck on what I think is a threading issue. I want to be able to run my method multiple times but each time it gets ran it should update a variable with new values. I am querying an API to get userID and then passing it into my main method by setting it as a global variable.
def setup():
try:
global gun_cascade, camera, frameRate, property_id, length, firstFrame, framecount,i,increment,start_Time,end_Time,statusCopy,userID
gun_cascade = cv2.CascadeClassifier('cascade.xml')
camera = cv2.VideoCapture('gun.mp4')
if camera.isOpened() == False:
print("Can't open video, isOpened is empty exiting now.")
exit(0)
frameRate = camera.get(5)
property_id = int(cv2.CAP_PROP_FRAME_COUNT)
length = int(cv2.VideoCapture.get(camera, property_id))
firstFrame = None
count = 0
gun_exist = False
increment = 0
start_Time = 0
end_Time = 0
i = 0
except Exception as e:
print(e)
exit(0)
Above I am setting userID to global
def decision():
setup()
user_object = listener.userIdListner()
tokenId = user_object.token
status = user_object.status
if user_object.status == "ON":
#status=statusCopy[:]
#aux = copy.deepcopy(matriz)
global statusCopy
statusCopy = copy.deepcopy(tokenId)
print("About to run mainscrt"+statusCopy)
#print(type(userID))
print(type(user_object))
post_object = listener.mainNullPostMethod()
post_object.null
print(post_object.null)
#opening a a new thread
Thread(target = main).start()
#post_object = listener.mainNullPostMethod()
#post_object.null
#print(post_object.null)
else:
print ("Program failed to run")
Here I am querying my API to get the userId and the status either on or off. At the moment this runs fine. But the problem is If this method is running and want to run it again with a new userID it works right up until the 'while camera.isOpened():' When I get to this point I get no error or anything
def main():
#printing out the userid to see if it's been changed
userID = statusCopy
print("The current userID is "+userID)
while isOpened:
framecount =0
framecount += 1
frameId = camera.get(1) #current frame number
(grabbed, frame) = camera.read()
if not grabbed:
break
# resize the frame, convert it to grayscale, and blur it
frame = imutils.resize(frame, width=500)
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
#gray = cv2.GaussianBlur(gray, (21, 21), 0)
#gray = cv2.dilate(gray, None, iterations=2)
#stuff to try in the future
#scaleFactor=1.1, minNeighbors=5, minSize=(30, 30), flags=cv2.CASCADE_SCALE_IMAGE, outputRejectLevels = True
gun = gun_cascade.detectMultiScale(gray, 3,5)
for (x,y,w,h) in gun:
randID = uuid.uuid4().hex
frame = cv2.rectangle(frame,(x,y),(x+w,y+h),(255,0,0),2)
roi_gray = gray[y:y+h, x:x+w]
roi_color = frame[y:y+h, x:x+w]
rects = gun[0]
neighbours = gun[0]
weights = gun[0]
if (frameId % math.floor(frameRate) == 1):
cv2.putText(frame, datetime.datetime.now().strftime("%A %d %B %Y %I:%M:%S%p"),(10, frame.shape[0] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.35, (255, 165, 0), 1)
cv2.imwrite('bin/' + userID+'-'+randID + '.jpg', frame)
if userID == "NULL":
print("failed due to user null")
break
print("working on pushing images to s3"+userID)
s3.uploadDirectory("bin/", "open-gun-recordings",userID)
picURL = "s3bucket.com/users/screenshots/"+userID+'/'+userID+'-'+randID+'.jpg'
text.fire(picURL)
cv2.imshow("Security Feed", frame)
key = cv2.waitKey(1) & 0xFF
camera.release()
cv2.destroyAllWindows()
Above I want to be able to have multiple instances of this method running at the same time and having a different userId for each instance.
if __name__ == '__main__':
#mainNullPostMethod()
#runDecision()
while True:
time.sleep(5)
decision()
Any help and suggestion would be greatly appreciated. I am not the best at python so apologies if this is a stupid question
First of all don't use global variables they are bad because it makes things difficult to track when changing from multiple functions (and in your case multiple threads) as described in this answer.
The problem I see is with initializing userID in the main function which you use to spawn threads, and the problem is that even though you initialize userID = statusCopy in main, even though you do a deepcopy in decision with statusCopy = copy.deepcopy(tokenId) it will still be overridden globally by any concurrent decision calls.
Let's imagine for a second that you call decision the first time, you initialize the userID and then you spawn a thread for main which makes use of that userID. Now I'm not sure how long it takes for main to execute but let's say for the sake of the argument that you wait for 5 seconds with the sleep and then do the whole thing again (while the first thread is still running). Now you basically change the userID with the second execution of the whole chain of functions and the first thread starts using the modified userID, which by definition is already bad practice since you wanted to use a specific userID information with your first thread. What I recommend is that you pass a deepcopy to the thread and initialize a local userID in the main, so that it does not get altered by concurrent threads.
Additionally, I'm not sure that you want to do a while True and spawn threads every 5 seconds, perhaps you should put a limit there also.
I'm using python and OpenCV to get video from a rtsp stream. I'm getting single frames from the stream and saving them to the file system.
I wrote a StreamingWorker which handles frame getting and saving. Additionally there is a StreamPool that has all the streaming objects. I thought that as the StreamingWorker would always be running, there should only be one per core, in order to take as much as possible. Then the StreamPool would provide the VideoCapture objects to the available StreamingWorker.
The problem is that most of the time that the script is running, is blocking:
import os
import time
import threading
import cv2 as cv
class StreamingWorker(object):
def __init__(self, stream_pool):
self.stream_pool = stream_pool
self.start_loop()
def start_loop(self):
while True:
try:
# getting a stream from the read_strategy
stream_object = self.stream_pool.next()
# getting an image from the stream
_, frame = stream_object['stream'].read()
# saving image to file system
cv.imwrite(os.path.join('result', stream_object['feed'], '{}.jpg'.format(time.time())))
except ValueError as e:
print('[error] {}'.format(e))
class StreamPool(object):
def __init__(self, streams):
self.streams = [{'feed': stream, 'stream': cv.VideoCapture(stream)} for stream in streams]
self.current_stream = 0
self.lock = threading.RLock()
def next(self):
self.lock.acquire()
if(self.current_stream + 1 >= len(self.streams)):
self.current_stream = 0
else:
self.current_stream += 1
result = self.streams[self.current_stream]
self.lock.release()
return result
def get_cores():
# This function returns the number of available cores
import multiprocessing
return multiprocessing.cpu_count()
def start(stream_pool):
StreamingWorker(stream_pool)
def divide_list(input_list, amount):
# This function divides the whole list into list of lists
result = [[] for _ in range(amount)]
for i in range(len(input_list)):
result[i % len(result)].append(input_list[i])
return result
if __name__ == '__main__':
stream_list = ['rtsp://some/stream1', 'rtsp://some/stream2', 'rtsp://some/stream3']
num_cores = get_cores()
divided_streams = divide_list(stream_list, num_cores)
for streams in divided_streams:
stream_pool = StreamPool(streams)
thread = threading.Thread(target=start, args=(stream_pool))
thread.start()
When I thought of this, I didn't take into account that most of the operations will be blocking operations like:
# Getting a frame blocks
_, frame = stream_object['stream'].read()
# Writing to the file system blocks
cv.imwrite(os.path.join('result', stream_object['feed'], '{}.jpg'.format(time.time())))
The problem with spending too much time blocking is that most of the processing power is wasted. I thought of using futures with a ThreadPoolExecutor but I can't seem to reach my goal of using the maximum amount of processing cores possible. Maybe I'm not setting enaugh threads.
Is there a standard way of handling blocking operations, in order to make the best use of the cores' processing power? I'm fine having a language-agnostic answer.
I ended up using the ThreadPoolExecutor using the add_done_callback(fn) function.
class StreamingWorker(object):
def __init__(self, stream_pool):
self.stream_pool = stream_pool
self.thread_pool = ThreadPoolExecutor(10)
self.start_loop()
def start_loop(self):
def done(fn):
print('[info] future done')
def save_image(stream):
# getting an image from the stream
_, frame = stream['stream'].read()
# saving image to file system
cv.imwrite(os.path.join('result', stream['feed'], '{}.jpg'.format(time.time())))
while True:
try:
# getting a stream from the read_strategy
stream_object = self.stream_pool.next()
# Scheduling the process to the thread pool
self.thread_pool.submit(save_image, (stream_object)).add_done_callback(done)
except ValueError as e:
print('[error] {}'.format(e))
I didn't actually want to do anything after the future finished, but if I used result() then the while True would stop, which whould also defeat all the purpose of using the thread pool.
Side note: I had to add a threading.Rlock() when calling self.stream_pool.next() because apparently opencv can't handle calls from multiple threads.