python opencv and tkinter capture webcam problem - python

hello everyone i have a code for read video from webcam and show it into the tkinter window with timer threading.
when user click show button, app make a thread and run it every x second to show frame by frame.
here's the problem : every several frame app shows first frame that captured from video source.
that's weird i know but i cant find out why!!!
here's my code:
import tkinter as tk
from tkinter import ttk
from tkinter import *
import threading
import cv2
import PIL.Image, PIL.ImageTk
from PIL import Image
from PIL import ImageTk
import time
class App(threading.Thread):
def __init__(self, root, window_title, video_source=0):
self.root = root
self.root.title(window_title)
self.video_source = video_source
self.show_switch=False
self.showButton = Button(self.root, text="PlayStream",command=self.showStram,width=15, padx="2", pady="3",compound=LEFT)
self.showButton.pack()
# Create a canvas that can fit the above video source size
self.canvas = tk.Canvas(root, width = 530, height = 397, cursor="crosshair")
self.canvas.pack()
self.root.mainloop()
def updateShow(self):
# Get a frame from the video source
cap=cv2.VideoCapture(0)
while True:
if(cap.isOpened()):
#read the frame from cap
ret, frame = cap.read()
if ret:
#show frame in main window
self.photo = PIL.ImageTk.PhotoImage(image = PIL.Image.fromarray(frame))
self.canvas.create_image(0, 0, image = self.photo, anchor = tk.NW)
else:
break
raise ValueError("Unable to open video source", video_source)
if self.show_switch==False:
cap.release()
return False
time.sleep(0.0416666666666667)
#release the cap
cap.release()
def showStram(self):
if self.show_switch:
self.showButton["text"]="StartStream"
# self.showButton.configure(image=self.playIcon)
self.show_switch=False
else:
self.showButton["text"]="StopStream"
self.show_switch=True
# self.showButton.configure(image=self.stopIcon)
self.showTimer=threading.Thread(target=self.updateShow,args=())
#self.showTimer.daemon=True
self.showTimer.start()
App(tk.Tk(), "Main")

This is a complicated question, because your example code is extensive and combines multiple things that can go wrong. I cannot test your code in its current form.
First off, you are accessing your Tk instance from a thread that is not the MainThread. This can cause all sorts of issues. There are also bugs present in the implementation of supposed thread-safety in Tkinter, and the solution has not yet been merged. Checkout mtTkinter if you really need to use multiple threads with Tkinter, and even then, it is better not to, especially not if you are building a new application and have the option to use Queues or some other system instead.
Second, you create a (subclass) instance of threading.Thread, but you never call threading.Thread.__init__ in App.__init__ (which is an absolute requirement if you want to use it as a Thread!). Then you create a new Thread in def showStream(self), while you actually already had a thread. Now, this does not break your code, but you do not need to subclass threading.Thread if you do not intend to use your class as a Thread. As you create a Thread in your class, there is no need to make a Thread out of your class.
Then, moving on through your code, you do start the Thread, so updateShow gets run. However, in your while loop, there is a problem:
while True:
if (cap.isOpened()):
ret, frame = cap.read()
if ret:
...
else:
break
# Error will never be raised because of break
# break exits the loop immediately, so the next line is never evaluated
raise ValueError()
cap.release()
# No notification of the loop having ended
There may be two things going wrong here. The first is that maybe your loop just ends because of the break in the else-clause. Break immediately exits the loop code. Anything following it will never be executed. If it does exit the loop because it failed to get the next frame, you cannot know for sure as you do not check whether the thread is still alive (threading.Thread.is_alive) or have any print statements to indicate that the loop has ended.
Secondly, your program might actually be hard-crashing on you accessing Tkinter from a second thread. Doing this causes undefined behaviour, including weird errors and Python-interpreter lockups because the Tk interpreter and Python interpreter are fighting for flow control in a deadlock (simply put).
Last but not least, there is a problem with the way you create your images:
self.canvas.create_image(0, 0, image = self.photo, anchor = tk.NW)
In this line, you create a new image on the Canvas. However, if an image already exists on the Canvas in the location where you are creating a new image, it will appear under the image already shown. If you do not delete your old image (which is in any case advisable to prevent a huge memory leak), it will remain visible on the Canvas.
def __init__(...):
...
self.im = None
def updateShow(self):
...
while True:
if (cap.isOpened()):
...
if ret:
if self.im is not None:
self.canvas.delete(self.im)
...
self.im: str = self.canvas.create_image(0, 0, image=self.photo, anchor=tk.NW)
Summarizing: With your current posted code, there are multiple things that can be going wrong. Without additional information, it is impossible to know. However, if you fix accessing Tkinter from a different Thread, adjust your while loop to not break but raise the error, you adjust your Canvas image creation code and your video source actually does work properly, it should work out.

Related

How to transport objects through modules with tkinter?

I am currently working on a project that takes a camera stream and after doing image processing, adds some overlays to measure some features of the image.
I am using tkinter for window management, threads to liberate the main thread (showing the camera stream) from image processing (sometimes quite lengthy operations and also non-live-showing critical dependency) and recently I moved to modules as the main script was +1000 lines.
Here I needed to make a decision about how I manage the project because I am using the window to show some updated values from the image processing, so I am quite literally "transporting" the window between modules (sending it as an argument in the buttons) so I can update values of the window in the aforementioned modules. Here is a MWE of what I am doing:
main.py:
import modules.win_manager as win_manager
if __name__ == "__main__":
win = win_manager.create_window()
win = setup_window(win)
win.root.mainloop()
win_manager.py in modules folder:
from functools import partial
from modules.cam import *
import tkinter as tk
def create_window():
root = tk.Tk()
return root
def img_proc(img,win):
img, val = process(img) #arbitrary image processing in a different file, imag_proc.py
win.objects.strv.set("Finished processing new value {}".format(val))
def cmd(win):
img = win.cam.get_camera()
img_proc(img,win)
def setup_window(win):
win.root = create_window()
win.objects= type('objects', (), {})()
win.objects.strv = strvar=tk.StringVar()
strvar.trace("w",lambda name, index, mode,strvar=strvar:win.cb(strvar))
root = tk.Tk()
win.objects.label=tk.Label(root,text="label 1")
win.objects.entry=tk.Entry(root,textvariable=strv)
win.button=tk.Button(root,text="action",command=partial(cmd,win))
win.cam = cam(win)
Finally the camera module cam.py in modules folder:
import threading
import cv2 as cv
class camera(threading.Thread):
def __init__(self,win,ini):
self.threads = type('threads', (object,), {})()
threading.Thread.__init__(self,daemon=True)#stops thread on exit
self.queue = queue.Queue()
self.queue.flag = False # Init to a "False" i.e. task is not completed
self.threads.flag = False
self.capture = init_camera() #arbitary function to get the camera with CV
def get_camera():
self.frame = self.capture.read()
return self.frame
My code is several more lines (i.e. more tkinter buttons which update several info fields) and the modules have many more functions, but the gist of what the functions do scale pretty much the same as in this code: Buttons take a window object using partial which has all the variables, objects and flags that I may need. In this case, my question is if it is worth all this internal transporting of my window object, and I should have instead made it a global object.
The code is working perfectly at the moment, the window gets updated with all the running threads, and so far there are no major bugs during runtime. My main question is if this
object passing through modules is something that is pythonic and somewhat good practice, or if I should opt for more global variables (especially the window object which has smaller objects that are frequently updated).
EDIT: Added martineau's correction of terminology.

Display continuously change of an image in python

I am writing a python program that gradually changes an image step by step, adjusting each pixel by a small amount in each step. To get a visualization of what the program is doing during runtime, I want it to display the image at each step, always overwriting the currently shown image so that it doesen't open bunch of display windows.
I already tried matplotlib, opencv and skimage, with their according possibilities to display an image and update the frame content in the course of the program:
# using scimage
viewer = ImageViewer(image)
viewer.show(main_window=False) # set the parameter to false so that the code doesn't block here but continues computation
..other code..
viewer.update_image(new_image)
# using matplotlib
myplot = plt.imshow(image)
plt.show(block=False)
.. other code..
myplot.set_data(new_image)
plt.show()
# using opencv
cv2.imshow('image',image)
.. other code ..
cv2.imshow('image', new_image)
I always ran into the problem that when it was supposed to open a frame with an image, it did not display the image but only a black screen. Weirdly enough, when I ran the code in IntelliJ in debug-mode and hit a breakpoint after the display-function, it worked.
What can I do so that it is displayed correctly when running the program normally and not with a breakpoint?
Here's the thing, I think your program does work, except it does and finishes unless you tell it to pause, which is why your breakpoint strategy is working.
Try pausing after showing image -
You can ask for user input. It'll pause until you enter some input to the program.
Put the program thread to sleep for some specified amount of time. This'll freeze your program for some given specified time, but you'll be able to see the image if it's already rendered.
Edit -
Since opencv's waitKey method is working for you now, you can use this method again to prevent the program from closing image window. Use waitKey(0) as your last program statement. It waits for a key press indefinitely, and returns the pressed key's code. Press any key to continue (but remember to have your image window in focus or it won't work), and your program should close if it's used in the end.
Also, I've striked earlier suggested options for pausing a program, because I'm unsure if it would've helped. I think waitKey method is more complex, and helps pause the program without freezing it.
Well, I am still not sure what exactly your goal is but here is a code piece that modifies an image inside of a window whenever the upper button is pressed.
from tkinter import Tk, Canvas, Button, Label, PhotoImage, mainloop
import random
WIDTH, HEIGHT = 800, 600
def modify_image():
print ("modifiying image...")
for x in range(1000):
img.put ( '#%06x' % random.randint(0, 16777215), # 6 char hex color
( random.randint(0, WIDTH), random.randint(0, HEIGHT) ) # (x, y)
)
canvas.update_idletasks()
print ("done")
canvas = Canvas(Tk(), width=WIDTH, height=HEIGHT, bg="#000000")
canvas.pack()
Button(canvas,text="modifiying image",command=modify_image).pack()
img = PhotoImage(width=WIDTH, height=HEIGHT)
Label(canvas,image=img).pack()
mainloop()
The function modify_image() adds 1000 random pixels to the image within the main window. Note the tkinter module is a default python module.

Trying to display opencv video in QGraphicsScene and QGraphicsView but Nothing shows up

I have a GUI app that I wrote in Python and I'm trying to display video in Qt app.
It uses a QGraphicsScene and QGraphicsView to display images but now I have to display a video. When I try to display a video here is what I do:
First, I create VideoCapture from cv2 package. After that I run loop and read frame in every iteration which is then casted into QPixmap (this is done correctly, I checked on a single frame). I return that QPixmap object to the Class which holds QGraphicsScene and QGraphicsView and try to add it to the scene.
Thing is, only when the video ends, the last frame is displayed, throughout video playing time I'm welcomed with this sighting
The loop that I mentioned earlier looks like this:
self.imageViewer.start_capturing()
self.timer = QtCore.QTimer()
self.fps = 24
while True:
retVal = self.imageViewer.read_frame()
if not retVal:
self.timer.stop()
break
self.timer.start(1000./self.fps)
self.imageViewer.stop_capturing()
self.ImageViewer is a component which holds the QGraphicsScene and QGraphicsView
I also put timer here so that it displays proper ammount of fps but it doesn't help.
The GUI app itself shows QGraphicsView with some buttons
middleLayout.addWidget(self.imageViewer.view)
This is what I mean. So it doesn't show imageViewer but it shows a subclass of QGraphicsView
Here is the code from read_frame method in ImageViewer class.
def read_frame(self):
"""
Reads a frame from the video, if video ended, then False is returned.
If there is a successful reading then True is returned
:return: True if video is read successfully, False otherwise.
"""
retVal, self.current_frame = self.capture.getFrame()
if not retVal:
return False
else:
self.pixmap = retVal
self.scene.addPixmap(self.pixmap)
return True
The method self.capture.getFrame() just returns QPixmap item and CV::mat item.
This whole process is done correctly because I tried manually reading frame by frame and everything worked OK, but when I try to display video the app freezed like shown in image above. So I manually tried to do the whole described process by manually clicking a button which loads me frame and puts it onto the QGraphicsScene so I assume that the core of the app works correctly (and I even tried to get roughly 5-7 fps by clicking fast :D )
I hope I made my problem clear and included all code that is needed.
I think your timing logic is wrong.. you do not use the timer correctly.. you could either use connect or you can use sleep.
I would do something like this:
def grabFrame
retVal = self.imageViewer.read_frame()
if not retVal:
self.timer.stop()
self.imageViewer.stop_capturing()
And somewhere in your main logic or some init function of your class:
yourObject.connect(timer,SIGNAL("timeout()"),yourObject,SLOT("grabFrame()"))
timer.start(1000./self.fps)
Or you can use some kind of sleep of time module after
import time
...
while True:
retVal = self.imageViewer.read_frame()
if not retVal:
break
time.sleep(1./self.fps)//in fraction of second
self.imageViewer.stop_capturing()

OpenCV imshow window cannot be reused when called within a thread

I'm having trouble using cv2 imshow when run in a separate python thread.
The code below works for the first call of start_cam, but the second call fails - the cv2 camera window does not reappear. This seems to have something to do with threading prevent that window being reused, because:
If the cv2 window is given a random name then it works indefinitely, although the window is not being reused as each window is new of course.
If _cam_loop() is called without threading this it also works and the same window can be reused indefinitely.
def start_cam(self):
self.cam_live = True
threading.Thread(target = self._cam_loop).start()
def stop_cam(self):
self.cam_live = False
def _cam_loop(self):
while self.cam_live:
img = self.cam.get_image()
cv2.imshow("cam", img)
cv2.waitKey(1)
self.start_cam() # live image window appears
self.stop_cam() # image window disappears (as thread is killed)
self.start_cam() # window fails to reappear
The window disappears when the thread finishes. Is there a way to keep a reference to the window after the thread finishes?
I didn't find a way to keep a reference to the named window, but if the window is destroyed it can be reused each time the thread is called. I simply added cv2.destroyAllWindows() to the end of the thread function and it worked. Curious to know why exactly.
def _cam_loop(self):
while self.cam_live:
img = self.cam.get_image()
cv2.imshow("cam", img)
cv2.waitKey(1)
cv2.destroyAllWindows()

OpenCV VideoCapture only updates after 5 read()s

I have a very strange error that has been plaguing my research for a few years now. I'm using OpenCV2 with Python to read image data from a webcam. However, the image is lagged by 5 frames. In other words, each call to read() is 5 frame behind real-time.
One bandage fix I have been using is to grab() 4 frames and then read the 5th every time I need an updated image, but that absolutely murders my performance.
Here's the code that I am using to display the images from the webcam
frame = self.getGoodFrame()
if self.DEBUG:
window = cv2.namedWindow("Angles")
imgHSV = cv2.cvtColor(frame, cv2.cv.CV_BGR2HSV)
... Reseach specific code I shouldn't be giving out here ...
... It finds the center of a few bright colors in an image
if self.DEBUG:
debugImage = numpy.zeros((self.CAMERA_HEIGHT, self.CAMERA_WIDTH), numpy.uint8) #blank image
... Then we draw some stuff to the image to be displayed ...
cv2.imshow("Angles", debugImage)
cv2.waitKey(1)
raw_input()
and getGoodFrame()
def getGoodFrame(self):
MIN_READS_FOR_GOOD_FRAME = 4
for i in xrange(MIN_READS_FOR_GOOD_FRAME):
successful_read = self.capture.grab()
successful_read, frame = self.capture.read()
if not successful_read:
print "Unable to read from webcam, exiting."
return frame
You'll notice that I have a raw_input() call. That makes it so I can see how many reads need to happen by pressing enter a few times in console. This shows that there is exactly 5 frames of lag.
I don't think it's a hardware issue, I've had this happen with multiple webcams and multiple USB cords. I haven't tried reproducing the error on another machine though.
So the problem was something to do with the way that my hardware was buffering the frames. I never quite got to the bottom of it but the solution I found was very simple paralellization. I modified my code to open up a thread and constantly update a variable holding the current frame. Then, when I need the current frame I just ask for whatever that variable is at the moment.
Note that this is still 5 read() calls behind because of the ever-present buffering, however because I am doing read() calls continuously and it's all on its own thread, it is very up to date. This is because I can call many read() calls per second. The image I get is not noticeably behind real-time at all.
The Python class I made to do this is below. It is not very nice code but it works. To do this properly, I would (and will) add graceful ways to exit the infinite loop. It will work for me though and it has improved the speed of my image detection code by well over 100 fold, which is extremely awesome and exciting for me :)
class webcamImageGetter:
def __init__(self):
self.currentFrame = None
self.CAMERA_WIDTH = #webcam width
self.CAMERA_HEIGHT = #webcam height
self.CAMERA_NUM = 0
self.capture = cv2.VideoCapture(0) #Put in correct capture number here
#OpenCV by default gets a half resolution image so we manually set the correct resolution
self.capture.set(cv2.cv.CV_CAP_PROP_FRAME_WIDTH,self.CAMERA_WIDTH)
self.capture.set(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT,self.CAMERA_HEIGHT)
#Starts updating the images in a thread
def start(self):
Thread(target=self.updateFrame, args=()).start()
#Continually updates the frame
def updateFrame(self):
while(True):
ret, self.currentFrame = self.capture.read()
while (self.currentFrame == None): #Continually grab frames until we get a good one
ret, frame = self.capture.read()
def getFrame(self):
return self.currentFrame
To use this, you initialize it then called start on the instance. This will make it so when you later called getFrame(), it will have the most up to date frame from the webcam. Woohoo!
Default buffer is set to 4.0 instead of 1.0.
To Fix it:
cap.set(38,1)
reference: https://docs.opencv.org/4.x/d4/d15/group__videoio__flags__base.html#ga41c5cfa7859ae542b71b1d33bbd4d2b4

Categories