I am using Python 3.5.1 and OpenCV 3.0.0.
I am working on a python program that can play games, so it needs to 'see' what is going on, on the screen. How can this be achieved?
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
#work with frames here
Is there a int 'a' such that cv2.VideoCapture(a) will take the desktop screen as video input? I tried making it and I followed a rather clumsy approach, I captured screen repeatedly using:
import os
os.system('screencapture test.jpg')
Then opening test.jpg using cv2.imread. This was a very slow approach, on searching online I found this question Screen Capture with OpenCV and Python-2.7 which does the same thing, but more efficiently. But the fact still remains that it is capturing individual screenshots and processing them one by one and not a true video stream. I also found this How to capture the desktop in OpenCV (ie. turn a bitmap into a Mat)? which I think is close to what I am trying but is in C++, if someone can help me convert this to Python, I will highly appreciate it.
The main thing is that the program will be doing something like MarI/O, so speed is a concern, any help is appreciated, go easy on me, I am (relatively) new to OpenCV.
Thanks.
Just an update on this question in case anyone wants a solution.
Taking screenshot can be achieved by using module pyautogui
import pyautogui
import matplotlib.pyplot as plt
image = pyautogui.screenshot()
plt.imshow(image)
If you want to read it as a stream,
while(True):
image = pyautogui.screenshot()
#further processing
if finished:
break
According to the documentation,
On a 1920 x 1080 screen, the screenshot() function takes roughly 100 milliseconds
So this solution can be used if your application does not demand high fps rate.
Taking screenshots in separate thread sounds good solution.
Also you can use virtual webcam, but it is a heavy solution.
Or you would capture desktop directly by using ffmpeg. https://trac.ffmpeg.org/wiki/Capture/Desktop
Related
I am trying to write a code for detecting the color green from a live video. I want to make a detector so that whenever the color green pops up in the screen, a counter starts counting how many times the color appears.
So for the video source, I am using the OBS Virtual Camera. But I have no idea how to input it as the source. I have seen codes inputting web cams as the source as shown below:
import numpy as np
import cv2
# Capturing video through webcam
webcam = cv2.VideoCapture(0)
Anyone have any idea how I can input the OBS virtual cam? Or does anyone know any alternative like switching to another language to do said task?
Windows will treat OBS Virtual Camera as a regular camera. The argument for cv2.VideoCapture is camera number. So up that number by 1 over and over again until the program uses the OBS Virtual Camera. And there you go.
Keep in mind that there is a bug currently reported that opencv is not parsing the stream from OBS virtual cam and just showing a black background.
https://github.com/obsproject/obs-studio/issues/3635
I'm trying to read a raw video file codified in bayer_rggb8 in python to store some specific frames. I Know that with FFmpeg I probably store all the frames in the format that I want and then work with them. Still, the point is that this is a first version of the program and the next step is to use it online with a grasshopper camera that sends the data in bayer_rggb8 format, and I want to process the frames without the need of storing them in the disk.
My first try was with the videocapture functions from OpenCV, but it didn't work. I think a right solution may be open the file with open(filepath,'rb') and read the number of bytes in a frame, and then transform from rggb8 space to OpenCV bgr. I haven't found a way to do it, however.
Have you ever done anything similar? Is there a bookstore that can help me? I don't ask for the code directly, but right now I think I'm so lost that I'm not able to see the solutions.
I'm using python 3.6 and opencv in Kubuntu 18.04. The frame size is 600X1920
Thanks!!
I found a solution; it was easier than I thought. I think I was blocked
video = open(filepath,'rb')
framesize = 600 * 1920
initbyte = framesize * initframe
video.seek(initbyte)
frame = self.video.read(framesize)
frame = np.frombuffer(frame, dtype=np.uint8).reshape(600,1920)
bgrframe = cv2.cvtColor(frame,cv2.COLOR_BayerBG2BGR)
I'm currently trying to build an ANN that can play the online game "Helicopter Game" (see picture below if you're unfamiliar) using only the pixels of screenshots for training.
I've built similar models in OpenAI Universe but was hoping to try my hand at training directly on an online game instead of using an emulator.
The first thing I tried was to use the Selenium screenshot method to capture 100 screenshots at 10 frames per second.
for i in range(100):
driver.save_screenshot(r'C:\Users\MyName\Desktop\Screenshots\shot'+str(i)+'.png')
time.sleep(0.1)
But Selenium doesn't seem to be able to handle that kind of speed, it can only capture about 2 or 3 screenshots per second, even when I take away the time delay, and this is before even doing any preprocessing of the images.
Does anyone know of a method faster than what I'm trying to accomplish with Selenium?
You can give a try to the MSS module, and more precisely to that example to only capture the revelant part of the screen.
The module can be used with PIL, Numpy and OpenCV for other work, just check the doc' :)
I am trying to write a program for image display and face capture. I am somewhat new to python and OpenCV.
Please find the details below:
I am running Python 2.7.5 on win32 on windows XP
Open cv2 version is 3.0.0
For the program,
Images from a predefined folder needs to be displayed for a fixed time of 500 millisecond in random sequence.
The gap between the images should be covered through a black screen, which should come for any random time gap interval between 1000-1500 millisecond.
Face capture of the viewer needs to be done via webcam once image showed, in between the image show, i.e. at the point of 250 millisecond. The output of the face should be stored in another newly created folder each time the program is run.
I have written the code below, but not getting the sequence right with a synchronised face capture with Haarcascade integration(perhaps required).
I also read somewhere that 'camera index' could be involved in this with possibly the value zero assigned to it. What exactly could be its role?
Please assist in this. Thanks in advance.
import cv2
from matplotlib import pyplot as plt
img = cv2.imread('C:\\Sourceimagepath.jpg', 1)
cv2.startWindowThread()
cv2.namedWindow("Demo")
cv2.imshow("Demo", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
I'm currently trying to use a .png of a Tic Tac Toe board in Pygame so that the white part is transparent. However, I'm getting "libpng warning: sBIT: invalid" and the white is showing when I blit the board onto the background. The code is as follows:
background = pygame.image.load("blah.bmp")
board_surface = pygame.image.load("board.png")
board_surface.convert_alpha()
I read on the Pygame docs that only .bmp is guaranteed to work, and since image.get_extended() is returning a 1, I'm assuming that I simply don't have enough image processing support.
I've installed Pillows and imported the module, but I have no idea how to use this with Pygame. The tutorial shows how to load images as Pillow's Image class and so forth, but I need to represent images as Pygame's Surfaces.
I would appreciate any help with integrating these two things, or any other solutions on how to get my .png to work! For reference, I'm using Python3.3 and Pygame1.9.2
I have encountered this problem too. I solved it by switching to .tif (only with one f !). If you have Gimp installed use that to turn the white spaces into alpha=0 spaces (if you have problems doing that I can help with a more detailed explanation). After that import the image just as normal and use the .convert_alpha() method.