We Keep Coding

results differ in computers when matching with cv2.matchTemplate and win32ui.CreateDCFromHandle

Here is my code
def get_screen(hwnd, zoom=1):
left, top, right, bot = [round(zoom*x) for x in win32gui.GetWindowRect(hwnd)]
width = right - left
height = bot - top
hwindc = win32gui.GetWindowDC(hwnd)
srcdc = win32ui.CreateDCFromHandle(hwindc)
memdc = srcdc.CreateCompatibleDC()
bmp = win32ui.CreateBitmap()
bmp.CreateCompatibleBitmap(srcdc, width, height)
memdc.SelectObject(bmp)
memdc.BitBlt((0, 0), (width, height), srcdc, (0, 0), win32con.SRCCOPY)
signedIntsArray = bmp.GetBitmapBits(True)
img = np.frombuffer(signedIntsArray, dtype='uint8')
img.shape = (height,width,4)
srcdc.DeleteDC()
memdc.DeleteDC()
win32gui.ReleaseDC(hwnd, hwindc)
win32gui.DeleteObject(bmp.GetHandle())
return cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
hwnd_list = []
def get_all_hwnd(hwnd,mouse):
if win32gui.IsWindow(hwnd) and win32gui.IsWindowEnabled(hwnd) and win32gui.IsWindowVisible(hwnd):
if(win32gui.GetWindowText(hwnd)=='梦幻西游三维版模拟器'):
hwnd_list.append(hwnd)
win32gui.EnumWindows(get_all_hwnd, 0)
hwnd = [hwnd_list[0]] # [hwnd_list[0]]
for hwnd in hwnd_list:
rect = win32gui.GetWindowRect(hwnd)
win32gui.MoveWindow(hwnd, rect[0], rect[1], 768, 461,True)
win32gui.SetBkMode(hwnd, win32con.TRANSPARENT)
img1 = get_screen(hwnd)
get = cv2.imread('D:/Code/template/get.jpg', 0)
cv2.matchTemplate(img1,get,cv2.TM_CCOEFF_NORMED).max()
This code works fine on the computer A (the similiarity can be up to 0.9), where templates were obtained. But when I run this code on computer B, the similiarity is only 0.5. I use the same template on computer B and only the screenshots are different. I belive this issue is caused by function get_screen.
Here is an example of screenshot
enter image description here
I expect the similarity in computer B is also above 0.9.

CreateDCFromHandle select object TOP coordinate in Python

I need to capture a specific application window on Windows, even if it is not focused or in the foreground, WITHOUT the window's top header bar, which height I specify manually in 'application_window_topbar_size' variable to keep it simple.
monitor_info = GetMonitorInfo(MonitorFromPoint((0,0))) # get screen information
work_area = monitor_info.get("Work")
screen_width = work_area[2] # if taskbar is on the left
screen_height = work_area[3] # if taskbar is on the left
target_window_hwnd = win32gui.FindWindow(None, ("Math Analysis - Google Chrome"))
application_window_topbar_size = 200 # just roughly for testing
hwndDC = win32gui.GetWindowDC(target_window_hwnd)
mfcDC = win32ui.CreateDCFromHandle(hwndDC)
saveDC = mfcDC.CreateCompatibleDC()
saveBitMap = win32ui.CreateBitmap()
saveBitMap.CreateCompatibleBitmap(mfcDC, screen_width, screen_height)
saveDC.SelectObject(saveBitMap)
# saveDC.BitBlt((0, 0), (screen_width, screen_height), mfcDC, (0, application_window_topbar_size),win32con.SRCCOPY)
result = windll.user32.PrintWindow(target_window_hwnd, saveDC.GetSafeHdc(), 3)
bmpinfo = saveBitMap.GetInfo()
bmpstr = saveBitMap.GetBitmapBits(True)
screen_image = Image.frombuffer('RGB', (bmpinfo['bmWidth'], bmpinfo['bmHeight']), bmpstr, 'raw', 'BGRX', 0, 1)
win32gui.DeleteObject(saveBitMap.GetHandle())
saveDC.DeleteDC()
mfcDC.DeleteDC()
win32gui.ReleaseDC(target_window_hwnd, hwndDC)
The capture routine should works for windows handle "WM_PRINT" (thanks for the comment of IInspectable), BUT I am unable to figure it out how to set the TOP coordinate of the capture area. In other words, I would like to capture the target window without the top bar. Changing the height is one thing, as logically, if the capture rectangle coordinates start at 0,0 then it cuts only the bottom area of the window, not the top. I tried to use BitBlt() but I failed.
----------------- Alternative version below based on IInspectable comment
target_window_hwnd = win32gui.FindWindow(None, ("Math Analysis - Google Chrome"))
application_window_topbar_size = 200 # just roughly for testing
hwndDC = win32gui.GetWindowDC(target_window_hwnd)
mfcDC = win32ui.CreateDCFromHandle(hwndDC)
saveDC = mfcDC.CreateCompatibleDC()
saveBitMap = win32ui.CreateBitmap()
l, t, r, b = win32gui.GetClientRect(target_window_hwnd)
screen_width = r - l
screen_height = b -t
saveBitMap.CreateCompatibleBitmap(mfcDC, screen_width, screen_height)
saveDC.SelectObject(saveBitMap)
result = windll.user32.PrintWindow(target_window_hwnd, saveDC.GetSafeHdc(), 3)
bmpinfo = saveBitMap.GetInfo()
bmpstr = saveBitMap.GetBitmapBits(True)
screen_image = Image.frombuffer('RGB', (bmpinfo['bmWidth'], bmpinfo['bmHeight']), bmpstr, 'raw', 'BGRX', 0, 1)
win32gui.DeleteObject(saveBitMap.GetHandle())
saveDC.DeleteDC()
mfcDC.DeleteDC()
win32gui.ReleaseDC(target_window_hwnd, hwndDC)
Still, altering top coordinate only effect height, not captured content (to be able to capture only the 'document area' of the window, excluding the address bar/toolbar).

After long search and trials, I realized that the most obvious solution is to CROP the screenshot after it made. This way, the unwanted part (in my case the top bar) can be easily get rid off.
Cropping is easy with PIL Image:
box = (left, top, right, bottom)
screen_image.crop(box)

Taking a screenshot of everything but 1 window

Ok, so I want to continuously screenshot the main screen (at something like 30 images per second), but I want my PyQt5 app to be invisible on the screenshot image. The app should stay on the screen at the same time (I am drawing things on the app based on the screenshot of the image)
Is there a way for my PyQt5 app to be invisible to my screenshots ?
Here is the function I currently use to take a screenshot :
def screenshot_screen(self, coords=None):
hwnd = win32gui.GetDesktopWindow()
if coords != None:
x0, y0, x1, y1 = coords
else:
x0, y0, x1, y1 = win32gui.GetWindowRect(hwnd)
width = x1 - x0
height = y1 - y0
wDC = win32gui.GetWindowDC(hwnd)
dcObj=win32ui.CreateDCFromHandle(wDC)
cDC=dcObj.CreateCompatibleDC()
dataBitMap = win32ui.CreateBitmap()
dataBitMap.CreateCompatibleBitmap(dcObj, width, height)
cDC.SelectObject(dataBitMap)
cDC.BitBlt((0, 0), (width, height), dcObj, (x0+1, y0+1), win32con.SRCCOPY)
im = dataBitMap.GetBitmapBits(True)
img = np.frombuffer(im, dtype='uint8')
img.shape = (height, width, 4)
img = cv2.cvtColor(img, cv2.COLOR_BGRA2BGR)
dcObj.DeleteDC()
cDC.DeleteDC()
win32gui.ReleaseDC(hwnd, wDC)
win32gui.DeleteObject(dataBitMap.GetHandle())
return img
To be clear, the yellow rects are from my pyqt5 app, I would want them drawing like they are on the right, but I wouldn't want them on the left window that shows the screenshot that is being taken:

Python - Screenshot of background/inactive window

I am attempting to take fast screenshots ready for processing with PIL/Numpy (~0.01s per screenshot) with Python 3.6. Ideally the window would not need to be in the foreground, i.e. even when another window is covering it, the screenshot is still successful.
So far I've modified the code for python 3 from this question: Python Screenshot of inactive window PrintWindow + win32gui
However, all it gets is black images.
import win32gui
import win32ui
from ctypes import windll
from PIL import Image
hwnd = win32gui.FindWindow(None, 'Calculator')
# Get window bounds
left, top, right, bot = win32gui.GetWindowRect(hwnd)
w = right - left
h = bot - top
hwndDC = win32gui.GetWindowDC(hwnd)
mfcDC = win32ui.CreateDCFromHandle(hwndDC)
saveDC = mfcDC.CreateCompatibleDC()
saveBitMap = win32ui.CreateBitmap()
saveBitMap.CreateCompatibleBitmap(mfcDC, w, h)
saveDC.SelectObject(saveBitMap)
result = windll.user32.PrintWindow(hwnd, saveDC.GetSafeHdc(), 1)
print(result)
bmp_info = saveBitMap.GetInfo()
bmp_str = saveBitMap.GetBitmapBits(True)
print(bmp_str)
im = Image.frombuffer(
'RGB',
(bmp_info['bmWidth'], bmp_info['bmHeight']),
bmp_str, 'raw', 'BGRX', 0, 1)
win32gui.DeleteObject(saveBitMap.GetHandle())
saveDC.DeleteDC()
mfcDC.DeleteDC()
win32gui.ReleaseDC(hwnd, hwndDC)
if result == 1:
im.save("screenshot.png")

This code worked for me with applications in background, not minimized.
import win32gui
import win32ui
def background_screenshot(hwnd, width, height):
wDC = win32gui.GetWindowDC(hwnd)
dcObj=win32ui.CreateDCFromHandle(wDC)
cDC=dcObj.CreateCompatibleDC()
dataBitMap = win32ui.CreateBitmap()
dataBitMap.CreateCompatibleBitmap(dcObj, width, height)
cDC.SelectObject(dataBitMap)
cDC.BitBlt((0,0),(width, height) , dcObj, (0,0), win32con.SRCCOPY)
dataBitMap.SaveBitmapFile(cDC, 'screenshot.bmp')
dcObj.DeleteDC()
cDC.DeleteDC()
win32gui.ReleaseDC(hwnd, wDC)
win32gui.DeleteObject(dataBitMap.GetHandle())
hwnd = win32gui.FindWindow(None, windowname)
background_screenshot(hwnd, 1280, 780)

Converting a WinAPI screenshot to a OpenCV compatible form

So I originally asked a question here about taking faster screen captures using win api as compared to PIL. I was able to succesfully capture the screen via BitBlt.
Now I am unsure how to convert the bitmap into a form that can be used with OpenCV. OpenCV doesn't have any support for bitmaps, and when I print(im) it is a ~14k long 1D array. OpenCV can't do anything with 1D, and I have tried to reshape it with NUMPY with no success.
def take_screenshot1(hwnd):
wDC = win32gui.GetWindowDC(hwnd)
dcObj=win32ui.CreateDCFromHandle(wDC)
cDC=dcObj.CreateCompatibleDC()
dataBitMap = win32ui.CreateBitmap()
dataBitMap.CreateCompatibleBitmap(dcObj, 765, 503)
cDC.SelectObject(dataBitMap)
cDC.BitBlt((0, 0), (765, 503), dcObj, (0, 0), win32con.SRCCOPY)
im = dataBitMap.GetBitmapBits(False)
#img = np.array(im)
#cv2.cvtColor(img, cv2.COLOR_RGB2BGR)
dcObj.DeleteDC()
cDC.DeleteDC()
win32gui.ReleaseDC(hwnd, wDC)
win32gui.DeleteObject(dataBitMap.GetHandle())
edit:
Here is the working code:
def take_screenshot1(hwnd):
wDC = win32gui.GetWindowDC(hwnd)
dcObj=win32ui.CreateDCFromHandle(wDC)
cDC=dcObj.CreateCompatibleDC()
dataBitMap = win32ui.CreateBitmap()
dataBitMap.CreateCompatibleBitmap(dcObj, 765, 503)
cDC.SelectObject(dataBitMap)
cDC.BitBlt((0, 0), (765, 503), dcObj, (0, 0), win32con.SRCCOPY)
im = dataBitMap.GetBitmapBits(False)
img = np.array(im).astype(dtype="uint8")
img.shape = (503,765,4)
cv2.cvtColor(img, cv2.COLOR_RGB2BGR)
dcObj.DeleteDC()
cDC.DeleteDC()
win32gui.ReleaseDC(hwnd, wDC)
win32gui.DeleteObject(dataBitMap.GetHandle())

GetBitmapBits() in its Python incarnation returns an array of signed ints instead of unsigned bytes. You should first convert it to unsigned bytes and then do as #DanMašek said.