So im trying to make a program that moves my mouse cursor and clicks every black pixel found on the screen.
I got to the point where I can input the screen, see it in a window and even let the mouse click on the black pixels. Without the clicking part in the program, I can see the windows changing in real time, but if I add the clicking part it stops refreshing.
import numpy as np
import pyautogui as py
from PIL import ImageGrab
import cv2 as cv
while(True):
# Record location of the program
screen_size = [1293, 171, 1647, 769]
screen = np.array(ImageGrab.grab(bbox=screen_size))
cv.imshow("window", cv.cvtColor(screen, cv.COLOR_BGRA2GRAY))
# Quit
if cv.waitKey(25) & 0xFF == ord("q"):
cv.destroyAllWindows()
break
for y in range(len(screen)):
for x in range(len(screen[y])):
if np.any(screen[y][x]) == 0:
py.click(x+1293,y+171)
I would like to have the screen refresh or something like that, so that lets say it looks at an video of black dots, it can see them and click on all of them. Now it is just stuck at the starting image and keeps clicking the starting dots even when they arent visiable anymore.
This is a video of the problem:
https://www.youtube.com/watch?v=QIrEnCgxe6E&feature=youtu.be
You can see here how it follows the black lines perfectly, but the window OpenCV creates doesnt change, and when I draw over some of the black parts, it still draws over it.
This is the window I see and how it converts the colors
The screen doesn't refresh/update, because the program is still working on the double for-loop that clicks all black pixels. If you want to use this for real-time video it is obviously way to slow a process. You could look into findContours, to click once on every black area. This will be much faster and gives the behavior that I think you want.
Below is some sample code that shows how findContours works. Here is additional info / examples.
Result:
Code:
# load image
img = cv2.imread("image.png")
# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# create a mask that hold only black values (below 10)
ret,thresh1 = cv2.threshold(gray,10,255,cv2.THRESH_BINARY_INV)
# find contours in mask
ret, contours, hierarchy = cv2.findContours(thresh1, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# draw outline of contour on image
for cnt in contours:
cv2.drawContours(img, [cnt], 0, (255,0,0), 2)
#show image
cv2.imshow("img", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Related
I will explain myself better, the point is that I want to develop a code that displays an image to me, then with the mouse as the image is displayed I can select or crop it as I want.
So, for example, as this code does, it would select any section of the image:
from PIL import Image
im = Image.open("test.jpg")
crop_rectangle = (50, 50, 200, 200)
cropped_im = im.crop(crop_rectangle)
cropped_im.show()
That is basically it, all I want is to crop an image at the points or coordinates that I please, I have been searching but can not find any library that helps me to do so.
NOTE: The code I am showing is an answer on this post, in case you want to check it out: How can i select a part of a image using python?
EDIT: Here is something I foud that may help me in a first instance, but is not entirely what I am looking for, it at least lets me find the coordinates that I want from the image - with some modifications on the code - and then I will keep processing the image. By now it is not solving my issue but it is a beginning. --> Using "cv2.setMouseCallback" method
The comments I received from Mark Setchell and bfris enlightened me, one with a C++ code and another one with a recommendation of OpenCV.
But in addition to their comments, I found this article where they explain exactly what I want to do, so I consider my question answered. Select ROI or Multiple ROIs [Bounding box] in OPENCV python.
import cv2
import numpy as np
#image_path
img_path="image.jpeg"
#read image
img_raw = cv2.imread(img_path)
#select ROI function
roi = cv2.selectROI(img_raw)
#print rectangle points of selected roi
print(roi)
#Crop selected roi from raw image
roi_cropped = img_raw[int(roi[1]):int(roi[1]+roi[3]), int(roi[0]):int(roi[0]+roi[2])]
#show cropped image
cv2.imshow("ROI", roi_cropped)
cv2.imwrite("crop.jpeg",roi_cropped)
#hold window
cv2.waitKey(0)
or
import cv2
import numpy as np
#image_path
img_path="image.jpeg"
#read image
img_raw = cv2.imread(img_path)
#select ROIs function
ROIs = cv2.selectROIs("Select Rois",img_raw)
#print rectangle points of selected roi
print(ROIs)
#Crop selected roi ffrom raw image
#counter to save image with different name
crop_number=0
#loop over every bounding box save in array "ROIs"
for rect in ROIs:
x1=rect[0]
y1=rect[1]
x2=rect[2]
y2=rect[3]
#crop roi from original image
img_crop=img_raw[y1:y1+y2,x1:x1+x2]
#show cropped image
cv2.imshow("crop"+str(crop_number),img_crop)
#save cropped image
cv2.imwrite("crop"+str(crop_number)+".jpeg",img_crop)
crop_number+=1
#hold window
cv2.waitKey(0)
I can't open a 24MP pictures on Python with opencv. It only opens the upper left part apparently and not the full image. The kernel also stops after running the code.
Here's my code:
import cv2
import numpy as np
PICTURE_PATH_NAME = "IMG.JPG"
img = cv2.imread(PICTURE_PATH_NAME)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.imshow("Gray Image", gray)
cv2.waitKey(0)
See the documentation for imshow as to how to get it to scale your image to fit the window at https://docs.opencv.org/4.1.1/d7/dfc/group__highgui.html#ga453d42fe4cb60e5723281a89973ee563
"If you need to show an image that is bigger than the screen
resolution, you will need to call namedWindow("", WINDOW_NORMAL)
before the imshow."
I have this image:
that I'm trying to paste on to this one:
I want to get rid of the black background (so I'm assuming "make it transparent"?).
I tried applying a black mask, but
the result is this:
I'm not quite understanding how to use a mask properly.
I thought creating a black mask would get rid of the black portions.
from PIL import Image
image = Image.open(r'..jpg')
image_2 = Image.open(r'...jpg')
image_copy = image.resize((300,300)).copy() #fish image
mask=Image.new('L', (300,300), color=0)
position = ((200,100))
image_2.paste(image_copy, position, mask = mask)
image_2.save('testing_1.png')
EDIT: Playing around with the code a bit more I'm able to overlay the image onto the background,
but it still has the black background surrounding it.
I would like to get RID of the black background.
1. I am doing a project regarding face detection, i just want to know how to display like this
I able to get that rectangle but not able to display the string within the box.
2. I have done the detection using haar and it is only detecting when we are looking directly to it. i want to make it more accurate when we turn left/right/up down - help me
My code looks like this.
import cv2
import sys
import logging as log
import datetime as dt
from time import sleep
cascPath = "haarcascade_frontalface_default.xml"
faceCascade = cv2.CascadeClassifier(cascPath)
log.basicConfig(filename='webcam.log',level=log.INFO)
video_capture = cv2.VideoCapture(0)
anterior = 0
while True:
if not video_capture.isOpened():
print('Unable to load camera.')
sleep(5)
pass
# Capture frame-by-frame
ret, frame = video_capture.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(
gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30)
)
# Draw a rectangle around the faces
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
if anterior != len(faces):
anterior = len(faces)
log.info("faces: "+str(len(faces))+" at "+str(dt.datetime.now()))
# Display the resulting frame
cv2.imshow('Video', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# Display the resulting frame
cv2.imshow('Video', frame)
# When everything is done, release the capture
video_capture.release()
cv2.destroyAllWindows()
You need: Posing and Projecting Faces
Humans can easily recognize that both images are of Will Ferrell, but computers would see these pictures as two completely different people:
To account for this, we will try to warp each picture so that the eyes and lips are always in the sample place in the image. This will make it a lot easier for us to compare faces in the next steps.
To do this, we are going to use an algorithm called face landmark estimation. There are lots of ways to do this, but we are going to use the approach invented in 2014 by Vahid Kazemi and Josephine Sullivan.
The 68 landmarks we will locate on every face. This image was created by Brandon Amos of CMU who works on OpenFace.
Here’s the result of locating the 68 face landmarks on our test image:
PROTIP: You can also use this same technique to implement your own version of Snapchat’s real-time 3d face filters!
Now that we know were the eyes and mouth are, we’ll simply rotate, scale and shear the image so that the eyes and mouth are centered as best as possible. We won’t do any fancy 3d warps because that would introduce distortions into the image. We are only going to use basic image transformations like rotation and scale that preserve parallel lines (called affine transformations):
Now no matter how the face is turned, we are able to center the eyes and mouth are in roughly the same position in the image. This will make our next step a lot more accurate.
If you want to try this step out yourself using Python and dlib, here’s the code for finding face landmarks and here’s the code for transforming the image using those landmarks.
This explanation taken from this medium post.
The source code is available on GitHub.
To your first question, you can use
cv2.putText(image, 'ABC', (x, y-12), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (30,255,30), 2)
This question is related with this one:
how to display a full screen images with python2.7 and opencv2.4
I want to display a black image full screen, i have created even a black image with the same resolution of the screen.
But i get a little white stripe on top and on the left of the screen.
I don't know if it is a problem of my screen that is not aligned or its my code. I have tried in 2 displays and the white stripe is displayed.
So if you run this code below, do you get a full black image?
import numpy as np
import cv2
if __name__ == "__main__":
img = cv2.imread('nero.jpg')
cv2.namedWindow("test", cv2.WND_PROP_FULLSCREEN)
cv2.setWindowProperty("test", cv2.WND_PROP_FULLSCREEN, cv2.cv.CV_WINDOW_FULLSCREEN)
cv2.imshow("test",img)
cv2.waitKey(0)
cv2.destroyAllWindows()
EDIT :
This method is not working for me. do you know anther way or libraries to display a full screen image?
EDIT 2: still unsolved, i am starting to think that it is an openCv bug
I have the same problem, there is a white stripe of 1 pixel on the left and on the top side of the window. Tested it with multiple monitors. OpenCV version 3.4.2
But there is a workaround which works perfectly fine in my case (see also https://gist.github.com/goraj/a2916da98806e30423d27671cfee21b6). Here's the code:
import cv2
import win32api
import win32gui
cv2.namedWindow("fullScreen", cv2.WINDOW_FREERATIO)
cv2.setWindowProperty("fullScreen",cv2.WND_PROP_FULLSCREEN,cv2.WINDOW_FULLSCREEN)
hwndMain = win32gui.FindWindow(None, "fullScreen")
rgb = win32gui.CreateSolidBrush(win32api.RGB(0, 0, 0))
GCLP_HBRBACKGROUND = -10
win32api.SetClassLong(hwndMain, GCLP_HBRBACKGROUND, rgb)
Yes I do.
img = np.zeros((900, 1600)) #my aspect ratio is 16x9
cv2.namedWindow("test", cv2.WND_PROP_FULLSCREEN)
cv2.setWindowProperty("test", cv2.WND_PROP_FULLSCREEN, cv2.cv.CV_WINDOW_FULLSCREEN)
cv2.imshow("test",img)
cv2.waitKey(0)
gives me a fully black screen. Are you sure you are using the right aspect ratio?