I am trying to build a GUI using pygame but when I try to stream a camera using opencv
and show it on the screen, pygame rotates the picture 90 degrees to the right.
How do I prevent pygame from rotating the picture?
image = self.cam.read() # getting the picture
if image is not None:
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # transforming the picture from BGR to RGB
image = pygame.surfarray.make_surface(image) # making the picture a pygame surface
self.snapshot = image
# blit it to the display surface. simple!
self.display.blit(self.snapshot, (0, 0)) # displaying the picture
pygame.display.update() # updating the screen
Use numpy.transpose to permute the x and y axis of the image. This causes a rotation by 90°:
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
image = image.transpose(1, 0, 2)
Related
With the below sample code, i am using basic dlib face detection.
I was initially drawing a bounding box around the detected face but I now wanted to display only within what is detected(AKA the face): img[top:bottom,left:right,:]
import sys
import dlib
import cv2
detector = dlib.get_frontal_face_detector()
cam = cv2.VideoCapture(1)
color_green = (0,255,0)
line_width = 3
while True:
ret_val, img = cam.read()
rgb_image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
dets = detector(rgb_image)
#for det in dets:
#cv2.rectangle(img,(det.left(), det.top()), (det.right(), det.bottom()), color_green, line_width)
new_img = img[top:bottom,left:right,:]
cv2.imshow('my webcam', new_img)
if cv2.waitKey(1) == 27:
break # esc to quit
cv2.destroyAllWindows()
The issue that I am facing, is that it is successfully showing me what is within the x,y,w,h but the image kept resizing depending on how close I am to the camera.
What I did is the following steps:
I got the coordinates of the detection: img[top:bottom,left:right,:]
I then resized the image to a 480 to 480 size focus_face = cv2.resize(img, (480, 480))
And then passed the image to show.
So the issue Im having is if I resize the array(img) it does not seem to be following the detected face but focusing at the centre of the screen, especially the more I move back.. So if im at the centre of the screen then it shows my whole face, if im at the sides, it will only show a part of my face.
I did my best to explain this, but If you have any questions please let me know.
Best.
I need to replace the transparency layer of a png image with a color white. I tried this
from PIL import Image
image = Image.open('test.png')
new_image = image.convert('RGB', colors=255)
new_image.save('test.jpg', quality=75)
but the transparency layer turned black. Anyone can help me?
Paste the image on a completely white rgba background, then convert it to jpeg.
from PIL import Image
image = Image.open('test.png')
new_image = Image.new("RGBA", image.size, "WHITE") # Create a white rgba background
new_image.paste(image, (0, 0), image) # Paste the image on the background. Go to the links given below for details.
new_image.convert('RGB').save('test.jpg', "JPEG") # Save as JPEG
Take a look at this and this.
The other answers gave me a Bad transparency mask error. The solution is to make sure the original image is in RGBA mode.
image = Image.open("test.png").convert("RGBA")
new_image = Image.new("RGBA", image.size, "WHITE")
new_image.paste(image, mask=image)
new_image.convert("RGB").save("test.jpg")
Lets make it straightforward.
I have private project to block or pixelate image using boundary box in open-cv, something like censoring image, inspired from this paper:
https://www.researchgate.net/publication/325746502_Seamless_Nudity_Censorship_an_Image-to-Image_Translation_Approach_based_on_Adversarial_Training
I have found the way to classify the area of censor using Keras, but still don't know the way how to use the boundary box to pixelate the classified area, and overlay it to original image. Any help are appreciated.
This is the example of the process that I want to do:
A simple method is to extract the ROI using Numpy slicing, pixelate, then paste it back into the original image. I will be using the pixelation technique found in how to pixelate image using OpenCV in Python?. Here's a simple example:
Input image and ROI to be extracted
Extracted ROI
Pixelated ROI
Result
Code
import cv2
def pixelate(image):
# Get input size
height, width, _ = image.shape
# Desired "pixelated" size
h, w = (16, 16)
# Resize image to "pixelated" size
temp = cv2.resize(image, (w, h), interpolation=cv2.INTER_LINEAR)
# Initialize output image
return cv2.resize(temp, (width, height), interpolation=cv2.INTER_NEAREST)
# Load image
image = cv2.imread('1.png')
# ROI bounding box coordinates
x,y,w,h = 122,98,283,240
# Extract ROI
ROI = image[y:y+h, x:x+w]
# Pixelate ROI
pixelated_ROI = pixelate(ROI)
# Paste pixelated ROI back into original image
image[y:y+h, x:x+w] = pixelated_ROI
cv2.imshow('pixelated_ROI', pixelated_ROI)
cv2.imshow('image', image)
cv2.waitKey()
Note: The ROI bounding box coordinates were found by using the script in how to get ROI Bounding Box Coordinates without Guess & Check. For your case, I will assume that you already have the x,y,w,h bounding box coordinates obtained by cv2.boundingRect.
so I'm making a bot that detects icons on the screen and moves the mouse to the detected icon the user chose. this is what the code looks like:
import numpy as np
import pyautogui
import cv2
from PIL import ImageGrab
fourcc = cv2.VideoWriter_fourcc(*'XVID')
face_csc = cv2.CascadeClassifier('improved_cascade.xml')
out = cv2.VideoWriter("output.avi", fourcc, 5.0, (1366, 768))
while True:
img = ImageGrab.grab(bbox=None)
# convert image to numpy array
img_np = np.array(img)
# convert color space from BGR to RGB
frame = cv2.cvtColor(img_np, cv2.COLOR_BGR2RGB)
# show image on OpenCV frame
faces = face_csc.detectMultiScale(frame, 1.1 , 15)
for (x,y,w,h) in faces:
detected_icon = cv2.rectangle(frame,(x,y),(x+w,y+h), (255,0,0), 2)
roi_gray = frame[y:y+h, x:x+w]
roi_color = img_np[y:y+h,x:x+w]
cv2.putText(frame,'icon',(x,y),cv2.FONT_HERSHEY_TRIPLEX,0.8,(0,0,255),1)
cv2.imshow("stream", frame)
# write frame to video writer
out.write(frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
out.release()
cv2.destroyAllWindows()
but I'm having trouble making my mouse click on an icon opencv detected. for example: lets say that I set my program so that when it detects chrome on the screen, it hovers the mouse automatically to the icon and click on it. how would I be able to do that? thanks
I don't have a handy Windows box to run ImageGrab on, but assuming it produces a screenshot with the same width and height (in pixels) as the actual screen, given that both Pyautogui and OpenCV put the origin in the top left, the translation to Pyautogui should be straightforward:
for (x,y,w,h) in faces:
center_x = x + 0.5 * w
center_y = y + 0.5 * h
pyautogui.click(center_x, center_y)
That teleports the mouse pointer to the center of the object rectangle and clicks it. In case you want to simulate more human-like mouse movement and clicking, Pyautogui has a number of tools for that purpose.
I am using Python and OpenCV. I am now using grabcut() to crop out the object I want. Here is my code:
img = cv2.imread('test.jpg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
mask = np.zeros(img.shape[:2], np.uint8)
bgdModel = np.zeros((1, 65), np.float64)
fgdModel = np.zeros((1, 65), np.float64)
rect = (2,2,630,930)
cv2.grabCut(img,mask,rect,bgdModel,fgdModel,5,cv2.GC_INIT_WITH_RECT)
mask2 = np.where((mask==2)|(mask==0), 0,1).astype('uint8')
img = img*mask2[:,:, np.newaxis]
Afterwards, I try to find out the contour.
I have tried to find the contour by the code below:
imgray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(imgray,127,255,0)
im2, contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
And it returns a contours array with length 48. When I draw this out:
First question is how can I get the contour (array) of this grab cut?
Second question: as you can see, the background color is black. How can I change the background color to white?
Thank you.
At first, you need to get the background. To this must be subtracted from the original image with the mask image. And then change the black background to white (or any color). And then back to add with the image of the mask.
import numpy as np
import cv2
cv2.namedWindow(‘image’, cv2.WINDOW_NORMAL)
#Load the Image
imgo = cv2.imread(‘input.jpg’)
height, width = imgo.shape[:2]
#Create a mask holder
mask = np.zeros(imgo.shape[:2],np.uint8)
#Grab Cut the object
bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)
#Hard Coding the Rect… The object must lie within this rect.
rect = (10,10,width-30,height-30)
cv2.grabCut(imgo,mask,rect,bgdModel,fgdModel,5,cv2.GC_INIT_WITH_RECT)
mask = np.where((mask==2)|(mask==0),0,1).astype(‘uint8’)
img1 = imgo*mask[:,:,np.newaxis]
#Get the background
background = imgo – img1
#Change all pixels in the background that are not black to white
background[np.where((background > [0,0,0]).all(axis = 2))] =[255,255,255]
#Add the background and the image
final = background + img1
#To be done – Smoothening the edges….
cv2.imshow(‘image’, final )
k = cv2.waitKey(0)
if k==27:
cv2.destroyAllWindows()
Information taken from the site
https://nxtify.wordpress.com/2015/02/24/image-background-removal-using-opencv-in-python/
If you want a single contour like boundary, you can go for edge detection on the output of grabcut and morphology dilation on the edge image to get as a proper connected contour and can get the array of pixels of the boundary.
For making the background white, All the pixels outside your bounding box can be default made to white. The black pixels inside your bounding box, you can compare with the original image corresponding gray level, if it is black, you can keep it, otherwise make it to white. Because if the original pixel is not black, but made black by grabcut, then it is considered as background. If black pixel is there in foreground, the grabcut never makes it to black (ideal case).