I am trying to understand how I can use PIL in Python 2.7 to search the whole screen for a certain image and click on it. I've been searching around and haven't been able to find a solution. I want to create a small GUI with one button in the middle of it that when clicked will search the entire screen for a predefined image. Once the image is found the program will then click in the centre of it and end. In short the program will detect if an image is present on the users screen and click it.
I did find an interesting bit on Sikuli, but that doesn't help me because it's unable to export to an .exe.
The image that the program will look for will most likely be in the same place each time it searches, but I didn't want to hard-code the location as it has the potential to move and I don't want that being an issue later on.
What I need is the code method I would use to search for the image on screen and send back the cords to a variable.
Image explanation/example:
Reference image of rifle:
PIL is the wrong tool for this job. Instead you should look into openCV (open source computer vision), which has fantastic python bindings. Here is a link to an example (in C but should be easy to redo with the python bindings) that does what you are looking for, but even allows the image to be rotated, scaled, etc.
http://docs.opencv.org/doc/tutorials/features2d/feature_homography/feature_homography.html
http://docs.opencv.org/doc/tutorials/features2d/detection_of_planar_objects/detection_of_planar_objects.html
Edit:
I assume you are using windows, as your example image looks like window. In this case you can use:
from PIL import ImageGrab
pil_img = ImageGrab.grab()
opencv_img = numpy.array(pil_img)
then use opencv to process the image to find sub image you are looking for.
If you want to do this cross platform, then you will need to use wxWidgets to do the screengrab: https://stackoverflow.com/a/10089645/455532
Even I wanted to do the same but using different module - pyautogui. I finally found the solution for my problem and I am sure this solution will also help you.
You have to just go to this webpage and read the locate function topic completely
and you'll be able to solve your problem.
I recommend you give a look on PyAutoGUI, a well documented library to control mouse and keyboard, also can locate imagens on screen, find the position, move the mouse to any location and clicks on location, also can simulate drag and drop, type on input fields, give double clicks and much more.
Related
I would like to show an image with transparent background to indicate something when a key combination is pressed.
Let's say I pressed ctrl+f3, I trigger a python script. Is there anyway I can make that happen?
What python library can I use to show an image without window border and background?
I have figured out how to trigger the file on key press. How to I deal with the (imshow) thing?
Thank you.
show an image without window border and background
This sound like task for some GUI library. There are many available but you would need test them in order to find which one can do it. First feature is generally known as frameless or borderless window. tkinter which ships with python has ability to work this way, see for example tutorialspoint.com tutorial, though I do not know how it will work with alpha channel of your image.
I wanted to use Python to create animations (video) containing text and simple moving geometric objects (lines, rectangles, circles and so on).
In the book titled "Python 2.6 Graphics Cookbook" I found examples using Tkinter library. First, it looked like what I need. I was able to create simple animation but then I realized that in the end I want to have a file containing my animation (in gif or mp4 format). However, what I have, is an application with GUI running on my computer and showing me my animation.
Is there a simple way to save the animation that I see in my GUI in a file?
There is no simple way.
The question Programmatically generate video or animated GIF in Python? has answers related strictly to creating these files with python (ie: it doesn't mention tkinter).
The question How can I convert canvas content to an image? has answers related to saving the canvas as an image
You might be able to take the best answers from those two questions and combine them into a single program.
I've accomplished this before, but not in a particularly pretty way.
Tl;dr save your canvas as an image at each step of the iteration, use external tools to convert from image to gif
This won't require any external dependencies or new packages except having imagemagick already installed on your machine
Save the image
I assume that you're using a Tkinter canvas object. If you're posting actual images to the tk widgets, it will probably be much easier to save them; the tk canvas doesn't have a built-in save function except as postcript. Postscript might actually be fine for making the animation, but otherwise you can
Concurrently draw in PIL and save the PIL image https://www.daniweb.com/software-development/python/code/216929/saving-a-tkinter-canvas-drawing-python
Take a screenshot at every step, maybe using imagegrab http://effbot.org/imagingbook/imagegrab.htm
Converting the images to to an animation
Once the images are saved, I used imagemagick to dump them into either a gif, or into a mpg. You can run the command right from python using How to run imagemagick in the background from python or something similar. It also means that the process is implictely run on a separate thread, so it won't halt your program while it happens. You can query the file to find out when the process is done.
The command
convert ../location/*.ps -quality 100 ../location/animation.gif
should do the trick.
Quirks:
There are some small details, and the process isn't perfect. Imagemagick reads files in order, so you'll need to save the files so that alphabetical and chronological line up. Beware that the name
name9.ps
Is alphabetically greater than
name10.ps
From imagemagick's point of view.
If you don't have imagemagick, you can download it easily (its a super useful command-line tool to have) on linux and mac, and cygwin comes with it on windows. If you're worried about portability... well... PIL isn't standard either
There is a way of doing that, with the "recording screen method", this was explained in other question: "how can you record your screen in a gif?".
Click the link -->LICEcap : https://github.com/lepht/licecap
They say that it's free software for Mac (OS X) and Windows
You could look at Panda3D, but it could be a little over killed for what you need.
I would say you can use Blender3d too but i'm not really sure of how it works. Someone more experimented then me could tell you more about this.
Is it possible to take screenshots of a running program (with GUI) from another python program ?
If so, what could be the steps and libraries that I could use ? (On Windows)
For example, let's say I have calc.exe running. I'd want to take screenshots of what is displayed to the user from myprogram.py.
My goal is to analyze what's displayed on the monitored program.
If it's not possible to isolate the screenshot to a running predefined program, I think I will have to take screenshots of the fullscreen but it's not very practical.
Capturing an screenshot is easy. Just install the Python Imaging Library and use the ImageGrab.grab() function to return an Image instance with the screenshot.
Capturing an specified window is a little more complicated, because you need the window coordinates. I recommend you to install the win32api modules and use a little module called winGuiAuto.py. Once you do that, you can do something like this:
hwnd = winGuiAuto.findTopWindow(title)
rect = win32gui.GetWindowPlacement(hwnd)[-1]
image = ImageGrab.grab(rect)
However, capturing the screen is the easy part. If you want to analyze the contents from screenshots, you're in for a lot of complications. This is probably the wrong approach for doing what you want and should be left as a last resort.
In most cases, it's easier to use the windows api to read the contents of a window's elements directly, but that won't work with some 3rd party GUI toolkits. That's not within the scope of your question so I'm not detailing it here, but you should read the source of the winGuiAuto.py module mentioned above for examples on how to do that, as well as checking the pywinauto library.
The ImageGrab Module, works on Windows only. The pyscreenshot module, is a better replacement for that, can be used to copy the contents of the screen to a PIL or Pillow image memory. Read more at link below.
https://pypi.python.org/pypi/pyscreenshot
I am using the Pygame module in python to take pictures with my webcam. The problem is that I would like to export a video file (don't care what type) to use elsewhere. Since pygame cannot export video directly, I guess that there is two ways to do it:
Somehow stitch the photos Pygame creates into a video. (my preferred method)
Use an external library.
I only need 4 frames per second, and I don't care about the picture quality.
How can I make a video with python / pygame?
I have the same problem and am searching the solution.
I found this
This seems work well though I didn't try yet.
Hope this helps.
I'd like to write a screenshot tool in Python. Right now i'm studying how I could do. I've got a script making a screenshot using win32 api like here, but want to include the mouse pointer in the screenshot.
With win32gui.GetCursorInfo() i get its state (shown/hidden), its handle and its position. My goal is to use the handle to access the actual bitmap of the cursor, copy it, and add it to my screenshot at the right coordinates. However, I can't seem to find any doc saying what I can do with that handle once I got it.
I'm thinking i might have to do like the screenshot itself, i.e. get a DC of the cursor, create a bitmap compatible with it and copy into it with BitBlt. Am I on the right track?
Does the python win32 api include DrawIconEx:
http://msdn.microsoft.com/en-us/library/ms648065%28v=vs.85%29.aspx
you need to make a drawable context to write it on, and then pass that and the cursor handle...
(or plain DrawIcon)
Anther option is to actually use Python Imaging Library (PIL)
you can easily edit bitmap with it.
(much simpler then actually using win32 apis.)
you can use ImageGrab, and ImageDraw
Once you get the cursor's icon handle from GetCursorInfo, you can draw it with DrawIcon or DrawIconEx, which I presume the Win32 API module you're using supports. You will need to create a DC for the screenshot bitmap to render the icon to, yes, but I don't think you'll need to create a DC for the cursor icon itself -- DrawIcon should be sufficient.