I have a task to create an augmented reality application which has a GUI similar to this one (the overlay route and the FollowMe concept). The interesting part of the task is that the application should be done on a desktop OS (Debian/Ubuntu) with the help of the Qt framework. Python is the preferred language for the application.
So, the big difference to the aforementioned mobile app is that my application will use a USB camera plugged into a PC.
I have tried a few different approaches and none of them seems to satisfy the requirements:
this example does not work;
this solution uses a 'two layers' approach, but when I try to render the upper widget it fully covers the bottom widget;
this solution works, but it has 2 disadvantages: OpenCV's main usage is not video stream rendering (wrong tool, I guess GStreamer could fit better) and also it renders only simple 4-point shapes and it does not allow any complex polygons with holes;
I have also skimmed through the Nokia's Qt camerality source code, but it seems that this problem is solved on Symbian in a completely different way which is inapplicable on PC (embedded camera API vs a generic USB camera on PC).
I tend to think that the problem can be divided into two questions:
What is the best way to render a camera video stream in a Qt PC application?
What is the best way to overlay an animated 3D image (probably OpenGL generated) over a camera video stream in a Qt PC application?
If you are targeting a desktop I would use openGL for the whole process.
Use a quad in the background with a texture streaming the video (ie draw each video frame to the openGL texture) then render all your 3D model in openGL ontop of this.
Generaly all the video player frameworks (phonon, directshow etc) all work well as long as you ONLY want to do the simplest "show video in a box" task - as soon as you try and mix them with anything else it's harder than just drawing the images.
(Of course I may be wrong and I would love it if phonon allowed showing a video in an openGL quad with a couple of lines of code!)
Related
Is there any way of embedding a video player into the GUI of a python program and then displaying video from a piCamera to it in real time?
Currently, when I preview it fills the whole screen and makes any GUI unusable.
I would like to overlay information over the video capture.
I'm not too hot on Python but on Visual Studio for example you could use the VLC plug-in and use it as a display component.
There's lots of tutorials about how to host video and stream it to a server but nothing covers local display.
Cheers.
I'm not sure if this is exactly what you are looking for, but I use RPi-Cam-Web-Interface. If you are not aware of it you may want to take a look: http://elinux.org/RPi-Cam-Web-Interface
I am looking to make a GUI in python but currently do not have much experience. The GUI must have a few key features, namely a slider bar to control audio, and a few basic menu buttons. I realize essentially all GUI development tools could handle these simple features, but I am also interested in some custom content as well. The basic look of the GUI I am looking to create is shown here:
In the image, the slider volume bar, "button1," "button2," the colored circles, and any slice of the circle (one highlighted section is shown) needs to be clickable and interactive. Moreover, the small colored circles must be able to dynamically revolve around the edge of the circle and remain clickable at all times. I have not used any GUI development tools yet, but have looked into pyGTK, pyQT, wxWidgets, and Kivy. Can anyone who has used these tools recommend which would be best suited? As far as shapes of buttons, am I restricted?
You can use pyopengl, pygame , pygtk .
I have one example how to show this images if you want take a look at :
http://free-tutorials.org software free section is decor.tgz example.
In my opinion you can use pyOpenGL is very portable to another language like C,C++ and also you can make it to Linux , Windows , Android.
I am starting out on a project in which I need to build a customized annotation tool for movies and video. Some person (not technically minded) will need to pop open a GUI that I create, open either a video file or a directory of frames that result from chopping up a video file, and then use a window (much like QuickTime or VLC player, etc., i.e. a video window with a simple slider bar allowing the user to move back and forth at will). In this window, the user will be able to click on interesting points, give them semantic labels and meta-data (such as whether or not the point is occluded by something else in the picture), and then basically "press go" and start a tracker. The tracker will follow the points, frame by frame and the user can press the space bar or something to move forward and backward. The idea is to allow the human to intervene any time the tracker gets confused, but hopefully the tracker works well enough that humans don't have to hand-label every frame in an entire multi-thousand frame video sequence.
I am planning to do this all in Python, (a) because it is the language I know best for non-trivial programming, (b) I have easy access to both OpenCV Python (for image processing algorithms) and PyQt which seems to have a powerful enough GUI toolbox for what I want to do and (c) some other aspects of this same project are being developed by other programmers to work in Python and with MySQL databases. Python just seems like the natural choice to streamline it all together.
I am experienced using computer vision algorithms for the tracking, and I am reasonably sure that I can figure out simple PyQt GUI devices for making points clickable, buttons, entering simple text data, etc. However, the part I am having trouble understanding is how to actually construct my own video window with a slider bar that either moves ahead according to a frame's number or else is actually manipulating a video file. Can I leverage other movie players like VLC from within PyQt when programming in Python? Any suggestions or links that describe similar movie/video editing GUIs and how to develop them at home would be greatly appreciated.
Qt(PyQt) has a good multimedia support via Phonon module. You can easily use that module to achieve Video window, it can provide an easy-to-use video player and you can get playing position etc.
I need a tile/sprite editor kind of like Pixen, but I couldn't find one for Windows so I thought it might be a good exercise for me to try and put one together. I use Python, so are there any libraries out there that are suited to the task of putting together a simple tile/sprite editor?
You just need a gui toolkit (gtk, qt, wx) a image library (PIL) and 500 hours of free time ...
Have you looked at the Python Imaging Library (PIL)?
So, the fact is that creating a complex app with a nice UI takes time - I am just expanding a little bit on the answer by THC4k.
PIL, at least PIL alone is useless for this: it does have some functions to manipulate images, but the complicate task here is creating and tunning your desired UI.
That's where the widgets toolkits come in: You would have to pick a toolkit platform that can offer you buttons, images, load and save the image files, maybe some specialzed widgets you can use to create your color swatches, etc.
both GTK+ and QT4.5 have a liberal license, are very complete and very unpythonic on their use :-(
(While you are at it, when using these libraries and toolkits our app can easily be multiplatform: you don't have to make it windows specific, it is equally easy to create an app that will run on Windows, Linux and Mac using python and either GTK+ or Qt4)
One thing I would suggest is for you to learn to proper use GIMP: it is an Image editor, and certainly it will lack a lot of tools you are needing for sprites: but you can expand it's capabilities with Python plug-ins. On the other hand GIMP does have thousands of features that you'd no longer will need to create for your stand-alone app. (think on layer support, color filters, image rotation etc...)
Check around on how to install GIMP with Python support on Windows, then spend some hours learning the app, with some book-like text around preferably so you can find the hidden features.
Ah, ok, finally:
If you want a very simple thing, just for the taste of "i did it" - you can use Pygame: You have to do all the drawing on the window, including text - but have straighter access to pixels, colors, mouse clicks and coordinates than with GTK+ or Qt, in a sense it would be a lot less of overhead for you to learn in terms of API's and internal working.
You could try PyGame but, seriously, you couldn't find a freeware graphics editor for Windows??!!
EDIT: In the past I've used Aha-Soft's IconXP for pixel work, but it costs USD 30 and doesn't offer all of the Pixen features that I guess you'll want.
I'm looking for a Python framework that will enable me to play video as well as draw on that video (for labeling purposes).
I've tried Pyglet, but this doesn't seem to work particularly well - when drawing on an existing video, there is flicker (even with double buffering and all of that good stuff), and there doesn't seem to be a way to get the frame index in the video during the per-frame callback (only elapsed time since the last frame).
Try a Python wrapper for OpenCV such as ctypes-opencv. The C API reference is here, and the wrapper is very close (see docstrings for any changes).
I have used it to draw on video without any flicker, so you should have no problems with that.
A rough outline of calls you need:
Load video with cvCreateFileCapture, load font with cvFont.
Grab frame with cvQueryFrame, increment your frame counter.
Draw on the frame with cvPutText, cvEllipse, etc etc.
Display to user with cvShowImage.
Qt (PyQt) has Phonon, which might help out. PyQt is available as GPL or payware. (Qt has LGPL too, but the PyQt wrappers don't)
Try the Python bindings for GStreamer.