I have been thoroughly searching this forum, the internet, and pretty much everything. I want to be able to create a new graphics window in python without using something else, for a project in which I want specialized optimized for whatever a need without any other extra mumbo-jumbo, and understanding fully how it works. Anyways, I want to be able to access pixel by pixel of a new window without using tkinter/turtle, installations, or anything of the sort. Just basic, simple, pixel access of a new window. If this isn't possible, just let me know. I imagine the program could look something like this:
import NOTHING_AT_ALL_GRAPHICS_WISE
window.pixel(1,2,True)
I program in Python 3.6.3
What you seek to accomplish isn't as simple as you might believe. Applications draw graphics on the screen largely with the help of calls to the graphics card API. Those API's can be complicated, as they should be, because all the graphics that you see on your screen, whether it be your browser GUI, to video playback, to your Starcraft II, has to use combinations of these calls to produce the beautiful and seamless graphics you see on your screen (graphics would NOT look good without use of the GPU).
However, not all manipulation of graphics has to be so complicated, and many languages/frameworks write wrappers around some of the graphics libraries that expose a simpler subset of the total functionality. Tkinter is a good example.
If you don't want to import ANYTHING, you will have to make the calls to the API yourself, and you're not in for a good time. Nobody does this nowadays for personal projects because it really is quite complicated. Also, Python is a very high-level language, and although it might expose the OS's system calls that you need, you're probably better off doing it in C. Before you proceed, I suggest you read up on the fundamentals of computer graphics work. Here's a nice image that shows how OpenGL, a very popular graphics library, works.
There is no way to do what you want. Python has no way of changing a single pixel on the screen. This is the whole reason why such "mumbo jumbo" libraries exist: it's a complex problem that requires specialized libraries that know how each operating system works.
Related
TLDR: Is there a Python library that allows me to get a application window frame as an image and rewrite it to the said application?
So the whole story is that I want to write an application using Python that does something similar to Lossless Scaling and Magpie. I want to grab an application window (a videogame window, for example), get the current frame as an image, then use some Machine Learning/Deep Learning algorithm (like FSR or DLSS) to upscale said image, then rewrite the current frame from the application with said upscaled image.
So far, I have been playing around with some upscaling algorithms like the one from Real-ESRGAN, but now my main problem is how to upscale the video game images in real-time. The only thing I found that does something related to what I need to do is PyAutoGUI. But this package only allows you to take screenshots of an application but not rewrite the graphics of said application.
I hope I have clarified my problem; feel free to comment if you still have any questions.
Thank you for reading this post, and have a good day.
Doing this with Python is going to be very difficult. A lot of the performance involved in this sort of thing is in avoiding as many memory copies as possible, and Python's idiom for string and bytes processing unfortunately makes quite a few additional copies in the course of any idiomatic program. I say this as a die-hard Python fan who is constantly trying to cram Python in everywhere it doesn't belong: you'd be better off doing this in Rust.
Update: After receiving some feedback from some folks with more direct experience in this sort of thing, I may have overstated the difficulty here. Many ML tools in Python provide zero-copy access, you can easily access and manipulate memory-mapped data from numpy and there is even a CUDA protocol for doing this to data in GPU memory, so while it's not exactly easy, as long as your operations are implemented as numpy operations and not as pure-python pixel-by-pixel logic, it shouldn't be much harder than other python machine learning applications which require access to native APIs for accessing their source data.
However, there's no way to access framebuffer data directly from python, so step 1 is going to be writing your own bindings over the relevant DirectX APIs. Since Magpie is open source, you can see which APIs it's using, for example, in its various C++ "Frame Source" backends. For example, this looks relevant: https://github.com/Blinue/Magpie/blob/42cfcba1222b07e4cec282eaff639aead229f123/Runtime/GraphicsCaptureFrameSource.cpp#L87
You can then look those APIs up on MSDN; that one, for example, is here: https://learn.microsoft.com/en-us/uwp/api/windows.graphics.capture.direct3d11captureframepool.createfreethreaded?view=winrt-22621
CFFI is a good choice for writing native wrappers: https://cffi.readthedocs.io/en/latest/
Gluing these together appropriately is left as an exercise for the reader :).
I'm writing a simple window manager as a hobby project, and I've chosen python and xlib to implement it. I'd like to have some fancy effects like windows sliding from left and right, and from what I've been able to dig up, the best way to go about this is to use the composite extension to render windows to offscreen buffers, and then manipulate those buffers and paint them however I want.
However, with python-xlib documentation somewhat lacking, I found no way to do this, and I haven't found any examples of using python-xlib with the composite extension. Where can I find this information, and has anyone used python-xlib with composite?
I suppose I could always switch to xcb which seems to support composite for python, but I'd prefer not having to rewrite the whole thing and figure out a different API with somewhat lacking python documentation.
To summarize, my questions are:
Is there a way to use composite with python-xlib?
Would it be a better idea to switch to xcb for this?
Any additional suggestions and advice are welcome.
Yes, there is support for Composite extension in python-xlib. Take a look at extension client implementation You'll probably need composite_redirect_window and composite_name_window_pixmap
I am using python inside another application (CINEMA 4D) create a nice connection to out issue tracker (Jira) inside the application. Rationale behind this is to make it really easy for our plugin users to report and track bugs and have things like machine specs, screenshots or attaching scene files (including textures) automatically.
So far it as been a really smooth ride and the integration is coming along great. I started grabbing the icons for issue priorities, projects, issue types, etc. from Jira as well so they can be displayed for better overview. To read the image files I am using CINEMA 4D functionality that is available inside its python binding.
The problem now is, that most icons from Jira come in GIF format and the CINEMA 4D SDK doesn't read GIF files directly (actually it does read them, but only through a back door so users can load them, but I can't use that functionality through Python or the SDK). So I need another way to read the GIF files.
There are a few questions on stackoverflow that go into this direction, but they all seem to recommend PIL. This doesn't feel like the right solution for a few reasons:
While that looks nice, it's not part of the standard distribution and seems to be really only maintained for Windows (even though there are builds for Mac OS X).
It also seems to install itself into the current system installation of Python, but CINEMA 4D comes with its own, so I'd have to rip it apart and distribute it with my plugin.
And then it is quite large, while I really only want a compact script to have a compact solution (preferably out of the box, but that doesn't seem to be an option)
I was wondering if there is a simpler or at least more compact way. Since GIF seems to be a relatively simple file format, I am wondering if there may even be a simple parser as a python function/class.
I found a link where somebody disassembles a gif files embedded frames, but doesn't actually grab the image contents: Python, how i can get gif frames
I'm fine with putting in some time on my own, and I would've already been coding away if the file format was something uncompressed, but I am a little reluctant since the compression seems to raise the bar slightly.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm looking to make a 2d side scrolling game in Python, however I'm not sure what library to use. I know of PyGame (Hasn't been updated in 3 years), Pyglet, and PyOpenGL. My problem is I can't find any actually shipped games that were made with Python, let alone these libraries - so I don't know how well they perform in real world situations, or even if any of them are suitable for use in an actual game and not a competition.
Can anyone please shed some light on these libraries? Is PyGame still used effectively? Is Pyglet worth while? Have either of them been used to make a game? Are there any other libraries I'm forgetting?
Honestly I'm not even sure I want to use Python, it seems too slow, unproven (For games written solely in Python), etc.... I have not found any game that was made primarily in Python, for sure, that has been sold. If I don't end up going with Python, what would a second/better choice be? Java? LUA?
Pygame is just a wrapper for SDL, so technically a lot of games have been made using it, most of the work is done C-side anyways. It's a little slow, though if you are doing a 2D game, it should be sufficient. Frets-on-Fire was written in pure python with PyGame, and FoFiX was written in python with C extensions, and those games are both pretty great, altough on slower computers FoFiX can have trouble running 4 player.
Pyglet is also pretty nice, though I've only used it for emulator development, drawing single layers, so I don't know how well it actually works for game development. I've found that it's much easier to use direct OpenGL with Pyglet than it is with SDL/PyGame, so if you are doing a 3D game, it's probably better than using SDL. Also, since it's written in python, it's very easy to extend classes to change how a function is handled. For instance, I was trying to load data into an openGL texture through the pyglet Texture class, but the class couldn't load the data in a format I wanted, so I extended that class and changed the function that got the texture type (comes from a string) to include the texture I wanted, quick and painless.
One library I don't see mentioned enough is SFML. The 2.0 API (currently in beta) is pretty stellar and simple to use, and, if I remember correctly, runs a little faster than SDL. It is also more "object oriented" than SDL, which is really nice if you are working in an OO language like Python. It has some nice Python bindings written by Bastien Leonard in Cython that are pretty easy to use (or you can use the old C bindings if you don't want to use the most up to date version). I used this a while back and it was quite an enjoyable experience.
The reason why there aren't a lot of shipped games using Python is pretty simple: You can't close Python source. Even if you "compile" python into an executable using Py2Exe or something, it really just packs up the interpreter and python source into an exe file, and the source can be got just by opening up the exe in a hex editor.
While I'm a supporter of the "write it in Python, write time critical parts in C", if you feel python is going to be too slow, I would probably suggest C or, preferably, C++ as the way to go, simply because you don't get that much of a usability advantage for writing in Java, and the C languages can be quite a bit faster running than Java, and Lua will just take away some of the usability of Python without giving you any noticeable performance enhancement. SFML and SDL can both be used straight from C(++).
Advice if you do decide to use python: Use it as it is supposed to be used, as a scripting language. Do not try to bit twiddle or load images in pure python, either use the interfaces available in the library you choose (they all have very good interfaces for those kinds of things) or write an extension module to do that for you. Use python for logic control and fetching data from your libraries. If you follow that simple rule, you really shouldn't run into any performance issues.
In my experience, the things that are really going to kill you aren't generally related to graphics at all. Sure, it's a challenge to get all your frame rendering done in less than 16 ms without techniques like dirty-rectangle updating, but that's not going to be the big thing you'll lose sleep over. The biggest performance considerations are going to come from much more elementary things - collision detection, spatial searching, and pathfinding are three of the big ones - that can easily send your logic frame rate into the floor if done inefficiently over even a few hundred objects. For problems like these, pure Python may not be the best choice, but it's not hard to offload these to C/C++ while retaining Python for higher-level logic. This is essentially what games like Civilization 4 and EVE Online (client and server!) do - offload performance-critical code to C++ while retaining the flexibility and simplicity of Python for 'everyday' tasks.
pygame is actively developed (they haven't had a stable release in a while, but the Mercurial repository is quite lively) and fairly popular. It also uses the well-regarded (C-based!) SDL library under the hood. pyglet is also actively developed and uses OpenGL for all of its rendering, making moving to 3D development easier if you're planning to go in that direction (since you'll already have most of the basics).
If you decide to go the not-Python route, you can use SDL directly with C/C++, along with addon libraries like SDL_gfx (for primitives), SDL_ttf (for text), and SDL_net (for basic networking). Another excellent and widely used alternative for C/C++ is Allegro. I've used both, and I will claim that it's largely a matter of taste which one you go with.
Pygame should suffice for what you want to do. Pygame is stable and if you look around the websites you will find games which have been coded in pygame. What type of game are you looking to implement?
I want to learn about graphical libraries by myself and toy with them a bit. I built a small program that defines lines and shapes as lists of pixels, but I cannot find a way to access the screen directly so that I can display the points on the screen without any intermediate.
What I mean is that I do not want to use any prebuilt graphical library such as gnome, cocoa, etc. I usually use Python to code and my program uses Python, but I can also code and integrate C modules with it.
I am aware that accessing the screen hardware directly takes away the multiplatform side of Python, but I disregard it for the sake of learning. So: is there any way to access the hardware directly in Python, and if so, what is it?
No, Python isn't the best choice for this type of raw hardware access to the video card. I would recommend writing C in DOS. Well, actually, I don't recommend it. It's a horrible thing to do. But, it's how I learned to do it, and it's probably about as friendly as you are going to get for accessing hardware directly without any intermediate.
I say DOS rather than Linux or NT because neither of those will give you direct access to the video hardware without writing a driver. That means having to learn the whole driver API, and you need to invoke a lot of "magic," that won't be very obvious because writing a video driver in Windows NT is fairly complicated.
I say C rather than Python because it gives you real pointers, and the ability to do stupid things with them. Under DOS, you can write to arbitrary physical memory addresses in C, which seems to be what you want. Trying to get Python working at all under an OS terrible enough to allow you direct hardware access would be a frustration in itself, even if you only wanted to do simple stuff that Python is good at.
And, as others have said, don't expect to use anything that you learn with this project in the real world. It may be interesting, but if you tried to write a real application in this way, you'd promptly be shot by whoever would have to maintain your code.
This seems a great self-learning path and I would add my two-cents worth and suggest you consider looking at the GObject, Cairo and Pygame modules some time.
The Python GObject module may be at a higher level than your current interest, but it enables pixel level drawing with Cairo (see the home page) as well as providing a general base for portable GUI apps using Python
Pygame also has pixel level methods as well as access methods to the graphics drivers (at the higher level) - here is a quick code example
This is rather an old thread now, but I stumbled upon it while musing the same question.
I used to program in assembly language. In my day, drawing on screen was simply(?) a matter of poking a value into a memory location. The value turned a pixel on or off and defined its colour.
The term 'poke' comes from Basic by the way, not assembler. In assembler, you had to write a value into a data register then tell the processor where to put the data using another command and specifying an address register, usually in hexadecimal form! And each different processor had its own assembly language. But hec was the code fast!
As hardware progressed, I found that graphics hardware programming became more and more complex. There's much more to it than now simply defining a pixel. The graphics subsystem has its own processor -- or processors -- and it's that that you've got to learn to talk to. The processor doesn't just plonk stuff in memory locations. (I believe that what used to be the fastest supercomputer in the world for a while ran on graphics chips!) 'Plonk' is not a Basic command by the way.
Sorry; I digress. In answer to the original poster's query, I believe that the goal of understanding the graphics-drawing process could have been best achieved by experimenting with a Raspberry Pi. It's Python compatible and hence perfect for the job. Its hardware is well documented and it's cheap and easy to use.
Hope this helps someone, Cheers, M