We Keep Coding

Compiling python to a small exe file

I know that python is interpreter language and that it means it converts script to machine code when you run it, but can someone clarify to me why can't it be pre converted to machine code (compiled).
I'm asking this because I have been making these simple programs with GUI using tkinter and I am using pyInstaller to create executable file. The problem is that sometimes I get this simple program (only a few lines of code) as big as 10s of MBs.
As I understand, PyInstaller packs all libraries and interpreter with script in exe which results in these sizes, but is there a way I can get this to somewhat rational size? Because I know same programs made in C++ much smaller (less than MB).
I don't know any of C or C++ so that isn't an option as I don't have time to learn it. Size really isn't that big to be much of a problem, but still, it's nice to have smaller file.

Check if the virtual environment that you are using does not have site-packages that you don't need. Usually it's these packages that are large. Create a new venv and only install the libraries you need and try.

Deploying Python modules for 3D software

I have been developing a fairly extensive library of python modules that automate the more time consuming parts of "3D character development" for games/film/tv.
All of my code up until a few months ago has been run within Maya's dedicated python interpreter, however, my GUIs are built in PySide/PyQt, and so, run just fine in mac/windows/linux or a few other Graphics programs such as Nuke, XSI, Max.
What I would really like to figure out is a "simple" way to distribute my code to various different people ---> using various different operating Systems ---> potentially using various applications (Nuke, XSI, Max), which, in turn, have their own dedicated python interpreters.
The obvious option would be pip and easy_install.. These modules are clearly the "right" way to go, but its not really clear how a user would install/run them under the dedicated python installs that ship with Maya/Nuke/ etc...Though, it does seem possible (as explained here). Still Its going to be a pretty big barrier for a less-technical user.
Any help or points in the right direction would be immensely appreciated..

I would not say that pip/easy_install are the 'right' way for this problem. They are pretty good (not quite 'great') tools for motivated, technically inclined users -- but even in that context they have issues (such as unintended upgrades or deletions). Most importantly, they are opt-in methods: nobody can make you pip unless you want to. This means users can accidentally or deliberately get themselves into very different positions from each other, which makes support and maintenance a nightmare.
I've had very good luck in Maya distributing a zipped file containing a complete environment - all the modules etc. userSetup.py adds that zip to the path and the Python's native zipimport functionality handles the rest. This makes sure that there is only one file to maintain and distribute. It also fixes the common problem of leftover .pyc files creating havok after .py files get moved or renamed. Since this is all standard python, I'd assume this will work for any app-specific python that uses a 2.6+ version of python, though I've never tried it in Nuke or Max.
The main wrinkle will be modules with .pyd or other binary components, typically these don't work inside the zip files. I include a bootstrap routine which unpacks those to a (disposable) location on the user's disk and adds that to the path.
There's a detailed discussion of the method here and some background here

Python .pyc files and Windows UAC

I'm working on an Inno Setup installer for a Python application for Windows 7, and I have these requirements:
The app shouldn't write anything to the installation directory
It should be able to use .pyc files
The app shouldn't require a specific Python version, so I can't just add a set of .pyc files to the installer
Is there a recommended way of handling this? Like give the user a way to (re)generate the .pyc files? Or is the shorter startup time benefit from the .pyc files usually not worth worrying about?

PYC files aren't guaranteed to be compatible for different python versions. If you don't know that all your customers are running the same python versions, you really don't want to distribute pyc's directly. So, you have to choose between distributing PYCs and supporting multiple python versions.
You could create build process that compiles all your files using py_compile and zips them up into a version-specific package. You can do this with setuptools.; however it will be awkward to do because you'll have to run py_compile in every version you need to support.
If you are basically distributing a closed application and don't want people to have trivial access to your source code, then py2exe is probably a simpler alternative. If your python is supposed to be integrated into the user's python install, then it's probably simpler to just create a zip of your .py files and add a one-line .py stub that imports the zipped package(s) using zipfile
if it makes you feel better, PYC doesn't provide much extra security and it doesn't really boost perf much either :)

If you haven't read PEP 3147, that will probably answer your questions.
I don't mean the solution described in that PEP and implemented as of Python 3.2. That's great if your "multiple Python versions" just means "3.2, 3.3, and probably future 3.x". Or even if it means "2.6+ and 3.1+, but I only really care about 3.2 and 3.3, so if I don't get the pyc speedups for other ones that's OK".
But when I asked your supported versions, you said, "2.7", which means you can't rely on PEP 3147 to solve your problems.
Fortunately, the PEP is full of discussion of earlier attempts to solve the problem, and the pitfalls of each, and there should be more than enough there to figure out what the options are and how to implement them.
The one problem is that the PEP is very linux-centric—mainly because it's primarily linux distros that tried to solve the problem in the past. (Apple also did so, but their solution was (a) pretty much working, and (b) tightly coupled with the whole Mac-specific "framework" thing, so they were mostly ignored…)
So, it largely leaves open the question of "Where should I put the .pyc files on Windows?"
The best choice is probably an app-specific directory under the user's local application data directory. See Known Folders if you can require Vista or later, CSIDL if you can't. Either way, you're looking for the FOLDERID_LocalAppData or CSIDL_LOCAL_APPDATA, which is:
The file system directory that serves as a data repository for local (nonroaming) applications. A typical path is C:\Documents and Settings\username\Local Settings\Application Data.
The point is that it's a place for applications to store data that's separate for each user (and inside that user's profile directory), and also for each machine the user's roaming profile might end up on, which means you can safely put stuff there and know that the user has the permissions to write there without UAC getting involved, and also know (as well as you ever can) that no other user or machine will interfere with what's there.
Within that directory, you create a directory for your program, and put whatever you want there, and as long as you picked a unique name (e.g., My Unique App Name or My Company Name\My App Name or a UUID), you're safe from accidental collision with other programs. (There used to be specific guidelines on this in MSDN, but I can no longer find them.)
So, how do you get to that directory?
The easiest way is to just use the env variable %LOCALAPPDATA%. If you need to deal with older Windows, you can use %USERPROFILE% and tack \Local Settings\Application Data onto the end, which is guaranteed to either be the same, or end up in the same place via junctions.
You can also use pywin32 or ctypes to access the native Windows APIs (since there are at least 3 different APIs for this and at least two ways to access those APIs, I don't want to give all possible ways to write this… but a quick google or SO search for "pywin32 SHGetFolderPath" or "ctypes SHGetKnownFolderPath" or whatever should give you what you need).
Or, there are multiple third-party modules to handle this. The first one both Google and PyPI turned up was winshell.

Re-reading the original question, there's a much simpler answer that probably fits your requirements.
I don't know much about Inno, but most installers give you a way to run an arbitrary command as a post-copy step.
So, you can just use python -m compileall to create the .pyc files for you at install time—while you've still got elevated privileges, so there's no problem with UAC.
In fact, if you look at pywin32, and various other Python packages that come as installer packages, they do exactly this. This is an idiomatic thing to do for installing libraries into the user's Python installation, so I don't see why it wouldn't be considered reasonable for installing an executable that uses the user's Python installation.
Of course if the user later decides to uninstall Python 2.6 and install 2.7, your .pyc files will be hosed… but from your description, it sounds like your entire program will be hosed anyway, and the recommended solution for the user would probably be to uninstall and reinstall anyway, right?

How to create a Mac OS X app with Python?

I want to create a GUI application which should work on Windows and Mac. For this I've chosen Python.
The problem is on Mac OS X.
There are 2 tools to generate an ".app" for Mac: py2app and pyinstaller.
py2app is pretty good, but it adds the source code in the package. I
don't want to share the code with the final users.
Pyinstaller generates UNIX executable, so how to run it on Mac? I
created a bundles with this executable, but the resulted ".app" is
not working.
The questions are:
How to configure py2app to include the source code in the
executable, so the final users will not have access to my program?
How to convert UNIX executable to Mac ".app" ?
Is there a way to compile Python code with GCC ?
In Windows it's easy, I created an "exe" file from Python code and
it works. Is it possible to create a single file "app" for Mac ?
P.S. I use two computers (Windows and for Mac), Python 2.7, wxPython, py2exe, py2app and pyinstaller.
Also, I have checked out these sites:
http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html
http://www.pyinstaller.org/export/develop/project/doc/Manual.html?format=raw
http://www.pyinstaller.org/wiki/Features/MacOsCompatibility
http://www.stackoverflow.com/questions/2933/an-executable-python-app

How to configure py2app to include the source code in the executable,
so the final users will not have access to my program?
Unless you very seriously hack the python interpreter (and include the mangled version) there is no really good way to hide the source from a moderately skilled and determined user. I strongly believe this is true on Windows also. Basically, whether you include true source or bytecode, a pretty clean version of the source can be recovered. More importantly, in my opinion, unless you include the actual source code (as opposed to bytecode, you will introduce a possible dependency on the interpreter version).
How to convert UNIX executable to Mac ".app" ?
What do you mean by a UNIX executable? A Darwin (OS X) binary [which isn't actually UNIX]? That can be done using the kinds of tools you already mentioned, but it must be done carefully to avoid library dependencies.
If all you want it a simple wrapper to put a command-line binary into a window, it's pretty easy to accomplish and the free XCode suite has several examples that would serve (depending on what output
you wan to deliver, if any).
Is there a way to compile Python code with GCC ?
GCC does not compile Python. It's a different language (although there tools in the gcc family rthat support multiple language front-ends, but not Python). There are tools that attempt to translate Python into C, and then you can compile that into a true binary, but this only works for programs that avoid certain types of construct, and the process (and restrictions) need to apply your libraries as well.
One project to allow this is Cython. It works well for some types
of code, mostly numerical code, but it is not trivial to install and
exploit, very especially if you want to produce something that runs on multiple
different computers.
In Windows it's easy, I created an "exe" file from Python code and it
works. Is it possible to create a single file "app" for Mac ?
I would have to say I am skeptical -- very skeptical -- about this. Just like the OS X case, the exe almost certainly has the source code trivially accessible within it.
One fairly easy trick is to encrypt the source code and then decrypt it on the fly, but this
seems to me like more trouble than it's worth.

PyInstaller will automatically create bundles under Mac OSX for windowed executables. When running ypinstaller.py, make sure to pass the option "--windowed".
This feature is documented in the website of pyinstaller

If you're not completely committed to wxPython (and for anyone else looking for a cross platform Python GUI framework), I recommend you check out Kivy. It's cross platform, GPU accelerated, and it will do the app packaging for you. It's easy to jump into, has a well thought-out architecture, and gives you an incredible amount of flexibility in terms of the interface. It's the best way I've found to make a cross platform Python GUI app.

cxFreeze was the choice.
I use it pack my python program to a Mac OS X app. Which works like a charm.

Automator was already mentioned as a quick and simple solution for Pythons scripts that are contained in a single file, but since the Automator UI has so many options, and it is not obvious how to actually do it, I'll provide step-by-step instructions (verified to work on Yosemite):
In Automator select File > New and pick Application as document type.
Next, make sure Actions tab is selected on the left, and then in the search box type run. Among other options you'll see Run Shell Script — doubleclick it, and an editor window will appear in the right panel.
From the Shell dropdown menu select /usr/bin/python.
Paste your Python code into the edit window and then pick File > Save.
By default, the app will be saved under $HOME/Applications and will appear in Spotlight.
If you want to be able to set your own icon and have some fancy features, like task bar icons with a menu, log windows etc, then have a look at Platypus — an open-source app for creating MacOS native bundles.

2: You can't "convert" it, but you can move the executable to App.app/Contents/MacOS/something in a .app file, with CFBundleExecutable set to "something". This would not generally be recommended.

A motivated person could probably reconstruct usable source code from the Python bytecode in your app, so you might reconsider your opposition to py2app. If you don't trust your final users, why are you doing business with them?

Having used py2exe for windows users so they wouldn't have to deal with library versions, I've torn apart the compiled programs, they include the python bytecode files. While you can make it a violation of the license to look inside those, the fact is that if a computer can execute them, I can read them. It is possible to compile python programs with gcc, via a C preprocessor (try looking for 2c.py on google), I don't know if any of them support GCC. Again, you don't gain any security through using them, but you can get a significant speed improvement.

I haven't tried it with big Python projects, but for my own scripts, the easiest way I found was to use Automator
You can interactively create an app project with Run Shell Script action, then paste in your script in its editor, select your shell program (/usr/bin/python), finally save the project. And you have yourself a Mac native app.
Automator can also be driven by AppleScript. So you can pipeline this py-2-app conversion process to your build scripts.
I've never tested a GUI program with it so I don't know if you'll be happy with it. But I'd give it a try since you may wonder how well all the cited 3rd-party python modules/applications are maintained, and how long they are gonna last. Coming bundled with OS X, Automator will likely stay, unless Apple got REALLY tired of it.

cxFreeze is best solution available, first create your program or application using python and than make setup file for your application, and than build the app using build command python setup.py build, according to your requirement you need to make some changes.

The only way is py2app. You have no other way. Sorry.
The research you did seems very solid and you did not miss anything.

Trimming Python Runtime

We've got a (Windows) application, with which we distribute an entire Python installation (including several 3rd-party modules that we use), so we have consistency and so we don't need to install everything separately. This works pretty well, but the application is pretty huge.
Obviously, we don't use everything available in the runtime. I'd like to trim down the runtime to only include what we really need.
I plan on trying out py2exe, but I'd like to try and find another solution that will just help me remove the unneeded parts of the Python runtime.

One trick I've learned while trimming down .py files to ship: Delete all the .pyc files in the standard library, then run your application throughly (that is, enough to be sure all the Python modules it needs will be loaded). If you examine the standard library directories, there will be .pyc files for all the modules that were actually used. .py files without .pyc are ones that you don't need.

Both py2exe and pyinstaller (NOTE: for the latter use the SVN version, the released one is VERY long in the tooth;-) do their "trimming" via modulefinder, the standard library module for finding all modules used by a given Python script; you can of course use the latter yourself to identify all needed modules, if you don't trust pyinstaller or py2exe to do it properly and automatically on your behalf.

This py2exe page on compression suggests using UPX to compress any DLLs or .pyd files (which are actually just DLLs, still). Obviously this doesn't help in trimming out unneeded modules, but it can/will trim down the size of your distribution, if that's a large concern.