We Keep Coding

What are the bare minimum files/libraries required by Python to run?

I am new to Python having come from a proprietary compiled language (Xojo) that produces self-contained executables.
I understand that Python is an interpreted language. I understand that it requires an interpreter (let’s stick with CPython) and presumably it requires a number of accessory frameworks/C libraries in order to run. What I don’t understand is why is it so hard to create a folder containing the interpreter and all required files and libraries and simply bundle these up with my script to distribute.
I have discovered that there are a bunch of tools that attempt to do this (py2app, cx_freeze, etc) but many of them seem either broken, not maintained or really buggy.
I guess my question is: is there any documentation that describes the exact things I need to bundle with a “Hello World” script to get it running? This seems to be a really straightforward problem to solve but it hasn’t been (which suggests that it is far more complex than I appreciate).

My understanding is that PyInstaller works fine for making a single exe for distribution. But barring packaging tools like that, in general, there isn't an obvious "bare minimum"; the modules don't have documented dependencies, so it's usually best to ship the whole standard library.
Typically, if you need a redistributable version, you use the embedded Python zip redistributable, shipping Python alongside your main application.

The exact list of files/libraries depends on how the python interpreter is built. In windows for example, you can obtain CPython binaries built from Visual Studio, Cygwin and Mingw-w64. They have different dependency of cause. In Linux distributions, python is normally installed by default.
Below is the list of .dll and .exe files that you can find in the official CPython release for windows.
libcrypto-1_1-x64.dll python.exe python37.dll sqlite3.dll
libssl-1_1-x64.dll pythonw.exe python3.dll vcruntime140.dll
The total size of this ZIP file release is only 6.7 MB. So it would be easy to bundle it in your main executable. You can use whatever bundler at hand, not necessary those designed for python. Quoting from the documentation here:
extracting the embedded distribution to a subdirectory of the application installation is sufficient to provide a loadable Python interpreter.

I feel the absolute best way to experience Python for beginners in thonny and an esp32.

A very good way to get started with python is to use Anaconda https://www.anaconda.com/distribution/#download-section - this distribution contains the CPython interpreter and the most commonly used packages. For quite a while you will get along without installing more packages.
To be able to make a simple distributable piece of code just include a requirements.txt along with your code which should list down the packages (and versions) you are using in your code.
More on that here : https://www.idiotinside.com/2015/05/10/python-auto-generate-requirements-txt/
pip freeze generates a superset of all packages in your running environment so you would ideally go with the second smarter option in the link : pipreqs
So, in short along with your code just an additional requirements.txt should be fine using which people can install all required packages as
pip install -r requirements.txt
and they are good to go to run your code.
For advanced scenarios you might want to look up creating virtual environments using conda.
What is a conda environment?
https://docs.conda.io/projects/conda/en/latest/user-guide/concepts.html#conda-environments
How to create/manage a conda environment
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
All the best in your Python journey!

How to include python with script package? [duplicate]

I need to package my Python application, its dependencies, and Python itself into a single MSI installer for distribution to users. The end result should desirably be:
Python is installed in the standard location
the package and its dependencies are installed in a separate directory (possibly site-packages)
the installation directory should contain the Python uncompressed and a standalone executable is not required

Kind of a dup of this question about how to make a python into an executable.
It boils down to:
py2exe on windows, Freeze on Linux, and
py2app on Mac.

I use PyInstaller (the svn version) to create a stand-alone version of my program that includes Python and all the dependencies. It takes a little fiddling to get it to work right and include everything (as does py2exe and other similar programs, see this question), but then it works very well.
You then need to create an installer. NSIS Works great for that and is free, but it creates .exe files not .msi. If .msi is not necessary, I highly recommend it. Otherwise check out the answers to this question for other options.

My company uses the free InnoSetup tool. It is a moderately complex program that has tons of flexibility for building installers for windows. I believe that it creates .exe and not .msi files, however. InnoSetup is not python specific but we have created an installer for one of our products that installs python along with dependencies to locations specified by the user at install time.

I've had much better results with dependencies and custom folder structures using pyinstaller, and it lets you find and specify hidden imports and hooks for larger dependencies like numpy and scipy. Also a PITA, though.

py2exe will make windows executables with python bundled in.

py2exe is the best way to do this. It's a bit of a PITA to use, but the end result works very well.

Ok, I have used py2exe before and it works perfectly except for one thing... It only works on executable windows machines. I then learned about Jython which turn a python script into a .Jar file. Which as you know is executable from any machine that has Java ("To your latest running version") installed. Which is great because both unix, windows, and ios (Most of the time) Run java. That means its executable from all of the following machines. As long as they run Java. No need for "py2mac + py2exe + freeze" just to run on all operating systems. Just Jython
For more information on how it works and how you can use it click here.
http://www.jython.org/

xcopy python deployment

I'm new to python and I'm writing my first program. I would like after I finish to be able to run the program from the source code on a windows or mac machine. My program has dependencies on 3rd party modules.
I read about virtualenv but I don't think it helps me because it says it's not relocatable and it's not cross-platform (see Making Environments Relocatable http://pypi.python.org/pypi/virtualenv).
The best scenario is to install the 3rd party modules locally in my project, aka xcopy installation.
I will be really surprised if python doesn't support this easily especially since it promotes simplicity and frictionless programming.

You can do what you want, you just have to make sure that the directory containing your third-party modules is on the python path.
There's no requirement to install modules system-wide.

Note, while packaging your whole app with py2exe may not be an option, you can use it to make a simple launcher environment. You make a script with imports your module/package/whatever and launches the main() entry-point. Package this with py2exe but keep your application code outside this, as python code or an egg. I do something similar where I read a .pth text file to learn what paths to add to the sys.path in order to import my application code.

Simply, that's generally not how python works. Modules are installed site-wide and used that way. Are you familiar with pip and/or easy_install? Those + pypi let you automatically install dependencies no matter what you need.
If you want to create a standalone executable typically you'd use py2exe, py2app or something like that. Then you would have no dependencies on python at all.

I also found about zc.buildout that can be used to include dependencies in an automatic way.

Can a python webapp be bundled into a single file for deployment?

Is it possible for a python webapp to be bundled (gzipped?) into a single file, along with any required resources (js/css files) & modules (including modules like lxml which are mostly C-based), for easy deployment onto a linux webserver?

Yes. You can create a windows executable using py2exe. A better way to manage deployment is to package your app with a setup.py file, listing all needed dependencies, and listing non-python support files in the MANIFEST.in file. Then you can package it into a bundle using setup.py sdist, and install it with pip. See Tarek Ziade's Hitchhiker's Guide to Packaging for more details.

For standalones ,you could try :
OS X : py2app
For Windows py2exe.
Not sure if this would apply to your 'webapp' though.

PyInstaller is perhaps the best cross-platform packager (in the SVN trunk version: 1.3, the latest packaged release, is very old and crufty -- no idea why my friends who took over maintenance over a year ago and did so many enhancements keep resisting my entreaties to make another release, already!-) -- you can package for Windows, Linux or Mac, and it directly supports many popular third-party packages too.

How to use Python distutils?

I wrote a quick program in python to add a gtk GUI to a cli program. I was wondering how I can create an installer using distutils. Since it's just a GUI frontend for a command line app it only works in *nix anyway so I'm not worried about it being cross platform.
my main goal is to create a .deb package for debian/ubuntu users, but I don't understand make/configure files. I've primarily been a web developer up until now.
edit: Does anyone know of a project that uses distutils so I could see it in action and, you know, actually try building it?
Here are a few useful links
Ubuntu Python Packaging Guide
This Guide is very helpful. I don't know how I missed it during my initial wave of gooling. It even walks you through packaging up an existing python application
The Ubuntu MOTU Project
This is the official package maintaining project at ubuntu. Anyone can join, and there are lots of tutorials and info about creating packages, of all types, which include the above 'python packaging guide'.
"Python distutils to deb?" - Ars Technica Forum discussion
According to this conversation, you can't just use distutils. It doesn't follow the debian packaging format (or something like that). I guess that's why you need dh_make as seen in the Ubuntu Packaging guide
"A bdist_deb command for distutils
This one has some interesting discussion (it's also how I found the ubuntu guide) about concatenating a zip-file and a shell script to create some kind of universal executable (anything with python and bash that is). weird. Let me know if anyone finds more info on this practice because I've never heard of it.
Description of the deb format and how distutils fit in - python mailing list

See the distutils simple example. That's basically what it is like, except real install scripts usually contain a bit more information. I have not seen any that are fundamentally more complicated, though. In essence, you just give it a list of what needs to be installed. Sometimes you need to give it some mapping dicts since the source and installed trees might not be the same.
Here is a real-life (anonymized) example:
#!/usr/bin/python
from distutils.core import setup
setup (name = 'Initech Package 3',
description = "Services and libraries ABC, DEF",
author = "That Guy, Initech Ltd",
author_email = "that.guy#initech.com",
version = '1.0.5',
package_dir = {'Package3' : 'site-packages/Package3'},
packages = ['Package3', 'Package3.Queries'],
data_files = [
('/etc/Package3', ['etc/Package3/ExternalResources.conf'])
])

apt-get install python-stdeb
Python to Debian source package conversion utility
This package provides some tools to produce Debian packages from Python packages via a new distutils command, sdist_dsc. Automatic defaults are provided for the Debian package, but many aspects of the resulting package can be customized via a configuration file.
pypi-install will query the Python Package Index (PyPI) for a
package, download it, create a .deb from it, and then install
the .deb.
py2dsc will convert a distutils-built source tarball into a Debian
source package.

Most Python programs will use distutils. Django is a one - see http://code.djangoproject.com/svn/django/trunk/setup.py
You should also read the documentation, as it's very comprehensive and has some good examples.

I found the following tutorial to be very helpful. It's shorter than the distutils documentation and explains how to setup a typical project step by step.

distutils really isn't all that difficult once you get the hang of it. It's really just a matter of putting in some meta-information (program name, author, version, etc) and then selecting what files you want to include. For example, here's a sample distutils setup.py module from a decently complex python library:
Kamaelia setup.py
Note that this doesn't deal with any data files or or whatnot, so YMMV.
On another note, I agree that the distutils documentation is probably some of python's worst documentation. It is extremely inclusive in some areas, but neglects some really important information in others.