We Keep Coding

What is `build` in the context of python?

I'm currently learning about Python distribution packages, and reading through this article, and it says:
pyproject.toml tells build tools (like pip and build) what is required to build your project. This tutorial uses setuptools, so open pyproject.toml and enter the following content:
It mentions a few times more the concept of building Python packages.
As far as I know, at least when talking about pure Python code, Python distributions (sdists and wheels) only contain .py source files. So what is meant by the author when speaking about building?

build may have two meaning.
In your description building means to put all project files into one file with extension .whl and/or .gz - so later users (using pip) will download all as a single .whl (or .gz).
So we could say that this building can mean packing.
Second build (but not in your description) can be when people install package. Some packages may have C/C++ code (ie. numpy) and after downloading it needs to compile it (for your CPU).
So we could say that this building can means compiling.

Create binary python package that depends on system-wide installed libssl.so

I am trying to create a binary python package that can be installed without compilation. The python package consists only one extension module written in C using the Python API. The extension module depending on the stable python ABI by using Py_LIMITED_API with 0x03060000 (3.6). Up to my knowledge, this means the extension module can work for all CPython versions that are not older than 3.6. I managed to create the sdist package, and I explored dumb, egg and wheel formats. I managed to create the binary packages, but none of them is perfect for my use case.
The problem is the extension module is depending on libssl.so, and this is the only "external" dependency of it. Because the python package itself is very stable, it doesn't require frequent releases. Therefore I wouldn't like to include libssl.so and take the burden of releasing new versions because of the security updates of OpenSSL (not mentioning to educate the users to update their python package regularly). I think because of this, the wheel format is not suitable, because the linux_*.whl packages cannot be uploaded to PyPI, but the manylinux2014_*.whl tag has to include libssl.so (and its dependencies) in the package. The dumb package are not suitable for PyPI based distribution, the egg format is not supported by pip, so they are also not suitable.
Because of the stable ABI and the widespread of libssl.so, I think it should be possible to release a single binary package for the most linux distributions and multiple python versions (similarly to the manylinux tags with wheel). Of course the pacakge would require libssl to be installed on the machine, but that is something I can accept for achieving better security. And that's where I am stuck.
My question is how can I create a binary python package, which
contains a python extension module,
depends on the system-wide installed libssl.so,
and can uploaded to and installed via pip and PyPI?
I tried to explore other possibilities, but I couldn't find anything else, so if you have any tips for other formats to look after, I would appreciate that also.

How to structure and distribute Pybind11 extension with stubs?

I'm trying to create and distribute (with pip) a Python package that has Python code, and C++ code compiled to a .pyd file with Pybind11 (using Visual Studio 2019). I also want to include .pyi stub files, for VScode and other editors. I can't find much documentation on doing this correctly.
I'd like to be able to just install the package via pip as normal, and write from mymodule.mysubmodule import myfunc etc like a normal Python package, including autocompletes, type annotations, VScode intellisense etc using the .pyi files I'd write.
My C++ code is in multiple cpp and header files. It uses a few standard libraries, and a few external libraries (such as boost). It defines a single module, and 2 submodules. I want to be able to distribute this on Windows and Linux, and for x86 and x64. I am currently targeting Python 3.9, and the c++17 standard.
How should I structure and distribute this package? Do I include the c++ source files, and create a setup.py similar to the Pybind11 example? If so, how do I include the external libraries? And how do I structure the .pyi stub files? Does this mean whoever tries to install my package would need a c++ compiler as well?
Or, should I compile my c++ to a .pyd/.so file for each platform and architecture? If so, is there a way to specify which one gets installed through pip? And again, how would I structure the .pyi stubs?

Generating .pyi stubs
The pybind11 issue mentions a couple of tools (1, 2) to generate stubs for binary modules. There could be more, but I'm not aware of others. Unfortunately both are far from being perfect, so you probably still need to check and adjust the generated stubs manually.
Distribution of .pyi stubs
After correction of stubs you just include those .pyi files in you distribution (e.g. in wheel or as sources) along with py.typed indication file or, alternatively, distribute them separately as standalone package (e.g. mypackage-stubs).
Building wheels
Wheels allows users of your library to install it in binary form, i.e. without compilation. Wheels makes use of older compilers in order to be compatible with greater number of systems/platforms, so you might face some troubles with a C++17 library. (C++11 is old enough and should have no problems with wheels).
Building wheels for various platforms is tedious, the pybind11's python_example uses cibuildwheels package to do that, I would recommend this route if you are already using CI.
If wheels are missing for target platform the pip will attempt to install from source. This would require compiler and 3rd party libraries you are using to be already installed.
Maybe conda?
If your setup is complex and requires a number of 3rd party libraries it might be worth to write a conda recipe and use conda-forge to generate binary versions of the package. Conda is superior to pip, since it can manage non-python dependencies as well.

Python wheel packages Linux vs windows

Can one use the same python package (wheel file) for Linux , windows etc.? I am asking this as some packages include not only python files but EXEs as well, which I assume are python code turned into exe (at least with pip.exe and Django admin tool). Exe files are platform specific in the same way there are separate python interpreters for windows and Linux so that arises a question.

Some wheel packages are cross-platform; some are platform-specific.
This information is included in the wheel's name. For example:
pytz-2018.4-py2.py3-none-any.whl (510kB)
That py2.py3 means that it works in any Python implementation, both Python 2.x and 3.x, and that none-any means that it works on any platform.
This one is more specific:
numpy-1.14.3-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
That cp36-cp36m means that it works only in CPython 3.6, and that macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64 means that it's built for x86_64 macOS versions 10.9-10.10. (Roughly speaking, that's the minimum and recommended versions of macOS; most other platforms aren't quite as complicated.)
The most common reason for a package to be platform-specific is that it includes C API extension modules, as is the case with numpy. But there can be other reasons. For example, it may include a native executable to subprocess, or it may use ctypes to access system APIs, etc.

A Python wheel is a packaging format, NOT an execution format. It's basically a .zip file.
Furthermore:
https://packaging.python.org/discussions/wheel-vs-egg/
...when the distribution only contains Python files (i.e. no compiled
extensions), and is compatible with Python 2 and 3, it’s possible for
a wheel to be “universal”, similar to an sdist.
From the same link:
A single wheel archive can indicate its compatibility with a number of
Python language versions and implementations, ABIs, and system
architectures.
In other words, the "wheel" format is designed to be as portable as possible ... and it also allows you to include platform-specific contents as required.

What is the difference between an 'sdist' .tar.gz distribution and an python egg?

I am a bit confused. There seem to be two different kind of Python packages, source distributions (setup.py sdist) and egg distributions (setup.py bdist_egg).
Both seem to be just archives with the same data, the python source files. One difference is that pip, the most recommended package manager, is not able to install eggs.
What is the difference between the two and what is 'the' way to do distribute my packages?
(Note, I am not wanting to distribute my packages through PyPI, but I want to use a package manager that fetches my dependencies from PyPI)

setup.py sdist creates a source distribution: it contains setup.py, the source files of your module/script (.py files or .c/.cpp for binary modules), your data files, etc. The result is an archive that can then be used to recompile everything on any platform.
setup.py bdist (and bdist_*) creates a built distribution: it includes .pyc files, .so/.dll/.dylib for binary modules, .exe if using py2exe on Windows, your data files... but no setup.py. The result is an archive that is specific to a platform (for example linux-x86_64) and to a version of Python, and that can be installed simply by extracting it into the root of your filesystem (executables are in /usr/bin (or equivalent), data files in /usr/share, modules in /usr/lib/pythonX.X/site-packages/...). You can even build rpm archives that can be directly installed using your package manager.

2021 update: the tools to build and use eggs no longer exist in Python.
There are many more than two different kind of Python (distribution) packages. This command lists many subcommands:
$ python setup.py --help-commands
Notice the various different bdist types.
An egg was a new package type, introduced by setuptools but later adopted by the standard library. It is meant to be installed monolithic onto sys.path. This differs from an sdist package which is meant to have setup.py install run, copying each file into place and perhaps taking other actions as well (building extension modules, running additional arbitrary Python code included in the package).
eggs are largely obsolete at this point in time. EDIT: eggs are gone, they were used with the command "easy_install" that's been removed from Python.
The favored packaging format now is the "wheel" format, notably used by "pip install".
Whether you create an sdist or an egg (or wheel) is independent of whether you'll be able to declare what dependencies the package has (to be downloaded automatically at installation time by PyPI). All that's necessary for this dependency feature to work is for you to declare the dependencies using the extra APIs provided by distribute (the successor of setuptools) or distutils2 (the successor of distutils - otherwise known as packaging in the current development version of Python 3.x).
https://packaging.python.org/ is a good resource for further information about packaging. It covers some of the specifics of declaring dependencies (eg install_requires but not extras_require afaict).