We Keep Coding

setup.py - how to set a sub-folder as the main package directory?

My project structure:
/myproject/ <- I would like to skip that folder
/mypackage/
/subpackage/
mymodule.py
run.py
- setup.py
Within run.py I'd like to import from mymodule.py like that:
from mypackage.subpackage.mymodule import something
instead of:
from myproject.mypackage.subpackage.mymodule import something
Is there a way in setup() to define the entrypoint as being mypackage and skip myproject?

You can use what's called a src-layout (since src/ is more typically used as a top-level directory for packages. See https://setuptools.readthedocs.io/en/latest/setuptools.html#using-a-src-layout
If using setup.cfg you can write this like:
[options]
package_dir=
=src
packages=find:
[options.packages.find]
where=src
or equivalently, using an old-style setup.py:
from setuptools import find_packages
setup(
...
package_dir={'': 'src'}
packages=find_packages(where='src')
...
)
packages

Building Python Package with source in different directory

When building a python package where the source tree looks like this:
src -\
+- module -\
<stuff>
+- setup.py
is pretty clear.
Is it possible to build a package where module source doesn't reside in the same location as the setup.py? For more specific use case the code for the module is either partially or full autogenerated in a location other then src
E.g.
src -\
+- setup.py
generated -\
module -\
<module code>

You can control the directory where packages reside by using the package_dir argument to setup(...)
and while it does appear to build a proper source distribution when package_dir is a relative path starting with .., it appears that pip will refuse to install it -- I'd suggest instead nesting your generated code inside that src directory instead and then using package_dir to select that.
Here's an example which moves all modules inside a generated subdir:
setup(
name='mypkg',
package_dir={'': 'generated'},
packages=find_packages('generated'),
)
Using a setup like:
$ tree .
.
├── generated
│   ├── mod1
│   │   └── __init__.py
│   └── mod2
│   └── __init__.py
└── setup.py
This would make the following succeed after install: import mod1; import mod2
If you wanted to make those modules available under a different prefix, you would do:
setup(
name='mypkg',
package_dir={'hello': 'generated'},
packages=[f'hello.{mod}' for mod in find_packages('generated')],
)
This would make import hello.mod1; import hello.mod2 succeed after installation

You can use relative paths in package lookup configuration. Examples:
all distributed sources are in generated
from setuptools import setup, find_packages
setup(
...
package_dir={'': '../generated'},
packages=find_packages(where='../generated'),
)
selected packages should be included from generated
In this example, only packages spam and eggs from generated will be included:
import pathlib
from setuptools import setup, find_packages
setup(
name='so',
package_dir={'spam': '../generated/spam', 'eggs': '../generated/eggs'},
packages=find_packages(where='../generated'), # or just ['spam', 'eggs']
)
Or implement a dynamic lookup like e.g.
package_dir={p.name: p.resolve() for p in pathlib.Path('..', 'generated').iterdir()}
better implementation by resolving all paths relative to the setup.py file
Resolving all paths relative to the setup.py script allows you to run the script from any other directory than src, e.g. you can run python src/setup.py bdist_wheel etc. You may or may not need it, depending on your use case. Nevertheless, the recipe is as usual: resolve all paths to __file__, e.g.
import pathlib
from setuptools import setup, find_packages
src_base = pathlib.Path(__file__, '..', '..', 'generated').resolve()
setup(
...
package_dir={'': str(src_base)},
packages=find_packages(where=src_base),
)

Add a virtual namespace for a Python package

I have a python package with the following directory structure:
my_package/
my_subpackage/
__init__.py
my_module.py
__init__.py
setup.py
When I generate the Python wheel and pip install it, I am not going to have the my_package namespace, so my_subpackage is going to be part of the global namespace of my virtualenv or whatever.
I know that the solution here is to create another directory called my_package and put everything inside it:
my_package/
my_package/
my_subpackage/
__init__.py
my_module.py
__init__.py
setup.py
But let's say that I cannot change the directory structure for some reason. Is there a way to add a virtual my_package namespace in setup.py with the first layout?

You can specify the package_dir option in your setup.py.
Something like this should work:
setup(
...
package_dir={'': '..'},
packages=['my_package']
...
)
I never used relative paths with package_dir though, so your confirmation would be appreciated.

Putting top-level modules in a "lib" directory

Context
I have a project set up like this
setup.py
MANIFEST.in
lib/
foo.py
bar.py
magic/
__init__.py
alchemy.py
submagic/
__init__.py
wizard.py
with this setup.py file:
from setuptools import setup, find_packages
setup(
...
py_modules=['foo', 'bar'],
packages=find_packages(exclude=['lib', 'lib.*']),
package_dir={'foo': 'lib', 'bar': 'lib'}
)
and MANIFEST.in
graft magic
graft lib
Problem
When I use pip install, I end up with the modules lib.foo and lib.bar instead of top-level modules foo and bar.
How should I change my setup to make this work the way I want? If possible, I would prefer not to move foo.py and bar.py out of lib into the top level of the project.

How do I distribute all contents of root directory to a directory with that name

I have a project named myproj structured like
/myproj
__init__.py
module1.py
module2.py
setup.py
my setup.py looks like this
from distutils.core import setup
setup(name='myproj',
version='0.1',
description='Does projecty stuff',
author='Me',
author_email='me#domain.com',
packages=[''])
But this places module1.py and module2.py in the install directory.
How do I specify setup such that the directory /myproj and all of it's contents are dropped into the install directory?

In your myproj root directory for this project, you want to move module1.py and module2.py into a directory named myproj under that, and if you wish to maintain Python < 3.3 compatibility, add a __init__.py into there.
├── myproj
│   ├── __init__.py
│   ├── module1.py
│   └── module2.py
└── setup.py
You may also consider using setuptools instead of just distutils. setuptools provide a lot more helper methods and additional attributes that make setting up this file a lot easier. This is the bare minimum setup.py I would construct for the above project:
from setuptools import setup, find_packages
setup(name='myproj',
version='0.1',
description="My project",
author='me',
author_email='me#example.com',
packages=find_packages(),
)
Running the installation you should see lines like this:
copying build/lib.linux-x86_64-2.7/myproj/__init__.py -> build/bdist.linux-x86_64/egg/myproj
copying build/lib.linux-x86_64-2.7/myproj/module1.py -> build/bdist.linux-x86_64/egg/myproj
copying build/lib.linux-x86_64-2.7/myproj/module2.py -> build/bdist.linux-x86_64/egg/myproj
This signifies that the setup script has picked up the required source files. Run the python interpreter (preferably outside this project directory) to ensure that those modules can be imported (not due to relative import).
On the other hand, if you wish to provide those modules at the root level, you definitely need to declare py_modules explicitly.
Finally, the Python Packaging User Guide is a good resource for more specific questions anyone may have about building distributable python packages.