Context
I have a project set up like this
setup.py
MANIFEST.in
lib/
foo.py
bar.py
magic/
__init__.py
alchemy.py
submagic/
__init__.py
wizard.py
with this setup.py file:
from setuptools import setup, find_packages
setup(
...
py_modules=['foo', 'bar'],
packages=find_packages(exclude=['lib', 'lib.*']),
package_dir={'foo': 'lib', 'bar': 'lib'}
)
and MANIFEST.in
graft magic
graft lib
Problem
When I use pip install, I end up with the modules lib.foo and lib.bar instead of top-level modules foo and bar.
How should I change my setup to make this work the way I want? If possible, I would prefer not to move foo.py and bar.py out of lib into the top level of the project.
Related
I have a Cython extension module for a Python project I want to install to the same namespace as the project on installation. When I try to specify in the extension to install it inside the package itself, it can't be found and imported. If I specify the extension goes in the root of the Python namespace, it works fine, but it's not in the module namespace I want. How do I get the Extension Module to be importable from the same namespace as the package itself?
I've made a simple test case for this question.
Folder Structure:
mypkg
├── foo
│ ├── __init__.py
│ └── barCy.pyx
└── setup.py
The .barCy.pyx file:
cpdef long bar():
return 0
The setup.py code:
import distutils.extension
from Cython.Build import cythonize
from setuptools import setup, find_packages
extensions = [distutils.extension.Extension("foo.bar",
['foo/barCy.pyx'])]
setup(
name='foo',
packages=find_packages(),
ext_modules=cythonize(extensions),
)
The __init.py__ is empty.
I want to be able to do:
>>> import foo.bar as bar
>>> bar.bar()
0
Instead, I get
>>> import foo.bar as bar
ModuleNotFoundError: No module named 'foo.bar'
If I were to change the Extension("foo.bar",... to Extension("bar",..., then I can do import bar like it were a top level package. Although that is the expected behavior, it's not what I want as I only want this extension module to be accessible through the foo namespace.
My Python interpreter was being run from the same folder as the setup.py script, so doing import foo was importing the local package, rather than the one installed in Python interpreter's site-packages directory. Because they have the same folder structure, the local directory was chosen as the superseding imported package.
Do testing/running from a different folder than the source.
My project structure:
/myproject/ <- I would like to skip that folder
/mypackage/
/subpackage/
mymodule.py
run.py
- setup.py
Within run.py I'd like to import from mymodule.py like that:
from mypackage.subpackage.mymodule import something
instead of:
from myproject.mypackage.subpackage.mymodule import something
Is there a way in setup() to define the entrypoint as being mypackage and skip myproject?
You can use what's called a src-layout (since src/ is more typically used as a top-level directory for packages. See https://setuptools.readthedocs.io/en/latest/setuptools.html#using-a-src-layout
If using setup.cfg you can write this like:
[options]
package_dir=
=src
packages=find:
[options.packages.find]
where=src
or equivalently, using an old-style setup.py:
from setuptools import find_packages
setup(
...
package_dir={'': 'src'}
packages=find_packages(where='src')
...
)
packages
I have a python package with the following directory structure:
my_package/
my_subpackage/
__init__.py
my_module.py
__init__.py
setup.py
When I generate the Python wheel and pip install it, I am not going to have the my_package namespace, so my_subpackage is going to be part of the global namespace of my virtualenv or whatever.
I know that the solution here is to create another directory called my_package and put everything inside it:
my_package/
my_package/
my_subpackage/
__init__.py
my_module.py
__init__.py
setup.py
But let's say that I cannot change the directory structure for some reason. Is there a way to add a virtual my_package namespace in setup.py with the first layout?
You can specify the package_dir option in your setup.py.
Something like this should work:
setup(
...
package_dir={'': '..'},
packages=['my_package']
...
)
I never used relative paths with package_dir though, so your confirmation would be appreciated.
I have a project named myproj structured like
/myproj
__init__.py
module1.py
module2.py
setup.py
my setup.py looks like this
from distutils.core import setup
setup(name='myproj',
version='0.1',
description='Does projecty stuff',
author='Me',
author_email='me#domain.com',
packages=[''])
But this places module1.py and module2.py in the install directory.
How do I specify setup such that the directory /myproj and all of it's contents are dropped into the install directory?
In your myproj root directory for this project, you want to move module1.py and module2.py into a directory named myproj under that, and if you wish to maintain Python < 3.3 compatibility, add a __init__.py into there.
├── myproj
│ ├── __init__.py
│ ├── module1.py
│ └── module2.py
└── setup.py
You may also consider using setuptools instead of just distutils. setuptools provide a lot more helper methods and additional attributes that make setting up this file a lot easier. This is the bare minimum setup.py I would construct for the above project:
from setuptools import setup, find_packages
setup(name='myproj',
version='0.1',
description="My project",
author='me',
author_email='me#example.com',
packages=find_packages(),
)
Running the installation you should see lines like this:
copying build/lib.linux-x86_64-2.7/myproj/__init__.py -> build/bdist.linux-x86_64/egg/myproj
copying build/lib.linux-x86_64-2.7/myproj/module1.py -> build/bdist.linux-x86_64/egg/myproj
copying build/lib.linux-x86_64-2.7/myproj/module2.py -> build/bdist.linux-x86_64/egg/myproj
This signifies that the setup script has picked up the required source files. Run the python interpreter (preferably outside this project directory) to ensure that those modules can be imported (not due to relative import).
On the other hand, if you wish to provide those modules at the root level, you definitely need to declare py_modules explicitly.
Finally, the Python Packaging User Guide is a good resource for more specific questions anyone may have about building distributable python packages.
I have a package with a structure that would (simplified) look like:
mypackage/
__init__.py
setup.py
module1.py
module2.py
mysubpackage/
__init__.py
mysubmodule1.py
mysubmodule2.py
I'm using a configuration for setup.py like this:
from setuptools import setup, find_packages
setup(
name = "mypackage",
version = "0.1",
author = "Foo",
author_email = "foo#gmail.com",
description = ("My description"),
packages=find_packages(),
)
The default where argument for find_packages() is '.', but it doesn't include my top-level modules (module1.py nor module2.py). However, all child submodules and subpackages are added when running python setup.py build.
How could I get top-level Python modules added too, without moving setup.py one level higher?
Thank you all for your responses.
Finally, I added a directory (not Python package) containing mypackage and the setup.py module. The structure now looks as follows:
myapp/
setup.py
mypackage/
__init__.py
module1.py
module2.py
mysubpackage/
__init__.py
mysubmodule1.py
mysubmodule2.py
Now using find_packages() works as expected. Thanks!