We Keep Coding

Setup.py duplicates sub packages as separate one

I have a following project structure:
.
└── Project/
├── package_1/
│ ├── package_2
│ ├── __init__.py
│ ├── file_1.py
│ ├── file_2.py
│ └── file_3.py
└── __init__.py
As the package_2 contains the ported files from another project, I want to be able to install package_1 using setuptools so that it doesn't conflict with the original project and import it like this :
import package_1.package_2
here is my setup.py file content :
setup(
name="Project",
...
packages=find_packages(exclude=["package_1"]),
)
So far everything works perfectly except that in \Lib\site-packages directory in addition to
package_1/package_2 I also have the package_2 as a separate module which I think it is not OK.

Is there a way to include extra files when building a PEX (i.e. with MANIFEST.in)?

I have a directory structure similar to the following:
├── myproj
│ ├── utils.py
│ ├── __init__.py
│ ├── routes
│ │ ├── __init__.py
│ │ ├── auth.py
│ │ └── stuff.py
├── html
│ ├── index.html
│ └── about.html
├── MANIFEST.in
├── setup.cfg
└── setup.py
The contents of MANIFEST.in are:
graft html
The following post alludes to being able to use MANIFEST.in with PEX (Python PEX: Pack a package with its sub-packages) but when I run either pex . -o myproject or python setup.py bdist_pex the html/ directory is not included, verified via unzip -Z1 myproject on the resulting output, but it is included when running python setup.py sdist.
How do I include these extra html files when building a PEX binary?

Defining a MANIFEST.in alone isn't enough. You also need to set the include_package_data option to True in setup.cfg.
This option will include extra files found in the package so you must also move the html directory inside the myproj package.
So the directory structure looks like:
├── myproj
│ ├── utils.py
│ ├── __init__.py
│ ├── routes
│ │ ├── __init__.py
│ │ ├── auth.py
│ │ └── stuff.py
│ ├── html
│ │ ├── index.html
│ │ └── about.html
├── MANIFEST.in
├── setup.cfg
└── setup.py
The contents of MANIFEST.in are:
graft myproj/html
And setup.cfg contains in the [options] section:
include_package_data = True

What is the correct way to distribute "bin" and "tests" directories for a Python package?

I have created a python package.
At the advice of several internet sources (including https://github.com/pypa/sampleproject ), I have set up the directory structure like so:
root_dir
├── bin
│ └── do_stuff.py
├── MANIFEST.in
├── README.md
├── my_lib
│ ├── __init__.py
│ ├── __main__.py
│ └── my_lib.py
├── setup.cfg
├── setup.py
├── important_script.py
└── tests
├── __init__.py
└── test_lib.py
I have included tests, bin, and important_script.py in the manifest, and set include_package_data in setup.py to True.
However, after running pip install root_dir, I see that it correctly installed my_lib but bin and tests were just placed directly into Lib/site-packages as if they were separate packages.
I can't find important_script.py at all, and I don't think it was installed.
How do I correctly include these files/directories in my installation?
EDIT
So, it turns out that the bin and tests directories being placed directly into the site-packages directory was caused by something I was doing previously, but I can't discover what. At some point a build and a dist directory were generated in my root_dir (I assume by pip or setuptools?), and any changes I made to the project after that were not actually showing up in the installed package. After deleting these directories, I am no longer able to reproduce that issue.

The sample project distributes neither bin nor tests, it even explicitly excludes tests.
To include bin you should use scripts or entry_points (like in the sample project). Add this to your setup.py to setup() call:
scripts=['bin/do_stuff.py'],
To include tests you should restructure your tree to include the directory tests under the package directory:
root_dir
├── bin
│ └── do_stuff.py
├── MANIFEST.in
├── README.md
├── my_lib
│ ├── __init__.py
│ ├── __main__.py
│ └── my_lib.py
│ └── tests
│ ├── __init__.py
│ └── test_lib.py
├── setup.cfg
├── setup.py
├── important_script.py

Configure python sub-packages

I'm trying to set sub-packages for a python project. Please refer to the structure below. The main setup.py will call "setup.py" in each sub packages.
my_project
├── my_sub_package1
│ ├── foo2.py
│ ├── foo.py
│ └── setup.py
├── my_sub_package2
│ ├── bar2.py
│ ├── bar.py
│ └── setup.py
└── setup.py [main]
With this structure, in other projects, if the user only needs a sub_package, the user can choose to install "my_sub_package1" only, instead of installing the whole package (which can become bulky over time as number of packages increases).
Does anyone know if this is the correct way of doing it? Thanks!

How to use find_packages() to package all files from subdirectory

I am creating a python package (for the first time) and am able to package contents but I am having issue with packaging one of the data files that I have in subdirectory. My directory structure looks like this
├── Jenkinsfile
├── MANIFEST.in
├── README.md
├── __init__.py
├── country
│ ├── __init__.py
│ └── folder1
│ ├── file1.py
│ ├── file2.py
│ ├── file3.py
│ ├── folder2
│ ├── file4.xlsx
│ ├── __init__.py
├── repoman.yaml
├── requirements.txt
├── setup.py
└── test
└── unit
├──── __init__.py
├──── test_unit.py
my setup.py has the following get_packages() method
def get_packages():
return find_packages(exclude=['doc', 'imgs', 'test'])
when I build the package, my package does not include file4.xlsx, can anyone tell me why is that the case and how can I fix it?

I found this answer which is similar to what I want to do
I had to update my setup.py to include package_data and MANIFEST.in to include *.xlsx file (by providing the full directory path to excel file)