We Keep Coding

setup.py not correctly including package_data

I am trying to include spreadsheets (.xlsx) and text files (.txt) in a directory (with an init.py file, so it is a package). I can get the .txt files to install properly but CANNOT get the .xlsx files to appear in the package directly after performing a pip install. I have tried with and without MANIFEST.in file and include_package_data=True in setup.py.
My package structure is as follows:
- setup.py
- requirements.py
- myproject
- __init__.py
- main.py
- Data
- __init__.py
- test.xlsx
- test.txt
And my setup.py files is as follows:
from setuptools import setup, find_packages
setup(
name='myproject',
packages=find_packages(),
package_data={'myproject': ['Data/*.xlsx', 'Data/*.txt']}
)
I install using pip and when I look at the package directory, all of the files appear as expected, except for the .xlsx files. It is strange that the .txt file shows up but the .xlsx file(s) do not.
I've seen tutorials, documentation and SO posts on this topic and have tried all of those suggestions.

How to include an executable file in setup.py

I have a Python project that uses an executable file.
The package structure is something like that:
/Project
/Package
__init__.py
aClass.py
executableFile
LICENSE
README.md
and I've this setup.py:
...
setup(
author=...
author_email=....
classifiers=[...]
description=....
install_requires=[...]
license=..
long_description=...
include_package_data=True
packages=find_packages(include=['Package*'])
url=..
version=x.x.x
)
but when I upload the package as stated here with twine in PyPI, the executable file is not loaded.
How can I properly include that file in the package?
PS: I've also read about adding scripts=[..] in setup.py, but it is limited to python files.

You need to use a manifest.
In MANIFEST.in:
include Package/executableFile

Python setup.py include .json files in the egg

I want to package .json files as well in the python egg file.
For example: boto package has endpoints.json file. But when I run python setup.py bdist_egg it does not include the json file in the egg. How do I include the Json file in the egg?
How do I include *.json file in the egg?
Below is the setup.py code
from setuptools import setup, find_packages, Extension
setup(
name='X-py-backend',
version='tip',
description='X Python backend tools',
author='meme',
packages=find_packages('python'),
package_dir={'': 'python'},
data_files=[('boto', ['python/boto/endpoints.json'])],
namespace_packages = ['br'],
zip_safe=True,
)
setup(
name='X-py-backend',
version='tip',
packages=find_packages('protobuf/target/python'),
package_dir={'': 'protobuf/target/python'},
namespace_packages = ['br'],
zip_safe=True,
)

You only need to list the file on data_files parameter. Here is the example.
setup(
name='X-py-backend',
version='tip',
description='XXX Python backend tools',
author='meme',
packages=find_packages('python'),
package_dir={'': 'python'},
data_files=[('boto', ['boto/*.json'])]
namespace_packages = ['br'],
zip_safe=True
)
You can see the details here. https://docs.python.org/2/distutils/setupscript.html#installing-additional-files
Another way to do this is to use MANIFEST.in files. you need to create a MANIFEST.in file in your project root. Here is the example.
include python/boto/endpoints.json
Please visit here for more information.https://docs.python.org/2/distutils/sourcedist.html#manifest-template

Well this works for me.
setup.py:
from setuptools import setup, find_packages
setup(
name="clean",
version="0.1",
description="Clean package",
packages=find_packages() + ['config'],
include_package_data=True
)
MANIFEST.in:
recursive-include config *
where there is a config file under the project root directory which contains a whole bunch of json files.
Hope this helps.

How to include package sub-folders in my project distribution?

I have a project with this structure:
SomeProject/
bin/
CHANGES.txt
docs/
LICENSE.txt
MANIFEST.in
README.txt
setup.py
someproject/
__init__.py
location.py
utils.py
static/
javascript/
somescript.js
And a "setup.py" as follows:
#!/usr/bin/env python
import someproject
from os.path import exists
try:
from setuptools import setup, find_packages
except ImportError:
from distutils.core import setup, find_packages
setup(
name='django-some-project',
version=someproject.__version__,
maintainer='Some maintainer',
maintainer_email='some#manteiner.com',
packages=find_packages(),
include_package_data=True,
scripts=[],
url='https://github.com/xxx/some-project',
license='LICENSE',
description='Some project description.',
long_description=open('README.markdown').read() if exists("README.markdown") else "",
install_requires=[
"Django >= 1.4.0"
],
)
Then, when I upload it using the command:
python setup.py sdist upload
It seems ok, but there is no "static" folder with this "javascript" subfolder in the package. My "setup.py" was inspired on github.com/maraujop/django-crispy-forms that has a similar structure. Any hint on what is wrong on uploading this subfolders?

You should be able to add those files to source distributions by editing the MANIFEST.in file to add a line like:
recursive-include someproject/static *.js
or just:
include someproject/static/javascript/*.js
This will be enough to get the files included in source distributions. If the setuptools include_package_data option you're using isn't enough to get the files installed, you can ask for them to be installed explicitly with something like this in your setup.py:
package_data={'someproject': ['static/javascript/*.js']},

Use following
packages = ['.','templates','static','docs'],
package_data={'templates':['*'],'static':['*'],'docs':['*'],},

Include non python files in RPM with setuptools

I have some fixture directories that contain xml files that I would like included with my python project when building the RPM with bdist_rpm. I thought I could do this by having MANIFEST.in do a recursive-include * *, however, it does not include anything other than *.py files. Is there anyway to have bdist_rpm include non python files in the package or specifically include *.xml files as well?

Where are you trying to install them? If you put them inside a package directory, like this...
myproject/
mypackage/
__init__.py
resources/
file1.xml
file2.xml
...you can use the package_data option in your setup.py file, like this:
from setuptools import setup, find_packages
setup(
name='myproject',
version='0.1',
description='A description.',
packages=find_packages(),
include_package_data=True,
package_data = { '': [ '*.xml' ] },
install_requires=[],
)
This will recursively include any *.xml files inside of any packages. They'll get installed with the rest of your package(s) somewhere inside of the Python library path. You can do the same thing with a MANIFEST.in that looks like this:
recursive-include * *.xml
If you're trying to install them into specific filesystem locations outside of the Python library, I'm not sure if you can do that via setup.py.

You can use data_files parameter of setup to do what you need. Something like this:
setup(
...
package_data = { '/usr/share/yourapp/xmls': [ 'xmls/1.xml', 'xmls/2.xml' ] },
...
)
This would install following files:
/usr/share/yourapp/xmls/1.xml
/usr/share/yourapp/xmls/2.xml
I generally create the list of files in a function like this:
def get_xmls():
xmlfiles = []
for filename in os.listdir('xmls/'):
if filename.endswith('.xml'):
xmlfiles.append('xmls/%s' % filename)
return xmlfiles