We Keep Coding

PyPI - folder inside the package folder not getting uploaded [duplicate]

When using setuptools, I can not get the installer to pull in any package_data files. Everything I've read says that the following is the correct way to do it. Can someone please advise?
setup(
name='myapp',
packages=find_packages(),
package_data={
'myapp': ['data/*.txt'],
},
include_package_data=True,
zip_safe=False,
install_requires=['distribute'],
)
where myapp/data/ is the location of the data files.

I realize that this is an old question, but for people finding their way here via Google: package_data is a low-down, dirty lie. It is only used when building binary packages (python setup.py bdist ...) but not when building source packages (python setup.py sdist ...). This is, of course, ridiculous -- one would expect that building a source distribution would result in a collection of files that could be sent to someone else to built the binary distribution.
In any case, using MANIFEST.in will work both for binary and for source distributions.

I just had this same issue. The solution, was simply to remove include_package_data=True.
After reading here, I realized that include_package_data aims to include files from version control, as opposed to merely "include package data" as the name implies. From the docs:
The data files [of include_package_data] must be under CVS or Subversion control
...
If you want finer-grained control over what files are included (for example, if
you have documentation files in your package directories and want to exclude
them from installation), then you can also use the package_data keyword.
Taking that argument out fixed it, which is coincidentally why it also worked when you switched to distutils, since it doesn't take that argument.

Following #Joe 's recommendation to remove the include_package_data=True line also worked for me.
To elaborate a bit more, I have no MANIFEST.in file. I use Git and not CVS.
Repository takes this kind of shape:
/myrepo
- .git/
- setup.py
- myproject
- __init__.py
- some_mod
- __init__.py
- animals.py
- rocks.py
- config
- __init__.py
- settings.py
- other_settings.special
- cool.huh
- other_settings.xml
- words
- __init__.py
word_set.txt
setup.py:
from setuptools import setup, find_packages
import os.path
setup (
name='myproject',
version = "4.19",
packages = find_packages(),
# package_dir={'mypkg': 'src/mypkg'}, # didnt use this.
package_data = {
# If any package contains *.txt or *.rst files, include them:
'': ['*.txt', '*.xml', '*.special', '*.huh'],
},
#
# Oddly enough, include_package_data=True prevented package_data from working.
# include_package_data=True, # Commented out.
data_files=[
# ('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
('/opt/local/myproject/etc', ['myproject/config/settings.py', 'myproject/config/other_settings.special']),
('/opt/local/myproject/etc', [os.path.join('myproject/config', 'cool.huh')]),
#
('/opt/local/myproject/etc', [os.path.join('myproject/config', 'other_settings.xml')]),
('/opt/local/myproject/data', [os.path.join('myproject/words', 'word_set.txt')]),
],
install_requires=[ 'jsonschema',
'logging', ],
entry_points = {
'console_scripts': [
# Blah...
], },
)
I run python setup.py sdist for a source distrib (haven't tried binary).
And when inside of a brand new virtual environment, I have a myproject-4.19.tar.gz, file,
and I use
(venv) pip install ~/myproject-4.19.tar.gz
...
And other than everything getting installed to my virtual environment's site-packages, those special data files get installed to /opt/local/myproject/data and /opt/local/myproject/etc.

include_package_data=True worked for me.
If you use git, remember to include setuptools-git in install_requires. Far less boring than having a Manifest or including all path in package_data ( in my case it's a django app with all kind of statics )
( pasted the comment I made, as k3-rnc mentioned it's actually helpful as is )

Using setup.cfg (setuptools ≥ 30.3.0)
Starting with setuptools 30.3.0 (released 2016-12-08), you can keep your setup.py very small and move the configuration to a setup.cfg file. With this approach, you could put your package data in an [options.package_data] section:
[options.package_data]
* = *.txt, *.rst
hello = *.msg
In this case, your setup.py can be as short as:
from setuptools import setup
setup()
For more information, see configuring setup using setup.cfg files.
There is some talk of deprecating setup.cfg in favour of pyproject.toml as proposed in PEP 518, but this is still provisional as of 2020-02-21.

Update: This answer is old and the information is no longer valid. All setup.py configs should use import setuptools. I've added a more complete answer at https://stackoverflow.com/a/49501350/64313
I solved this by switching to distutils. Looks like distribute is deprecated and/or broken.
from distutils.core import setup
setup(
name='myapp',
packages=['myapp'],
package_data={
'myapp': ['data/*.txt'],
},
)

I had the same problem for a couple of days but even this thread wasn't able to help me as everything was confusing. So I did my research and found the following solution:
Basically in this case, you should do:
from setuptools import setup
setup(
name='myapp',
packages=['myapp'],
package_dir={'myapp':'myapp'}, # the one line where all the magic happens
package_data={
'myapp': ['data/*.txt'],
},
)
The full other stackoverflow answer here

I found this post while stuck on the same problem.
My experience contradicts the experiences in the other answers.
include_package_data=True does include the data in the
bdist! The explanation in the setuptools
documentation
lacks context and troubleshooting tips, but
include_package_data works as advertised.
My setup:
Windows / Cygwin
git version 2.21.0
Python 3.8.1 Windows distribution
setuptools v47.3.1
check-manifest v0.42
Here is my how-to guide.
How-to include package data
Here is the file structure for a project I published on PyPI.
(It installs the application in __main__.py).
├── LICENSE.md
├── MANIFEST.in
├── my_package
│ ├── __init__.py
│ ├── __main__.py
│ └── _my_data <---- folder with data
│ ├── consola.ttf <---- data file
│ └── icon.png <---- data file
├── README.md
└── setup.py
Starting point
Here is a generic starting point for the setuptools.setup() in
setup.py.
setuptools.setup(
...
packages=setuptools.find_packages(),
...
)
setuptools.find_packages() includes all of my packages in the
distribution. My only package is my_package.
The sub-folder with my data, _my_data, is not considered a
package by Python because it does not contain an __init__.py,
and so find_packages() does not find it.
A solution often-cited, but incorrect, is to put an empty
__init__.py file in the _my_data folder.
This does make it a package, so it does include the folder
_my_data in the distribution. But the data files inside
_my_data are not included.
So making _my_data into a package does not help.
The solution is:
the sdist already contains the data files
add include_package_data=True to include the data files in the bdist as well
Experiment (how to test the solution)
There are three steps to make this a repeatable experiment:
$ rm -fr build/ dist/ my_package.egg-info/
$ check-manifest
$ python setup.py sdist bdist_wheel
I will break these down step-by-step:
Clean out the old build:
$ rm -fr build/ dist/ my_package.egg-info/
Run check-manifest to be sure MANIFEST.in matches the
Git index of files under version control:
$ check-manifest
If MANIFEST.in does not exist yet, create it from the Git
index of files under version control:
$ check-manifest --create
Here is the MANIFEST.in that is created:
include *.md
recursive-include my_package *.png
recursive-include my_package *.ttf
There is no reason to manually edit this file.
As long as everything that should be under version control is
under version control (i.e., is part of the Git index),
check-manifest --create does the right thing.
Note: files are not part of the Git index if they are either:
ignored in a .gitignore
excluded in a .git/info/exclude
or simply new files that have not been added to the index yet
And if any files are under version control that should not be
under version control, check-manifest issues a warning and
specifies which files it recommends removing from the Git index.
Build:
$ python setup.py sdist bdist_wheel
Now inspect the sdist (source distribution) and bdist_wheel
(build distribution) to see if they include the data files.
Look at the contents of the sdist (only the relevant lines are
shown below):
$ tar --list -f dist/my_package-0.0.1a6.tar.gz
my_package-0.0.1a6/
...
my_package-0.0.1a6/my_package/__init__.py
my_package-0.0.1a6/my_package/__main__.py
my_package-0.0.1a6/my_package/_my_data/
my_package-0.0.1a6/my_package/_my_data/consola.ttf <-- yay!
my_package-0.0.1a6/my_package/_my_data/icon.png <-- yay!
...
So the sdist already includes the data files because they are
listed in MANIFEST.in. There is nothing extra to do to include
the data files in the sdist.
Look at the contents of the bdist (it is a .zip file, parsed
with zipfile.ZipFile):
$ python check-whl.py
my_package/__init__.py
my_package/__main__.py
my_package-0.0.1a6.dist-info/LICENSE.md
my_package-0.0.1a6.dist-info/METADATA
my_package-0.0.1a6.dist-info/WHEEL
my_package-0.0.1a6.dist-info/entry_points.txt
my_package-0.0.1a6.dist-info/top_level.txt
my_package-0.0.1a6.dist-info/RECORD
Note: you need to create your own check-whl.py script to produce the
above output. It is just three lines:
from zipfile import ZipFile
path = "dist/my_package-0.0.1a6-py3-none-any.whl" # <-- CHANGE
print('\n'.join(ZipFile(path).namelist()))
As expected, the bdist is missing the data files.
The _my_data folder is completely missing.
What if I create a _my_data/__init__.py? I repeat the
experiment and I find the data files are still not there! The
_my_data/ folder is included but it does not contain the data
files!
Solution
Contrary to the experience of others, this does work:
setuptools.setup(
...
packages=setuptools.find_packages(),
include_package_data=True, # <-- adds data files to bdist
...
)
With the fix in place, redo the experiment:
$ rm -fr build/ dist/ my_package.egg-info/
$ check-manifest
$ python.exe setup.py sdist bdist_wheel
Make sure the sdist still has the data files:
$ tar --list -f dist/my_package-0.0.1a6.tar.gz
my_package-0.0.1a6/
...
my_package-0.0.1a6/my_package/__init__.py
my_package-0.0.1a6/my_package/__main__.py
my_package-0.0.1a6/my_package/_my_data/
my_package-0.0.1a6/my_package/_my_data/consola.ttf <-- yay!
my_package-0.0.1a6/my_package/_my_data/icon.png <-- yay!
...
Look at the contents of the bdist:
$ python check-whl.py
my_package/__init__.py
my_package/__main__.py
my_package/_my_data/consola.ttf <--- yay!
my_package/_my_data/icon.png <--- yay!
my_package-0.0.1a6.dist-info/LICENSE.md
my_package-0.0.1a6.dist-info/METADATA
my_package-0.0.1a6.dist-info/WHEEL
my_package-0.0.1a6.dist-info/entry_points.txt
my_package-0.0.1a6.dist-info/top_level.txt
my_package-0.0.1a6.dist-info/RECORD
How not to test if data files are included
I recommend troubleshooting/testing using the approach outlined
above to inspect the sdist and bdist.
pip install in editable mode is not a valid test
Note: pip install -e . does not show if data files are
included in the bdist.
The symbolic link causes the installation to behave as if the
data files are included (because they already exist locally on
the developer's computer).
After pip install my_package, the data files are in the
virtual environment's lib/site-packages/my_package/ folder,
using the exact same file structure shown above in the list of
the whl contents.
Publishing to TestPyPI is a slow way to test
Publishing to TestPyPI and then installing and looking in
lib/site-packages/my_packages is a valid test, but it is too
time-consuming.

Ancient question and yet... package management of python really leaves a lot to be desired. So I had the use case of installing using pip locally to a specified directory and was surprised both package_data and data_files paths did not work out. I was not keen on adding yet another file to the repo so I ended up leveraging data_files and setup.py option --install-data; something like this
pip install . --install-option="--install-data=$PWD/package" -t package

Moving the folder containing the package data into to module folder solved the problem for me.
See this question: MANIFEST.in ignored on "python setup.py install" - no data files installed?

Just remove the line:
include_package_data=True,
from your setup script, and it will work fine. (Tested just now with latest setuptools.)

Like others in this thread, I'm more than a little surprised at the combination of longevity and still a lack of clarity, BUT the best answer for me was using check-manifest as recommended in the answer from #mike-gazes
So, using just a setup.cfg and no setup.py and additional text and python files required in the package, what worked for me was keeping this in setup.cfg:
[options]
packages = find:
include_package_data = true
and updating the MANIFEST.in based on the check-manifest output:
include *.in
include *.txt
include *.yml
include LICENSE
include tox.ini
recursive-include mypkg *.py
recursive-include mypkg *.txt

For a directory structure like:
foo/
├── foo
│   ├── __init__.py
│   ├── a.py
│   └── data.txt
└── setup.py
and setup.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from setuptools import setup
NAME = 'foo'
DESCRIPTION = 'Test library to check how setuptools works'
URL = 'https://none.com'
EMAIL = 'gzorp#bzorp.com'
AUTHOR = 'KT'
REQUIRES_PYTHON = '>=3.6.0'
setup(
name=NAME,
version='0.0.0',
description=DESCRIPTION,
author=AUTHOR,
author_email=EMAIL,
python_requires=REQUIRES_PYTHON,
url=URL,
license='MIT',
classifiers=[
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.6',
],
packages=['foo'],
package_data={'foo': ['data.txt']},
include_package_data=True,
install_requires=[],
extras_require={},
cmdclass={},
)
python setup.py bdist_wheel works.

Starting with Setuptools 62.3.0, you can now use recursive wildcards ("**") to include a (sub)directory recursively. This way you can include whole folders with all their folders and files in it.
For example, when using a pyproject.toml file, this is how you include two folders recursively:
[tool.setuptools.package-data]
"ema_workbench.examples.data" = ["**"]
"ema_workbench.examples.models" = ["**"]
But you can also only include certain file-types, in a folder and all subfolders. If you want to include all markdown (.md) files for example:
[tool.setuptools.package-data]
"ema_workbench.examples.data" = ["**/*.md"]
It should also work when using setup.py or setup.cfg.
See https://github.com/pypa/setuptools/pull/3309 for the details.

include extra file in a Python package using setuptools

I am attempting to build a python wheel using setuptools. The package needs to include two files:
mymodule.py - a python module in the same directory as setup.py
myjar.jar - a java .jar file that exists outside of my package directory
I am building my package using python3 setup.py bdist_wheel.
If I call setup() like so:
setup(
name="mypkg",
py_modules=["mymodule"],
data_files=[('jars', ['../target/scala-2.11/myjar.jar'])]
)
then myjar.jar does successfully get included in the .whl (good so far) however when I pip install mypkg it places the jar at /usr/local/myjar.jar (this kinda explains why) which isn't what I want at all, I want it to exist in the same place as mymodule.py, i.e. /usr/local/lib/python3.7/site-packages/
If I change setup.py to
setup(
name="mypkg",
py_modules=["mymodule"],
package_data={'jars': '../target/scala-2.11/myjar.jar'}
)
or
setup(
name="mypkg",
py_modules=["mymodule"],
package_data={'jars': ['../target/scala-2.11/myjar.jar']}
)
then myjar.jar simply doesn't get included in the .whl. I tried copying myjar.jar into the same directory and changing setup.py to:
setup(
name="mypkg",
py_modules=["mymodule"],
package_data={'jars': 'myjar.jar'}
)
or
setup(
name="mypkg",
py_modules=["mymodule"],
package_data={'jars': ['myjar.jar']}
)
but still myjar.jar does not get included in the .whl.
I've been tearing my hair out over this for hours, hence why I'm here.
I've read a myriad of SO posts on this:
How to include package data with setuptools/distribute?
MANIFEST.in ignored on "python setup.py install" - no data files installed?
How do you add additional files to a wheel?
setuptools: adding additional files outside package
which suggest different combinations of data_files, package_data, include_package_data=True and/or use of a Manifest.in file but still I can't get this working as I would like, so I'm here hoping someone can advise what I'm doing wrong.

The data files (in that case myjar.jar) should really be package data files, and as such they should be part of a Python package. So having such files in parent directories makes things much more complicated, but probably not impossible. So let's start with a simpler example. I believe something like the following should work...
Project directory structure:
MyProject
├ MANIFEST.in
├ mymodule.py
├ setup.py
└ myjars
├ __init__.py
└ myjar.jar
MANIFEST.in:
recursive-include myjars *.jar
setup.py:
#!/usr/bin/env python3
import setuptools
setuptools.setup(
name='MyProject',
version='1.2.3',
#
include_package_data=True,
packages=['myjars'],
py_modules=["mymodule"],
)
myjars/__init__.py might not be strictly necessary, but I believe it's better to have it. And as always, an empty __init__.py file is perfectly good enough.
(This assumes the myjars/myjar.jar file exists before the source distribution sdist is built.)
As to dealing with the data files in parent directories, my recommendation would be to simply copy (or symlink) those files before calling setup.py, maybe as part of a shell script or anything like that. There are probably ways to do the copy as part of a custom setuptools command in setup.py, but it's not worth the effort in my opinion, and really it's not part of setup.py's job.

Why current working directory affects install path of setup.py? How to prevent that?

I have created a custom python package following this guide, so I have the following structure:
mypackage/ <-- VCS root
mypackage/
submodule1/
submodule2/
setup.py
And setup.py contains exactly the same information as in the guide:
from setuptools import setup, find_packages
setup(name='mypackage',
version='0.1',
description='desc',
url='vcs_url',
author='Hodossy, Szabolcs',
author_email='myemail#example.com',
license='MIT',
packages=find_packages(),
install_requires=[
# deps
],
zip_safe=False)
I have noticed if I go into the folder where setup.py is, and then call python setup.py install in a virtual environment, in site-packages the following structure is installed:
.../site-packages/mypackage-0.1-py3.6.egg/mypackage/
submodule1/
submodule2/
but if I call it from one folder up like python mypackage/setup.py install, then the structure is the following:
.../site-packages/mypackage-0.1-py3.6.egg/mypackage/
mypackage/
submodule1/
submodule2/
This later one ruins all imports from my module, as the path is different for the submodules.
Could you explain what is happening here and how to prevent that kind of behaviour?
This is experienced with Python 3.6 on both Windows and Linux.

Your setup.py does not contain any paths, but seems to only find the files via find_packages. So of course it depends from where you run it. The setup.py isn't strictly tied to its location. Of course you could do things like chdir to the basename of the setup file path in sys.argv[0], but that's rather ugly.
The question is, WHY do you want to build it that way? It looks more like you would want a structure like
mypackage-source
mypackage
submodule1
submodule2
setup.py
And then execute setup.py from the work directory. If you want to be able to run it from anywhere, the better workaround would be to put a shellscript next to it, like
#!/bin/sh
cd ``basename $0``
python setup.py $#
which separates the task of changing to the right directory (here I assume the directory with setup.py in the workdir) from running setup.py

jinja2 and distutils - how can I make distutils install template together with modules [duplicate]

How do I make setup.py include a file that isn't part of the code? (Specifically, it's a license file, but it could be any other thing.)
I want to be able to control the location of the file. In the original source folder, the file is in the root of the package. (i.e. on the same level as the topmost __init__.py.) I want it to stay exactly there when the package is installed, regardless of operating system. How do I do that?

Probably the best way to do this is to use the setuptools package_data directive. This does mean using setuptools (or distribute) instead of distutils, but this is a very seamless "upgrade".
Here's a full (but untested) example:
from setuptools import setup, find_packages
setup(
name='your_project_name',
version='0.1',
description='A description.',
packages=find_packages(exclude=['ez_setup', 'tests', 'tests.*']),
package_data={'': ['license.txt']},
include_package_data=True,
install_requires=[],
)
Note the specific lines that are critical here:
package_data={'': ['license.txt']},
include_package_data=True,
package_data is a dict of package names (empty = all packages) to a list of patterns (can include globs). For example, if you want to only specify files within your package, you can do that too:
package_data={'yourpackage': ['*.txt', 'path/to/resources/*.txt']}
The solution here is definitely not to rename your non-py files with a .py extension.
See Ian Bicking's presentation for more info.
UPDATE: Another [Better] Approach
Another approach that works well if you just want to control the contents of the source distribution (sdist) and have files outside of the package (e.g. top-level directory) is to add a MANIFEST.in file. See the Python documentation for the format of this file.
Since writing this response, I have found that using MANIFEST.in is typically a less frustrating approach to just make sure your source distribution (tar.gz) has the files you need.
For example, if you wanted to include the requirements.txt from top-level, recursively include the top-level "data" directory:
include requirements.txt
recursive-include data *
Nevertheless, in order for these files to be copied at install time to the package’s folder inside site-packages, you’ll need to supply include_package_data=True to the setup() function. See Adding Non-Code Files for more information.

To accomplish what you're describing will take two steps...
The file needs to be added to the source tarball
setup.py needs to be modified to install the data file to the source path
Step 1: To add the file to the source tarball, include it in the MANIFEST
Create a MANIFEST template in the folder that contains setup.py
The MANIFEST is basically a text file with a list of all the files that will be included in the source tarball.
Here's what the MANIFEST for my project look like:
CHANGELOG.txt
INSTALL.txt
LICENSE.txt
pypreprocessor.py
README.txt
setup.py
test.py
TODO.txt
Note: While sdist does add some files automatically, I prefer to explicitly specify them to be sure instead of predicting what it does and doesn't.
Step 2: To install the data file to the source folder, modify setup.py
Since you're looking to add a data file (LICENSE.txt) to the source install folder you need to modify the data install path to match the source install path. This is necessary because, by default, data files are installed to a different location than source files.
To modify the data install dir to match the source install dir...
Pull the install dir info from distutils with:
from distutils.command.install import INSTALL_SCHEMES
Modify the data install dir to match the source install dir:
for scheme in INSTALL_SCHEMES.values():
scheme['data'] = scheme['purelib']
And, add the data file and location to setup():
data_files=[('', ['LICENSE.txt'])]
Note: The steps above should accomplish exactly what you described in a standard manner without requiring any extension libraries.

It is 2019, and here is what is working -
despite advice here and there, what I found on the internet halfway documented is using setuptools_scm, passed as options to setuptools.setup. This will include any data files that are versioned on your VCS, be it git or any other, to the wheel package, and will make "pip install" from the git repository to bring those files along.
So, I just added these two lines to the setup call on "setup.py". No extra installs or import required:
setup_requires=['setuptools_scm'],
include_package_data=True,
No need to manually list package_data, or in a MANIFEST.in file - if it is versioned, it is included in the package. The docs on "setuptools_scm" put emphasis on creating a version number from the commit position, and disregard the really important part of adding the data files. (I can't care less if my intermediate wheel file is named "*0.2.2.dev45+g3495a1f" or will use the hardcoded version number "0.3.0dev0" I've typed in - but leaving crucial files for the program to work behind is somewhat important)

create MANIFEST.in in the project root with recursive-include to the required directory or include with the file name.
include LICENSE
include README.rst
recursive-include package/static *
recursive-include package/templates *
documentation can be found here

Step 1: create a MANIFEST.in file in the same folder with setup.py
Step 2: include the relative path to the files you want to add in MANIFEST.in
include README.rst
include docs/*.txt
include funniest/data.json
Step 3: set include_package_data=True in the setup() function to copy these files to site-package
Reference is here.

I wanted to post a comment to one of the questions but I don't enough reputation to do that >.>
Here's what worked for me (came up with it after referring the docs):
package_data={
'mypkg': ['../*.txt']
},
include_package_data: False
The last line was, strangely enough, also crucial for me (you can also omit this keyword argument - it works the same).
What this does is it copies all text files in your top-level or root directory (one level up from the package mypkg you want to distribute).

None of the above really worked for me. What saved me was this answer.
Apparently, in order for these data files to be extracted during installation, I had to do a couple of things:
Like already mentioned - Add a MANIFEST.in to the project and specify the folder/files you want to be included. In my case: recursive-include folder_with_extra_stuff *
Again, like already mentioned - Add include_package_data=True to your setup.py. This is crucial, because without it only the files that match *.py will be brought.
This is what was missing! - Add an empty __init__.py to your data folder. For me I had to add this file to my folder-with-extra-stuff.
Extra - Not sure if this is a requirement, but with my own python modules I saw that they're zipped inside the .egg file in site-packages. So I had to add zip_safe=False to my setup.py file.
Final Directory Structure
my-app/
├─ app/
│ ├─ __init__.py
│ ├─ __main__.py
├─ folder-with-extra-stuff/
│ ├─ __init__.py
│ ├─ data_file.json
├─ setup.py
├─ MANIFEST.in

This works in 2020!
As others said create "MANIFEST.in" where your setup.py is located.
Next in manifest include/exclude all the necessary things. Be careful here regarding the syntax.
Ex: lets say we have template folder to be included in the source package.
in manifest file do this :
recursive-include template *
Make sure you leave space between dir-name and pattern for files/dirs like above.
Dont do like this like we do in .gitignore
recursive-include template/* [this won't work]
Other option is to use include. There are bunch of options. Look up here at their docs for Manifest.in
And the final important step, include this param in your setup.py and you are good to go!
setup(
...
include_package_data=True,
......
)
Hope that helps! Happy Coding!

In setup.py under setup( :
setup(
name = 'foo library'
...
package_data={
'foolibrary.folderA': ['*'], # All files from folder A
'foolibrary.folderB': ['*.txt'] #All text files from folder B
},

Here is a simpler answer that worked for me.
First, per a Python Dev's comment above, setuptools is not required:
package_data is also available to pure distutils setup scripts
since 2.3. – Éric Araujo
That's great because putting a setuptools requirement on your package means you will have to install it also. In short:
from distutils.core import setup
setup(
# ...snip...
packages = ['pkgname'],
package_data = {'pkgname': ['license.txt']},
)

I just wanted to follow up on something I found working with Python 2.7 on Centos 6. Adding the package_data or data_files as mentioned above did not work for me. I added a MANIFEST.IN with the files I wanted which put the non-python files into the tarball, but did not install them on the target machine via RPM.
In the end, I was able to get the files into my solution using the "options" in the setup/setuptools. The option files let you modify various sections of the spec file from setup.py. As follows.
from setuptools import setup
setup(
name='theProjectName',
version='1',
packages=['thePackage'],
url='',
license='',
author='me',
author_email='me#email.com',
description='',
options={'bdist_rpm': {'install_script': 'filewithinstallcommands'}},
)
file - MANIFEST.in:
include license.txt
file - filewithinstallcommands:
mkdir -p $RPM_BUILD_ROOT/pathtoinstall/
#this line installs your python files
python setup.py install -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES
#install license.txt into /pathtoinstall folder
install -m 700 license.txt $RPM_BUILD_ROOT/pathtoinstall/
echo /pathtoinstall/license.txt >> INSTALLED_FILES

None of the answers worked for me because my files were at the top level, outside the package. I used a custom build command instead.
import os
import setuptools
from setuptools.command.build_py import build_py
from shutil import copyfile
HERE = os.path.abspath(os.path.dirname(__file__))
NAME = "thepackage"
class BuildCommand(build_py):
def run(self):
build_py.run(self)
if not self.dry_run:
target_dir = os.path.join(self.build_lib, NAME)
for fn in ["VERSION", "LICENSE.txt"]:
copyfile(os.path.join(HERE, fn), os.path.join(target_dir,fn))
setuptools.setup(
name=NAME,
cmdclass={"build_py": BuildCommand},
description=DESCRIPTION,
...
)

For non-python files to be included in an installation, they must be within one of the installed package directories. If you specify non-python files outside of your package directories in MANIFEST.in, they will be included in your distribution, but they will not be installed. The "documented" ways of installing arbitrary files outside of your package directories do not work reliably (as everyone has noticed by now).
The above answer from Julian Mann copies the files to your package directory in the build directory, so it does work, but not if you are installing in editable/develop mode (pip install -e or python setup.py develop). Based on this answer to a related question (and Julian's answer), below is an example that copies files to your installed package location either way after all the other install/develop tasks are done. The assumption here is that files file1 and file2 in your root-level data directory will be copied to your installed package directory (my_package), and that they will be accessible from python modules in your package using os.path.join(os.path.dirname(__file__), 'file1'), etc.
Remember to also do the MANIFEST.in stuff described above so that these files are also included in your distribution. Why setuptools would include files in your distribution and then silently never install them, is beyond my ken. Though installing them outside of your package directory is probably even more dubious.
import os
from setuptools import setup
from setuptools.command.develop import develop
from setuptools.command.install import install
from shutil import copyfile
HERE = os.path.abspath(os.path.dirname(__file__))
NAME = 'my_package'
def copy_files (target_path):
source_path = os.path.join(HERE, 'data')
for fn in ["file1", "file2"]:
copyfile(os.path.join(source_path, fn), os.path.join(target_path,fn))
class PostDevelopCommand(develop):
"""Post-installation for development mode."""
def run(self):
develop.run(self)
copy_files (os.path.abspath(NAME))
class PostInstallCommand(install):
"""Post-installation for installation mode."""
def run(self):
install.run(self)
copy_files (os.path.abspath(os.path.join(self.install_lib, NAME)))
setup(
name=NAME,
cmdclass={
'develop': PostDevelopCommand,
'install': PostInstallCommand,
},
version='0.1.0',
packages=[NAME],
include_package_data=True,
setup_requires=['setuptools_scm'],
)

Figured out a workaround: I renamed my lgpl2.1_license.txt to lgpl2.1_license.txt.py, and put some triple quotes around the text. Now I don't need to use the data_files option nor to specify any absolute paths. Making it a Python module is ugly, I know, but I consider it less ugly than specifying absolute paths.

How can I get my setup.py to use a relative path to my files?

I'm trying to build a Python distribution with distutils. Unfortunately, my directory structure looks like this:
/code
/mypackage
__init__.py
file1.py
file2.py
/subpackage
__init__.py
/build
setup.py
Here's my setup.py file:
from distutils.core import setup
setup(
name = 'MyPackage',
description = 'This is my package',
packages = ['mypackage', 'mypackage.subpackage'],
package_dir = { 'mypackage' : '../mypackage' },
version = '1',
url = 'http://www.mypackage.org/',
author = 'Me',
author_email = 'me#here.com',
)
When I run python setup.py sdist it correctly generates the manifest file, but doesn't include my source files in the distribution. Apparently, it creates a directory to contain the source files (i.e. mypackage1) then copies each of the source files to mypackage1/../mypackage which puts them outside of the distribution.
How can I correct this, without forcing my directory structure to conform to what distutils expects?

What directory structure do you want inside of the distribution archive file? The same as your existing structure?
You could package everything one directory higher (code in your example) with this modified setup.py:
from distutils.core import setup
setup(
name = 'MyPackage',
description = 'This is my package',
packages = ['mypackage', 'mypackage.subpackage'],
version = '1',
url = 'http://www.mypackage.org/',
author = 'Me',
author_email = 'me#here.com',
script_name = './build/setup.py',
data_files = ['./build/setup.py']
)
You'd run this (in the code directory):
python build/setup.py sdist
Or, if you want to keep dist inside of build:
python build/setup.py sdist --dist-dir build/dist
I like the directory structure you're trying for. I've never thought setup.py was special enough to warrant being in the root code folder. But like it or not, I think that's where users of your distribution will expect it to be. So it's no surprise that you have to trick distutils to do something else. The data_files parameter is a hack to get your setup.py into the distribution in the same place you've located it.

Have it change to the parent directory first, perhaps?
import os
os.chdir(os.pardir)
from distutils.core import setup
etc.
Or if you might be running it from anywhere (this is overkill, but...):
import os.path
my_path = os.path.abspath(__file__)
os.chdir(os.normpath(os.path.join(my_path, os.pardir)))
etc. Not sure this works, but should be easy to try.

Run setup.py from the root folder of the project
In your case, place setup.py in code/
code/ should also include:
LICENSE.txt
README.txt
INSTALL.txt
TODO.txt
CHANGELOG.txt
The when you run "setup.py sdist' it should auto-gen a MANIFEST including:
- any files specified in py_modules and/or packages
- setup.py
- README.txt
To add more files just hand-edit the MANIFEST file to include whatever other files your project needs.
For a somewhat decent explanation of this read this.
To see a working example checkout my project.
Note: I don't put the MANIFEST under version control so you won't find it there.

A sorta lame workaround but I'd probably just use a Makefile that rsynced ./mypackage to ./build/mypackage and then use the usual distutils syntax from inside ./build. Fact is, distutils expects to unpack setup.py into the root of the sdist and have code under there, so you're going to have a devil of time convincing it to do otherwise.
You can always nuke the copy when you make clean so you don't have to mess up your vcs.

Also a lame workaround, but a junction/link of the package directory inside of the build project should work.