I'm trying to include some data files in a python package using the setuptools package_data option. I'm then accessing the files with pkg_resources. This works perfectly when the python .egg file is installed as-is (i.e. still zipped). But when the egg file is unzipped during installation, the data files are not installed.
In other words, if I run:
python setup.py bdist_egg
cd dist
sudo easy_install -z EnrichPy-0.1.001-py2.6.egg
then the egg file is installed (with the data safely zipped inside) and everything works.
On the other hand, if I run
sudo easy_install -Z EnrichPy-0.1.001-py2.6.egg
then the data files are not installed. I have a directory called
EnrichPy-0.1.001-py2.6.egg/enrichpy/ under dist-packages, but it contains only my source files, not my data files.
Can anyone suggest what I need to do to get the package_data files to be installed when easy_install unzips the egg file?
Notes:
The package is available at http://github.com/roban/EnrichPy
I can test it by running:
import enrichpy.yields
enrichpy.yields.Data_vdHG().data
If that exits without errors, then pkg_resources is finding the installed data.
Problem solved, thanks to help from P.J. Eby on the distutils-sig email list:
http://mail.python.org/pipermail/distutils-sig/2010-November/017054.html
Just needed to rename files to avoid the '..' string.
Related
I'm new to packaging in Python. I've tried to specify my non-python files within setup.py's 'scripts' argument, and also specifying the file within MANIFEST.in, however after I package the file using python setup.py build sdist and install using pip, only the files with the .py extension make it to the site-packages/my_package directory.
Am I missing something?
15 minutes later I find the answer. sdist only includes the *.py files. I just changed the command to use bdist_wheel and all the files I needed were included.
I have several packages in folder Top. The path is set at the command prompt such that each package contains some python files that use other packages' modules.
I have the following files in Top directory: setup.py, MANIFEST, MANIFEST.in, README. I wish to modify the setup files such that the path is set during installation. Does PYTHONPATH set it, and does it need to go into a new file?
The appropriate actions here are
Packages do not mess with PYTHONPATH. Never.
Instead, you write an setup.py entry point to your command line scripts
When the user installs the package using pip install the command line script is automatically added to user's PATH, which is hopefully inside virtualenv
This command line script is generated during the install, so that it points to the PYTHONPATH of virtualenv or system-wide Python installation. The path is hardcoded at the script head, pointing to the current Python interpreter.
More info
https://packaging.python.org/en/latest/distributing/
I have run setup.py and installed, my script/package it all appears in Python27\Lib\site-packages in a folder with the relevant .egg.info file.
However, after adding site-packages to my PATH I cannot run the relevant scripts.
python: cant open file 'mypackage': [Errno 2] No such file or directory
What have I missed?
Essentially I would like to install the package, several modules are included in a folder with __init and __main files. Once installed I would like to just run
python mypackage inputfile.txt
however mypackage is not found. I will use optparse for taking in the txt files, but that is irrelevant as I can't get the packages installed.
As a rule, scripts do not go in site-packages. On the MS-windows platform they generally go in C:\Python27\Scripts\.
So on MS-Windows you probably want to add C:\Python27\;C:\Python27\Scripts\ to your $PATH.
Additionally, you have to associate the .py filetype with python.
In general, the whole chapter Using Python on Windows of the official CPython documentation should be considered required reading for users of Python on Windows.
Python has ability to "pseudoinstall" a package by running it's setup.py script with develop instead of install. This modifies python environment so package can be imported from it's current location (it's not copied into site-package directory). This allows to develop packages that are used by other packages: source code is modified in place and changes are available to rest of python code via simple import.
All works fine except that setup.py develop command creates an .egg-info folder with metadata at same level as setup.py. Mixing source code and temporary files is not very good idea - this folder need to be added into "ignore" lists of multiple tools starting from vcs and ending backup systems.
Is it possible to use setup.py develop but create .egg-info directory in some other place, so original source code is not polluted by temporary directory and files?
setup.py develop creates a python egg, in-place; it does not [modify the] python environment so package can be imported from it's current location. You still have to either add it's location to the python search path or use the directory it is placed in as the current directory.
It is the job of the develop command to create an in-place egg, which may include compiling C extensions, running the 2to3 python conversion process to create Python3 compatible code, and to provide metadata other python code may be relying on. When you install the package as an egg in your site-packages directory, the same metadata is included there as well. The data is certainly not temporary (it is extracted from your setup.py file for easy parsing by other tools).
The intent is that you can then rely on that metadata when using your package in a wider system that relies on the metadata being present, while still developing the package. For example, in a buildout development deployment, we often use mr.developer to automate the process of fetching the source code for a given package when we need to work on it, which builds it as a develop egg and ties it into the deployment while we work on the code.
Note that the .egg-info directory serves a specific purpose: to signal to other tools in the setuptools eco-system that your package is installed and available. If your package is a dependency of another egg in your setup, then that dependency is satisfied. pip and easy_install and buildout will not try and fetch the egg from somewhere else instead.
Apart from creating the .egg-info directory, the only other thing the command does, is to build extensions, in-place. So the command you are looking for instead is:
setup.py build_ext --inplace
This will do the exact same thing as setup.py develop but leave out the .egg-info directory. It also won't generate the .pth file.
There is no way of generating only the .pth file and leave out the .egg-info directory generation.
Technically speaking, setup.py develop will also check if you have the setuptools site.py file installed to support namespaced packages, but that's not relevant here.
The good manner is to keep all source files inside special directory which name is your project name (programmers using other languages keep their code inside src directory). So if your setup.py file is inside myproject directory then you should keep the files at myproject/myproject. This method keeps your sources separated from other files regardless what happen in main directory.
My suggestion would be to use whitelist instead of blacklist -- tell the tools to ignore all files excluding these which are inside myproject directory. I think that this is the simplest way not to touch your ignore lists too often.
Try the --install-dir option. You may also want to use --build-dir to change building dir.
Sorry about the nooby question, but, when I download and unzip a third-party python package, and then python setup.py install it thereby making an egg directory in site-packages, what do I do with the original unzipped directory in the Download folder? Should I sudo copy & paste all the test/docs/README files along with the rest of the corresponding site-packages files? I've typically deleted them but don't think that's a smart thing to do..
If all you want is to use the installed Python package, then you don't need the downloaded directory at all. You can delete it if you like. If you want to use it for its docs, then you can keep it, or move it somewhere else. There's no connection between the installed package and the original unzipped directory you installed from, so you are free to do what you like with it.