When doing
python setup.py sdist bdist_wheel
it creates build, dist, packagename.egg-info directories. I'd like to have them out of the current folder.
I tried:
--dist-dir=../dist: works with sdist but packagename.egg-info is still there
--bdist-dir=../dist: for example:
python setup.py sdist bdist_wheel --dist-dir=../dist --bdist-dir=../dist2
works and the final bdist package is in ../dist. But the current folder still gets new directories build, dist, packagename.egg-info, which I don't want.
Question: how to have everything (the output of sdist and bdist_wheel) outside of the current folder?
Of course I can write a script with mv, rm -r, etc. but I wanted to know if there exists a built-in solution.
setup.py takes a series of commands (setup.py --help shows usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]). Critically, each command is its own script with its own command line options (which is why setup.py bdist_wheel --help shows different options than setup.py sdist --help).
The subtlety here is that certain commands generate calls to other commands under the hood but aren't nice enough to pass along common flags to the generated commands. For example, bdist_wheel ends up calling build and also egg_info but it does not pass along any bdist-dir you may specify. There's no global "use such and such working directory for the whole setup.py command" because all the commands are running independently and without knowledge of each other.
In order to redirect all temp directories somewhere else you have to manually specify each command and use its temp directory flag. In my case for bdist_wheel the full invocation was:
python setup.py ^
build --build-base \path\to\working\dir ^
egg_info --egg-base \path\to\working\dir ^
bdist_wheel --dist-dir \path\to\final\output\dir
(Side note, I found that if build-base and egg-base didn't match I got a weird error about not having used relative paths.)
This was sufficient to put all temp directories outside of the source folder.
Unfortunately it's not directly obvious which temp directory is the result of which command. You can use the list of commands (setup.py --help-commands) and some guesswork to determine which command created each temp directory. Then use --help on that command to see how to change its working directory.
I tried some time again with -d, --dist-dir, --bdist-dir but I found no way to do it in one-line.
I'm afraid the shortest we could find (on Windows) is:
python setup.py sdist bdist_wheel
rmdir /s /q packagename.egg-info build ..\dist
move dist ..
Why don't you try the command:
python setup.py egg_info --egg-base /tmp sdist bdist_wheel
That will put the .egg_info folder in the tmp-folder.
At least, it will be outside of your source folder.
Related
I have some protocol buffer definitions which need to be built to Python source as part of the pip install process. I've subclassed the setuptools.command.install command in setup.py but I think it's trying to run the Makefile after the package is installed so the sources aren't recognised.
I can't find information about what happens during a pip installation. Can anyone shed any light?
setup.py:
import subprocess
import sys
from setuptools import setup
from setuptools.command.install import install
class Install(install):
"""Customized setuptools install command - builds protos on install."""
def run(self):
protoc_command = ["make", "python"]
if subprocess.call(protoc_command) != 0:
sys.exit(-1)
install.run(self)
setup(
name='myprotos',
version='0.0.1',
description='Protocol Buffers.',
install_requires=[],
cmdclass={
'install': Install,
}
)
Output of $ pip install -vvv .:
Processing /path/to/myprotos
Running setup.py (path:/private/var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/pip-jpgCby-build/setup.py) egg_info for package from file:///path/to/myprotos
Running command python setup.py egg_info
running egg_info
creating pip-egg-info/myprotos.egg-info
writing pip-egg-info/myprotos.egg-info/PKG-INFO
writing top-level names to pip-egg-info/myprotos.egg-info/top_level.txt
writing dependency_links to pip-egg-info/myprotos.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/myprotos.egg-info/SOURCES.txt'
reading manifest file 'pip-egg-info/myprotos.egg-info/SOURCES.txt'
writing manifest file 'pip-egg-info/myprotos.egg-info/SOURCES.txt'
Source in /private/var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/pip-jpgCby-build has version 0.0.1, which satisfies requirement myprotos==0.0.1 from file:///path/to/myprotos
Building wheels for collected packages: myprotos
Running setup.py bdist_wheel for myprotos: started
Destination directory: /var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/tmpD7dfGKpip-wheel-
Running command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/pip-jpgCby-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/tmpD7dfGKpip-wheel- --python-tag cp27
running bdist_wheel
running build
installing to build/bdist.macosx-10.12-x86_64/wheel
running install
# THIS IS MY MAKEFILE RUNNING
Grabbing github.com/google/protobuf...
Building Python protos...
# MAKEFILE COMPLETE
running install_egg_info
running egg_info
creating myprotos.egg-info
writing myprotos.egg-info/PKG-INFO
writing top-level names to myprotos.egg-info/top_level.txt
writing dependency_links to myprotos.egg-info/dependency_links.txt
writing manifest file 'myprotos.egg-info/SOURCES.txt'
reading manifest file 'myprotos.egg-info/SOURCES.txt'
writing manifest file 'myprotos.egg-info/SOURCES.txt'
Copying myprotos.egg-info to build/bdist.macosx-10.12-x86_64/wheel/myprotos-0.0.1-py2.7.egg-info
running install_scripts
creating build/bdist.macosx-10.12-x86_64/wheel/myprotos-0.0.1.dist-info/WHEEL
Running setup.py bdist_wheel for myprotos: finished with status 'done'
Stored in directory: /Users/jds/Library/Caches/pip/wheels/92/0b/37/b5a50146994bc0b6774407139f01d648ba3a9b4853d2719c51
Removing source in /private/var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/pip-jpgCby-build
Successfully built myprotos
Installing collected packages: myprotos
Found existing installation: myprotos 0.0.1
Uninstalling myprotos-0.0.1:
Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/DESCRIPTION.rst
Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/INSTALLER
Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/METADATA
Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/RECORD
Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/WHEEL
Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/metadata.json
Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/top_level.txt
Successfully uninstalled myprotos-0.0.1
Successfully installed myprotos-0.0.1
Cleaning up...
Should my Makefile be running early in the process to generate the source files? Do the files need to be there before egg_info runs for example?
If I manually run the Makefile and then install the package then it works.
Update
Here is the structure of my project:
myprotos
├── Makefile
├── README.md
├── document.proto
├── myprotos # Generated by Makefile
│ ├── __init__.py # Generated by Makefile
│ └── proto_pb2.py # Generated by Makefile
└── setup.py
Here is the section of the Makefile which generates the Python source from Potocol Buffer definitions:
python: protoc deps
# the protoc and deps command above just downloads
# the `protoc` binary to a local bin directory
#echo "Building Python protos..."
#mkdir -p "${PYTHON_OUT}"
#touch "${PYTHON_OUT}"/__init__.py
#printf "__all__ = ['proto_pb2']" > "${PYTHON_OUT}"/__init__.py
#PATH="${LOCAL_BINARY_PATH}:$$PATH" protoc \
--proto_path="${BASE}" \
--proto_path="${GOPATH}/src/github.com/google/protobuf/src" \
--python_out="${PYTHON_OUT}/" \
${PROTOS}
OK, there are three things you need to change here:
Add Makefile and document.proto to a new file MANIFEST.in.
Makefile
document.proto
If you do that, the .zip file created by python setup.py sdist (which is also uploaded to PyPI) will contain those files.
You need to run your make command during python setup.py build, not during install. Since you are generating Python code, you will need to change the build_py command here:
import sys
import subprocess
from setuptools import setup
from setuptools.command.build_py import build_py
class Build(build_py):
"""Customized setuptools build command - builds protos on build."""
def run(self):
protoc_command = ["make", "python"]
if subprocess.call(protoc_command) != 0:
sys.exit(-1)
super().run()
setup(
name='buildtest',
version='1.0',
description='Python Distribution Utilities',
packages=['buildtest'],
cmdclass={
'build_py': Build,
}
)
If your Makefile generated machine code, i.e. from C or any other compiled language, you should change the build_ext command:
import sys
import subprocess
from setuptools import setup
from setuptools.command.build_ext import build_ext
class Build(build_ext):
"""Customized setuptools build command - builds protos on build."""
def run(self):
protoc_command = ["make", "python"]
if subprocess.call(protoc_command) != 0:
sys.exit(-1)
super().run()
setup(
name='buildtest',
version='1.0',
description='Python Distribution Utilities',
packages=['buildtest'],
has_ext_modules=lambda: True,
cmdclass={
'build_ext': Build,
}
)
Lastly, you need to tell setuptools to install the resulting package on install by defining an attribute packages in setup():
setup(
...
packages=['myprotos']
)
The reason for deciding to run build_py or build_ext lies in the situation when the two are run:
build_py is also run when creating source distributions, which have to be cross-platform. Compiled extensions are usually not cross-platform, so you cannot compile them in this step.
build_ext is only run when you are creating binary distributions, which are platform-specific. It is OK to compile to platform-specific machine code here.
I have a simple Python package I've created. I copied the setup.py file from the Python docs, but when I run pip install . I get the following error:
~/Projects/wolfgang (master) $ pip install . Processing /Users/Cisplatin/Projects/wolfgang
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/wolfgang.egg-info
writing pip-egg-info/wolfgang.egg-info/PKG-INFO
writing top-level names to pip-egg-info/wolfgang.egg-info/top_level.txt
writing dependency_links to pip-egg-info/wolfgang.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/wolfgang.egg-info/SOURCES.txt'
error: package directory 'wolfgang' does not exist
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/k5/grp3jdnn5jvcd14ffvqyr9z00000gn/T/pip-pWAjtJ-build/
I've tried searching online and there seem to be some similar problems on GitHub or StackOverflow, but they all apply to something being broken in the project itself as opposed to the setup.py file. My program does not mess around with any command line arguments or anything like that, so I'm not sure why this is happening.
Here's a link to the actual setup.py file I'm using, if that helps.
There is no setup.py in your repository because you removed it.
There have to be a subdirectory wolfgang. Both __init__.py and vector.py must be in the subdirectory.
I am using setuptools to build my sphinx documentation of a python project (python setup.py build_sphinx).
As found on, e.g., this site, I have configured the build process using the setup.cfg:
[build_sphinx]
source-dir = docs/source
build-dir = docs/build
all_files = 1
However, I would like to add some more options. Specifically, I would like to turn all warnings into errors, which would work with the sphinx-build command with the option -W:
sphinx-build --help
Sphinx v1.1.3
Usage: /usr/bin/sphinx-build [options] sourcedir outdir [filenames...]
Options: -b <builder> -- builder to use; default is html
-a -- write all files; default is to only write new and changed files
-E -- don't use a saved environment, always read all files
-t <tag> -- include "only" blocks with <tag>
-d <path> -- path for the cached environment and doctree files
(default: outdir/.doctrees)
-c <path> -- path where configuration file (conf.py) is located
(default: same as sourcedir)
-C -- use no config file at all, only -D options
-D <setting=value> -- override a setting in configuration
-A <name=value> -- pass a value into the templates, for HTML builder
-n -- nit-picky mode, warn about all missing references
-N -- do not do colored output
-q -- no output on stdout, just warnings on stderr
-Q -- no output at all, not even warnings
-w <file> -- write warnings (and errors) to given file
-W -- turn warnings into errors
-P -- run Pdb on exception
Modi:
* without -a and without filenames, write new and changed files.
* with -a, write all files.
* with filenames, write these.
I do not see a similar option for python setup.py build_sphinx:
python setup.py build_sphinx --help
Common commands: (see '--help-commands' for more)
setup.py build will build the package underneath 'build/'
setup.py install will install the package
Global options:
--verbose (-v) run verbosely (default)
--quiet (-q) run quietly (turns verbosity off)
--dry-run (-n) don't actually do anything
--help (-h) show detailed help message
--no-user-cfg ignore pydistutils.cfg in your home directory
Options for 'BuildDoc' command:
--fresh-env (-E) discard saved environment
--all-files (-a) build all files
--source-dir (-s) Source directory
--build-dir Build directory
--config-dir (-c) Location of the configuration directory
--builder (-b) The builder to use. Defaults to "html"
--project The documented project's name
--version The short X.Y version
--release The full version, including alpha/beta/rc tags
--today How to format the current date, used as the replacement
for |today|
--link-index (-i) Link index.html to the master doc
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
Does anyone know, if turning all warnings into errors can be achieved when building the sphinx docu with setuptools?
Edit:
The option -W is not recognized by setuptools:
python setup.py build_sphinx -W
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option -W not recognized
If instead, like me, you're using make to build your html docs with Sphinx, then you can do this to turn warnings into errors and cause make to fail:
make html SPHINXOPTS="-W"
This will cause the build to fail immediately when a warning is encountered. If you add --keep-going then the docs build will still fail but it will run to completion so you can see all the warnings. And -n will invoke the 'nit-picky' option to check for broken links. So I find this useful when building the docs in my CI framework:
make html SPHINXOPTS="-W --keep-going -n"
See here for a list of options.
In recent versions of Sphinx, you do this by adding an additional option to the section in setup.cfg:
[build_sphinx]
all-files = 1
source-dir = docs/source
build-dir = docs/build
warning-is-error = 1
Support for this was added in Sphinx 1.5, thus, this will not work with older versions.
The only solution I can manage is both simple and sub-optimal.
Change from:
python setup.py build_sphinx
to:
python -W error setup.py build_sphinx
That will turn all warnings into errors, including errors from setuptools, etc., which isn't exactly what you want, but it will stop on sphinx errors.
If you're doing this to try and set up Continuous Integration or something, maybe this is good enough?
UPDATE: See stephenfin's answer if using Sphinx 1.5+
I want to use nose.collector as a test suite for setuptools, as described here. My package's source lives in mypackage/src, and I have tests in mypackage/tests. I have a setup.py that looks like this:
import setuptools
setuptools.setup(
name='mypackage',
version='1.2.3',
package_dir={'': 'src'},
packages=setuptools.find_packages('src'),
tests_require=['nose'],
test_suite='nose.collector',
provides=setuptools.find_packages('src'),
)
However, when I run python setup.py test, it doesn't test anything:
$ python setup.py test
running test
running egg_info
writing src/mypackage.egg-info/PKG-INFO
writing top-level names to src/mypackage.egg-info/top_level.txt
writing dependency_links to src/mypackage.egg-info/dependency_links.txt
reading manifest file 'src/mypackage.egg-info/SOURCES.txt'
writing manifest file 'src/mypackage.egg-info/SOURCES.txt'
running build_ext
----------------------------------------------------------------------
Ran 0 tests in 0.002s
OK
How can I tell nose where to look for tests? Up until now, I've been doing nosetests -d tests, which works fine. But I'd like to change to use setuptools so that I can follow the python setup.py test convention.
from the docs
When running under setuptools, you can configure nose settings via the
environment variables detailed in the nosetests script usage message,
or the setup.cfg, ~/.noserc or ~/.nose.cfg config files.
http://nose.readthedocs.org/en/latest/setuptools_integration.html
Fix them with chmod -x $(find tests/ -name '*.py')
This question already has an answer here:
convert python programme to windows executable
(1 answer)
Closed 10 years ago.
I wrote a very simple program named test.py which looks like this:
print 'hello world'
then I wrote a setup program called setup.py which looks like this:
from distutils.core import setup
import py2exe
setup(console=['test.py'])
They are both in the same folder so it should work. When I run the setup.py it gives me error messages that look like this:
C:\Python26\lib\sets.py:85: DeprecationWarning: functions overriding warnings.showwarning() must support the 'line' argument
stacklevel=2)
Traceback (most recent call last):
File "C:\Users\python2.6\Desktop\program\pygametests\setup.py", line 5, in <module>
setup(console=['test.py'])
File "C:\Python26\lib\distutils\core.py", line 140, in setup
raise SystemExit, gen_usage(dist.script_name) + "\nerror: %s" % msg
SystemExit: usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: no commands supplied
I am running windows vista.
you should go to the folder where you have your files and run:
C:\Python27\mydir> python setup.py py2exe
or directly
C:\Python27\mydir> setup.py py2exe
the traceback is telling you you are not sending any command ('py2exe' in my example) for setup.py in the command line
You can see available commands with:
C:\Python27\mydir>setup.py --help-commands
Standard commands:
build build everything needed to install
build_py "build" pure Python modules (copy to build directory)
build_ext build C/C++ extensions (compile/link to build directory)
build_clib build C/C++ libraries used by Python extensions
build_scripts "build" scripts (copy and fixup #! line)
clean clean up temporary files from 'build' command
install install everything from build directory
install_lib install all Python modules (extensions and pure Python)
install_headers install C/C++ header files
install_scripts install scripts (Python or otherwise)
install_data install data files
sdist create a source distribution (tarball, zip file, etc.)
register register the distribution with the Python package index
bdist create a built (binary) distribution
bdist_dumb create a "dumb" built distribution
bdist_rpm create an RPM distribution
bdist_wininst create an executable installer for MS Windows
upload upload binary package to PyPI
check perform some checks on the package
py2exe
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
C:\Python27\mydir>