I'm having trouble getting a python package to build with python -m build .. setup.py fails on:
FileNotFoundError: [Errno 2] No such file or directory: 'requirements/requirements.txt'
It's caused by the fact that build is copying files to a temporary directory first. But it's only copying the source/, README.md, setup.py, setup.cfg. It's not copying requirements/.
For complex reasons my setup.py needs to reference other files at the root of the source repo - a directory containing multiple requirements.txt files. It's not really worth discussing why it needs to be structured this way, I've already been through that long debate with colleagues.
This works fine when we install the package through pip install -e . or as a git dependency git+ssh://... but fails when building before we push to a pypi repo.
setup.cfg
setup.py
source/
source/my_package/
requirements/
requirements/requirements.txt
requirements/some-other-requirements.txt
setup.py references this directory before calling setup().
from pathlib import Path
from setuptools import setup, find_namespace_packages
requirements_dir = Path("requirements")
# This is the line that fails:
with (requirements_dir / "requirements.txt").open() as f:
install_requires = list(f)
setup(
packages=find_namespace_packages(where="source", include=["acme_corp.*"], exclude=["tests", "tests.*"]),
package_dir={"": "source"},
install_requires=install_requires,
extras_require=optional_packages,
)
build does the correct job here. After building the source dist, it checks whether the built result can be actually installed. And without including the files under requirements into the source dist, the source dist can not be installed and is thus unusable. Try it out:
$ python -m build --sdist # builds fine, but the tar is broken:
$ cd dist
$ tar tf pkg-0.0.1.tar.gz | grep requirements # will be empty
$ pip wheel pkg-0.0.1.tar.gz # will fail
Processing ./pkg-0.0.1.tar.gz
File was already downloaded
Preparing metadata (setup.py) ... error
ERROR: Command errored out with exit status 1:
command:
...
Complete output (9 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-req-build-wfzz27_k/setup.py", line 9, in <module>
with (requirements_dir / "requirements.txt").open() as f:
File "/usr/lib64/python3.9/pathlib.py", line 1252, in open
return io.open(self, mode, buffering, encoding, errors, newline,
File "/usr/lib64/python3.9/pathlib.py", line 1120, in _opener
return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-wfzz27_k/requirements/requirements.txt'
To fix, write a MANIFEST.in alongside the setup.py that includes requirements directory in the source dist. Example file contents:
graft requirements
Building a source dist should work now; you can additionally verify that the sdist now contains all files to install from:
$ tar tf dist/pkg-0.0.1.tar.gz | grep requirements
pkg-0.0.1/requirements/
pkg-0.0.1/requirements/requirements.txt
...
This only concerns the source dist, as the wheel is built on host (your machine) already and does not contain a setup script anymore. Wheel building will also ignore the MANIFEST.in file.
Related
tosca-parser downloaded from github, when using python setup.py develop, it complains: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. So I use python setup.py sdist instead, the error is the same:
ERROR:root:Error parsing
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pbr/core.py", line 96, in pbr
attrs = util.cfg_to_args(path, dist.script_args)
File "/usr/lib/python2.7/site-packages/pbr/util.py", line 270, in cfg_to_args
pbr.hooks.setup_hook(config)
File "/usr/lib/python2.7/site-packages/pbr/hooks/__init__.py", line 25, in setup_hook
metadata_config.run()
File "/usr/lib/python2.7/site-packages/pbr/hooks/base.py", line 27, in run
self.hook()
File "/usr/lib/python2.7/site-packages/pbr/hooks/metadata.py", line 26, in hook
self.config['name'], self.config.get('version', None))
File "/usr/lib/python2.7/site-packages/pbr/packaging.py", line 874, in get_version
name=package_name))
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name tosca-parser was given, but was not able to be found.
error in setup command: Error parsing /home/tiina/tosca/tosca-parser-master/setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name tosca-parser was given, but was not able to be found.
I run the same command after git init, then the error is gone. What I don't understand is from where it requires an upstream git repository?
To disable all version calculation logic of pbr set PBR_VERSION
$ export PBR_VERSION=1.2.3
$ python setup.py sdist
or on one line
$ PBR_VERSION=1.2.3 python setup.py sdist
According to pbr docs
pbr, when run in a git repo, derives the version of a package from the git tags. When run in a tarball with a proper egg-info dir, it will happily pull the version from that. So for the most part, the package maintainers shouldn’t need to care. However, if you are doing something like keeping a git repo with the sources and the packaging intermixed and it’s causing pbr to get confused about whether its in its own git repo or not, you can set PBR_VERSION
I am working on creating a private package repository for my company and am trying to download the package from Github. I believe the package should compile, as I have uploaded this as a test function to PyPI.
The error I get is: FileNotFoundError: [Errno 2] No such file or directory: setup.py I am cloning the repo back into the exact same environment in which the function was created - what could be wrong?
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\ProgramData\Anaconda3\lib\tokenize.py", line 454, in open
buffer = _builtin_open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory:
'C:\\Users\\ALLENB~1\\AppData\\Local\\Temp\\pip-07k46aoa-build\\setup.py'
Command "python setup.py egg_info" failed with error code 1 in
C:\Users\ALLENB~1\AppData\Local\Temp\pip-07k46aoa-build\setup.py
setup.py is not part of the Python/Anaconda 3 installation location!
It's included with the package that you wish to install. Change directories to the location of the source that you download it with pip install (or with: py -m pip install) and then run the setup.py file.
EDIT:
Add #subdirectory=folder_name to your code.
There is a github code I am trying to use that is located here.
I am trying to run params.py which is a code that will take a binary file and converts it so that I can plot it (or so I think).
I tried to run:
pip install git+https://github.com/PX4/pyulog.git
However, that gave me an error:
C:\Users\Mike\Documents>pip install git+https://github.com/PX4/pyulog.git
Collecting git+https://github.com/PX4/pyulog.git
Cloning https://github.com/PX4/pyulog.git to c:\users\mike\appdata\local\temp\pip-t_vvh_b0-build
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Anaconda3\lib\tokenize.py", line 454, in open
buffer = _builtin_open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Mike\\AppData\\Local\\Temp\\pip-t_vvh_b0-build\\setup.py'
Pip install tries to install the module from :
PyPI (and other indexes) using requirement specifiers.
VCS project urls.
Local project directories.
Local or remote source archives.
When looking at the items to be installed, pip checks what type of item each is, in the following order:
Project or archive URL.
local directory (which must contain a setup.py, or pip will report an error).
Local file (a sdist or wheel format archive, following the naming conventions for those formats).
A requirement, as specified in PEP 440.
In your case, git repo doesn't meet the requirement. It doesn't have setup.py that's why you get the error.
Instead try cloning the repo on your local machine.
The setup.py file in a Python package I've sent to pip:
#!/usr/bin/env python
from distutils.core import setup
setup(
#......
long_description=open('README.md').read(),
#....
)
The file README.md exists. When put a breakpoint in setup.py and execute it locally, it reads the file well. However, when I install it from pip (pip install my_lib), it throws an exception during the installation that it's not found:
File "/private/var/folders/ty/0nvksfhn29z_cjb6md2t3x8c0000gn/T/pip_build_alex/my_app123/setup.py", line 14, in <module>
long_description=open('README.md').read(),
IOError: [Errno 2] No such file or directory: 'README.md'
Complete output from command python setup.py egg_info:
UPDATE:
I just downloaded my library from pip, unzipped and discovered that the file README, LICENCE, MANIFEST aren't in it. And they're in gitignore either because they exist at github.
I needed to create MANIFEST.in with the following content:
include README.md
include LICENSE.txt
I uploaded my package on pypi using this guide.
But it seems that there is an error with this line in setup.py
long-description=open(os.path.join(os.path.dirname(__file__),'README.md')).read()
which is giving me, on trying to install via pip
IO Error no such file or directory.
So how can I fix this? should a simple open('README.md')?
Is the long-description line really needed when I already have this in my setup.cfg
[metadata]
description-file = README.md
You need to add 'include README.md' in MANIFEST.in, if you do not have this file (the Manifest one), create it.
Just execute this on your repository root directory (where the setup.py file is located)
echo "include README.md" >> MANIFEST.in
Cheers.