According to Dependency section in the setuptools manual git repository URLs can be specified in the dependency_links argument to setup with git+URL. Yet,
cd /tmp
mkdir py-test
cd py-test
touch __init__.py
and creation of a setup.py file with
from setuptools import setup, find_packages
from pkg_resources import parse_version
setup(
name = "py-test",
version = "1.0",
packages = ["."],
dependency_links = [
"git+https://github.com/wxWidgets/wxPython.git"
],
install_requires = ["wxPython"],
)
causes the error Download error on git+https://github.com/wxWidgets/wxPython.git: unknown url type: git+https -- Some packages may not be found! when I run python setup.py build && sudo setup.py install.
The installation of the package python-setuptools-git doesn't help.
I'm using setuptools 18.2 with python 2.7 on Ubuntu 15.04.
From the setuptools docs:
In the case of a VCS checkout, you should also append #egg=project-version in order to identify for what package that checkout should be used
So the fix is just to append the #egg=wxPython fragment onto the end:
dependency_links = [
"git+https://github.com/wxWidgets/wxPython.git#egg=wxPython"
]
Related
from distutils.core import setup
import setuptools
setup(name = 'my_project_name',
version = '0.0.1',
description = 'My project',
py_modules = ['main'],
packages = ['generated'],
python_requires = '>=3.5',
install_requires = [
'requests>=2.20.0',
'grpcio>=1.48.2',
'grpcio-tools>=1.48.2' ],
)
Now I do python setup.py sdist and it successfully builds package and places in dist/my_project_name-0.0.1.tar.gz in the current directory.
However when I install it with pip install dist/my_project_name-0.0.1.tar.gz, it does install the package in $HOME/.local/lib/python3.6/site-packages (which is fine, I don't run it as root), but in two different pieces:
$HOME/.local/lib/python3.6/site-packages/main.py
$HOME/.local/lib/python3.6/site-packages/generated/*
I was expecting that both main.py and generated/ will go under lib/python3.6/site-packages/my_project_name/. Is there a way to do what I want, or this is a python way?
Will appreciate helpful advices!
I've read a discussion where a suggestion was to use the requirements.txt inside the setup.py file to ensure the correct installation is available on multiple deployments without having to maintain both a requirements.txt and the list in setup.py.
However, when I'm trying to do an installation via pip install -e ., I get an error:
Obtaining file:///Users/myuser/Documents/myproject
Processing /home/ktietz/src/ci/alabaster_1611921544520/work
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory:
'/System/Volumes/Data/home/ktietz/src/ci/alabaster_1611921544520/work'
It looks like pip is trying to look for packages that are available on pip (alabaster) on my local machine. Why? What am I missing here? Why isn't pip looking for the required packages on the PyPi server?
I have done it before the other way around, maintaining the setup file and not the requirements file. For the requirements file, just save it as:
*
and for setup, do
from distutils.core import setup
from setuptools import find_packages
try:
from Module.version import __version__
except ModuleNotFoundError:
exec(open("Module/version.py").read())
setup(
name="Package Name",
version=__version__,
packages=find_packages(),
package_data={p: ["*"] for p in find_packages()},
url="",
license="",
install_requires=[
"numpy",
"pandas"
],
python_requires=">=3.8.0",
author="First.Last",
author_email="author#company.com",
description="Description",
)
For reference, my version.py script looks like:
__build_number__ = "_LOCAL_"
__version__ = f"1.0.{__build_number__}"
Which Jenkins is replacing the build_number with a tag
This question consists of two separate questions, for the rather philosopihc choice of how to arrange setup requirements is actually unrelated to the installation error that you are experiencing.
First about the error: It looks like the project you are trying to install depends on another library (alabaster) of which you apparently also did an editable install using pip3 install -e . that points to this directory:
/home/ktietz/src/ci/alabaster_1611921544520/work
What the error tells you is that the directory where the install is supposed to be located does not exist anymore. You should only install your project itself in editable mode, but the dependencies should be installed into a classical system directory, i. e. without the option -e.
To clean up, I would suggest that you do the following:
# clean up references to the broken editable install
pip3 uninstall alabaster
# now do a proper non-editable install
pip3 install alabaster
Concerning the question how to arrange setup requirements, you should primarily use the install_requires and extras_require options of setuptools:
# either in setup.py
setuptools.setup(
install_requires = [
'dep1>=1.2',
'dep2>=2.4.1',
]
)
# or in setup.cfg
[options]
install_requires =
dep1>=1.2
dep2>=2.4.1
[options.extras_require]
extra_deps_a =
dep3
dep4>=4.2.3
extra_deps_b =
dep5>=5.2.1
Optional requirements can be organised in groups. To include such an extra group with the install, you can do pip3 install .[extra_deps_name].
If you wish to define specific dependency environments with exact versions (e. g. for Continuous Integration), you may use requirements.txt files in addition, but the general dependency and version constraint definitions should be done in setup.cfg or setup.py.
This is an extension of SO setup.py ignores full path dependencies, instead looks for "best match" in pypi
I am trying to write setup.py to install a proprietary package from a .tar.gz file on an internal web site. Unfortunately for me the prop package name duplicates a public package in the public PyPI, so I need to force install of the proprietary package at a specific version. I'm building a docker image from a Debian-Buster base image, so pip, setuptools and tox are all freshly installed, the image brings python 3.8 and pip upgrades itself to version 21.2.4.
Solution 1 - dependency_links
I followed the instructions at the post linked above to put the prop package in install_requires and dependency_links. Here are the relevant lines from my setup.py:
install_requires=["requests", "proppkg==70.1.0"],
dependency_links=["https://site.mycompany.com/path/to/proppkg-70.1.0.tar.gz#egg=proppkg-70.1.0"]
Installation is successful in Debian-Buster if I run python3 setup.py install in my package directory. I see the proprietary package get downloaded and installed.
Installation fails if I run pip3 install . also tox (version 3.24.4) fails similarly. In both cases, pip shows a message "Looking in indexes" then fails with "ERROR: Could not find a version that satisfies the requirement".
Solution 2 - PEP 508
Studying SO answer pip ignores dependency_links in setup.py which states that dependency_links is deprecated, I started over, revised setup.py to have:
install_requires=[
"requests",
"proppkg # https://site.mycompany.com/path/to/proppkg-70.1.0.tar.gz#egg=proppkg-70.1.0"
],
Installation is successful in Debian-Buster if I run pip3 install . in my package directory. Pip shows a message "Looking in indexes" but still downloads and installs the proprietary package successfully.
Installation fails in Debian-Buster if I run python3 setup.py install in my package directory. I see these messages:
Searching for proppkg# https://site.mycompany.com/path/to/proppkg-70.1.0.tar.gz#egg=proppkg-70.1.0
..
Reading https://pypi.org/simple/proppkg/
..
error: Could not find suitable distribution for Requirement.parse(...).
Tox also fails in this scenario as it installs dependencies.
Really speculating now, it almost seems like there's an ordering issue. Tox invokes pip like this:
python -m pip install --exists-action w .tox/.tmp/package/1/te-0.3.5.zip
In that output I see "Collecting proppkg# https://site.mycompany.com/path/to/proppkg-70.1.0.tar.gz#egg=proppkg-70.1.0" as the first step. That install fails because it fails to import package requests. Then tox continues collecting other dependencies. Finally tox reports as its last step "Collecting requests" (and that succeeds). Do I have to worry about ordering of install steps?
I'm starting to think that maybe the proprietary package is broken. I verified that the prop package setup.py has requests in its install_requires entry. Not sure what else to check.
Workaround solution
My workaround is installing the proprietary package in the docker image as a separate step before I install my own package, just by running pip3 install https://site.mycompany.com/path/to/proppkg-70.1.0.tar.gz. The setup.py has the PEP508 URL in install_requires. Then pip and tox find the prop package in the pip cache, and work fine.
Please suggest what to try for the latest pip and tox, or if this is as good as it gets, thanks in advance.
Update - add setup.py
Here's a (slightly sanitized) version of my package's setup.py
from setuptools import setup, find_packages
def get_version():
"""
read version string
"""
version_globals = {}
with open("te/version.py") as fp:
exec(fp.read(), version_globals)
return version_globals['__version__']
setup(
name="te",
version=get_version(),
packages=find_packages(exclude=["tests.*", "tests"]),
author="My Name",
author_email="email#mycompany.com",
description="My Back-End Server",
entry_points={"console_scripts": [
"te-be=te.server:main"
]},
python_requires=">=3.7",
install_requires=["connexion[swagger-ui]",
"Flask",
"gevent",
"redis",
"requests",
"proppkg # https://site.mycompany.com/path/to/proppkg-70.1.0.tar.gz#egg=proppkg-70.1.0"
],
package_data={"te": ["openapi_te.yml"]},
include_package_data=True, # read MANIFEST.in
)
Hello Denizens of the Exchange of Stackness,
I have a library that I'm trying to distribute. I have created a setup.py and run
python setup.py sdist
I see that it creates a .tar.gz file under the dist/ directory, which has all my files and folders that I want in it. However, when I install it on a Windows 8 box (running Python 3.6.5rc1), I don't get any files- only a kivydnd-0.5.dist-info directory. When I install it on Linux (running Fedora 26, Python 2.7), I do see the package's files but I don't see the examples directory.
Can you tell me what I'm doing wrong?
The setup.py is here:
from setuptools import setup, find_packages
setup(
name='kivydnd',
version='0.5',
description='Kivy Drag-n-Drop for Widgets',
url='https://github.com/GreyGnome/KivyDnD',
author='GreyGnome',
author_email='myemail#example.com',
license='Apache License 2.0',
#packages=find_packages('kivydnd'),
packages=['kivydnd'],
zip_safe=False,
scripts=[
'examples/dndexample1.py',
'examples/dndexample2.py',
'examples/dndexample3.py',
'examples/dndexample_copy_draggable.py',
'examples/dndexample_drop_groups.py',
'examples/dndexample_relative_layout.py',
'examples/example_base_classes.py',
'examples/example_base_classes.pyc',
]
)
In my development directory, I perform:
python setup.py sdist
The resulting .tar.gz looks like this; this will also reflect the structure of the directory where I'm doing my development:
kivydnd-0.5/
kivydnd-0.5/setup.py
kivydnd-0.5/PKG-INFO
kivydnd-0.5/examples/
kivydnd-0.5/examples/example_base_classes.pyc
kivydnd-0.5/examples/dndexample1.py
kivydnd-0.5/examples/dndexample_copy_draggable.py
kivydnd-0.5/examples/dndexample3.py
kivydnd-0.5/examples/dndexample_relative_layout.py
kivydnd-0.5/examples/dndexample_drop_groups.py
kivydnd-0.5/examples/dndexample2.py
kivydnd-0.5/examples/example_base_classes.py
kivydnd-0.5/README.md
kivydnd-0.5/RELEASE_NOTES.md
kivydnd-0.5/LICENSE
kivydnd-0.5/kivydnd.egg-info/
kivydnd-0.5/kivydnd.egg-info/top_level.txt
kivydnd-0.5/kivydnd.egg-info/PKG-INFO
kivydnd-0.5/kivydnd.egg-info/not-zip-safe
kivydnd-0.5/kivydnd.egg-info/SOURCES.txt
kivydnd-0.5/kivydnd.egg-info/dependency_links.txt
kivydnd-0.5/setup.cfg
kivydnd-0.5/MANIFEST.in
kivydnd-0.5/kivydnd/
kivydnd-0.5/kivydnd/dnd_storage_singletons.py
kivydnd-0.5/kivydnd/debug_print.py
kivydnd-0.5/kivydnd/__init__.py
kivydnd-0.5/kivydnd/dropdestination.py
kivydnd-0.5/kivydnd/dragndropwidget.py
Here is what happens on Windows 8:
F:\>pip install kivydnd-0.5.tar.gz
Processing f:\kivydnd-0.5.tar.gz
Building wheels for collected packages: kivydnd
Running setup.py bdist_wheel for kivydnd ... done
Stored in directory: C:\Users\schwager\AppData\Local\pip\Cache\wheels\9a\11\cd
\68bfb0d34c7b73ec7e25c6f9c40c5926377747b5951ac2e6ab
Successfully built kivydnd
Installing collected packages: kivydnd
Successfully installed kivydnd-0.5
` c:\users\schwager\python\Lib\site-packages\kivydnd-0.5.dist-info\` I have:
DESCRIPTION.rst
INSTALLER
METADATA
metadata.json
RECORD
top_level.txt
WHEEL
Here is what happens on Linux:
pip install --target=/home/schwager/lib/python kivydnd-0.5.tar.gz
Processing ./kivydnd-0.5.tar.gz
Installing collected packages: kivydnd
Running setup.py install for kivydnd ... done
Successfully installed kivydnd-0.5
You are using pip version 9.0.1, however version 9.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ ls /home/schwager/lib/python
kivydnd kivydnd-0.5-py2.7.egg-info
$ ls -R /home/schwager/lib/python
/home/schwager/lib/python:
kivydnd kivydnd-0.5-py2.7.egg-info
/home/schwager/lib/python/kivydnd:
debug_print.py dnd_storage_singletons.py dragndropwidget.py dropdestination.py __init__.py
debug_print.pyc dnd_storage_singletons.pyc dragndropwidget.pyc dropdestination.pyc __init__.pyc
/home/schwager/lib/python/kivydnd-0.5-py2.7.egg-info:
dependency_links.txt installed-files.txt not-zip-safe PKG-INFO SOURCES.txt top_level.txt
I appears that my setup.py should look like this. The package will get installed under Python's site-packages directory, the examples under <path-to-share>/kivydnd-examples.
from setuptools import setup, find_packages
from codecs import open
from os import path
with open(path.join('.', 'README.md'), encoding='utf-8') as f:
long_description = f.read()
setup(
name='kivydnd',
version='0.5.0',
description='Kivy Drag-n-Drop for Widgets',
long_description=long_description,
long_description_content_type='text/markdown',
url='https://github.com/GreyGnome/KivyDnD',
author='GreyGnome',
author_email='myemail#example.com',
license='Apache License 2.0',
keywords='kivy drag-n-drop',
packages=find_packages(exclude=[]),
data_files=[('share/kivydnd-examples',
[
'examples/dndexample1.py',
'examples/dndexample2.py',
'examples/dndexample3.py',
'examples/dndexample_copy_draggable.py',
'examples/dndexample_drop_groups.py',
'examples/dndexample_relative_layout.py',
'examples/example_base_classes.py',
'examples/example_base_classes.pyc',
]
)],
)
I am trying to install hw3 package which has a dependency package hw2. My setup.py looks as follows -
setup(
name='hw3',
version='0.1',
packages = find_packages(),
install_requires = 'hw2',
dependency_links = [
r'svn+https://server.local/svn/Libraries/testPkg2/trunk#egg=hw2'
]
)
I get the following error when I run python setup.py install in windows cmd
svn: E170013: Unable to connect to a repository at URL 'svn+https://server.local/svn/Libraries/testPkg2/trunk'
svn: E125002: Undefined tunnel scheme 'https'
Alternatively, I have requirements.txt which is as follows
svn+https://server.local/svn/Libraries/testPkg2/trunk#egg=hw2
If I run pip install -r requirements.txt, it installs hw2 package successfully.
My svn version is
svn, version 1.9.7 (r1800392) compiled Aug 8 2017, 22:14:48 on
x86-microsoft-windows
how to resolve this error? Thanks
I am getting the same error for 'http' and 'svn'.
For 'ssh' it is
svn: E170012: Can't create tunnel
svn: E720002: Can't create tunnel: The system cannot find the file specified.
Maybe try it directly with the install_requires option (requires pip>=18.1):
setup(
name='hw3',
version='0.1',
packages = find_packages(),
install_requires = ['hw2#svn+https://server.local/svn/Libraries/testPkg2/trunk#egg=hw2'],
)
See also this answer to a related question https://stackoverflow.com/a/54216163/13835019.