`setup.py test` cannot find package in own repo - python

I'm trying to run tests with my setup.py as usually with python setup.py test with appropriate setup.py configuration. It was ok until I added a package from my own repository.
I can easily install it with pip install my_package, having the --extra-index-url in /etc/pip.conf or credentials in ~/.pypirc. I also entered the url in dependency_links in the setup.py file!
But when I run it now, I see it knows about the module, but it is unable to download it!
(testpy3) honza#mycomp:~/.../xxxx_module$ python setup.py test
running pytest
Searching for package_yyyy
Reading https://specimen:specimen#repo.xxxxx.net/repository/pypi-private/simple
Reading https://pypi.python.org/simple/package_yyyy/
Reading https://pypi.python.org/simple/package_yyyy-grpc/
Couldn't find index page for 'package_yyyy' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
No local packages or working download links found for package_yyyy
error: Could not find suitable distribution for Requirement.parse('package_yyyy')
In fact I want to use this in docker, but does not work locally nor in docker. Any ideas, please?

Related

Python build error - Could not find a version that satisfies the requirement wheel

I am trying to build python wheel following the instructions as described in the link below. I am doing this first time.
https://packaging.python.org/en/latest/tutorials/packaging-projects/
I set up the folder structure, files and all. I have added this in pyproject.toml file.
[build-system]
requires= ["setuptools>=57.4.0","wheel>=0.37.1"]
build-backend = "setuptools.build_meta"
I have installed setuptoos and wheel on my virtual environment.
When I tried to run the build command, I am getting an SSL warnings and below error.
Could not find a version that satisfies the requirement wheel>=0.37.1
Could not fetch from URL https://pypi.org/simple
Even though have installed setuptools and wheel on my virtual environment, I think it is hitting the pypi to find and download these packages.
I don't know how the build module finds the modules/packages in "requires". I am not finding a way to direct to use the already installed setuptools and wheel on my machine instead of fetching from pypi.
Even if it tries to doanload again, how can derect to use our artifactory instead of pypi.
Any help in this is greatly appreciated.
I tried all below with differenct combinations but did not work. Obviously I am missing something.
1.
I added a pip.ini in my virtual environment (Lib\site-packages\pip).
Added the index-url with our organization's artifactory url.
Added trusted-host
Also tried pip.config
I downloaded the wheels for setuptools and wheel.
Added another argument in pyproject.toml
[easy-install]
find-links = c:\wheels
Added the wheels directly in the src folder.
Thanks.

pip install package still not following dependency links

I'm having serious trouble with using setup.py to pip install my package which also has dependency links. I have read this answer and this one thoroughly and none of the answers including the accepted ones help.
Here is the setup.py for the package trying to install.
Basically, it reads the requirements.txt to fill install_requires and dependency_links, most of the rest of the code is boilerplate from cookie-cutter. requirements.txt has a private github repo in it which is causing the issues. e.g. git+https://${GITHUB_OAUTH_TOKEN}#github.com/jmerkow/pripy.git#egg=pripy
When I run pip install -r requirements.txt everything works great, it installs the private repository. However, if I try to install using pip install . --process-dependency-links, I get this error:
Could not find a version that satisfies the requirement pripy (from mypackage==<sha>) (from versions: )
No matching distribution found for pripy (from mypackage==<sha>)
If I take off the #egg=xxx from the link in requirements, the private repo package is completely ignored by pip install . but not by pip install -r requirements.txt.
I have confirmed that dependency_links contains 'git+https://<actual-token>#github.com/jmerkow/pripy.git#egg=pripy' and that install_requires includes 'pripy'
How do you get setup to properly Is this a problem with the sub-package? setup.py in that repo is done pretty much the same, except there are no private links.
Ugh, this always happens. I put in all the work to the question, then I figure it out myself.
The issue is two things, first, all dependeny_links need to have a version, second to pull the version from the requirements file properly you need to do some magic on the string.
Comparing to the above setup.py, I changes the way requirements are added to the two lists (updated here). Then add the version to #egg=xxx on the link e.g.
git+https://${GITHUB_OAUTH_TOKEN}#github.com/jmerkow/pripy.git#egg=pripy-0.
Now setup.py will parse that file, take the egg version info, convert it to a pip version (basically replace the first '-' with an '==') for the install_requires, and you're good to go.

Error: setup script specifies an absolute path: .gitignore

On Ubuntu 16.04.4, I suspect some recent update of some Python system package of having broken my Python 2.7 configuration. Whatever package I try to install or reinstall with a basic sudo python setup.py install it always fails because of gitignore:
running install
running bdist_egg
running egg_info
[...]
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
error: Error: setup script specifies an absolute path:
/home/me/some_repo/.gitignore
setup() arguments must *always* be /-separated paths relative to the
setup.py directory, *never* absolute paths.
Lately I found a temporary workaround by manually cleaning the /usr/local/lib/python2.7/dist-packages/some_package directory manually before installing some_package. However tonight I'm facing the same issue on another repository and it keeps failing whatever I clean up. I also tried cleaning all compiled folders .egg-info/ build/ dist/ without success.
Notes: The setup script does not actually specify an absolute path to gitignore. An example of failing repo is https://github.com/philchristensen/python-artnet/blob/master/setup.py This repo has a setuptools_git entry which might lead to a clue, but other packages without this git entry point also fail because of the gitignore while a couple of months ago I never faced such issue with the same repos. Deleting the gitignore causes the setup to fail because of another non-py local file.
Any clue?
It looks like some other package that I had previously installed broke my system-wide Python.
Here's how I fixed in order to install package xyz:
Browsed /usr/local/lib/python2.7/dist-packages in search for occurences of "gitignore"
Deleted folders of all matching occurences (including setuptools_git itself that matches "gitignore")
pip install setuptools_git
In package xyz, rm -rf dist/ build/ *.egg-info/
Reinstall package xyz, which now succeeds to install
Use virtual envs as a lesson

Python pip install gives “Command ”python setup.py egg_info“ failed with error code 1”

Edit: Yes I know this question already exists, except my question is a bit different and none of the solutions fixed it.
I do most of my Python stuff when I'm at work and not on my personal machine, but I decided to install it on my personal computer as well. I fresh installed python 3.6.1, and created a virtual environment with virtualenv. Then within the virtualenv I tried to pip install urllib (or any module) and I received the error:
(pdbot) C:\Users\user\Documents\pdbot>pip install urllib
Collecting urllib
Using cached urllib-1.21.1.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\setup.py", line 191
s.connect((base64.b64decode(rip), 017620))
^
SyntaxError: invalid token
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\
I read elsewhere that this error had something to do with setuptools not being properly installed. So I ran this to attempt to fix the issue:
easy_install -U setuptools
I ended up receiving an even weirder error next:
(pdbot) C:\Users\zeke\Documents\pdbot>easy_install -U setuptools
Searching for setuptools
Reading https://pypi.python.org/simple/setuptools/
Downloading https://pypi.python.org/packages/a9/23/720c7558ba6ad3e0f5ad01e0d6ea2288b486da32f053c73e259f7c392042/setuptools-36.0.1.zip#md5=430eb106788183eefe9f444a300007f0
Best match: setuptools 36.0.1
Processing setuptools-36.0.1.zip
Writing C:\Users\zeke\AppData\Local\Temp\easy_install-jhg1val_\setuptools-36.0.1\setup.cfg
Running setuptools-36.0.1\setup.py -q bdist_egg --dist-dir C:\Users\zeke\AppData\Local\Temp\easy_install-jhg1val_\setuptools-36.0.1\egg-dist-tmp-8apak7kn
warning: no files found matching '*' under directory 'setuptools\_vendor'
Copying setuptools-36.0.1-py3.6.egg to c:\users\zeke\documents\pdbot\lib\site-packages
Adding setuptools 36.0.1 to easy-install.pth file
Installing easy_install-script.py script to c:\users\zeke\documents\pdbot\Scripts
Installing easy_install.exe script to c:\users\zeke\documents\pdbot\Scripts
error: [WinError 5] Access is denied: 'c:\\users\\zeke\\documents\\pdbot\\Scripts\\easy_install.exe'
This looks like a permissions error, but I ran these both in an administrator command prompt (Windows 10) and got the same result. I am the only user on this computer and I have all admin permissions. Is this virtualenv causing an issue? How do I remedy it?
EDIT: I was able to fix the permissions issue by leveraging the python executable like so:
python -m easy_install -U setuptools
But it didn't fix the python setup.py egg_info issue. I still get this error message when trying to pip install anything:
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\
I have tried both python -m pip install urllib and pip install urllib and neither work.
I had the same problem when trying to install urllib, but after doing a pip search urllib, I discovered that the problem was due to the version of urllib. From the search:
$ pip search urllib
...
> urllib5 (5.0.0) - Just increment the number and create a new lib. Never fix the original one.
At the end, a simple
pip install urllib5
within an elevated shell solved it.
Your problem has to do with permissions. The related/similar tools setup_tools, easy_install, and pip all tend to set a default set of permissions on files and folders they try to create in the package installation folder(s), rather than trying to match access permissions of the location they're installing in.
On Linux systems, where files and folders individually have permissions, this is frequently bypassed with the sudo command. On Windows, the equivalent is to run the installer as an Administrator. Since you're in the console, you have to open a console with Administrator privileges to run the pip command in.
Notable under Windows, the modules installed with pip from an Administrator console are still accessible to all users of the system that have the proper path in the PYTHONPATH system environment variable. Under Linux however, the problem is exacerbated by the fact that the files themselves may not be created with read and execute access for other users and may need to have their permissions manually modified after installation.
WARNING: urllib vs urllib2 vs urllibx
Both other answers claim that the problem is you're not specifying the correct "version" of the module in the call to pip. Neither is correct, as the error clearly indicates an installation folder access permissions violation causing the failure, but they also incorrectly recommended VERY unsafe behavior.
pip install urllib != pip install urllib5 these are two completely different packages.
The documentation for pip (https://packaging.python.org/tutorials/installing-packages/#id17) clearly says the way to specify a module version explicitly is pip install 'urllib==5'.
As part of how the package management engine implemented by pip works, running the command pip install urllib will always try to use the latest version of the urllib package, so you shouldn't need to specify the version unless you have some reason that you need a very specific version of the module.
There are two points to make in order to answer your question:
1. You are lucky you did not install that package!
The package you were trying to install was a maliciously created python package that was designed to look like a real package (in this case urllib3). If you had installed it, the package would have operated as normal except it would have sent some basic information about the system on which you installed the package to a URL (you can see more details on this here). You can read more about this fake package at either of the following links:
https://app.threatconnect.com/auth/incident/incident.xhtml?incident=5256822&owner=Common%20Community (you can sign up for a free account to view this one)
http://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/index.html
Sending basic information about your systems to an unknown source isn't the worst thing you could do, but is certainly something you want to avoid when possible.
2. To properly install a package...
Specifically urllib:
To install urllib, you need to specify the version of the package you would like to install. For example, pip install urllib3.
Any package in general:
As #Elisabete Coelho suggested, you can use the pip search <package-name> feature to view the available packages. This is not perfect, however, as it may list malicious libraries like the one you were trying to install. A good guideline is that you should follow the installation instructions in a package's documentation closely to avoid any unforeseen issues. This is just an unfortunate necessity of living in a world where people make pretend python packages.

PyPi versioning

Newbie here. I've created my first Python package and I managed to register it on Pypi, as well as upload the tar.gz file. Now whenever I want to run:
pip install myPackage
I get this error in console:
Could not find a version that satisfies the requirement myPackage (from versions: 1.0dev)
No distributions matching the version for flashCardStudy
Storing debug log for failure in /Users/xxx/Library/Logs/pip.log
I believe this is because my version is development version I guess? So yeah, I can install it by adding --pre argument but what I'd really like is to turn it into a normal version so to speak.
I've tried figuring out how to do it and looking at some docs but I can't still figure it out. In my setup.py my version is set to '1.0' so I don't see where to problem is. If anyone wants to have a look at the file, here it is.
So I found the problem. I used utility called Paster which generates package structure, including setup.py and setup.cfg files among others. My utility hasn't been updated in a while and meanwhile submission rules to PyPi have changed. It now requires certain setup.py structure and unless it passes via pip, it's labeled as development version - which pip does not install without --pre argument.
So I just went to PyPi pages and looked at setup.py tutorial, did it their way and now it works.

Categories