The situation I'm trying to resolve is installing a package from a private repository on bitbucket which has it's own dependency on another private repository in bitbucket.
I use this to kick off the install:
pip install -e git+https://bitbucket.org/myuser/project-one.git/master#egg=django_one
which then attempts to download it's dependencies from setup.py that look like:
install_requires = ['project-two',],
dependency_links = ['git+https://bitbucket.org/myuser/project-two.git/master#egg=project_two'],
This fails, the pip log looks like:
Downloading/unpacking project-two (from project-one)
Getting page https://pypi.python.org/simple/project-two/
Could not fetch URL https://pypi.python.org/simple/project-two/: HTTP Error 404: Not Found (project-two does not have any releases)
Will skip URL https://pypi.python.org/simple/project-two/ when looking for download links for project-two (from project-one)
Getting page https://pypi.python.org/simple/
URLs to search for versions for project-two (from project-one):
* https://pypi.python.org/simple/project-two/
* git+https://bitbucket.org/myuser/project-two.git/master#egg=project-two
Getting page https://pypi.python.org/simple/project-two/
Cannot look at git URL git+https://bitbucket.org/myuser/project-two.git/master#egg=project-two
Could not fetch URL https://pypi.python.org/simple/project-two/: HTTP Error 404: Not Found (project-two does not have any releases)
Will skip URL https://pypi.python.org/simple/project-two/ when looking for download links for project-two (from project-one)
Skipping link git+https://bitbucket.org/myuser/project-two.git/master#egg=project-two; wrong project name (not project-two)
Could not find any downloads that satisfy the requirement project-two (from project-one)
The curious thing about this setup is, if I take a clone of project-one and run
python setup install
from there, project-two is fetched from bitbucket and installed into my virtualenv. My understanding was that pip was using setup tools under the hood, so my assumption was the success of that test validated my approach.
Any suggestions appreciated.
FOLLOW UP:
So the accepted answer is quite right - but my problem had the additional complexity of being a private repo (https + http auth-basic). Using the syntax
dependency_links=["http://user:password#bitbucket.org/myuser/..."]
still caused a 401. Running up a shell and using pip.download.py to run urlopen demonstrates the underlying problem (ie pip needs additional setup in urllib2 to get this working).
The problem is mentioned here but I couldn't get that working.
pip created the idea of a VCS installation, so you can use git+https://path/to/repo.git, but setuptools does not understand that.
When you create a setup.py file you are using only setuptools (no pip involved), and setuptools does not understand that kind of URL.
You can use dependency_links with tarballs or zip files, but not with git repositories.
Replace your depencency_links by:
dependency_links=["https://bitbucket.org/myuser/project-two/get/master.zip#egg=project-two"]
And check if it works.
There is a similar question at https://stackoverflow.com/a/14928126/565999
References:
http://peak.telecommunity.com/DevCenter/setuptools#dependencies-that-aren-t-in-pypi
Related
I am trying to add the Haystack library as a dependency of a python project. The installation instructions that Haystack provides are as follows
git clone https://github.com/deepset-ai/haystack.git
cd haystack
pip install -e .[all]
I am trying to translate this into a single line that I can include in a requirements.txt. My current best guess is
farm-haystack[all] # git+https://github.com/deepset-ai/haystack.git
However this emits a bunch of warnings that various versions of farm-haystack don't provide the desired extras, such as
WARNING: farm-haystack 0.1.0.post2 does not provide the extra 'ray'
before failing with the error message
ERROR: Requested dill from https://files.pythonhosted.org/packages/3e/ad/31932a4e2804897e6fd2f946d53df51dd9b4aa55e152b5404395d00354d1/dill-0.3.1.tar.gz#sha256=d3ddddf2806a7bc9858b20c02dc174396795545e9d62f243b34481fd26eb3e2c (from farm-haystack[all]# git+https://github.com/deepset-ai/haystack.git->-r /dss_data/tmp/pip-requirements-install/req3361828774079305889.txt (line 1)) has different version in metadata: '0.3.1.dev0'
What is the proper way to go about doing this?
Since you mention it would be a dependency of another project, the format for listing Haystack in your requirements.txt should be the following (I pin the version here but it's not mandatory):
farm-haystack[all]==1.5.0
If you want to pin a specific git commit instead, the line in your requirements.txt should just be:
git+https://github.com/deepset-ai/haystack.git#egg=farm-haystack[all]
I am currently trying to install many different libraries into my Python environment. I've currently tried using a virtual environment and conda environment still resulting in similar errors.
As I'm doing this on a VM, I have to go through my corporate proxy, hence my pip install command looks a little verbose like the following:
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org h5py --proxy http://user:password#proxy.internet.mycompany.local:8080
This had been working fine to download many different packages (including ones that are currently failing now), I'm not too sure what has changed, but now the command results in the following error:
Collecting h5py
Downloading h5py-3.1.0-cp37-cp37m-win_amd64.whl
\ 4.8 kB ...
ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the >package versions
, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered >with them
.
h5py from https://files.pythonhosted.org/packages/53/c2/77bd81922264520b492bd7bfd1a51a845bc1187445408a7a83db284fd566/h5py-3.1.0-cp37-cp37m-win_amd64.whl#sha256=02c391fdb980762a1cc03a4bcaecd03dc463994a9a63a02264830114a96e111f
:
Expected sha256 02c391fdb980762a1cc03a4bcaecd03dc463994a9a63a02264830114a96e111f
Got 3fbaf847f810d5f6970abb0c7754abcca7dea17d315036fe5e4b26f427adf3a6
Although this error will still stay largely the same, the "Got" sha256 will change on each attempt, i.e. the following for the last returned line:
Got ff8197d74c233317597b89612691bf420806046093628ac02f0a92df3ac8123a
Got 8be06806b22d49aaa9b94113d8b3e075539303da96386971600c21cb8b9f8460
To rectify this error I've attempted the following:
using '--no-cache-dir' as suggested other Stack Overflow answers to similar questions
reinstalling python/pip or trying other versions of Python.
trying to use miniconda/anaconda instead
contacting owner of libraries about the issue (although I'm fairly sure it isn't library specific as I can't download any packages)
attempted to access the link in the error output: https://files.pythonhosted.org/packages/53/c2/77bd81922264520b492bd7bfd1a51a845bc1187445408a7a83db28%204fd566/h5py-3.1.0-cp37-cp37m-win_amd64.whl#sha256=02c391fdb980762a1cc03a4bcaecd03dc463994a9a63a02264830114a96e111f. This results in the following page response - the console section of developer tools states the following error: Failed to load resource: the server responded with a status of 404 (Not Found)
I've found a few similar issues to this already on Stack Overflow, but I've found nothing that resolves this.
I am trying test a python package I want to release using test test.pypi.
In the setup.py file I have install_requires=['numpy>=1.15','scipy>=0.0','scikit-learn>=0.2','numba>=0.0'],
Scipy and Numpy get downloaded and install as expected.
I get the following error: ERROR: Could not find a version that satisfies the requirement numba>=0.0
As a note, if I do pip install numba before my test package it will work, but I am trying to make the package work correctly.
I notice when it does the scipy requirement first, it shows Downloading https://test-files.pythonhosted.org/packages/68/72/eb962a3ae2755af6b1f31f7a94dccc21bfc41bb1637c5877a043e711b1d7/scipy-0.1.tar.gz .
So from the url, it seems like it is using test-files, but is this the regular pypi or just the test one?
My question is: what is the appropriate way to write the install_requires so I can make sure the test works before putting it on the actual pypi site?
There is nothing wrong with your syntax, it is just that unlike scipy, numpy, and scikit-learn, there is no numba hosted on the test PyPI instance. Compare:
https://pypi.org/project/numba/ <-- 200 OK
https://test.pypi.org/project/numba/ <-- 404 Not Found
(Note: that 404 was true at the time of writing this answer, but it appears as though a version of numba has now been uploaded to the test index, on Feb 9th 2021)
My question is: what is the appropriate way to write the install_requires so I can make sure the test works before putting it on the actual pypi site?
How you wrote install_requires is OK. To smoke test it, by uploading your package to test PyPI and checking that it installs properly, use the test PyPI as an extra index url, rather than as a replacement --index-url:
pip install yourpackage --extra-index-url=https://test.pypi.org/simple/
This way yourpackage can be found in test PyPI but the install requirements such as numba can still be resolved on main PyPI.
Whether I use PIP or download manually and run 'python setup.py install' on Pyramid I'm getting the same error: accessing standard pypi websites.
If I visit the sites in my browser it all works, but if I run the command it doesn't. Error as below.
I guess the fault will occur on most packages using these url's because I'm behind a proxy and my proxy is disallowing web access via python executable?
How do I specify pip or simply the apps setup.py to use my proxy, assuming this is the problem.
Installed c:\pymon\python34\lib\site-packages\pyramid-1.5.1-py3.4.egg
Processing dependencies for pyramid==1.5.1
Searching for PasteDeploy>=1.5.0
Reading https://pypi.python.org/simple/PasteDeploy/
Download error on https://pypi.python.org/simple/PasteDeploy/: timed out -- Some
packages may not be found!
Couldn't find index page for 'PasteDeploy' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: timed out -- Some packages may not be found!
No local packages or download links found for PasteDeploy>=1.5.0
error: Could not find suitable distribution for Requirement.parse('PasteDeploy>=
1.5.0')
My apologies I should have searched further:
Plone: pypi specific download issue behind a proxy server - can't install eggs
I created two environment variables: HTTP_PROXY and HTTPS_PROXY
Note the above comment regarding the --proxy argument also worked via pip.
I would like to install PRAW so I can make reddit bots and stuff, but all the install guides are confusing to me so could someone explain how to as noob friendly as possible. I've had some experience with vanilla python. Thanks!
praw is best installed, according to the documentation, via pip. To install pip, you need setuptools. Here is a simple guide on installing pip via setuptools.
Basically, download ez_setup.py and get-pip.py, two Python scripts that automatically get and install setuptools and pip. You'll want to run the following commands in the terminal in the same directory as the location of the files, in order:
python ez_setup.py
python get-pip.py
Finally, you'll want to use pip to get praw. pip is an executable file that is usually located in your python build directory. For example, in Windows, it's located in C:\Python27\scripts. You can add that directory to your system path variable, but right now you can just navigate to that directory where pip.exe is installed. Then, run the following command in the terminal:
pip install praw
I recently had trouble with this so I thought I would add what I did.
Install Pip - https://pip.pypa.io/en/stable/installing/
install praw pip install praw; This is done on your pc/mac/linux(?)
Installation guide
Register on reddit as a developer and register the app. To be able to use the api you need to have a client_id and a client_secret. Get those by registering here. More information about the types of applications can be found here.
Now you are ready to begin coding. This is a good script to verify that you are connecting to the reddit api. The client_id and client_secret are from the previous step and the user_agent is a string that is unique to your app. I used something like 'my first app by /u/myUsername'. The password and username is your reddit login
Run this code and it should output your username.
import praw
reddit = praw.Reddit(client_id='CLIENT_ID',
client_secret="CLIENT_SECRET", password='PASSWORD',
user_agent='USERAGENT', username='USERNAME')
print(reddit.user.me())