We Keep Coding

Installing from github (with extras) via a requirements.txt file

I am trying to add the Haystack library as a dependency of a python project. The installation instructions that Haystack provides are as follows
git clone https://github.com/deepset-ai/haystack.git
cd haystack
pip install -e .[all]
I am trying to translate this into a single line that I can include in a requirements.txt. My current best guess is
farm-haystack[all] # git+https://github.com/deepset-ai/haystack.git
However this emits a bunch of warnings that various versions of farm-haystack don't provide the desired extras, such as
WARNING: farm-haystack 0.1.0.post2 does not provide the extra 'ray'
before failing with the error message
ERROR: Requested dill from https://files.pythonhosted.org/packages/3e/ad/31932a4e2804897e6fd2f946d53df51dd9b4aa55e152b5404395d00354d1/dill-0.3.1.tar.gz#sha256=d3ddddf2806a7bc9858b20c02dc174396795545e9d62f243b34481fd26eb3e2c (from farm-haystack[all]# git+https://github.com/deepset-ai/haystack.git->-r /dss_data/tmp/pip-requirements-install/req3361828774079305889.txt (line 1)) has different version in metadata: '0.3.1.dev0'
What is the proper way to go about doing this?

Since you mention it would be a dependency of another project, the format for listing Haystack in your requirements.txt should be the following (I pin the version here but it's not mandatory):
farm-haystack[all]==1.5.0
If you want to pin a specific git commit instead, the line in your requirements.txt should just be:
git+https://github.com/deepset-ai/haystack.git#egg=farm-haystack[all]

pip install failing due to repeatedly changing sha256 (ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE)

I am currently trying to install many different libraries into my Python environment. I've currently tried using a virtual environment and conda environment still resulting in similar errors.
As I'm doing this on a VM, I have to go through my corporate proxy, hence my pip install command looks a little verbose like the following:
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org h5py --proxy http://user:password#proxy.internet.mycompany.local:8080
This had been working fine to download many different packages (including ones that are currently failing now), I'm not too sure what has changed, but now the command results in the following error:
Collecting h5py
Downloading h5py-3.1.0-cp37-cp37m-win_amd64.whl
\ 4.8 kB ...
ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the >package versions
, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered >with them
.
h5py from https://files.pythonhosted.org/packages/53/c2/77bd81922264520b492bd7bfd1a51a845bc1187445408a7a83db284fd566/h5py-3.1.0-cp37-cp37m-win_amd64.whl#sha256=02c391fdb980762a1cc03a4bcaecd03dc463994a9a63a02264830114a96e111f
:
Expected sha256 02c391fdb980762a1cc03a4bcaecd03dc463994a9a63a02264830114a96e111f
Got 3fbaf847f810d5f6970abb0c7754abcca7dea17d315036fe5e4b26f427adf3a6
Although this error will still stay largely the same, the "Got" sha256 will change on each attempt, i.e. the following for the last returned line:
Got ff8197d74c233317597b89612691bf420806046093628ac02f0a92df3ac8123a
Got 8be06806b22d49aaa9b94113d8b3e075539303da96386971600c21cb8b9f8460
To rectify this error I've attempted the following:
using '--no-cache-dir' as suggested other Stack Overflow answers to similar questions
reinstalling python/pip or trying other versions of Python.
trying to use miniconda/anaconda instead
contacting owner of libraries about the issue (although I'm fairly sure it isn't library specific as I can't download any packages)
attempted to access the link in the error output: https://files.pythonhosted.org/packages/53/c2/77bd81922264520b492bd7bfd1a51a845bc1187445408a7a83db28%204fd566/h5py-3.1.0-cp37-cp37m-win_amd64.whl#sha256=02c391fdb980762a1cc03a4bcaecd03dc463994a9a63a02264830114a96e111f. This results in the following page response - the console section of developer tools states the following error: Failed to load resource: the server responded with a status of 404 (Not Found)
I've found a few similar issues to this already on Stack Overflow, but I've found nothing that resolves this.

Getting Error as "ValueError: Zip does not support timestamps before 1980" while installing Setuptools-scm Pypi

Describe the bug
Am working on a Client machine where I don't have access to external servers. So I have to download the packages from Python websites and extract the zip-files and I have to install the packages in my machine by passing a command python setup.py install in command prompt. First two packages (Selenium and urllib3) working fine in my machine and setup also good.
I have tried to install the Pytest Pypi but that requires setuptools-scm. So I downloaded the setuptools-scm packages and I tried to install it but I am getting an error ValueError: Zip does not support timestamps before 1980.
Expected behavior
Setuptools-scm should be installed
To Reproduce
Download the setuptools-scm package from https://pypi.org/project/setuptools-scm/
Extract the zip-files and installing the setuptools-scm by running python setup.py install
Observe the error ValueError: Zip does not support timestamps before 1980.
Command Prompt response:
File "C:\Program Files\Python 3.8\lib\zipfile.py", line 360, in __init__
raise ValueError('ZIP does not support timestamps before 1980')
ValueError: ZIP does not support timestamps before 1980

In my case, it was because files had a last modified date on them on 1st of january 1970. I simply touch all the files to update the last modified to today and everything works.
$ touch `find . -type f`

I got this error in python3.9. I could resolve it by changing all instance's of strict_timestamps from True to False (i.e. strict_timestamps=False) in zipfile.py inside Lib folder (..\Python\Python39\Lib\). Reference

I got this error too. That's because I use "WinRAR" to decompress package( *.tar.gz), so many files don't have dates on them. Then I use cmd tools and the command tar -zxvf *.tar.gz to decompress this package, and the problem is solved.

I noticed that the files in src/setuptools_scm/ didn't have a timestamp for date created or modified. I simply opened the files in a text editor and saved them without changes to establish a timestamp.
After that, >python ./setup.py install worked as expected.

Checking setuptools install_requires on testpypi

I am trying test a python package I want to release using test test.pypi.
In the setup.py file I have install_requires=['numpy>=1.15','scipy>=0.0','scikit-learn>=0.2','numba>=0.0'],
Scipy and Numpy get downloaded and install as expected.
I get the following error: ERROR: Could not find a version that satisfies the requirement numba>=0.0
As a note, if I do pip install numba before my test package it will work, but I am trying to make the package work correctly.
I notice when it does the scipy requirement first, it shows Downloading https://test-files.pythonhosted.org/packages/68/72/eb962a3ae2755af6b1f31f7a94dccc21bfc41bb1637c5877a043e711b1d7/scipy-0.1.tar.gz .
So from the url, it seems like it is using test-files, but is this the regular pypi or just the test one?
My question is: what is the appropriate way to write the install_requires so I can make sure the test works before putting it on the actual pypi site?

There is nothing wrong with your syntax, it is just that unlike scipy, numpy, and scikit-learn, there is no numba hosted on the test PyPI instance. Compare:
https://pypi.org/project/numba/ <-- 200 OK
https://test.pypi.org/project/numba/ <-- 404 Not Found
(Note: that 404 was true at the time of writing this answer, but it appears as though a version of numba has now been uploaded to the test index, on Feb 9th 2021)
My question is: what is the appropriate way to write the install_requires so I can make sure the test works before putting it on the actual pypi site?
How you wrote install_requires is OK. To smoke test it, by uploading your package to test PyPI and checking that it installs properly, use the test PyPI as an extra index url, rather than as a replacement --index-url:
pip install yourpackage --extra-index-url=https://test.pypi.org/simple/
This way yourpackage can be found in test PyPI but the install requirements such as numba can still be resolved on main PyPI.

NameError: name 'json' is not defined

Trying to import Json, my command is pip install json.
I'm working on windows 8.1
The error i'm getting in command prompt is
Could not find a version that satisfies the requirements json <from versions:>
No matching distribution found for json.
and the error i'm getting on pycharm is
NameError: name 'json' is not defined
I tried importing numpy and it worked just fine .
I also did check Pip "Could not find a that satisfies the requirement"
and Could not find a version that satisfies the requirement <package>
Edit : Referred also to this link Python 3.5.1 : NameError: name 'json' is not defined and getting an error that sudo is not recognized

If it's not defined in your code, you need to import it. This is exactly the same as any name in Python; you can't use something until you have defined it.
import json

I have also encountered a similar issue, pip failing to install json and math modules (using python 3.x).
Finally, I discovered that some modules you simply do not have to install - they are already BUILT-IN. :)
Of course, you still have to add "import json" at the top of your .py file.
Hope this helps somebody.

Use the command as simplejson instead of json like below,
pip install simplejson
If the above command also throwing an error Use the following command
easy_install simplejson
easy_install works sometimes if pip cannot serve the request.