We Keep Coding

cannot install fairseq using Anaconda?

I install the package fairseq but it raises some errors as bellow. I'm using Anaconda Prompt to install on the base environment. Could anyone help to fix this problem?? Thanks so much!
(base) PS C:\WINDOWS\system32> pip install fairseq --user
Collecting fairseq
Using cached fairseq-0.10.1.tar.gz (924 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\johnn\Anaconda3\python.exe' 'C:\Users\johnn\Anaconda3\lib\site-packages\pip\_vendor\pep517\_in_process.py' get_requires_for_build_wheel 'C:\Users\johnn\AppData\Local\Temp\tmpr4o1lvo4'
cwd: C:\Users\johnn\AppData\Local\Temp\pip-install-5efwp00y\fairseq_8f7b526c143c4da7b9cf30edcc3c3ab0
Complete output (31 lines):
Traceback (most recent call last):
File "setup.py", line 214, in <module>
do_setup(package_data)
File "setup.py", line 136, in do_setup
setup(
File "C:\Users\johnn\AppData\Local\Temp\pip-build-env-8dy_kf5o\overlay\Lib\site-packages\setuptools\__init__.py", line 152, in setup
_install_setup_requires(attrs)
File "C:\Users\johnn\AppData\Local\Temp\pip-build-env-8dy_kf5o\overlay\Lib\site-packages\setuptools\__init__.py", line 147, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "C:\Users\johnn\AppData\Local\Temp\pip-build-env-8dy_kf5o\overlay\Lib\site-packages\setuptools\build_meta.py", line 60, in fetch_build_eggs
raise SetupRequirementsError(specifier_list)
setuptools.build_meta.SetupRequirementsError: ['cython', 'numpy', 'setuptools>=18.0']
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\johnn\Anaconda3\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 280, in <module>
main()
File "C:\Users\johnn\Anaconda3\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\Users\johnn\Anaconda3\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 114, in get_requires_for_build_wheel
return hook(config_settings)
File "C:\Users\johnn\AppData\Local\Temp\pip-build-env-8dy_kf5o\overlay\Lib\site-packages\setuptools\build_meta.py", line 149, in get_requires_for_build_wheel
return self._get_build_requires(
File "C:\Users\johnn\AppData\Local\Temp\pip-build-env-8dy_kf5o\overlay\Lib\site-packages\setuptools\build_meta.py", line 130, in _get_build_requires
self.run_setup()
File "C:\Users\johnn\AppData\Local\Temp\pip-build-env-8dy_kf5o\overlay\Lib\site-packages\setuptools\build_meta.py", line 145, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 217, in <module>
os.unlink(fairseq_examples)
PermissionError: [WinError 5] Access is denied: 'fairseq\\examples'
----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\johnn\Anaconda3\python.exe' 'C:\Users\johnn\Anaconda3\lib\site-packages\pip\_vendor\pep517\_in_process.py' get_requires_

my solution is below
git clone https://github.com/pytorch/fairseq
cd fairseq
activate env(that you created existed)
pip install --editable .

TL; DR: conda install fairseq --channel conda-forge
Conda Channels
Unfortunately, fairseq is not in the list of default conda channel. However, you can use
conda install fairseq --channel conda-forge
to install fairseq. The option --channel (-c for short) specifies the channel (it uses conda-forge in here) for conda to retrieve packages. You get a more detailed description in Conda channels | Conda Dos.
A similar example is when you follow the offical guide to install PyTorch with conda, it gives
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
which means you install these 3 packages through channel pytorch.
About fairseq
If you simply would like to use fairseq rather than doing some specialized modifications (e.g. create an experimental neural network structure for research), conda install is enough. Otherwise, it is recommended to install as #yeonsook kwak said. The option --editable (-e for short) means you can directly do editions in this pulled repository.
conda install v.s. pip install
One advantage of doing conda install (instead of pip install) is that you can share the packages through all conda environments, which may save disk spaces. However, if you would like to install the same package but with different versions in different environments, you should use pip install.
A possible drawback for conda install is the package from external channels may not be the lastest comparing to those from PyPI/offical source.

Why does Pip disregard configured repository with nested dependencies?

Problem
Let us say I have a completely empty Python+Pip+R (pip 19.3.1) environment on a Linux machine and I want to install the package rpy2 with pip. Since I am behind a corporate firewall I configure pip to use a private repository.
[global]
index-url = http://private.com/artifactory/api/pypi/PyPI/simple
trusted-host = private.com
Now I execute pip install rpy2 and I will get back the following error:
Couldn't find index page for 'cffi'
Download error on https://pypi.python.org/simple/
So pip tries to resolve the nested dependency by looking and installing cffi from the official PyPi repository. It completely ignores the repo I have configured.
When I run pip install cffi && pip install rpy2 one after another everything works as expected.
Here is the full error output:
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8vuadu93/rpy2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-8vuadu93/rpy2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-8vuadu93/rpy2/pip-egg-info
cwd: /tmp/pip-install-8vuadu93/rpy2/
Complete output (25 lines):
Download error on https://pypi.python.org/simple/cffi/: [Errno -2] Name or service not known -- Some packages may not be found!
Couldn't find index page for 'cffi' (maybe misspelled?)
Download error on https://pypi.python.org/simple/: [Errno -2] Name or service not known -- Some packages may not be found!
No local packages or working download links found for cffi>=1.13.1
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-8vuadu93/rpy2/setup.py", line 183, in <module>
'rinterface_lib/R_API_eventloop.h']}
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 128, in setup
_install_setup_requires(attrs)
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 123, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 513, in fetch_build_eggs
replace_conflicting=True,
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 774, in resolve
replace_conflicting=replace_conflicting
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1057, in best_match
return self.obtain(req, installer)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1069, in obtain
return installer(requirement)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 580, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 692, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cffi>=1.13.1')
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Question
Is this a bug in Pip, a specific problem with rpy2 or am I missing something?
Updates
Running the following commands gives me the same error:
pip install rpy2 --no-index --find-links http://private.com/artifactory/api/pypi/PyPI/simple
pip install rpy2 --index-url http://private.com/artifactory/api/pypi/PyPI/simple
I used -vvv and it seems like the problem occurs somewhere inside of setuptools
Solution
The package rpy2 uses setuptools which again uses easy_install.py. It also works with an index_url variable. But it gets the value not from pip.config but distutils.cfg.
I identified all Python versions I have installed with find / -name "distutils". Then I added the a distutils.cfg with the following content to each of these directories:
[easy_install]
index_url = blablabla
And now it works, I execute pip install rpy2 and all missing requirements are installed in one go

I believe it could be caused by the fact that cffi is listed as setup_requires in rpy2's setup.py. Most likely because cffi is required to build the project itself before it can be installed. This kind of build dependencies are not handled by pip directly, so its index-url option has no effect.
The solution is to tell setuptools about the alternative index in a distutils configuration file
[easy_install]
index_url = https://my.index-mirror.com
References:
"Controlling setup_requires" in pip's documentation
setuptools, easy_install, and a custom pypi server
distutils configuration file

Pip install --target=. Alfred-Workflow gives an error

I am trying to install a python library on macOS following through these instructions.
However I get an error every time I run this command : pip install --target=. Alfred-Workflow
And I always get this error for running it :
pip install --target=. Alfred-Workflow
Collecting Alfred-Workflow
Installing collected packages: Alfred-Workflow
Exception:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/local/lib/python2.7/site-packages/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/usr/local/lib/python2.7/site-packages/pip/req/req_set.py", line 784, in install
**kwargs
File "/usr/local/lib/python2.7/site-packages/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/usr/local/lib/python2.7/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/usr/local/lib/python2.7/site-packages/pip/wheel.py", line 247, in move_wheel_files
prefix=prefix,
File "/usr/local/lib/python2.7/site-packages/pip/locations.py", line 153, in distutils_scheme
i.finalize_options()
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/install.py", line 264, in finalize_options
"must supply either home or prefix/exec-prefix -- not both"
DistutilsOptionError: must supply either home or prefix/exec-prefix -- not both
I tried googling and searching for this but still can't figure it out. Thank you for any help.

This question answers that (I found it when googling for the last line of your error message).
First I also got the same error message as you did, but after doing this:
$ echo "[install]
prefix=" > ~/.pydistutils.cfg
It works:
$ pip install --target=. Alfred-Workflow
Collecting Alfred-Workflow
Installing collected packages: Alfred-Workflow
Successfully installed Alfred-Workflow-1.24
Important note: it breaks normal pip install commands, so you need to rm ~/.pydistutils.cfg afterward.

I have a similar error installing python modules using pip with -t(--target) option.
The pip log show the next message:
Complete output from command /usr/bin/python -c "import setuptools, tokenize;file='/tmp/pip-build-LvB_CW/xlrd/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-UNJizV-record/install-record.txt --single-version-externally-managed --compile --user --home=/tmp/tmphjBN23
and the next error:
can't combine user with prefix, exec_prefix/home, or install_(plat)base
Reading about alternate installation on python docs I see the next info
Note that the various alternate installation schemes are mutually exclusive: you can pass --user, or --home, or --prefix and --exec-prefix, or --install-base and --install-platbase, but you can’t mix from these groups.
So the command executed by pip has two mutually exclusive schemes --user and --home (i think it could be a bug on pip).
I use the --system option to avoid the error, eliminating the --user flag on the setup command.
pip install -t path_to_dir module_name --system
I don't known the aditional implications of this usage, but i think it's better than modifing the config file that messed up with normal installations.
PD: I use ubuntu 15.10 with pip 1.5.6

pip uninstall working but giving error

I have downloaded, built and installed the sample pypi project. The project is supposed to be used to give an insight to python packaging and is referenced in Python Packaging User Guide and Packaging and Distributing Projects. I am therefore confused to why on earth following their instructions on creating the package I have an error when uninstalling it.
As mentioned above I have used their sample project. If anyone can shed some light it would be appreciated.
Key notes: I am running Mac OSX, my installed python version is 3.5.1
Below are steps to reproduce the problem:
mkdir testdirectory
cd testdirectory/
pyvenv venv # Creating a virtual environment
source venv/bin/activate
git clone https://github.com/pypa/sampleproject.git # Getting sample project
cd sampleproject/
python setup.py build
python setup.py install # Installing on the virtual environment
pip list
peppercorn (0.5)
pip (7.1.2)
sample (1.2.0)
setuptools (18.2)
pip uninstall sample
Resulting in the following output:
Uninstalling sample-1.2.0:
/Users/steve/testdirectory/venv/bin/sample
/Users/steve/testdirectory/venv/lib/python3.5/site-packages/sample-1.2.0-py3.5.egg
Proceed (y/n)? y
Successfully uninstalled sample-1.2.0
Traceback (most recent call last):
File "/Users/steve/testdirectory/venv/bin/pip3", line 11, in <module>
sys.exit(main())
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/__init__.py", line 217, in main
return command.main(cmd_args)
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/basecommand.py", line 248, in main
pip_version_check(session)
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/utils/outdated.py", line 102, in pip_version_check
installed_version = get_installed_version("pip")
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/utils/__init__.py", line 858, in get_installed_version
working_set = pkg_resources.WorkingSet()
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 629, in __init__
self.add_entry(entry)
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 685, in add_entry
for dist in find_distributions(entry, True):
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2075, in find_eggs_in_zip
if metadata.has_metadata('PKG-INFO'):
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1605, in has_metadata
return self.egg_info and self._has(self._fn(self.egg_info, name))
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1963, in _has
return zip_path in self.zipinfo or zip_path in self._index()
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1843, in zipinfo
return self._zip_manifests.load(self.loader.archive)
File "/Users/steve/testdirectory/venv/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1783, in load
mtime = os.stat(path).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/Users/steve/testdirectory/venv/lib/python3.5/site-packages/sample-1.2.0-py3.5.egg'
It appears that the package has uninstalled correctly however the error is still thrown.

This is documented:
$ pip help uninstall
Usage:
pip uninstall [options] <package> ...
pip uninstall [options] -r <requirements file> ...
Description:
Uninstall packages.
pip is able to uninstall most installed packages. Known exceptions are:
- Pure distutils packages installed with ``python setup.py install``, which
leave behind no metadata to determine what files were installed.
- Script wrappers installed by ``python setup.py develop``.
i.e. you are getting the error because you didn't use pip itself to install - you used setup.py and so pip doesn't know what was installed.

Install package not with python setup.py install but with pip install .
Then uninstalling will work without errors

Why am I getting this error (related to pip and easy_install) when trying to setup a virtualenv?

I'm trying to create a Flask application to push to Heroku. When I try to create a virtualenv in the folder, I'm thrown this error:
photo-crawl › sudo virtualenv venv --distribute
Password:
New python executable in venv/bin/python
Installing distribute............................................................................................................................................................................................................................done.
Installing pip....
Complete output from command /Users/evansiegel/In...rawl/venv/bin/python -x /Users/evansiegel/In...env/bin/easy_install /Library/Python/2.7/...ort/pip-1.2.1.tar.gz:
/Users/evansiegel/Involvio/photo-crawl/venv/bin/python: can't open file '/Users/evansiegel/Involvio/photo-crawl/venv/bin/easy_install': [Errno 2] No such file or directory
----------------------------------------
...Installing pip...done.
Traceback (most recent call last):
File "/usr/local/bin/virtualenv", line 9, in <module>
load_entry_point('virtualenv==1.8.4', 'console_scripts', 'virtualenv')()
File "/Library/Python/2.7/site-packages/virtualenv.py", line 964, in main
never_download=options.never_download)
File "/Library/Python/2.7/site-packages/virtualenv.py", line 1076, in create_environment
install_pip(py_executable, search_dirs=search_dirs, never_download=never_download)
File "/Library/Python/2.7/site-packages/virtualenv.py", line 667, in install_pip
filter_stdout=_filter_setup)
File "/Library/Python/2.7/site-packages/virtualenv.py", line 1042, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /Users/evansiegel/In...rawl/venv/bin/python -x /Users/evansiegel/In...env/bin/easy_install /Library/Python/2.7/...ort/pip-1.2.1.tar.gz failed with error code 2
It seems that for some reason virtualenv is not putting everything it should be putting into the venv/bin folder. Right now, only python and two aliases, python2 and python2.7 are in there.
Thanks!

I believe this is caused by a bug in older versions of virtualenv and/or pip. It looks like you're using virtualenv 1.8.4 and pip 1.2.1. Is it possible for you to upgrade to latest virtualenv and pip (currently virtualenv 1.9.1 and pip 1.3.1)?
This should do it, if you have install privileges:
pip install pip --upgrade
pip install virtualenv --upgrade
At that point running the same command (i.e. sudo virtualenv venv --distribute) should give you what you want.