I am working on a project and have shifted my environment from local windows to a linux server (via SSH). I only have limited access as the host server is from my college, I've installed many packages without issues (both with and without virtualenv). I'm working on Python 3.6.9.
I was able to install spacy and import it but I need to use the en_core_web_sm package which has to be installed additionally using the command python3 -m spacy download en_core_web_sm. However, I consistently face a PermissionError as seen in the logs below.
Why am I facing this error? Is it because I don't have administrator access on the /usr level (refer to last line of logs)? If yes, how come only this package in particular requires a higher level access? If no, are there any workaround for me to install the package, or do I need to contact the server administrator?
jiayi#cdas1:~/adverse_drug_event_extraction/app/preprocessing$ python3 -m spacy download en_core_web_sm
Collecting en-core-web-sm==3.1.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.1.0/en_core_web_sm-3.1.0-py3-none-any.whl#egg=en_core_web_sm==3.1.0
Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.1.0/en_core_web_sm-3.1.0-py3-none-any.whl (13.6MB)
100% |████████████████████████████████| 13.6MB 117kB/s
Requirement already satisfied: spacy<3.2.0,>=3.1.0 in /home/jiayi/.local/lib/python3.6/site-packages (from en-core-web-sm==3.1.0)
Requirement already satisfied: pathy>=0.3.5 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: typer<0.4.0,>=0.3.0 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: jinja2 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: srsly<3.0.0,>=2.4.1 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.7 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/lib/python3/dist-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: setuptools in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: catalogue<2.1.0,>=2.0.4 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: packaging>=20.0 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4; python_version < "3.8" in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: thinc<8.1.0,>=8.0.8 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: numpy>=1.15.0 in /home/jiayi/.local/lib/python3.6/site-packages (from spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /home/jiayi/.local/lib/python3.6/site-packages (from pathy>=0.3.5->spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: dataclasses<1.0,>=0.6; python_version < "3.7" in /home/jiayi/.local/lib/python3.6/site-packages (from pathy>=0.3.5->spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: click<7.2.0,>=7.1.1 in /home/jiayi/.local/lib/python3.6/site-packages (from typer<0.4.0,>=0.3.0->spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/jiayi/.local/lib/python3.6/site-packages (from jinja2->spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: zipp>=0.5; python_version < "3.8" in /home/jiayi/.local/lib/python3.6/site-packages (from catalogue<2.1.0,>=2.0.4->spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: pyparsing>=2.0.2 in /home/jiayi/.local/lib/python3.6/site-packages (from packaging>=20.0->spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: contextvars<3,>=2.4; python_version < "3.7" in /home/jiayi/.local/lib/python3.6/site-packages (from thinc<8.1.0,>=8.0.8->spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Requirement already satisfied: immutables>=0.9 in /home/jiayi/.local/lib/python3.6/site-packages (from contextvars<3,>=2.4; python_version < "3.7"->thinc<8.1.0,>=8.0.8->spacy<3.2.0,>=3.1.0->en-core-web-sm==3.1.0)
Installing collected packages: en-core-web-sm
Exception:
Traceback (most recent call last):
File "/home/jiayi/.local/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/home/jiayi/.local/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/home/jiayi/.local/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
**kwargs
File "/home/jiayi/.local/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/home/jiayi/.local/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/home/jiayi/.local/lib/python3.6/site-packages/pip/wheel.py", line 345, in move_wheel_files
clobber(source, lib_dir, True)
File "/home/jiayi/.local/lib/python3.6/site-packages/pip/wheel.py", line 316, in clobber
ensure_dir(destdir)
File "/home/jiayi/.local/lib/python3.6/site-packages/pip/utils/__init__.py", line 83, in ensure_dir
os.makedirs(path)
File "/usr/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/en_core_web_sm-3.1.0.dist-info'
As you suspect, this is happening because you don't have permission to install to /usr. Normally pip would install there, but it looks like your pip is basically running with --user, which will install to your user home directory instead.
The spaCy models can't be installed directly via pip because they are large data files, so they can't be hosted on PyPI like ordinary code. An unfortunate side effect of this is that some of the options around pip configuration are ignored.
You can pass extra arguments to the pip install command by appending them to your command, so in your case you can do this:
python3 -m spacy download en_core_web_sm --user
And everything should work.
That said, I would strongly recommend you learn how to use virtual environments with the virtualenv tool, which will make working with Python projects easier and allow to you avoid this problem, as anything you install will just go in the local virtualenv rather than your global pip install.
It seems all other packages are installed under your user /home/jiayi/.local/python3.6/lib and this one tries to install itself globally in /usr/local/lib/python3.6/, not sure why. I guess you can give it installation folder or something.
Look here Where does spacy language model download?
When I try pip install statsmodels.api command,
I had these errors
ERROR: Could not find a version that satisfies the requirement statsmodels.api (from versions: none)
ERROR: No matching distribution found for statsmodels.api
Python version is Python 3.8.5
C:\Windows\system32>pip install statsmodels
Requirement already satisfied: statsmodels in c:\python38\lib\site-packages (0.12.0)
Requirement already satisfied: numpy>=1.15 in c:\python38\lib\site-packages (from statsmodels) (1.19.1)
Requirement already satisfied: patsy>=0.5 in c:\python38\lib\site-packages (from statsmodels) (0.5.1)
Requirement already satisfied: pandas>=0.21 in c:\python38\lib\site-packages (from statsmodels) (1.1.2)
Requirement already satisfied: scipy>=1.1 in c:\python38\lib\site-packages (from statsmodels) (1.5.2)
Requirement already satisfied: six in c:\python38\lib\site-packages (from patsy>=0.5->statsmodels) (1.15.0)
Requirement already satisfied: python-dateutil>=2.7.3 in c:\python38\lib\site-packages (from pandas>=0.21->statsmodels) (2.8.1)
Requirement already satisfied: pytz>=2017.2 in c:\python38\lib\site-packages (from pandas>=0.21->statsmodels) (2020.1)
I think you want this:
pip install statsmodels
You're trying to install one submodule of a Python module/package. You have to install the whole package.
If you've already done this, then you already have the statsmodels.api module and don't have to install anything else.
I do not know how you have configured your environments (if you have many). but ensure that you install your package in the correct environment. However, please try the following for me:
conda install -c anaconda statsmodels
I tried installing pandas and matplotlib using the pip-command in Anaconda prompt
pip install matplotlib and it gave me the following:
Requirement already satisfied: matplotlib in c:\programdata\anaconda3\lib\site-p
ackages (3.1.3)
Requirement already satisfied: python-dateutil>=2.1 in c:\programdata\anaconda3\
lib\site-packages (from matplotlib) (2.8.1)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\pr
ogramdata\anaconda3\lib\site-packages (from matplotlib) (2.4.6)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\programdata\anaconda3\lib
\site-packages (from matplotlib) (1.1.0)
Requirement already satisfied: cycler>=0.10 in c:\programdata\anaconda3\lib\site
-packages (from matplotlib) (0.10.0)
Requirement already satisfied: numpy>=1.11 in c:\programdata\anaconda3\lib\site-
packages (from matplotlib) (1.18.1)
Requirement already satisfied: six>=1.5 in c:\programdata\anaconda3\lib\site-pac
kages (from python-dateutil>=2.1->matplotlib) (1.14.0)
Requirement already satisfied: setuptools in c:\programdata\anaconda3\lib\site-p
ackages (from kiwisolver>=1.0.1->matplotlib) (45.2.0.post20200210)
And when trying to import in Spyder, it says no module found.
What should I do? Thanks in advance.
Make sure your spyder points to your anaconda environment. Currently it is not pointing to your anaconda env that is the reason you are getting this error.
In an attempt to access and run Matlab code via Jupyter notebook and Python, I tried installing 'matlab_kernal' (w/ typo) and 'matlab_kernel' as suggested by a blog, I didn't realize I needed MatLab, not MatLab Compiler Runtime.
I received the error:
"Collecting matlab_kernal
Could not find a version that satisfies the requirement matlab_kernal (from versions: )
No matching distribution found for matlab_kernal"
As a work around, I found python code on GitHub that calculated the metric I wanted, but the author said I'd need to install 'MPI' from 'mpi4py'.
However, it seems my Terminal is now stuck on my previous installation attempts of 'matlab_kernal'. It lists my previous attempts to install these packages (see all white text above red error message in the attached screen capture).
I'm running: pip 18.1 from /anaconda3/lib/python3.7/site-packages/pip (python 3.7)
I've tried:
pip uninstall 'package'
Terminal restart
laptop restart
Screenshot of Terminal error
Please use:
pip install matlab-kernel
Try with conda as well:
conda install -c pchrapka matlab_kernel
I think you're making a typo with kernal.
I am on py3.7 with pip v19.3.
Output:
$ pip install matlab-kernel
Collecting matlab-kernel
Downloading https://files.pythonhosted.org/packages/64/ad/5f471160ec33e2f0f3586d285cd5e3b6dff51027849f28369d43d1d57fd1/matlab_kernel-0.16.7-py3-none-any.whl
Requirement already satisfied: jupyter-client>=4.4.0 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from matlab-kernel) (5.3.4)
Collecting metakernel>=0.23.0
Downloading https://files.pythonhosted.org/packages/ad/a9/0cd74cfbc9c4aeb117bdb7fd9ff12c3890be34ef5c77932e00ef4afaca98/metakernel-0.24.3-py2.py3-none-any.whl (208kB)
|████████████████████████████████| 215kB 246kB/s
Collecting wurlitzer>=1.0.2; platform_system != "Windows"
Downloading https://files.pythonhosted.org/packages/24/5e/f3bd8443bfdf96d2f5d10097d301076a9eb55637b7864e52d2d1a4d8c72a/wurlitzer-2.0.0-py2.py3-none-any.whl
Requirement already satisfied: ipython>=4.0.0 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from matlab-kernel) (7.8.0)
Requirement already satisfied: pyzmq>=13 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from jupyter-client>=4.4.0->matlab-kernel) (18.1.0)
Requirement already satisfied: tornado>=4.1 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from jupyter-client>=4.4.0->matlab-kernel) (6.0.3)
Requirement already satisfied: traitlets in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from jupyter-client>=4.4.0->matlab-kernel) (4.3.3)
Requirement already satisfied: python-dateutil>=2.1 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from jupyter-client>=4.4.0->matlab-kernel) (2.8.0)
Requirement already satisfied: jupyter-core>=4.6.0 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from jupyter-client>=4.4.0->matlab-kernel) (4.6.0)
Requirement already satisfied: pexpect>=4.2 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from metakernel>=0.23.0->matlab-kernel) (4.7.0)
Collecting portalocker
Downloading https://files.pythonhosted.org/packages/91/db/7bc703c0760df726839e0699b7f78a4d8217fdc9c7fcb1b51b39c5a22a4e/portalocker-1.5.2-py2.py3-none-any.whl
Requirement already satisfied: ipykernel in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from metakernel>=0.23.0->matlab-kernel) (5.1.2)
Collecting ipyparallel
Downloading https://files.pythonhosted.org/packages/3f/82/aaa7a357845a98d4028f27c799f0d3bb2fe55fc1247c73dc712b4ae2344c/ipyparallel-6.2.4-py2.py3-none-any.whl (198kB)
|████████████████████████████████| 204kB 698kB/s
Requirement already satisfied: decorator in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from ipython>=4.0.0->matlab-kernel) (4.4.0)
Requirement already satisfied: setuptools>=18.5 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from ipython>=4.0.0->matlab-kernel) (39.0.1)
Requirement already satisfied: pickleshare in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from ipython>=4.0.0->matlab-kernel) (0.7.5)
Requirement already satisfied: prompt-toolkit<2.1.0,>=2.0.0 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from ipython>=4.0.0->matlab-kernel) (2.0.10)
Requirement already satisfied: jedi>=0.10 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from ipython>=4.0.0->matlab-kernel) (0.15.1)
Requirement already satisfied: pygments in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from ipython>=4.0.0->matlab-kernel) (2.4.2)
Requirement already satisfied: appnope; sys_platform == "darwin" in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from ipython>=4.0.0->matlab-kernel) (0.1.0)
Requirement already satisfied: backcall in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from ipython>=4.0.0->matlab-kernel) (0.1.0)
Requirement already satisfied: ipython-genutils in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from traitlets->jupyter-client>=4.4.0->matlab-kernel) (0.2.0)
Requirement already satisfied: six in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from traitlets->jupyter-client>=4.4.0->matlab-kernel) (1.12.0)
Requirement already satisfied: ptyprocess>=0.5 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from pexpect>=4.2->metakernel>=0.23.0->matlab-kernel) (0.6.0)
Requirement already satisfied: wcwidth in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from prompt-toolkit<2.1.0,>=2.0.0->ipython>=4.0.0->matlab-kernel) (0.1.7)
Requirement already satisfied: parso>=0.5.0 in /Users/siddheshpisal/.pyenv/versions/3.6.7/envs/ori_venv/lib/python3.6/site-packages (from jedi>=0.10->ipython>=4.0.0->matlab-kernel) (0.5.1)
Installing collected packages: portalocker, ipyparallel, metakernel, wurlitzer, matlab-kernel
Successfully installed ipyparallel-6.2.4 matlab-kernel-0.16.7 metakernel-0.24.3 portalocker-1.5.2 wurlitzer-2.0.0
WARNING: You are using pip version 19.3; however, version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
It's a version issue for pip and MacOS
See this thread
terminal error in MacOS from command 'pip install 'package-name':
"Collecting 'package-name'
Could not find a version that satisfies the requirement 'package-name' (from versions: )
No matching distribution found for 'package-name'"
Uninstall pip and reinstall pip in terminal
I have followed the instructions to the letter on the website. The problem is inconsistency where things end up. When I type:
which virtualenv
I get back
/usr/local/bin/virtualenv
which looks OK to me (I am not a unix expert). But after doing
pip install virtualenvironmentwrapper
I get this:
Requirement already satisfied: virtualenvwrapper in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied: virtualenv-clone in /usr/local/lib/python2.7/dist-packages (from virtualenvwrapper)
Requirement already satisfied: stevedore in /usr/local/lib/python2.7/dist-packages (from virtualenvwrapper)
Requirement already satisfied: virtualenv in /usr/local/lib/python2.7/dist-packages (from virtualenvwrapper)
Requirement already satisfied: pbr!=2.1.0,>=2.0.0 in /usr/local/lib/python2.7/dist-packages (from stevedore->virtualenvwrapper)
Requirement already satisfied: six>=1.9.0 in ./.local/lib/python2.7/site-packages (from stevedore->virtualenvwrapper)
What do you do to fix this?
It rurns out warpper is not that much more useful. However, the original problem was never resolved.