When I install a package with pip install -t some_dir, I can usually then add some_dir to my PYTHONPATH when running python and import the package. Like this:
~/dev/scratch [venv] » pip install -t some_dir pytest
...
Successfully installed py pytest
~/dev/scratch [venv] » PYTHONPATH=some_dir python
...
>>> import pytest
>>>
However, I recently added the library "stompest" to my dependencies, which is apparently a "namespace package" (not entirely sure what that means). It doesn't seem to work with this same pattern:
~/dev/scratch [venv] » pip install -t some_dir stompest
...
Successfully installed stompest
~/dev/scratch [venv] » PYTHONPATH=some_dir python
...
>>> import stompest.config
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named stompest.config
It works fine if I just install stompest normally (into a virtual environment, say):
~/dev/scratch [venv] » pip install stompest
...
Successfully installed stompest-2.1.6
~/dev/scratch [venv] » python
...
>>> import stompest.config
>>>
The problem seems to be that pip(?) lays down a .pth file that expects to be in the site-dir (notice f_locals['sitedir'] below):
~/dev/scratch [venv] » cat some_dir/stompest-2.1.6-py2.7-nspkg.pth
import sys,types,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('stompest',)); ie = os.path.exists(os.path.join(p,'__init__.py')); m = not ie and sys.modules.setdefault('stompest',types.ModuleType('stompest')); mp = (m or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p)
I tried also setting PYTHONUSERBASE=some_dir, but that didn't seem to make any difference. It seems like a similar issue to this one, which suggests using --egg as a workaround. I can't get --egg to work with -t, though, because I get an error that I am "attempting to install a package to a directory that is not on PYTHONPATH and which Python does not read ".pth" files from." The problem is that I can't add the directory to PYTHONPATH because it seems to be trying to install to a temporary directory:
~/dev/scratch [venv] » PYTHONPATH=some_dir pip install -t some_dir --egg stompest
Collecting stompest
Using cached stompest-2.1.6.tar.gz
Installing collected packages: stompest
Running setup.py install for stompest
Complete output from command /home/nalderso/dev/scratch/venv/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-Wc3oaO/stompest/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-fZlXM2-record/install-record.txt --compile --install-headers /home/nalderso/dev/scratch/venv/include/site/python2.7/stompest --home=/tmp/tmpHNVOP0:
running install
Checking .pth file support in /tmp/tmpHNVOP0/lib/python/
/home/nalderso/dev/scratch/venv/bin/python -E -c pass
TEST FAILED: /tmp/tmpHNVOP0/lib/python/ does NOT support .pth files
error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/tmp/tmpHNVOP0/lib/python/
and your PYTHONPATH environment variable currently contains:
'some_dir'
Here are some of your options for correcting the problem:
* You can choose a different installation directory, i.e., one that is
on PYTHONPATH or supports .pth files
* You can add the installation directory to the PYTHONPATH environment
variable. (It must then also be on PYTHONPATH whenever you run
Python and want to use the package(s) you are installing.)
* You can set up the installation directory to support ".pth" files by
using one of the approaches described here:
https://pythonhosted.org/setuptools/easy_install.html#custom-installation-locations
Please make the appropriate changes for your system and try again.
----------------------------------------
Command "/home/nalderso/dev/scratch/venv/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-Wc3oaO/stompest/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-fZlXM2-record/install-record.txt --compile --install-headers /home/nalderso/dev/scratch/venv/include/site/python2.7/stompest --home=/tmp/tmpHNVOP0" failed with error code 1 in /tmp/pip-build-Wc3oaO/stompest
Am I doing something wrong, or is this an issue with pip?
It works when you do this:
import site
site.addsitedir('some_dir')
See https://docs.python.org/2/library/site.html#site.addsitedir
I found this answer in a post by Matt Iversen (Ivoz) in this pip issue. Note that talk in this issue continues about some corner cases you might run into.
Related
I am not quite experienced using pip. When I tried to install some packages, I had this error and had no idea what that meant. This happened to multiple packages that I tried to install. Any suggestions would be appreciated.
sudo pip install mysql-python
Password:
WARNING: The directory '/Users/yzr/Library/Caches/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting mysql-python
Downloading MySQL-python-1.2.5.zip (108 kB)
|████████████████████████████████| 108 kB 2.2 MB/s
ERROR: Command errored out with exit status 1:
command: /Users/yzr/opt/anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/tmp/pip-install-0kcqh5vw/mysql-python/setup.py'"'"'; __file__='"'"'/private/tmp/pip-install-0kcqh5vw/mysql-python/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/tmp/pip-install-0kcqh5vw/mysql-python/pip-egg-info
cwd: /private/tmp/pip-install-0kcqh5vw/mysql-python/
Complete output (7 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/tmp/pip-install-0kcqh5vw/mysql-python/setup.py", line 13, in <module>
from setup_posix import get_config
File "/private/tmp/pip-install-0kcqh5vw/mysql-python/setup_posix.py", line 2, in <module>
from ConfigParser import SafeConfigParser
ModuleNotFoundError: No module named 'ConfigParser'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
I’m assuming you are using an Unix system such as Linux or MacOS.
Maybe you are accidentally using the python2.7 version installed on your system.
You can use a specific version of python for pip with:
python3 -m pip install mysql-python
Let me explain myself, every MacOS or Linux system has two Python versions (2.7 and 3.x) by default, and both of those versions has a pip module related to it, in other words, you have a pip module for Python 2.7 and other pip module for Python 3.x.
However, I’m assuming you are using python3 for your project, therefore, you must specify the pip module to use.
Other way to avoid this problem is using a virtual environment manager, such as pipenv or built in virtualenv module.
It looks like your pip package doesn't have the module "ConfigParser" installed. You should be able to install it by running:
pip install ConfigParser
Alternatively, depending on your Python version, try installing mysqlclient instead:
pip install mysqlclient
Installing PyAudio on Windows 10 error
I want to install PyAudio
I have Windows 10, Python 3.7, pip 8.1
I'm installing it in Command Prompt with command pip install PyAudio
But there's an error error: command 'D:\\VisualStudio2017\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x86\\cl.exe' failed with exit status 2
Command "c:\python37\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-is1aobj6\\pyaudio\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\USER\AppData\Local\Temp\pip-record-50bh4kz0\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\USER\AppData\Local\Temp\pip-install-is1aobj6\pyaudio\
Here's a
screenshot
Thanks for reading my question. I hope you can help :D
Well, I've had just this problem when I wanted to install pylisten (which has PyAudio as a dependency).
There are two problems here - first, your MSVC can't find portaudio.h and after that there will be a problem with the min macro when building. But first things first.
Get portaudio
This is a C dependency for PyAudio, so you need to have include files, in particular portaudio.h like you see on the screenshot. It's possible to get binaries for Windows, but if your using Anaconda, the simplest way to do it is to install it using conda:
(base) C:\> conda search portaudio
Loading channels: done
# Name Version Build Channel
portaudio 19.6.0 h0c8e037_3 pkgs/main
portaudio 19.6.0 hfa6e2cd_3 pkgs/main
(base) C:\> conda install portaudio
(...)
Next, we would like to know where the files from portaudio library actually went. There is no conda package --list command... (see here: https://github.com/conda/conda/pull/7457), so we need to do some magic found elsewhere on SO:
(base) C:\>dir %CONDA_PREFIX%\conda-meta\portaudio*
(...)
26.01.2020 13:17 5 577 portaudio-19.6.0-hfa6e2cd_3.json
We need to check this JSON and find the files property:
"extracted_package_dir": "C:\\Users\\(...)\\Anaconda3\\pkgs\\portaudio-19.6.0-hfa6e2cd_3",
"features": "",
"files": [
"Library/bin/portaudio.dll",
"Library/include/pa_asio.h",
"Library/include/pa_jack.h",
"Library/include/pa_linux_alsa.h",
"Library/include/pa_mac_core.h",
"Library/include/pa_win_ds.h",
"Library/include/pa_win_wasapi.h",
"Library/include/pa_win_waveformat.h",
"Library/include/pa_win_wdmks.h",
"Library/include/pa_win_wmme.h",
"Library/include/portaudio.h",
"Library/lib/portaudio.lib",
"Library/lib/portaudio_static.lib",
".nonadmin"
],
The extracted_package_dir (username removed above) actually amounts to:
%CONDA_PREFIX%\pkgs\portaudio-19.6.0-hfa6e2cd_3.
So, for this specific build, the necessary include directories (-I) are:
%CONDA_PREFIX%\pkgs\portaudio-19.6.0-hfa6e2cd_3\Library\include
And link directories (-L) are:
%CONDA_PREFIX%\pkgs\portaudio-19.6.0-hfa6e2cd_3\Library\lib
This will be useful below.
Simple way, which doesn't work
Now if your Conda environment has MSVC 14.0 for building native extensions you should be fine with:
(base) C:\> pip install --global-option=build_ext --global-option="-I%CONDA_PREFIX%\pkgs\portaudio-19.6.0-hfa6e2cd_3\Library\include" --global-option="-L%CONDA_PREFIX%\pkgs\portaudio-19.6.0-hfa6e2cd_3\Library\lib" pyaudio
Now, it will find portaudio.h!
But alas, at the time of writing, this fails with a compile error. I still put this here, because maybe the incompatibility with MSVC will be fixed.
The hard, but successful, way
We need to hack the sources of PyAudio. This can be done like so:
(base) C:\>pip download pyaudio
This will download a file like PyAudio-0.2.11.tar.gz (version number may differ in the future). You need to unpack it, using tar. I have it in C:\Windows\system32 - maybe because of WSL. You can also use 7-zip to unpack it. Anyway:
(base) C:\>tar xf PyAudio-0.2.11.tar.gz
(... unpacked ...)
(base) C:\>cd PyAudio-0.2.11
(base) C:\PyAudio-0.2.11>
Now to get the same result as before, you need to pass the special arguments to setup.py (in a different way than it's done for pip, of course):
(base) C:\PyAudio-0.2.11>python setup.py build_ext --include-dirs="%CONDA_PREFIX%\pkgs\portaudio-19.6.0-hfa6e2cd_3\Library\include" --library-dirs="%CONDA_PREFIX%\pkgs\portaudio-19.6.0-hfa6e2cd_3\Library\lib" install
(compile errors: min macro redefinition, __typeof__ undefined)
Now, edit the file src\_portaudiomodule.c, and delete/comment out these lines (right at the top, line ~39):
#define min(a, b) \
({ \
__typeof__(a) _a = (a); \
__typeof__(b) _b = (b); \
_a < _b ? _a : _b; \
})
This trivial macro is the culprit. If you want to submit a fix to Mr Pham (http://people.csail.mit.edu/hubert/pyaudio/), don't delete it, but use something like
#ifndef _MSC_VER ... #endif.
Save the file and run this (again):
(base) C:\PyAudio-0.2.11>python setup.py build_ext --include-dirs="%CONDA_PREFIX%\pkgs\portaudio-19.6.0-hfa6e2cd_3\Library\include" --library-dirs="%CONDA_PREFIX%\pkgs\portaudio-19.6.0-hfa6e2cd_3\Library\lib" install
Everything should be fine now.
EDIT: TL;DR easy way
After checking once again, I saw pyaudio in https://repo.anaconda.com/pkgs/main/win-64. This means, if you're using Anaconda you can just do:
conda install pyaudio
I'm leaving the rest of this answer for educational purposes.
I am trying to install the spacy python library for python 3.4 on windows 7. i keep getting install errors (the important-looking error messages are below).
I installed it for 2.7, but I can't seem to install it for 3.4.
I tried doing:
pip3.4 install -U spacy
But it does not seem to work. Does anybody have any ideas? I'm trying to avoid crazy system surgery. There must be some simple way to do this.
Warning: The nvcc binary could not be located in your $PATH. For GPU capability, either add it to your path, or set $CUDA_HOME
warning: no files found matching '*.cpp' under directory 'include'
warning: no files found matching 'buildbot.json'
***********************************************************
WARNING: One of the following dependencies is unresolved:
ctypes >=1.0.1
readline >=1.7.1
***********************************************************
error: Unable to find vcvarsall.bat
Cleaning up...
Command C:\Python34\python.exe -c "import setuptools, tokenize;file='C:\Users\joe\AppData\Local\Temp\pip_build_joe\spacy\setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record C:\Users\joe\AppData\Local\Temp\pip-sxqy1vof-record\install-record.txt --single-version-externally-managed --compile failed with error code 1 in C:\Users\joe\AppData\Local\Temp\pip_build_joe\spacy
Storing debug log for failure in C:\Users\joe\pip\pip.log
In order for python to detect it you must place it in the following directory.
C:\Python27\
or where ever python is located.
I'm very new to Python and programming world and has been going along with tutorials from newcoder.io This Here! I have been doing as per the instructions but when I try to install NumPy I get an error.
"
error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
Command "C:\Users\HP.virtualenvs\DataVizProj\Scripts\python.exe -c "import setuptools, tokenize;file='c:\users
\appdata\local\temp\pip-build-lsj5sj\numpy\setup.py';exec(compile(getattr(tokenize, 'open', open)(file).re
.replace('\r\n', '\n'), file, 'exec'))" install --record c:\users\hp\appdata\local\temp\pip-6jei4k-record\instal
cord.txt --single-version-externally-managed --compile --install-headers C:\Users\HP.virtualenvs\DataVizProj\includ
te\python2.7" failed with error code 1 in c:\users\hp\appdata\local\temp\pip-build-lsj5sj\numpy
"
But that's not enough, I tried to install VCForPython27.msi from the given link. But still, gets the same error.
Please Help!
I recommend installing the Anaconda distribution of Python. It contains more packages than you can dream of, including numpy of course: http://continuum.io/downloads
The installation is as straightforward as installing the usual Python, no matter what OS you are on.
A good solution is to download the wheel file which match your Python version here:unofficial python binaries
And then go to the folder where the .whl file is present using the command prompt and unpack the wheel file xxxxx.whl with:
>python -m pip install xxxxx.whl
This will install the library in the Lib\site-packages folder, you can check it afterwards.
I am trying to create virtualenv and install project dependencies using pip.
$ mkvirtualenv --no-site-packages --distribute myenv
(myenv)$ pip install -r requirements.txt
I also set up export VIRTUALENV_DISTRIBUTE=true in ~/.bash_profile
After installing some packages pip shows following error:
.....
Could not find the /Users/me/.virtualenvs/myenv/lib/python2.7/site-packages/site.py element of the Setuptools distribution
Patched done.
Relaunching...
Traceback (most recent call last):
File "<string>", line 1, in <module>
NameError: name 'install' is not defined
----------------------------------------
Cleaning up...
Command /Users/me/.virtualenvs/myenv/bin/python -c "import setuptools;__file__='/Users/me/.virtualenvs/myenv/build/distribute/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/r0/2b441j6x5rq8y964bhd15gkm0000gn/T/pip-wyn1Ys-record/install-record.txt --single-version-externally-managed --install-headers /Users/me/.virtualenvs/myenv/include/site/python2.7 failed with error code 1 in /Users/me/.virtualenvs/myenv/build/distribute
Exactly the same happens without --distribute switch and without export VIRTUALENV_DISTRIBUTE=true
Here is my requirements.txt file:
Django==1.5
Pillow==1.7.6
South==0.7.3
amqplib==1.0.2
anyjson==0.3.1
celery==2.5.3
distribute==0.6.10
django-celery==2.4.2
django-indexer==0.3.0
django-kombu==0.9.4
django-mptt==0.5.2
django-paging==0.2.4
django-picklefield==0.2.1
django-social-auth==0.7.22
django-tagging==0.3.1
django-taggit==0.9.3
django-templated-email==0.4.7
django-templatetag-sugar==0.1
eventlet==0.9.16
greatape==0.3.0
greenlet==0.3.4
html5lib==0.90
httplib2==0.8
kombu==2.1.7
lockfile==0.9.1
oauth2==1.5.211
pycrypto==2.3
python-daemon==1.6
python-dateutil==1.5
python-openid==2.2.5
raven==1.0.4
sentry==2.0.0-RC6
simplejson==2.3.2
ssh==1.7.8
wsgiref==0.1.2
I am using Mac OS X 10.9.2.
I don't want to change anything in requirements.txt I just want to install all dependencies and run this project.
Remove the distribute package from the list, recreate your environment and reinstall the requirements.
You may remove the wsgiref as well (although not much important).