I create and activate a virtualenv (venv) using Python 3.3's built-in way of doing it:
$ python3.3 -m venv env
$ source env/bin/activate
At this point python is the python in my virtualenv, which I expect:
(env) $ which python
/my_home_directory/env/bin/python
Now I want to install distribute and pip, so I download the setup scripts and run them:
(env)$ wget http://python-distribute.org/distribute_setup.py
(env)$ wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
(env)$ python distribute_setup.py
(env)$ python get-pip.py
These commands complete successfully. At this point I inspect my venv to find another directory called "local" which wasn't there before. env/local/bin contains my easy_install and pip executables, and they're still aliased to my system's existing easy_install and pip:
(env)$ ls env
bin include lib local pyvenv.cfg
(env)$ ls env/bin
activate pydoc python python3 python3.3
(env)$ ls env/local/bin
easy_install easy_install-3.3 pip pip-3.3
(env)$ which easy_install
/usr/bin/easy_install
(env)$ which pip
/usr/bin/pip
I believe this is a departure from Python 2.x's behavior. At this point I expect easy_install and pip to be using the virtualenv's copies, and using them to install eggs will put them in the virtualenv.
Going a bit further, I crack open env/bin/activate to find that env/bin is prepended to the system path, but env/local/bin is not. That explains the behavior I'm seeing. I can work around this problem by editing env/bin/activate to add the env/local/bin directory to the path, something like:
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
PATH="$VIRTUAL_ENV/local/bin:$PATH" # my new line
export PATH
So, what's going on here? Is this a bug, or am I missing something?
I'm on Ubuntu 12.10 in case that makes a difference.
I have a feeling there's a bug in Ubuntu's python packages or distribute somewhere… but I haven't tracked it down (and I'm not sure I care to).
For whatever reason, the VIRTUAL_ENV environment variable needs to be set the root of the virtualenv for distribute and pip to install properly.
This gist, adopted from Vinay Sajip's code sample in the Python 3 docs, sets said variable; both distribute and pip will install properly when using it.
It's in the python docs.
'/usr/local' is the default exec_prefix. Read the venv docs for detail how to change the default behaviour. There's even an example there that shows how to make a venv.EnvBuilder that installs distribute and pip for you.
if you find distribute docs, please let me know ;-)
I had the same problem.
In activate script file I need to add as first line (of cource after #!...):
unset PYTHON_PATH
Related
I am newbie in Automation (Python + Pytest).
I try to use clear virtualenv on my mac, but somehow Pytest runs from a local directory.
So, what I try to do:
I have a project with many pip packages (pytest, selene etc.).
I do :
$ pip3 install --user virtualenv
$ virtualenv --no-site-packages venv
$ cd venv/
$ source venv/bin/activate
$ echo $PATH
> /Users/.../venv/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/puppetlabs/bin
$ which python
> /Users/.../venv/bin/python
$ which pytest
> /usr/local/bin/pytest
And when I try to run my pytest - it does! But shouldn't!
If I delete Pytest from my mac as:
$ pip3 uninstall pytest
it works, and I got clear venv.
Python 3.7.4
This is pytest version 5.2.1, imported from /usr/local/lib/python3.7/site-packages/pytest.py
setuptools registered plugins: pytest-reportportal-1.0.4 at /usr/local/lib/python3.7/site-packages/pytest_reportportal/plugin.py
Can someone say what I do wrong? Or what is the problem with it?
How to create clear venv without dependencies from local PATH libs?
The behavior seems perfectly normal, working as intended.
Install pytest in your virtual environment, it will then take priority over the one globally installed, since it will be located higher up in your $PATH. Consider uninstalling the global one if you don't intend to use it.
I am working on mac OS X Yosemite, version 10.10.3.
I installed python2.7 and pip using macport as done in
http://johnlaudun.org/20150512-installing-and-setting-pip-with-macports/
I can successfully install packages and import them inside my python environment and python scripts. However any executable associated with a package that can be called from the command line in the terminal are not found.
Does anyone know what might be wrong? (More details below)
For example while installing a package called "rosdep" as instructed in http://wiki.ros.org/jade/Installation/Source
I can run: sudo pip install -U rosdep
which installs without errors and corresponding files are located in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
However if I try to run : sudo rosdep init,
it gives an error : "sudo: rosdep: command not found"
This is not a package specific error. I get this for any package installed using pip on my computer. I even tried adding
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
to my $PATH.
But the executables are not found on the command line, even though the packages work perfectly from within python.
I know the question asks about macOS, but here is a solution for Linux users who arrive here via Google.
I was having the issue described in this question, having installed the pdfx package via pip.
When I ran it however, nothing...
pip list | grep pdfx
pdfx (1.3.0)
Yet:
which pdfx
pdfx not found
The problem on Linux is that pip install ... drops scripts into ~/.local/bin and this is not on the default Debian/Ubuntu $PATH.
Here's a GitHub issue going into more detail: https://github.com/pypa/pip/issues/3813
To fix, just add ~/.local/bin to your $PATH, for example by adding the following line to your .bashrc file:
export PATH="$HOME/.local/bin:$PATH"
After that, restart your shell and things should work as expected.
Solution
Based on other answers, for linux and mac you can run the following:
echo "export PATH=\"`python3 -m site --user-base`/bin:\$PATH\"" >> ~/.bashrc
source ~/.bashrc
instead of python3 you can use any other link to python version: python, python2.7, python3.6, python3.9, etc.
instead of .bashrc, choose the rc file from your favourite shell.
Explanation
In order to know where the user packages are installed in the current OS (win, mac, linux), we run:
python3 -m site --user-base
We know that the scripts go to the bin/ folder where the packages are installed.
So we concatenate the paths:
echo `python3 -m site --user-base`/bin
Then we export that to an environment variable.
export PATH=\"`python3 -m site --user-base`/bin:\$PATH\"
Finally, in order to avoid repeating the export command we add it to our .bashrc file and we run source to run the new changes, giving us the suggested solution mentioned at the beginning.
On macOS with the default python installation you need to add /Users/<you>/Library/Python/2.7/bin/ to your $PATH.
Add this to your .bash_profile:
export PATH="/Users/<you>/Library/Python/2.7/bin:$PATH"
That's where pip installs the executables.
Tip: For non-default python version which python to find the location of your python installation and replace that portion in the path above. (Thanks for the hint Sanket_Diwale)
check your $PATH
tox has a command line mode:
audrey:tests jluc$ pip list | grep tox
tox (2.3.1)
where is it?
(edit: the 2.7 stuff doesn't matter much here, sub in any 3.x and pip's behaving pretty much the same way)
audrey:tests jluc$ which tox
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/tox
and what's in my $PATH?
audrey:tests jluc$ echo $PATH
/opt/chefdk/bin:/opt/chefdk/embedded/bin:/opt/local/bin:..../opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin...
Notice the /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin? That's what allows finding my pip-installed stuff
Now, to see where things are from Python, try doing this (substitute rosdep for tox).
$python
>>> import tox
>>> tox.__file__
that prints out:
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tox/__init__.pyc'
Now, cd to the directory right above lib in the above. Do you see a bin directory? Do you see rosdep in that bin? If so try adding the bin to your $PATH.
audrey:2.7 jluc$ cd /opt/local/Library/Frameworks/Python.framework/Versions/2.7
audrey:2.7 jluc$ ls -1
output:
Headers
Python
Resources
bin
include
lib
man
share
If you're installing using --user (e.g. pip3.6 install --user tmuxp), it is possible to get the platform-specific user install directory from Python itself using the site module. For example, on macOS:
$ python2.7 -m site --user-base
/Users/alexp/Library/Python/2.7
By appending /bin to this, we now have the path where package executables will be installed. We can dynamically populate the PATH in your shell's rc file based on the output; I'm using bash, but with any luck this is portable:
# Add Python bin directories to path
python3.6 -m site &> /dev/null && PATH="$PATH:`python3.6 -m site --user-base`/bin"
python2.7 -m site &> /dev/null && PATH="$PATH:`python2.7 -m site --user-base`/bin"
I use the precise Python versions to reduce the chance of the executables just "disappearing" when Python upgrades a minor version, e.g. from 3.5 to 3.6. They'll disappear because, as can be seen above, the user installation path may include the Python version. So while python3 could point to 3.5 or 3.6, python3.6 will always point to 3.6. This needs to be kept in mind when installing further packages, e.g. use pip3.6 over pip3.
If you don't mind the idea of packages disappearing, you can use python2 and python3 instead:
# Add Python bin directories to path
# Note: When Python is upgraded, packages may need to be re-installed
# or Python versions managed.
python3 -m site &> /dev/null && PATH="$PATH:`python3 -m site --user-base`/bin"
python2 -m site &> /dev/null && PATH="$PATH:`python2 -m site --user-base`/bin"
The Installing Packages tutorial on python.org describes how to locate the binary directory:
On Windows
You can find the user base binary directory by running
python -m site --user-site and replacing site-packages with Scripts.
For example, this could return
C:\Users\Username\AppData\Roaming\Python36\site-packages so you
would need to set your PATH to include
C:\Users\Username\AppData\Roaming\Python36\Scripts.
On Linux and macOS
On Linux and macOS you can find the user base binary directory by
running python -m site --user-base and adding bin to the end. For
example, this will typically print ~/.local (with ~ expanded to the
absolute path to your home directory) so you’ll need to add
~/.local/bin to your PATH.
I stumbled upon this question because I created, successfully built and published a PyPI Package, but couldn't execute it after installation. The $PATHvariable was correctly set.
In my case the problem was that I hadn't set the entry_pointin the setup.py file:
entry_points = {'console_scripts':
['YOUR_CONSOLE_COMMAND=MODULE_NAME.FILE_NAME:FUNCTION_NAME'],},
On Windows, you need to add the path %USERPROFILE%\AppData\Roaming\Python\Scripts to your path.
Windows and Python 3.9 from MS Store
I have a different path with python -m site --user-base and python -m site - yes, the second command without --user-base to get all sites - as the other answers here state:
C:\Users\<your User>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages
Why is my path different
Because I installed python from MS Store
Solution
Put the above path in your path and replace site-packages with scripts
When you install Python or Python3 using MacOS installer (downloaded from Python website) - it adds an exporter to your ~/.profile script. All you need to do is just source it. Restarting all the terminals should also do the trick.
WARNING - I believe it's better to just use pip3 with Python3 - for future benefits.
If you already have Python3 installed, the following steps work for me on macOS Mojave:
Install ansible first using sudo - sudo -H pip3 install ansible
you create a symlink to the Python's bin path
sudo ln -s /Library/Frameworks/Python.framework/Versions/Current/bin /Library/Frameworks/Python.framework/current_python_bin
and staple it to .profile
export PATH=$PATH:/Library/Frameworks/Python.framework/current_python_bin
run source ~/.profile and restart all terminal shells.
Type ansible --version
In addition to adding python's bin directory to $PATH variable, I also had to change the owner of that directory, to make it work. No idea why I wasn't the owner already.
chown -R ~/Library/Python/
I solve the problem!
Use pip3 instead pip.
pip3 install foobaz
vim ~/.zshrc and add:
export PATH="/Users/your_name/Library/Python/3.8/bin:$PATH"
source ~/.zshrc
Now MacOS has shifted the default terminal from bash to zsh. Therefore, you have to source zshrc but not bashrc or bash_profile.
foobaz -v
had the same issue with macOS Monterey. I had to modify my .bash_profile file and add the following entry
export PATH="~/Library/Python/3.8/bin:$PATH"
The default python version on macOS Monterey is 3.8, but you will have to double check your python version to make sure you're using the correct one
I am trying to create a virtual environment using virtualenv on Mac OS X El Capitan. I have installed Python 2.7.11 with brew, which includes pip, wheel and setuptools by default.
Hovewer, when I try to install virtualenv following instructions in the documentation or from any other resource, I get several problems:
virtualenv executable is not placed in /usr/local/bin after pip makes its job, so I need to ln -s it by hand (it may indicate, that there is something wrong with installation on this step).
After I run virtualenv venv, it creates new environment, catches Python 2.7.11 from brew-installation, but: there is no pip inside bin folder. That means, that if I try which pip, having venv activated, it returns a global position of pip — /usr/local/bin/pip, not /path/to/venv/bin/pip.
As a consequence, installing packages inside venv uses global pip and installs them to a global sites-packages, not that inside venv, and it's quite the opposite of what environment should do.
Could you please suggest what may be wrong and how to fix it?
EDIT: The thing to mention is that I used to have other versions of Python installed on my computer, which I have recently deleted as it is described in this answer. Maybe it causes the issue, and some more thorough cleaning is needed.
Try removing or renaming the .pydistutils.cfg file in your home directory, e.g. by renaming with mv ~/.pydistutils.cfg ~/oldpydistutils.cfg
I'm putting a detailed answer here to help others, but the original credit goes to this answer. If you know what specifically in .pydistutils.cfg was causing the problem, let me know!
I was having the same issue: my virtual environments were created without a local copy of pip, although they had a local copy of python. This meant that using $ pip from within the virtual environment installed to the global package location, and was not visible to the environment's python.
How I diagnosed this on my machine:
I create a virtualenvironment with $ virtualenv env
Activated the virtual environment with $ source env/bin/activate
Checked python location: run (env)$ which python with output /Users/<username>/env/bin/python (as expected)
Checked pip location: run (env)$ which pip with output /usr/local/bin/pip (NOT expected)
To check where our packages are going, we can try to install a package in the virtual environment:
Try to install a package: (env)$ pip install HTTPServer which succeeds
Try to run the package: (env)$ python -m HTTPServer which fails with error /Users/emunsing/env/bin/python: No module named HTTPServer
To double-check, try to install again: (env)$ pip install HTTPServer which produces Requirement already satisfied (use --upgrade to upgrade): HTTPServer in /usr/local/lib/python2.7/site-packages
Double-checking, we see that there's no Pip in the environment's /bin folder:
$ ls env/bin
activate activate.fish python python2
activate.csh activate_this.py python-config python2.7
And so while the system finds the local python version, it can't find a local pip to use and traverses the $PATH. It ended up using pip from /usr/local/bin, leaving me unable to install packages locally to the virtual environment.
Here's what I tried:
- Reinstalling python brew uninstall python followed by brew upgrade and brew install python --build-from-source
- Installing pip using the get-pip.py command as described in the Pip documentation
Here's what I ruled out:
- I was not using sudo pip ... which caused similar problems in this other question and haven't done so at any time on this Python/pip install
- My virtual environment didn't show a local installation of pip, as was the case in these similar questions: This one for Windows, This one for Mac OS X.
Ultimately, I found that eliminating the ~/.pydistutils.cfg file fixed the problem, allowing for fresh virtual environments that had their own local pip. The contents of my ~/.pydistutils.cfg file were:
[global]
verbose=1
[install]
install-scripts=$HOME/bin
[easy_install]
install-scripts=$HOME/bin
Simply renaming the ~/.pydistutils.cfg file appears to fix the problem: it seems that although this file was created by the homebrew installation, some settings in this file may be incompatible with virtualenv. While removing this file hasn't had any bad effects on my system, you may need to use the --user flag when installing packages with pip to the global environment (e.g. $ pip install --user HTTPServer). Here are more details on .pydistutils.cfg if you want to work on tailoring it for your needs.
virtualenv executable is not placed in /usr/local/bin after pip makes its job, so I need to ln -s it by hand (it may indicate, that there is something wrong with installation on this step).
Don't do that. That will only hide the bug and not solve the problem. Here's a short guide how to debug this kind of issues:
Start with which -a python. The first path you see should be /usr/local/bin/python, if not check your PATH variable.
Next, check which -a pip. Again the first path should be /usr/local/bin/pip. If not, run python -m ensurepip and recheck.
Now install virtualenv using pip install virtualenv, after that check the output of which -a virtualenv. The first path should be /usr/local/bin/virtualenv, if not check the output of env |grep PYTHON for unexpected environment variables.
Finally check the output of virtualenv --version to make sure you have the latest version.
I had the issue when running virtualenv: "ImportError: No module named pip."
My solution was to downgrade virtualenv. I had 16.2.0.
pip uninstall virtualenv
pip install virtualenv==15.1.0
Just hit same issue on Linux. Seems like there are multiple causes of this issue, but for me answer was to remove ~/.pip/.
Details: I had this in my .pip/pip.conf for some reason I can't remember:
$ cat ~/.pip/pip.conf
[global]
use_https = True
index = https://pypi.python.org/pypi
prefix = /home/sam/local/
and was using local versions on Python, Pip installed at ~/local/. For some reason virtualenv installed pip must pick up prefix = /home/sam/local/ setting and pip was being installed there.
Try this: sudo apt install pythonV.v-distutils.
In my case V.v == 3.8.
This worked for me.
I recently did a clean install of Mac OS 10.10.3 and am trying to set up a python environment. But I can't get virtualenvwrapper to work.
I installed python 2 and 3 using Homebrew.
$ which python
/usr/local/bin/python
$ which python3
/usr/local/bin/python3
$ which pip
/usr/local/bin/pip
$ which virtualenv
/usr/local/bin/virtualenv
$ which virtualenvwrapper.sh
/usr/local/bin/virtualenvwrapper.sh
Here are the lines in my .bash_profile
export PATH=/usr/local/bin:$PATH
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python
export VIRTUALENVWRAPPER_VIRTUALENV=/usr/local/bin/virtualenv
source /usr/local/bin/virtualenvwrapper.sh
The virtualenv works on its own with: $ source bin/activate
But when I source the virtualenvwrapper.sh. I get the following error message.
/usr/local/opt/python/bin/python2.7: No module named virtualenvwrapper
virtualenvwrapper.sh: There was a problem running the initialization hooks.
If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenvwrapper has been installed for
VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python and that PATH is
set properly.
I can't figure out what went wrong. When I echo VIRTUALENVWRAPPER_PYTHON in the terminal. It returns the right path. In fact, all my paths are correct. They all point to the homebrew directory.
How can I get it to work?
It has been some time since this question was asked, but encountering the same issue, I'd like to add my solution in case someone might find it usefull:
Since virtualenvwrapper was installed using Python3, it is axpected that VIRTUALENVWRAPPER_PYTHON variable will point to Python3 path and not the Python2 path.
That is, in .bash_profile change:export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python
To:export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python3
Then you can start using virtualwrapper, even if not installed for Python2 (unless you want to explicitly create the environment using Python2 in which case you'd probebly have to install it using pip2 and specify the relevant python path on the virtual environment creation command).
Per our comment exchange, you've installed virtualenvwrapper only on
python 3. You need to also install it on python2. To do so:
curl -O https://bootstrap.pypa.io/get-pip.py python2.7 ./get-pip.py
--user ~/.local/bin/pip install virtualenvwrapper My own setup is the opposite -- I use python2 for work, so install virtualenv for 3.x in
~/local -- hope that helps.
Thank you very much. Your answer points me to the right direction.
I solved the problem by uninstalling all things virtualenv.
pip uninstall virtualenv virtualenvwrapper
Then I installed everything again using pip2.7.
pip2.7 install virtualenv virtualenvwrapper
Now the terminal doesn't complain about can't find the module when I source virtualwrapper.sh
. /usr/local/bin/virtualenvwrapper.sh
However, python3 is the default python.
$ mkvirtualenv myenv
Using base prefix '/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4'
New python executable in myenv/bin/python3.4
Also creating executable in myenv/bin/python
Installing setuptools, pip...done.
To create a python2 env. I need to do this:
$ mkvirtualenv myenv --python /usr/local/bin/python
Running virtualenv with interpreter /usr/local/bin/python
New python executable in myenv/bin/python2.7
Also creating executable in myenv/bin/python
Installing setuptools, pip...done.
Per our comment exchange, you've installed virtualenvwrapper only on python 3. You need to also install it on python2. To do so:
curl -O https://bootstrap.pypa.io/get-pip.py
python2.7 ./get-pip.py --user
~/.local/bin/pip install virtualenvwrapper
My own setup is the opposite -- I use python2 for work, so install virtualenv for 3.x in ~/local -- hope that helps.
This has fixed my issue
Assuming you have already added an alias for python2 (the default version of python in mac)
Add these on ~/.bash_profile
export WORKON_HOME=~/.virtualenvs
VIRTUALENVWRAPPER_PYTHON='/usr/local/bin/python3'
source /usr/local/bin/virtualenvwrapper.sh
just modify the VIRTUALENVWRAPPER_PYTHON environment variable into this:
export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python3
and then run it by typing
source ~/.bash_profile
When I activate a venv, which pip returns /usr/local/bin/pip instead of path/to/my/apps/venv/bin/pop. Why is that?
I am inclined to just rm- rf the pip in /usr/local/bin/pip and install again, but since this is a production server I prefer not to guess too much :-)
My concern is that I have (in usr/local/bin):
easy_install
easy_install-2.6
pip
pip-2.6
virtualenv
virtualenv-2.6
python --version returns 2.6.6 and which python returns /usr/bin/python even though venvis activated?
Running Debian Squeeze
Check your virtualenv for a local directory. If venv/local exists, does it contain pip and easy_install? If it does, you can try working around this problem by editing venv/bin/activate and prepending venv/local/bin to your path just like venv/bin is being prepended.
Something like:
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
PATH="$VIRTUAL_ENV/local/bin:$PATH" # my new line
export PATH
I'm having a problem similar to what I've described, and unfortunately I have not run it to ground yet. See also: Why do distribute and pip install to my virtualenv's ./local/bin?
What shell are you using? What specific command did you use to activate the virtualenv?
In my case (also using squeeze) I am using bash and it if I run "source bin/activate" then everything in my path (pip, python, etc) is correct.