After using virtualenv with pip off-and-on for a couple of days, I've found that the version of PIP that is used after the virtualenv is actived is the global PIP instead of the PIP relative to that environment; such that if you don't set the shell environment variable export PIP_RESPECT_VIRTUALENV=true, pip will install whatever new package (e.g. pip install argparse) to the global scope instead of only to the virtualenv.
I would expect PIP to install to the virtualenv by default, if that virtualenv is activated.
Is there a reasoning behind it not working that way by default?
See explanation here for how PIP_RESPECT_VIRTUALENV works.
When you create a virtualenv, the activate file hardcodes the variable VIRTUAL_ENV to the location in which you first created the root directory. This variable is then exported when you source <your-venv>/bin/activate.
Consequently, if you move the virtualenv directory subsequent to its creation, the hardcoded file path will be incorrect.
Just open <your-venv>/bin/activate in a text editor and make sure VIRTUAL_ENV is set to the new path of your virtualenv directory:
VIRTUAL_ENV="/Full/path/to/<your-venv>"
export VIRTUAL_ENV
before running source <your-venv>/bin/activate again.
Then of course you can test the version of pip with which pip which should produce:
/Full/path/to/<your-venv>/bin/pip
rather than /usr/bin/pip or /bin/pip etc.
It is not the first time I see someone reporting the same issue. I don't know what is happening, but some people discourage the use o source /path/to/venv/bin/activate because it can mess up your $PATH.
There is a way pip will always respect your virtualenv: don't rely on $PATH. Use:
/path/to/venv/bin/pip install MYPACKAGE
It would be nice to find out what is happening to you and share your solution with others. Meanwhile, it may be ok to use the absolute path to pip.
Related
I am trying to install and use python3 packages to /home/myname/pp folder. I should be able to run python3 from anywhere. Also, pip3 should be able to update the packages in this folder. I should also be able to copy this folder to a new Linux system and it should work there as well (by changing PYTHONPATH there).
I searched and found following options:
pip install -t <direct directory> <package> # I prefer this.
and
pip install --install-option="--prefix=$PREFIX_PATH" package_name
or use:
virtualenv
and then I need to do:
echo 'export PYTHONPATH="/home/myname/pp:$PYTHONPATH"' >> ~/.bash_profile
What should be my approach for these requirements? Thanks for your help.
pip install -t <direct directory> <package>
Will install the package globally in the given directory.
pip install --install-option="--prefix=$PREFIX_PATH" package_name
Will run the package's setup.py with the given parameters, as mentioned in the pip's help:
--install-option Extra arguments to be supplied to the setup.py install command (use like
--install-option="--install-scripts=/usr/local/bin"). Use multiple --install-
option options to pass multiple options to setup.py install. If you are using an option with a directory path,
be sure to use absolute path.
The recommended way to do to install packages is to use a virtual environment. It keeps your global packages sanitized, in case you want the same package but different versions for it in two different projects for example.
virtualenv basically creates a folder for where to store the installed packages.
In a linux-based system, you would have to run the virtualenv commmand to create the folder and after that "activate" it.
virtualenv my_virtual_environment
source my_virtual_environment/bin/activate
You will notice that the environment's name will appear at the end of shell line. What activate does is simply changing some paths in your PATH environment variable to point to your current virtual environment folder.
It will still used the system's python interpreter but when trying to import packages in your program, it will look in the virtual environment's folder first.
To return to the global python packages, just type deactivate.
If you want the environment you're using to be active right when you start the terminal, add the source command to your .bash_profile or .bashrc. I recommend using the absolute path to the python virtual environment.
If you're working on multiple projects and want to keep the packages separate from each other, create multiple virtual environments and just switch to them. You could take a look at virtalenvwrapper which makes starting the virtual environment when you open the terminal and switching between other environments really easy.
I am trying to instal virtualenv and/or virtualenvwrapper on a Mac OS X 10.8.3.
I have been fighting with python for the last two days. Finally I was able to install Python 2.7.4 using brew. Before I had virtualenv installed using easy_install. Then I tried to uninstall it, trying to get my computer in the same situation as the one of my colleagues. Maybe I uninstalled it with success, maybe not. I don't know how to test it. Now I am supposed to install virtualenv using:
pip install virtualenv
But it gives me:
Could not find an activated virtualenv (required).
pip install virtualenvwrapper gives exactly the same output.
Also the variable: PIP_RESPECT_VIRTUALENV is null:
echo $PIP_RESPECT_VIRTUALENV
How can I solve this issue?
Open your ~/.bashrc file and see if this line is there -
export PIP_REQUIRE_VIRTUALENV=true
It might be causing the trouble. If it's there, change it to false and run -
source ~/.bashrc
If not, run export PIP_REQUIRE_VIRTUALENV=false from terminal.
Note: everything works the same if you have .bash_profile instead of .bashrc in your current user's root directory.
#Bibhas has it; +1 to look for export PIP_REQUIRE_VIRTUALENV=true in ~/.profile or ~/.bashrc. You can confirm the setting in your current shell with env |grep PIP_REQUIRE_VIRTUALENV.
This setting is a good safety check; more often than not, you'll want to be installing things into virtualenvs. However, sometimes you do want to be working with the global/system python. In those cases, take a look at --isolated:
Run pip in an isolated mode, ignoring environment variables and user configuration.
$ pip install --upgrade pip
Could not find an activated virtualenv (required).
$ pip install --upgrade pip --isolated
Requirement already up-to-date: pip in /usr/local/lib/python2.7/site-packages
$ pip freeze --isolated
...
An additional solution to those already presented is to add a shell command that will allow you to install py packages by temporarily overriding the default setting. Add this to your ~/.profile, ~/.bashrc or wherever you maintain your shell's exports/settings (in my case, ~/.zshrc).
syspip(){
PIP_REQUIRE_VIRTUALENV="" pip "$#"
}
With this simple addition, you can install pip packages to the system via syspip install <package>.
Verify contents of ~/.pip/pip.conf like:
[global]
index=https://pypi.python.org/simple/
require-virtualenv=false
if previous it was set like require-virtualenv=true
Another place where you may possibly have this "lock" is the pip.conf file. In my case I had one in my ~/Library/Application Support/pip folder and forgot about it.
Typical content of the file could be:
[install]
require-virtualenv = true
[uninstall]
require-virtualenv = true
Similar to other answers, false should be changed to true in the file.
Important to heed #JCotton's advice here-- keeping your pip setup so as to only install into virtualenvs is a great practice.
His solution to get virtualenv setup again of pip install --upgrade pip --isolated is exactly what should be done.
You should NOT turn off requiring a virtualenv, either by config file or by editing ~/.bash_rc or ~/.bash_profile, to get your project's pip packages installed. We're only doing it here because OP needs virtualenv itself installed.
In general, I see people get this message when their virtualenv wasn't setup correctly for their project in the first place. Reminder that to create a virtualenv with its own python and pip so that you don't run into the "could not find an activated virtualenv" error, you run virtualenv -p python3
for matchbook you must go to '.bash_profile '
1) open with your favorite editor in terminal
nano .bash_profile OR vim .bash_profile
2) find the text line that says
export PIP_REQUIRE_VIRTUALENV=true
3) delete it or set it equal to "false"
4) finally restart your terminal
I'm looking for a way to make a virtualenv which will contain just some libraries (which I chose) of the base python installation.
To be more concrete, I'm trying to import my matplotlib to virtualenv during the creation of virtualenv. It can't be installed efficiently with pip or easy_install since it misses some fortran compiler libs. The way I did it until now was to manually copy from:
/usr/lib/python2.7/dist-packages/ to virtualenv_name/lib/python2.7/dist-packages/
However this prevents the manully imported links to be registerd by yolk (which prints all currently available libs in virtualenv).
So, is there a way to do a selective variant of the
virtualenv --system-site-packages
Create the environment with virtualenv --system-site-packages . Then, activate the virtualenv and when you want things installed in the virtualenv rather than the system python, use pip install --ignore-installed or pip install -I . That way pip will install what you've requested locally even though a system-wide version exists. Your python interpreter will look first in the virtualenv's package directory, so those packages should shadow the global ones.
You can use the --system-site-packages and then "overinstall" the specific stuff for your virtualenv. That way, everything you install into your virtualenv will be taken from there, otherwise it will be taken from your system.
I am late to the game using python.3.8 and pip3 on Ubuntu 20.04.
The ONLY way to get rid of the annoying .local install for me was to set an environment variable (bash):
export PYTHONNOUSERSITE="true"
This does not need to be "true" anything will work. I would not go for a 0. ;-)
Install virtual env with
virtualenv --system-site-packages
and use pip install -U to install matplotlib
When I activate a venv, which pip returns /usr/local/bin/pip instead of path/to/my/apps/venv/bin/pop. Why is that?
I am inclined to just rm- rf the pip in /usr/local/bin/pip and install again, but since this is a production server I prefer not to guess too much :-)
My concern is that I have (in usr/local/bin):
easy_install
easy_install-2.6
pip
pip-2.6
virtualenv
virtualenv-2.6
python --version returns 2.6.6 and which python returns /usr/bin/python even though venvis activated?
Running Debian Squeeze
Check your virtualenv for a local directory. If venv/local exists, does it contain pip and easy_install? If it does, you can try working around this problem by editing venv/bin/activate and prepending venv/local/bin to your path just like venv/bin is being prepended.
Something like:
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
PATH="$VIRTUAL_ENV/local/bin:$PATH" # my new line
export PATH
I'm having a problem similar to what I've described, and unfortunately I have not run it to ground yet. See also: Why do distribute and pip install to my virtualenv's ./local/bin?
What shell are you using? What specific command did you use to activate the virtualenv?
In my case (also using squeeze) I am using bash and it if I run "source bin/activate" then everything in my path (pip, python, etc) is correct.
I've created folder and initialized a virtualenv instance in it.
$ mkdir myproject
$ cd myproject
$ virtualenv env
When I run (env)$ pip freeze, it shows the installed packages as it should.
Now I want to rename myproject/ to project/.
$ mv myproject/ project/
However, now when I run
$ . env/bin/activate
(env)$ pip freeze
it says pip is not installed. How do I rename the project folder without breaking the environment?
You need to adjust your install to use relative paths. virtualenv provides for this with the --relocatable option. From the docs:
Normally environments are tied to a
specific path. That means that you
cannot move an environment around or
copy it to another computer. You can
fix up an environment to make it
relocatable with the command:
$ virtualenv --relocatable ENV
NOTE: ENV is the name of the virtual environment and you must run this from outside the ENV directory.
This will make some of the files
created by setuptools or distribute
use relative paths, and will change
all the scripts to use
activate_this.py instead of using the
location of the Python interpreter to
select the environment.
Note: you must run this after you've
installed any packages into the
environment. If you make an
environment relocatable, then install
a new package, you must run virtualenv
--relocatable again.
I believe "knowing why" matters more than "knowing how". So, here is another approach to fix this.
When you run . env/bin/activate, it actually executes the following commands (using /tmp for example):
VIRTUAL_ENV="/tmp/myproject/env"
export VIRTUAL_ENV
However, you have just renamed myproject to project, so that command failed to execute.
That is why it says pip is not installed, because you haven't installed pip in the system global environment and your virtualenv pip is not sourced correctly.
If you want to fix this manually, this is the way:
With your favorite editor like Vim, modify /tmp/project/env/bin/activate usually in line 42:
VIRTUAL_ENV='/tmp/myproject/env' => VIRTUAL_ENV='/tmp/project/env'
Modify /tmp/project/env/bin/pip in line 1:
#!/tmp/myproject/env/bin/python => #!/tmp/project/env/bin/python
After that, activate your virtual environment env again, and you will see your pip has come back again.
NOTE: As #jb. points out, this solution only applies to easily (re)created virtualenvs. If an environment takes several hours to install this solution is not recommended
Virtualenvs are great because they are easy to make and switch around; they keep you from getting locked into a single configuration. If you know the project requirements, or can get them, Make a new virtualenv:
Create a requirements.txt file
(env)$ pip freeze > requirements.txt
If you can't create the requirements.txt file, check env/lib/pythonX.X/site-packages before removing the original env.
Delete the existing (env)
deactivate && rm -rf env
Create a new virtualenv, activate it, and install requirements
virtualenv env && . env/bin/activate && pip install -r requirements.txt
Alternatively, use virtualenvwrapper to make things a little easier as all virtualenvs are kept in a centralized location
$(old-venv) pip freeze > temp-reqs.txt
$(old-venv) deactivate
$ mkvirtualenv new-venv
$(new-venv) pip install -r temp-reqs.txt
$(new-venv) rmvirtualenv old-venv
I always install virtualenvwrapper to help out. From the shell prompt:
pip install virtualenvwrapper
There is a way documented in the virtualenvwrapper documents - cpvirtualenv
This is what you do. Make sure you are out of your environment and back to the shell prompt. Type in this with the names required:
cpvirtualenv oldenv newenv
And then, if necessary:
rmvirtualenv oldenv
To go to your newenv:
workon newenv
You can fix your issue by following these steps:
rename your directory
rerun this: $ virtualenv ..\path\renamed_directory
virtualenv will correct the directory associations while leaving your packages in place
$ scripts/activate
$ pip freeze to verify your packages are in place
An important caveat, if you have any static path dependencies in script files in your virtualenv directory, you will have to manually change those.
Yet another way to do it that worked for me many times without problems is virtualenv-clone:
pip install virtualenv-clone
virtualenv-clone old-dir/env new-dir/env
Run this inside your project folder:
cd bin
sed -i 's/old_dir_name/new_dir_name/g' *
Don't forget to deactivate and activate.
In Python 3.3+ with built-in venv
As of Python 3.3 the virtualenv package is now built-in to Python as the venv module. There are a few minor differences, one of which is the --relocatable option has been removed. As a result, it is normally best to recreate a virtual environment rather than attempt to move it. See this answer for more information on how to do that.
What is the purpose behind wanting to move rather than just recreate any virtual environment? A virtual environment is intended to manage the dependencies of a module/package with the venv so that it can have different and specific versions of a given package or module it is dependent on, and allow a location for those things to be installed locally.
As a result, a package should provide a way to recreate the venv from scratch. Typically this is done with a requirements.txt file and sometimes also a requirements-dev.txt file, and even a script to recreate the venv in the setup/install of the package itself.
One part that may give headaches is that you may need a particular version of Python as the executable, which is difficult to automate, if not already present. However, when recreating an existing virtual environment, one can simply run python from the existing venv when creating the new one. After that it is typically just a matter of using pip to reinstall all dependencies from the requirements.txt file:
From Git Bash on Windows:
python -m venv mynewvenv
source myvenv/Scripts/activate
pip install -r requirements.txt
It can get a bit more involved if you have several local dependencies from other locally developed packages, because you may need to update local absolute paths, etc. - though if you set them up as proper Python packages, you can install from a git repo, and thus avoid this issue by having a static URL as the source.
virtualenv --relocatable ENV is not a desirable solution. I assume most people want the ability to rename a virtualenv without any long-term side effects.
So I've created a simple tool to do just that. The project page for virtualenv-mv outlines it in a bit more detail, but essentially you can use virtualenv-mv just like you'd use a simple implementation of mv (without any options).
For example:
virtualenv-mv myproject project
Please note however that I just hacked this up. It could break under unusual circumstances (e.g. symlinked virtualenvs) so please be careful (back up what you can't afford to lose) and let me know if you encounter any problems.
Even easier solution which worked for me: just copy the site-packages folder of your old virtual environment into a new one.
Using Visual Studio Code (vscode), I just opened the ./env folder in my project root, and did a bulk find/replace to switch to my updated project name. This resolved the issue.
Confirm with which python
If you are using an conda env,
conda create --name new_name --clone old_name
conda remove --name old_name --all # or its alias: `conda env remove --name old_name`