We Keep Coding

What will happen to packages installed when the virtual environment is deleted?

I create a new project with the following commands:
mkdir ~/my_project
python -m venv ~/my_project
source ～/my_project/Scripts/activate
pip install flask
pip install kivy
pip install foo
pip install bar
And let's say that I decide to remove my_project and create a new project:
rm -rf ~/my_project
mkdir ~/new_project
python -m venv ~/new_project
source ~/Scripts/activate
Questions:
What happens to previously installed packages "flask", "kivy", "foo", and "bar"? Do I need to re-install them for my new_project's virtual environment?
If I don't need to re-install these packages, then I am missing the point of virtual environment? I thought the whole point of virtual environment is that packages installed in the virtual environment is isolated in that environment. Can someone elaborate?

Indeed, pip installs libraries into a subdirectory of the virtualenv when the env is active. Anything in there is independent of any system-wide installs, and vice versa. Removing the env obviously removes the stuff in the subdirectories.
On the other hand, virtualenv can optionally fall back to system-installed packages if you let it. The system-wide installs will obviously still be there after you remove the virtualenv.
As an aside, the current activate hard-codes some things so that you cannot even rename a virtualenv directory; you have to zap and reinstall it. Keeping all the things it needs in a requirements.txt or similar is a good way to simplify this process, as well as document the dependencies.

The point of Virtual Environment is to separate your development environment from your actual environment. Different projects have different package dependencies. So for these cases Virtual Environment comes in handy.
If you installed flask, kivy, foo, and bar inside a virtual environment, then YES, you need to install them when you move into another Virtual Environment (That's the whole purpose of Virtula Environment).
If you find that the packages are available globally then may be you are not using it correctly. You need to activate your Virtual Environment before you install anything or the packages will be installed globally.
For your case I can see you are activating before installing packages. So it should work right. you can always use pip freeze to see what packages are installed.

Running homebrew while in a virtualenv

Simple question: is running homebrew while in a virtualenv a bad idea?
If so, is there any way that I can automatically deactivate the virtualenv each time I run a homebrew command? I don't trust myself to always remember to deactivate the virtualenv or open a new terminal window.

Since I just ran across this myself in the documentation, it's probably better in general not to brew install from within an active virtualenv. To quote from said docs:
WARNING: When you brew install formulae that provide Python bindings, you should not be in an active virtual environment.
Activate the virtualenv after you've brewed, or brew in a fresh Terminal window. Homebrew will still install Python modules into Homebrew's site-packages and not into the virtual environment's site-package.
Virtualenv has a switch to allow "global" (i.e. Homebrew's) site-packages to be accessible from within the virtualenv.
As such, if you are certain the formula you are brewing will not attempt to provide new Python bindings then let it rip. But for me, if there's no specific reason to brew install with my virtual environment activated, I try and remember to deactivate beforehand.

I don't think if they are related. You have to use pip for python package management when you use virtualenv. this way you make sure that your new stuff is on the sandbox you created. AFAIK home-brew installs stuff globally. So better not use it to get the python modules. hope it helps.

Make virtualenv inherit specific packages from your global site-packages

I'm looking for a way to make a virtualenv which will contain just some libraries (which I chose) of the base python installation.
To be more concrete, I'm trying to import my matplotlib to virtualenv during the creation of virtualenv. It can't be installed efficiently with pip or easy_install since it misses some fortran compiler libs. The way I did it until now was to manually copy from:
/usr/lib/python2.7/dist-packages/ to virtualenv_name/lib/python2.7/dist-packages/
However this prevents the manully imported links to be registerd by yolk (which prints all currently available libs in virtualenv).
So, is there a way to do a selective variant of the
virtualenv --system-site-packages

Create the environment with virtualenv --system-site-packages . Then, activate the virtualenv and when you want things installed in the virtualenv rather than the system python, use pip install --ignore-installed or pip install -I . That way pip will install what you've requested locally even though a system-wide version exists. Your python interpreter will look first in the virtualenv's package directory, so those packages should shadow the global ones.

You can use the --system-site-packages and then "overinstall" the specific stuff for your virtualenv. That way, everything you install into your virtualenv will be taken from there, otherwise it will be taken from your system.

I am late to the game using python.3.8 and pip3 on Ubuntu 20.04.
The ONLY way to get rid of the annoying .local install for me was to set an environment variable (bash):
export PYTHONNOUSERSITE="true"
This does not need to be "true" anything will work. I would not go for a 0. ;-)

Install virtual env with
virtualenv --system-site-packages
and use pip install -U to install matplotlib

System PIP instead of virtualenv PIP by default?

After using virtualenv with pip off-and-on for a couple of days, I've found that the version of PIP that is used after the virtualenv is actived is the global PIP instead of the PIP relative to that environment; such that if you don't set the shell environment variable export PIP_RESPECT_VIRTUALENV=true, pip will install whatever new package (e.g. pip install argparse) to the global scope instead of only to the virtualenv.
I would expect PIP to install to the virtualenv by default, if that virtualenv is activated.
Is there a reasoning behind it not working that way by default?
See explanation here for how PIP_RESPECT_VIRTUALENV works.

When you create a virtualenv, the activate file hardcodes the variable VIRTUAL_ENV to the location in which you first created the root directory. This variable is then exported when you source <your-venv>/bin/activate.
Consequently, if you move the virtualenv directory subsequent to its creation, the hardcoded file path will be incorrect.
Just open <your-venv>/bin/activate in a text editor and make sure VIRTUAL_ENV is set to the new path of your virtualenv directory:
VIRTUAL_ENV="/Full/path/to/<your-venv>"
export VIRTUAL_ENV
before running source <your-venv>/bin/activate again.
Then of course you can test the version of pip with which pip which should produce:
/Full/path/to/<your-venv>/bin/pip
rather than /usr/bin/pip or /bin/pip etc.

It is not the first time I see someone reporting the same issue. I don't know what is happening, but some people discourage the use o source /path/to/venv/bin/activate because it can mess up your $PATH.
There is a way pip will always respect your virtualenv: don't rely on $PATH. Use:
/path/to/venv/bin/pip install MYPACKAGE
It would be nice to find out what is happening to you and share your solution with others. Meanwhile, it may be ok to use the absolute path to pip.

Renaming a virtualenv folder without breaking it

I've created folder and initialized a virtualenv instance in it.
$ mkdir myproject
$ cd myproject
$ virtualenv env
When I run (env)$ pip freeze, it shows the installed packages as it should.
Now I want to rename myproject/ to project/.
$ mv myproject/ project/
However, now when I run
$ . env/bin/activate
(env)$ pip freeze
it says pip is not installed. How do I rename the project folder without breaking the environment?

You need to adjust your install to use relative paths. virtualenv provides for this with the --relocatable option. From the docs:
Normally environments are tied to a
specific path. That means that you
cannot move an environment around or
copy it to another computer. You can
fix up an environment to make it
relocatable with the command:
$ virtualenv --relocatable ENV
NOTE: ENV is the name of the virtual environment and you must run this from outside the ENV directory.
This will make some of the files
created by setuptools or distribute
use relative paths, and will change
all the scripts to use
activate_this.py instead of using the
location of the Python interpreter to
select the environment.
Note: you must run this after you've
installed any packages into the
environment. If you make an
environment relocatable, then install
a new package, you must run virtualenv
--relocatable again.

I believe "knowing why" matters more than "knowing how". So, here is another approach to fix this.
When you run . env/bin/activate, it actually executes the following commands (using /tmp for example):
VIRTUAL_ENV="/tmp/myproject/env"
export VIRTUAL_ENV
However, you have just renamed myproject to project, so that command failed to execute.
That is why it says pip is not installed, because you haven't installed pip in the system global environment and your virtualenv pip is not sourced correctly.
If you want to fix this manually, this is the way:
With your favorite editor like Vim, modify /tmp/project/env/bin/activate usually in line 42:
VIRTUAL_ENV='/tmp/myproject/env' => VIRTUAL_ENV='/tmp/project/env'
Modify /tmp/project/env/bin/pip in line 1:
#!/tmp/myproject/env/bin/python => #!/tmp/project/env/bin/python
After that, activate your virtual environment env again, and you will see your pip has come back again.

NOTE: As #jb. points out, this solution only applies to easily (re)created virtualenvs. If an environment takes several hours to install this solution is not recommended
Virtualenvs are great because they are easy to make and switch around; they keep you from getting locked into a single configuration. If you know the project requirements, or can get them, Make a new virtualenv:
Create a requirements.txt file
(env)$ pip freeze > requirements.txt
If you can't create the requirements.txt file, check env/lib/pythonX.X/site-packages before removing the original env.
Delete the existing (env)
deactivate && rm -rf env
Create a new virtualenv, activate it, and install requirements
virtualenv env && . env/bin/activate && pip install -r requirements.txt
Alternatively, use virtualenvwrapper to make things a little easier as all virtualenvs are kept in a centralized location
$(old-venv) pip freeze > temp-reqs.txt
$(old-venv) deactivate
$ mkvirtualenv new-venv
$(new-venv) pip install -r temp-reqs.txt
$(new-venv) rmvirtualenv old-venv

I always install virtualenvwrapper to help out. From the shell prompt:
pip install virtualenvwrapper
There is a way documented in the virtualenvwrapper documents - cpvirtualenv
This is what you do. Make sure you are out of your environment and back to the shell prompt. Type in this with the names required:
cpvirtualenv oldenv newenv
And then, if necessary:
rmvirtualenv oldenv
To go to your newenv:
workon newenv

You can fix your issue by following these steps:
rename your directory
rerun this: $ virtualenv ..\path\renamed_directory
virtualenv will correct the directory associations while leaving your packages in place
$ scripts/activate
$ pip freeze to verify your packages are in place
An important caveat, if you have any static path dependencies in script files in your virtualenv directory, you will have to manually change those.

Yet another way to do it that worked for me many times without problems is virtualenv-clone:
pip install virtualenv-clone
virtualenv-clone old-dir/env new-dir/env

Run this inside your project folder:
cd bin
sed -i 's/old_dir_name/new_dir_name/g' *
Don't forget to deactivate and activate.

In Python 3.3+ with built-in venv
As of Python 3.3 the virtualenv package is now built-in to Python as the venv module. There are a few minor differences, one of which is the --relocatable option has been removed. As a result, it is normally best to recreate a virtual environment rather than attempt to move it. See this answer for more information on how to do that.
What is the purpose behind wanting to move rather than just recreate any virtual environment? A virtual environment is intended to manage the dependencies of a module/package with the venv so that it can have different and specific versions of a given package or module it is dependent on, and allow a location for those things to be installed locally.
As a result, a package should provide a way to recreate the venv from scratch. Typically this is done with a requirements.txt file and sometimes also a requirements-dev.txt file, and even a script to recreate the venv in the setup/install of the package itself.
One part that may give headaches is that you may need a particular version of Python as the executable, which is difficult to automate, if not already present. However, when recreating an existing virtual environment, one can simply run python from the existing venv when creating the new one. After that it is typically just a matter of using pip to reinstall all dependencies from the requirements.txt file:
From Git Bash on Windows:
python -m venv mynewvenv
source myvenv/Scripts/activate
pip install -r requirements.txt
It can get a bit more involved if you have several local dependencies from other locally developed packages, because you may need to update local absolute paths, etc. - though if you set them up as proper Python packages, you can install from a git repo, and thus avoid this issue by having a static URL as the source.

virtualenv --relocatable ENV is not a desirable solution. I assume most people want the ability to rename a virtualenv without any long-term side effects.
So I've created a simple tool to do just that. The project page for virtualenv-mv outlines it in a bit more detail, but essentially you can use virtualenv-mv just like you'd use a simple implementation of mv (without any options).
For example:
virtualenv-mv myproject project
Please note however that I just hacked this up. It could break under unusual circumstances (e.g. symlinked virtualenvs) so please be careful (back up what you can't afford to lose) and let me know if you encounter any problems.

Even easier solution which worked for me: just copy the site-packages folder of your old virtual environment into a new one.

Using Visual Studio Code (vscode), I just opened the ./env folder in my project root, and did a bulk find/replace to switch to my updated project name. This resolved the issue.
Confirm with which python

If you are using an conda env,
conda create --name new_name --clone old_name
conda remove --name old_name --all # or its alias: `conda env remove --name old_name`