Create Python Virtual Environment in a Specific Directory - python

I have Anaconda and I've been playing around with setting up virtual environments since I have scripts that I need to run that have been written in Python 2 and Python 3. I want to be able to activate my Python 3 virtual environment in a specific directory - ie Python 3 will only run in that directory and all other directories will remain at the default Python 2 that I have set in Anaconda. My problem is that every time I try to activate a new environment, it changes the version of python used everywhere on my machine instead of just in the one directory that I want. Is it possible to create a virtual environment that is limited to a specific location?
I've tried the following:
conda create --prefix=testEV1 python=3.5
source activate testEV1
and this changes my version of python everywhere in my workspace to 3.5.

No. You only have a single default Python installation in effect at any one time.
Once you're done using one virtualenv you can use deactivate to go "back" to the standard, physical default Python installation.
Or you can use different command sessions with different virtual environments activated in each session.
Or you can explicitly invoke one version or another of Python from the command line each time, rather than just using the currently active-by-default one.

It is possible to autoactivate a conda environment when entering a specific directory.
https://github.com/conda/conda/issues/5179
BUT, it doesn't change the fact that source activate xxx affect your shell/prompt instead of your directory structure. You can still manually activate an environment and it will still affect your whole prompt.

Related

How do I reroute a project to another Python version in PyCharm? (Error: No Python at 'C:\Users\...\python.exe')

I wanted to replace Python 3.8 32-bit with the 64-bit version to install the face_recognition module, so I deleted the previous version and tried to re-route the project to the new Python version by going to File > Settings > Project Interpreter > Show all > Show Paths for Selected Interpreter, and adding all the Python files from the new folder and getting rid of the old ones.
However, it's still showing me this error when I try to install the module:
(Will2.0) C:\Users\solei\PycharmProjects\Will>pip install face_recognition
No Python at 'C:\Users\solei\AppData\Local\Programs\Python\Python38-32\python.exe'
I've also tried going to the Windows System Properties and changing everything that says "Python38-32" there, but it's still not working. It does work when I make a new environment, though, so at least I know that Python installed properly. It's just this one environment that is tripping me up (I'd prefer not to make a new project for this, btw. I've already installed a lot of modules in it.).
Your selected interpreter is not the system interpreter you've replaced with the 64-bit version, but your project's virtual environment interpreter. The virtual environment's files weren't changed in that process and need to be updated before you can use that environment again.
The system interpreter is your Python interpreter installed using the installation executable. In your case it is located in C:\Users\solei\AppData\Local\Programs\Python\Python38\. You can have multiple system interpreters installed, such as having Python 2.7, Python 3.7 and Python 3.8 side-by-side.
The virtual environment interpreter is a copy of another interpreter created using the venv package from the Python standard library. You can have many virtual environments interpreters in the system (one or more for every project, for example)
The base interpreter is the interpreter that was used as a template for the venv package. Every virtual environment interpreter has its base interpreter (usually a system interpreter) that it requires to run. Changing or upgrading the base interpreter requires updating the virtual environment.
If we take a quick look at the documentation, a virtual environment is described as
a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages.
That means you can setup an individual environment for every project, which will contain its own packages. The environment is a very efficient way of managing project packages, that's why PyCharm suggests a creation of such environment over the system interpreter by default. In short, it allows you to have two different versions of the same package used by two different projects, without the packages conflicting with each other.
This also explains why your virtual environment files weren't affected by your upgrade.
Now, I am unfortunately no Python expert. I had to spend some time examining how Python handles virtual environments on Windows and Ubuntu. It seems the environment always requires the base system interpreter present in the system. If you remove or change the location of the base interpreter, the environment will fail to function.
As I mentioned before editing this answer, you can in theory simply edit the pyenv.cfg file located in the root folder of the virtual environment. In practice, that will only work in simple cases and it is not the intended way of updating virtual environments.
You need to upgrade your virtual environment's files to work with your new system interpreter. That can mean the 64-bit version over the 32-bit version, or even a newer version of Python - such us upgrading from 3.7 to 3.8.
Close PyCharm
Check if the system interpreter you want to upgrade to is on the system Path
You can quickly check by running
python -c "import platform; print(platform.architecture())"
For you, the output should look like this
('64bit', 'WindowsPE')
If your output is different, you'll need to prefix the absolute path to the Python executable in step 4).
Navigate to the virtual environment's directory
The directory you're looking for contains the Include, Lib and Scripts directories and the pyenv.cfg file. From your screenshots, it seems this directory is your project's root directory, so in your case:
cd C:\Users\solei\PycharmProjects\Will2.0\
Upgrade the virtual environment
python -m venv --upgrade .
... or if Python is not on your path
C:\Users\solei\AppData\Local\Programs\Python\Python38\python.exe -m venv --upgrade .
The . in the commands refers to the current directory.
Open PyCharm and verify your environment is working correctly
... or simply try to run pip directly from the command line. Note you need to first activate the virtual environment by running the Scripts\activate.bat batch file.
If the above-mentioned method doesn't work, you might have to create a new virtual environment. You can create one easily without making a new PyCharm project. See this PyCharm documentation for reference. However, you'll still need to redownload all the required packages again.
For the simplicity, I recommend creating the new virtual environment in a .venv folder located in the project's root.
Disclaimer
I tested only the Python's behavior alone on a fresh Windows installation inside the Windows Sandbox. I was able to install the 32-bit Python, create a virtual environment, replace Python with the 64-bit version and upgrade the virtual environment to have it launch correctly again.

Virtual Environment calling global installation of Python instead of venv-specifc version

I'm having an issue where for some reason the virtual environments that I'm creating are accessing my system-wide installations of Python and pip when they shouldn't be.
Here's my fairly simple workflow just make sure I'm not missing anything obvious (Windows 10, Python 3.8.2):
python -m venv venv
venv\Scripts\activate.bat
My path is now prepended by (venv), as you'd expect. However,
pip list
lists all of the system-wide pip packages I have instead of just the ones that should be in that venv.
pyvenv.cfg indicates that
include-system-site-packages = false
When I open the interpreter using
python
In the virtual environment,
sys.executable
Returns the path on my C drive and
print(pip.__file__)
Does the same. I suspect they should instead be pointing to the interpreter and pip in the virtual environment but don't know how to make that happen.
Edit: 4/27/20, Still dealing with this issue, I have tried:
uninstalling and reinstalling Python, both from python.org and the MS Store
Installing python in a new location
Clearing my user and system environment variables and then adding in
just the ones for Python 3.8.
I'm really at a loss here, would appreciate any help.
To anyone looking at this later on, I eventually realized that the problem was only occurring in a specific folder. Making virtual environments elsewhere on my computer seems to work fine. Why this is the case? I have no idea. I'm curious, but not enough to spend another second on this problem. So there you go.
This happened to me too. My problem was that I created the virtual environment under the directory including the global executable (I mean C:\Program Files\Python39). Once I removed this virtual environment and created a new one somewhere else (for example C:\Python), running python command took the executable in the virtual environment rather than the global one.
I had the same problem too. It might be that the path to your virtual environment has a character with accent in it. This basically means that you will have to alter the \Scripts\activate.bat file.
In order to see if that's the case, use the following procedure:
Activate the virtual environment by running the file \Scrips\activate.bat
Run the following command:
echo %PATH%
There should appear a bunch of paths, being path\to\your\virtual\environment\Scripts the first of them (this is obligatory). Check if this path is exactly the same as it is supposed to be (meaning it should have no "strange" characters instead of characters with accents).
If there are any discrepancies, I suggest you alter your \Scripts\activate.bat according to https://stackoverflow.com/a/22348546/11923771, reopen your Command Prompt and follow steps 1 and 2 again to see if your problem was solved.

Change conda environment during execution

I have two pieces of code that require different versions of python and versions of packages. I have two conda environments that allow each piece of code to work separately. It seems impossible to create an environment that will support both of them. Is there a way to switch conda environment during the run (in python code), so that I could execute one part using one environment and the second part using the second environment in the same script? The form and format of the result of the first part is definitely supported by the second part, so I don't see a reason why it can't work.
I have had success using conda within a shell script that calls the python process. e.g. something along these lines
conda activate my_env_1
python some_process.py
conda deactivate
conda activate my_env_2
python some_process_2.py
conda deactivate
You will have to enable your shell to use conda. See this Python - Activate conda env through shell script
If you want to change the environments in the same code at the same time, unfortunately as far as I know that will be painful, or just impossible with one word. In order to fix this issue, just add the libraries that your other environment has into the one that you use mainly.
You can achieve this by opening either your cmd (commnand line prompt), or just anaconda prompt:
activate yourenvironment name
pip/pip3 install modulename

Source Activate Conda

I am a new user to conda environment and was setting up to use TensorFlow , on Windows.
I came across a command -
source activate IntroToTensorFlow.
I understood IntroToTensorFlow is an environment we are creating, but does it mean we need to create this environment every time?. I am using jupyter notebook, so if I shutdown the kernel will the environment get deactivated?
And if I restart my PC, should I activate the environment everytime ?
Conda is a package manager that installs and manages (usually) Python libraries and (sometimes) non-Python packages. A conda environment is a sort of virtualenv virtual environment; its typical use case is to have a Python interpreter (any version) along with your choice of compatible Python libraries (any version).
The following example might most likely pertain to you. Suppose you have downloaded the implementation of a very nice paper implemented in TF and you want to try it out. But the authors implemented that when Tensorflow was just growing. The APIs have changed now, and so is the required CUDA version. You want to work ideally on the latest TF. Now, what do you do? An easy way to just try out this implementation is to create a different conda environment with the libraries needed for that implementation, run that in this environment, and perhaps if you like it, you might consider upgrading the TF APIs and use it in your code.
The conda environments are also pretty simple in its construction. If you installed conda using Anaconda and default options, you will have your environments in ~/anaconda3/envs. The environments are nothing but directories here, each having various configurations of Python interpreter and libraries of your choice. (So when you shutdown your PC/Jupyter, the environments will of course persist.) At the time of usage, you just switch between the environments to suit your needs. That is, when you source activate an environment, you will be allowed to use the Python interpreter and installed libraries from that environment. Note if you source deactivate or start a new terminal session, you will still be using the root environment.
Besides, Jupyter notebook, if setup with this plugin, will allow you to have nice integration with conda environments and you wouldn't even need to source activate everytime you want to switch. You can choose between the various settings (or conda environments), which are interpreted as different kernels in the notebook. So it would as simple as choosing some environment using a drop-down.
source activate IntroToTensorFlow does not create an environment, it simply activates an environment that has already been created. To create that environment (with tensorflow installed), use conda create -n IntroToTensorFlow tensorflow.
You do not need to create the environment every time, but you do need to activate it every time in order to use the packages installed in it. This is done using source activate IntroToTensorFlow
If you shutdown a kernel, the environment does not get deactivated automatically. To do so, you have to explicitly say source deactivate, or activate a separate environment using source activate xxx, replacing xxx with whatever environment name you want (that you have created previously).
When restarting your PC, (or starting a new session at the command line), you have to manually activate your desired environment to use it. Otherwise, by default, it will be running in your root environment. So, if you've only installed tensorflow in IntroToTensorFlow environment, you have to use source activate IntroToTensorFlow every time in order to use it.
Take a look here for more info

anaconda virtual env imports modules from global

Working in Windows PowerShell, I created a virtual environment within Anaconda using
>create conda -n test python=2.7.8
I activated it using
>activate test
activating environment "test"...
The test env contains no packages. Python is the only thing in there. I checked the test\Lib\site-packages folder and it is empty.
I started python
>python
>>>import pandas
>>>pandas.Series(range(1,5))
0 1
1 2
2 3
3 4
dtype: int64
I expected error messages because there is no pandas installed in this virtual env. Instead it seems to be importing pandas from the global Anaconda environment, where pandas is installed.
I thought virtual environments were supposed to be isolated. Have I misunderstood how they are supposed to work?
If they are supposed to be isolated, what might the problem be?
New to stack overflow, but I think I can help. First, the commands I needed to run were a bit different (I am on Linux). That is, in fact, how virtual environments are supposed to work. A few things to try: conda create --dry-run -n test python=2.7.8 to see if pandas is being installed to the venv (I don't know why it would). Another idea, when you activate test does the prompt actually change to the venv? Maybe see: virtualenv-in-powershell
Windows has a little idiosyncrasy that can lead to confusing situations like this, which is that it always implicitly puts . (the current directory) in the front of the search PATH. This means that if you are currently in the Anaconda directory when you start python, it will run the python.exe in that directory, regardless of what the rest of your PATH looks like (the activate command, in case you don't know, works by modifying the order of directories in the PATH variable).
The solution is to cd away from the Anaconda directory before starting Python.

Categories