In order to not alter the default python installation, I have a virtualenv set up that automatically runs on startup. I've also set up my profile so that unless a virtualenv is running, pip will not run. I've been using python2.x up to now, and have now installed python3 via homebrew. When I attempt to install a new package via pip3, I get the following error: Could not find an activated virtualenv (required).. However, when I check for the existence of a virtual env by checking the $VIRTUAL_ENV variable, it tells me that a virtual env is in fact running: -bash: /Users/me/virtualenvs/r: is a directory. Do I need to create separate instances of virtualenvs for each version of python? Why will one of my versions of python recognize the virtualenv, while the other will not?
All your virtualenvs are stored in a directory -- /Users/me/virtualenvs in your case. If you go there, you'll notice that it executes totally separate python interpreters in each virtualenv. For example, this is a freshly created virtualenv:
I'd imagine that running pip3 while on the python2.7 virtualenv wouldn't work for that reason.
It seems that there are two different kinds of virtualenvs, one of which is compatible with python2 and one of which is compatible with python3. I'm not sure whether there is a way to get one that is compatible with both, but I went ahead and created a new virtualenv using python 3 (python3 -m venv myPython3Env), and have been able to successfully use that.
Related
I know it possible to have two installs of Python of different versions on a Windows system. But I cannot manage to have two installs of the same revision (in my case 3.8.10) to coexist.
I'm designing an application that creates a Python process. That process needs to run from a specific version of Python with packages of specific versions installed on it. In order to fully control the Python install, decision was made to install it inside the application distribution directory, segregating it from any other Python installed on the system. No environment variable refers to it.
As part of the the deployment/install process for the application, a PowerShell script downloads the Python installer and installs Python and the necessary packages into the application distribution directory. The Python installer is invoked as follows:
.\\python-3.8.10-amd64.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0 TargetDir="$curDir\\Python" Include_exe=1 Include_lib=1 Include_pip=1 Include_tcltk=1 | Out-Null
It works well unless the system has already a Python install of the same version installed on it. In that case, running the installer will break the existing install, and not fully install the new one.
I tried to run the installer manually and I noticed that it is able, somehow, to detect that an install of the same revision exist on the system. In that case, it does not allow an new install. To do so, I would have to uninstall Python at its current location to be able to install it somewhere else.
Is there a way to have two distinct installs of Python 3 of the same revision on a Windows system? And if yes, how can it be done?
A better aproach instead of installing python again would be using virtual environments.
To create a new python env. Open the command line (Powershell) on Windows and navigate to the directory you want your python env to be.
Type python3 -m venv tutorial-env. This will create a new python virtual env named tutorial-env
To activate that env on Windows powershell type: tutorial-env\Scripts\activate.bat
To deactivate the env type deactivate
If you are wondering what python virtual environments do. They basically do what you are trying to do but without installing python globally again. When you create a new python env, a new python3 is placed in your env directory, in this case in the tutorial-env directory, and when you activate the environment, it replaces the python global path to the path in your env (in this case in tutorial-env). Now when you are on this virtual env and install new python packages, they will only be available when you activate that env.
For more information about virtual environments please refer to Python official docs.
I have a python library that I am wanting to help out with and fix some issues. I just don't know how to test my changes given the complexity of how python/pip installs libraries.
I have the library installed with pip and I can run python code connecting to the library by doing an "from import *". But now that I want to make changes to it I pulled the code with git and plan to branch to work on my changes. That's fine. I will then do a pull request to merge any changes given tests pass.
But after I make a change, how do I integrate my changes into python to test out the changes I made with the library? Can pip install my custom/modified version of the library?
I have looked around and haven't successfully found an answer to this but perhaps I'm not looking in the right spot.
Can pip install my custom/modified version of the library?
Yes.
There are various ways of approaching this question. A common solution is the use of Python virtual environments. This allows you to create an isolated Python environment that does not share the same packages as your system Python install. You can then install things into it (such as your modified Python library) to test it out.
To get started, you need the virtualenv tool. This is probably available as a package for your distribution, but you can also install it using pip. Once you have it, you can run in the same directory as your code:
virtualenv .venv
This creates a virtuelenv named .venv. You can call it anything you want, but naming it .venv (or anything starting with a .) means it won't clutter up the output of ls in your workspace.
Next, you need to activate the virtualenv:
. .venv/bin/activate.sh
This modifies your $PATH to place the virtualenv at the front of the list of directories. Now when you type python or pip, you'll be using the virtualenv version.
If your code has a setup.py file, you can install it like this:
pip install -e .
The -e means you want to perform an "editable" install, which means python will use the code "in place", and any changes you make will be immediately visible to the code you use for testing.
When you're done, you can run:
deactivate
This will remove the changes that activate made to your environment.
For more information:
Pipenv & Virtual Environments discusses a higher level tool for managing virtual environments.
Virtualenvwrapper is another take on a higher level management tool.
I am working on several projects on the same PyCharm. Like I "attached" them all together. But I recently noticed some weird behaviors. Like when I import a library I haven't installed yet to my script. It shows me a little error as expected. But when I try to install that using python -m pip install my_library, it tells me that it has already installed. I recently noticed that this is because it's using and other pip from another project. I doesn't use the one in the venv folder in the project. Also to run the scripts sometimes it uses python.exe from pythons original directory. It's a whole mess and I have no idea how I can solve it. Sometimes my projects requires different versions of the same library and you can imagine what happens when I change the version.
I make sure each project is using their own interpreter. Don't know what else to do other than this. I am using Python3.6.4 PyCharm2018.3.2 running on Windows10
it sounds like all your projects are configured to use the system's interpreter instead of the virtual environment you set up for each of them.
Follow this instruction to fix it https://www.jetbrains.com/help/pycharm-edu/creating-virtual-environment.html
In terms of using different version of the python library, you can address that by specifying it in requirements.txt file, which you can put in your venv folder for each project. then you can just do pip install -r requirements.txt after you set up your venv. (you need to ensure that the venv is activated - you don't need to worry about this if you have configured the project in PyCharm to use the venv's python interpreter.) You can check this by going to Terminal in your PyCharm and you should see (venv_name) hostusername#host:~/project_folder$
I have an old computer with dozens of old python projects installed, each with its own different virtualenv, and many of which built with different versions of python.
I'd prefer not to have to download these different versions when I create new virtualenvs via virtualenv -p whatever path that version of python has
My question is: within a virtualenv, is there a command I can run to find the path to the version of python which was used to create that particular environment?
For example, if I created a venv with 'virtualenv -p /usr/bin/python3.4' and then ran this command with the venv activated, it would return '/usr/bin/python3.4'
Since virtualenv copies python completely (including the binary) there is no way to know the exact path it originated from.
However, you can easily find the version by running ./python --version inside the environment's bin folder.
You could try running something like this from the command line:
python -c "import sysconfig; print(sysconfig.get_config_var('BINDIR'))"
In my Ubuntu16.04, there are python 2 and python 3 default. In addition, i have installed anaconda too. I am sucked by the 'python' cmd. Every time i use pip or pip3 install, I don't know where the package install, python2 or python 3? And I use conda install to install anaconda package. I also use anaconda env to manage different virtual env. But I think it mix with my local Python 2 and 3.
For example, in directory /usr/bin, I found many soft links like this:
When i try 'python' cmd, it just confuse me！
Why python3m are local, shouldn't it be anaconda? Why python3 are anaconda, shouldn't it be local? Then I found that if I use ./python2 or ./python3, I found it is correct now!
So I know it is caused by environment variables. I echo $PATH, Found it like this: /home/kinny/.pyenv/shims:/home/kinny/.pyenv/bin:/home/kinny/anaconda3/bin:/home/kinny/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/ant/bin:/snap/bin:/opt/maven/bin:/usr/lib/jvm/java-8-oracle/bin
I have used update-alternative --config python to configure default python, but it doesn't work! It sames mixed with each others.
Now I just want to install tensorflow 0.11 in local python3, because in anaconda it is 0.10 version by default. So how can I change this. I just want to use python python3 and python3m represents python2.7 python3.5 and anaconda python respectively, How can I do that! use pip and pip3 for local python2 and python3 respectively!
I ran into a similiar problem when setting up PyCharm Edu to work with Anaconda. I found that I had several versions of Python installed and it was very hard to keep track of which version the IDE was referencing. My CS professor gave me the advice of simply removing the versions of Python I didn't frequent. I now just have Anaconda installed; and use the Anaconda Prompt as my Python console. I also rely on PyCharm's IPython for the developer console. However, if you still want differing versions of Python installed (say your doing QA testing for older devices); there is the really helpful command: which python. When entered into the python console or Anaconda Prompt: which python will display the directory associated with the currently executing Python Shell. This enables you to better keep track of to what particular python.exe the current window is referring to.
Follow up to the comments mentioning using virtualenv and virtualenvwrapper.
Here are the official docs and a good blog post to follow for getting started using virtualenv's is here:
https://virtualenv.pypa.io/en/stable/installation/
http://virtualenvwrapper.readthedocs.io/en/latest/install.html
http://exponential.io/blog/2015/02/10/install-virtualenv-and-virtualenvwrapper-on-ubuntu/
Also, once you are setup you can create virtualenv's specifying which python installation you want to use.
which python3
returns
/usr/bin/python3
Then create a virtualenv with that python path. Where example_env is the name of the virtualenv.
mkvirtualenv -p /usr/bin/python3 example_env
Then activate the virtualenv using virtualenvwrapper.
workon example_env
Finally, install tensorflow and other dependencies with pip.
pip install tensorflow
the which command is very useful for finding the path to the executable that is first in your path. Zsh also has the where command, which will show you all instances of the given executable that show up in your path. For managing different python versions, you have a lot of options. The easiest for most people tends to be anaconda, using conda environments. The installer will ask you to add some stuff to your .bashrc file, which will then make anaconda's binaries come first in your path. Anything else you run after the .bashrc gets sourced after that, will then use that first, including PyCharm. For graphical desktop apps to pick up the change, you may need to log out and back in again. If you only need one version each of python 2 and python 3, you can just use the ones available via apt. Depending on your Ubuntu version, Python 2 is definitely installed by default as it is used by many system utilities, including apt itself. Some newer versions may also install python 3 by default, but I do not remember for sure. Another option is to install the versions of python you need in an alternate location, such as /opt/python/<version> and then using environment-modules (installed via apt install environment-modules) or Lmod to control which versions are being used, but that may or may not be easy/convenient to use with a desktop application such as PyCharm.
for TensorFlow, 1.11 is available in anaconda, but I don't remember if it's in the default channel or not.