I need to make a fully self-contained Python environment for Mac, which can be transferred between different Macs running OSX and function independently of any other Python installations (or lack thereof).
In the enviroments I have created (using venv), I find that the /bin/python file and others are in fact aliases. When I replace these aliases with the original executables, the enviroment still does not work on other computers due references to files within a Python installation, such as
Library not loaded: /Library/Frameworks/Python.framework/Versions/3.9/Python
Is there a way to make the environment fully self-contained?
I would guess so, since tools such as pyinstaller and cx_freeze exist to make executables out of Python applications. So these must contain standalone environments somehow. But for my purposes, it is the enviroment that I need. Not an executable as these tools provide.
You can try using a virtual environment. Python comes with venv which allows you to create virtual environments. You could consider them self-contained, since all the necessary scripts for running Python are in the virtual environment. This goes with the package dependencies as well. You can have a variety of packages installed on your computer, but the virtual environment will have none of them installed (unless, of course, you install them yourself).
To create a virtual environment, run:
$ py -3.9 -m venv virtualenv
You can replace 3.9 with whichever Python version you are using and you can replace virtualenv with whatever you want to name your virtual environment.
To activate the virtual environment, you would run:
$ source virtualenv/Scripts/activate
And to deactivate it, you would run:
$ deactivate
Related
I know it possible to have two installs of Python of different versions on a Windows system. But I cannot manage to have two installs of the same revision (in my case 3.8.10) to coexist.
I'm designing an application that creates a Python process. That process needs to run from a specific version of Python with packages of specific versions installed on it. In order to fully control the Python install, decision was made to install it inside the application distribution directory, segregating it from any other Python installed on the system. No environment variable refers to it.
As part of the the deployment/install process for the application, a PowerShell script downloads the Python installer and installs Python and the necessary packages into the application distribution directory. The Python installer is invoked as follows:
.\\python-3.8.10-amd64.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0 TargetDir="$curDir\\Python" Include_exe=1 Include_lib=1 Include_pip=1 Include_tcltk=1 | Out-Null
It works well unless the system has already a Python install of the same version installed on it. In that case, running the installer will break the existing install, and not fully install the new one.
I tried to run the installer manually and I noticed that it is able, somehow, to detect that an install of the same revision exist on the system. In that case, it does not allow an new install. To do so, I would have to uninstall Python at its current location to be able to install it somewhere else.
Is there a way to have two distinct installs of Python 3 of the same revision on a Windows system? And if yes, how can it be done?
A better aproach instead of installing python again would be using virtual environments.
To create a new python env. Open the command line (Powershell) on Windows and navigate to the directory you want your python env to be.
Type python3 -m venv tutorial-env. This will create a new python virtual env named tutorial-env
To activate that env on Windows powershell type: tutorial-env\Scripts\activate.bat
To deactivate the env type deactivate
If you are wondering what python virtual environments do. They basically do what you are trying to do but without installing python globally again. When you create a new python env, a new python3 is placed in your env directory, in this case in the tutorial-env directory, and when you activate the environment, it replaces the python global path to the path in your env (in this case in tutorial-env). Now when you are on this virtual env and install new python packages, they will only be available when you activate that env.
For more information about virtual environments please refer to Python official docs.
I use vevn with python 3.6.8, pip 18.1.
I set powershell executionPolicy RemoteSigned,
create virtual environment with venv, refed to venv official site : (https://docs.python.org/3.6/library/venv.html)
and also activated according to powershell, used .<venv>\Scripts\activate.ps1
But when I execute 'pip list' command, it shows all packages installed in global, not in venv.
And when execute python - import sys - sys.prefix, to check python directory,
I can find, powershell venv uses environmental variables python.
Of course, run it on cmd in same way, it works correctly.
pip list shows
The pip list shows only packages installed in venv, and sys.prefix is python in venv directory.
Does anyone know why it works like this?
Powershell works correctly like cmd
Maybe you activated the "--system-site-packages" switch by accident when installing the virtual env. This switch makes all python packages in the global system available to the virtual env.
Also check that after activating the environment your powershell indicates that you are actually using the virtual env. This is indicated by (env_name) at the beginning of the new power shell line.
I wanted to replace Python 3.8 32-bit with the 64-bit version to install the face_recognition module, so I deleted the previous version and tried to re-route the project to the new Python version by going to File > Settings > Project Interpreter > Show all > Show Paths for Selected Interpreter, and adding all the Python files from the new folder and getting rid of the old ones.
However, it's still showing me this error when I try to install the module:
(Will2.0) C:\Users\solei\PycharmProjects\Will>pip install face_recognition
No Python at 'C:\Users\solei\AppData\Local\Programs\Python\Python38-32\python.exe'
I've also tried going to the Windows System Properties and changing everything that says "Python38-32" there, but it's still not working. It does work when I make a new environment, though, so at least I know that Python installed properly. It's just this one environment that is tripping me up (I'd prefer not to make a new project for this, btw. I've already installed a lot of modules in it.).
Your selected interpreter is not the system interpreter you've replaced with the 64-bit version, but your project's virtual environment interpreter. The virtual environment's files weren't changed in that process and need to be updated before you can use that environment again.
The system interpreter is your Python interpreter installed using the installation executable. In your case it is located in C:\Users\solei\AppData\Local\Programs\Python\Python38\. You can have multiple system interpreters installed, such as having Python 2.7, Python 3.7 and Python 3.8 side-by-side.
The virtual environment interpreter is a copy of another interpreter created using the venv package from the Python standard library. You can have many virtual environments interpreters in the system (one or more for every project, for example)
The base interpreter is the interpreter that was used as a template for the venv package. Every virtual environment interpreter has its base interpreter (usually a system interpreter) that it requires to run. Changing or upgrading the base interpreter requires updating the virtual environment.
If we take a quick look at the documentation, a virtual environment is described as
a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages.
That means you can setup an individual environment for every project, which will contain its own packages. The environment is a very efficient way of managing project packages, that's why PyCharm suggests a creation of such environment over the system interpreter by default. In short, it allows you to have two different versions of the same package used by two different projects, without the packages conflicting with each other.
This also explains why your virtual environment files weren't affected by your upgrade.
Now, I am unfortunately no Python expert. I had to spend some time examining how Python handles virtual environments on Windows and Ubuntu. It seems the environment always requires the base system interpreter present in the system. If you remove or change the location of the base interpreter, the environment will fail to function.
As I mentioned before editing this answer, you can in theory simply edit the pyenv.cfg file located in the root folder of the virtual environment. In practice, that will only work in simple cases and it is not the intended way of updating virtual environments.
You need to upgrade your virtual environment's files to work with your new system interpreter. That can mean the 64-bit version over the 32-bit version, or even a newer version of Python - such us upgrading from 3.7 to 3.8.
Close PyCharm
Check if the system interpreter you want to upgrade to is on the system Path
You can quickly check by running
python -c "import platform; print(platform.architecture())"
For you, the output should look like this
('64bit', 'WindowsPE')
If your output is different, you'll need to prefix the absolute path to the Python executable in step 4).
Navigate to the virtual environment's directory
The directory you're looking for contains the Include, Lib and Scripts directories and the pyenv.cfg file. From your screenshots, it seems this directory is your project's root directory, so in your case:
cd C:\Users\solei\PycharmProjects\Will2.0\
Upgrade the virtual environment
python -m venv --upgrade .
... or if Python is not on your path
C:\Users\solei\AppData\Local\Programs\Python\Python38\python.exe -m venv --upgrade .
The . in the commands refers to the current directory.
Open PyCharm and verify your environment is working correctly
... or simply try to run pip directly from the command line. Note you need to first activate the virtual environment by running the Scripts\activate.bat batch file.
If the above-mentioned method doesn't work, you might have to create a new virtual environment. You can create one easily without making a new PyCharm project. See this PyCharm documentation for reference. However, you'll still need to redownload all the required packages again.
For the simplicity, I recommend creating the new virtual environment in a .venv folder located in the project's root.
Disclaimer
I tested only the Python's behavior alone on a fresh Windows installation inside the Windows Sandbox. I was able to install the 32-bit Python, create a virtual environment, replace Python with the 64-bit version and upgrade the virtual environment to have it launch correctly again.
Using Python 3.7.0 on Mac. Trying to use venv module that was added post python 3.4.
I setup my virtual env using python3 venv -m path/to/my/dir - my question is do I need to activate this virtual env to use?
The documentation seem to imply I don't need to?
You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically.
If I don't have to activate, what is the benefit of prepending venv to binary directory? Wouldn't this have to happen regardless for a venv to work?
Activating the virtualenv gives you convenience. It is never required.
Even for scripts that are configured to run with #!/usr/bin/env python, (which looks up the python executable on your path), you could manually update the PATH environment variable:
$ PATH="/path/to/venv/bin" some_script
Activating makes the PATH update stick until you deactivate again, and that can be convenient.
For example, I regularly have several virtualenvs in use at any one time. Some of them are there only to install some command-line tools I have symlinked into my ~/bin/ directory, another is home to a Jupyter notebook, and 2 more are used to quickly verify code snippets in different Python versions with access to 3rd-party libraries. I don't activate any of those.
When you don’t activate a virtualenv, all that happens is that your terminal PATH variable is not updated to put the bin directory of the virtualenv first, so when you enter python or pip or other script without any path into the terminal, the shell will find a different Python binary to run. You can always use any of the commands in the virtualenv bin/ directory by giving the full path to that command.
I have an existing django project which I have developed using python libraries installed in system and adding missing ones to the system. But now conflict has come for python-requests as system has 2.2 version but I need >2.5. Dont't want to uninstall and put newer one as it may break the OS. So now, I want to use virtual env and install packages there in complete isolation to that of OS.
I think the solution you're looking for is to download a different version of python without uninstalling your original, then start up virtualenv venv, but by passing in a path to the new python.exe file. Like this: virtualenv -p venv <path-to-executable-here>, then just do source bin\activate, as usual. This starts the virtual environment using the python executable you passed to virtualenv through the terminal.
Also, this might not be the only way, but, there's something called ModuleFinder which enables you to get a list of all the modules your script is importing--That is if you don't want to type them out manually, and you have extra modules installed (otherwise pip freeze > requirements.txt would do the job, and your new virtual environment would install all the packages in requirements.txt).