I'm working on a python CLI project that, unfortunately, is a little complex on the virtual environments side, as it has to deal with multiple ones.
In my quest to get my tool to work properly and robustly, I came across Virtual Env's "activate_this.py" file, which is generated inside the .venv/bin/ directory.
I thought this could be useful for my needs so I started experimenting with it, but I haven't truly understood what it is actually doing under the hood.
For example, it says that the script should be used when a file must be called from an existing python interpreter (which is my case), but the virtual environment must be activated.
What I'm struggling to understand is: What exactly does activating a virtual environment on a given file mean? (Since we're not changing the interpreter)
Also, on the activate_this.py file there's this bit of code:
# add the virtual environments libraries to the host python import mechanism
prev_length = len(sys.path)
for lib in "../lib/python3.9/site-packages".split(os.pathsep):
path = os.path.realpath(os.path.join(bin_dir, lib))
site.addsitedir(path.decode("utf-8") if "" else path)
sys.path[:] = sys.path[prev_length:] + sys.path[0:prev_length]
Does this mean that you can now import libraries that are installed on the virtual env you've just activated, even if they are not installed in the environment of the base interpreter?
You've got it. It rewrites your sys.path to expose the stuff installed in the virtual environment, and it cuts off access to the globally installed third-party libraries not in the virtual environment.
Related
I am aware that there are many questions regarding Django and virtual environments, but I cannot wrap my head around the use of virtual environments with respect to deploying my Django app (locally) via uwsgi/nginx.
My setup includes a virtual environment (with Django and uwsgi), my Django app, nginx and PostgreSQL. The app was created before the virtual environment, and I applied only a single change to manage.py:
#!/Users/snafu/virtualdjango/bin/python3
When I start up the uwsgi located in the virtual environment (with the appropriate .ini file), everything works right away, but I wonder why. I did not need to fiddle around with the $PYTHONPATH, or append the site packages directory to the system path in manage.py, or activate the virtual environment at any point (apart from the initial installation of packages), although the boilerplate comment in manage.py explicitly mentions an inactive virtual environment as a possible reason for an import error.
Activating a virtual environment does nothing but prepend the virtual environment's bin/ to the $PATH thus making python and pip without explicit paths running from the virtual environment. Everything else related to virtual environments is implemented inside Python — it automatically changes sys.path and other paths (sys.prefix, sys.exec_prefix, etc).
This means that when you run python with an absolute path from a virtual environment Python automatically activates the virtual environment for this particular Python session. So you don't need to activate the virtual environment explicitly.
There is a minor warning sign on the road though: to run any Python script from a non-activated virtual environment you must set the shebang for all scripts to point to the virtual environment or use sys.executable. Do not use explicit python because that could be a different Python from the $PATH.
I wanted to replace Python 3.8 32-bit with the 64-bit version to install the face_recognition module, so I deleted the previous version and tried to re-route the project to the new Python version by going to File > Settings > Project Interpreter > Show all > Show Paths for Selected Interpreter, and adding all the Python files from the new folder and getting rid of the old ones.
However, it's still showing me this error when I try to install the module:
(Will2.0) C:\Users\solei\PycharmProjects\Will>pip install face_recognition
No Python at 'C:\Users\solei\AppData\Local\Programs\Python\Python38-32\python.exe'
I've also tried going to the Windows System Properties and changing everything that says "Python38-32" there, but it's still not working. It does work when I make a new environment, though, so at least I know that Python installed properly. It's just this one environment that is tripping me up (I'd prefer not to make a new project for this, btw. I've already installed a lot of modules in it.).
Your selected interpreter is not the system interpreter you've replaced with the 64-bit version, but your project's virtual environment interpreter. The virtual environment's files weren't changed in that process and need to be updated before you can use that environment again.
The system interpreter is your Python interpreter installed using the installation executable. In your case it is located in C:\Users\solei\AppData\Local\Programs\Python\Python38\. You can have multiple system interpreters installed, such as having Python 2.7, Python 3.7 and Python 3.8 side-by-side.
The virtual environment interpreter is a copy of another interpreter created using the venv package from the Python standard library. You can have many virtual environments interpreters in the system (one or more for every project, for example)
The base interpreter is the interpreter that was used as a template for the venv package. Every virtual environment interpreter has its base interpreter (usually a system interpreter) that it requires to run. Changing or upgrading the base interpreter requires updating the virtual environment.
If we take a quick look at the documentation, a virtual environment is described as
a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages.
That means you can setup an individual environment for every project, which will contain its own packages. The environment is a very efficient way of managing project packages, that's why PyCharm suggests a creation of such environment over the system interpreter by default. In short, it allows you to have two different versions of the same package used by two different projects, without the packages conflicting with each other.
This also explains why your virtual environment files weren't affected by your upgrade.
Now, I am unfortunately no Python expert. I had to spend some time examining how Python handles virtual environments on Windows and Ubuntu. It seems the environment always requires the base system interpreter present in the system. If you remove or change the location of the base interpreter, the environment will fail to function.
As I mentioned before editing this answer, you can in theory simply edit the pyenv.cfg file located in the root folder of the virtual environment. In practice, that will only work in simple cases and it is not the intended way of updating virtual environments.
You need to upgrade your virtual environment's files to work with your new system interpreter. That can mean the 64-bit version over the 32-bit version, or even a newer version of Python - such us upgrading from 3.7 to 3.8.
Close PyCharm
Check if the system interpreter you want to upgrade to is on the system Path
You can quickly check by running
python -c "import platform; print(platform.architecture())"
For you, the output should look like this
('64bit', 'WindowsPE')
If your output is different, you'll need to prefix the absolute path to the Python executable in step 4).
Navigate to the virtual environment's directory
The directory you're looking for contains the Include, Lib and Scripts directories and the pyenv.cfg file. From your screenshots, it seems this directory is your project's root directory, so in your case:
cd C:\Users\solei\PycharmProjects\Will2.0\
Upgrade the virtual environment
python -m venv --upgrade .
... or if Python is not on your path
C:\Users\solei\AppData\Local\Programs\Python\Python38\python.exe -m venv --upgrade .
The . in the commands refers to the current directory.
Open PyCharm and verify your environment is working correctly
... or simply try to run pip directly from the command line. Note you need to first activate the virtual environment by running the Scripts\activate.bat batch file.
If the above-mentioned method doesn't work, you might have to create a new virtual environment. You can create one easily without making a new PyCharm project. See this PyCharm documentation for reference. However, you'll still need to redownload all the required packages again.
For the simplicity, I recommend creating the new virtual environment in a .venv folder located in the project's root.
Disclaimer
I tested only the Python's behavior alone on a fresh Windows installation inside the Windows Sandbox. I was able to install the 32-bit Python, create a virtual environment, replace Python with the 64-bit version and upgrade the virtual environment to have it launch correctly again.
I want to find the Python virtual environments ("venvs" for short) in my file system, and for this I need to know what set of files constitutes a venv.
Some tools - when they want to make sure a directory is a venv - check for the pyvenv.cfg file, but older virtual environments and environments created by Pipenv do not have this file. - Furthermore pyvenv.cfg does not seem to be necessary for activating the venv either.
So what is the minimal set of files to define a working virtual environment? And what is an easy way to check for this in Python?
Clarification:
With "working virtual environment" I mean:
A way to use a specific version of Python executable defined in that environment (and possibly different to the default system Python)
A way to use specific packages (usually installed via pip, bit that's not a requirement).
The Python executable in the venv should be aware of being run inside a venv (see
sys.base_prefix and sys.prefix in the Python documentation.)
I am trying to set up a virtual environment on my machine, and I am really having trouble.
EDIT: I set up my virtual environment by installing virtualenvironment1.9 and running:
python virtualenv.py -p myVE
Then I add the line
alias goVE='source ~/virtualenv-1.9/myVE/bin/activate'
to my bashrc, and I use goVE to activate the virtual environment. I get a (myVE) at the beginning of the commandline in terminal, so I think it's working. It's just losing the gtk module somehow.
END EDIT
Basically, if I try to
import gtk
in python, it tells me there is no module named gtk. But outside my virtual environment, gtk imports with no problem. I decided it was a problem with the path, so I added
export VIRTUALENV_EXTRA_SEARCH_DIR=$PYTHONPATH
export VIRTUALENV_EXTRA_SEARCH_DIR=/usr/lib64/python2.6/site-packages/gtk-2.0
to my bashrc. Since evidently the gtk module was found in the pythonpath, I thought the first line would do it. When that was insufficient, I tried adding a path specifically to the directory where gtk was located, which I verified by trying
import gtk
gtk.__file__
Since this still didn't work, I tried modifying the virtualenvironment/bin/activate file, so:
PATH = ="$VIRTUAL_ENV/bin:$PATH:/usr/lib64/python2.6/site-packages/gtk-2.0/"
Again, this should be included in the original PATH (unless I'm mistaken), but I'm getting desperate so I tried adding it explicitly to the PATH. For completeness, my PYTHONPATH is set by:
export PYTHONPATH=${PYTHONPATH}:/data/monroe/ebexcode/trunk/GetData/:/home/user1/geach/:/usr/lib64/python2.6/site-packages/
You haven't explained how you created your virtual environment so there's no way to tell if that succeeded. If your virtual environment was created successfully, there's no need to manually munge the paths.
Assuming you have a virtual environment, you haven't demonstrated that you were able to activate it. In order to activate it, you have to run a script in the virtual environment's bin directory. Example, if your virtual environment is in ~/myvenv and you run bash, activate your virtual environment by issuing this at your prompt:
source ~/myvenv/bin/activate
At this point you can use this instance of your shell to install packages into your virtual environment:
pip install PyGTK
Depending on your system installation, this might fail if you do not have all the tools and libraries installed to build the Python GTK bindings.
I'm trying to set up WAMP server. I've got Apache working correctly, and I've installed mod_wsgi without a hitch.
Problem is, I'm using virtual environments (using virtualenv) for my projects. So obviously, mod_wsgi is having problems locating my installation of Django.
I'm trying to understand how I can get mod_wsgi to work well with the virtualenvs. The documentation seems to think this isn't possible:
Note that the WSGIPythonHome directive can only be used on UNIX systems and is not available on Windows systems. This is because on Windows systems the location of the Python DLL seems to be what dictates where Python will look for the Python library files. It is not known at this point how one could create a distinct baseline environment independent of the main Python installation on Windows.
From here: mod_wsgi + virtualenv docs.
Does anyone have some idea on how to make this work?
You can activate the environment programmatically from Python adding this to your .wsgi file before importing anything else.
From virtualenv's docs:
Sometimes you can't or don't want to
use the Python interpreter created by
the virtualenv. For instance, in a
mod_python or mod_wsgi environment,
there is only one interpreter.
Luckily, it's easy. You must use the
custom Python interpreter to install
libraries. But to use libraries, you
just have to be sure the path is
correct. A script is available to
correct the path. You can setup the
environment like:
activate_this = '/path/to/env/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))
This will change sys.path and even
change sys.prefix, but also allow you
to use an existing interpreter. Items
in your environment will show up first
on sys.path, before global items.
However, this cannot undo the
activation of other environments, or
modules that have been imported. You
shouldn't try to, for instance,
activate an environment before a web
request; you should activate one
environment as early as possible, and
not do it again in that process.