After learning how to code Python I'm starting to learn and figure out how to structure projects and set virtual environments up, but I can't make my mind up on how are packages managed when the virtual environment is activated. To make an example: I want to make a Django project. So, first of all, mkdir the project folder, cd into it and then execute python -m venv [whatever]. cd into Scripts folder and execute 'activate'. Then, pip install Django and pip list and shows Django. At last, I deactivate the virtual environment and make pip list again. Why is Django listed there? Should it?
You might have installed Django both inside and outside the venv (outside being the system python installation). Deactivate the venv and run pip uninstall django then try again.
Okay, finally I understood what was happening and I didn't realize. On one hand, virtual environments are completely independent from the global or system, so what is being installed in one side it shouldn't affect the other. On the other hand, what happened was, that for being inside the Scripts folder, when trying to execute Python commands I was actually executing the scripts with the same name, which are copies to use with the virtual environment and which can be used calling the "activate" script. There was actually no problem, it was me messed up.
Thanks to all contributors for their help.
Related
To be honest, up until recently I was hoping to avoid everything related to vurtualenvs. Yep, very naïve.
All of the code I write using python is to be deployed using containers, where "one container - one app" is totally true for now. And also number of CLIs being distributed also to my coworkers as wheels.
What I came up with is two "global" environments living in my homedir:
"default" virtual env with python 3.10 and some general packages installed. Using this venv I develop current versions of my applications and do my regular job. It is sourced automatically in .bashrc.
Virtual env with python 3.9. This is used to support legacy versions of/not yet updated applications in case if something should be done to them.
And also for some of the projects there are dedicated virtual envs living in a project folder needed only for this particular project, python version there may vary from 3.7 to 3.10.
As a consequence, I have to change current environment by manually sourcing the right activate script several times a day, and this isn't funny.
After some research on the topic, I found things like virtualenvwrapper, Pipenv and Poetry, but none of them seem to meet all my needs: "global" env to use across multiple projects and outside projects (because of CLIs); ability to create env with given python version; easy switch between virtual envs.
Is there any solution for this?
UPD: Some illustrations
Next virtual environments were created inside ~/pyenv/:
$ virtualenv -p /usr/local/bin/python3.9 default3.9
$ virtualenv -p /usr/local/bin/python3.10 default3.10
Next, .bashrc was updated to do source ~/pyenv/default3.10/bin/activate.
Now, for the vast majority of tasks it is enough: everything needed for day-to-day job is installed in default3.10 venv.
In case I need to switch to python3.9 the following is required:
$ deactivate
$ source ~/pyenv/default3.9/bin/activate
And if then I need to work on some project with its own venv, inside project folder I have to do:
$ deactivate
$ source env/bin/activate
Too many manual inputs. At minimal, I'd like to do something like $ workon default3.9 no matter of the current directory I'm in, and it should activate default3.9 venv correctly if no venv is active right now or if any venv if already active.
The same should work inside project folder: $ workon <local_venv_name> to switch to the venv living in current project folder.
And something like venv create <enterpreter path/py version> to create new "global" or "local" (in current folder only) venv, and venv list to get the list of available "global" and "local" environments.
to use the global environment inside the project's local virtual environment.
set
include-system-site-packages = true
in pyvenv.cfg file located in .venv (virtual environment installation directory) directory of each project.
My question is do i have to install django every single time in my virtual environment in order to run my python files? and is this taking up bunch of space on my machine? My project also uses "matplotlib" and every virtual environment i create it also asks me to import the matplotlib module too. its getting annoying. do i have to do this every time?
Im new to Django. I wanted to run some python files in django but they weren't working, so after some research i found out i needed to run my pycharm project in a virtual environment in order to run these python files.
my folders look like this pycharmProjects -> my project
I enter pycharmProjects and I set up virtual environment using "pienv shell". Then i run "python3 manage.py runserver". It turns out i must install django in the virtual environment before the files run.
Short answer is no, you don't have to use a virtual environment at all and can install your dependancies globally instead. However you will soon find that it will cause a lot of issues. The main reason you would create a virtual environment is to give control of your dependancies and prevent bugs that could be caused because of them having their wires crossed between projects.
Short answer yes.
If you create a virualenv you have to install all packages, that your program needs.
Long answer:
You could install django system wide and then create a virtualenv with the option
--system-site-packages then django would be used from your globally installed python.
(Or you install everything just in your global python, put I personally don't think this is good practice)
If you work with many different projects I think you will avoid a lot of trouble if you use one virtualenv per project.
Trouble meaning that one project breaks, because one pip install for another project changed the version of one package and one project can't handle the newer version.
I would recommend to create a requirements.txt file for each project, that lists the dependencies then you can create the virtualenv with following command
pip install -r requirements.txt
if you have requirement.txt files, then you can create virtualenvs rather quickly if going back to an old project and you can delete the virtualenvs whenever you run out of disk space. If you want to be an the safe side, type pip freeze > pipfreeze.txt prior to deleting the virtualenv and use pip install -r pipfreeze.txt if you want to create one with the same modules and the same versions.
You also might want to look at direnv or autoenv if working on a linux like system.
This will automatically switch to the required virtualenv when changing to a project's working directory.
complete beginner here. Trying to build a flask web app. Using Windows 8.
Having some problems activating my python virtualenv in Cygwin. I have been using git shell up till now with no problems.
I copied my folder ("app") into my cygwin home directory and it is set up like so:
app - templates
- static
- flask - env - scripts - python
- ...
- hello.py
- ...
I change directory into the app folder, then when I type the command to activate my virtualenv:
$ source flask/env/scripts/activate
The terminal shows:
(env)
so I assume that it is working, until I double check which python:
$ which python
and that returns my original global python install, not the virtual environment. I've checked the installed packages to double check which python environment I am using.
I use the same command in git shell and it activates the right virtualenv. Where am I going wrong / what do I need to change? Please let me know if you need any more information.
I created a new virtual environment using cygwin and when I activated the new env, it switched to that environment fine. Why won't it work for the folder which I copied in?
I created a new virtual environment using cygwin and when I activated the new env, it switched to that environment fine. Why won't it work for the folder which I copied in?
This last sentence is the real problem. The way you try to activate is correct. The problem is that the virtualenv directory must not be moved.
The activate script inside the virtualenv uses absolute paths internally. If you move the directory, the paths will no longer work, and so which python finds the first valid binary on PATH, which is your global binary.
If you need to move the project to a different location, and the virtualenv together with it, then recreate the virtualenv, do not copy it.
The recommended practice is to have a requirements.txt file, and install packages using pip install -r requirements.txt.
That way, recreating a virtualenv is very easy: create an empty virtualenv, and run the pip ... command. There should be nothing else inside the virtualenv that needs moving, only what pip put there, or other python installer scripts, if you used any (and which you would need to re-run, in addition to pip).
So yesterday I had to create a virtualenv in order to be able to install Python modules that wouldn't install thanks to OS X El Capitan's new SIP. I thought I did everything right, but today I'm reaching a different conclusion. I hope I can be clear about it.
my python custom install is at myname/learnp/imdb_module, this is where I created it with virtualenv. Edit: I later moved it to myname/learnp/ayr2/imdb_module.
However, when I try to run the interpreter, it seems to always default to the Python that is in Library or something along these lines. I found out about this because a certain module that I managed to install in this custom python env wouldn't import, when I checked what modules I have, it wasn't the same as what I expected.
Furthermore, it seems that ALL other modules that I wanted to install on the CUSTOM virtualenv were installed on the main python env, and that I wasn't installing those modules on the custom env all along.
Excuse me, but I'm very confused right now.
I know how to create a virtual env
I know how to activate it (it appears to the right on Terminal line)
I don't know how to install modules to my virtual env
I don't know how to make the interpreter run from the virtual env so I can do python operations that are only possible by using custom env modules
Any advice is much appreciated!
Update:
Followed Will Hogan's answer for troubleshooting,and I think something weird is happening, quoting my comment to his answer:
HI, thanks for taking the time to answer. This is basically the way I understood this. However, let me attach a screenshot: http://i.imgur.com/DfpngJq.jpg . Am I right to assume something is wrong here? My prompt is changed with the virtualenv named "imdb_module", but when I type in which python it doesn't list ayr2/imdb_module/bin but rather a folder with the path usr/bin/python, which if I understand correctly is the "default" environment.
And not if this helps in any way, but echo $PATH when (imdv_module) appears to the right of the prompt, gives this (I redacted my name): /Users/REDACTEDNAME/learnp/imdb_module/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
While creating the virtualenv you should see it installing setuptools and pip:
$ virtualenv testvenv
New python executable in testvenv/bin/python2.7
Also creating executable in testvenv/bin/python
Installing setuptools, pip...done.
After ensuring the virtualenv is activated you should see your prompt change:
$ . ./testvenv/bin/activate
(testvenv)$
Now you can confirm the paths to python and pip, which should be in the virtualenv:
(testvenv)$ which python
/private/tmp/testvenv/bin/python
(testvenv)$ which pip
/private/tmp/testvenv/bin/pip
If you aren't seeing the python and pip locations as being under the virtualenv's directory, then the virtualenv has not been activated.
I would also ensure that, if you're executing the .py file directly (and not with "python foo.py"), that your shebang line uses:
#!/usr/bin/env python
Or even the full path to the virtualenv's python, e.g.:
#!/tmp/testvenv/bin/python
As opposed to, say:
#!/usr/bin/python
The first will search in the current environment, which will be set by the virtualenv activation. The second explicitly points to the virtualenv's `python'.
I'm aware of how to use virtualenv for isolating Python dependencies in a long-running script, like a Flask or Twisted app. But I've been sort of puzzled about how you're supposed to go about this for a script intended to be invoked from the command line.
Suppose I wanted to make a CLI tool for interacting with some API, perhaps using Click or docopt. Obviously you don't want to have to source venv/bin/activate every time you want to use this tool. But I'd assume that it's still best to still use virtualenv to keep a clean environment even beyond development.
Sorry for the newbie question, but...what are you supposed to do to package up a script so it can be cleanly used in this manner? (I'm more used to RubyGems, and am still figuring out Pip and VirtualEnv.)
In general, if a package you've installed in a virtual env that provides binary command line script, say in ~/.virtualenv/bin/ you can symlink it into ~/bin/ (or wherever on your path you'd like to put local scripts).
There are a couple projects that aim to solve this issue:
pipsi the pip script installer -- amounts to doing the virtual env creation and symlinking for you
pipx pip for executable binaries
An excellent article on virtualenv by Dabapps would make it clear for you:
http://www.dabapps.com/blog/introduction-to-pip-and-virtualenv-python/
As for invoking it from a CLI script:
1. cd to your project root
2. env/bin/python your_main_file.py (Assuming your virtualenv is named env)
This way you don't need to source the virtualenv everytime.
Each virtualenv has it's own Python site_packages, builtin modles, and Python interpreter. So virtualenv is meant to be used at the project level, not at the "package by package" level. It isolates a collection of Python modules and possible dependencies. Each virtualenv has it's own location where pip will install packages. In theory, virtualenv should not be necessary, but in practice, it's nice to have a way to have different "environments" with different versions of Python modules and Python interpreters. I don't know if Ruby has something similar, that would allow you to have different "sets" of gems for different projects.
People that use straight virtualenv will add aliases to their .bashrc, for example:
alias workonawesomeproject="source ~/venv/awesomeproject/bin/activate"
They would activate the virtualenv with the alias
workonawesomeproject
To leave a virtualenv you use the command deactivate
An easier way to deal with virtualenvs is to use virtualenvwrapper
pip install virtualenvwrapper
Add these lines to your .bashrc (or other shell initialization file)
export WORKON_HOME=$HOME/venv # this directory is your choice
export PROJECT_HOME=$HOME/src # this directory is your choice
source /usr/local/bin/virtualenvwrapper.sh # leave this alone
If you just modified your .bashrc make sure to source it
source ~/.bashrc
Then to make a new virtualenv you simply run
mkvirtualenv awesomeproject
To use that virtualenv
workon awesomeproject
To deactivate that virtualenv
deactivate
Virtualenvwrapper Docs:
http://virtualenvwrapper.readthedocs.org/en/latest/install.html