I have created a Python application and now I need to deploy it to UAT and production environments which don't have access to pip library (the environment doesn't have access to internet).
I was told to put all the libraries in a folder python-lib. Should I just copy the all the content in anaconda3\lib\site-packages\ to the folder? However, it has several hundred mega bytes. Is it a way to copy the libraries just needed by my application?
What I would do is create a virtual environment locally and install the packages needed to it like so:
$ python3 -m venv prod_env
$ source prod_env/bin/activate
$ pip install <package_name>
If the folder is too large you can always compress and unzip on the UAT/Prod box.
Once the packages are installed in the virtual environment that are dependencies for your app, move the prod_env folder to the prod/UAT environment using scp or whatever method you normally would.
After the folder is moved to the environment you'll need to go into the bin folder and edit the activate script. You'll need to edit the VIRTUAL_ENV variable to be the path that the virtual environment is located on your prod/UAT box.
Once that's done you should be able to activate the virtual environment as normal:
$ source /path/to/prod_env/bin/activate
Using pip -t
mkdir libs
python -m pip -r requirements.txt -t libs
Then add the path to PYTHONPATH. Now sure if there is any default PYTHONPATH for a module/script.
So am working in a group project, we are using python and of the code is on GitHub. My question is how do I activate the virtual environment? Do I make one on my own using the "python virtual -m venv env" or the one that's on the repo, if there is such a thing. Thanks
virtual env is used to make your original env clean. you can pip install virtualenv and then create a virtual env like virtualenv /path/to/folder then use source /path/to/folder/bin/activate to activate the env. then you can do pip install -r requirements.txt to install dependencies into the env. then everything will be installed into /path/to/folder/lib
alteratively, you can use /path/to/folder/bin/pip install or /path/to/folder/bin/python without activating the env.
Yes, you'll want to create your own with something like: python -m venv venv. The final argument specifies where your environment will live; you could put it anywhere you like. I often have a venv folder in Python projects, and just .gitignore it.
After you have the environment, you can activate it. On Linux: source venv/bin/activate. Once activated, any packages you install will go into it; you can run pip install -r requirements.txt for instance.
I'm learning django and just installed pipenv via pip install pipenv and then pipenv shell and I notice that the virtual environment files are installed or created in some random default directory, I have two questions regarding this:
1) How can I customize that installation/creation directory for the virtual environment? Do I have to use a different command line from pipenv shell?
2) Can you have multiple folders with different virtual environments inside each folder/project?
According to the pipenv advanced readme (https://github.com/pypa/pipenv/blob/master/docs/advanced.rst#-custom-virtual-environment-location):
You can set the environment variable WORKON_HOME to whichever directory you want,
e.g.: by setting export WORKON_HOME=~/.venvs in your .bashrc file (if you are using bash).
According to this https://github.com/pypa/pipenv/issues/1071#issuecomment-370561179 comment (from the pipenv github repo), you can use a workaround for achieving this:
To be super clear, you can still get your own custom environments set
up just by sourcing virtualenvs.
virtualenv 35 --python=python3.5
virtualenv 36 --python=python3.6
source 35/bin/activate && pipenv install
source 36/bin/activate && pipenv install
source 35/bin/activate && pipenv run <whatever>
a tiny bit of additional visual clutter to the commands but is pretty
straightforward.
You would execute the virtualenv x commands inside the project folder.
I am learning Flask and am trying to install it using virtualenv but the pip installs the flask folder in Lib-->site-packages instead of the main venv folder.
This question has been asked before, but the answer did not work for me because the OP used GIT Bash install. I didn't (using Windows cmd):
C:\Users\Owner\> cd Desktop
C:\Users\Owner\Desktop> md python_projects
C:\Users\Owner\Desktop> cd python_projects
C:\Users\Owner\Desktop\python_projects> md project1
C:\Users\Owner\Desktop\python_projects> cd project1
C:\Users\Owner\Desktop\python_projects\project1> virtualenv venv
C:\Users\Owner\Desktop\python_projects\project1\venv> cd Scripts
C:\Users\Owner\Desktop\python_projects\project1\venv\Scripts> activate
(venv) C:\Users\Owner\Desktop\python_projects\project1\venv\Scripts> pip install Flask
This is my first time using virturalenv and installing Flask, so the mistake may be silly.
I tried starting over and creating a brand new directory, and same results. I also tried using virtualenv venv --no-site-packages.
Am I not supposed to put the file in the main venv folder instead of it's Lib -- site-packages?
pip installs the flask folder in Lib-->site-packages instead of the main venv folder.
Not clear enough. I'm betting there is isn't a file name anywhere on your computer that reports its path as:
Lib-->site-packages
I don't use windows, but the whole purpose of virtualenv is to install python and all the packages you install into your virtualenv directory. However, that does not mean the contents of your virtualenv directory won't have lots of subdirectories. pip will locate all the packages you install somewhere in the subdirectories in your virtualenv directory.
And if you've ever installed packages without using virtualenv, at some point you will discover that they get installed into a directory named:
.../lib/python3.x/site-packages
which is located deep within the directory structure of your python install. virtualenv mimics that directory structure.
So, are you saying that pip installed the packages in:
/Lib/site-packages
or in:
.../your_virtual_env/Lib/site-packages
or, somewhere else?
I've created folder and initialized a virtualenv instance in it.
$ mkdir myproject
$ cd myproject
$ virtualenv env
When I run (env)$ pip freeze, it shows the installed packages as it should.
Now I want to rename myproject/ to project/.
$ mv myproject/ project/
However, now when I run
$ . env/bin/activate
(env)$ pip freeze
it says pip is not installed. How do I rename the project folder without breaking the environment?
You need to adjust your install to use relative paths. virtualenv provides for this with the --relocatable option. From the docs:
Normally environments are tied to a
specific path. That means that you
cannot move an environment around or
copy it to another computer. You can
fix up an environment to make it
relocatable with the command:
$ virtualenv --relocatable ENV
NOTE: ENV is the name of the virtual environment and you must run this from outside the ENV directory.
This will make some of the files
created by setuptools or distribute
use relative paths, and will change
all the scripts to use
activate_this.py instead of using the
location of the Python interpreter to
select the environment.
Note: you must run this after you've
installed any packages into the
environment. If you make an
environment relocatable, then install
a new package, you must run virtualenv
--relocatable again.
I believe "knowing why" matters more than "knowing how". So, here is another approach to fix this.
When you run . env/bin/activate, it actually executes the following commands (using /tmp for example):
VIRTUAL_ENV="/tmp/myproject/env"
export VIRTUAL_ENV
However, you have just renamed myproject to project, so that command failed to execute.
That is why it says pip is not installed, because you haven't installed pip in the system global environment and your virtualenv pip is not sourced correctly.
If you want to fix this manually, this is the way:
With your favorite editor like Vim, modify /tmp/project/env/bin/activate usually in line 42:
VIRTUAL_ENV='/tmp/myproject/env' => VIRTUAL_ENV='/tmp/project/env'
Modify /tmp/project/env/bin/pip in line 1:
#!/tmp/myproject/env/bin/python => #!/tmp/project/env/bin/python
After that, activate your virtual environment env again, and you will see your pip has come back again.
NOTE: As #jb. points out, this solution only applies to easily (re)created virtualenvs. If an environment takes several hours to install this solution is not recommended
Virtualenvs are great because they are easy to make and switch around; they keep you from getting locked into a single configuration. If you know the project requirements, or can get them, Make a new virtualenv:
Create a requirements.txt file
(env)$ pip freeze > requirements.txt
If you can't create the requirements.txt file, check env/lib/pythonX.X/site-packages before removing the original env.
Delete the existing (env)
deactivate && rm -rf env
Create a new virtualenv, activate it, and install requirements
virtualenv env && . env/bin/activate && pip install -r requirements.txt
Alternatively, use virtualenvwrapper to make things a little easier as all virtualenvs are kept in a centralized location
$(old-venv) pip freeze > temp-reqs.txt
$(old-venv) deactivate
$ mkvirtualenv new-venv
$(new-venv) pip install -r temp-reqs.txt
$(new-venv) rmvirtualenv old-venv
I always install virtualenvwrapper to help out. From the shell prompt:
pip install virtualenvwrapper
There is a way documented in the virtualenvwrapper documents - cpvirtualenv
This is what you do. Make sure you are out of your environment and back to the shell prompt. Type in this with the names required:
cpvirtualenv oldenv newenv
And then, if necessary:
rmvirtualenv oldenv
To go to your newenv:
workon newenv
You can fix your issue by following these steps:
rename your directory
rerun this: $ virtualenv ..\path\renamed_directory
virtualenv will correct the directory associations while leaving your packages in place
$ scripts/activate
$ pip freeze to verify your packages are in place
An important caveat, if you have any static path dependencies in script files in your virtualenv directory, you will have to manually change those.
Yet another way to do it that worked for me many times without problems is virtualenv-clone:
pip install virtualenv-clone
virtualenv-clone old-dir/env new-dir/env
Run this inside your project folder:
cd bin
sed -i 's/old_dir_name/new_dir_name/g' *
Don't forget to deactivate and activate.
In Python 3.3+ with built-in venv
As of Python 3.3 the virtualenv package is now built-in to Python as the venv module. There are a few minor differences, one of which is the --relocatable option has been removed. As a result, it is normally best to recreate a virtual environment rather than attempt to move it. See this answer for more information on how to do that.
What is the purpose behind wanting to move rather than just recreate any virtual environment? A virtual environment is intended to manage the dependencies of a module/package with the venv so that it can have different and specific versions of a given package or module it is dependent on, and allow a location for those things to be installed locally.
As a result, a package should provide a way to recreate the venv from scratch. Typically this is done with a requirements.txt file and sometimes also a requirements-dev.txt file, and even a script to recreate the venv in the setup/install of the package itself.
One part that may give headaches is that you may need a particular version of Python as the executable, which is difficult to automate, if not already present. However, when recreating an existing virtual environment, one can simply run python from the existing venv when creating the new one. After that it is typically just a matter of using pip to reinstall all dependencies from the requirements.txt file:
From Git Bash on Windows:
python -m venv mynewvenv
source myvenv/Scripts/activate
pip install -r requirements.txt
It can get a bit more involved if you have several local dependencies from other locally developed packages, because you may need to update local absolute paths, etc. - though if you set them up as proper Python packages, you can install from a git repo, and thus avoid this issue by having a static URL as the source.
virtualenv --relocatable ENV is not a desirable solution. I assume most people want the ability to rename a virtualenv without any long-term side effects.
So I've created a simple tool to do just that. The project page for virtualenv-mv outlines it in a bit more detail, but essentially you can use virtualenv-mv just like you'd use a simple implementation of mv (without any options).
For example:
virtualenv-mv myproject project
Please note however that I just hacked this up. It could break under unusual circumstances (e.g. symlinked virtualenvs) so please be careful (back up what you can't afford to lose) and let me know if you encounter any problems.
Even easier solution which worked for me: just copy the site-packages folder of your old virtual environment into a new one.
Using Visual Studio Code (vscode), I just opened the ./env folder in my project root, and did a bulk find/replace to switch to my updated project name. This resolved the issue.
Confirm with which python
If you are using an conda env,
conda create --name new_name --clone old_name
conda remove --name old_name --all # or its alias: `conda env remove --name old_name`