I would like to create a conda environment on a machine that has no network connection. What I've done so far is:
On a machine that is connected to the internet:
conda create -n python3 python=3.4 anaconda
Conda archived all of the relevant packages into \Anaconda\pkgs. I put these into a separate folder and moved it to the machine with no network connection. The folder has the path PATHTO\Anaconda_py3\win-64
I tried
conda create -n python=3.4 anaconda --offline --channel PATHTO\Anaconda_py3
This gives the error message
Fetching package metadata:
Error: No packages found in current win-64 channels matching: anaconda
You can search for this package on Binstar with
binstar search -t conda anaconda
What am I doing wrong? How do I tell conda to create an environment based on the packages in this directory?
You could try cloning root which is the base env.
conda create -n yourenvname --clone root
Short answer: copy the whole environment from another machine with the same OS.
Why
Dependency. A package depends on other packages. When you install a package online, the package manager conda analyzes the package dependencies and install all the required packages for you.
The dependency is especially heavy in anaconda. Cause anaconda is a meta package depends on another 160+ packages.
Meta packages,are packages do not contain actual softwares and simply depend on other packages to be installed.
It's totally absurd to download all these dependencies one by one and install them on the offline machine.
Detail Solution
Get conda installed on another machine with same OS. Install the packages you need in an isolated virtual environment.
# create a env named "myvenv", name it whatever you want
# and install the package into this env
conda create -n myvenv --copy anaconda
--copy is used to
Install all packages using copies instead of hard- or
soft-linking.
Find where the environments are stored with
conda info
The 1st value of key "envs directories" is the location. Go there and package the whole sub-folder named "myvenv" (the env name in previous step) into an archive.
Copy the archive to your offline machine. Check "envs directories" from conda info. And extract the environment from the archive into the env directory on the offline machine.
Done.
In addition to copying the pkgs folder, you need to index it, so that conda knows how to find the dependencies. See this ticket for more details and this script for an example of indexing the pkgs folder.
Using --unknown as #asmeurer suggests will only work if the package you're trying to install has no dependencies, otherwise you will get a "Could not find some dependencies" error.
Cloning is another option, but this will give you all root packages, which may not be what you want.
A lot of the answers here are not 100% related to the "when offline" part. They talk about the rest of OP's question, not reflected in question title.
If you came here because you need offline env creation on top of an existing Anaconda install you can try:
conda create --offline --name $NAME
You can find the --offline flag documented here
Have you tried without the --offline?
conda create -n anaconda python=3.4 --channel PATHTO\Anaconda_py3
This works for me if I am not connected to the Internet if I do have anaconda already on the machine but in another location. If you are connected to the Internet when you run this command you will probably get an error associated with not finding something on Binstar.
I'm not sure whether this contradicts the other answers or is the same but I followed the instructions in the conda documentation and set up a channel on the local file system.
Then it's a simple matter of moving new package files to the local directory, running conda index on the channel sub-folder (which should have a name like linux-64).
I also set the Anaconda config setting offline to True as described here but not sure if that was essential.
Hope that helps.
The pkgs directory is not a channel. The flag you are looking for is --unknown, which causes conda to include files in the pkgs directory even if they aren't found in one of the channels.
Here's what worked for me in Linux -
(a) Create a blank environment - Just create an empty directory under $CONDA_HOME/envs. Verify with - conda info --envs.
(b) Activate the new env - source activate
(c) Download the appropriate package (*.bz2) from https://anaconda.org/anaconda/repo on a machine with internet connection and move it to the isolated host.
(d) Install using local package - conda install . For example - conda install python-3.6.4-hc3d631a_1.tar.bz2, where python-3.6.4-hc3d631a_1.tar.bz2 exists in the current dir.
That's it. You can verify by the usual means (python -V, conda list -n ). All related packages can be installed in the same manner.
I found the simplest method to be as follows:
Run 'conda create --name name package' with no special switches
Copy the URL of the first package it tried (unsuccessfully) to download
Use the URL on a connected machine to fetch the tar.bz2
Copy the tar.bz2 to the offline machine's /home/user/anaconda3/pkgs
Deploy the tar.bz2 in place
Delete the now unneeded tar.bz2
Repeat until the 'conda create' command succeeds
Here's a solution that may help. It's not very pretty but it gets the job done. So i suppose you have a machine where you have a conda environment in which you've installed all the packages you need. I will refer to this as ENV1 You will have to go to this environment directory and locate it. It is usually found in \Anaconda3\envs. I suggest compressing the folder but you could just use it as is. Copy the desired environment folder into your offline machine's directory for anaconda environments. This first step should get your new environment to respond to commands like conda activate.
You will notice though that software like spyder and jupyter don't work anymore (probably because of path differences). My solution to this was to clone the base environment in the offline machine into a new environment that i will refer to as ENV2. What you need to do then is copy the contents of ENV2 into those of ENV1 and replace files.
This should overwrite the files related to spyder, jupyter.. and keep your imported packages intact.
Related
I am following the tutorial on MLFlow website. I was able to run the train.py and mlflow ui worked fine. Packaging the project tries to use env variable MLFLOW_CONDA_HOME but can't find conda.
I have tried setting the variable to the path of anaconda3/condabin but it doesn't seem to find my executable. This is the error I get:
ERROR mlflow.cli: === Could not find Conda executable at /anaconda3/condabin\bin/conda. Ensure Conda is installed as per the inst
ructions at https://conda.io/docs/user-guide/install/index.html. You can also configure MLflow to look for a specific Conda executable by setting the MLFLOW_CONDA_HOME environment variable
to the path of the Conda executable ===
Adding \bin/conda at the end of my path seems to be the problem, I am not sure why mlflow is doing it. I even tried setting it to my python.exe in my conda env, but no luck. I can't find bin/conda folder in my Anaconda folder anywhere.
I resolved this by running it from Anaconda Prompt. Make sure mlflow is installed in anaconda first as well, nothing else. But the problem then is that it's not well compatible on windows, you would need to split into two steps, activate the conda environment and then run with --no-conda as mentioned here https://github.com/mlflow/mlflow/issues/2674
MLflow 1.5 was just released today.
It doesn't specifically mention it in the github notes, but I had the same issue, where it affixed \bin/conda, and now it doesn't do that anymore.
If you don't have conda environment then you can execute the following command from your terminal
mlflow run <enter your local directory name> --no-conda -P alpha=0.5
This should solve the issues with the environment variable.
I solved the issue by removing the MLFLOW_CONDA_HOME environment variable alltogether. Make sure you have added the path to the conda executable to your PATH variable.
Here is one possible solutions (the fastest one, in my opinion).
Key points:
The project virtual environment should be created with conda.
Use pip to install MLFlow.
Follow the steps for Windows:
Install miniconda (in my case, version 3)
Set conda bat file (installation path + condabin dir + conda.bat) in PATH
Create your project without virtual environment (in my case, I set in PyCharm conda instead of venv and it did not create any virtual environment, just added some external libraries), at least not in the project directory.
Create conda virtual environment manually in the project directory. In your project directory, execute conda create -n venv and follow the instructions (I used default for all the questions there).
Open a terminal and activate conda virtual environment. If you use PyCharm, you will be positioned properly, otherwise just prompt yourself in the project directory. Execute conda activate venv where venv is my virtual environment created at point 4.
Execute python -m pip install mlflow
If you want to test it, you can try one of the tests from MLFlow. E.g., you can use mlflow run https://github.com/mlflow/mlflow-example.git -P alpha=5.0
In my case, it worked.
If you're using mlflow.pyfunc.spark_udf and get an error saying Could not find Conda executable conda then try to define the environment variable MLFLOW_CONDA_HOME in spark-env.sh as Spark doesn't recognize variables defined elsewhere. Also make sure to use the absolute path for the Conda executable.
I faced this issue within a kubernetes deployment with miniconda3 as the base image. Fixed this by setting the MLFLOW_CONDA_HOME env variable to "/opt/conda/"
I am using Windows 10 (all commands run as administrator). I created an environment called myenv. Then I used
conda env remove -n myenv
Now, if I try
conda info --envs
I only see the base environment. However, if I try
conda activate myenv
I'm still able to activate it! I think because under the folder envs, there is still a folder with the name myenv there which doesn't get deleted.
How do I delete the environment for good?
Command-line options can only go so far, unless you get very specific; perhaps the simplest approach is to delete things manually:
Locate Anaconda folder; I'll use "D:\Anaconda\"
In envs, delete environment of interest: "D:\Anaconda\envs\myenv"
Are you done? Not quite; even while in myenv, conda will still sometimes install packages to the base environment, in "D:\Anaconda\pkgs\"; thus, to clean traces of myenv,
Delete packages installed to myenv that ended up in "D:\Anaconda\pkgs\"
(If above don't suffice) Anaconda Navigator -> Environments -> myenv -> Remove
(If above don't suffice) Likely corrupted Anaconda; make note of installed packages, completely uninstall Anaconda, reinstall.
Note: step 3 is redundant for the goal of simply removing myenv, but it's recommended to minimize future package conflicts.
In addition to the first command in the question posted, I had to complete one additional step to completely remove the environment. I had to go to the folder where the environment was stored (e.g. C:\Users*username*.conda\envs\ on a windows machine) and remove the folder with the same name as the environment I deleted. After this second step, I was able to reuse the environment name without any errors.
When I run the following command:
conda env create -f virtual_platform_mac.yml
I get this error
Collecting package metadata (repodata.json): done
Solving environment: failed
ResolvePackageNotFound:
- pytables==3.4.2=np113py35_0
- h5py==2.7.0=np113py35_0
- anaconda==custom=py35_0
How can I solve this?
I am working on Mac OS X.
Conda v4.7 dropped a branch of the Anaconda Cloud repository called the free channel for the sake of improving solving performance. Unfortunately, this includes many older packages that never got ported to the repository branches that were retained. The requirements failing here are affected by this.
Restore free Channel Searching
Conda provides a means to restore access to this part of the repository through the restore_free_channel configuration option. You can verify that this is the issue by seeing that
conda search pytables=3.4.2[build=np113py35_0]
fails, whereas
CONDA_RESTORE_FREE_CHANNEL=1 conda search pytables=3.4.2[build=np113py35_0]
successfully finds the package, and similarly for the others.
Option 1: Permanent Setting
If you expect to frequently need older packages, then you can globally set the option and then proceed with installing:
conda config --set restore_free_channel true
conda env create -f virtual_platform_mac.yml
Option 2: Temporary Setting
As with all Conda configuration options, you can also use the corresponding environment variable to temporarily restore access just for the command:
Unix/Linux
CONDA_RESTORE_FREE_CHANNEL=1 conda env create -f virtual_platform_mac.yml
Windows
SET CONDA_RESTORE_FREE_CHANNEL=1
conda env create -f virtual_platform_mac.yaml
(Yes, I realize the cognitive dissonance of a ..._mac.yaml, but Windows users need help too.)
Including Channel Manually
One can also manually include the channel as one to be searched:
conda search -c free pytables=3.4.2[build=np113py35_0]
Note that any of these approaches will only use the free channel in this particular search and any future searches or changes to the env will not search the channel.
Pro-Tip: Env-specific Settings
If you have a particular env that you always want to have access to the free channel but you don't want to set this option globally, you can instead set the configuration option only for the environment.
conda activate my_env
conda config --env --set restore_free_channel true
A similar effect can be accomplished by setting and unsetting the CONDA_RESTORE_FREE_CHANNEL variable in scripts placed in the etc/conda/activate.d and etc/conda/deactivate.d folders, respectively. See the documentation for an example.
Another solution might be explained here. Basically, if you import an environment.yml file to a different OS (e.g., from macOS to Windows) you will get build errors.
The solution is to use the flag "--no-buils", but it does not guarantee that the environment.yml will actually be compatible. Some libraries, e.g. libgfortran, are not found on Windows channels for Anaconda (see here).
I would use
CONDA_RESTORE_FREE_CHANNEL=1 conda env create -f
to keep using outdated/older packages
I have Anaconda for Python 2, It came packed with a lot of useful packages. During my work, I have added several packages to it using conda install command. Now I have to format my system, and I want to backup/pack all the added libraries, either as full packages or even by knowing the installation command of each one.
I searched StackOverflow, I found one unanswered question with a similar problem, the question suggested conda list -e >file_list.txt to create a file contains all the installed packages, but this is not sufficient for me, I want Anaconda to determine which package is added by me, and by which command, or to pack the added packages in full.
Thanks for help.
I think you can find the solution you are looking for here.
Open the Anaconda prompt
Activate the environment you are interested in
Type conda env export > environment.yml
In the yml you will find all the dependencies and you can use it to create a new virtual environment as a copy of the current one.
For example, on the new/rebooted machine, you can do:
conda env create -f environment.yml
By error, I forgot to specify the WORKON_HOME variable before creating my virtual environments, and they were created in /root/.virtualenvs directory. They worked fine, and I did some testing by activating certain environment and then doing (env)$ pip freeze to see what specific modules are installed there.
So, whe I discovered the workon home path error, I needed to change the host directory to /usr/local/pythonenv. I created it and moved all the contents of /root/.virtualenvs directory to /usr/local/pythonenv, and changed the value of WORKON_HOME variable. Now, activating an environment using workon command seems to work fine (ie, the promt changes to (env)$), however if I do (env)$ pip freeze, I get way longer list of modules than before and those do not include the ones installed in that particular env before the move.
I guess that just moving the files and specifying another dir for WORKON_HOME variable was not enough. Is there some config where I should specify the new location of the host directory, or some config files for the particular environment?
Virtualenvs are not by default relocatable. You can use virtualenv --relocatable <virtualenv> to turn an existing virtualenv into a relocatable one, and see if that works. But that option is experimental and not really recommended for use.
The most reliable way is to create new virtualenvs. Use pip freeze -l > requirements.txt in the old ones to get a list of installed packages, create the new virtualenv, and use pip install -r requirements.txt to install the packages in the new one.
I used the virtualenv --relocatable feature. It seemed to work but then I found a different python version installed:
$ . VirtualEnvs/moslog/bin/activate
(moslog)$ ~/VirtualEnvs/moslog/bin/mosloganalisys.py
python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory
Remember to recreate the same virtualenv tree on the destination host.