I have updated the anaconda packages for the python program I was working on. I now want to install these new packages on computers that are offline.
I have generated a list of the installed packages using the following command:
>conda list -e > packagelist.txt
I want to be able to update the packages on the offline computers using something like this:
>conda install --file (location of 'packagelist') --channel file://(location of the folder where the packages are located)
So my question is: how do I easily make a folder, "pkgs", that contains all the package files indicated in my package list?
Can I simply just use the entire "pkgs" folder that is located in my current Anaconda installation location ? Or are the packages in my package list not necessarily installed inside there?
I apologize if the question is 'nooby', I honestly don't know how to. Looking forward to any help.
If anyone else is struggling with this, these are the steps I took:
Create a file containing the names of all the packages installed in the current environment. One way to do this(already shown in question) is by typing the following on a command prompt (run as admin):
conda list -e > packagelist.txt
Go to the "pkgs" folder that is located in your Anaconda installation directory. Copy all the zipped packages there (only the ones that have a ".tar.bz2" file extension), and paste them inside another folder somewhere, lets call it "pkgs2".
Compare the packages in the "packagelist" text file you created with the packages inside the the new folder you created, "pkgs2". Delete older package versions.For example, there might be two different numpy versions in your "pkgs2" folder. Delete the outdated one if necessarily.
Create index and repodata files. I'm not entirely sure why. The important one is the "repodata.json.bz2" file. You can create this by typing the following in command prompt (obviously your "pkgs2" might be located somewhere else, so adjust accordingly):
conda index C:\Users\myName\Desktop\pkgs2
If you get the following error:
Indexing a copy of the Anaconda conda package channel is neither
necessary nor supported. If you which to add your own packages,
you can do so by adding them to a separate channel.
then you have to go to your "pkgs2" folder and delete the "_license" package.I don't know why it works but it does (saw the solution to the error from a google search).
You should see that 3 new files were added to the "pkgs2" folder after the indexing was completed. They are called ".index.json", "repodata.json" and "repodata.json.bz2".
You can now transfer the "pkgs2" folder and the "packagelist.txt" to another computer. You can also include a setup file for installing anaconda if the computer doesn't already have it installed. To install the new/updated packages offline, enter the following in a command prompt on the offline computer that has anaconda installed (once again, obliviously adjust the file names and locations accordingly) :
conda install --file C:\Users\myName\Desktop\OfflineInstall\packagelist.txt --channel file://C:\Users\myName\Desktop\OfflineInstall\pkgs2
If all went well, you should see something like this in your command prompt:
. And once the new packages are installed, you are done!
You might however, run into the following problem during Step 5 (this is just an example):
UnsatisfiableError: The following specifications were found to be in conflict:
- matplotlib 1.4.3 np19py27_1 -> pyparsing 2.0.3
- pyparsing 2.2.0 py27_0
Use "conda info <package>" to see the dependencies for each package.
Adjust your packages accordingly, either deleting conflicting ones you don't need, or updating the ones shown. Repeat from Step 4 and you should be ok at the end.
Hope this helps someone !
Related
I am going completely mental as I fail to understand and keep my python environment running.
So :
I have a python environment running under Miniconda 3 and that I manage through an environment.yaml file
Environment is used to develop within VSCode
It installs a bunch of external package through conda and then installs my local development package in editable mode (simplified version of the environment.yaml)
name: DLL_ETL
channels:
- conda-forge
- msys2
- defaults
dependencies:
- pandas
- numpy
- pip:
- -e .
- datatable
I update it regularly through "mamba env update"
However, since today this is what I see :
Pip states that the package install is sucessful
In the environment, when running Python from the command line, I can import my package
Package is installed pointing to the correct location (a one-drive)
When opening up an interactive window in VSCode with that environment as kernel : package cannot be found
What am I doing wrong?
What I tried :
Deleting the environment and reinstalling from scratch
Deleting the pycache and dll.egg-info
Manually removing and installing again through pip (and not through mamba, even if it runs pip underneath).
conda list shows the package as properly installed
As can be understood :
Yes I am running a virtual environment
Yes I am in the correct directory (especially when updating through environment.yaml)
Even when manually installing
Yes the directory with my local install is in my Python path and python is running
I managed to find the solution, but no idea why. I found a comment somewhere (I believe : pip: module installed as editable not found) that there is an issue in how some files are created when installing as editable.
More specifically, when installing as editable, two files are created :
___editable___dll_etl_1_0_0_finder.py
editable.dll_etl-1.0.0.pth
both in the environment
Miniconda3\envs\DLL_ETL\Lib\site-packages
When comparing two environments I have, I noticed that the .pth created in the one that broke was not formatted similarly, the .py didn't even exist. More specifically
The original file
What it should look like
When creating the files manually, everything seems to work fine. I haven't tried to update since. Likely it's an issue with setuptools if I understand correctly.
Restart the jupyter kernel.
Add the path to the package above the import statement.
import sys
// path is the directory where the package is located, not the full path of the package
sys.path.append("path/to/package")
import dll_etl
I need to create new virtual environment in PyCharm. There are already some files in the folder.
This is how my project looks like:
I want to create venv in heureka-negativni-reviews
I try: File -> New project.
I switch this settings:
to have Python 3.10 as an interpreter and to create venv in heureka-negativni-reviews
But when I click on create, I got:
EDIT:
I found that I am not able to select Python 3.10 as an interpreter:
But I cannot click OK when I select Python 3.10:
What is the problem, please?
Now we see the reason, why your IDE is not able to create your venv, since the command line isn't either, which is the set of commands your IDE typically uses.
The pip and pip3 belongs to python3.8 and that is another problem, if you want to install python packages for python 3.10, which isn't possible this time without system invention.
Also your python setup seems to be messed.
/home/vojta/Desktop/INTERNET_HANDEL/HEUREKA/heureka-negativni-reviews/venv/bin/python
seems not to be a general system path. It looks that you built your own version of python and installed it by a specific prefix. Installing python correctly is off the topic here.
There are different installers for different platforms which provide install instruction inside their archives.
Before doing this, it's recommended to clean up your messed installation first.
You may find information here and there.
It could also be, that you are trying to create another venv from within an already sourced venv. Such kind of composed venvs can lead to confusion and that would explain your path above a bit more, which contains /venv/.
Also manually remove all your custom python versions from your /home/vojta/ directory, like $ rm -r /home/vojta/Desktop/INTERNET_HANDEL
First ensure that the Python interpreter is added to the list available (by adding as an 'Existing environment').
Then add a venv via link
Then drag and drop your existing files to the venv directory.
before with venv after drag and drop
Currently I have installed a software called ArcGIS Pro 2.1 in C:\Program Files and it comes with a conda that is the only python I have on this computer. Now I need python for other projects and I don't want to go through IT department every time I need to install some python packages because I don't have the admin right.
Will installing another conda in a folder that I have right to edit mess up anything of the ArcGIS conda?
Or can I use the ArcGIS conda to create another python environment outside C:\Program Files in a folder I have right to edit?
Thanks.
You should be able to leverage the existing conda. You just need to make sure you always use the --prefix command to actually write the environment folder into a directory you can write to.
Check out the conda-create help docs. Look at the --prefix option
-- Edit --
There's actually an existing thread discussion about it. Check out this other SO link how to specify new environment location for conda create
I have Anaconda for Python 2, It came packed with a lot of useful packages. During my work, I have added several packages to it using conda install command. Now I have to format my system, and I want to backup/pack all the added libraries, either as full packages or even by knowing the installation command of each one.
I searched StackOverflow, I found one unanswered question with a similar problem, the question suggested conda list -e >file_list.txt to create a file contains all the installed packages, but this is not sufficient for me, I want Anaconda to determine which package is added by me, and by which command, or to pack the added packages in full.
Thanks for help.
I think you can find the solution you are looking for here.
Open the Anaconda prompt
Activate the environment you are interested in
Type conda env export > environment.yml
In the yml you will find all the dependencies and you can use it to create a new virtual environment as a copy of the current one.
For example, on the new/rebooted machine, you can do:
conda env create -f environment.yml
I would like to create a conda environment on a machine that has no network connection. What I've done so far is:
On a machine that is connected to the internet:
conda create -n python3 python=3.4 anaconda
Conda archived all of the relevant packages into \Anaconda\pkgs. I put these into a separate folder and moved it to the machine with no network connection. The folder has the path PATHTO\Anaconda_py3\win-64
I tried
conda create -n python=3.4 anaconda --offline --channel PATHTO\Anaconda_py3
This gives the error message
Fetching package metadata:
Error: No packages found in current win-64 channels matching: anaconda
You can search for this package on Binstar with
binstar search -t conda anaconda
What am I doing wrong? How do I tell conda to create an environment based on the packages in this directory?
You could try cloning root which is the base env.
conda create -n yourenvname --clone root
Short answer: copy the whole environment from another machine with the same OS.
Why
Dependency. A package depends on other packages. When you install a package online, the package manager conda analyzes the package dependencies and install all the required packages for you.
The dependency is especially heavy in anaconda. Cause anaconda is a meta package depends on another 160+ packages.
Meta packages,are packages do not contain actual softwares and simply depend on other packages to be installed.
It's totally absurd to download all these dependencies one by one and install them on the offline machine.
Detail Solution
Get conda installed on another machine with same OS. Install the packages you need in an isolated virtual environment.
# create a env named "myvenv", name it whatever you want
# and install the package into this env
conda create -n myvenv --copy anaconda
--copy is used to
Install all packages using copies instead of hard- or
soft-linking.
Find where the environments are stored with
conda info
The 1st value of key "envs directories" is the location. Go there and package the whole sub-folder named "myvenv" (the env name in previous step) into an archive.
Copy the archive to your offline machine. Check "envs directories" from conda info. And extract the environment from the archive into the env directory on the offline machine.
Done.
In addition to copying the pkgs folder, you need to index it, so that conda knows how to find the dependencies. See this ticket for more details and this script for an example of indexing the pkgs folder.
Using --unknown as #asmeurer suggests will only work if the package you're trying to install has no dependencies, otherwise you will get a "Could not find some dependencies" error.
Cloning is another option, but this will give you all root packages, which may not be what you want.
A lot of the answers here are not 100% related to the "when offline" part. They talk about the rest of OP's question, not reflected in question title.
If you came here because you need offline env creation on top of an existing Anaconda install you can try:
conda create --offline --name $NAME
You can find the --offline flag documented here
Have you tried without the --offline?
conda create -n anaconda python=3.4 --channel PATHTO\Anaconda_py3
This works for me if I am not connected to the Internet if I do have anaconda already on the machine but in another location. If you are connected to the Internet when you run this command you will probably get an error associated with not finding something on Binstar.
I'm not sure whether this contradicts the other answers or is the same but I followed the instructions in the conda documentation and set up a channel on the local file system.
Then it's a simple matter of moving new package files to the local directory, running conda index on the channel sub-folder (which should have a name like linux-64).
I also set the Anaconda config setting offline to True as described here but not sure if that was essential.
Hope that helps.
The pkgs directory is not a channel. The flag you are looking for is --unknown, which causes conda to include files in the pkgs directory even if they aren't found in one of the channels.
Here's what worked for me in Linux -
(a) Create a blank environment - Just create an empty directory under $CONDA_HOME/envs. Verify with - conda info --envs.
(b) Activate the new env - source activate
(c) Download the appropriate package (*.bz2) from https://anaconda.org/anaconda/repo on a machine with internet connection and move it to the isolated host.
(d) Install using local package - conda install . For example - conda install python-3.6.4-hc3d631a_1.tar.bz2, where python-3.6.4-hc3d631a_1.tar.bz2 exists in the current dir.
That's it. You can verify by the usual means (python -V, conda list -n ). All related packages can be installed in the same manner.
I found the simplest method to be as follows:
Run 'conda create --name name package' with no special switches
Copy the URL of the first package it tried (unsuccessfully) to download
Use the URL on a connected machine to fetch the tar.bz2
Copy the tar.bz2 to the offline machine's /home/user/anaconda3/pkgs
Deploy the tar.bz2 in place
Delete the now unneeded tar.bz2
Repeat until the 'conda create' command succeeds
Here's a solution that may help. It's not very pretty but it gets the job done. So i suppose you have a machine where you have a conda environment in which you've installed all the packages you need. I will refer to this as ENV1 You will have to go to this environment directory and locate it. It is usually found in \Anaconda3\envs. I suggest compressing the folder but you could just use it as is. Copy the desired environment folder into your offline machine's directory for anaconda environments. This first step should get your new environment to respond to commands like conda activate.
You will notice though that software like spyder and jupyter don't work anymore (probably because of path differences). My solution to this was to clone the base environment in the offline machine into a new environment that i will refer to as ENV2. What you need to do then is copy the contents of ENV2 into those of ENV1 and replace files.
This should overwrite the files related to spyder, jupyter.. and keep your imported packages intact.