I'm using Ubuntu 16.04 LTS with Anaconda 2, which takes over 5 gb disk space. Is it normal to take such large space, or I can make it smaller by removing some unnecessary folders?
P.S. Some commands such as "conda clean" have been used, I just wonder if there are some repeated modules installed...
I have seen anaconda accumulate lots of garbage package caches and tarballs. To delete caches, tarballs and lock files which are not used and reduce a little bit the space used, you can try:
conda clean -a
It depends what do you really want to use.
Install miniconda instead of Anaconda and then install required packages 1 by 1 using
conda install
this will definitely reduce the size. :)
Important to note that conda is a package/environment manager, which can build an anaconda environment.
More often than not, they are considered as one and the same which is untrue.
If you want to simply use conda, the package/environment manager and not have the entire anaconda environment packaged with it, you can install miniconda -
https://conda.io/miniconda.html- which gives you the conda application, without the bloat* of the entire scientific stack of anaconda.
miniconda will have a far smaller footprint, and then using conda to create specific (streamline/smaller) environments for usage will also be more storage-efficient.
Make the separation between conda and anaconda in your thinking, and it should make things easier to understand.
*not quite fair, it is not that it is bloat, it is just overkill for 99% of users
Related
I have downloaded the anaconda3 a complete newer version from official site.. I tried to create an environment but the 'solving environment' keeps on running..
I tried turning off windows defender but it didn't work.. someone plz help.. I am using windows 11 pro, and I have downloaded and installed Anaconda3-2022.10-Windows-x86_64 on 23/12/2022
The actively available Anaconda builds for Python 3.6 all use 3.6.10, so you would have a possibly easier solve with:
conda create -n ssd_env python=3.6 anaconda
However, there is the custom build, anaconda=custom=py36_1 that should be generally compatible with python>=3.6,<3.7. That is what I get when simulating the solve from OP:
CONDA_SUBDIR=win-64 mamba create -n ssd_env --override-channels -c anaconda python=3.6.8 anaconda
which solves almost instantly with Mamba.
Using conda also takes unreasonably long for me, even when explicitly identifying the anaconda=custom=py36_1. The reason this takes so long is that this anaconda package has no version constraints on the 100+ packages it specifies, which means a huge search space. This is aggravated by the fact that the solvers work from latest to oldest package versions, and the versions that are expected to be identified are ~3 years down the stack.
I recommend:
Use Mamba as the solver.
Don't use the anaconda package unless absolutely needed. Most users do not ever need all those packages - instead just specify the packages actually required.
Many Python packages don't have pre-built conda packages, so Anaconda users are frequently forced to use pip to install packages. I have to do this routinely, since so many packages don't have conda packages, not even in the most common alternate channel(s) like conda-forge or bio-conda.
This open issue was already reported in the Anaconda github support repo https://github.com/ContinuumIO/anaconda-issues/issues/10634. However, no answers have been forthcoming in almost 1 year. I am asking here because responses are typically faster and shared more widely than in support forums for individual products.
I hate the productivity loss of re-installing Anaconda, particularly a long-standing installation, because it can take 3-4 hours to backup and export existing environment build files as requirements.txt, remove an existing Anaconda installation, clean out the Windows Registry, search the Windows file system for leftover detritus, and then rebuild all of my environments one at a time.
Does anyone know a trick, or have a Python script or some other workaround(s) to refresh the Package Index within a conda environment or for ALL environments created and managed by the Anaconda Navigator GUI. It would be awesome if there was an updater widget within the Anaconda Navigator GUI to allow users to select for which virtual environment(s) they want the package index to be updated.
What I have tried
In the Anaconda Navigator GUI "Environments" tab, clicking on the "Update Index..." button does not get updated with the list of packages installed "behind the scenes" from a Anaconda Command Prompt.
The result I get
pip-installed packages are NOT included in the Anaconda Package Index update process. It does not find packages in environments installed inside and outside of the Anaconda3 root directory. It doesn't even find all packages underneath the \envs folder. This makes me think packages are not installed into the currently selected environments, so it takes time to verify their location in C:\ProgramData\Anaconda3\envs, C:\Users\username\AppData\Local, or elsewhere.
What else I have tried: after having a corrupted Anaconda and Spyder installation that would not start at all, I posted requests for help on various support forums. I got answers that were not much help, like "Just don't mix pip and conda packages, use one or the other". That is not practical since so many Python packages are not available in the conda package format. I have gotten that impractical advice from Anaconda and Spyder developers in the past.
Severity and impact
This is an important issue, since it is possible to use / misuse conda and pip and inadvertently corrupt Anaconda so badly that it requires a painful and time-consuming removal and re-installation of the entire Anaconda distribution.
A possible solution beyond my current cable-tow... If someone can build an intelligent and transparent converter built into the PyPi, Anaconda Cloud, Conda-forge, and other channels that made this conversion automatic and validated, then this conversation might not be needed.
Anaconda is a powerful and popular toolkit normally associated to Python development. Its snake-name is probably chosen to remind of Python, but the feeling I get using it, is that all it does is handling virtual environments, pre-installing useful Python libraries, tools and packages, and then (and probably most importantly) dealing with dependencies and installations.
Can I set Anaconda up to create virtual environments for Go, too?
From the fact Anaconda can install Go, I get to think it's actually possible, but then I also get the feeling all dependencies-solving power of Anaconda is probably made thinking of Python only.
...Can it be done?
I also get the feeling all dependencies-solving power of Anaconda is probably made thinking of Python only.
Absolutely not. Anaconda installs Conda. Conda is a package and environment manager, and not just for Python, but also for R, Julia, Perl, Scala, etc... including Go.
You will need to create a new environment if you want to run Go:
conda create --name go --channel-name conda-forge go
Then move to the new environment to run go using:
activate go
Since Go supports vendoring, you can use an utility like https://github.com/golang/dep to have all dependencies in the project.
I have just spent 2 days trying to build Tensorflow from source, and finally succeeded when I realized that sudo pip (even with the -H flag) was not finding my anaconda pip, but instead finding a pip installed with apt. Running, then, sudo -H ~/anaconda3/bin/pip ... fixed my problem.
In order to avoid this kind of issue ever again (I had several issues in this process with the "wrong" python being used), is it possible for me to completely remove python from my system, keeping only Anaconda? Is it advisable?
No, you can't. Python is an essential part of ubuntu (and of just any linux distrib nowadays), so removing it will just break your system. Well, if you want a broken system then it's of course up to you ;)
The right solution is to use virtualenvs for your own workspaces instead, so you can have different isolated installs with different python and 3rd part packages versions. This also avoids breaking anything when two projects depends on different versions of a same package. For a local use you probably want to install virtualenvwrapper too (it's an addon over virtualenv that makes working with virtualenvs easier - switching between virtualenvs, running hooks on env switch etc).
Oh and yes: those virtualenvs are per-user so no more sudo and you don't risk messing up your system install (thx Attie for mentioning this).
And to be even more exhaustive: darthbith mentions in a comment that anaconda has it's own virtualization/package management system conda that might make virtualenv redundant and is anyway a best fit if you're using anaconda.
This is not just a Ubuntu issue but also a linux world wide issue. The system python is at the core of apt-get and yum package managers. Also the modern grub is based on python so removing it can make your machine unbootable.
In short, this will affect RHEL related distributions (CentOS/Fedora) and Debian related distributions (Debian/Ubuntu).
So, I have seen What is the difference between pip and conda?. However, all of the answers there appear to be from Anaconda supporters. So, that made me wonder: why is pip still the standard? why hasn't everyone just moved to anaconda?
I understand that anaconda only works with its own python, but is that the only disadvantage?
Based on my limited experience, I would guess that the main advantage of pip over conda is the ability to still install packages that are not available from conda or Anaconda.org.
https://conda.io/docs/using/pkgs.html#install-non-conda-packages - says basically the same.
I have been using conda for a while now, mostly studying Machine Learning and related subjects. I am a happy user 99.99% of the time. But when one faces challenges like building and installing tensorflow with GPU support for Mac that would support his or her rather specific/outdated GPU, one can't really rely on conda.
One huge advantage of pip is the built-in ability to install packages system-wide via f.ex.
sudo -H pip install ipython
It actually is smart enough to do this by default if run as the root user, installing to some directory in the global execution path. (/usr/local/bin?)
What can actually be considered an advantage for some things is that pip compiles packages (by default). So some packages like f.ex. theano which are actually optimized upon installation should not be installed via conda, or you are possibly missing out on this.
Finally, as mentioned, pip is directly linked to Python's package archive, whereas conda assumedly needs to be told when a new package was uploaded via a new configuation.