Is it safe to run conda updates that leads to custom package? - python

To be on the safe side, I was advised to keep to
$ conda update anaconda
However, some tutorials on internet recommends
$ conda update conda
$ conda update --all
The above two commands lead to the installation of custom packages. Will it cause anaconda to be unstable? Is it safer to simply keep to conda update anaconda?
Custom packages refer to packages with names that contain the word "custom". They do not belong to the standard anaconda package.
conda packages with version name of 'custom'
I am using Anaconda Python version 3.

Open "Command or Conda Prompt" and run:
conda update conda
conda update anaconda
It's a good idea to run both commands twice (one after the other) to be sure that all the basic files are updated.
This should put you back on the latest 'releases', which contains packages that are selected by the people at Continuum to work well together.
If you want the last version of each package run (this can lead to an unstable environment):
conda update --all

If you are implementing some projects and they depend on the previous version of some packages then running $ conda update --all will be disastrous. It may break some of the code from your project.
Running $ conda update anaconda will be a safe option because along with anaconda it will update all of the required dependencies to the required version.

Related

Segmentation fault: 11 when trying to run Spyder in conda environment

I'm setting up a separate environment to work in. However, whenever I try running spyder, I receive this error:
/Users/user/anaconda3/envs/myenv/bin/pythonw: line 3: 84700 Segmentation fault: 11 /Users/user/anaconda3/envs/myenv/python.app/Contents/MacOS/python "$#"
I have no idea what this means. The commands I used to get to this point were:
conda create -n myenv python
conda activate myenv
conda config --add channels conda-forge
conda install -c conda-forge spyder
spyder
I have anaconda navigator installed, and spyder runs fine in the root environment. I do receive the statement "link image0 hasn't been detected!" every time I run it.
I've always used spyder and anaconda, and I've never had these issues before
I was dealing with this same problem for several weeks and spent many days trying to fix it. Of the countless operations I attempted, I finally got spyder to successfully run from its own conda environment without throwing the segmentation fault error you've referenced.
With a fresh install of anaconda3 on macOS 10.14.6, I created a conda environment from the command line just as you’ve shown:
conda create -n myenv python
conda activate myenv
A comment about:
conda create -n myenv python
Note 1: if you list “spyder” to install during the environment creation there’s no need to include “python” as a package unless you wish to have a specific version. For example, “python=2.7.9”, or “python=3.10.*”. Otherwise, python gets added automatically when the environment is created.
Next,
conda config --add channels conda-forge
instead of setting the conda configuration to use the conda-forge channel and instead of including it as a channel when installing spyder into your conda environment, simply install spyder using the virgin (unmodified) default channel configuration by only typing:
conda install spyder
Now when you run spyder from the command line within your active environment it should open as expected.
As previously indicated, you can install spyder during the environment creation:
conda create -n myenv spyder
Then you can activate the environment to test everything:
conda activate myenv
spyder
Note 2: the official spyder documentation recommends including the -c conda-forge channel, but this was the entry that I removed to finally avoid the segmentation fault. I suspect that although the community repository at conda-forge keeps the most up-to-date versions, there are conflicts not being resolved at creation of the new local environment.
Furthermore, you may notice that the method described above installs spyder version 5.3.3 and spyder-kernels version 2.3.3, both of which are not the most up-to-date. You could check this by running conda list from the command line and then searching for the corresponding entries.
However, if you try to update spyder by calling conda update spyder, the system will tell you that all packages are currently installed. This is because the virgin channel defaults are looking at the anaconda repository, not conda-forge. My suggestion for right now is not to attempt updating spyder in your conda environment because it will update other packages as dependencies that will result in breaking the application again.
In case you’re curious, to get the most recent version of spyder based on the procedure I've described above would be by setting the channel as follows: conda update spyder -c conda-forge. Depending on your OS, this will install spyder 5.4.1 and spyder-kernels 2.4.1 (as of 16-Jan-23), but it will also remove and update several other packages, hence causing compatibility issues.
UPDATE:
My key takeaway is to remove the -c conda-forge from the installation line of code to prevent the segmentation fault when using conda. Everything else is up to the user. Installing with pip, brew, or any other method is discretionary but could also result in more trouble than it’s worth for the already frustrated user.
If you run conda create -n myenv spyder at the environment creation or conda install spyder after creating the environment, the packages pyqt and qtpy are installed alongside spyder because they are dependencies and not explicitly needed (unless specifying specific versions). If you run pip install spyder, the packages pyqt5 and qtpy get installed. (as of 16-Jan-23)
Caution:
(with reservation because if you can follow instructions and you are both okay with and the only one affected by the consequences of your choices when things go wrong, I support you to thoughtfully experiment and explore if you so desire)
According to the docs, using pip to install packages within the conda environment is reserved for experienced users. A big reason for this caution is because pip and conda don’t communicate and so package versions and dependencies can become incompatible if the user is not aware of what’s happening. It should also be pointed out that if and when using pip, one should generally install all the desired conda packages prior to installing a package with pip. Problems usually present themselves when updating, or installing with conda after pip. When this becomes necessary the general advice is to start over and install new packages as well as updated versions in a new environment.
Installing with brew is unsupported by the spyder devs. This can be referenced from the spyder documentation.

How to update Python in Anaconda base environment?

How can I update Python in the Anaconda base environment? Is Python intended to be upgradable in the base environment at all, or should one completely remove and reinstall Anaconda? Any version will eventually go out of support, so there should be some solution.
What I have tried so far, and did not result in Python getting updated:
conda update --all
conda update python and conda update anaconda
conda install python=3.9 or conda install anaconda=2021.11 do not finish after an hour and a half.
mamba install python=3.9 results in "package python_abi-3.7-2_cp37m requires python 3.7.*, but none of the providers can be installed"
No other suggestion is put forward here.
I had a similar problem:
conda install anaconda=2022.10 resulted in unresolvable conflicts, conda install python=3.9 as well.
I finally managed to update by specifying both requirements within the same update command:
conda install anaconda=2022.10 python=3.9
In the base environment, run mamba update python --no-pin (of course, you could do it with conda, but then you're up for a long wait!).
Official documentation by Anaconda advises against upgrading to another major version of Python. It mentions the method you have already tried:
conda install python=3.9
but the process did not finish for you. This is in line with their documentation which says:
It is not recommended, rather it is preferable to create a new environment. The resolver has to work very hard to determine exactly which packages to upgrade.
Instead, you should create a new environment as suggested by the documentation and answers to this question.
conda create -n py39 python=3.9 anaconda
Open the Anaconda prompt by searching for it in the start menu or by running "anaconda-prompt" in the command prompt.
Run this command in the Anaconda prompt to update the base environment:
conda update --all

Updating a package in a virtual environment with conda

I am trying to update the package flopy, within a virtual environment called flopyenv using the Anaconda Prompt command line. First, I activate the virtual environment using conda activate flopyenv. Then to update flopy, I've tried conda update flopy. I get the following error:
PackageNotInstalledError: Package is not installed in prefix.
prefix: C:\Users\person\Anaconda3\envs\flopyenv
package name: flopy
which makes sense since the flopy directory was installed in a different directory (C:\Users\person\Anaconda3\envs\flopyenv\lib\site-packages\flopy). Also, I have checked using conda list and flopy is listed in the environment. How do I point conda update to the proper directory to update flopy within the virtual environment?
Edit: Per merv's comment I've included the output below.
(flopyenv) C:\Users\person>conda list -n flopyenv flopy
# packages in environment at C:\Users\person\Anaconda3\envs\flopyenv:
#
# Name Version Build Channel
flopy 3.3.1 pypi_0 pypi
Looks like I used pip to install flopy not conda which I guess is why the directories weren't lining up when I tried updating using conda. I was able to successfully update the flopy package using pip.
Seems like OP figured it out, but it may be worth mentioning that in addition to using pip to update, it might also work to enable the pip_interop_enabled configuration option. I would only do this on a per-environment basis:
conda activate flopyenv
conda config --env --set pip_interop_enabled true
conda update flopy
However, this is still (as of Conda v 4.9) considered an experimental feature, AFAIK.

Best practice to manage dependencies between conda and pip

I'm developing a Python library, which depends on multiple packages. I'm struggling to find the most straightforward of managing all those dependencies with the following constraints :
Some of those dependencies are only available as conda packages (technically the source is available but the build process is not something I want to get into)
Other dependencies are only available via pip
I need to install my own library in editable or developer mode
I regularly need to keep the dependencies up-to-date
My current setup for the initial install :
Create a new conda environment
Install the conda-only dependencies with conda install ...
Install my library with pip install -e .
At this point, some packages were installed and are now managed by conda, others by pip. When I want to update my environment, I need to:
Update the conda part of the environment with conda update --all
Update the pip part of the environment by hand
My problem is that this is unstable : when I update all conda packages, it ensures the consistency of the packages it manages. However, I can't guarantee that the environment as a whole stays consistent, and I just realized that I was missing some updates because I forgot to check for updates in the pip part of the environment.
What's the best way to do this ? I've thought of :
Using conda's pip interoperability feature : this seems to work, but I've had some dubious results, probably because of my use of extras_require
Since pip can see the conda packages, the initial install is consistent, which means I can simply reinstall everything when I want to update. This works but is not exactly elegant.
The recommendation in the official documentation for managing a Conda environment that also requires PyPI-sourced or pip-installed local packages is to define all dependencies (both Conda and Pip) in a YAML file. Something like:
env.yaml
name: my_env
channels:
- defaults
dependencies:
- python=3.8
- numpy
- pip
- pip:
- some_pypi_only_pkg
- -e path/to/a/local/pkg
The workflow for updating in such an environment is to update the YAML file (which I would recommend to keep under version control) and then either create a new environment or use
conda env update -f env.yaml
Personally, I would tend to create new envs, rather than mutate (update) an existing one, and use minimal constraints (i.e., >=version) in the YAML. When creating a new env, it should automatically pull the latest consistent packages. Plus, one can keep the previous instances of the env around in case a regression is need during the development lifecycle.

Do these commands do the same action?

Basically I would like to know if those 2 snippets do the same thing :
conda install -n myEnv myPackage
VS
conda activate myEnv
pip install myPackage
Or in a different way, does a pip install when a conda environment is activated equal doing a conda install on myEnv ?
EDIT : I thought it was obvious but => more precisely, does the second snippet only install the package on the environment or on the overall system ?
PS : Asking because there's a package available with pip but not with conda and I want it to only be installed on myEnv
The Anaconda docs make it clear that if you use conda as your virtual environment manager, you should stick to conda install to install new packages as far as possible:
Unfortunately, issues can arise when conda and pip are used together
to create an environment, especially when the tools are used
back-to-back multiple times, establishing a state that can be hard to
reproduce. … Running conda after pip has the potential to
overwrite and potentially break packages installed via pip. Similarly,
pip may upgrade or remove a package which a conda-installed package
requires.
If you can't get all the packages you need from a conda channel, they say this, which is good advice even if you don't use pip:
If there is an expectation to install software using pip along-side
conda packages it is a good practice to do this installation into a
purpose-built conda environment to protect other environments from any
modifications that pip might make.
Finally the same document notes:
Use conda environments for isolation
create a conda environment to isolate any changes pip makes
environments take up little space thanks to hard links
care should be taken to avoid running pip in the “root” environment
Provided you activate the correct conda environment first, the pip install command(s) should use that environment's pip and install only into that environment.
Yes and no.
pip downloads and installs the package from PyPI whereas conda does the same from Anaconda repositories.
There are packages in PyPI not present in Anaconda, and the other way around.
For managing the environment I would choose one way or the other, since with pip you can freeze into a requirements.txt (pip freeze > requirements.txt) and conda you can either export the whole environment (conda env export) or the list of packages (conda list --export > requirements.txt). However if you try to use a conda-generated file from pip, it will most probably fail.

Categories