I am using snakemake as workflow manager. I first create a conda environment in JupyterLab (anaconda) with: conda create base. Then I activate a .yaml file with mamba env create --name <name of the env> --file environment.yaml. In the .yaml file I include among other the following dependencies: R = 4.1.3, tensorflow = 2.8.0. At this point I get an error message:
Encountered problems while solving:
nothing provides requested r 4.1.3**
nothing provides requested tensorflow 2.8.0**.
As far as I understand, the dependencies are going to be take care of by conda in the local environment. I eventually installed manually tensorflow in the local environment with pip install tensorflow. Nevertheless, I get the same error message when I try to activate the .yaml file. I know that I have already installed R.
What should I do ? I have Ubuntu 20.04.4 LTS.
Many thanks.
Packages are not found because the current highest versions of tensorflow from conda is 2.7.0 and the highest available for R is 4.1.0, so you have to downgrade if you want to install using conda.
For tensorflow, you can tell conda to use pip instead, but I don't know if there is a similar workaround for R.
Edit: Fixed R version
I've been using Anaconda for a few years now, but since I started using a Mac with a M1 processor I had to deal with a bunch of problem with the installation of some packages, which left me a little confused about some basic concepts.
For example, I was trying to install Tensorflow, and it turns out the proper way is to install miniforge, and get Tensorflow from the conda-forge channel (which is the default for miniforge), as explained here.
Then, I was wondering whether I could do the same using Anaconda/Miniconda...set up the conda-forge channel as default, and install Tensorflow (or any other arm-compatible package), but I've been told it's not possible
So, now I'm trying to understand how this all works.
If a Tensorflow version compatible with M1 processors exists in the conda-forge channel (and it does exist), why can't I install it by using Anaconda/Miniconda, after configuring it to use said channel? To phrase it in another way, what is the difference between Anaconda/Miniconda and Miniforge, other than the channels they look into for packages (and, as I understand, some licenses)?
Here there is a similar question, but the answers don't seem to address my main concern (why Anaconda/Miniconda with conda-forge as default channel is different than miniforge).
It's not impossible, but you'll have to jump through hoops to get it done.
First, if you have an Anaconda installation, you can't install conda-forge packages into the base environment consistently, because the anaconda package in the base environment of Anaconda will conflict with packages from conda-forge.
Second, since Anaconda is only x86_64 at the moment, you can only install it via Rossetta emulation. After that, you need to tell conda that you need arm64 compatible packages by setting the env variable CONDA_SUBDIR.
CONDA_SUBDIR=osx-arm64 conda create -n native numpy -c conda-forge
will get you a new env with native arm64 packages. However if you want to update this env, you have to prefix all your conda commands with CONDA_SUBDIR=osx-arm64.
To fix this permanently, you can do the following
conda activate native
conda config --env --set subdir osx-arm64
which will make conda use osx-arm64 for this environment.
Seriously, I have no idea what is up. My minimal working example (Windows 10 Pro, with administrative access, updated as of 2019-11-06):
Fresh install of most recent MiniConda (I used "all users")
.condarc as per conda-forge:
channels:
- conda-forge
- defaults
channel_priority: strict
conda create --name TEST
conda activate TEST
conda install pillow
python -c "from PIL import Image"
ImportError: DLL load failed while importing _imaging: The specified
module could not be found.
I'm tearing my hair out here. Things I have tried/noticed:
Despite using conda-forge strict, the vc and vc2015 packages are installing from pkgs/main.
I have upgraded by base environment, which triggers a switch to conda-forge python. However, the base environment continues to use vc from pkgs/main. (for all I know, this is the correct behavior, but with DLL load failures and vc/vc2015 coming from a different repo than the package, I get suspiscious).
I have ensured that all paths involved, including base install and my environment install paths, have no spaces.
The hack of conda remove pillow --force then pip install pillow...works? It worries me, because I can't figure out why, and I don't know if there will be downstream C incompatibilities
If the above instructions are repeated, but without the switch to conda forge, no error occurs.
NOTE: conda remove pillow does NOT work with --force, because it takes ALL the dependent packages with them in a real environment, as well as any unused dependencies now that those packages are removed. The above method with --force allows a drop-in replacement of pillow in an otherwise well-formed environment.
I installed opencv 3.4.2 from source using this guide
One major change i did was point the python-executable to my anaconda3 env python3.6.
The configuration was passed successfully. However, when I activate the anaconda environment and try to import cv2. I get the no module named cv2 still.
But the opencv installed is shown in the conda list as here:
conda list -n anaconda-env | grep -i cv
Ouptut is :
libopencv 3.4.2 h765d7f9_1
opencv 3.4.2 py36h40b0b35_1
py-opencv 3.4.2 py36h765d7f9_1
Can anyone help me to know whats going wrong ? I am aware anaconda channel has a opencv pacakage. But it has only opencv3.1
The conda-forge organization contains one repository for each of the installable packages. Such a repository is known as a feedstock. A feedstock is made up of a conda recipe (the instructions on what and how to build the package) and the necessary configurations for automatic building using freely available continuous integration services.
If you want to check the Feedstock for opencv, it can be found here.
To install a package from conda-forge in a conda environment like anaconda, use : conda install -c conda-forge opencv
I use an out-of-the-box Anaconda installation to work with Python. Now I have read that it is possible to also "include" the R world within this installation and to use the IR kernel within the Jupyter/Ipython notebook.
I found the command to install a number of famous R packages:
conda install -c r r-essentials
My beginner's question:
How do I install R packages that are not included in the R-essential package? For example R packages that are available on CRAN. "pip" works only for PyPI Python packages, doesn't it?
Now I have found the documentation:
This is the documentation that explains how to generate R packages that are only available in the CRAN repository:
https://www.continuum.io/content/conda-data-science
Go to the section "Building a conda R package".
(Hint: As long as the R package is available under anaconda.org use this resource. See here: https://www.continuum.io/blog/developer/jupyter-and-conda-r)
alistaire's answer is another possibility to add R packages:
If you install packages from inside of R via the regular install.packages (from CRAN mirrors), or devtools::install_github (from GitHub), they work fine. #alistaire
How to do this:
Open your (independent) R installation, then run the following command:
install.packages("png", "/home/user/anaconda3/lib/R/library")
to add new package to the correct R library used by Jupyter, otherwise the package will be installed in /home/user/R/i686-pc-linux-gnu-library/3.2/png/libs mentioned in .libPaths() .
To install other R Packages on Jupyter beyond R-essentials
install.packages('readr', repos='http://cran.us.r-project.org')
One issue is that the specific repository is the US.R-Project (as below). I tried others and it did not work.
N.B. Replace readr with any desired package name to install.
Here's a conda-centric answer. It builds on Frank's answer and the continuum website: https://www.continuum.io/content/conda-data-science with a bit more detail.
Some packages not available in r-essentials are still available on conda channels, in that case, it's simple:
conda config --add channels r
conda install r-readxl
If you need to build a package and install using conda:
conda skeleton cran r-xgboost
conda build r-xgboost
conda install --use-local r-xgboost
that last line is absent in the continuum website because they assume it gets published to anaconda repository first. Without it, nothing will be put in the envs/ directory and the package won't be accessible to commandline R or Jupyter.
On a mac, I found it important to install the Clang compiler for package builds:
conda install clangxx_oxs-64
I found an easy workaround. I suppose that you have an RStudio IDE for you R. It is weird to use RStudio for that, but I tried straight from R in my terminal and it didn't work. So, in RStudio console, just do the usual adding the path to your anaconda directory (in OSX,'/Users/yourusernamehere/anaconda/lib/R/library')
So, for example,
install.packages('package','/Users/yourusernamehere/anaconda/lib/R/library')
I feel ashamed to post such a non-fancy answer, but that is the only one that worked for me.
Adding it here so other beginners already working with Jupyter notebooks with Python and interested in using it with R: additional packages available for Anaconda can be installed via terminal using the same command used to instal the essential packages.
Install r-essentials
conda install -c r r-essentials
Install microbenchmark (infrastructure to accurately measure and compare the execution time of R expressions)
conda install -c r r-microbenchmark
To install a CRAN package from the command line:
R --slave -e "install.packages('missing-package', repos='http://cran.us.r-project.org')"
I had a problem when trying to install package from github using install_github("user/package") in conda with r-essentials. Errors were multiple and not descriptive.
Was able to resolve a problem using these steps:
download and unzip the package locally
activate correct conda environment (if required)
run R from command line
library(devtools)
install('/path/to/unzipped-package')
Command failed due to missing dependancies, but now I know what's missing!
run install.packages('missing-package', repos='http://cran.us.r-project.org') for all dependancies
run install('/path/to/unzipped-package') again. Now it should work!
Use Conda Forge
Five years out from the original question, I'd assert that a more contemporary solution would simply be: use Conda Forge. The Conda Forge channel not only provides broader coverage of CRAN, but also has a simple procedure and great turnaround time (typically under 24 hours) for adding a missing CRAN package to the channel.
Start from Conda Forge
I'd recommend using Conda Forge for the full stack, and use a dedicated environment for each R version you require.
conda create -n r41 -c conda-forge r-base=4.1 r-irkernel ...
where ... is whatever additional packages you require (like r-tidyverse). The r-irkernel package is optional, but included here because OP mentions using R in Jupyter.
If your environment with Jupyter (which should be in a separate environment) also has nb_conda_kernels installed, then this environment will automatically be discovered by Jupyter.
Install from Conda Forge
Generally, all R packages on CRAN have a r- prefix to the package name on Conda Forge. So, if your package of interest is pkgname, first try
conda install -n r41 -c conda-forge r-pkgname
If the package is not available, then proceed to either add it or request it.
Submit a CRAN package with Conda R Skeleton Helper
There is a helpful script collection, called conda_r_skeleton_helper for creating new Conda Forge recipes for CRAN packages. There are clear directions in the README.
In broad strokes, one will
clone the conda_r_skeleton_helper repository
edit the packages.txt file to include r-pkgname
run the script to generate the recipe
fork and clone the conda-forge/staged-recipes
copy the new recipe folder to the stage-recipes/recipes folder
commit changes, push to the fork, then submit a Pull Request back to Conda Forge
This takes maybe ~15 mins of work. Once submitted, most packages take under 24 hours to get accepted, feedstocked, and deployed to the Conda Forge channel. Once the feedstock is up and running, the Conda Forge infrastructure uses a bot to auto-detect version updates, generate new pull requests, and even auto-merge Pull Requests that successfully build. That is, maintainers have a very minimal workload, and if there are issues, a team is available to help out.
File a Package Request
For users uncomfortable with creating and maintaining a Conda Forge build, packages can be requested on Conda Forge's staged-recipes repository by filing a new Issue. There is a template for Package Request, that includes some information fields to be filled in.
Install rpy2 with conda and add following line in your Jupyter notebook.
%load_ext rpy2.ipython
In next chunks, you can simply run any r code by specifying %R
Below is my favorite method to install and/or load r package
%R if (!require("pacman")) install.packages("pacman")
%R pacman::p_load(dplyr, data.table, package3, package4)
p_load argument will install + load the package if it's not in your lib else it will simply load it.
Someone suggested a not so elegant way around it, but actually it doesn't matter as long as it works fine.
install.packages('package','/Users/yourusernamehere/anaconda/lib/R/library')
I spent almost an entire morning looking for an answer to this problem. I was able to install the libraries on RStudio but not on Jupyter Notebook (they have different versions of R) The above solution "almost" worked, it's just that I found the Jupyter Notebook was trying to install in a different directory, and it will report what directory. So I only changed that and it worked as a charm... thanks to Dninhos
What worked for me is install.packages("package_name", type="binary").
None of the other answers have worked.