First of all, I’m new to rpy2 / jupyter so please don’t judge me if this isn’t the correct place to ask my question.
I am trying to set up an integrated workflow for data analysis using R and Python and I encounter the following error:
I am on Ubuntu 19.04. running a conda environment using Jupyter 1.0.0, Python 3.7.4, R 3.5.1, r-irkernel 1.0.2 and rpy2 3.1.0 and I installed the R-package Seurat through R.
When I create a Jupyter notebook using the R-kernel, I can load Seurat with library(Seurat) just fine.
I can also use R code in python using rpy2 and the rmagic such as:
%load_ext rpy2.ipython
%%R
data(allen, package = 'scRNAseq')
adata_allen <- as(allen, 'SingleCellExperiment')
However when I try to load Seurat using rpy2 the kernel crashes:
%%R
library(Seurat)
And I get the following message:
Kernel Restarting
The kernel appears to have died. It will restart automatically
Jupyter gives the following message in the command line:
[I 16:39:01.388 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 23284ec0-63d5-4b61-9ffa-b52d19851eab restarted
Note that other libraries such as library(dplyr) load just fine using rpy2.
My complete conda environment can be found in the attached text file.
I just can’t seem to figure out what is causing the problem. Is there a way to get a more verbose error message from Jupyter?
Your help would be greatly appreciated!
Regards Felix
The R package Seurat is using an other R package called reticulate, providing a bridge to Python from R.
Unfortunately, whenever rpy2 and reticulate are involved R ends up being initialized twice, which results inevitably in a segfault. This is still an open bug at the time of writing. The issue tracking on the rpy2 side (a link to the reticulate side of the tracking can be found there) is here:
https://bitbucket.org/rpy2/rpy2/issues/456/reticulate-rpy2-sharing-r-process
I've got the same problem with you. But I downgrade to Seurat 3.0.2, your problem will be fixed. To use the user defined R kernel for rpy2 with conda, run the code before at the very beginning (before imoort rpy2)
# user defined R installation
import os
os.environ['R_HOME'] = '/path/to/miniconda/envs/seurat/lib/R' #path to your R installation
os.environ['R_USER'] = '/path/to/miniconda/lib/python3.7/site-packages/rpy2' #path depends on where you installed Python.
This worked for me, while facing issue of kernel getting dead during importing robjects from rpy2:
import os
os.environ['R_HOME'] = '/Users/<your user>/anaconda3/envs/<env name>/lib/R'
# import your desired module
from rpy2.robjects.packages import importr
I had the same problem and I am also using R and python with a Jupyter notebook in docker.
I solved the Kernel crash issue by starting my notebook or Python code with this:
import os \
os.environ['R_HOME'] = '/usr/lib/R'
/usr/lib/R is where I have my system's R installation and libraries, and should be an R version needed by rpy2. Hope this helps.
I tried to install rpy2 in the jupyter/r-notebook:hub-2.3.1 Docker image which comes with Python 3.10.5, IPython 8.4.0, R 4.1.3.
If I install rpy2 in a Terminal window with pip:
python3 -m pip install rpy2
and I start IPython in the Terminal, and type import rpy2,this first step works. But the next step, namely: import rpy2.robjects as robjects results in the following not-so-instructive error message:
Error in glue(.Internal(R.home()), "library", "base", "R", "base", sep = .Platform$file.sep) :
4 arguments passed to .Internal(paste) which requires 3
Error: could not find function "attach"
Error: object '.ArgsEnv' not found
Fatal error: unable to initialize the JIT
The reason is some subtle incompatibility between the rpy2 package on PyPI and the Python and R installations in the jupyter/r-notebook image. The incompatibility occurs because Python and R were installed using Conda in the r-notebook image.
If I install rpy2 also with Conda, like this:
conda install --yes rpy2
then everything works as advertised.
Lessons learned
If Python and R were installed with OS package installers, then you can probably install rpy2 with pip.
If Python and R were installed with Conda, then install rpy2 also with Conda.
(the most embarrassing bit): There is a jupyter/datascience-notebook which comes with rpy2 preinstalled (plus a lot of other goodies), no need to install anything:
jupyter/datascience-notebook includes libraries for data analysis from
the Julia, Python, and R communities.
Everything in the jupyter/scipy-notebook and jupyter/r-notebook
images, and their ancestor images rpy2 package The Julia compiler and
base environment IJulia to support Julia code in Jupyter notebooks
HDF5, Gadfly, RDatasets packages
Related
I'm trying to use the leiden algorithm in R, I would like to use it in a shiny app.
But I can't even get it to work in Rstudio console.
When I use reticulate (v 1.26) as is, i can't set a python path or conda environment. I get following error:
Error in Sys.setenv(PATH = new_path) : wrong length for argument
When I use reticulate (also v 1.26) in renv (v 0.16), at least I can set a python path and run python code,
but as soon as I'm using code that uses the leidenalg internally I get this error, although it is installed in the conda environment it is using:
Error in py_call_impl(callable, dots$args, dots$keywords) :
ImportError: Please install the leiden algorithm: `conda install -c conda-forge leidenalg` or `pip3 install leidenalg`.
If I specifically try to import the module with import leidenalg, the error is different:
Error in py_run_file_impl(file, local, convert) :
ImportError: DLL load failed while importing _c_leiden: The specified procedure could not be found.
I have tried using the R package leiden , I have tried importing python modules and using it in R , I've tried sourcing a file with python functions.
I have also tried more than once to create a new conda environment hoping it would fix the DLL problem.
when I try to import leidenalg in a python-consule (in the conda environment) I can load it without issues
What do I need to sacrifice to the R-gods to be able to use the leiden algorithm?
I want to use spacy in a Python project on Juptyer Notebook but when i try to import the module I get the Error "ImportError: cannot import name Deque". I installed spacy in PyCharm in my virtual env via pip install -U spacy
I've seen another question on here that was similar (ImportError cannot import name Deque with spacy) but I can import spacy in PyCharm just fine so I assume it's a Jupyter Notebook specific problem.
I also checked the spacy version on PyCharm and JN and they're both the same, however it seems like my virtual env and JN don't use the same Python Version (3.8.1 in my venv on PyCharm and 3.6.0 on JN)
[Python Versions in PyCharm and JN]
May that be the problem or is something else causing it?
I also tried to uninstall and install again in PyCharm but it didn't change anything and also tried to pip install spacy in JN again but that also didn't solve it, JN told me I might have to restart the kernel but after restarting it still couldn't import spacy without that error.
I've tried so much that after some time i just get the error ImportError: cannot import name errors but I assume it's still the same problme.
I'm very grateful for any suggestions and tips!
So for future references:
I just 'solved' this problem by making a new environment with Python 3.6 since afaik Jupyter Notebook doesn't support Python versions highter than 3.6.x as of now yet, set up a new venv and now both PyCharm and JN use the same Python version (3.6.12) and I could successfully import spacy.
So I'm assuming the different Python versions really were the problem.
Maybe it's not even necessary to create a new venv and everything but I wanted to start clean again to not have further problems.
I have installed the netcdf4 library through
conda install -c anaconda netcdf4
And it shows up fine on my conda environment:
However, when I try to import it in Spyder, I am getting an
"ModuleNotFoundError: No module named 'netcdf4'"
I have tried the solutions here. As you can see below, both the interpreters are the same (in tools/preferences it's set as python.exe, not pythonw.exe):
Yet when I try to import the netcdf4 module, it says there is no module named this.
Any ideas what the cause could be? It's been two days of trying to get Anaconda to work and it has not been a smooth experience at all, really close on giving up on Anaconda as a whole.
Added per request, a screenshot of my entire IDE. Notice in the console I ran pip install netcdf4, where it says I already installed it. Directly below is importing the module and the error:
And how the console looks on each startup:
The docs/tutorial used stated
import netcdf4
While now it is case sensitive
import netCDF4
Such a simple fix..
I would like to use both R and Python languages inside a Kaggle Kernel. Thus, when running
!pip install rpy2
inside a Kaggle Notebook I got the following error
Error: rpy2 in API mode cannot be built without R in the PATH or R_HOME defined. Correct this or force ABI mode-only by defining the environment variable RPY2_CFFI_MODE=ABI
I've found out a solution for users of Python within R, but a solution for calling R within Python in a Kaggle Kernel has not yet been provided.
One can notice that a Kaggle Kernel is using behind an anaconda environment. For example,
/opt/conda/bin/python3.7
Also, it is necessary to have R installed on this conda environment. Thus, we can use the subprocess library to run the following script for installing R
import subprocess
subprocess.run('conda install -c conda-forge r-base', shell=True)
and the corresponding rpy2 package
!pip install rpy2
I have provided a notebook on Kaggle with a complete explanation. I'll appreciate your comments.
I received an error on atom, it asked to install ipkernal using pip.
Not sure what to do. I have Anaconda on my system and not pip. Can someone explains whats the error about and how can I solve it in using anaconda.
I was running a python code and saved the file as .py.
import pandas as pd
wd = pd.read_csv("winequality-red", sep = ";")
five = wd.head()
print ("five")
Error message:
No kernel for grammar Python found <br>
Check that the language for this file is set in Atom and that you have a Jupyter kernel installed for it.<br>
To detect your current Python install you will need to run:<br>
python -m pip install ipykernel<br>
python -m ipykernel install --user
This isn't really an answer, but you might have better luck on the dedicated Atom forums.
In your case though, it looks like you haven't installed the proper kernels Hydrogen needs to run Python with. (Of course, I'm just assuming you're using Hydrogen. You haven't actually provided any details about how you are trying to run it).
From the Hydrogen documentation, it takes you to this page for Python kernels.
https://nteract.io/kernels/python
In particular, I think you want to run the command conda install ipykernel