I'm trying to use reticulate to run some simple Python code in an RMarkdown document. I've found that if Matplotlib is in the conda environment, I get errors when trying to run a python code chunk, but I can run Python from R directly. Here's a simple example of what I see:
---
title: "Reticulate Test"
date: "9/21/2020"
output: html_document
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(reticulate)
use_condaenv('Toy_MPL') # this environment contain matplotlib and produces the error
#use_condaenv('Toy') # this environment does not contain matplotlib and no error
```
```{r}
# this works regardless of which environment I use
pysys <- import('sys')
pysys$version
```
[1] "3.8.5 (default, Sep 4 2020, 02:23:17) \n[Clang 10.0.0 ]"
```{python, engine.path = '/opt/miniconda2/envs/Toy_MPL/bin/python'}
# if Toy_MPL conda environment is used, the error is generated
# if Toy conda environment is used, I get the same output as above
import sys
print(sys.version)
```
Error in py_call_impl(callable, dots$args, dots$keywords) :
TypeError: use() got an unexpected keyword argument 'warn'
My first thought was that reticulate was not seeing the various system libraries that are installed in the conda environment lib/ folder when Matplotlib is installed - there are a LOT of dependencies that come along with Matplotlib. I tried the following, but none worked:
Set LD_LIBRARY_PATH in .Renviron to point to the correct library path.
Call use_python() in addition to or instead of use_condaenv()
set engine.path in the Python code chunk
I tried downgrading matplotlib to v3.2 (suggested here), but that caused a new set of errors:
Error in if (has_compatible_arch && has_preferred_numpy) valid_python_versions <- c(valid_python_versions, : missing value where TRUE/FALSE needed
Checking NumPy, I see I have v1.19.1 (other errors suggest needing >1.6). And, reinstalling matplotlib v3.3.1 does not prevent the error. After this "fix" I end up having to rebuild the entire environment.
traceback() gives me a CPP stack trace from the reticulate.so which is not interpretable.
My interpretation is that the environment created for RMarkdown does not point to the correct library locations, but I cannot determine how to set it correctly.
System info:
Mac OS Catalina 10.15.6
RStudio v1.3.1073
reticulate v1.16
conda v4.8.4
Python in conda environments v3.8.5
Matplotlib in Toy_MPL environment v3.3.1
In my original question, I referred to this question in which there was a suggestion to downgrade matplotlib to version 3.2.0 because reticulate was not up to date with changes in matplotlib. I followed up further on that suggestion and have found a resolution (for now).
TL;DR
Removing pip and conda installed versions of matplotlib, and then installation of matplotlib version 3.2.2 with conda (NOT pip) resolves the problem. Installing matplotlib with pip leads to other errors.
Details
In the response to the other question, the suggestion was to do:
pip install matplotlib==3.2
I tried this and ended up with other errors that I also could not track down. So, I uninstalled matplotlib and then reinstalled it with
pip install matplotlib==3.3.1
in hopes of getting back to where I was. This also did not work and the new errors persisted. I then removed matplotlib completely and with pip and reinstalled version 3.3.1 with conda:
pip uninstall matplolib
conda install matplotlib
This got me back to matplotlib version 3.3.1 and the original error I mention in my question. I then tried installing matplotlib version 3.2 with conda:
conda install matplotplib==3.2
The installed version is 3.2.2 and not 3.2.0, as suggested in the response, but when I did this, the original problem seems to be resolved.
There is clearly a difference in dependency resolution between pip and conda in this case, and conda provides a version of matplotlib that plays nicely with reticulate. I do not at this point know what the difference is, however.
I ran into a similar issue as well. I'm new to python so I used the Anaconda Navigator to manage my python environments and packages. I fixed my problem by doing the following.
Remove the reticulate package from R
Open the Anaconda Navigator and remove the r-reticulate environment from environments page. I also removed an additional conda environment that I had created with reticulate.
Re install the reticulate package and run your code
This won't help too much if you still want to use matplotlib with reticulate, but it should at least get your script running again if it doesn't use matplotlib.
Related
I want to import MeCab and use it, but that error comes out repeatedly.
MeCab-python is well installed.
I've tried Brew install and so on, and it's still the same.
I'd appreciate it if you could help me if you knew the solution.
Hmm. On a fresh macOS 13.1 Ventura, I just did this:
Installed miniconda https://docs.conda.io/en/latest/miniconda.html#latest-miniconda-installer-links
conda create -n mecab-tutorial and answering questions
conda activate mecab-tutorial
conda install python ipython and saying 'yes'
python -m pip install mecab-python3 unidic-lite
Then I was able to run this script:
import MeCab
wakati = MeCab.Tagger("-Owakati")
wakati.parse("pythonが大好きです").split()
# ['python', 'が', '大好き', 'です']
Looking at the path in the error in the screenshot, it looks like you installed a global Python in /Library which I don't want to try because I very much prefer to keep my Python environments local, hence using Conda above to create a custom environment to install MeCab packages in to ensure no cross-contamination later.
There's no guarantee that this will work if you try it since the error you're seeing seems to be a compiler issue (Python is finding MeCab, just having trouble calling a C++ function inside the binary), but if possible, can you try the Conda approach above?
I need to use the sksparse.chomod package however my pycharm does not let me install it as it can't seem to find it.
I found the sksparse package on github and downloaded it but I do not know how to add a package downloaded from the internet into a conda environment. So, my first question would be can you download a package from github and add it to your conda environment, and how do you do this?
As I did not know how to do the above I instead saved the package within my project and thought I could simply import sksparse.cholmod. However, the line in my code that says import sksparse.cholmod as sks has no errors with it, so I assumed that meant this was ok, but when I try to run my file I get this error:
import sksparse.cholmod as sks
ModuleNotFoundError: No module named 'sksparse.cholmod'
If I have downloaded the package into my project why can't it be found, yet there are no errors when importing?
The cholmod file is a pyx file which I've been told should not be a problem.
Please could anyone help, I am reasonably new to python and I am looking for a straight forward solution that won't be time consuming.
It was an issue with windows, I was able to fix this using the instructions on this link
https://github.com/EmJay276/scikit-sparse
We must follow these steps precisely:
(This was tested with a Anaconda 3 installation and Python 3.7)
Install these requirements in order:
'''
conda install -c conda-forge numpy - tested with v1.19.1
conda install -c anaconda scipy - tested with v1.5.0
conda install -c conda-forge cython - tested with v0.29.21
conda install -c conda-forge suitesparse - tested with v5.4.0
'''
Download Microsoft Build Tools for C++ from https://visualstudio.microsoft.com/de/visual-cpp-build-tools/ (tested with 2019, should work with 2015 or newer)
Install Visual Studio Build Tools
Choose Workloads
Check "C++ Buildtools"
Keep standard settings
Run ''' pip install git+https://github.com/EmJay276/scikit-sparse '''
Test ''' from sksparse.cholmod import cholesky '''
Use all the versions stated for numpy etc, however with scipy I installed the latest version and it worked fine.
I've been trying to install CartoPy recently. I have all the dependencies that CartoPy requires, but when I try to install through the Anaconda Navigator, I'm given this message:
UnsatisfiableError: The following specifications were found to be incompatible with the existing python installation in your
environment:
Specifications:
- cartopy -> python[version='>=2.7,<2.8.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0']
Your python: python=3.8
If python is on the left-most side of the chain, that's the version
you've asked for.
When python appears to the right, that indicates that the thing on the
left is somehow not available for the python version you are constrained to. Note that conda will not change your python version to a different minor version unless you explicitly specify that.
The following specifications were found to be incompatible with your CUDA driver:
- feature:/win-64::__cuda==11.0=0
Your installed CUDA driver is: 11.0
I'm confused with the last part of the error message, because the CUDA drivers are matching versions (11.0). I've googled around, and have seen similar problems, but none that explicitly mention CartoPy, and the responses are a bit over my head.
How do I fix this error? Thanks for the help!
It looks like the main channel for Anaconda doesn't currently have a build of CartoPy for Python 3.8. You'll either need to change to Python 3.7, or install from conda-forge--but I don't know how to add a channel to Anaconda Navigator. So the path I'd take would be:
conda create -n myenv python=3.8 cartopy
conda activate myenv
To create a new environment with cartopy and activate it from the command line.
I am trying to package some R packages from CRAN to use in a conda environment because I am using a combination of Python and R packages for a bioinformatics pipeline. Because of other dependencies, I need to keep R at version 3.3
I made a brandnew environment with the version of Python and R I want:
$ conda create -n bioinfo python=3.6.3 r=3.3.2
There is no R installed in the root environment. Then I follow the instructions for conda skeleton:
(bioinfo)$ conda skeleton cran rootSolve
(bioinfo)$ conda skeleton cran rootSolve
(bioinfo)$ conda build r-rootsolve
For some reason, this keeps coming up with an R3.4 dependency, even though according to CRAN, the rootSolve package only needs R>=2.01! Where is this coming from??
The following NEW packages will be INSTALLED:
r-base: 3.4.2-haf99962_0
Though building the package does not actually change the version of R running in my environment, the package does not load. Any ideas, please?
R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
> library('rootSolve')
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/usr/people/bioc1402/miniconda3/envs/bioinfo2/lib/R/library/rootSolve/libs/rootSolve.so':
/usr/people/bioc1402/miniconda3/envs/bioinfo2/lib/R/library/rootSolve/libs/rootSolve.so: undefined symbol: R_ExternalPtrAddrFn
In addition: Warning message:
package ‘rootSolve’ was built under R version 3.4.2
Error: package or namespace load failed for ‘rootSolve’
Apparently this was a bug, now fixed in conda-build 3.1.3,
https://github.com/conda/conda-build/issues/2562
Thanks conda team!
'conda build r-rootsolve --R=3.3.1' now works appropriately with the recipe generated by conda skeleton.
I have been trying to get the gdal library work using Python 2.7 and Anaconda in Windows 8 environment.
Besides gdal, I have also installed libgdal (frankly, I don't really understand the difference between the two). I now seem to have gdal 2.1.0 and 2.0.2 as well as libgdal 2.1.0.
However, when I run my Py code, there is a gdal error:
'gdalwarp' is not recognized as an internal or external command,
operable program or batch file.
I have already set the GDAL_DATA environmental variable to point to
C:\Anaconda\pkgs\libgdal-2.1.0-vc9_0\Library\share\gdal
I have also added a path, although I am not entirely sure where this should point to:
C:\Anaconda\pkgs\libgdal-2.1.0-vc9_0\Library\bin
I have tried the same with gdal 2.0.2 without success. gdalwarp.exe does seem to exist under libgdal 2.1.0 and gdal 2.0.2.
Any ideas? Is there an issue with the installation or have I not set the environmental variables correctly?
FYI, I have tried various installation commands, notably:
conda install gdal
conda install -c conda-forge gdal
conda install -c anaconda gdal
Addendum: I have found a manual solution: I set the GDAL_DATA and PATH variables in the terminal (pointing to libgdal 2.1.0) before running the code...
However, there is still an issue when I run my Py code: it is supposed to convert a tiff file to shp with gdal_polygonize:
cmd = 'gdal_polygonize.py %s -f "ESRI Shapefile" %s'%(dst_tif, dst_shp)
There is no error but the shapefile is not created (which leads to an error later on in the code). Any ideas as to why gdal is still not working correctly?
I have tried pointing the env variables to osgeo:
set PATH=%PATH%;C:\Anaconda2\Lib\site-packages\osgeo\scripts
set GDAL_DATA=C:\Anaconda2\Lib\site-packages\osgeo\data\gdal
Gdalinfo works but the gdal_polygonize used in my Py code does not appear to work.
The key is the activation script which is (potentially) executed when activating the environment. Not every GDAL build for Conda contains this. In my experience recent Conda-Forge builds are really good.
With your requirements of py27 and GDAL 2.1 i can get it working by following these steps:
1) Create a new environment: conda create -n gdaltest python=2.7
2) Activate: activate gdaltest
3) Install GDAL: conda install gdal=2.1 -c conda-forge
4) Reactivate environment: deactivate + activate gdaltest
This forces the just installed activation script to be executed, this sets the environment variables.
If i start python and run os.system("gdalinfo"), i can see its picked up correctly. And running os.environ['GDAL_DATA'] confirms the path is set correctly.
You can view the (de)activation script yourself at:
C:\Miniconda3\envs\<env name>\etc\conda\activate.d\gdal-activate.bat
A few years ago this didn't work as well as it does today, so make sure you have a recent Conda version (4.3.x) etc.
The benefit of this method is, that when switching environments, your paths are also changed accordingly. A "hard coded" GDAL_DATA path could potentially cause some compatibility issues if you mix and match different GDAL versions (although normally is should work OK).