We are building a User Interface APP (predicting a continuous variable through a machine learning model) through R Shiny.
Since we built the machine learning model in Python3 sklearn module, we hope that we could write python codes in R Shiny to call that model and corresponding functions.
We used R-package "reticulate" to create virtual python environment where it would save python packages, and through which we could call python3 functions.
We created the virtual environment using the following line of code (the function in R package "reticulate")
use_virtualenv("env", required = TRUE)
Where we indeed have the following directory "env/bin" in which there are python and python3 to execute.
The Shiny APP worked perfectly locally. HOWEVER, when we made attempts to publish, it gave the following error (please see picture) (after the APP was successfully deployed and on shinyapps.io, it said the APP was running).
The issue was "Error 126", which denied the permission for our APP to access the virtual environment. This issue had no previous (similar) case on Stackoverflow, and therefore we spent a long time to debug (issue not resolved).
If anyone knows how to solve this problem, would it be possible for you to kindly mark your solution tips below? (We hope your solution would not modify our basic layout, i.e. "calling python-made model in Shiny and publish through Shiny") We really appreciate your efforts to help us out!
Thank you so much!
Could you share the code where actual call to python script is being made? is it a python module function that you are calling from Rshiny? what does the python module/function do and return? I have used reticulate inside shiny to call Python scripts and it works fine. Didn't require to set the environment. Just provide the source to python script and call it like any other R function.
If you're trying to deploy to shinyapps.io, you may need to set the RETICULATE_PYTHON env variable so that reticulate uses the correct version of Python when running your app:
VIRTUALENV_NAME = 'env'
Sys.setenv(RETICULATE_PYTHON = paste0('/home/shiny/.virtualenvs/',
VIRTUALENV_NAME,
'/bin/python'))
Full example here demonstrates one method for configuring a Shiny + reticulate app so that it can easily run both locally and on shinyapps.io.
Related
all.
I recently started working with Jenkins, in an attempt to replace cronjob with Jenkins pipeline. I have really a bit knowledge of programming jargon. I learned what I learned from questions on stackoverflow. So, if you guys need any more info, I would really appreciate if you use plain English.
So, I installed the lastest version of Jenkins and suggested plugins plus all the plugins that I could find useful to python running.
Afterwards, I searched stackoverflow and other websites to make this work, but all I could do was
#!/usr/bin/env python
from __future__ import print_function
print('Hello World')
And it succeeded.
Currently, Jenkins is running on Ubuntu 16.04, and I am using anaconda3's python (~/anaconda3/bin/python).
When I tried to run a bit more complicated python code (by that I mean import pandas), it gives me import error.
What I have tried so far is
execute python script build: import pandas - import error
execute shell build: import pandas (import pandas added to the code that worked above)
python builder build: import pandas - invalid interpreter error
pipeline job: sh python /path_to_python_file/*.py - import error
All gave errors. Since 'hello world' works, I believe that using anaconda3's python is not an issue. Also, it imported print_function just fine, so I want to know what I should do from here. Change workspace setting? workdirectory setting? code changes?
Thanks.
Since 'hello world' works, I believe that using anaconda3's python is not an issue.
Your assumption is wrong.
There are multiple ways of solving the issue but they all come down to using the correct python interpreter with installed pandas. Usually in ubuntu you'll have at least two interpreters. One for python 2 and one for python 3 and you'll use them in shell by calling either python pth/to/myScript.py or python3 pth/to/myScript.py. python and python3 are in this case just a sort of labels which point to the correct executables, using environmental variable PATH.
By installing anaconda3 you are adding one more interpreter with pandas and plenty of other preinstalled packages. If you want to use it, you need to tell somehow your shell or Jenkins about it. If import pandas gives you an error then you're probably using a different interpreter or a different python environment (but this is out of scope here).
Coming back to your script
Following this stack overflow answer, you'll see that all the line #!/usr/bin/env python does, is to make sure that you're using the first python interpreter on your Ubuntu's environment path. Which almost for sure isn't the one you installed with anaconda3. Most likely it will be the default python 2 distributed with ubuntu. If you want to make sure which interpreter exactly is running your script, instead of 'Hello World' put inside:
#!/usr/bin/env python
import sys
print(sys.executable) # this line will give you the exact path to the interpreter
print(sys.version) # this one will give you the version
Ok, so what to do?
Well, run your script using the correct interpreter. Remove #!/usr/bin/env python from your file and if you have a pipeline, add there:
sh "/home/yourname/anaconda3/bin/python /path_to_python_file/myFile.py"
It will most likely solve the issue. It's also quite flexible in the sense that if you ever want to use this python file on a different machine, you won't have your username hardcoded inside.
I am a big fan of Rstudio Cloud and would like to inter-grate R and Python by using the package Reticulate.
It looks like Rstudio Cloud is using python 2.7 (no problems with that). When I try to write Python Code in an R markdown document, nothing gets run.
---
title: "reticulate"
output: html_document
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```
```{r}
library(reticulate)
py_config()
```
```{python}
import pandas
x = 4
```
Python code does not get run.
I am also finding that if I want to install python packages in an R script using reticulate. I have to create a virtual environment. What is the reason behind that?
library(reticulate)
virtualenv_create("r-reticulate")
virtualenv_install("r-reticulate", "scipy")
virtualenv_install("r-reticulate", "pandas")
If I use conda_install, I get an error message.
conda_create("r-reticulate")
Error: Unable to find conda binary. Is Anaconda installed?
conda_install("r-reticulate", "scipy")
Error: Unable to find conda binary. Is Anaconda installed?
The goal is to have python working in Rstudio cloud on R markdown. I can not install packages and execute code.
I just succeeded in getting Conda installed in Rstudio cloud after receiving the same error message as you1, so thought I'd share how I got this working.
I created two scripts:
to install miniconda (i think that's the step you're missing, and why Conda didn't work for you) and then restartSession for this to be accessible
to seperately store the commands for setting up Conda, with the running of this script passed as a command to the call to restartSession (because otherwise the commands are triggered before R has restarted, and they fail; sys.sleep() didn't seem to work, but this method did)
setup.R
setwd("/cloud/project") # to ensure students get required resources
install.packages("rstudioapi") # to restart R session w/ installations
install.packages("reticulate") # for python
reticulate::install_miniconda("miniconda") # for python
# Restart again to make sure all system things are loaded
# and then create a new Conda environment
rstudioapi::restartSession(command="source('nested_reticulate_setup.R')")
nested_reticulate_setup.R
reticulate::conda_create("r-reticulate")
reticulate::conda_install("r-reticulate", "scipy")
Sys.setenv(RETICULATE_PYTHON="/cloud/project/miniconda/envs/r-reticulate/bin/python")
reticulate::use_condaenv("r-reticulate")
osmnx <- reticulate::import("scipy")
Then if you make a call to scipy, eg scicpy$`__version__` , I believe it should work for you without that error you observed.
I couldn't find a solution to this issue elsewhere, so thought it worth responding to this old post in case it helps somebody some day. I am sure there are other ways of approaching this.
1 Perhaps for a different reason; i'll explain later in the post...
I am new in Python and Basic. I am trying to replicate the IMPORTHTML function from Google Sheets in LibreOffice (LO) Calc. In a nutshell, I want to create a GetHtmTable( Url, Table Index) Basic function in Calc which will call a Python script to do the heavy work.
So based on Villeroy's great example, I implemented in LO 5.1.6.2. the Basic SOUNDEX function which calls the Python script sheetFunctions.py to get familiar with the process. My environment is Linux Mint 18, I use Python 3, I imported all kind of libraries such as Uno, PIP etc.. I use PycharmProjects as a Python editor.
I see clearly under the LO Calc menu tools->macro->organize macros->python the sheetFunctions.py Python script, which indeed is in the folder /usr/lib/libreoffice/share/Scripts/python.
Whenever, I run the SOUNDEX Basic function I see the following error message:
BASIC runtime error. An exception occurred Type:
com.sun.star.script.provider.ScriptFrameworkErrorException Message:
: an
error occurred during file opening
/usr/lib/libreoffice/program/pythonscript.py:429 in function
getModuleByUrl() [lastRead = self.sfa.getDateTimeModified( url )]
/usr/lib/libreoffice/program/pythonscript.py:993 in function
getScript() [mod = self.provCtx.getModuleByUrl( fileUri )]
I tried to debug the SOUNDEX basic function and found out the blocking point is when the program runs getScript("vnd.sun.star.script:sheetFunctions.py$soundex?language=Python&location=user").
I've been trying for days now to overcome this error, unsuccessfully I must confess.
I wonder if I need to bring some extra extensions in the Basic environment or a missing add-in in the Linux/Python one?
I changed the location=user by location=document and got stuck again. I added recently libreoffice-script-provider-python thanks to the command sudo apt-get install libreoffice-script-provider-python but this did not help. I also embedded in the Calc document the Python script but same this did not solve the issue.
The location name does not match. The standard place for self-written scripts is under the user directory. This is location=user, for example ~/.config/libreoffice/4/user/Scripts/python.
Then there is location=share, which refers to the path in your question. These parameters are described under Python Script in the URI Specification.
See also my answer to this question. Be sure to try the APSO extension if you haven't yet. Especially, APSO helps when using location=document, because embedding requires several steps including editing manifest.xml.
I am working one file python project.
I integrated google-cloud-API for realtime speech streaming and recognition.
It works with python aaa.py command well.
Now I need windows build file(.exe), so I used pyinstaller program and I got aaa.exe file successfully.
But I got this error while running speech streaming by using Google cloud API.
[Errno 2] No such file or directory:
'D:\AI\ai\dist\AAA\google\cloud\gapic\speech\v1\speech_client_config.json'
So I copied this speech_client_config.json file in needed path, after that I got below error again.
Exception in 'grpc._cython.cygrpc.ssl_roots_override_callback'
ignored E0511 01:13:14.320000000 3108
src/core/lib/security/security_connector/security _connector.cc:1170]
assertion failed: pem_root_certs != nullptr
Then, I can not find solution to get working version with google-cloud API.
I am using python version 2.7.14
I need your friendly help.
Thanks.
I had the same problem. If you are willing to distribute roots.pem with your executable (just search for the file - it should be buried deep within the installation directory of grpcio), I had luck fixing this by setting GRPC_DEFAULT_SSL_ROOTS_FILE_PATH environment variable to the full path of this roots.pem file.
Update 2021
To anyone who is experiencing this issue. I got it working thanks to these amazing people. See the full conversation on this github issue.
Here is the link
Step 1
Credits to #cbenhagen & #rising-stark on this github link.
A PyInstaller hook called hook-grpc.py looking like this would do the trick:
Create a python file named hook-grpc.py with this code.
from PyInstaller.utils.hooks import collect_data_files
datas = collect_data_files('grpc')
Step 2
Put the hook-grpc.py file in your \site-packages\PyInstaller\hooks directory of the python environment you are running on. So basically you can find it at
C:\Users\yourusername\AppData\Local\Programs\Python\Python37\Lib\site-packages\PyInstaller\hooks
Note:
Just change the yourusername and Python37 to your
respective username and python version you are using.
For Anaconda users it might be different. Check this site
to find the anaconda python environment path you are using.
Step 3
Once you've done that you can now convert your .py python program to .exe using pyinstaller and it should work.
This looks to me like a SSL credentials mistake. I think you are not being allowed to GC. Check this code snippet and this documentation.
I am currently working on a project to embed a Python interpreter in my website--I would like users to be able to execute their own Python code on the server-side in a secure Python sandbox. I have been using PyPy's sandboxing features and have set my virtual tmp/ directory (which corresponds to the real directory that the sandbox can read from) to my server-side files. So ideally, users will be able to run their own code on the server and use whatever libraries I make available to them (with no ability to write to any of these files/make system calls/do anything in general that would mess up my files).
The issue is that within my server-side code I use standard Python libraries as well as others I have installed separately (e.g. "import datetime", "import pandas", etc.). While the sandbox can read my server-side code it ends up failing whenever I try to import additional libraries within this code. I have thought of setting my virtual tmp/ directory to include my server-side files + the standard Python libraries + separately installed libraries such as PANDAS. However, when I tried setting it to just the Python libraries it ended up not working, not to mention that this solution seems pretty inelegant.
Any thoughts on how to go about fixing this? I appreciate whatever advice anyone may have. Thanks!
EDIT: I think part of the answer lies in the following portion of pypy_interact.py (the full source code can be found here: https://github.com/ojii/sandlib/blob/master/sandlib/pypy_interact.py):
def build_virtual_root(self):
# build a virtual file system:
# * can access its own executable
# * can access the pure Python libraries
# * can access the temporary usession directory as /tmp
exclude = ['.pyc', '.pyo']
if self.tmpdir is None:
tmpdirnode = Dir({})
else:
tmpdirnode = RealDir(self.tmpdir, exclude=exclude)
libroot = str(LIB_ROOT)
return Dir({
'bin': Dir({
'pypy-c': RealFile(self.executable),
'lib-python': RealDir(os.path.join(libroot, 'lib-python'),
exclude=exclude),
'lib_pypy': RealDir(os.path.join(libroot, 'lib_pypy'),
exclude=exclude),
}),
'tmp': tmpdirnode,
})
I am relatively new to Python and can't seem to figure out how to use this method properly. Any suggestions would be greatly appreciated.