I am new to Julia. I developed a few lines of code to get the results I needed from packages I was not able to find in Python or R. Now, I am trying to get this file to be easily accessible, and wrap the code in Python or R. Has anyone done this before? I have tried a few methods and have not found anything that has helped.
The most simple way to do this would be just a few lines of code that calls the .jl file, runs it (which the code is then added to a .txt file from julia), and then alerts you when the code is done.
Any help would be greatly appreciated. R is the preferable method and at this point Python would be appreciated as well.
Please find below instructions for Python, R and just an external process (which of course is an executable command that can be run from any other process). I recommend putting your code in a package and loading it in one of those languages rather than executing this as an external process.
Python
Use Python Anaconda (not in-built system Python) and install Julia
Run Julia and install PyCall
using Pkg
ENV["PYTHON"]="/path/to/your/python/executable"
pkg"add PyCall"
pkg"build PyCall"
Put your code into a Julia package
using Pkg
Pkg.generate("MyPackage")
In the folder src you will find MyPackage.jl, edit it to look like this:
module MyPackage
function main(x,y)
#do very complex staff or place it in your_other_file.jl
2x.+y
end
include("your_other_file.jl")
export main, and_whatever_other_functio_you_defined
end
Install pyjulia
python -m pip install julia
(On Linux systems you might want to use python3 instead of python command)
For this step note that while an external Python can be used with Julia. However, for a convenience it might be worth
to consider using a Python that got installed together with Julia as PyCall.
In that case for installation use a command such this one:
%HOMEPATH%\.julia\conda\3\python -m pip install julia
or on Linux
~/.julia/conda/3/python -m pip install julia
Note that if you have JULIA_DEPOT_PATH variable defined you can replace %HOMEPATH%\.julia or ~/.julia/ with its value.
Run the appropiate Python and tell it to configure the Python-Julia integration:
import julia
julia.install()
Now you are ready to call your Julia code:
>>> from julia import Pkg
>>> Pkg.activate(".\\MyPackage") #use the correct path
Activating environment at `MyPackage\Project.toml`
>>> from julia import MyPackage
>>> MyPackage.main([1,2],5)
[7,9]
Gnu R
Configure your system PATH variable to point to your Julia location. Hence when you type julia in the console it should start Julia
Run the script below to install R-Julia integration
install.packages("JuliaCall")
library(JuliaCall)
julia <- julia_setup()
Follow the above instructions for Python (step 3 only) and create the package named MyPackage
Run the code
library(JuliaCall)
julia_eval("using Pkg;Pkg.activate(\"C:/temp/rrr/MyPackage\")")
julia_library("MyPackage")
julia_eval("MyPackage.main(3,5)")
Bash (or just any language)
Build the package following instructions for Python (step 3 only)
Configure the system PATH variable
Being in the package directory run the command (note string(:.) is a Julian trick that I use to avoid apostrophe escaping in bash commands):
julia -e "using Pkg;Pkg.activate(string(:.));Pkg.instantiate();using MyPackage;MyPackage.main(3,4)"
This will install all dependencies for your package. In order to skip the installation remove Pkg.instantiate() from the above command.
The answer from #przemyslaw-szufel is correct but maybe a bit overcomplicated. You don't necessarily need to wrap your code in a module or define a custom environment (yes it is good practice, but a step at the time...)
First create a file juliaScripts.jl with content:
function getAnElement(array,n)
return array[n]
end
To run functions defined in a .jl file in R
Then in R you just do:
> install.packages("JuliaCall")
> library(JuliaCall)
> julia_setup() # on every new R session !
> julia_source("juliaScript.jl")
> out <- julia_call("getAnElement",c(10,20,30),2)
> out
[1] 20
Note that the R vector has been automatically converted to a Julia Array.
To run functions defined in a .jl file in Python
In Python, it is even easier:
$ python3 -m pip install --user julia
>>> import julia
>>> julia.install() # only once, not every session
>>> jl=julia.Julia(compiled_modules=False)
>>> from julia import Main
>>> Main.include("juliaScript.jl")
>>> Main.getAnElement([1,2,3],2)
20
Also in Python arrays (native python lists as well Numpy arrays and other commonly used data structures) are automatically converted between Python and Julia.
Not to make advertising, but more details on interfacing R <-> Julia or Python <-> Julia are on my Apress(2019) book "Julia Quick Syntax reference" in Chapter 7 "Interfacing Julia with other languages" (I shouldn't say it, but you can easily find the pdf online in well-known sites...)
Using the JuliaConnectoR package:
library(JuliaConnectoR)
fun <- juliaCall("include", "/path/to/file.jl") # you may need to provide the full path
For more info on JuliaConnectoR, see the link above as well as this paper, which additionally compares it to alternative packages suck as JuliaCall and XRJulia.
Related
Iam new bee to the jmeter
My code is working in the Python 2.7 with importing additional packages Dateutil, parser .
Problme : But when I am trying to run same code in the J Meter-JSR-223 PreProcessors , an error saying No module named dateutil in.
So , I have tried another approach to use Jython .
Installed the Jython ( downloaded the dateutil) and provide the packages reference under
import sys
sys.path.append('C:/Jython27/Lib/site-packages')
sys.path.append('C:/Jython27/Lib/site-packages/python_dateutil-2.4.2-py2.7/dateutil')
sys.path.append('C:/Jython27/Lib/site-packages/python_dateutil-2.4.2-py2.7/dateutil')
Now packages error is gone but string syntax error is present .
java.sql.Date' object has no attribute .
I believe dateutil package can be picked up from CPython as it doesn't require any extra wrappers for Java.
Install dateutil normally using pip like:
pip install python-dateutil
Add site-packages folder of Python (not Jython) installation to sys.path like:
sys.path.append("C:\Python27\Lib\site-packages")
That's it, now you should be able to use dateutil module functions from the JSR223 Test Elements:
Be aware that invoking Python scripts via Jython interpreter is not the best idea from performance perspective and if you're about to invoke your Python code only limited number of times and/or with a single thread - it might be better to go for the OS Process Sampler.
If you plan to use the Python code to create the main load - consider using Locust tool instead of JMeter. If you don't want to change JMeter a good approach would be rewriting your Python code in Groovy - it will be way better from the performance perspective.
hi please find follwing
import sys
sys.path.append('C:/Python27/Lib/site-packages')
sys.path.append('C:/Python27/Lib/site-packages/python_dateutil-2.4.2-py2.7/dateutil')
from dateutil.parser import *
sourceDateTimeOfEvent = ""
dateTimeOfEvent = ""
a=parse('2016-07-01 13:00:00')
sourceDateTimeOfEvent = a.isoformat()+"+05:30Z"
dateTimeOfEvent = a.isoformat()+ "Z"
vars.put("sourceDateTimeOfEvent", sourceDateTimeOfEvent)
vars.put("dateTimeOfEvent", dateTimeOfEvent)
This sourceDateTimeOfEvent and dateTimeOfEvent considered as two variables and passed it to the json file
Is it possible to run MATLAB functions from within Python?
I search the internet, I could only find PyMat. The bad thing is the compiled version only supports Python2.2 and I am using 2.6. So I tried to download the source code, so I can compile it for myself. But I cannot compile it, VC++ express seems not to have the necessary functionalities to compile it. Does anyone have the compile version for PC?
or any substitutes for PyMat?
Thanks
I know this is an old question and has been answered. But I was looking for the same thing (for the Mac) and found that there are quite a few options with different methods of interacting with matlab and different levels of maturity. Here's what I found:
pymat
A low level interface to Matlab using the matlab engine (libeng) for communication (basically a library that comes with matlab). The module has to be compiled and linked with libeng.
http://pymat.sourceforge.net
Last updated: 2003
pymat2
A somewhat short lived continuation of the pymat development. Seems to work on windows (including 64bit), linux and mac (with some changes).
https://code.google.com/p/pymat2/
Last updated: 2012
mlabwrap
A high level interface that also comes as a module which needs compilation and linking against libeng. It exposes Matlab functions to python so you can do fun stuff like
mlab.plot(x, y, 'o')
http://mlabwrap.sourceforge.net
Last updated: 2009
mlab
A repackaging effort of mlabwrap. Basically it replaces the c++ code that links against 'libeng' in mlabwrap with a python module (matlabpipe) that communicates with matlab through a pipe. The main advantage of this is that it doesn't need compilation of any kind.
Unfortunately the package currently has a couple of bugs and doesn't seem to work on the mac at all. I reported a few of them but gave up eventually. Also, be prepared for lots of trickery and a bunch of pretty ugly hacks if you have to go into the source code ;-) If this becomes more mature it could be one of the best options.
https://github.com/ewiger/mlab
last update: 2013
pymatlab
A newer package (2010) that also interacts with Matlab through libeng. Unlike the other packages this one loads the engine library through ctypes thus no compilation required. Its not without flaws but still being maintained and the (64bit Mac specific) issues I found should be easy enough to fix.
(edit 2014-05-20: it seems those issues have already been fixed in the source so things should be fine with 0.2.4)
http://pymatlab.sourceforge.net
last update: 2014
python-matlab-bridge
Also a newer package that is still actively maintained. Communicates with Matlab through some sort of socket. Unfortunately the exposed functions are a bit limited. I couldn't figure out how to invoke a function that takes structs as parameters. Requires zmq, pyzmq and IPython which are easy enough to install.
http://arokem.github.io/python-matlab-bridge
last update: 2014
Another option is Mlabwrap:
Mlabwrap is a high-level python to Matlab® bridge that lets Matlab look like a normal python library.
It works well with numpy arrays. An example from the home page:
>>> from mlabwrap import mlab; from numpy import *
>>> xx = arange(-2*pi, 2*pi, 0.2)
>>> mlab.surf(subtract.outer(sin(xx),cos(xx)))
PyMat looks like it's been abandoned.
I'm assuming you are on windows so you could always do the simplest approach and use Matlab's COM interface:
>>> import win32com.client
>>> h = win32com.client.Dispatch('matlab.application')
>>> h.Execute ("plot([0 18], [7 23])")
>>> h.Execute ("1+1")
u'\nans =\n\n 2\n\n'
More info here
There is a python-matlab bridge which is unique in the sense that Matlab runs in the background so you don't have the startup cost each time you call a Matlab function.
https://github.com/jaderberg/python-matlab-bridge
it's as easy as downloading and the following code:
from pymatbridge import Matlab
mlab = Matlab(matlab='/Applications/MATLAB_R2011a.app/bin/matlab')
mlab.start()
res = mlab.run('path/to/yourfunc.m', {'arg1': 3, 'arg2': 5})
print res['result']
where the contents of yourfunc.m would be something like this:
%% MATLAB
function lol = yourfunc(args)
arg1 = args.arg1;
arg2 = args.arg2;
lol = arg1 + arg2;
end
see this page: An Open-Source MATLAB®-to-Python® Compiler
I would like to add one more option to the excellent summary by Lukas:
matlab_wrapper
The advantage of matlab_wrapper is that it is pure Python library and you will not need to compile anything. Works in GNU/Linux, Windows and OSX.
https://github.com/mrkrd/matlab_wrapper
Disclaimer: I'm the author of matlab_wrapper
You can use the official matlab engine by installing Matlab, then building python engine from its extern files. You can check the guide website below:
---Thanks for the advice in the first comment of this answer ---
the essential step in brief are (On Windows platform, other can checked in the url below):
1. download and then install matlab, the version must be R2014 or later.
2. open a PowerShell window under admin, then:
cd "matlabroot\extern\engines\python"
3. use command-line below to install:
python setup.py install
The admin is essential, or you'll fail to build it.
For more information, you can click the official start sheet below:
http://cn.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html
newer versions of matlab seem to provide a module that allows you to call matlab functions from within python. see here and here.
2 more options for you to consider:
Follow the official MATLAB docs:
Create a Python Application with MATLAB Code. This will create a Python library that includes MATLAB runtime which you can call from within your Python code.
Run your MATLAB code in GNU Octave then call it from Python using Oct2Py
This is the solution from Mathworks.
In your current folder, create a MATLAB script in a file named triarea.m.
function a = triarea(b,h)
a = 0.5*(b.* h);
Meanwhile, you run the python code as follows,
import matlab.engine
eng = matlab.engine.start_matlab()
eng.addpath('your/code/folders/')
ret = eng.triarea(1.0,5.0)
print(ret)
>>> 2.5
Matlab already provides the python module of the engine, to install that you can do the following,
cd matlab_root_folder/extern/engines/python
python setup.py install
You are all done!
Tips: you need to be careful about the data type, the engine is not friendly with numpy. You need to convert the data first.
mat_array = matlab.double(list(my_numpy_array))
eng.my_matlabe_function(mat_array )
I just installed ycm, everything looks good, but I found small problem. The problem is as following:
import os # os is built-in library
os. # ycm helps to complete members of the class.
import numpy # numpy is not built-in library, where its location is site-packages.
numpy. # nothing happened. ycm shows 'pattern not found' message.
I think, this would be a simple problem. But I could not find the solution yet. I think, there is some configuration file in which I can define 'search path' for my project.
It would be grateful if I can find a way to solve it.
Best,
Je-Hoon Song
I had the same issue with module 'mpmath' and fixed it in the following manner:
First I retrieved the path where the module was located:
%python3
>>>import mpmath
>>>print(mpmath.__file__)
/usr/lib/python3.4/site-packages/mpmath/__init__.py
Here I found the path of all my "installed" python3 packages to be:
/usr/lib/python3.4/site-packages/
I then simply added to my PYTHONPATH environment variable this path:
%export PYTHONPATH=/usr/lib/python3.4/site-packages/
Then when I used vim sample.py typing import mpmath and following it up with mpmath. YCM showed me all the autocompletions for the mpmath module.
Hope this helps.
I use anaconda python to be my python interpreter in ycm to solve this.
First I modified my vimrc according to full pythong setting in vim.
Then I change g:ycm_python_interpreter_path by
let g:ycm_python_interpreter_path = '/usr/local/anaconda3/bin/python3.8'
In this way I didn't change the system environment variables.
Addtional Info 1:
I think the main problem is that,
my Python interpreter for YCM is my system python (/usr/local/opt/python#3.9/bin/python3.9 ),
which only has limited locally built libraries.
So using the libraries comes with anaconda (/usr/local/anaconda3/bin/python3.8 ) can solve.
Additional Info 2:
By reading :YcmDebugInfo, the main different after edited g:ycm_python_interpreter_path is that:
-- Python completer debug information:
-- Python interpreter: /usr/local/opt/python#3.9/bin/python3.9
-- Python path: ['/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python39.zip', '/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9', '/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload', '/usr/local/lib/python3.9/site-packages']
-- Python version: 3.9.6
change to
-- Python completer debug information:
-- Python interpreter: /usr/local/anaconda3/bin/python3.8
-- Python path: ['/usr/local/anaconda3/lib/python38.zip', '/usr/local/anaconda3/lib/python3.8', '/usr/local/anaconda3/lib/python3.8/lib-dynload', '/usr/local/anaconda3/lib/python3.8/site-packages', '/usr/local/anaconda3/lib/python3.8/site-packages/aeosa', '/usr/local/anaconda3/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg']
-- Python version: 3.8.8
Additional Info 3: about how to read list of locally installed pyton modules
use https://stackoverflow.com/a/740018/11226687
e.g. in my case
$ /usr/local/opt/python#3.9/bin/python3.9
>>> help('modules')
# only return limitted modules
$ /usr/local/anaconda3/bin/python3
>>> help('modules')
# list out all the modules included in Anaconda, including numpy/matplotlib/scipy ect
numpy is kind of a difficult library because it dynamically builds its namespace on import, making it hard for static code analysis tools to know when you're write the code what names should be available. Because the names available in the namespace numpy are only really known at runtime, YCM probably doesn't have any useful suggestions for you.
One simple way to fix is activate your python environment, then open vim. For example
(django_mdn) ➜ locallibrary git:(master) ✗ vim
and in the vim run :echo $PATH.
Then you should be able to see that your venv path is at the first like this:
/Users/gwanghyeongim/.virtualenvs/django_mdn/bin:/usr/local/opt/tcl-tk/bin:...
Then see if your python packages are auto-complete.
It worked.
If you want to set a certain site-packages to be auto complete permanently, you need to make a file called .ycm_ extra_conf.py in your project root directory or global_extra_conf.py and set vim configuration if you want to set it globally.
P.S.
By running export PYTHONPATH=/usr/lib/python3.4/site-packages/ in the shell before opening vim didn't work for me. Besides, unless setting PYTHONPATH permanently, which will cause issue, you will have to set export PYTHONPATH everytime you want dependencies to be auto complete.
I'm writing a simple IronWorker in Python to do some work with the AWS API.
To do so I want to use the boto library which is distributed via PyPi repository. The boto library is not installed by default in the IronWorker runtime environment.
How can I bundle the boto library dependancy with my IronWorker code?
Ideally I'm hoping I can use something like the gem dependancy bundling available for Ruby IronWorkers - i.e in myRuby.worker specify
gemfile '../Gemfile', 'common', 'worker' # merges gems from common and worker groups
In the Python Loggly sample, I see that the hoover library is used:
#here we have to include hoover library with worker.
hoover_dir = os.path.dirname(hoover.__file__)
shutil.copytree(hoover_dir, worker_dir + '/loggly') #copy it to worker directory
However, I can't see where/how you specify which hoover library version you want, or where to download it from.
What is the official/correct way to use 3rd party libraries in Python IronWorkers?
Newer iron_worker version has native support of pip command.
So, you need:
runtime "python"
exec "something.py"
pip "boto"
pip "someotherpip"
full_remote_build true
[edit]We've worked on our toolset a bit since this answer was written and accepted. The answer from my colleague below is the recommended course moving forward.[/edit]
I wrote the Python client library for IronWorker. I'm also employed by Iron.io.
If you're using the Python client library, the easiest (and recommended) way to do this is to just copy over the library's installed folder, and include it when uploading the package. That's what the Python Loggly sample is doing above. As you said, that doesn't specify a version or where to download the library from, because it doesn't care. It just takes the one installed on your system and uses it. Whatever you get when you enter "import boto" on your local machine is what would be uploaded.
The other option is using our CLI to upload your worker, with a .worker file.
To do this, here's what you'd need to do:
Create a botoworker.worker file:
runtime "binary"
build 'pip install --install-option="--prefix=`pwd`/pips" boto'
file 'botoworker.py'
exec "botoworker.sh"
That second line is the pip command that will be run to install the dependency. You can modify it like you would any pip command run from the command line. It's going to execute that command on the worker during the "build" phase, so it's only executed once instead of every time you run a task.
The third line should be changed to the Python file you want to run--it's your Python worker file. Here's the one we used to test this:
import boto
If you save that as botoworker.py, the above should work without any modification. :)
The fourth line is a shell script that's going to actually run your worker. I've included the one we used below. Just save it as botoworker.sh, and you won't have to worry about modifying the .worker file above.
PYTHONPATH="$HOME/pips/lib/python2.7/site-packages:$PYTHONPATH" python botoworker.py "$#"
You'll notice it refers to your Python file--if you don't name your Python file botoworker.py, remember to change it here, too. All this does is set your PYTHONPATH to include the installed library, and then runs your Python file.
To upload this, just make sure you have the CLI installed (gem install iron_worker_ng, making sure your Ruby version is 1.9.3 or higher) and then run "iron_worker upload botoworker" in your shell, from the same directory your botoworker.worker file is in.
Hope this helps!
For context: My goal is to create a graphic interface so that the user can run a program that I have been developing in R. The interface is done using the Tkinter module from python (version 3.3). Right now I am trying to make it work in Windows.
I believe I have successfully called the R interpreter using python to run my R file (run.R). In this file I call the R package 'seqinr'. However when I call the R interpreter from python I obtain this:
I run this:
os.system('C:/"Program Files"/R/R-3.6.1/bin/Rscript run.R')
and I obtain this:
Error in library(seqinr) : there is no package called 'seqinr'
Calls: source -> withVisible -> eval -> eval -> library
Execution halted
However, when I run the 'C:/"Program Files"/R/R-3.6.1/bin/Rscript run.R' command in the Command Prompt, it works perfectly and I have no problems.
I also checked where my package is installed and it's in the default directory that R uses to install all the packages. I also do not have any other R versions installed in my machine.
I have no clue what is happening now, so I would appreciate any help.
Thank you!
Although probably not satisfactory, I'd recommend changing your "run.R" file to include the following lines:
if (!("seqinr" %in% installed.packages())) {
install.packages('seqinr', repos = "https://cloud.r-project.org")
}
library(seqinr)
This will install the package if it isn't available to wherever R is looking for packages when being called from Python. Additionally, you could include print(rownames(installed.packages())) in your "run.R" file to see what packages are currently available to your R installation at run time when calling from Python. Inclusion of the line print(.libPaths()) will tell you where R is looking for installed packages at run time as well.