I have a platform for python scripting, and I would like to call matlab functions inside. I have found several threads tackling the issue, among them those two
How do I interact with MATLAB from Python?
Running m-files from Python
However, threads are either not recent, or not very detailed.
is mlabwrap reliable?
what would you advocate as a solution to call matlab functions / .m files in python script?
using win32com from python to call a matlab session --> is this a good idea ? could you point to more doc or examples on this topic?
looks like link to sourceForge is not up to date, last update 2010,
http://sourceforge.net/projects/mlabwrap/
Could you hint at some latest version ?
Thanks
I would still recommend mlabwrap as the solution for this.
I use mlabwrap on a regular (weekly?) basis, on both Linux and Windows, across several different versions of Python, and several different versions of Matlab. To answer your specific questions:
mlabwrap will reliably perform across platforms, Python, and Matlab versions. It does however have limitations, and it will reliably fail when pushed past those limitations. Usually, these can be worked around.
See my answer here for more information on calling Matlab functions vs. Matlab scripts through mlabwrap. This answer also describes how to workaround one of the primary limitations of mlabwrap, which is that not all Matlab objects can be directly converted into Python objects.
I don't know anything about calling Matlab using win32com.
I have used mlabwrap in what I'll term 'Python-primary' style, where the majority of the programming in Python, using Matlab as a library for specific mathematical functions that aren't available in scipy/numpy, and in a 'Matlab-primary' style, the majority of the programming is in Matlab, and the final results are importedinto Python for use in some external process.
For Python-primary, the thing to keep in mind is that not all Matlab functions will return Python-readable data. mlabwrap will return a MLabObjectProxy object from these functions. These commonly occur when you use Matlab functions to create objects that are passed into other Matlab functions to actually process the data. For example, you can use the digital signal processing toolbox to create a Welch spectrum object which you can then use to get the power spectrum of your data. Theoretically, you can pass these MLabObjectProxies into Matlab functions that require them. In my experience the more you pass these back and forth, the more likely you are to find a bug in mlabwrap. What you can do instead is write a simple Matlab wrapper function obtains the object, processes the data, and then returns appropriate output as an array.
You can also get around problems with the MLabObjectProxies by using the low-level commands in mlabwrap. For example, if I have a matlab_struct that is a struct array with field matlab_struct.label, and I only want the labels on the Python side, I can do the following:
# place matlab_struct into the Matlab workspace
mlab._set('matlab_struct', matlab_struct)
# convert the labels into a cell array
matlab_struct_labels = mlab.eval('{matlab_struct.labels}')
The main low-level commands available are mlab._set('variable_name', variable), mlab.eval('command string'), and mlab.get('variable_name').
If I'm doing a lot of heavy-duty processing in Matlab, say in a toolbox or plugin that isn't available elsewhere, I'll write what I call 'Matlab-primary' code, where I try to avoid passing data back and forth through mlabwrap, instead manipulating variables in the Matlab workspace by calling .m scripts, saving the resulting output to a data file, and importing that into my Python code.
Good luck!
mlabwrapper
is a great solution for python <-> MATLAB bridging. If something is not working for you just report concrete problems on SO :)
You have to note that mlabwrapper as project has been around for quite a while. http://mlabwrap.sourceforge.net/
I had problems with mlabwrap.cpp recently, for which I found the following github fork
Related Thereis is a copy of mlabwrap v1.1-pre
(http://mlabwrap.sourceforge.net/) patched as described here:
http://sourceforge.net/mailarchive/message.php?msg_id=27312822
with a patch fixing the error:
mlabraw.cpp:225: error: invalid conversion from ‘const mwSize*’ to
‘const int*’ Also note that in Ubuntu you need to sudo apt-get install
csh
For details see http://github.com/aweinstein/mlabwrap
mlab
After spending more time, I made a github mirror to update, bugfix and maintain the wrapper https://github.com/ewiger/mlab (patches and pull-requests are welcomed!)
It can be pip installed, i.e.
pip install mlab
I have excluded the cpp implementation for now. In the current way it works as the following:
For Linux/Mac library creates a pipe connection with MATLAB instance. The rest is serialization (partially pointed out by #brentlance), which is done using numpy.
For Windows library uses DCOM to communicate. (But I am still to fix version lookup using registry).
When to use mlab?
I would recommend to call very high level user-functions in MATLAB (mostly returning logical results or very standard built-in types as matrices) to minimize any communication with MATLAB. This approach is perfect for legacy code, but might require writing some wrapping interfaces to simplify function declarations.
Overall, the code is a bit cumbersome and is a patchwork of many. The core part (now this is matlabpipe and matlabcom) seems to do the job pretty well. Ultimately, I would not recommend mlab for full-scale productive application unless you are willing to spend time testing, bug-reporting, bug-fixing and feature-requesting all of your use-cases.
To answer your questions:
We used mlabwrap and it was reliable. The only difficulty was to compile it. We had a few issues with that. However, once you compile it, it runs well.
I would recommend matlab_wrapper, because it's much easier to install (pure Python, no compilation), supports various data structures (numeric, logical, struct, cell arrays) and runs on GNU/Linux, Windows, OSX.
Using win32com to communicate with MATLAB should work, but it's quite low-level and not portable.
Disclaimer: I'm the author of matlab_wrapper.
Since MATLAB R2014b, there is now a MATLAB API for Python. See the MATLAB documentation for more information. It however requires to start the MATLAB engine which can take some time.
Related
I am prototyping a simple Drake simulation. I have some simple Python LeafSystems that implement controllers, and find that without these systems, my simulation can run at realtime; however, with these systems, my simulation runs much slower than realtime.
I don't think it's the math, but instead just the overhead of Python vs. C++.
For this code:
https://github.com/EricCousineau-TRI/repro/tree/2e3865a7aefe8adc19a6ff69e84025def03da7fd/drake_stuff/python_profiling
If I try to use Python's cProfile and then use snakeviz to visualize the results, I can see that my Python code seems slow, but I can't tell how it compares to the C++ Drake code that pydrake is binding.
Without Python LeafSystems (--no_control):
With the Python LeafSystem:
My tracepoint is in main(), but it does not appear in either of those.
How do I get better information about relative timing, without rolling my own timers?
I'm not sure if this is best answer, but I found this post:
https://stackoverflow.com/a/61253170/7829525
py-spy seems like an excellent tool for seeing relative performance information for Python code that involves CPython API extensions.
From my naive usage mentioned here:
https://github.com/benfred/py-spy/issues/531
https://github.com/EricCousineau-TRI/repro/tree/6048da3/drake_stuff/python_profiling
I can now see more information.
Looking at interactive SVG flamegraphs from py-spy with default rate of 100 samples/sec:
Without Python LeafSystems (--no_control):
With Python LeafSystems:
Per #nicho's suggestion below, using py-spy --native can provide much better detail. Using Ctrl+F for .py:, here's what it looks like:
Without Python LeafSystems (--no_control):
With Python LeafSystems:
As additional note, quick transcription to C++ with some Python bindings yields sufficient performance to run at real-time.
Example of the transcription from Python to C++ w/ bindings:
components.py ->
components_cc_py.cc
I am learning Python and how to make classes. I was curious how the classes are made inside Python itself! For example, in datetime.py (I find it by googling) I was checking how they used __add__ or __sub__ which is using "if isinstance(other, timedelta):" that was an interesting learning. Also, I am learning what professionally written programs look like.
My question is how can I find the source codes of internal classes and functions inside Python, for example, I am interested to see how they implement add in print(), that can print(1+2) -> 3 and
print('a'+'b') -> ab
The source code for the reference implementation of Python is available here at their GitHub mirror. However, it's worth noting that large parts of the Python language are implemented in C, including the core engine and many of the standard library libraries. Really understanding how everything is implemented under the hood requires a fair amount of C fluency.
ipython is a great tool for exploring how things work. Just add "??" after a function or other callable, and it show the code when possible, ie. when it's "pure python".
Eg:
import this
this??
Python is an open source language which means the source code is available to any interested party. I would suggest looking at the source files on the machine you are using or looking at the CPython Github repo.
print() is a built in module. It is written in C and the source can be viewed in the file bltinmodule.c.
You may also find it useful to learn about the functions available in Python for getting help, like help() (documentation available here). To learn about the print() function you can call:
help(print)
I recommend reading the Beginner's Guide as a starting point for more resources.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Call python function from MATLAB
Is there a way to call functions in a Python script from MATLAB, and if so, how?
You're trying to do something non-trivial. Matlab doesn't have any binding of Python-scripts, and also I don't know of a really good substitute. You can choose between two roads, maybe one can help you a little bit (though both are non-trivial, just as your task):
Convert to Python
In nowadays, Python is a valid substitute for most use cases of Matlab. There are libraries for munerical calculations, scientific routines, a great plotting library and a fantastic community that will try to help you with all your problems. And, it's free.
If you only have a couple of Matlab scripts running or you just started using Matlab I'd recommend consideringa a change of platform. To convert your existing Matlab-scripts, you could try the Open Matlab-Python-Converter.
Create a webservice using Python and access it from Matlab
The cleanest way to execute Python-Code from Matlab is to setup a webservice with Python and call this webservice from Matlab. This sounds really complicated, but it isn't. For small projects, I'd recommend XML-RPC. You can look at my other post to see an example of a small XML-RPC-server. If your methods exist, it'Ss easy to adapt to your needs, just import these methods and offer them as webservice calls.
On the Matlab side, you must stick to third-party tools for connecting to an XML-RPC server, e.g., Apache XML-RPC jars.
It might become complicated when you want to pass in other variables than primitives, e.g., arrays. I have no experience how well this works.
I have a poorly designed and big (> 300 public functions, >200 numeric constants defined with #define in the header file) that I have to wrap in Python. I have the dll and the h file. The library is updated yearly, till now in a backwards compatible way (i.e. just functions were added, a constant keep their numerical values, etc). But I have no guarantees as I do not control the library.
Using ctypes, I see two ways of wrapping this in Python:
Mapping every constant and function to python, 1 to 1
Redefining the API in Python and making calls to the library.
The first can be done in a (roughly) automatic way from the header file and therefore is easier to maintain and upgrade, the second requires a lot of python code but it will be easier to use.
I would appreciate some opinions based on your experience with this type of problem and some examples.
I recently used ctypesgen to create a ctypes wrapping for SDL, and complementary libraries (SDL_image, SDL_ttf, SDL_mixer).
For me, it worked fairly well. It generates Python 2.x, but I was able to get the desired 3.x code by using the "2to3" utility.
I think it's a good idea to use the ctypes wrapping as a foundation for a more "pythonic" api, and that's basically what I did (on a very simple level) with my pslab module.
So, if you're looking to do something similar, that would be one way.
Maintaining a Python library with a ctypes backend isn't an unmanageable approach. Obviously the initial investment is larger than using automated tools, but the API you are left with should be much better.
If you do take that route, aim to separate the Python API entirely from the C library though. Supporting multiple ctypes backends with one Python front end api isn't too bad - just query at runtime and dynamically load the correct ctypes wrapper module. I've done that to wrap different dll files and .so files for windows and linux but it would work for versions of a library as well.
This question already has answers here:
How do Rpy2, pyrserve and PypeR compare?
(4 answers)
Closed 7 years ago.
I am quite new to R, and pretty much used to python. I am not so comfortable writing R code. I am looking for python interface to R, which lets me use R packages in pythonic way.
I have done google research and found few packages which can do that:
Rpy2
PypeR
pyRserve
But not sure which one is better ? Which has more contributers and more actively used ?
Please note my main requirement is pythonic way for accessing R packages.
As pointed out by #lgautier, there is already another answer on this subject. I leave my answer here as it adds the experience of approaching R as a novice, knowing Python first.
I use both Python and R and sympathise with your need as a newcomer to R.
Since any answer you get will be subjective, I summarise a few points from my experience:
I use rpy2 as my interface and find it is 'Pythonic', stable, predictable, and effective enough for my needs. I have not used the other packages so this is not a comment on them, rather on the merits of rpy2 itself.
BUT do not expect that there will be an easy way of using R in Python without learning both. I find that adding an interface between the two languages allows ease of coding when you know both, but a nightmare of debugging for someone who is deficient in one of the languages.
My advice:
For most applications, Python has packages that allow you to do most of the things that you want to do in R, from data wrangling to plotting. Check out SciPy, NumPy, pandas, BioPython, matplotlib and other scientific packages, or even the full Anaconda or Enthought python distributions. This allows you to stay within the Python environment and provides you most of the power that you need.
At the same time, you will want R's vast range of specialised packages, so spend some time learning it in an interactive environment. I found it almost impossible to master even basic R on the command line, but RStudio and the tutorials at Quick-R and Learn-R got me going very fast.
Once you know both, then you will do magic with rpy2 without the horrors of cross-language debugging.
New Resources
Update on 29 Jan 2015
This answer has proved popular and so I thought it would be useful to point out two more recent resources:
Ralph Heinkel gave a great talk on this subject at EuroPython 2014. The video on Combining the powerful worlds of Python and R is available on the EuroPython YouTube channel. Quoting him:
The triplet R, Rserve, and pyRserve allows the building up of a network bridge from Python to R: Now R-functions can be called from Python as if they were implemented in Python, and even complete R scripts can be executed through this connection.
It is now possible to combine R and Python using rmagic in IPython/Jupyter greatly easing the work of producing reproducible research and notebooks that combine both languages.
A question about comparing rpy2, pyrserve, and pyper with each other was answered on the site earlier.
Regarding the number of contributors, I'd say that all 3 have a relatively small number. A site like Ohloh can give a more detailled answer.
How actively a package is used is tricky to determine. One indication might be the number of downloads, an other might be the number of posts on mailing lists or the number questions on a site like stackoverflow, the number of other packages using it or citing it, the number of CVs or job openings mentioning the package. As much as I believe that I could give a fair evaluation, I might also be seen as having a conflict of interest. ;-)
All three have their pros and cons. I'd say that you base you choice on that.
My personal experience has been with Rpy, not Rpy2. I used it for a while, but dropped it in favor of using system commands. A typical case for me was running a FORTRAN model using Python scripts, and post-processing with R. In my experience the easiest solution was to create a command line tool using R, which is quite straightforward (at least under Linux). The command line tool could be executed in the root of the model run, and the script would produce a set of R objects and plots in an Routput directory. The advantage of disconnecting R and Python in this way was that I could easily debug the R code separate from the Python code.
I think Rpy really shines when a lot of back and forth communication between R and Python is needed. But if the functionality is nicely separable, and the overhead of disk i/o is not too bad, I would stick to system calls. See ?system for more information regarding system calls, and Rscript for running R scripts as a command line tool.
Regarding your wish to write R code in a Python way, this is not possible as all the solutions require you to write R code in R syntax. For Rpy this means R syntax, but a little different (no . for example). I agree with #gauden that there is no shortcut in using R through Rpy.