I have an executable (face recognition) written by one of our programmers who's no longer with us. It takes a commandline argument and prints the result to STDOUT like this.
I need to be able to use this in Python but performance is important so I'm not sure if I should just use subprocess. I need to capture the output of the program so is it a better idea to modify the code to make it a C++ library instead and write a wrapper for it in Python? How would that impact performance compared to using subprocess?
I have looked into the Python docs for running C code and found this but the answer here is different. The second method does not even use Python.h so I'm a little confused. What's the best way to do what I need when performance is important?
At first I have to tell you I think subprocess method and running external application and reading output from STD output would be slow and unreliable. I won't recommend that method.
Instead, here is some suggestions:
You can create a Python extension in C or C++, you can find a lot of resources online about it. So you can convert your current code to Python extension format and just use Python extension skeleton template, integrate your functions and code in it and you'll have a python extension, simply you'll be able to call it's functions from Python code. See: http://docs.python.org/2/extending/index.html
Convert your current C++ code directly into a DLL and exports functions of your code you want to call from python if you are in Windows or simply convert it to a shared library if you are using Linux. It will be compiled to .so . Then you can call functions as an external library in your Python code. Example with explanation: http://www.linuxforu.com/2010/05/extending-python-via-shared-libraries/
I've never tried them, but it seems these projects can help you with what you want:
http://sourceforge.net/projects/pygccxml/?source=dlp
https://code.google.com/p/pybindgen/
http://www.cython.org/
Related
I have a Fortran library wrapped to Python (with f2py), and I want to call the library from MATLAB (2016b). Although it's possible to call the Fortran library from MATLAB using MEX files, to save some work I'm hoping it's possible to call it using the Python interface. I know that one can call Python code from MATLAB (e.g. py.calendar.isleap(2016)), but unfortunately it seems that this only gives access to standard Python libraries.
Since the library has initialization cost, I prefer to initialize once and maintain a running Python session, and write MATLAB wrappers for library initialization and the library routines I wish to call. I'm not entirely sure how the MATLAB and Python sessions can best communicate, but I guess writing and reading temporary text files is a viable option.
Any suggestions for a good way to maintain an active Python session in MATLAB, and pass input to Python and output back to MATLAB?
I am trying to create an application in which the GUI is programmed in Python (Tkinter) and I have a library in C++ that I want to interface with this GUI code. (Please, no comments on why the GUI and the application library are in separate languages).
One approach that immediately comes to mind is to compile the C++ library into an executable and write python wrappers that call this executable ( via system() ) with specific arguments and consume the output.
I am not sure what the performance implications are for such an implementation. Also I do not want to change the library into Python. Any better suggestions or comments on the approach?
There are several ways for doing this. One obvious way was already stated by chis. Another good way of interfacing C++ with Python is by using swig. It all comes down how complex your structures / classes are.
So the C++ code is going to be a module in python and can be called by the interface as any other python function.
I'm designing musical training games using JUCE -- a multiplatform C++ framework that allows me to code audio/visuals close to the wire.
However, I have coded my gameplay (control flow / data-processing) in Python -- it is complex and I wish to keep changing it so I can experiment with different gameplays. Python is ideal for this kind of rapid prototyping work.
So I would like my (platform independent, so Win/OSX/Lin/iOS/And) C++ to start up a Python runtime, feed it a .py file, and then call various functions in that .py. Also I would like to be able to call back to the C++ code from the .py.
Here is the relevant official Python documentation: https://docs.python.org/2/extending/extending.html
And here is a CodeProject article: http://www.codeproject.com/Articles/11805/Embedding-Python-in-C-C-Part-I
However, neither of them seem to address the issue of multiplatform.
The technique seems to be to link with the library libpython.a, and #include which contains the various functions for starting up the runtime environment, loading scripts, executing python-code, etc.
But surely this libpython.a would need to be compiled separately per platform? If so, this wouldn't be a very clean solution, so could I instead add the Python source code to my project and get it to compile the .a?
How can I go about doing this?
EDIT: https://wiki.python.org/moin/boost.python/EmbeddingPython
EDIT2: I'm pretty sure trying to bring in the full CPython source code is overkill here -- someone must have made some stripped down Python implementation in C/C++ that doesn't support any system-calls/multithreading/fancy-stuff -- just works through Python syntax line by line. Looking thru https://wiki.python.org/moin/PythonImplementations but I can't see an obvious candidate.
EDIT3: https://github.com/micropython/micropython should be added to that last page, but still it doesn't look like it is what I'm after
There's an entire chapter of the Python docs that explain the different approaches you can take embedding a Python interpreter into another app.
Embedding Python is similar to extending it, but not quite. The
difference is that when you extend Python, the main program of the
application is still the Python interpreter, while if you embed
Python, the main program may have nothing to do with Python — instead,
some parts of the application occasionally call the Python interpreter
to run some Python code.
So if you are embedding Python, you are providing your own main
program. One of the things this main program has to do is initialize
the Python interpreter. At the very least, you have to call the
function Py_Initialize(). There are optional calls to pass command
line arguments to Python. Then later you can call the interpreter from
any part of the application.
There are several different ways to call the interpreter: you can pass
a string containing Python statements to PyRun_SimpleString(), or you
can pass a stdio file pointer and a file name (for identification in
error messages only) to PyRun_SimpleFile(). You can also call the
lower-level operations described in the previous chapters to construct
and use Python objects.
A simple demo of embedding Python can be found in the directory
Demo/embed/ of the source distribution.
I recently decided to create a project that mixes C++ with Python, thus getting the best of both worlds. My idea was to do rapid prototyping of classes and functions in Python for obvious reasons, but still being able to call C++ code within Python (for obvious reasons as well). So instead of embedding Python in the C++ framework, I suggest you do the opposite: embed your C++ framework into a Python project. In order to do so, you just have to write very simple interface files and let Swig take care of the interfacing part.
If you want to start from scratch, there's a nice tool called cookiecutter that can be used to generate a project templates. You can choose either the cookiecutter-pypackage, or the cookiecutter-pylibrary, the latter improving over the former as described here. Interestingly, you can also use the cookiecutter code to generate the structure of a C++ project. This empty project uses the CMake build system, which IMHO is the best framework for developing platform independent C++ code. I then had to decide on the directory structure for this mixed project, so one of my previous posts describes this in detail. Good luck!
I'm using SWIG to embed Python into my C++ application, and to extend it as well, i.e. access my C++ API in Python outside my application. SWIG and Python are multi-platform, so that is not really an issue. One of the main advantage of SWIG is that it can generate bindings for a lot of languages. There are also a lot of C++ code wrappers that could be used, for example boost.python or cython.
Check these links on SO:
Extending python - to swig, not to swig or Cython
Exposing a C++ API to Python
Or you can go the hard way and use plain Python/C API.
I know that it is possible to write bash extension modules (loadable builtins) on C or lua (see luabash), but is it possible on Python/Cython? Is there any projects that make steps in this direction?
The way you would probably do this is start out with a C library which includes the appropriate exported functions, then within the exported function load and run the python interpreter, run your python code, then tear down the python interpreter.
You can see how to load the python interpreter into a C program/library here:
http://docs.python.org/extending/embedding.html
http://docs.python.org/extending/extending.html#calling-python-functions-from-c
http://www.linuxjournal.com/article/8497
If you do this a lot, then it may be simpler to write a single generic handler when you can use with multiple different python scripts.
I used Bash examples and the linked resources #tylerl mentioned to make bashpy. It's a proof of concept and currently lacks support for both passing variables and calling functions. So not very useful yet, but maybe it can help someone ending up here.
A researcher has created a small simulation in MATLAB and we want to make it accessible to others. My plan is to take the simulation, clean up a few things and turn it into a set of functions. Then I plan to compile it into a C library and use SWIG to create a Python wrapper. At that point, I should be able to call the simulation from a small Django application. At least I hope so.
Do I have the right plan? Are there are any serious pitfalls that I'm not aware of at the moment?
One thing to remember is that the MATLAB compiler does not actually compile the MATLAB code into native machine instructions. It simply wraps it into a stand-alone executable or a library with its own runtime engine that runs it. You would be able to run your code without MATLAB installed, and you would be able to interface it with other languages, but it will still be interpreted MATLAB code, so there would be no speedup.
Matlab Coder, on the other hand, is the thing that can generate C code from Matlab. There are some limitations, though. Not all Matlab functions are supported for code generation, and there are things you cannot do, like change the type of a variable on the fly.
I remember that I was able to wrap a MATLAB simulation into a DLL file and then call it from a Delphi application. It worked really well.
I'd also try ctypes first.
Use the MATLAB compiler to compile the code into C.
Compile the C code into a DLL.
Use ctypes to load and call code from this DLL
The hardest step is probably 1, but if you already know MATLAB and have used the MATLAB compiler, you should not have serious problems with it.
Perhaps try ctypes instead of SWIG. If it has been included as a part of Python 2.5, then it must be good :-)