In Python, is it possible at run time to convert a Google Protocol Buffers .proto file into a python class that reads that data? Python is a very dynamic language. When you use protoc to convert a .proto file to python source code, the generated code makes a lot of use of python metaclasses, so it's already very dynamic.
Ideally, I'm thinking of something like this:
import whatever
module = whatever.load_from_file("myfile.proto")
Is this possible?
(I am new to protocol buffers, please let me know if my question makes no sense)
In theory, all the pieces exist to make this work. The Python protobuf implementation could call the C++ .proto parser library (libprotoc) as a C extension to get Descriptors, and then could feed those into the metaclasses.
However, as far as I know, no one has quite tied it altogether. (Disclaimer: My knowledge is a few years old, I may have missed a new development, but I don't see anything in the docs.)
Incidentally, Cap'n Proto's Python implementation does do what you describe, proving it is possible. But that doesn't help you if you need to work with Protobuf format.
(Disclosure: I was the author of most of Google's open source Protobuf code, and I am also the author of Cap'n Proto.)
The most elegant solution to this problem is to use init.py. See: What is __init__.py for?
You can invoke your script to generate Python protobuf classes in init.py. Upon importing, that script will be automatically invoked to generate Python protobuf classes.
Related
I am new to kafka, and trying work some basic examples via VScode. The problem is I can't seem to make parameter hints work at all for all the artefacts imported via confluent_kafka. The module itself is a wrapper, and I was wondering if there was a way to get parameter hinting the same way other python classes and modules work?
It's impossible. Because what you want was stored in the file of cimpl.cp39-win_amd64.pyd.
The Language Server can not provide Intellisense from the file with pyd filename extension.
You can have a look at the cimply with PyCharm, as it can decompilation it.
Currently, it is not supported by VScode. One way to get the tooltips is to create a .pyi file for your package which defines the interface the functions. If you ship this with the package (or have this file in your VSCode working directory) it will be able to give parameter hints and such!
You can create one using mypy's stubgen, but it does not automatically include the docstrings. You can write one yourself however. Have a look here: Python: Generate function stubs from C module
I'm writing some sort of Python C extension. It uses my own *.so library and headers from another project (let's say they're in /usr/local/lib/otherproject.so and /usr/local/include/otherproject.h).
I don't know which strategy to follow. I came up with two:
As a pure Python extension
Write a Python C extension just as described in the official docs. The problem here is that I don't know how to link with my own library and headers; to compile, I write a setup.py file and run python3.4 setup.py build. I don't know if I can include some option to the former command, or if I can write something in setup.py to include my headers and binaries (if so, I will also have to worry about making this distributable?).
With ctypes
Write a C library (with my other project's build system). Include Python by passing '/usr/include/python2.7' to find headers and the python2.7 binary. Then use ctypes to wrap around that library and get the functions, types, objects, etc. The inconvenience here is that I need to manually wrap around every single function/type/variable from ctypes; I don't think I can use PyModule_AddObject since I'm not creating the module in C but in the Python wrapper (with ctypes).
Also, I tried the second approach, but I could not successfully get my custom PyTypeObject from ctypes. If the second approach sounds good to any more expert brain here on SO, I would post the code to get any help =).
The second approach also yields problems with distribution. And if you create a Python object in C you should do it in the context of a module. For scenarios where distribution is problematic, you could link this module statically instead.
For your issue with linking you'll find more information about Library options in the documentation. Since your library resides in a directory which should be in the standard library search path, you'd only need to define your library with the libraries option of the Extension class:
mymodule_ext = Extension('mymodule', ['mymodule.c'], libraries=['otherproject'])
If you're not using the standard lib* prefix you'd need to use libraries=[':otherproject.so'] instead.
I'm designing musical training games using JUCE -- a multiplatform C++ framework that allows me to code audio/visuals close to the wire.
However, I have coded my gameplay (control flow / data-processing) in Python -- it is complex and I wish to keep changing it so I can experiment with different gameplays. Python is ideal for this kind of rapid prototyping work.
So I would like my (platform independent, so Win/OSX/Lin/iOS/And) C++ to start up a Python runtime, feed it a .py file, and then call various functions in that .py. Also I would like to be able to call back to the C++ code from the .py.
Here is the relevant official Python documentation: https://docs.python.org/2/extending/extending.html
And here is a CodeProject article: http://www.codeproject.com/Articles/11805/Embedding-Python-in-C-C-Part-I
However, neither of them seem to address the issue of multiplatform.
The technique seems to be to link with the library libpython.a, and #include which contains the various functions for starting up the runtime environment, loading scripts, executing python-code, etc.
But surely this libpython.a would need to be compiled separately per platform? If so, this wouldn't be a very clean solution, so could I instead add the Python source code to my project and get it to compile the .a?
How can I go about doing this?
EDIT: https://wiki.python.org/moin/boost.python/EmbeddingPython
EDIT2: I'm pretty sure trying to bring in the full CPython source code is overkill here -- someone must have made some stripped down Python implementation in C/C++ that doesn't support any system-calls/multithreading/fancy-stuff -- just works through Python syntax line by line. Looking thru https://wiki.python.org/moin/PythonImplementations but I can't see an obvious candidate.
EDIT3: https://github.com/micropython/micropython should be added to that last page, but still it doesn't look like it is what I'm after
There's an entire chapter of the Python docs that explain the different approaches you can take embedding a Python interpreter into another app.
Embedding Python is similar to extending it, but not quite. The
difference is that when you extend Python, the main program of the
application is still the Python interpreter, while if you embed
Python, the main program may have nothing to do with Python — instead,
some parts of the application occasionally call the Python interpreter
to run some Python code.
So if you are embedding Python, you are providing your own main
program. One of the things this main program has to do is initialize
the Python interpreter. At the very least, you have to call the
function Py_Initialize(). There are optional calls to pass command
line arguments to Python. Then later you can call the interpreter from
any part of the application.
There are several different ways to call the interpreter: you can pass
a string containing Python statements to PyRun_SimpleString(), or you
can pass a stdio file pointer and a file name (for identification in
error messages only) to PyRun_SimpleFile(). You can also call the
lower-level operations described in the previous chapters to construct
and use Python objects.
A simple demo of embedding Python can be found in the directory
Demo/embed/ of the source distribution.
I recently decided to create a project that mixes C++ with Python, thus getting the best of both worlds. My idea was to do rapid prototyping of classes and functions in Python for obvious reasons, but still being able to call C++ code within Python (for obvious reasons as well). So instead of embedding Python in the C++ framework, I suggest you do the opposite: embed your C++ framework into a Python project. In order to do so, you just have to write very simple interface files and let Swig take care of the interfacing part.
If you want to start from scratch, there's a nice tool called cookiecutter that can be used to generate a project templates. You can choose either the cookiecutter-pypackage, or the cookiecutter-pylibrary, the latter improving over the former as described here. Interestingly, you can also use the cookiecutter code to generate the structure of a C++ project. This empty project uses the CMake build system, which IMHO is the best framework for developing platform independent C++ code. I then had to decide on the directory structure for this mixed project, so one of my previous posts describes this in detail. Good luck!
I'm using SWIG to embed Python into my C++ application, and to extend it as well, i.e. access my C++ API in Python outside my application. SWIG and Python are multi-platform, so that is not really an issue. One of the main advantage of SWIG is that it can generate bindings for a lot of languages. There are also a lot of C++ code wrappers that could be used, for example boost.python or cython.
Check these links on SO:
Extending python - to swig, not to swig or Cython
Exposing a C++ API to Python
Or you can go the hard way and use plain Python/C API.
I'm writing a Python-based [web] application that needs to be able to read and write EXIF data.
libexif seems to have all the right ingredients, but I can't work out how (or if) I could access it access it by using Python's ctypes library? I'm new to C, suppose I need see a .so for this to work?
You need to be running on an os that you can obtain the required library, to download the .h files, (usually the -dev package gives you these).
Then you need to work your way through the ctypes tutorial found here which explains all the steps you need to take.
I used cpython api to load py from C/C++.
But, if i want not setup cpython in client, can I take package dll of cpython in my program?
How to do that?
Installer-builders like PyInstaller (cross-platform) and py2exe (Windows only) basically do that job for you in a general way, except that the executable at the heart of the produced package is their own instead of yours.
But basically, you can imitate their behavior in terms of setting up a .zip file with all the Python library modules you need (or just zip up everything in the standard python library if you want to allow python code running form your app to import anything from there), and follow the simple advice in the Embedding Python in Another Application section of the Python docs.
Note that embedding Python equals extending Python plus a little bit of code to initialize and finalize the interpreter itself and a little bit of packaging as I just mentioned; if you've never writted Python extensions I would suggest practicing that first since it's the most substantial part of the task (not all that hard with helpers such as boost python, but more work if you choose to do it as the "bare C" level instead).
You don't need to setup Python to embed it in applications. The core of the Python interpreter is available as a shared library which you can dynamically load in your application and distribute with it.
Read on embedding Python in the official docs. Also, this article seems nice and comprehensive for Linux. For Windows, read the notes here.
Here's another SO question that discusses this issue.
The Python license is probably hard to understand for a non-lawyer, non-native English speaker. So yes, you can redistribute the unmodified DLL as it contains the copyright notice within it.
It would be polite to give credit like "This program contains the Python Language Interpreter version X.XX http://python.org for more information" or similar somewhere in the program or documentation.