Is it possible to import arbitrary modules in cython, compile them to shared object files and then use them in python 3.1?
The reason for this is, that I am writing an extension for the program "blender", which has an internal python 3.1 interpreter. But i would also like to make use of some python-modules which are not ported to 3.x, yet
I have specifically numpy in my mind (but also some other libraries). I have a module, which makes use of numpy. As I want to redistribute that module, I don't want poeple to install numpy on their machines. would that work?
In principle, I believe it's possible. Cython works by translating Python-like code to C code. That code can be compiled for either Python 2 or Python 3 (it uses C preprocessor statements to change which code is used).
The bad news is that it will only work for extensions written in Python-like code that Cython can translate. You can't use Cython on extensions written in C, like Numpy.
The good news is that, at least for Numpy, you shouldn't have to. Since version 1.5, Numpy supports Python 3. There's a binary available for Windows; on other systems, you might have to work out how to compile the code yourself.
Check your Python documentation, the section "Python/C API Reference Manual" describes in detail how to do it.
EDITED:
So what you want is porting a 2.x lib to 3.x. That's a big work to do.
If the solution your wished exists. Python 2.x should have been eliminated by now.
Sure there is a 2to3 tool. But a Python lib written by C is not applicable in this way.
So, you may follow the instruction in the "Python/C API Reference Manual" to port the lib to 3.x or just wait.
Related
I'm in the process of learning Cython and I wasn't able to find a direct answer to this. Also please bear with me as my understanding of C is limited as of now. As far as I understand, with the cythonize command, .pyx files are converted to C, and are compiled to platform-specific libraries (.so / .pxd). My questions are:
If .pyx files are fully converted to C, does it mean that these generated extensions are no longer dependent on python runtime after compilation?
Assuming the same host architecture, can these generated extensions be loaded into other languages, eg. via Java's JNI? If so, are there any hello world examples on this?
Cython extensions are fully C, but they heavily use the Python C API. This means they can't be run independent of libpython (and usually the Python standard library). However it is possible to load libpython into other languages and then use a Cython extension. Also bear in mind that anything you import within Cython is not compiled but needs to be available for import.
I don't plan to answer this fully, but both ImageJ (in Java) and Julia do support Python interoperability layers and do Cython extensions would work there. You're much better off searching for "Python-Java interoperability layer" than trying to create your own way of using Cython specifically.
I am trying to import pygr:
It fails on:
>>> import seqfmt
ImportError: No module named seqfmt
The program that uses this works fine in Python. However its calling a C library called seqfmt (which has a C file and a PYX files). Is this possible to import over to Jython or since its C am I out of luck?
.PYX is the file extension used by cython, a tool for writing C extensions for python in a python-like syntax. Cython creates an intermediate file (that's presumably the .C file you see, at least it's not in the git repository) and compiles it into a python extension.
Jython does not support CPython extensions yet. From its homepage:
There are a number of differences. First, Jython programs cannot currently use CPython
extension modules written in C. These modules usually have files with the extension .so,
.pyd or .dll. If you want to use such a module, you should look for an
equivalent written in pure Python or Java. However, it is technically
feasible to support such extensions, as demonstrated by IronPython.
For the next release of Jython, we plan to support the C Python
Extension API.
Some cython modules can be easily translated into python, and seqfmt is one of them, but pygr has a second cython module, cnestedlist, which involves C calls: The lines
cdef extern from "apps/maf2nclist.h":
[..]
int readMAFrecord(IntervalMap im[],int n,SeqIDMap seqidmap[],int nseq,
int lpoStart,int *p_block_len,FILE *ifile,int maxseq,
long long linecode_count[],int *p_has_continuation)
define an external library call. You'd have to translate this library to Python, too.
Just as a side-note regarding the translation: cython can not only be used to wrap C libraries, but also to simply speed up certain parts of a program. In these cases it is pretty straight-forward to translate them into a python module. Have a look into the seqfmt.pyx source file, its pretty self-explanatory if you know python.
That all being said, there is a project related to Jython, JyNI, which aims to support CPython extensions from Jython. It is work-in-progress, so I can't tell if your libraries are supported by it. There are some examples in the github repository, maybe you can get it to work. The readme file claims binary compatibility, so with JyNI enabled you should be able to run your code without any recompiling.
Basically I want to make a Python program call functions written in C.
So (as far as I know) my options are:
CTypes/CFFI
Create a DLL/SO/DyLib containing the C functions and access them using CTypes or CFFI. Apparently CFFI is way faster with the only drawback of having to declare in python all the functions signatures.
Pros:
Don't have to make any adaptation in my C functions. All type-translation is done in Python.
Cons:
Performance ?
Python Binary Module
Write a python interface in C, converting my C module into a binary python module
Pros:
Performance ?
Cons:
All type-translation is done in C. Using [SIP][3] this might be automated.
Convert the C module into a python binary module is really faster ?
Does both solutions support sending python callbacks to C functions ?
Is SIP a good option to generate a python interface ? Are there any other options ?
Are there any other particularities in any of them ?
I was just reviewing an old list of options I published related to this:
http://stromberg.dnsalias.org/~strombrg/speeding-python/
If you're only targeting CPython (2.x or 3.x), I'd probably go for Cython.
If you want to be able to run on Pypy too, CFFI might be good; I've not tried it yet, but it sounds great. It's not entirely like ctypes though - ctypes is more ABI level, while CFFI is more API level - which is nice.
If you want to be able to run on Jython too, subprocess is probably your best bet.
I'm working in an embedded Linux environment and I have some Python code which I would like to use. My Python code is just doing some math, not using any library other than Numpy and the common ones.
Is there any way to build up a library that I can call from C or C++ code?
Embedding the CPython interpreter into a C or C++ program is actually pretty straightforward.
The official documentation has some complete examples.
Also, check out SWIG and Boost.Python.
I am learning Python. My intentions are:
to write a webapp in Python/Django
create an android app (using Jython)
write some python scripts for unix box
I was under (incorrect) impression that because Python has been implemented in Java (Jython) and .NET (IronPython), I could simply write my Python code and run it through either interpreter/compiler.
I thought if I wrote a hello world in CPython and compiled it with Jython, I'd get Java bytecode. If I compliled it with IronPython, I'd get .NET bytecode.
But now it seems like regular Python code won't work with Jython compiler/interpreter. You've to import some fancy Java specific modules. So, that means, I would have to re-write my program for Java using Java modules/libraries.
Any tips on how to write my Python code so that it works everywhere? Web, Unix, Android.
NOTE: I don't want to have to learn Java.
Thanks
print 'Hello, World!'
This works just fine on any Python implementation worthy of the name. So will most other pure-Python code. Where it gets tricky is when using libraries, as Jython and IronPython are missing some standard library modules and don't support C extensions. Dealing with platform-specific code can also present some issues.
If you want your code to be portable, you need to remove as many dependencies as possible from the shared code. The standard library is generally OK (but not complete in either), and pure-Python external modules are generally OK if they only depend on other pure-Python modules.
If you do need to detect them, I believe the canonical checks are:
if os.name == 'java': # Jython
if sys.platform == 'cli': # IronPython
Neither Jython nor IronPython will produce programs that will run without Jython/IronPython being present. In principle it's possible, and it's even possible to compile a subset of Python to pure bytecode; the former requires linking in the Python engine, and the latter would require restricting what parts of Python you could use.
If someone were to provide this for IronPython I wouldn't turn it down, and I doubt the Jython team would either, but I'm not holding my breath. Either option is a lot of work.
Please be more specific about what you are trying to do. What is your regular Python code ? What does not work with it as you expected ?
According to the Jython FAQ, Jython is an implementation of the Python language. The same Python code should produce the same result on Jython or CPython.