I am trying to import pygr:
It fails on:
>>> import seqfmt
ImportError: No module named seqfmt
The program that uses this works fine in Python. However its calling a C library called seqfmt (which has a C file and a PYX files). Is this possible to import over to Jython or since its C am I out of luck?
.PYX is the file extension used by cython, a tool for writing C extensions for python in a python-like syntax. Cython creates an intermediate file (that's presumably the .C file you see, at least it's not in the git repository) and compiles it into a python extension.
Jython does not support CPython extensions yet. From its homepage:
There are a number of differences. First, Jython programs cannot currently use CPython
extension modules written in C. These modules usually have files with the extension .so,
.pyd or .dll. If you want to use such a module, you should look for an
equivalent written in pure Python or Java. However, it is technically
feasible to support such extensions, as demonstrated by IronPython.
For the next release of Jython, we plan to support the C Python
Extension API.
Some cython modules can be easily translated into python, and seqfmt is one of them, but pygr has a second cython module, cnestedlist, which involves C calls: The lines
cdef extern from "apps/maf2nclist.h":
[..]
int readMAFrecord(IntervalMap im[],int n,SeqIDMap seqidmap[],int nseq,
int lpoStart,int *p_block_len,FILE *ifile,int maxseq,
long long linecode_count[],int *p_has_continuation)
define an external library call. You'd have to translate this library to Python, too.
Just as a side-note regarding the translation: cython can not only be used to wrap C libraries, but also to simply speed up certain parts of a program. In these cases it is pretty straight-forward to translate them into a python module. Have a look into the seqfmt.pyx source file, its pretty self-explanatory if you know python.
That all being said, there is a project related to Jython, JyNI, which aims to support CPython extensions from Jython. It is work-in-progress, so I can't tell if your libraries are supported by it. There are some examples in the github repository, maybe you can get it to work. The readme file claims binary compatibility, so with JyNI enabled you should be able to run your code without any recompiling.
Related
I'm in the process of learning Cython and I wasn't able to find a direct answer to this. Also please bear with me as my understanding of C is limited as of now. As far as I understand, with the cythonize command, .pyx files are converted to C, and are compiled to platform-specific libraries (.so / .pxd). My questions are:
If .pyx files are fully converted to C, does it mean that these generated extensions are no longer dependent on python runtime after compilation?
Assuming the same host architecture, can these generated extensions be loaded into other languages, eg. via Java's JNI? If so, are there any hello world examples on this?
Cython extensions are fully C, but they heavily use the Python C API. This means they can't be run independent of libpython (and usually the Python standard library). However it is possible to load libpython into other languages and then use a Cython extension. Also bear in mind that anything you import within Cython is not compiled but needs to be available for import.
I don't plan to answer this fully, but both ImageJ (in Java) and Julia do support Python interoperability layers and do Cython extensions would work there. You're much better off searching for "Python-Java interoperability layer" than trying to create your own way of using Cython specifically.
I am working on a project which requires me to create some wrappers in Python for the C library that I need to call from Python. For context, the C library I am using is a bunch of header files (.h) and statically linked library files (.a)
I have decided to use either CFFI or Cython to get my work done. I followed examples similar to this for CFFI - Interfacing C code with CFFI, and this for Cython - Making your C library callable from Python by wrapping it with Cython. Now small sample programs I've tried in both these modules more or less have the following steps
Create the interfacing code to call C APIs
In CFFI, it's a python file declaring the C functions and headers needed
In Cython, it's a .pyx file and modifications to setup.py
Build the interfacing code to generated the .so files for the interfacing glue code.
Call the wrapped functions from a different python script, by importing the interfacing library from the .so file.
Now, this works perfectly for me. But, I'll have to go through two execution steps in the process (generating the .so file, and then actually running the python script with the C API being called).
What I need is to know if there is a way to do all the above in a single execution step. Like, I want to run my final python script, and it should build the interfacing code and import it on the fly in a single execution.
For more context, I have tried SWIG, but wasn't able to find a way to wrapped .a statically linked libraries with it. Same goes for ctypes.
Can you not do this?
import os
os.system('command to build your .so here')
...
import what_ever_you_need
...
For CFFI you just need to execute at runtime the code that is now in the builder script. Move it all in a function, and then you have a function you can call when needed.
I'm developing a C++ library, which has python embedded. What I would like to do is to statically link Python library, so that there won't be configuration issues, when I switch to production server. So far, I'm able to link libpython3.5m.a statically (I had to build Python from sources though, because it seems, that packaged libraries aren't compiled with -fPIC flag). However, I came to a problem, that it seems, there's no Numpy: When I run application, which uses my library, it prompts me with an error:
ImportError: numpy.core.multiarray failed to import
And this error is caused by import_array1() macro, that (AFAIK) is used to import the numpy routines to C++. I tried linking libnpymath.a as well as libnpysort.a, which I found in numpy build dir, but to no avail. Do you happen to know, if such static linking is possible and how to do it? I guess it should be possible, since numpy is written in C...
What I would like to do is to statically link Python library, so that there won't be configuration issues, when I switch to production server.
This would only be the Python core, it would exclude all of the Python libraries. You still need to ship all of the Python code.
...since numpy is written in C...
This is incorrect. NumPy is written about half in C and half in Python. It looks like the C part is the part that's not loading here, since numpy.core.multiarray is written in C, and you wouldn't normally import that yourself, it would normally be imported by the Python part of NumPy.
Linking in the C code is not enough anyway, you need to load initialize the associated Python modules exported by the C code. Without static linking, Python would just find the multiarray.so file in the right place and load it. When you build Python statically, you would would normally edit the Modules/Setup.local file with the modules you want statically compiled into Python. However, this is not designed to work with arbitrary third-party modules like NumPy. See: Compile the Python interpreter statically?
Honestly, if you are just trying to make sure that the same version of Python runs on both development and production systems, there are vastly easier ways to do this, like virtualenv. CPython is simply not designed to be statically linked.
I have made an application in Python. It contains several plugins, organized into different subdirectories. I need to compile entirely to C code to improve security of source code. I have dealt with Cython, but cannot find how to compile the entire directory, with all plugin dependencies. I need a way to compile each of the dependencies to C, and that the application runs from C compiled.
http://docs.cython.org/src/quickstart/build.html
How to compile and link multiple python modules (or packages) using cython?
Python does not compile to native code. Scripts can be "frozen" with a few different tools, which makes them into standalone executables, but it's not actually compiling to C, it's packaging the script (or just its Python byte code representation) with a copy of the interpreter and all of its dependencies; the binary still has all the Python code (or the trivial byte code transform thereof) in it.
Cython lets you compile a syntactic variant of Python into Python C extensions, but they still run on the Python interpreter, and they still expose enough information to reverse the transformation.
Get the proper legal protections in place and freeze your Python executable if you like (freezing is enough to make the source code "non-obvious" even if anyone who went to trivial effort could get it back), but Python does not compile to plain C directly (if it did, I'd expect the CPython reference interpreter to do that more often just to gain performance with the built-in modules, yet they only write C accelerators by hand).
Is it possible to import arbitrary modules in cython, compile them to shared object files and then use them in python 3.1?
The reason for this is, that I am writing an extension for the program "blender", which has an internal python 3.1 interpreter. But i would also like to make use of some python-modules which are not ported to 3.x, yet
I have specifically numpy in my mind (but also some other libraries). I have a module, which makes use of numpy. As I want to redistribute that module, I don't want poeple to install numpy on their machines. would that work?
In principle, I believe it's possible. Cython works by translating Python-like code to C code. That code can be compiled for either Python 2 or Python 3 (it uses C preprocessor statements to change which code is used).
The bad news is that it will only work for extensions written in Python-like code that Cython can translate. You can't use Cython on extensions written in C, like Numpy.
The good news is that, at least for Numpy, you shouldn't have to. Since version 1.5, Numpy supports Python 3. There's a binary available for Windows; on other systems, you might have to work out how to compile the code yourself.
Check your Python documentation, the section "Python/C API Reference Manual" describes in detail how to do it.
EDITED:
So what you want is porting a 2.x lib to 3.x. That's a big work to do.
If the solution your wished exists. Python 2.x should have been eliminated by now.
Sure there is a 2to3 tool. But a Python lib written by C is not applicable in this way.
So, you may follow the instruction in the "Python/C API Reference Manual" to port the lib to 3.x or just wait.