I have the following package structure:
+ repo/
+ setup.py
+ package/
+ module1/
+ submodule1.py
+ submodule2.pyx
+ module2/
+ submodule3.py
I would like to use submodule2.pyx from submodule1.py by something like:
import submodule2
but I have absolutely no idea how to do this. I tried adding the following lines to my setup.py:
from distutils.core import setup
from setuptools import setup
from Cython.Distutils import build_ext
ext_modules = cythonize(Extension(
"zindex",
sources=["ndmg/graph/zindex.pyx"],
language="c",
))
for e in ext_modules:
e.pyrex_directives = {"boundscheck": False}
setup(
name='ndmg',
ext_modules = ext_modules,
packages=[
'package',
'package.module1',
....
)
but was unsuccessful. All of the tutorials I could find had very very simplified examples, so I am not sure how to include Cython modules in my python package when the rest of the package is all just normal python code. Does anybody have any good examples I could follow, or can somebody tell me what I'm doing wrong?
Thanks in advance!
The name given to cythonize is what Cython will use to call the module and what it will be have to be imported as.
The above setup.py will generate a native extension called zindex and will have to be imported as import zindex even within python files in the zindex package.
Here is an example of how to do this:
from distutils.core import setup
from setuptools import setup
from Cython.Distutils import build_ext
ext_modules = cythonize(Extension(
"ndmg.graph.zindex",
sources=["ndmg/graph/zindex.pyx"],
language="c",
))
<..>
Build and install extension.
In a python file under ndmg/graph/py_index.py you can then do.
from zindex import <..>
to import from the cython module.
Related
I am attempting to compile a *.pyx file. It uses some definitions and constants inside a __init__.py in the same directory. The project structure is:
setup.py
Foo/__init__.py
Foo/Foo.pyx
and the setup command is as follows:
from setuptools import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy
setup(
cmdclass = {'build_ext': build_ext},
ext_module = [ Extension(name='Foo', sources=['Foo/Foo.pyx']) ],
include_dirs=[numpy.get_include()],
name='Foo',
packages=['Foo'],
zip_safe=True
)
Problem arises when the egg is built and deployed. The resultant egg has the following structure:
Foo.so
Foo.py
Foo/__init__.py
Now, Foo.py contains some dynamic import code that basically imports the *.so file. However, because of the presence of Foo/__init__.py, import Foo attempts to import symbols only from __init__.py, which contains just some constants (all the relevant code is actually in Foo.so).
I've hacked around this issue by pasting all the definitions from __init__.py into Foo.pyx, but I'm trying to figure out what a proper solution might be.
Any advice is appreciated!
I tracked down my problem to an extraneous argument to the setup() command. Judging by the documentation at https://docs.python.org/2/distutils/setupscript.html, I do not need the packages=['Foo'] argument, and in fact that's what's causing it to create the inner Foo package that's messing everything up.
Following these examples (https://github.com/cython/cython/wiki/PackageHierarchy , https://groups.google.com/forum/#!msg/cython-users/6trL0V1bLx4/7bxhj0xCK50J and Cython: ImportError: No module named 'myModule': how to call a cython module containing a cimport to another cython nodule?), I have created the following cython package setup:
test_outside.py
cython_pckg/
__init__.py
setup.py
test_inside.py
Code/
__init__.py
worker/
__init__.py
worker_1.pyx
worker_1.pxd
worker_2.pyx
worker_2.pxd
worker_3.pyx
worker_3.pxd
My setup.py file looks as follows:
from distutils.core import setup
from distutils.extension import Extension
import numpy
from Cython.Distutils import build_ext
ext_modules = [
Extension("Code.worker.worker_1", ["Code/worker/worker_1.pyx"], include_dirs=[".", numpy.get_include()]),
Extension("Code.worker.worker_2", ["Code/worker/worker_2.pyx"], include_dirs=["."]),
Extension("Code.worker.worker_3", ["Code/worker/worker_3.pyx"], include_dirs=[".","./Code/worker/", numpy.get_include()])
]
setup(name="C_Extensions",
cmdclass={"build_ext": build_ext},
ext_modules=ext_modules,
script_args=["build_ext"],
options={'build_ext':{'inplace':True, 'force':True}}
)
Please note, that worker_3 imports worker_2. If I try to import these modules in test_inside.py, everything works fine. However, importing the modules into test_outside.py throws an ImportError: No module named Code.worker.worker_2 for the file worker_3.pyx. On the top level __init__.py, I import everything.
from Code.worker.worker_1 import *
from Code.worker.worker_2 import *
from Code.worker.worker_3 import *
How can I make this work?
In worker_3.pyx:
cimport worker_2
Note - not cimport Code.<..>
Cython only looks at available .pxd files when cimport is used.
It has no knowledge of module level name spacing, meaning the module name defined in setup.py's Extension.
Reference - Cython docs.
I am a beginner in python and I have just got familiar with cython as well. I am using Anaconda on Windows 64-bit. I am trying to run the "helloworld" example as follows:
1- I build a helloworld.pyx file containing:
print("Hello World")
2- I build a setup.py file containing:
from distutils.core import setup
from Cython.Build import cythonize
setup(name='Hello world app',ext_modules=cythonize("helloworld.pyx"),)
But I get the following error:
'helloworld.pyx' doesn't match any files
Could you please tell me what should I do now? Where should I save these two files?
From here: https://github.com/cython/cython/wiki/enhancements-distutils_preprocessing
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = 'MyProject',
ext_modules = cythonize(["*.pyx"]),
)
Looks like cythonize accepts a list of strings, but you're providing a string.
I am constantly working on a Python module which contains C++ extensions wrapped with Cython. The setup.py currently handles the building of the extension module, and is called as python3 setup.py --build_ext --inplace.
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext
srcDir = "../src"
src = ["_MyProject.pyx"] # list of source files
print("source files: {0}".format(src))
modules = [Extension("_MyProject",
src,
language = "c++",
extra_compile_args=["-fopenmp", "-std=c++11", "-O3", "-DNOGTEST"],
extra_link_args=["-fopenmp", "-std=c++11"],
libraries=["MyProjectLib", "log4cxx"],
library_dirs=["../"])]
for e in modules:
e.cython_directives = {"embedsignature" : True}
setup(name="_MyProject",
cmdclass={"build_ext": build_ext},
ext_modules=modules)
On top of the Cython module _MyProject, there is a pure Python module MyProject which imports stuff from _MyProject.
Currently I use and test the module by cd-ing into its directory and importing it from there. How do I need to modify my setup.py so that I can install MyProject into my site packages and have the package always up to date?
Add the argument py_modules = ["MyProject.py",] to your setup() function.
I'm trying to import a cython module data.pyx into another cython module user.pyx.
Everything compile fine, but when I try to call user.pyx in a python module, I am getting the error 'ImportError: No module named data'.
Everything is in the same directory.
package/
__init__.py #empty
setup.py
data.pxd
data.pyx
user.pyx
My setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [
Extension("data", ["data.pyx"]),
Extension("user", ["user.pyx"],include_dirs = ['myPackageDir'])
]
setup(
name = 'app',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
Running the following test.py will raised the error.
import user #this line raised the 'ImportError: No module named data' below
user.doSomething()
The exception I get is
Traceback:
File "test.py", line 1, in <module>
import package.user
File "user.pyx", line 1, in init user (user.c:3384)
ImportError: No module named data
How can I make the import work? Thanks for any help.
I encounter this problem again in an another project. To solve it, here is what I did:
all import and cimport statement must be fully qualified
all the python code must be contained in a rootFolder
the setup.py must be at the same level than the rootFolder
all folder in the rooFolder including the rootFolder must contains a __init__.py
in your setup.py the extension's include_dirs must contains '.'
I created a simple project which illustrates this here.
This page helped me created it.
But my project is simpler and I think it would have helped me a lot if I had it.
My project also illustrate how to automatically build all cython files in a project.
I might be missing something about Cython, but I think it's:
import package.user
user.doSomething()