I am trying to invoke a Cython file in a Python script. I have read this answer and followed the instruction. However, even though I have successfully compiled the C code, PyCharm does not recognize the Cython file 'hello.pyx' while executing the import command, as shown in the screenshot below. What is the remedy?
The Cython file hello.c is generated by setup.py the content of which is shown below.
from distutils.core import setup
from Cython.Build import cythonize
setup(name='Hello world app',
ext_modules=cythonize("hello.pyx")
)
When you compile your module with cythonize, the resulting module will be put into a subfolder. By default, this subfolder is not in the PATHS list. To fix this problem you can just move the resulting file .dll file into the folder where your hello.py and hello.pyx are located.
An alternative approach is to add Extension like so:
from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension
exts = [Extension(name='hello',sources=['hello.pyx'])]
setup(name='Hello world app',
ext_modules=cythonize(exts)
)
And then compile with:
python setup.py build_ext --inplace
This code will put resulting .dll into the same folder as your source files.
Related
I am a beginner in python and I have just got familiar with cython as well. I am using Anaconda on Windows 64-bit. I am trying to run the "helloworld" example as follows:
1- I build a helloworld.pyx file containing:
print("Hello World")
2- I build a setup.py file containing:
from distutils.core import setup
from Cython.Build import cythonize
setup(name='Hello world app',ext_modules=cythonize("helloworld.pyx"),)
But I get the following error:
'helloworld.pyx' doesn't match any files
Could you please tell me what should I do now? Where should I save these two files?
From here: https://github.com/cython/cython/wiki/enhancements-distutils_preprocessing
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = 'MyProject',
ext_modules = cythonize(["*.pyx"]),
)
Looks like cythonize accepts a list of strings, but you're providing a string.
I have a Python application to which I recently added a Cython module. Running it from script with pyximport works fine, but I also need an executable version which I build with cx_Freeze.
Trouble is, trying to build it gives me an executable that raises ImportError trying to import the .pyx module.
I modified my setup.py like so to see if I could get it to compile the .pyx first so that cx_Freeze could successfully pack it:
from cx_Freeze import setup, Executable
from Cython.Build import cythonize
setup(name='projectname',
version='0.0',
description=' ',
options={"build_exe": {"packages":["pygame","fx"]},'build_ext': {'compiler': 'mingw32'}},
ext_modules=cythonize("fx.pyx"),
executables=[Executable('main.py',targetName="myproject.exe",base = "Win32GUI")],
requires=['pygcurse','pyperclip','rsa','dill','numpy']
)
... but then all that gives me is No module named fx within cx_Freeze at build-time instead.
How do I make this work?
The solution was to have two separate calls to setup(); one to build fx.pyx with Cython, then one to pack the exe with cx_Freeze. Here's the modified setup.py:
from cx_Freeze import Executable
from cx_Freeze import setup as cx_setup
from distutils.core import setup
from Cython.Build import cythonize
setup(options={'build_ext': {'compiler': 'mingw32'}},
ext_modules=cythonize("fx.pyx"))
cx_setup(name='myproject',
version='0.0',
description='',
options={"build_exe": {"packages":["pygame","fx"]}},
executables=[Executable('main.py',targetName="myproject.exe",base = "Win32GUI")],
requires=['pygcurse','pyperclip','rsa','dill','numpy']
)
I am trying to build a Python multi-file code with PyInstaller. For that I have compiled the code with Cython, and am using .so files generated in place of .py files.
Assuming the 1st file is main.py and the imported ones are file_a.py and file_b.py, I get file_a.so and file_b.so after Cython compilation.
When I put main.py, file_a.so and file_b.so in a folder and run it by "python main.py", it works.
But when I build it with PyInstaller and try to run the executable generated, it throws errors for imports done in file_a and file_b.
How can this be fixed? One solution is to import all standard modules in main.py and this works. But if I do not wish to change my code, what can be the solution?
So I got this to work for you.
Please have a look at Bundling Cython extensions with Pyinstaller
Quick Start:
git clone https://github.com/prologic/pyinstaller-cython-bundling.git
cd pyinstaller-cython-bundling
./dist/build.sh
This produces a static binary:
$ du -h dist/hello
4.2M dist/hello
$ ldd dist/hello
not a dynamic executable
And produces the output:
$ ./dist/hello
Hello World!
FooBar
Basically this came down to producing a simple setup.py that builds the extensions file_a.so and file_b.so and then uses pyinstaller to bundle the application the extensions into a single executeble.
Example setup.py:
from glob import glob
from setuptools import setup
from Cython.Build import cythonize
setup(
name="test",
scripts=glob("bin/*"),
ext_modules=cythonize("lib/*.pyx")
)
Building the extensions:
$ python setup.py develop
Bundling the application:
$ pyinstaller -r file_a.so,dll,file_a.so -r file_b.so,dll,file_b.so -F ./bin/hello
Just in case someone's looking for a quick fix.
I ran into the same situation and found a quick/dirty way to do the job. The issue is that pyinstaller is not adding the necessary libraries in the .exe file that are needed to run your program.
All you need to do is import all the libraries (and the .so files) needed into your main.py file (the file which calls file_a.py and file_b.py). For example, assume that file_a.py uses opencv library (cv2) and file_b.py uses matplotlib library. Now in your main.py file you need to import cv2 and matplotlib as well. Basically, whatever you import in file_a.py and file_b.py, you have to import that in main.py as well. This tells pyinstaller that the program needed these libraries and it includes those libraries in the exe file.
I wrote a simple program in python:
// main.py
import re
links = re.findall('(https?://\S+)', 'http://google.pl http://youtube.com')
print(links)
Then I execute this:
cython main.py
It was generated a file: main.c
And then I tried this:
gcc main.c
And I have an error:
main.c:8:10: fatal error: 'pyconfig.h' file not found
#include "pyconfig.h"
^
1 error generated.
How to compile python to c ? How to get started with cython with xcode on mac ?
You have to tell the gcc compiler where is the pyconfig.h file on your system using the -I flag. You can find it using the find program.
A simpler way to compile the module is using a setup.py module. Cython provides a cythonize function that starts this process for a .pyx module.
Another point you are missing is that Cython files usually define helper functions to be used from a main Python module.
Suppose you have the following setup as for dirs and files:
cython-start/
├── main.py
├── setup.py
└── split_urls.pyx
The contents of the setup.py are
from distutils.core import setup
from Cython.Build import cythonize
setup(name="My first Cython app",
ext_modules=cythonize('split_urls.pyx'), # accepts a glob pattern
)
The contents of the split_urls.pyx file are
import re
def do_split(links):
return re.findall('(https?://\S+)', links)
And it is the main.py module which uses the defined Cython function:
import split_urls
URLS = 'http://google.pl http://youtube.com'
print split_urls.do_split(URLS)
Compile the Cython module by issuing:
$ python setup.py build_ext --inplace
Cythonizing split_urls.pyx
running build_ext
building 'split_urls' extension
creating build
creating build/temp.macosx-10.9-x86_64-2.7
... compiler output ...
And check that your main module is doing what it is supposed to do:
$ python main.py
['http://google.pl', 'http://youtube.com']
I am constantly working on a Python module which contains C++ extensions wrapped with Cython. The setup.py currently handles the building of the extension module, and is called as python3 setup.py --build_ext --inplace.
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext
srcDir = "../src"
src = ["_MyProject.pyx"] # list of source files
print("source files: {0}".format(src))
modules = [Extension("_MyProject",
src,
language = "c++",
extra_compile_args=["-fopenmp", "-std=c++11", "-O3", "-DNOGTEST"],
extra_link_args=["-fopenmp", "-std=c++11"],
libraries=["MyProjectLib", "log4cxx"],
library_dirs=["../"])]
for e in modules:
e.cython_directives = {"embedsignature" : True}
setup(name="_MyProject",
cmdclass={"build_ext": build_ext},
ext_modules=modules)
On top of the Cython module _MyProject, there is a pure Python module MyProject which imports stuff from _MyProject.
Currently I use and test the module by cd-ing into its directory and importing it from there. How do I need to modify my setup.py so that I can install MyProject into my site packages and have the package always up to date?
Add the argument py_modules = ["MyProject.py",] to your setup() function.