I am a newbie to cython, so pardon me if I am missing something obvious here. I am trying to build c extensions to be used in python for enhanced performance. I have fc.py module with a bunch of function and trying to generate a .dll through cython using dsutils and running on win64:
c:\python26\python c:\cythontest\setup.py build_ext --inplace
I have the dsutils.cfg in C:\Python26\Lib\distutils. As required the disutils.cfg has the following config settings:
[build]
compiler = mingw32
My startup.py looks like this:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [Extension('fc', [r'C:\cythonTest\fc.pyx'])]
setup(
name = 'FC Extensions',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
This is what the command line outputs looks like :
C:\python26\python C:\cythontest\setup.py build_ext --inplace
running build_ext
cythoning C:\cythonTest\fc.pyx to C:\cythonTest\fc.c
building 'FC' extension
C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -Ic:\python26\include -Ic:\pytho
n26\PC -c C:\cythonTest\fc.c -o c:\cythontest\fc.o
I have latest version mingw for target/host amdwin64 type builds. I have the latest version of cython for python26 for win64. Cython does give me an fc.c without errors, only a few warning for type conversions, which I will handle once I have it right. Further it produces fc.def an fc.o files Instead of giving a .dll. I get no errors. I find on threads that it will create the .so or .dll automatically as required, which is not happening.
Finally, I was able to build extension for win64. Apparently, if you have VC 2010 express, you can tweak the disuilts to use msvc9compiler for compiling the module. The details can be found here. Many thanks to the guys at nukeitdotorg for putting up this, and also to J.F. Sebastian for his tips.
Did you try to compile it with
python setup.py build --compiler=mingw32
?
Related
I 'm fighting 3 days now to setup my venv...
I need cython, openmp....
My IDE is PyCharm Prof in macOS... I have tried plenty solutions with no result...
when i try to build with PyCharm... the building result is:
UserWarning: Unknown distribution option: 'cmd_class' warnings.warn(msg)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: no commands supplied
so, when i try with:
python setup.py install build_ext --inplace
No build again.... The error is following:
ld: warning: directory not found for option '-L/install/prefix/lib'
ld: warning: -L path '/usr/local/Cellar/llvm/9.0.1/lib/libomp.dylib' is not a directory
ld: warning: directory not found for option '-L/install/prefix/lib'
ld: warning: directory not found for option '-L/usr/local/Cellar/gcc/9/gcc/x86_64-apple-darwin19/9.3.0/include/omp.h'
ld: library not found for -lomp
clang-9: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/usr/local/opt/llvm/bin/clang++' failed with exit status 1
i have already install:
brew install llvm
brew install libomp
clang version 9.0.1
Target: x86_64-apple-darwin19.3.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin
The setup.py is according to https://cython.readthedocs.io/en/latest/src/userguide/parallelism.html but i have try a lot of differents mods... i am trying to setup 3-days now...
Finally... i know that the compilation is completed with errors. Although if i try to run it... i have the following resutlts... maybe it helps...
ImportError: dlopen(myfile.pyx, 2): Symbol not found: _omp_get_num_threads
Referenced from: myfile.pyx
Expected in: flat namespace
in myfile.pyx.cpython-36m-darwin.so
I 've read somewhere that there is no way to implement in OSX Cython-App with usage of openMP but i m still believe that there is a way...
My system is macOS Catalina 10.15.3 ...
I need your help!!!
Finally i find the solution...
In your venv:
pip install Cython setuptools
Find where is your gcc compiler... if you have already install do as follows, in other case find and install one...
$ mdfind gcc | grep gcc
Put the whole gcc directory in your project's venv.
(path: /My_Project/venv/gcc)
and setup.py should be the following... (path: /My_Project/package/setup.py)
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext
import os
os.environ["CC"]="../venv/gcc/9.3.0/bin/gcc-9"
os.environ["CXX"]="../venv/gcc/9.3.0/bin/gcc-9"
ext_modules = [Extension(
"filename",
["filename.pyx"],
language='c',
extra_compile_args=['-fopenmp',"-Os",],
extra_link_args=['-fopenmp', ],
)
]
setup(
name='filename',
cmd_class = {'build_ext': build_ext},
ext_modules= cythonize(ext_modules),
)
Your cython file: filename.pyx ... could starts with a compiler directive comment at the top of filename.pyx as following:
# distutils: extra_compile_args = -fopenmp
# distutils: extra_link_args = -fopenmp
Run in your venv the compilation command:
$ python setup.py build_ext -i
Finally you can import your cython file in your code... at the top of your python file etc. mycode.py :
import filename
filename.myCythonFunc()
The above gives the solution for OSX-10.15.3 macOS Catalina.
When I try to install python-pcl(PCL is the point cloud library for presentation like laser radar data. I followed the instruction on https://github.com/strawlab/python-pcl ,and I have already copied travis/pcl-2d-1.8.pc file to /usr/local/lib/pkgconfig folder) in my computer. I typed AppledeMacBook-Pro-3:python-pcl-0.3.0rc1 apple$ python setup.py install in my terminal.Then I encountered a problem below:
running install
running bdist_egg
running egg_info
writing requirements to python_pcl.egg-info/requires.txt
writing python_pcl.egg-info/PKG-INFO
writing top-level names to python_pcl.egg-info/top_level.txt
writing dependency_links to python_pcl.egg-info/dependency_links.txt
reading manifest file 'python_pcl.egg-info/SOURCES.txt'
writing manifest file 'python_pcl.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.6-x86_64/egg
running install_lib
running build_py
running build_ext
skipping 'pcl/_pcl_180.cpp' Cython extension (up-to-date)
building 'pcl._pcl' extension
/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/apple/miniconda3/include -I/Users/apple/miniconda3/include -DEIGEN_YES_I_KNOW_SPARSE_MODULE_IS_NOT_STABLE_YET=1 -I/Users/apple/miniconda3/pkgs/python-3.5.4-h8f450c2_22/lib/python3.5/site-packages/numpy/core/include -I/usr/local/include/pcl-1.8 -I/usr/local/Cellar/openni/1.5.7.10/include/ni -I/usr/local/include/pcl-1.8 -I/usr/local/Cellar/flann/1.9.1_6/include -I/usr/local/include/pcl-1.8 -I/opt/local/include/eigen3 -I/usr/include/ni -I/usr/include/vtk-5.8 -I/usr/local/include/vtk-8.0 -I/usr/local/Cellar/vtk/8.0.1/include -I/Users/apple/miniconda3/pkgs/python-3.5.4-h8f450c2_22/include/python3.5m -c pcl/_pcl_180.cpp -o build/temp.macosx-10.6-x86_64-3.5/pcl/_pcl_180.o
warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the
command line to use the libc++ standard library instead
[-Wstdlibcxx-not-found]
pcl/_pcl_180.cpp:447:10: fatal error: 'vector' file not found
#include <vector>
^~~~~~~~
Somebody says it has something to do with my Cython version. But I have already switched Cython to version 0.25.2, still not working. Does anyone have the same problem? Many thanks. By the way, my python version is 3.5.4 and my Mac Version is macOS Mojave 10.14.1
This is a special issue with current MacOS-installations. You could tweak setup.py and add, as the warning suggest, -std=libc++ to the compile-options, i.e.
from distutils.core import setup
from Cython.Build import cythonize
... some stuff
#passing `-stdlib=libc++` to compiler and linker:
ext_modules = [Extension(...,
language='c++',
extra_compile_args=["-stdlib=libc++"], # + anything else you need
extra_link_args= ["-stdlib=libc++"] # + anything else you need]
... some more stuff
I have also added -stdlib=libc++ to the linker options, because it will be probably the next problem you will run into.
More background: In the MacOS world, for long time, there where two different implementations of c++'s standard library: -libstdc++ associated with gcc and libc++ associated with clang. At the beginning-libstdc++ was also used per default with clang-compiler. However, this is no longer the case - it is not even installed now and that is the reason why the headers cannot be found. I'm not sure why your clang-version doesn't take libc++ per default - so you have to pass it manually to compiler/linker.
I'm new to python, and try to compile python module in HP-UX 11.31.
% CC=gcc python setup.py build
unix
running build
running build_ext
building 'pygoldilocks' extension
gcc -DNDEBUG -O +z -DPYGOLDILOCKS_VERSION=1.0
gcc: +z: not recognized ...
this produces '+z' not recognized.
How to solove it? Can I remove the option '+z'?
I am trying to get cython to realize I have a c compiler in MinGW 32-bit and I've tried everything I can find on the web but it's still not working. I am running Windows 7 Professional 64-bit. Here is what I have tried:
(1) I have Python 2.7 and I just installed MinGW with options gcc and g++ and some other options
(2) I edited the PATH environmental variable so it includes
C:\MinGW\bin;C:\MinGW\MSYS\1.0\local\bin;C:\MinGW\MSYS\1.0\bin
(3) I told Python to use MinGW as the default compiler by creating a file named
C:\Python27\Lib\distutils\distutils.cfg, containing
[build]
compiler = mingw32
(I do have MinGW32 by the way)
(4) I removed all instances of -mno-cygwin from the file C:\Python27\Lib\distutils\cygwincompiler.py
(5) I have a file called setup.py and a module called tryingcython.pyx that is written in python. My setup.py says
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext':build_ext},
ext_modules=[Extension("tryingcython",["tryingcython.pyx"])]
)
So then I open Command Prompt and get into the directory that contains setup.py and tryingcython.pyx, and I type
python setup.py build_ext --inplace --compiler=mingw32
Then it tells me:
running build_ext
skipping 'tryingcython.c' Cython extension (up-to-date)
building 'tryingcython.c' extension
gcc -mdll -O -Wall -IC:\Python27\include -IC:\Python27\PC -c tryingcython.c -o build\
temp.win32-2.7\Release\tryingcython.o
error: command 'gcc' failed: No such file or directory
So I guess Cython can't tell that I have gcc and it can't find it or what, even though I've tried about every single piece of advice I can find online for making it realize that I have MinGW which has gcc included.
Any help/additional ideas on how I can get cython to actually work would be much appreciated.
You are using exactly the same operational system and versions than me.
Try to cal gcc using:
SET input=intput.c
SET output=output.pyd
gcc -shared -IC:\Python27\include -LC:\Python27\libs -O2 -o %output% %input% -lpython27
Usually I put this call in a cythongcc.bat file, in a directory recognized by the PATH environment variable as:
gcc -shared -IC:\Python27\include -LC:\Python27\libs -O3 -mtune=native -o %1.pyd %2.c -lpython27
So that I can , from where my cython .pyx files are, just do:
cython input.pyx
cythongcc input input
To get the compiled .pyd working!
I'm using the basic "hello world" demo straight out of the Cython documentation. It works fine unless I try to import py2app in the same setup.py file:
import py2app
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension("helloworld", ["helloworld.pyx"])]
)
Py2app itself works fine as long as I pre-generate the .c files for my Cython modules. But if I haven't, then build_ext fails with:
running build_ext
gcc-4.2 not found, using clang instead
building 'helloworld' extension
clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c helloworld.c -o build/temp.macosx-10.6-intel-2.7/helloworld.o
clang: error: no such file or directory: 'helloworld.c'
clang: error: no input files
error: command 'clang' failed with exit status 1
If I comment out import py2app in setup.py, build_ext works fine and I get the missing intermediate step in my compilation:
...
gcc-4.2 not found, using clang instead
cythoning helloworld/helloworld.pyx to helloworld/helloworld.c
building 'helloworld' extension
...
So what is it about py2app that breaks Cython? And what can I do about it? I'd like to have just one setup.py for my project, obviously.
I have Cython 0.18 and py2app 0.7.2, installed from PyPI. I'm using Mac OS X 10.8 with the python.org Python 2.7.3, not Apple's build.
The build fails because py2app uses setuptools, and older versions of setuptools (and distribute) are not compatible with Cython.
The solution is to install a newer version of setuptools or distribute.