We Keep Coding

How to make Python look for missing DLLs in its own folder?

I have written a Python C-API package that uses openssl to perform https requests. This builds fine using the below setup:
from distutils.core import setup, Extension
my_module = Extension('my_module',
sources = ['my_module.cpp'],
include_dirs = ['<...>/openssl-1.1/x64/include', '<...>/cpp-httplib'],
library_dirs=['<...>/openssl-1.1/x64/lib'],
libraries = ['libssl', 'libcrypto']
)
setup(name = 'my_module', version = '1.0',description = '', ext_modules = [my_module])
But when I use the module (test.pyd), i.e.
import my_module
It complains that the DLLs for openSSL (libcrypto-1_1.dll and libssl-1_1.dll) are missing:
ImportError: DLL load failed while importing my_module: The specified module could not be found.
I could add these to the folder to make the import work, but I find this somewhat unnecessary because these DLLs can also be found in the installation of Python itself (at C:\Users\<username>\AppData\Local\Programs\Python\Python38\DLLs. Is there anyway I can modify the setup so that Python looks for the missing DLLs in the Python DLLs folder?

Use Cython to wrap C/C++-program depending on multiple libraries

I have a large C/C++ project which I usually just build as a standalone executable. It also requires some libraries (system libraries like pthread, but also MKL from intel for example) which I specify when compiling the project.
Now, I want to use some functions from this project in Python. So my first plan was to simply build the project as a static library and then use that to write a cython wrapper. I.e.
Build the c project: icc -c src/main.cpp .. options and stuff.. linking to all libraries needed .. -o main.o
Create a static library (including the libraries I needed in step 1): ar -rc libmain.a main.o ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ... /usr/lib/x86_64-linux-gnu/libpthread.a ...
I use the generated static library to build my cython wrapper
when I try to execute a test python script calling a random function from my C program I get this error ImportError: /home/.../main.cpython-37m-x86_64-linux-gnu.so: undefined symbol: __kmpc_ok_to_fork
It seems like I need to tell cython to link to the corresponding libraries again or something, but I'm not really sure how to resolve this.. Also I don't want the python project to be dependent on some special libraries that I have installed on my system (that's what I try to achieve by adding all the libraries in step 2), is this possible at all?
My setup.py looks like this
from setuptools import setup, Extension
from Cython.Build import cythonize
import os
os.environ["CC"] = "icc"
os.environ["CXX"] = "icc"
setup(
ext_modules = cythonize([Extension("test", ["test.pyx"], language="c++",
library_dirs=[r'.'], libraries=['main'])])
)

Create a shared library (.so on linux). Python can only support dynamic or shared libraries. The linker is looking for a .so not a .a.
So your setup.py would be remain the same,
from setuptools import setup, Extension
from Cython.Build import cythonize
import os
os.environ["CC"] = "icc"
os.environ["CXX"] = "icc"
setup(
ext_modules = cythonize([Extension("test", ["test.pyx"], language="c++",
library_dirs=[r'.'], libraries=['main'])])
)
Related - Using Cython To Link Python To A Shared Library.

Python package Cython module

I have the following package structure:
+ repo/
+ setup.py
+ package/
+ module1/
+ submodule1.py
+ submodule2.pyx
+ module2/
+ submodule3.py
I would like to use submodule2.pyx from submodule1.py by something like:
import submodule2
but I have absolutely no idea how to do this. I tried adding the following lines to my setup.py:
from distutils.core import setup
from setuptools import setup
from Cython.Distutils import build_ext
ext_modules = cythonize(Extension(
"zindex",
sources=["ndmg/graph/zindex.pyx"],
language="c",
))
for e in ext_modules:
e.pyrex_directives = {"boundscheck": False}
setup(
name='ndmg',
ext_modules = ext_modules,
packages=[
'package',
'package.module1',
....
)
but was unsuccessful. All of the tutorials I could find had very very simplified examples, so I am not sure how to include Cython modules in my python package when the rest of the package is all just normal python code. Does anybody have any good examples I could follow, or can somebody tell me what I'm doing wrong?
Thanks in advance!

The name given to cythonize is what Cython will use to call the module and what it will be have to be imported as.
The above setup.py will generate a native extension called zindex and will have to be imported as import zindex even within python files in the zindex package.
Here is an example of how to do this:
from distutils.core import setup
from setuptools import setup
from Cython.Distutils import build_ext
ext_modules = cythonize(Extension(
"ndmg.graph.zindex",
sources=["ndmg/graph/zindex.pyx"],
language="c",
))
<..>
Build and install extension.
In a python file under ndmg/graph/py_index.py you can then do.
from zindex import <..>
to import from the cython module.

cx_Freeze fails to include Cython .pyx module

I have a Python application to which I recently added a Cython module. Running it from script with pyximport works fine, but I also need an executable version which I build with cx_Freeze.
Trouble is, trying to build it gives me an executable that raises ImportError trying to import the .pyx module.
I modified my setup.py like so to see if I could get it to compile the .pyx first so that cx_Freeze could successfully pack it:
from cx_Freeze import setup, Executable
from Cython.Build import cythonize
setup(name='projectname',
version='0.0',
description=' ',
options={"build_exe": {"packages":["pygame","fx"]},'build_ext': {'compiler': 'mingw32'}},
ext_modules=cythonize("fx.pyx"),
executables=[Executable('main.py',targetName="myproject.exe",base = "Win32GUI")],
requires=['pygcurse','pyperclip','rsa','dill','numpy']
)
... but then all that gives me is No module named fx within cx_Freeze at build-time instead.
How do I make this work?

The solution was to have two separate calls to setup(); one to build fx.pyx with Cython, then one to pack the exe with cx_Freeze. Here's the modified setup.py:
from cx_Freeze import Executable
from cx_Freeze import setup as cx_setup
from distutils.core import setup
from Cython.Build import cythonize
setup(options={'build_ext': {'compiler': 'mingw32'}},
ext_modules=cythonize("fx.pyx"))
cx_setup(name='myproject',
version='0.0',
description='',
options={"build_exe": {"packages":["pygame","fx"]}},
executables=[Executable('main.py',targetName="myproject.exe",base = "Win32GUI")],
requires=['pygcurse','pyperclip','rsa','dill','numpy']
)

Installing a Python/Cython (extension) module under development

I am constantly working on a Python module which contains C++ extensions wrapped with Cython. The setup.py currently handles the building of the extension module, and is called as python3 setup.py --build_ext --inplace.
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext
srcDir = "../src"
src = ["_MyProject.pyx"] # list of source files
print("source files: {0}".format(src))
modules = [Extension("_MyProject",
src,
language = "c++",
extra_compile_args=["-fopenmp", "-std=c++11", "-O3", "-DNOGTEST"],
extra_link_args=["-fopenmp", "-std=c++11"],
libraries=["MyProjectLib", "log4cxx"],
library_dirs=["../"])]
for e in modules:
e.cython_directives = {"embedsignature" : True}
setup(name="_MyProject",
cmdclass={"build_ext": build_ext},
ext_modules=modules)
On top of the Cython module _MyProject, there is a pure Python module MyProject which imports stuff from _MyProject.
Currently I use and test the module by cd-ing into its directory and importing it from there. How do I need to modify my setup.py so that I can install MyProject into my site packages and have the package always up to date?

Add the argument py_modules = ["MyProject.py",] to your setup() function.