We Keep Coding

Accessing DLL function using ctypes

I have been trying to convert a C-code to DLL and load it from Python but I have never succeeded. I am using Windows10
(64bit). The original C code is quite simple:
//Filename: my_functions.c
# include <stdio.h>
int square(int i){
return i*i
}
I summarize the two approaches I have tried so far.
Method1
First, I built the 64-bit dll by cl.exe on Command Prompt:
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\Tools\VsDevCmd.bat" -arch=x64
cl.exe /D_USRDL /D_WINDLL my_functions.c /MT /link /DLL /OUT:my_functions.dll /MACHINE:X64
As a result, I got a file my_functions.dll. Subsequently, I tried to load it by the following Python code (Python 3.8, Spyder):
# Filename: calling_c_functions.py
from ctypes import *
my_functions = CDLL("./my_functions.dll")
print(type(my_functions))
print(my_functions.square(10))
print("Done")
However, I got the error:
AttributeError: function 'square' not found.
Why does this happen? According to the document in Microsoft: Exporting from a DLL, a DLL file contains an exports table including the name of every function that the DLL exports to other executables. These functions need to be accessed either by ordinal number or by name (different from the original name 'square').
This brought me to Method 2 below.
Method2
Based on Python: accessing DLL function using ctypes -- access by function name fails, I first created a DLL file by myself using the keyword __declspec(dllexport):
//Filename: my_functionsDLL2.dll
#include <stdio.h>
extern "C"
{
__declspec(dllexport) int square(int i);
}
__declspec(dllexport) int square(int i)
{
return i * i;
}
The next step is to find a real function name in the my_functionsDLL2.dll by link.exe. I did the following on the Command Prompt.
call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\link.exe"
link /dump /exports my_functionsDLL2.dll
I should be able to see a list of function names but ended up getting the following output:
Microsoft (R) COFF/PE Dumper Version 14.29.30037.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file myfunctionsDLL2.dll
myfunctionsDLL2.dll : warning LNK4048: Invalid format file; ignored
Summary
Any idea what I'm doing wrong?

You're on the right track but got a bit confused along the way - taking a code file and giving it a .dll extension isn't enough to turn it into a DLL, you do still need to compile it. Your linker is telling you that it doesn't know how to parse your file:
myfunctionsDLL2.dll : warning LNK4048: Invalid format file; ignored
because it only knows how to parse real executables.
You should rename your second code file (the one you named my_functionsDLL2.dll) back to my_functions.c - and compile it the same way you did the first time. But this time, because you added __declspec(dllexport) to your function definition, your compiler (cl.exe) will know that these functions are meant to be exported from the DLL and will export them, making them available through ctypes.

I needed to make 2 modifications to my current approach.
The first modification is the one pointed out by #Daniel Kleinstein. But only with that, I stil got an error: C2059: syntax error: 'string'. This error is due to extern "C" { guards that are only understood by C++ (reference: C2059 syntax error 'string' ? ). For a C source file, I needed to add __cplusplus ifdef.
I summarize the overall procedures that led me to the correct result (there may be some redundancies though).
I want to call the following C file from Python.
//Filename: my_functions.c
#include <stdio.h>
int square(int i){
return i * i;
}
Create a C file using __declspec(dllexport) and __cplusplus ifdef.
//Filename: my_functions2.c
#include <stdio.h>
#ifdef __cplusplus
extern "C" {
#endif
__declspec(dllexport) int square(int i);
#ifdef __cplusplus
}
#endif
__declspec(dllexport) int square(int i)
{
return i * i;
}
Create a 64-bit dll file my_functions2.dll by running the following commands on Command Prompt. You need to locate the cl.exe carefully on your computer.
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\Tools\VsDevCmd.bat" -arch=x64
cl.exe /D_USRDL /D_WINDLL my_functions2.c /MT /link /DLL /OUT:my_functions2.dll /MACHINE:X64
Call the my_functions2.dll from Python using ctypes. In this example, I test the function square and obtain a correct result.
#Filename: call_my_functions2.py
import ctypes as ctypes
my_functions = ctypes.CDLL("./my_functions2.dll")
print(type(my_functions))
print(my_functions.square(10))
print("Done")
I get the following output:
<class 'ctypes.CDLL'>
100
Done

Wrapper DLL for Python: "fatal error LNK1127: library is corrupt"

Brief description
I have a DLL programmed in ADA with GNAT. I want to compile with MSVC another DLL in C as a wrapper to the ADA_DLL in order to use it with Python.
I have compiled the ada_DLL, then I have generated the .lib file according to gnat documentation about MSVC. And finally I tried to compile the C_DLL with Visual-Studio, getting the error:
libmath.lib : fatal error LNK1127: library is corrupt
Update: In the case of compiling with gcc as suggested by #Brian, I get the following output:
>"C:\GNAT\2015\bin\gcc.exe" -c -IC:\Python27\include -o libmath_c.o libmath_c.c
>"C:\GNAT\2015\bin\gcc.exe" -shared -LC:\Python27\libs -L./ -l libmath -o DIVISION_CPP.pyd libmath_c.o -lpython27
.//libmath.lib: error adding symbols: Malformed archive
collect2.exe: error: ld returned 1 exit status
Things I tried & more data:
I have tried importing the ADA_DLL directly with ctypes in Python and it works, so I believe that the ADA_DLL is correctly compiled. Also, forgetting about the C_DLL is not really an option.
I did a small example with a division example module. My .def file looks something like:
; dlltool -z libmath.def --export-all-symbols libmath.dll
EXPORTS
[...]
div # 259
[...]
The libmath_c.c:
#include "libmath_c.h"
PyObject* _wrap_DIVISION(PyObject *self, PyObject *args){
div(10, 2);
return Py_None;
}
__declspec(dllexport) void __cdecl initDIVISION_CPP(void){
Py_InitModule("DIVISION_CPP", LIB_METHODS_methods);
}
The libmath_c.h:
#include <windows.h>
#include <stdio.h>
#include <Python.h>
PyObject* _wrap_DIVISION(PyObject *self, PyObject *args);
static PyMethodDef LIB_METHODS_methods[] = {
{ "CPP_DIVISION", _wrap_DIVISION, METH_VARARGS },
{NULL, NULL, 0, NULL} //Added as indicated by #Brian. Thanks!
};
__declspec(dllexport) void __cdecl initDIVISION_CPP(void);
Any idea of what is happening? Any help would be really appreciated. Thanks!

Preamble: Apologies if this turns out to be a non-answer; I want to be able to come back to this and find the links again, and comments tend to rot...
First, gcc (in the version matching Gnat) may work as an alternative C compiler, and if it does, it may eliminate difficulties with incompatible library versions.
GCC can be used for building Windows DLLs so the result should be usable from other Windows executables.
Following comments; gcc does appear to allow compilation, but the result is not currently usable from Python - here, my Python knowledge is shallow, and we don't have an MCVE, so this is speculative:
This Q&A addresses the same error message between Python and pure C, with no Ada, suggesting this error may not be specific to C-wrapped Ada.
You have already bypassed the asker's specific error,
static PyMethodDef* _npfindmethods = { ... };
which was using a pointer; you are (correctly according to the answer) statically allocating an array. However, the accepted answer terminates the list of methods
static PyMethodDef _npfindmethods[] = {
{"add", py_add, METH_VARARGS, py_add_doc},
{NULL, NULL, 0, NULL}
};
with a NULL method; your example does not:
static PyMethodDef LIB_METHODS_methods[] = {
{ "CPP_DIVISION", _wrap_DIVISION, METH_VARARGS }
};
So my hypothesis is that when you run setup() on this module, it finds CPP_DIVISION successfully, then in the absence of a NULL method it runs off into the weeds, producing the same symptoms despite the difference in cause.
I could test this hypothesis using the MCVE in that question by deleting the NULL method; however I don't have a Windows system handy, only Linux.
Alternatively, I see no reason for a C layer. If there isn't one, this Q&A addresses direct interaction between Python and Ada with no C layer, though it appears to use a different method, getattr() to import the external method. Might be an alternative?

Finally I managed to compile with gcc+gnat but not with MSVC+gnat.
With gcc+gnat, I was getting .//libmath.lib: error adding symbols: Malformed archive. The solution consists on using libmath.dll instead of building the .lib from the .dll.
So, in summary:
If you have a .dll generated by gnat, use it with gcc. You don't need to build a .lib.
If you have a .lib (for example python27.lib) or a .dll not generated by gnat, convert it to a .a using a tool like "pexport" (DO NOT USE SED!).
If you really need to compile using the MSVC... I'm sorry, I could not manage to make it work. Your princess is in another castle.

C++ Python Swig Compability with Scons and/or dynamic libraries

So what I want to do is fairly straight forward, I want to access a C++ class in Python using swig. I've managed to do this for stand alone applications but that's not enough.
What I have: a fairly huge C++ library which is compiled using SCons. This generates a static (lib---.a) library. I also have a small C++ class using some of the library functionality.
What I tried to do was letting SCons compile everything, including my custom class, and then do the swig magic. I defined an swig interface file including the structure of my header file which I also included in the swig interface file.
In the header file there are dependencies to the library, but since these already have been compiled (into the static lib) swig can't find it. Replacing SCons is not an option. However, I can make a dynamic library instead of a static one.
So my question boils down to this: is there a proper way of using Swig inside SCons, or, can you somehow include the library dependency as an existing dynamic library? Furthermore, I can (using SCons) either compile my custom class into an object (.o) file or a dynamic library (.so) as well.
Regarding the dynamic library, this is where things get messy since this is also what swig generates (?), or at least generated by someone.
Does this even make sense? I'm clearly on thin ice and don't necessarily know what I'm talking about.
Below are the c++ header and swig interface, I excluded c++ source since it felt redundant.
My Header File:
#include "../include/LogCabin/Client.h"
#include "../include/LogCabin/Debug.h"
#include "../include/LogCabin/Util.h"
class Ops{
private:
void dumpTree(std::string);
void dumpTreeLocal(std::string);
public:
Ops(std::string, uint64_t, std::string);
~Ops();
...
void makeLeader();
void getLeader();
int reconfigure(std::vector<std::string>);
};
My corresponding swig interface file:
%module Ops
%{
#include "Ops.h"
%}
%include "Ops.h"
class Ops{
public:
Ops(std::string, uint64_t, std::string);
~Ops();
...
void makeLeader();
void getLeader();
};
Commands used:
swig -c++ -python Ops.i;
g++ -std=c++11 -c -fPIC Ops_wrap.cxx -I/usr/include/python2.7 -I/usr/lib/python2.7;
g++ -std=c++11 -shared -Wl,-soname,_Ops.so -o _Ops.so Ops.o Ops_wrap.o;
Thanks. #swig
EDIT:
I include the SConscript called by SConstruct.
Import('env', 'object_files')
libs = [ "pthread", "protobuf", "rt", "cryptopp" ]
src = [
"Ops.cc",
"Ops.i"
]
/*object_files['Impl'] = env.StaticObject(src)*/
env.Default([
env.SharedLibrary("Ops", src,
LIBS = libs)
])
Edit2:
I updated the SContruct env declaration to set some parameters for swig.
env = Environment(options = opts,
SWIGFLAGS=['-c++','-python'],
CPPPATH=["/usr/include/python2.7"]),
LIBPATH=["/usr/lib/python2.7"]),
SHLIBPREFIX="",
tools = ['default', 'protoc', 'packaging'],
ENV = os.environ)
When compiling the generated Ops_wrap.cc file it gives me a lot of type cast warnings but still finish the compilation, e.g.
build/Impl/Ops_wrap.cc:654:23: warning: conversion to ‘unsigned char’ from ‘int’ may alter its value [-Wconversion]
uu = ((d - '0') << 4);
When trying to access the Ops class via the generated Ops.py file it gives me error when importing Ops:
Traceback (most recent call last):
File "test.py", line 3, in <module>
import Ops
ImportError: /home/erneborg/Applications/coherent/logcabin/build/Impl/Ops.so: undefined symbol: _ZNK8LogCabin6Client4Tree6readExERKSs
The undefined symbol corresponds to a function in the library "LogCabin::Client::Tree::readEx()". Thoughts?

How do I set constants at run-time in a c++ header file, imported through Cython?

I have some C++ code that currently relies on hard-coded constants, which are imported into multiple other cpp files, and I would like my python (pyx) file to set the constants once at runtime.
So, cython.pyx imports files a.cpp, b.cpp, and c.cpp, and constants.hpp
Files a.cpp, b.cpp, and c.cpp all import constants.hpp.
I would like instead to have one universal constants file, eg new_constants.yml, which python imports and sends through to the cpp files. This also means (I think) that I won't have to re-compile the c code every time I want to tweak the constants.
I'm used to scripting languages (python, js), so working with old C++ code is throwing me off a bit, and I'm sure parts of this question sound like I'm retarded, so, thanks for being patient with me.
These are just some weird dependencies, and I can't wrap my mind around unspooling it.

C++ literally inserts #include'd files into the code at compile time (technically before compile time - during the preprocessor run), so there is no way to change those values at runtime.
if you have the following
foo.h
const int value = 42;
and foo.cpp
#include "foo.h"
int foo(){ return value; }
When you compile foo.cpp, the preprocessor will substitute the exact contents of foo.h to replace #include "foo.h" in the cpp file and then the compiler will see
const int value = 42;
int foo(){ return value; }
and nothing else
The original source code for a c++ program is completely discarded once compilation is complete and is never used again.
You can see what the compiler sees using the -E flag to gcc which will make it output the pre-processed source.

Calling c++ from python using ctypes: g++ undefined reference

I've got a Camera (thorlabs dcc1645c) which comes with a "uc480.h" + "uc480.lib" to program it with c++. My problem is that I need to use the Camera with Python, so I tried to call a C++ Class with Python as told by Florian:
Calling C/C++ from python?
This is my c++ code (Cam2.cpp):
\#include "uc480.h"
class Cam{
public:
UC480_CAMERA_LIST* pucl;
int nNumberOfCameras;
int GetNumberOfCameras(){
return is_GetNumberOfCameras(&nNumberOfCameras);
}
};
extern "C" {
Cam* Cam_new(){ return new Cam(); }
int Cam_GetNumberOfCameras(Cam* cam){ cam->GetNumberOfCameras(); }
}
I tried to compile with g++:
g++ -fPIC -c Cam2.cpp -o Cam2.o
g++ -shared -Wl,-soname,libCam2.so -o libCam2.so Cam2.o
The second line gives an error:
Cam2.o:Cam2.cpp:(.text$_ZN3Cam18GetNumberOfCamerasEv[Cam::GetNumberOfCameras()]+
0x1a): undefined reference to `__imp_is_GetNumberOfCameras'
collect2.exe: error: ld returned 1 exit status
'is_GetNumberOfCameras' is defined in "uc480.lib", so in my opinion there is a problem with the linking. How do I fix it?
Also tell me please, if you now other possibilities to use the Camera with Python.

I am working from memory; it might be enough to simply add the lib file to the link command, as in
g++ -shared -Wl,-soname,libCam2.so -o libCam2.so uc480.lib Cam2.o
Or, you might need to add an option like -L. -luc480 to the link command, assuming the library is in the same directory as the code.

Thanks for your help so far. I found a solution for my problem. First of all here is my current c++ code:
#include "uc480.h"
class Cam{
public:
UC480_CAMERA_LIST* pucl = 0;
int nNumberOfCameras;
int GetNumberOfCameras(){
return is_GetNumberOfCameras(&nNumberOfCameras);
}
extern "C"{
__declspec(dllexport) Cam* Cam_New() { return new Cam(); }
__declspec(dllexport) int Cam_GetNumberOfCameras(Cam* obj){ return obj->GetNumberOfCameras(); }
}
I used Visual Studio to build a DLL. I followed the first steps of this instruction http://msdn.microsoft.com/de-de/library/ms235636.aspx :
To create a dynamic link library (DLL) project
On the menu bar, choose File, New, Project.
In the left pane of the New Project dialog box, expand Installed, Templates, Visual C++, and then select Win32.
In the center pane, select Win32 Console Application.
Specify a name for the solution — for example, MyDLL — in the Solution name box. Choose the OK button.
On the Overview page of the Win32 Application Wizard dialog box, choose the Next button.
On the Application Settings page, under Application type, select DLL.
Choose the Finish button to create the project.
After that I added my c++ code to the MyDLL.cpp and compiled to get MyDLL.dll
This can one use with python ctypes:
from ctypes import cdll
lib = cdll.LoadLibrary("PathToLib\MyDLL.dll")
class Camera(object):
def __init__(self):
self.obj = lib.Cam_New()
def GetNumberOfCameras(self):
return lib.Cam_GetNumberOfCameras(self.obj)
MyCam = Camera()
Numbers = MyCam.GetNumberOfCameras()
This works fine for me.