We Keep Coding

Mocking shared library (.so) functions in Python [duplicate]

New to cpp (Java guy).
I have 3rd party library that has method sendMail(txt).
I don't want to test the library. i want to test my own method, so in order to do this , i need to mock the library calls .
My own method is looking like this:
#include "mailsender.h"
int run(txt){
analysis(txt);
...
...
int status = sendMail(txt);//sendMail is a 3rd party library call. i need to mock it.its not part of the unit test
return status;
}
In Java the mailsender was interface and it was injected to my class, so in case of test i inject mock.
What is a good practice in cpp to mock library calls?
I can wrap the 3rd party library call in a class and inject this class, but i am looking for something simpler and for the common practice (maybe ifndf).
I am familiar with googlemock.
googlemock allow me to mock classes . i am not aware to option how to mock a call in my tested method.

So I assume you have a 'global' function that is implemented in a library that you both include a header file for (to get the definition) and link (to get the implementation).
You obviously need to replace the implementation of the library with your own - one that does "nothing", so you can do this in 2 ways:
you replace the .dll (or .so) with your own implementation that has all the methods the 3rd party library exposes. This is easy once you've written a new version of all the 3rd party lib functions, but writing them all out can be a pain.
you remove the library temporarily, and replace the calls you make to that in a .cpp source file that implements those functions. So you'd create your own sendMail() function in a .cpp file and include this into the program instead of the mailsender.h include.
The latter is easier, but you might also have to modify your program to not link with the 3rd party lib. This can also require changing the #include as well, as some compilers (eg VC++) allow you to embed linker directives in the source. If your does this, then you won't be able to stop the linker from including the 3rd party lib.
The other option is to modify your code to use a different call to the sendMail call, eg test__sendMail() that you implement yourself. Wrap this is a macro to conditionally include your, or the real, function call depending on your build options.
If this was a c++ library then you'd probably be able to use a mocking framework like you're used to, but it sounds like its a C library, and they simply provide a list of functions that you use directly in your code. You could wrap the library in your own class and use that instead of calling the 3rd party lib functions directly.
There is a list of C mocking frameworks.

This is an old question, with an already choosen response, but maybe the following contribution can help someone else.
First solution
You still have to create a custom library to redefine the functions, but you do not need to change Makefiles to link to your "fake-library", just use LD_PRELOAD with the path to the fake-library and that will be the first that the linker will find and then use.
example
Second solution
ld (GNU) linker has an option --wrap that let you wrap only one function with another provided by the user. This way you do not have to create a new library/class just to mock the behavior
Here is the example from the man page
--wrap=symbol
Use a wrapper function for symbol. Any undefined reference to symbol will be resolved to "__wrap_ symbol ". Any undefined reference
to "__real_ symbol " will be resolved to symbol.
This can be used to provide a wrapper for a system function. The wrapper function should be called "__wrap_ symbol ". If it wishes to
call the system function, it should call "__real_ symbol ".
Here is a trivial example:
void *
__wrap_malloc (size_t c)
{
printf ("malloc called with %zu\n", c);
return __real_malloc (c);
}
If you link other code with this file using --wrap malloc, then all calls to "malloc" will call the function "__wrap_malloc" instead.
The call to "__real_malloc" in "__wrap_malloc" will call the real
"malloc" function.
You may wish to provide a "__real_malloc" function as well, so that links without the --wrap option will succeed. If you do this, you
should not put the definition of "__real_malloc" in the same file as
"__wrap_malloc"; if you do, the assembler may resolve the call before
the linker has a chance to wrap it to "malloc".

Disclaimer: I wrote ELFspy.
Using ELFspy, the following code will allow you to fake/mock the sendMail function by replacing it with an alternative implementation.
void yourSendMail(const char* txt) // ensure same signature as sendMail
{
// your mocking code
}
int main(int argc, char** argv)
{
spy::initialise(argc, argv);
auto sendMail_hook = SPY(&sendMail); // grab a hook to sendMail
// use hook to reroute all program calls to sendMail to yourSendMail
auto sendMail_fake = spy::fake(sendMail_hook, &yourSendMail);
// call run here..
}
Your program must be compiled with position independent code (built with shared libraries) to achieve this.
Further examples are here:
https://github.com/mollismerx/elfspy/wiki

Though there is no interface keyword, you can use Abstract Base Classes for similar things in C++.
If the library you are using doesn't come with such abstractions, you can wrap it behind your own "interface". If your code separates construction of objects from usage (e.g. by IoC), you can either use this to inject a fake or use Mocks:
https://stackoverflow.com/questions/38493/are-there-any-good-c-mock-object-frameworks

Wrapping shared pointer object with SWIG don't give access to class member functions

I have a C++ CameraManager class that manages a list of Camera objects.
The camera objects are managed by a std::list, as shared pointers, i.e. each list item is of type: shared_ptr < Camera > .
I can obtain a Camera from a CameraManager object as
std::shared_ptr<Camera> c = cameraManager.getCamera();
Creating a Python module using Swig, the above is translated to python code as:
camera = cameraManager.getCamera()
The camera Python object above, however, don't allow me to access any of the Camera classes functions. Python says is an object of type: 'Swig object of type 'std::shared_ptr *' at ####
Adding the following in the Swig interface file
%include <std_shared_ptr.i>
%shared_ptr(Camera)
before including
%include "aiCamera.h"
don't change the behavior in the resulting Python module. Any ideas what might be missing?
Additional info:
The C++ code uses a typedef:
typedef CameraSP std::shared_ptr<Camera>;
The Camera class is derived from a base class, that is virtually empty.
class MVR_API MVRObject
{
public:
MVRObject();
MVRObject(const MVRObject& obj);
virtual ~MVRObject();
};
The code is compiled using VS 2013 and CMake. The CMake code looks like this:
set_source_files_properties(${PyModule}.i PROPERTIES CPLUSPLUS ON)
set_source_files_properties(${PyModule}.i PROPERTIES SWIG_FLAGS "-threads")
SWIG_ADD_LIBRARY(${PyModule}
TYPE MODULE
LANGUAGE python
SOURCES ${PyModule}.i
)
SWIG_LINK_LIBRARIES (${PyModule}
${PYTHON_LIB_FOLDER}/Python37.lib
dslFoundation
aimvr
)
# INSTALL PYTHON BINDINGS
# Get the python site packages directory by invoking python
execute_process(COMMAND python -c "import site; print(site.getsitepackages()[0])" OUTPUT_VARIABLE PYTHON_SITE_PACKAGES OUTPUT_STRIP_TRAILING_WHITESPACE)
message("PYTHON_SITE_PACKAGES = ${PYTHON_SITE_PACKAGES}")
SET(SWIG_RUNTIME ${CMAKE_CURRENT_BINARY_DIR}/mvr_swigpyrun.h)
execute_process(COMMAND ${SWIG_EXECUTABLE} -python -external-runtime ${SWIG_RUNTIME})
UPDATE:
The problem was not Swig and usage of shared pointers. It was a third party library having its own implementation of shared_ptr. Resolving the shared pointers by namespace names fixed the problem and the resulting Python module started working as expected.

The question above are dealing with a C/C++ API from Allied Vision, controlling their cameras. They have defined their own shared pointer class and named it using the same name as the std shared pointer class, i.e. shared_ptr.
The client code for this API is using std::shared_ptr's, and at some point the usage of a shared_ptr, without specifying the namespace caused the above problems with Swig.
By explicitly specifying the std namespace when using a shared_ptr, the problems were resolved and the resulting swigged objects, using shared pointers, started working perfectly fine.

Cython: std::function callbacks with custom parameter types

Please read this post before answering: Pass a closure from Cython to C++
In the accepted answers, it is neatly shown how a python function is converted into a std::function using Boost Python.
Following this example I'm able to wrap functions taking an std::function as an argument and call them using a python function as input. However, this only works when the std::function parameters are primitives like int, double, string etc.
Any guidance on how to make this work for custom types as well will be highly appreciated.

This won't be a complete answer - it assumes you can fill in the gaps from my previous answer that the question was based on. Unfortunately it is a little bit more complicated than that case.
Just to define the problem - assume you have a parameter of a custom C++ class, like:
class cpp_class {
// some non-trivial contents
};
and thus your C++ interface looks like this:
void call_some_std_func(std::function<void(cpp_class&)> callback) {
callback(5,std::string("hello"));
}
The first thing to do is to write a Cython wrapper for your C++ class (in principle you could make a Boost Python wrapper instead). Here you need to make a choice of about "ownership" of the C++ object. The first choice is to make a copy:
cdef extern from "cpp_file.hpp":
cppclass cpp_class:
pass # details
cdef class CyWrapper:
cdef cpp_class* ptr
def __dealloc__(self):
del self.ptr
# other details following standard wrapper pattern
cdef public make_CyWrapper(cpp_class& x):
obj = CyWrapper()
obj.ptr = new cpp_class(x)
return obj
I've created a wrapper class with a destructor that handles the memory and a publicly accessible constructor function that can be called from external code. This version is safe because your wrapper owns the object it holds and so there can be no writes to invalid memory. However, because it makes a copy, you can't make changes to the original C++ object.
A second option is to hold a pointer to an object you don't own. The code is basically identical except you remove the __dealloc__ and avoid making a copy in make_CyWrapper:
obj.ptr = &x // instead of new cpp_class(x)
This is unsafe - you need to ensure your C++ object outlives the Cython wrapper - but allows you to modify the object.
You could also imagine a few other options: you could take ownership of an existing object with your Cython wrapper (Such a scheme would have to pass by pointer rather than reference, or it could use move constructors); you could deconstruct your C++ class into a representation expressed in basic types and pass those to Python; you could use shared pointers to split the ownership; or you have a more elaborate way of marking your Cython wrapper as "invalid" once your held C++ instance is destructed.
What you do next depends on whether you're using Boost Python (for it's convenient, callable wrapping of Python objects) or if you're making your own version. (I showed both possibilities in the previous answer).
Assuming Boost Python, you need to do two things - tell it about the conversion and make sure that it imports the module that your wrapper is defined in (if you don't do this you get exciting segmentation faults)
struct convert_to_PyWrapper {
static PyObject* convert(const cpp_class& rhs) {
// the const_cast here is a bit dodgy, but was needed to make it work
return make_CyWrapper(const_cast<cpp_class&>(rhs));
}
};
inline void setup_boost_python() {
PyInit_your_module_name(); // named inityour_module_name in Python 2
boost::python::to_python_converter<
cpp_class,
convert_to_PyWrapper>();
}
You need to make sure that your Python/Cython code calls "setup_boost_python" before attempting to use the callback (if you put it at module level it's done on import, which is ideal).
If you're following my "manual" scheme (avoiding the dependency on Boost Python) then you need to modify the call_obj Cython function that does the C++ to Cython type conversion.
cdef public void call_obj(obj, cpp_class& c):
obj(make_CyWrapper(c))
You also need to ensure the wrapper Cython module is imported before use (otherwise you get segmentation faults). I did this in "py_object_wrapper.hpp" but providing it's done once somewhere you can place it where you like.
void operator()(cpp_class& a) {
PyInit_your_module_name();
if (held) {
call_obj(held,a);
}
}

Python NumPy arrays to boost::multi_array numpy_boost setup to interface between C++ and NumPy

I have a C/C++ DLL with libraries I need to call from NumPy, where the format inside the DLL is all boost::multi_array format. While I have seen some posts involving this project: https://github.com/mdboom/numpy-boost
there really is very little documentation and examples from users out there on the steps required to wrap the boost::multi_array to NumPy interface with this library. My questions: while the examples are enough to guess the C++ part of the interface (other than strings, how is this handled? Q1), what do you have to do on the Python side to use the compiled DLL (Q2)? It seems Ctypes wouldn't work (correct me if I'm wrong) so does this have to be rewritten in Cython?
For reference, this is the C part with all the datatypes I'm trying to wrap:
extern "C"
{
DECLDIR void Cfunction(
boost::multi_array<double, 2>& p_result,
const vector<string>& p_calcType,
const string& p_optionType,
unsigned long p_nTimeStep = 50,
const vector<double>& p_premium = vector<double>());

With ctypes you can load DLL dynamically and call functions from it. From my point of view, it's easiest way for using compiled code within Python code. Here is how: How can I use a DLL file from Python?
numpy allows you to get pointer to memory location. Look at that: http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.ndarray.ctypes.html
If you don't have source code of your DLL and cannot change its interface, you create C++ wrapper (your own separate DLL) that accepts raw pointer from Python instead of boost::multi_array.

Embedding Python: How to use custom type inside Python script?

I try to run some Python scripts from inside the C++ code. I reach the point, in which I need to use my custom type. I found article in Python doc about creating custom types and nice SOQ, explaining how to create instances of custom type on C++ side.
I am not sure, however, how am I suppose to use this type in Python. In doc sample, a 'module initializer' is defined:
PyMODINIT_FUNC PyInit_module_type(void)
{
CX_type.tp_new = PyType_GenericNew;
if (PyType_Ready(&CX_type) < 0)
return NULL;
//create module, return it
}
But there is no hint what is purpose of this function. How (and when) this function is called?
Currently, I run my scripts either by PyEval_EvalCode() to run whole script or PyObject_Call() to run specific function. How do I use my type in both cases? Do I need to import it first somehow?
If I import my scripts as modules:
PyObject* pm_1 = PyImport_Import("pm_1.py")
do I need to add my type to each module I create this way:
Py_INCREF(&CX_type);
PyModule_AddObject(pm_1, "CX", (PyObject*)&CX_type);
? I think, that types created after Py_Initialize() (so, during single interpreter session) should be visible automatically to all modules imported during this session. Am I wrong?