We Keep Coding

Shared library vs. opening a process performance

I have a certain base of Python code (a Flask server). I need the server to perform a performance-critical operation, which I've decided to implement in C++. However, as the C++ part has other dependencies, trying out ctypes and boost.python yielded no results (not found symbols, other libraries even when setting up the environment, etc., basically, there were problems). I believe a suitable alternative would be for me to just compile the C++ part into an executable (a single function/procedure is required) and run it from python using commands or subprocess, communicating through stdin/out for example. The only thing I'm worried about is that this will slow down the procedure enough to matter and since I'm unable to create a shared object library, calling its function from python directly, I cannot benchmark the speedup.
When I compile the C++ code into an executable and run it with some sample data, the program takes ~5s to run. This does not account for opening the process from python, nor for passing data between the processes.
The question is: How big of a speedup can one expect by using ctypes/boost with a SO compared to creating a new process to run the procedure? If I regard the number to be big enough, it would motivate me to solve the encountered problems, basically, I'm asking if it's worth it.

If you're struggling with creating binding using Boost.Python, you can manually expose your API via c-functions and use them via FFI.
Here's a simple example, which briefly explains my idea. At first, you create a shared library, but add some extra functions here, which in the example I put into extern "C" section. It's necessary to use extern "C" since otherwise function names will be mangled and their actual names are likely to be different from those you've declared:
#include <cstdint>
#include <cstdio>
#ifdef __GNUC__
#define EXPORT __attribute__ ((visibility("default")))
#else // __GNUC__
#error "Unsupported compiler"
#endif // __GNUC__
class data_processor {
public:
data_processor() = default;
void process_data(const std::uint8_t *data, std::size_t size) {
std::printf("processing %zu bytes of data at %p\n", size, data);
}
};
extern "C" {
EXPORT void *create_processor() {
return new data_processor();
}
EXPORT void free_processor(void *data) {
delete static_cast<data_processor *>(data);
}
EXPORT void process_data(void *object, const std::uint8_t *data, const std::uint32_t size) {
static_cast<data_processor *>(object)->process_data(data, size);
}
}
Then you create function bindings in python. As you can see function declarations are almost the same as they are in the cpp file below. I used built-in types only (like void *, uint8_t and anything, but I believe FFI allows you to declare and use custom structs as well):
from cffi import FFI
mylib_api = FFI()
mylib_api.cdef("""
void *create_processor();
void free_processor(void *object);
void process_data(void *object, const uint8_t *data, uint32_t size);
""")
mylib = mylib_api.dlopen("mylib.so-file-location")
processor = mylib.create_processor()
try:
buffer = b"buffer"
mylib.process_data(processor, mylib_api.from_buffer("uint8_t[]", python_buffer=buffer), len(buffer))
finally:
mylib.free_processor(processor)
And that's basically it.
In my opinion inter-processing is going to be the last resort when nothing else works since:
you need to put a lot of efforts implementing details of your communication protocol, either if you use something popular, there could be a lot of issues, especially from c++-side;
inter-process communication is generally more expensive in terms of processor time.

Interoperability between C++ Boost Multiprecision and Python's mpmath

I have experience working with both Boost Multiprecision and with Python's mpmath, separately.
When it gets to making both communicate (for example to create Python extensions in C++), my attempts have always involved some sort of wasteful float-to-string and string-to-float conversion.
My question is: is it possible to make both communicate in a more performant (and elegant) way? And by that I mean, is there a way to directly have C++ Boost Multiprecision load from and export to a Python mpmath.mpf object in the same vein as C's mpp does via pybind11?
I have been searching for this for quite a bit. The only other similar question I found was about just exporting from Boost Multiprecision to Python (in general) using pybind11, not to a mpmath object directly. And in that question, the OP ended up using the same approach I am trying to avoid (that is, converting from/to strings when communicating from/to C++ and Python).

This answers only partially to your question. Because the direct answer is: No, it is not possible in a clean way without a wasteful conversion to string, because mpmath is a purely python library without any parts of it written in C or C++, hence even if you try to skip "wasteful conversion" by seeking to use some sort of binary compatibility, your code will be very fragile: it will break every time when some python or mpmath internals are changed ever so slightly.
However I needed exactly the same thing. And so I settled down for an automated conversion registered via boost::python which checks and converts using strings. Actually inside python you also create mpmath.mpf objects from strings, so it's very much the same, except in the code below it is faster because it is written inside C++.
So here's what works for me:
#include <boost/python.hpp>
#include <iostream>
#include <limits>
#include <sstream>
#include <boost/math/constants/constants.hpp>
#include <boost/multiprecision/cpp_bin_float.hpp>
namespace py = ::boost::python;
using Prec80 = boost::multiprecision::number<boost::multiprecision::cpp_bin_float<80>>;
template<typename ArbitraryReal>
struct ArbitraryReal_to_python {
static PyObject* convert(const ArbitraryReal& val){
std::stringstream ss{};
ss << std::setprecision(std::numeric_limits<ArbitraryReal>::digits10+1) << val;
py::object mpmath = py::import("mpmath");
mpmath.attr("mp").attr("dps")=int(std::numeric_limits<ArbitraryReal>::digits10+1);
py::object result = mpmath.attr("mpf")(ss.str());
return boost::python::incref( result.ptr() );
}
};
template<typename ArbitraryReal>
struct ArbitraryReal_from_python {
ArbitraryReal_from_python(){
boost::python::converter::registry::push_back(&convertible,&construct,boost::python::type_id<ArbitraryReal>());
}
static void* convertible(PyObject* obj_ptr){
// Accept whatever python is able to convert into float
// This works with mpmath numbers. However if you want to accept strings as numbers this checking code can be a little longer to verify if string is a valid number.
double check = PyFloat_AsDouble(obj_ptr);
return (PyErr_Occurred()==nullptr) ? obj_ptr : nullptr;
}
static void construct(PyObject* obj_ptr, boost::python::converter::rvalue_from_python_stage1_data* data){
std::istringstream ss{ py::call_method<std::string>(obj_ptr, "__str__") };
void* storage=((boost::python::converter::rvalue_from_python_storage<ArbitraryReal>*)(data))->storage.bytes;
new (storage) ArbitraryReal;
ArbitraryReal* val=(ArbitraryReal*)storage;
ss >> *val;
data->convertible=storage;
}
};
struct Var
{
Prec80 value{"-71.23"};
Prec80 get() const { return value; };
void set(Prec80 val) { value = val; };
};
BOOST_PYTHON_MODULE(pysmall)
{
ArbitraryReal_from_python<Prec80>();
py::to_python_converter<Prec80,ArbitraryReal_to_python<Prec80>>();
py::class_<Var>("Var" )
.add_property("val", &Var::get, &Var::set);
}
Now you compile this code with this command:
g++ -O1 -g pysmall.cpp -o pysmall.so -std=gnu++17 -fPIC -shared -I/usr/include/python3.7m/ -lboost_python37 -lpython3.7m -Wl,-soname,"pysmall.so"
And here is an example python session:
In [1]: import pysmall
In [2]: a=pysmall.Var()
In [3]: a.val
Out[3]: mpf('-71.2299999999999999999999999999999999999999999999999999999999999999999999999999997072')
In [4]: a.val=123.12
In [5]: a.val
Out[5]: mpf('123.120000000000000000000000000000000000000000000000000000000000000000000000000000003')
The C++ code does not care whether mpmath is already imported in python. If it is, it obtains the exsiting library handle, if it is not then it imports it.
If you find any room for improvement in this snippet please let me know!
Here's a couple of useful references when I was writing this:
https://misspent.wordpress.com/2009/09/27/how-to-write-boost-python-converters/
https://github.com/bluescarni/mppp/blob/master/include/mp%2B%2B/extra/pybind11.hpp (but I didn't want to use pybind11, just boost::python)
EDIT: I have now finished implementing this in YADE , it works with EIGEN and CGAL libraries. The part concerning this question is in file ToFromPythonConverter.hpp

Adding swig pythoncode to set thisown flag on Python object

I have a swigged C++ class container, MyContainer, holding objects of type MyObject, also a C++ class.
The following is the C++ header code (freemenot.h)
#ifndef freemenotH
#define freemenotH
#include <vector>
#include <string>
using std::string;
class MyObject
{
public:
MyObject(const string& lbl);
~MyObject();
string getLabel();
private:
string label;
};
class MyContainer
{
public:
MyContainer();
~MyContainer();
void addObject(MyObject* o);
MyObject* getObject(unsigned int t);
int getNrOfObjects();
private:
std::vector<MyObject*> mObjects;
};
#endif
and this is the source (freemenot.cpp)
#include "freemenot.h"
#include <iostream>
using namespace std;
/* MyObject source */
MyObject::MyObject(const string& lbl)
:
label(lbl)
{ cout<<"In object ctor"<<endl; }
MyObject::~MyObject() { cout<<"In object dtor"<<endl; }
string MyObject::getLabel() { return label; }
/* MyContainer source */
MyContainer::MyContainer() { cout<<"In container ctor"<<endl; }
MyContainer::~MyContainer()
{
cout<<"In container dtor"<<endl;
for(unsigned int i = 0; i < mObjects.size(); i++)
{
delete mObjects[i];
}
}
int MyContainer::getNrOfObjects() { return mObjects.size(); }
void MyContainer::addObject(MyObject* o) { mObjects.push_back(o); }
MyObject* MyContainer::getObject(unsigned int i) { return mObjects[i]; }
Observe that the objects are stored as RAW POINTERS in the vector. The class is such designed, and the container is thus responsible to free the objects in its destructor, as being done in the destructors for loop.
In C++ code, like below, an object o1 is added to the container c, which is returned to client code
MyContainer* getAContainerWithSomeObjects()
{
MyContainer* c = new MyContainer();
MyObject* o1 = new MyObject();
c.add(o1);
return c;
}
The returned container owns its objects, and are responsible to de-allocate these objects when done. In C++, access to the containers objects is fine after the function exits above.
Exposing the above classes to python, using Swig, will need an interface file. This interface file looks like this
%module freemenot
%{ #include "freemenot.h" %}
%include "std_string.i"
//Expose to Python
%include "freemenot.h"
And to generate a Python module, using CMake, the following CMake script was used.
cmake_minimum_required(VERSION 2.8)
project(freemenot)
find_package(SWIG REQUIRED)
include(UseSWIG)
find_package(PythonInterp)
find_package(PythonLibs)
get_filename_component(PYTHON_LIB_FOLDER ${PYTHON_LIBRARIES} DIRECTORY CACHE)
message("Python lib folder: " ${PYTHON_LIB_FOLDER})
message("Python include folder: " ${PYTHON_INCLUDE_DIRS})
message("Python libraries: " ${PYTHON_LIBRARIES})
set(PyModule "freemenot")
include_directories(
${PYTHON_INCLUDE_PATH}
${CMAKE_CURRENT_SOURCE_DIR}
)
link_directories( ${PYTHON_LIB_FOLDER})
set(CMAKE_MODULE_LINKER_FLAGS ${CMAKE_CURRENT_SOURCE_DIR}/${PyModule}.def)
set_source_files_properties(${PyModule}.i PROPERTIES CPLUSPLUS ON)
set_source_files_properties(${PyModule}.i PROPERTIES SWIG_FLAGS "-threads")
SWIG_ADD_LIBRARY(${PyModule}
MODULE LANGUAGE python
SOURCES ${PyModule}.i freemenot.cpp)
SWIG_LINK_LIBRARIES (${PyModule} ${PYTHON_LIB_FOLDER}/Python37_CG.lib )
# INSTALL PYTHON BINDINGS
# Get the python site packages directory by invoking python
execute_process(COMMAND python -c "import site; print(site.getsitepackages()[0])" OUTPUT_VARIABLE PYTHON_SITE_PACKAGES OUTPUT_STRIP_TRAILING_WHITESPACE)
message("PYTHON_SITE_PACKAGES = ${PYTHON_SITE_PACKAGES}")
install(
TARGETS _${PyModule}
DESTINATION ${PYTHON_SITE_PACKAGES})
install(
FILES ${CMAKE_CURRENT_BINARY_DIR}/${PyModule}.py
DESTINATION ${PYTHON_SITE_PACKAGES}
)
Generating the make files using CMake, and compiling using borlands bcc32 compiler, a Python module (freemenot) is generated and installed into a python3 valid sitepackages folder.
Then, in Python, the following script can be used to illuminate the problem
import freemenot as fmn
def getContainer():
c = fmn.MyContainer()
o1 = fmn.MyObject("This is a label")
o1.thisown = 0
c.addObject(o1)
return c
c = getContainer()
print (c.getNrOfObjects())
#if the thisown flag for objects in the getContainer function
#is equal to 1, the following call return an undefined object
#If the flag is equal to 0, the following call will return a valid object
a = c.getObject(0)
print (a.getLabel())
This Python code may look fine, but don't work as expected. Problem is that, when the function getContainer() returns, the memory for object o1 is freed, if the thisown flag is not set to zero. Accessing the object after this line, using the returned container will end up in disaster. Observe, there is not nothing wrong with this per se, as this is how pythons garbage collection works.
For the above use case being able to set the python objects thisown flag inside the addObject function, would render the C++ objects usable in Python.
Having the user to set this flag is no good solution.
One could also extend the python class with an "addObject" function, and modifying the thisown flag inside this function, and thereby hiding this memory trick from the user.
Question is, how to get Swig to do this, without extending the class?
I'm looking for using a typemap, or perhaps %pythoncode, but I seem not able to find a good working example.
The above code is to be used by, and passed to, a C++ program that is invoking the Python interpreter. The C++ program is responsible to manage the memory allocated in the python function, even after PyFinalize().
The above code can be downloaded from github https://github.com/TotteKarlsson/miniprojects

There are a number of different ways you could solve this problem, so I'll try and explain them each in turn, building on a few things along the way. Hopefully this is useful as a view into the options and innards of SWIG even if you only really need the first example.
Add Python code to modify thisown directly
The solution most like what you proposed relies on using SWIG's %pythonprepend directive to add some extra Python code. You can target it based on the C++ declaration of the overload you care about, e.g.:
%module freemenot
%{ #include "freemenot.h" %}
%include "std_string.i"
%pythonprepend MyContainer::addObject(MyObject*) %{
# mess with thisown
print('thisown was: %d' % args[0].thisown)
args[0].thisown = 0
%}
//Expose to Python
%include "freemenot.h"
Where the only notable quirk comes from the fact that the arguments are passed in using *args instead of named arguments, so we have to access it via position number.
There are several other places/methods to inject extra Python code (provided you're not using -builtin) in the SWIG Python documentation and monkey patching is always an option too.
Use Python's C API to tweak thisown
The next possible option here is to use a typemap calls the Python C API to perform the equivalent functionality. In this instance I've matched on the argument type and argument name, but that does mean the typemap here would get applied to all functions which receive a MyObject * named o. (The easiest solution here is to make the names describe the intended semantics in the headers if that would over-match currently which has the side benefit of making IDEs and documentation clearer).
%module freemenot
%{ #include "freemenot.h" %}
%include "std_string.i"
%typemap(in) MyObject *o {
PyObject_SetAttrString($input, "thisown", PyInt_FromLong(0)); // As above, but C API
$typemap(in,MyObject*); // use the default typemap
}
//Expose to Python
%include "freemenot.h"
The most noteworthy point about this example other than the typemap matching is the use of $typemap here to 'paste' another typemap, specifically the default one for MyObject* into our own typemap. It's worth having a look inside the generated wrapper file at a before/after example of what this ends up looking like.
Use SWIG runtime to get at SwigPyObject struct's own member directly
Since we're already writing C++ instead of going via the setattr in the Python code we can adapt this typemap to use more of SWIG's internals and skip a round-trip from C to Python and back to C again.
Internally within SWIG there's a struct that contains the details of each instance, including the ownership, type etc.
We could just cast from PyObject* to SwigPyObject* ourselves directly, but that would require writing error handling/type checking (is this PyObject even a SWIG one?) ourselves and become dependent on the details of the various differing ways SWIG can produce Python interfaces. Instead there's a single function we can call which just handles all that for us, so we can write our typemap like this now:
%module freemenot
%{ #include "freemenot.h" %}
%include "std_string.i"
%typemap(in) MyObject *o {
// TODO: handle NULL pointer still
SWIG_Python_GetSwigThis($input)->own = 0; // Safely cast $input from PyObject* to SwigPyObject*
$typemap(in,MyObject*); // use the default typemap
}
//Expose to Python
%include "freemenot.h"
This is just an evolution of the previous answer really, but implemented purely in the SWIG C runtime.
Copy construct a new instance before adding
There are other ways to approach this kind of ownership problem. Firstly in this specific instance your MyContainer assumes it can always call delete on every instance it stores (and hence owns in these semantics).
The motivating example for this would be if we were also wrapping a function like this:
MyObject *getInstanceOfThing() {
static MyObject a;
return &a;
}
Which introduces a problem with our prior solutions - we set thisown to 0, but here it would already have been 0 and so we still can't legally call delete on the pointer when the container is released.
There's a simple way to deal with this that doesn't require knowing about SWIG proxy internals - assuming MyObject is copy constructable then you can simply make a new instance and be sure that no matter where it came from it's going to be legal for the container to delete it. We can do that by adapting our typemap a little:
%module freemenot
%{ #include "freemenot.h" %}
%include "std_string.i"
%typemap(in) MyObject *o {
$typemap(in,MyObject*); // use the default typemap as before
$1 = new $*1_type(*$1); // but afterwards call copy-ctor
}
//Expose to Python
%include "freemenot.h"
The point to note here is the use of several more SWIG features that let us know the type of the typemap inputs - $*1_type is the type of the typemap argument dereferenced once. We could have just written MyObject here, as that's what it resolves to but this lets you handle things like templates if your container is really a template, or re-use the typemap in other similar containers with %apply.
The thing to watch for here now is leaks if you had a C++ function that you were deliberately allowing to return an instance without thisown being set on the assumption that the container would take ownership that wouldn't now hold.
Give the container a chance to manage ownership
Finally one of the other techniques I like using a lot isn't directly possible here as currently posed, but is worth mentioning for posterity. If you get the chance to store some additional data along side each instance in the container you can call Py_INCREF and retain a reference to the underlying PyObject* no matter where it came from. Provided you then get a callback at destruction time you can also call Py_DECREF and force the Python runtime to keep the object alive as long as the container.
You can also do that even when it's not possible to keep a 1-1 MyObject*/PyObject* pairing alive, by keeping a shadow container alive somewhere also. That can be hard to do unless you're willing to add another object into the container, subclass it or can be very certain that the initial Python instance of the container will always live long enough.

You're looking for %newobject. Here's a small example:
%module test
%newobject create;
%delobject destroy;
%inline %{
#include <iostream>
struct Test
{
Test() { std::cout << "create" << std::endl; }
~Test() { std::cout << "destroy" << std::endl; }
};
Test* create() { return new Test; }
void destroy(Test* t) { delete t; }
%}
Use:
>>> import test
>>> t1 = test.create() # create a test object
create
>>> t2 = test.Test() # don't really need a create function :)
create
>>> t3 = test.create() # and another.
create
>>> test.destroy(t2) # explicitly destroy one
destroy
>>>
>>>
>>>
>>> ^Z # exit Python and the other two get destroyed.
destroy
destroy

I just wanted thisown to be set to zero in the constructor. I did it in two ways
I simply added one line sed statement to my makefile to add 'self.thisown = 0' at the end of init() function of my class.
Using pythonappend. I figured out two caveats (a) %pythonappend statement has to be placed before c++ class definition, (b) c++ constructor overloads do not matter
%pythonappend MyApp::MyApp() %{
self.thisown = 0
%}
%include <MyApp.hpp>

Returning arguments in SWIG/Python

According to Swig docs and the marvelous explanation at SWIG in typemap works, but argout does not by #Flexo, the argout typemap turns reference arguments into return values in Python.
I have a scenario, in which I pass a dict, which then is converted to an unordered_map in typemap(in), which then gets populated in the C++ lib. Stepping through the code, I can see the mapping changed after it returned from C++, so I wonder why there is not a possibility to just convert the unordered_map back in place in to the dict that was passed. Or is it possible by now and I'm just overlooking something?
Thanks!

I am a little confused as to what exactly you are asking, but my understanding is:
You have an "in" typemap to convert a Python dict to a C++ unordered_map for some function argument.
The function then modifies the unordered_map.
After completion of the function, you want the Python dict updated to the current unordered_map, and are somehow having trouble with this step.
Since you know how to convert a dict to an unordered_map, I assume you basically do know how to convert the unordered_map back to the dict using the Python C-API, but are somehow unsure into which SWIG typemap to put the code.
So, under these assumptions, I'll try to help:
"the argout typemap turns reference arguments into return values in Python". Not really, although it is mostly used for this purpose. An "argout" typemap simply supplies code to deal with some function argument (internally referred to as $1) that is inserted into the wrapper code after the C++ function is called. Compare this with an "in" typemap that supplies code to convert the supplied Python argument $input to a C++ argument $1, which is obviously inserted into the wrapper code before the C++ function is called.
The original passed Python dict can be referred to in the "argout" typemap as $input, and the modified unordered_map as $1 (see the SWIG docs linked above).
Therefore, all you need to do is write an "argout" typemap for the same argument signature as the "in" typemap that you already have, and insert the code (using the Python C-API) to update the contents of the Python dict ($input) to reflect the contents of the unordered_map ($1).
Note that this is different from the classical use of "argout" typemaps, which would typically convert the $1 back to a new Python dict and append this to the Python return object, which you can refer to by $result.
I hope this helps. If you are still stuck at some point, please edit your question to make clear at which point you are having trouble.

I am well aware of that the user has already solved his issue, but here goes a solution. Some validation of inputs may be introduced to avoid non-string values of the input dictionary.
Header file
// File: test.h
#pragma once
#include <iostream>
#include <string>
#include <unordered_map>
void method(std::unordered_map<std::string, std::string>* inout) {
for( const auto& n : (*inout) ) {
std::cout << "Key:[" << n.first << "] Value:[" << n.second << "]\n";
}
(*inout)["BLACK"] = "#000000";
};
Interface file
// File : dictmap.i
%module dictmap
%{
#include "test.h"
%}
%include "typemaps.i"
%typemap(in) std::unordered_map<std::string, std::string>* (std::unordered_map<std::string, std::string> temp) {
PyObject *key, *value;
Py_ssize_t pos = 0;
$1 = &temp;
temp = std::unordered_map<std::string, std::string>();
while (PyDict_Next($input, &pos, &key, &value)) {
(*$1)[PyString_AsString(key)] = std::string(PyString_AsString(value));
}
}
%typemap(argout) std::unordered_map<std::string, std::string>* {
$result = PyDict_New();
for( const auto& n : *$1) {
PyDict_SetItemString($result, n.first.c_str(),
PyString_FromString(n.second.c_str()));
}
}
%include "test.h"
Test
import dictmap
out = dictmap.method({'WHITE' : '#FFFFFF'})
Output is an updated dictionary
In[2]: out
Out[3] : {'BLACK': '#000000', 'WHITE': '#FFFFFF'}

Using SWIG to pass C++ object pointers to Python, than back to C++ again

I'm using SWIG to wrap 2 C++ objects, and I am embedding the Python interpreter in my application (i.e. calling PyInitialize() etc myself).
The first object is a wrapper for some application data.
The second is a "helper" object, also written in C++, which can perform certain operation based on what it finds in the data object.
The python script decides when/how/if to invoke the helper object.
So I pass a pointer to my C++ object to SWIG/Python thus:
swig_type_info *ty = SWIG_MangledTypeQuery("_p_MyDataObject");
if(ty == NULL)
{
Py_Finalize();
return false;
}
PyObject *data_obj = SWIG_NewPointerObj(PointerToMyDataObject, ty, 0);
if(data_obj == NULL)
{
Py_Finalize();
return false;
}
ty = SWIG_MangledTypeQuery("_p_MyHelperObject");
if(ty == NULL)
{
Py_Finalize();
return false;
}
PyObject *helper_obj = SWIG_NewPointerObj(PointerToMyHelperObject, ty, 0);
if(helper_obj == NULL)
{
Py_Finalize();
return false;
}
PyTuple_SetItem(pArgs, 0, data_obj);
PyTuple_SetItem(pArgs, 1, helper_obj);
PyObject *pValue = PyObject_CallObject(pFunc, pArgs);
if(pValue == NULL)
{
Py_Finalize();
return false;
}
In Python, we see something like:
def go(dataobj, helperobj):
## if conditions are right....
helperobj.helpme(dataobj)
Now, this largely works except for one thing. In my C++ code when I am preparing my arguments to pass on to the Python script, I observe the pointer value of PointerToMyDataObject.
When I set a breakpoint in the C++ implementation of helperobj.helpme(), I see that the memory address is different, though it seems to be a pointer to a valid instance of MyDataObject.
This is important to me, as "MyDataObject" is in fact a base class for a few possible derived classes. My helper object wants to perform an appropriate (determined by context) dynamic cast on the pointer it receives to point at the appropriate derived class. That's failing for what I think are obvious reasons now.
I've read some things about "shadow" objects in SWIG, which only adds to my confusion (apologies for my tiny brain :-P)
So, is SWIG making a copy of my object for some reason, and then passing a pointer to the copy? If it is, then I can understand why my assumptions about dynamic casts won't work.
I Tried to add this as a comment, but struggled with formatting, so..... more insight follows:
The problem has to do with pass-by-reference. Notice I have 2 implementations of the virtual method helpMe():
bool MyHelperObject::helpMe(MyDataObject mydata_obj)
{
return common_code(&mydata_obj);
}
bool MyHelperObject::helpMe(MyDataObject *mydata_obj)
{
return common_code(mydata_obj);
}
Although I provided python with a pointer, it is calling the pass-by-reference version. This explains why I'm getting different pointer values. But what can I do to force a call on the version that takes a pointer argument?

Based on what you've shown I think you want to make sure SWIG only gets to see the pointer version of helpMe. The non-pointer version will be creating a temporary copy and then passing that into the function and it sounds like that isn't what you want.
SWIG will have a hard time picking which version to use since it abstracts the pointer concept slightly to match Python better.
You can hide the non-pointer version from SWIG with %ignore before the declaration or %import that shows it to SWIG in your interface file:
%ignore MyHelperObject::helpMe(MyDataObject mydata_obj)
%import "some.h"