I am trying to embed python 2.7.3 in C++ and use Numpy library and I obtain runtime error while importing Numpy for the second time. Here is a simple code example (as smallest as possible) :
#include <Python.h>
int main() {
for(int i=0 ; i<2 ; i++) {
Py_Initialize() ;
PyImport_ImportModule("numpy");
Py_Finalize() ;
}
return 0 ;
}
What's wrong with this ?
From the Py_Finalize documentation docs you have:
Some extensions may not work properly if their initialization routine is called more than
once; this can happen if an application calls Py_Initialize() and
Py_Finalize() more than once.
I wouldn't be surprised if Numpy is one of these extensions.
Update: looks like it is, see this question.
Related
I have a C++ program and it has sort of plugin structure: when program starts up, it's looking for dll in the plugin folder with certain exported function signatures, such as:
void InitPlugin(FuncTable* funcTable);
Then the program will call the function in the dll to initialize and pass function pointers to the dll. From that time on, the dll can talk to the program.
I know Cython let you call C function in Python, but I'm not sure can I write a Cython code and compile it to a dll so my C++ program can initialize with it. An example code would be great.
Using cython-module in a dll is not unlike using a cython-module in an embeded python interpreter.
The first step would be to mark cdef-function which should be used from external C-code with public, for example:
#cyfun.pyx:
#doesn't need python interpreter
cdef public int double_me(int me):
return 2*me;
#needs initialized python interpreter
cdef public void print_me(int me):
print("I'm", me);
cyfun.c and cyfun.h can be generated with
cython -3 cyfun.pyx
These files will be used for building of the dll.
The dll will need one function to initialize the python interpreter and another to finalize it, which should be called only once before double_me and print_me can be used (Ok, double_me would work also without interpreter, but this is an implementation detail). Note: the initialization/clean-up could be put also in DllMain - see such a version further bellow.
The header-file for the dll would look like following:
//cyfun_dll.h
#ifdef BUILDING_DLL
#define DLL_PUBLIC __declspec(dllexport)
#else
#define DLL_PUBLIC __declspec(dllimport)
#endif
//return 0 if everything ok
DLL_PUBLIC int cyfun_init();
DLL_PUBLIC void cyfun_finalize();
DLL_PUBLIC int cyfun_double_me(int me);
DLL_PUBLIC void cyfun_print_me(int me);
So there are the necessary init/finalize-functions and the symbols are exported via DLL_PUBLIC (which needs to be done see this SO-post) so it can be used outside of the dll.
The implementation follows in cyfun_dll.c-file:
//cyfun_dll.c
#define BUILDING_DLL
#include "cyfun_dll.h"
#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include "cyfun.h"
DLL_PUBLIC int cyfun_init(){
int status=PyImport_AppendInittab("cyfun", PyInit_cyfun);
if(status==-1){
return -1;//error
}
Py_Initialize();
PyObject *module = PyImport_ImportModule("cyfun");
if(module==NULL){
Py_Finalize();
return -1;//error
}
return 0;
}
DLL_PUBLIC void cyfun_finalize(){
Py_Finalize();
}
DLL_PUBLIC int cyfun_double_me(int me){
return double_me(me);
}
DLL_PUBLIC void cyfun_print_me(int me){
print_me(me);
}
Noteworthy details:
we define BUILDING_DLL so DLL_PUBLIC becomes __declspec(dllexport).
we use cyfun.h generated by cython from cyfun.pyx.
cyfun_init inizializes python interpreter and imports the built-in module cyfun. The somewhat complicated code is because since Cython-0.29, PEP-489 is default. More information can be found in this SO-post. If the Python-interpreter isn't initialized or if the module cyfun is not imported, the chances are high, that calling the functionality from cyfun.h will end in a segmentation fault.
cyfun_double_me just wraps double_me so it becomes visible outside of the dll.
Now we can build the dll!
:: set up tool chain
call "<path_to_vcvarsall>\vcvarsall.bat" x64
:: build cyfun.c generated by cython
cl /Tccyfun.c /Focyfun.obj /c <other_coptions> -I<path_to_python_include>
:: build dll-wrapper
cl /Tccyfun_dll.c /Focyfun_dll.obj /c <other_coptions> -I<path_to_python_include>
:: link both obj-files into a dll
link cyfun.obj cyfun_dll.obj /OUT:cyfun.dll /IMPLIB:cyfun.lib /DLL <other_loptions> -L<path_to_python_dll>
The dll is now built, but the following details are noteworthy:
<other_coptions> and <other_loptions> can vary from installation to installation. An easy way is to see them is to run cythonize some_file.pyx` and to inspect the log.
we don't need to pass python-dll, because it will be linked automatically, but we need to set the right library-path.
we have the dependency on the python-dll, so later on it must be somewhere where it can be found.
Were you go from here depends on your task, we test our dll with a simple main:
//test.c
#include "cyfun_dll.h"
int main(){
if(0!=cyfun_init()){
return -1;
}
cyfun_print_me(cyfun_double_me(2));
cyfun_finalize();
return 0;
}
which can be build via
...
:: build main-program
cl /Tctest.c /Focytest.obj /c <other_coptions> -I<path_to_python_include>
:: link the exe
link test.obj cyfun.lib /OUT:test_prog.exe <other_loptions> -L<path_to_python_dll>
And now calling test_prog.exe leads to the expected output "I'm 4".
Depending on your installation, following things must be considered:
test_prog.exe depends on pythonX.Y.dll which should be somewhere in the path so it can be found (the easiest way is to copy it next to the exe)
The embeded python interpreter needs an installation, see this and/or this SO-posts.
IIRC, it is not a great idea to initialize, then to finalize and then to initialize the Python-interpreter again (that might work for some scenarios, but not all , see for example this) - the interpreter should be initialized only once and stay alive until the programs ends.
Thus, it may make sense to put initialization/clean-up code into DllMain (and make cyfun_init() and cyfun_finalize() private), e.g.
BOOL WINAPI DllMain(
HINSTANCE hinstDLL, // handle to DLL module
DWORD fdwReason, // reason for calling function
LPVOID lpReserved ) // reserved
{
// Perform actions based on the reason for calling.
switch( fdwReason )
{
case DLL_PROCESS_ATTACH:
return cyfun_init()==0;
case DLL_PROCESS_DETACH:
cyfun_finalize();
break;
case DLL_THREAD_ATTACH:
// Do thread-specific initialization.
break;
case DLL_THREAD_DETACH:
// Do thread-specific cleanup.
break;
}
return TRUE;
}
If your C/C++-program already has an initialized Python-interpreter it would make sense to offer a function which only imports the module cyfun and doesn't initialize the python-interpreter. In this case I would define CYTHON_PEP489_MULTI_PHASE_INIT=0, because PyImport_AppendInittab must be called before Py_Initialize, which might be already too late when the dll is loaded.
I'd imagine it'd be difficult to call it directly, with Cython depending so much on the Python runtime.
Your best bet is to embed a Python interpreter directly inside your app, e.g. as described in this answer, and call your Cython code from the interpreter. That's what I would do.
I'm very new to the whole CMake. Following this and this posts, now I want to call a MAXON function inside Python, using pybind11. What I have done so far:
The library can be downloaded from this page (direct download link).
wget https://www.maxongroup.com/medias/sys_master/root/8837358518302/EPOS-Linux-Library-En.zip
unzip:
unzip EPOS-Linux-Library-En.zip
make the install shell script executable and run it:
chmod +x ./install.sh
sudo ./install.sh
Then going to the example folder:
cd /opt/EposCmdLib_6.6.1.0/examples/HelloEposCmd/
Now combining the CMakeLists.txt files from here:
# CMakeLists.txt
cmake_minimum_required(VERSION 2.8.12)
project (HelloEposCmd)
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -Wall")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -Wall")
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
find_package(pybind11 REQUIRED)
pybind11_add_module(${PROJECT_NAME} HelloEposCmd.cpp)
add_executable(${PROJECT_NAME} HelloEposCmd.cpp)
target_link_libraries(${PROJECT_NAME} -lEposCmd)
and the HelloEposCmd.cpp this line is added right after other header files:
#include <pybind11/pybind11.h>
the main function is renamed to:
int run(int argc, char** argv)
and the pybind11 syntax to add the module is written at the end:
PYBIND11_MODULE(HelloEposCmd, m) {
m.def("run", &run, "runs the HelloEposCmd");
}
However, When I run the cmake . I get the error:
CMake Error at CMakeLists.txt:13 (add_executable):
add_executable can not create target "HelloEposCmd" because another target with the same name already exists. The existing target is a module library created in source directory "/opt/EposCmdLib_6.6.1.0/examples/HelloEposCmd" See documentation for policy CMP0002 for more details.
...
I was wondering if you could be kind to help me get the right CMakeList.txt file. Ideally, I should be able to call the compiled module in python:
# HelloEposCmd.py
import HelloEposCmd
HelloEposCmd.run()
Thanks for your support in advance.
pybind11_add_module already creates a target for you. So you don't need add_executable anymore. Just remove that line and when you will build you will get a library with the name HelloEposCmd
add_executable is needed if you are building an executable (.exe), which I believe is not what you want.
Documenation of pybind11 says.
This function behaves very much like CMake’s builtin add_library (in fact, it’s a wrapper function around that command).
Thanks to abhilb post and his kind followup in the comments I was able to figure the problem out. well, at least find a temporary workaround:
According to this post, the last two lines of the CMakeLists.txt file should change to
# this line can be removed
# add_executable(${PROJECT_NAME} HelloEposCmd.cpp)
target_link_libraries(${PROJECT_NAME} PRIVATE -lEposCmd)
and then because according to this post pybind11 doesn't support double pointers we change the run function to:
int run() {
int argc = 1;
char* argv[] = {"./HelloEposCmd"};
...
}
which I suppose to be a horrible workaround (inspired by information from this page). Now running cmake ., make and python3 HelloEposCmd.py should work properly (except a small c++ warning!).
P.S.1. Maybe someone could use std::vector<std::string> as suggested here. This idea was proposed here and there are already some answers worth investigating.
P.S.2. Following this discussion, another workaround could be something like:
#include <stdio.h>
#include <stdlib.h>
void myFunc(int argc, char* argv[]) {
for (int i = 0; i < argc; ++i) {
printf("%s\n", argv[i]);
}
}
int run(int argc, long* argv_) {
char** argv = (char**)malloc(argc * sizeof(char*));
for (int i = 0; i < argc; ++i) {
argv[i] = (char*)(argv_[i]);
}
myFunc(argc, argv);
free(argv);
return 0;
}
I'm getting a really weird crash when using ctypes in Python, but I'm not sure if the problem comes from Python or C.
Here is the C source (in test.c):
#include <stdio.h>
void compress(char *a, int b) {
printf("inside\n");
}
void run() {
printf("before\n");
compress("hi", 2);
printf("after\n");
}
Then here's what happens when I call run() with ctypes:
$ python -c 'import ctypes; ctypes.cdll.LoadLibrary("./test.so").run()'
before
Segmentation fault (core dumped)
The weirdest thing is that the crash doesn't happen when I rename compress() to anything else.
Other things that prevent it from crashing:
Calling compress() directly
Calling run() or compress() from C directly (If I add a main(), compile it directly, and execute it)
Removing either argument from the signature of compress() (but then the function doesn't seem to execute, based on the lack of "inside" being printed.
I'm pretty new to C, so I'm assuming there's something I'm missing here. What could be causing this?
System info:
Python 2.7.6
gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04)
Ubuntu 14.04
uname -r: 3.13.0-58-generic
According to the debugging, the program is trying to call compress in libz.so.1.
$ gdb python -c core
...
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `python -c import ctypes; ctypes.cdll.LoadLibrary("./test.so").run()'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f9ddea18bff in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
which accepts different parameters (zlib.h):
ZEXTERN int ZEXPORT compress OF((Bytef *dest, uLongf *destLen,
const Bytef *source, uLong sourceLen));
ZEXTERN int ZEXPORT compress2 OF((Bytef *dest, uLongf *destLen,
const Bytef *source, uLong sourceLen,
int level));
/*
You can modify the compress function to be static to work around the issue:
static void compress(char *a, int b)
{
printf("inside\n");
}
While #falsetru has diagnosed the problem, his solution won't work in the general case where you have a lot of files to statically link together (because the entire point of declaring things static is to not have them visible from other files).
And while #eryksun has posted a solution for when you want to declare a function the same name as another, in general, you may have a lot of C functions you don't want to export, and you don't want to have to worry about whether they collide with some random function in some library that Python happens to import, and you don't want to have to prefix every one of your internal functions with an attribute.
(GCC maintains documentation on function attributes, including this function visibility feature.)
A more general solution to avoiding namespace collisions is to tell the linker not to export any symbols by default, and then to mark only those functions you want exported, like run(), as visible.
There is probably a standard way to define the macro for this, but my C is so out-of-date I wouldn't know it. In any case, this will work:
#include <stdio.h>
#define EXPORT __attribute__((visibility("protected")))
void compress(char *a, int b) {
printf("inside\n");
}
EXPORT void run() {
printf("before\n");
compress("hi", 2);
printf("after\n");
}
You can link and run it like this:
$ gcc -x c test.c --shared -fvisibility=hidden -o test.so
$ python -c 'import ctypes; ctypes.cdll.LoadLibrary("./test.so").run()'
before
inside
after
I am a beginner in assembly, but a master in Python. I have just recently started to learn x86_64 NASM for windows, and I wish to combine the power of assembly, and the flexibility of Python. I have looked all over, and I have not found a way to use a NASM assembly procedure from within Python. By this I do not mean in-line assembly. I wish to write an assembly program, compile it, and then somehow extract the procedure to use in my Python program. Can someone illustrate a simple example of how to do this, as I am completely lost.
You could create a C extension wrapper for the functions implemented in assembly and link it to the OBJ file created by nasm.
A dummy example (for 32 bit Python 2; not tested):
myfunc.asm:
;http://www.nasm.us/doc/nasmdoc9.html
global _myfunc
section .text
_myfunc:
push ebp
mov ebp,esp
sub esp,0x40 ; 64 bytes of local stack space
mov ebx,[ebp+8] ; first parameter to function
; some more code
leave
ret
myext.c:
#include <Python.h>
void myfunc(void);
static PyObject*
py_myfunc(PyObject* self, PyObject* args)
{
if (!PyArg_ParseTuple(args, ""))
return NULL;
myfunc();
Py_RETURN_NONE;
}
static PyMethodDef MyMethods[] =
{
{"myfunc", py_myfunc, METH_VARARGS, NULL},
{NULL, NULL, 0, NULL}
};
PyMODINIT_FUNC initmyext(void)
{
(void) Py_InitModule("myext", MyMethods);
}
setup.py:
from distutils.core import setup, Extension
setup(name='myext', ext_modules=[
Extension('myext', ['myext.c'], extra_objects=['myfunc.obj'])])
Build and run:
nasm -fwin32 myfunc.asm
python setup.py build_ext --inplace
python -c"import myext;myext.myfunc()"
You can also embed assembly directly inside your Python program:
https://github.com/Maratyszcza/PeachPy
https://github.com//pycca/pycca
http://codeflow.org/entries/2009/jul/31/pyasm-python-x86-assembler/
https://github.com/AmihaiN/pyAsm
These work by compiling the assembly and loading it into executable memory at runtime. The first three projects implement x86 assemblers in Python, whereas the last calls out to an external compiler.
Not sure about the "power" of assembly, really.
You can start here:
https://docs.python.org/2/extending/extending.html
It's about extending python with compiled code, written in C or C++, but the principle should be the same (C is really just a portable macro-assembler).
I am facing a problem similar to the Py_initialize / Py_Finalize not working twice with numpy .. The basic coding in C:
Py_Initialize();
import_array();
//Call a python function which imports numpy as a module
//Py_Finalize()
The program is in a loop and it gives a seg fault if the python code has numpy as one of the imported module. If I remove numpy, it works fine.
As a temporary work around I tried not to use Py_Finalize(), but that is causing huge memory leaks [ observed as the memory usage from TOP keeps on increasing ]. And I tried but did not understand the suggestion in that link I posted. Can someone please suggest the best way to finalize the call while having imports such as numpy.
Thanks
santhosh.
I recently faced a very similar issue and developed a workaround that works for my purposes, so I thought I would write it here in the hope it might help others.
The problem
I work with some postprocessing pipeline for which I can write a own functor to work on some data passing through the pipeline and I wanted to be able to use Python scripts for some of the operations.
The problem is that the only thing I can control is the functor itself, which gets instantiated and destroyed at times beyond my control. I furthermore have the problem that even if I do not call Py_Finalize the pipeline sometimes crashes once I pass another dataset through the pipeline.
The solution in a Nutshell
For those who don't want to read the whole story and get straight to the point, here's the gist of my solution:
The main idea behind my workaround is not to link against the Python library, but instead load it dynamically using dlopen and then get all the addresses of the required Python functions using dlsym. Once that's done, one can call Py_Initialize() followed by whatever you want to do with Python functions followed by a call to Py_Finalize() once you're done. Then, one can simply unload the Python library. The next time you need to use Python functions, simply repeat the steps above and Bob's your uncle.
However, if you are importing NumPy at any point between Py_Initialize and Py_Finalize, you will also need to look for all the currently loaded libraries in your program and manually unload those using dlclose.
Detailed workaround
Loading instead of linking Python
The main idea as I mentioned above is not to link against the Python library. Instead, what we will do is load the Python library dynamically using dlopen():
#include
...
void* pHandle = dlopen("/path/to/library/libpython2.7.so", RTLD_NOW | RTLD_GLOBAL);
The code above loads the Python shared library and returns a handle to it (the return type is an obscure pointer type, thus the void*). The second argument (RTLD_NOW | RTLD_GLOBAL) is there to make sure that the symbols are properly imported into the current application's scope.
Once we have a pointer to the handle of the loaded library, we can search that library for the functions it exports using the dlsym function:
#include <dlfcn.h>
...
// Typedef named 'void_func_t' which holds a pointer to a function with
// no arguments with no return type
typedef void (*void_func_t)(void);
void_func_t MyPy_Initialize = dlsym(pHandle, "Py_Initialize");
The dlsym function takes two parameters: a pointer to the handle of the library that we obtained previously and the name of the function we are looking for (in this case, Py_Initialize). Once we have the address of the function we want, we can create a function pointer and initialize it to that address. To actually call the Py_Initialize function, one would then simply write:
MyPy_Initialize();
For all the other functions provided by the Python C-API, one can just add calls to dlsym and initialize function pointers to its return value and then use those function pointers instead of the Python functions. One simply has to know the parameter and return value of the Python function in order to create the correct type of function pointer.
Once we are finished with the Python functions and call Py_Finalize using a procedure similar to the one for Py_Initialize one can unload the Python dynamic library in the following way:
dlclose(pHandle);
pHandle = NULL;
Manually unloading NumPy libraries
Unfortunately, this does not solve the segmentation fault problems that occur when importing NumPy. The problems comes from the fact that NumPy also loads some libraries using dlopen (or something equivalent) and those do not get unloaded them when you call Py_Finalize. Indeed, if you list all the loaded libraries within your program, you will notice that after closing the Python environment with Py_Finalize, followed by a call to dlclose, some NumPy libraries will remain loaded in memory.
The second part of the solution requires to list all the Python libraries that remain in memory after the call dlclose(pHandle);. Then, for each of those libraries, grab a handle to them and then call dlcloseon them. After that, they should get unloaded automatically by the operating system.
Fortunately, there are functions under both Windows and Linux (sorry MacOS, couldn't find anything that would work in your case...):
- Linux: dl_iterate_phdr
- Windows: EnumProcessModules in conjunction with OpenProcess and GetModuleFileNameEx
Linux
This is rather straight forward once you read the documentation about dl_iterate_phdr:
#include <link.h>
#include <string>
#include <vector>
// global variables are evil!!! but this is just for demonstration purposes...
std::vector<std::string> loaded_libraries;
// callback function that gets called for every loaded libraries that
// dl_iterate_phdr finds
int dl_list_callback(struct dl_phdr_info *info, size_t, void *)
{
loaded_libraries.push_back(info->dlpi_name);
return 0;
}
int main()
{
...
loaded_libraries.clear();
dl_iterate_phdr(dl_list_callback, NULL);
// loaded_libraries now contains a list of all dynamic libraries loaded
// in your program
....
}
Basically, the function dl_iterate_phdr cycles through all the loaded libraries (in the reverse order they were loaded) until either the callback returns something other than 0 or it reaches the end of the list. To save the list, the callback simply adds each element to a global std::vector (one should obviously avoid global variables and use a class for example).
Windows
Under Windows, things get a little more complicated, but still manageable:
#include <windows.h>
#include <psapi.h>
std::vector<std::string> list_loaded_libraries()
{
std::vector<std::string> m_asDllList;
HANDLE hProcess(OpenProcess(PROCESS_QUERY_INFORMATION
| PROCESS_VM_READ,
FALSE, GetCurrentProcessId()));
if (hProcess) {
HMODULE hMods[1024];
DWORD cbNeeded;
if (EnumProcessModules(hProcess, hMods, sizeof(hMods), &cbNeeded)) {
const DWORD SIZE(cbNeeded / sizeof(HMODULE));
for (DWORD i(0); i < SIZE; ++i) {
TCHAR szModName[MAX_PATH];
// Get the full path to the module file.
if (GetModuleFileNameEx(hProcess,
hMods[i],
szModName,
sizeof(szModName) / sizeof(TCHAR))) {
#ifdef UNICODE
std::wstring wStr(szModName);
std::string tModuleName(wStr.begin(), wStr.end());
#else
std::string tModuleName(szModName);
#endif /* UNICODE */
if (tModuleName.substr(tModuleName.size()-3) == "dll") {
m_asDllList.push_back(tModuleName);
}
}
}
}
CloseHandle(hProcess);
}
return m_asDllList;
}
The code in this case is slightly longer than for the Linux case, but the main idea is the same: list all the loaded libraries and save them into a std::vector. Don't forget to also link your program to the Psapi.lib!
Manual unloading
Now that we can list all the loaded libraries, all you need to do is find among those the ones that come from loading NumPy, grab a handle to them and then call dlclose on that handle. The code below will work on both Windows and Linux, provided that you use the dlfcn-win32 library.
#ifdef WIN32
# include <windows.h>
# include <psapi.h>
# include "dlfcn_win32.h"
#else
# include <dlfcn.h>
# include <link.h> // for dl_iterate_phdr
#endif /* WIN32 */
#include <string>
#include <vector>
// Function that list all loaded libraries (not implemented here)
std::vector<std::string> list_loaded_libraries();
int main()
{
// do some preprocessing stuff...
// store the list of loaded libraries now
// any libraries that get added to the list from now on must be Python
// libraries
std::vector<std::string> loaded_libraries(list_loaded_libraries());
std::size_t start_idx(loaded_libraries.size());
void* pHandle = dlopen("/path/to/library/libpython2.7.so", RTLD_NOW | RTLD_GLOBAL);
// Not implemented here: get the addresses of the Python function you need
MyPy_Initialize(); // Needs to be defined somewhere above!
MyPyRun_SimpleString("import numpy"); // Needs to be defined somewhere above!
// ...
MyPyFinalize(); // Needs to be defined somewhere above!
// Now list the loaded libraries again and start manually unloading them
// starting from the end
loaded_libraries = list_loaded_libraries();
// NB: this below assumes that start_idx != 0, which should always hold true
for(std::size_t i(loaded_libraries.size()-1) ; i >= start_idx ; --i) {
void* pHandle = dlopen(loaded_libraries[i].c_str(),
#ifdef WIN32
RTLD_NOW // no support for RTLD_NOLOAD
#else
RTLD_NOW|RTLD_NOLOAD
#endif /* WIN32 */
);
if (pHandle) {
const unsigned int Nmax(50); // Avoid getting stuck in an infinite loop
for (unsigned int j(0) ; j < Nmax && !dlclose(pHandle) ; ++j);
}
}
}
Final words
The examples shown here capture the basic ideas behind my solution, but can certainly be improved to avoid global variables and facilitate ease of use (for example, I wrote a singleton class that handles the automatic initialization of all the function pointers after loading the Python library).
I hope this can be useful to someone in the future.
References
dl_iterate_phdr: https://linux.die.net/man/3/dl_iterate_phdr
PsAPI library: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684894(v=vs.85).aspx
OpenProcess: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684320(v=vs.85).aspx
EnumProcess: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682629(v=vs.85).aspx
GetModuleFileNameEx: https://msdn.microsoft.com/en-us/library/windows/desktop/ms683198(v=vs.85).aspx
dlfcn-win32 library: library: https://github.com/dlfcn-win32/dlfcn-win32
I'm not quite sure how you don't seem to understand the solution posted in Py_initialize / Py_Finalize not working twice with numpy. The solution posted is quite simple: call Py_Initialize and Py_Finalize only once for each time your program executes. Do not call them every time you run the loop.
I assume that your program, when it starts, runs some initialization commands (which are only run once). Call Py_Initialize there. Never call it again. Also, I assume that when your program terminates, it has some code to tear down things, dump log files, etc. Call Py_Finalize there. Py_Initialize and Py_Finalize are not intended to help you manage memory in the Python interpreter. Do not use them for that, as they cause your program to crash. Instead, use Python's own functions to get rid of objects you don't want to keep.
If you really MUST create a new environment every time you run your code, you can use Py_NewInterpreter and to create a sub-interpreter and Py_EndInterpreter to destroy that sub-interpreter later. They're documented near the bottom of the Python C API page. This works similarly to having a new interpreter, except that modules are not re-initialized each time a sub-interpreter starts.