I am trying to do some fairly simple list manipulation, using a Jupyter notebook that calls a DLL function. I'd like my Jupyter notebook/Python code to pass in a Python list to a C++ function, which modifies the list, and then I'd like the Python code to be able to access the new list values.
I can actually read (in Jupyter) the items that were not edited by the C++ code, so there must be some issue with how I'm writing, but every example I can find looks just like my code. When I try to access the item in the list that the C++ code writes, my Jupyter kernel dies with no explanation; I've tried to run the same Python code in the terminal, and the terminal session just exits, again with no explanation.
Running on Windows 10, environment with Python 3.9.2. Here's the Python:
import os
import ctypes
import _ctypes
# Import the DLL
mydll = ctypes.cdll.LoadLibrary(*path to DLL*)
# Set up
data_in = [3,6,9]
mydll.testChange.argtypes = [ctypes.py_object]
mydll.testChange.restype = ctypes.c_float
mydll.testChange(data_in)
# Returns 0.08
After running this and closing the DLL, running data_in[1] returns 6, data_in[2] returns 9, and data_in[0] causes my kernel to die.
C code for the DLL:
float testChange(PyObject *data_out) {
Py_SetPythonHome(L"%user directory%\\anaconda3");
Py_Initialize();
PyList_SetItem(data_out, 0, PyLong_FromLong(1L));
return 0.08;
}
I can also insert a number of print statements in this code that show that I can read out all three items in the DLL both before and after the call to PyList_SetItem using calls like PyLong_AsLong(PyList_GetItem(data_out, 1)). It's not clear to me that any reference counts need changing or anything like that, but perhaps I misunderstand the idea. Any ideas you all have would be greatly appreciated.
Related
I have this part of a C# DLL from a larger project. It has no dependencies, runs in either .NET 5.0 or .NET 6.0 C# without anything else in it. It looks something like this:
namespace Some.Namespace {
public static class Rename{
public static string RenameString(string stringToRename){
//Do some stuff to it
return stringToRename;
}
}
}
However, no matter what way we attempt to add the reference to Python 3.6 (ArcGIS Pro Distribution) using Python.NET...
sys.path.append(os.path.join(os.getcwd(),"Project_Folder","bin"))
clr.AddReference("SomeDLL")
from Some.Namespace import Rename
It throws this error on the 3rd line.
Exception has occurred: ModuleNotFoundError No module named Some
We've tried just about every possible means to load the DLL at this point and none of them have worked (Yes I know that 2.5.x Python.NET doesn't support .NET 6.0 - we switched to 5.0 and it didn't work either) Exposing the function through Robert's DllExport using ctypes throws a CLR Exception that we can't debug because it's cross-environment whenever the function is run.
Here's that attempt.
//The DLL With Robert's DLLExport
namespace Some.Namespace {
public static class Rename{
//We've also tried CallingConvention.Cdecl
[DllExport("Rename", System.Runtime.InteropServices.CallingConvention.StdCall)]
public static string RenameString(string stringToRename){
//Do some stuff to it
return stringToRename;
}
}
}
#In Python
#We've tried WINDLL
dll_utils = CDLL(os.path.join(os.getcwd(),"project_folder","bin","SomeDLL.dll"))
#We've tried using CFUNCTTYPE here too
encrypt_proto = ctypes.WINFUNCTYPE(ctypes.c_void_p, ctypes.c_void_p)
encrypt_args = (1, "p1", 0),(1, "p2", 0),
#It fails here when we add the encrypt_args flags to it, saying they don't match up,
#but there's no way to figure out how many it wants, we've tried many variations of
#the above
encrypt_call = encrypt_proto(("Rename",dll_utils),encrypt_args)'
#So because of that on one attempt we removed these two lines
p2 = ctypes.c_char_p(text_to_obfuscate)
p1 = ctypes.c_char_p(text_to_obfuscate)
#And the second argument here
return_value = encrypt_call(p1,p2)
#And...
OSError: [WinError -532462766] Windows Error 0xe0434352
Attempting to write a second function in the DLL to convert the C# string to a byte array and back and using a ctypes pointer doesn't work - it throws the same error just posted when it's called. Someone suggested in another question here IronPython next - which we want to avoid, or to try to use comtypes which I've never used before. At one point we even tried to do a COM decoration and it claimed something I've seen on an answer here is obsolete. I've researched this for 2-3 days and haven't been able to solve this.
At this point I'm at a loss. Is there any effective, simple way, to get an external .NET C# function in a DLL to call a function and pass a string in Python without installing any major dependencies? Just out of the box? There's gotta be something simple I'm missing.
I am new to cython(only use it for doing a little hw now).
I use the following code to see a general idea of it in jupyter notebook.
%load_ext Cython
%%cython
def cfunc(int n):
cdef int a = 0
for i in range(n):
a += i
return a
print(cfunc(10))
However, it only prints out the result 45 once. When I run the print function, the cell doesn't show 45 anyone.
Is there any problems with the code? How can I make the cell prints out 45 just the same as a normal python code? Thanks.
When running %%cython-magic a lot happens under the hood. One can see parts of it when calling the magic in verbose mode, i.e. %%cython --verbose:
A file called _cython_magic_b599dcf313706e8c6031a4a7058da2a2.pyx is generated. b599dcf313706e8c6031a4a7058da2a2 is the sha1-hash of the %%cython-cell, which is needed for example to be able to reload a %%cython-cell (see this SO-post).
This file is cythonized and build to a c-extension called _cython_magic_b599dcf313706e8c6031a4a7058da2a2.
This extension gets imported - this is the moment your code prints 45, and everything from this module is added to the global namespace.
When you execute the cell again, nothing of the above happens: given the sha-hash the machinery can see, that this cell was already executed and loaded - so nothing to be done. Only when the content of the cell is changed and thus its hash the cash will not be used but the 3 steps above executed.
To enforce that the steps above are performed one has to pass --force (or -f) options to the %%cython-magic-cell, i.e.:
%%cython --force
...
# 45 is printed
However, because building extension anew is quite time consuming one would probably prefer the following
%%cython
def cfunc(int n):
cdef int a = 0
for i in range(n):
a += i
return a
# put the code of __main__ into a function
def cython_main():
print(cfunc(10))
# execute the old main
cython_main()
and now calling cython_main() in a new cell, so it gets reevaluated the same way the normal python code would.
I'm currently using a python module called petsc4py (https://pypi.org/project/petsc4py/). My main issue is that none of the typical intellisense features seems to work with this module.
I'm guessing it might have something to do with it being a C extension module, but I am not sure exactly why this happens. I initially thought that intellisense was unable to look inside ".so" files, but it seems that numpy is able to do this with the array object, which in my case is inside a file called multiarray.cpython-37m-x86_64-linux-gnu (check example below).
Does anyone know why I see this behaviour in the petsc4py module. Is there anything that I (or the developers of petsc4py) can do to get intellisense to work?
Example:
import sys
import petsc4py
petsc4py.init(sys.argv)
from petsc4py import PETSc
x_p = PETSc.Vec().create()
x_p.setSizes(10)
x_p.setFromOptions()
u_p = x_p.duplicate()
import numpy as np
x_n = np.array([1,2,3])
u_n = x_n.copy()
In this example, when trying to work with a Vec object from petsc4py, doing u_p.duplicate() cannot find the function and the suggestion is simply a repetition of the function immediately before. However, using an array from numpy, doing u_n.copy() works perfectly.
If you're compiling in-place then you're bumping up against https://github.com/microsoft/python-language-server/issues/197.
I’m currently developing a script using the python script editor in Rhino. As I’m currently working in a Windows machine, the script editor uses IronPython as language.
In the same script, I want to interact with an FE software (Straus7) which has a Python API. When doing so, I have experienced some problems as the ctypes module does not seem to work in IronPython the same way it does in regular Python. Especially, I’m finding problems when initializing arrays using the command:
ctypes.c_double*3
For example, if I want to obtain the XYZ coordinates of a node #1 in the FE model, I regular Python I would write the following:
XYZType = ctypes.c_double*3
XYZ = XYZType()
node_num = 1
st.St7GetNodeXYZ(1,node_num,XYZ)
And this returns me a variable XYZ which is a 3D array such that:
XYZ -> <straus_userfunctions.c_double_Array_3 at 0xc5787b0>
XYZ[0] = -0.7xxxxx -> (X_coord)
XYZ[1] = -0.8xxxxx -> (Y_coord)
XYZ[2] = -0.9xxxxx -> (Z_coord)
On the other side, I copy the same exact script in IronPython, the following error message appears
Message: expected c_double, got c_double_Array_3
Obviously, If I change the variable XYZ to c_double; then it becomes a double variable which contains only a single entry, which corresponds to the first element of the array (in this case, the X-coordinate)
This situation is quite annoying as all FEM softwares, the usage of matrices and arrays is widely used. Consequently, I wanted to ask if anyone nows a simple fix to this situation.
I was thinking of using the memory allocation of the first element of the array to obtain the rest but I’m not so sure how to do so.
Thanks a lot. Gerard
I've found when working with IronPython you need to explicitly cast the "Array of three doubles" to a "Pointer to double". So if you're using Grasshopper with the Strand7 / Straus7 API you will need to add an extra bit like this:
import St7API
import ctypes
# Make the pointer conversion functions
PI = ctypes.POINTER(ctypes.c_long)
PD = ctypes.POINTER(ctypes.c_double)
XYZType = ctypes.c_double*3
XYZ = XYZType()
node_num = 1
# Cast arrays whenever you pass them to St7API from IronPython
St7API.St7GetNodeXYZ(1, node_num, PD(XYZ))
I don't have access to IronPython or Strand7 / Straus7 at the moment but from memory that will do it. If that doesn't work for you you can email Strand7 Support - you would typically get feedback on something like this within a day or so.
I have successfully created a Python module that appears to work in isolation, but doesn't affect the program that is running it.
I have the following module:
BOOST_PYTHON_MODULE(mandala)
{
class_<state_mgr_t, state_mgr_t*, noncopyable>("states", no_init)
.def("push", &state_mgr_t::push)
.def("pop", &state_mgr_t::pop)
.def("count", &state_mgr_t::count);
scope().attr("states") = boost::python::object(boost::python::ptr(&states));
}
The states object is referencing a global value, states:
extern state_mgr_t states;
I can run the following script lines from within my program:
from mandala import states
states.count()
> 0
All of that is fine and whatnot, but I was expecting that running this python script would affect the actual state of the program that is running it. It appears as though Python is actually just dealing with it's own instance of states and not affecting the parent program.
Now I'm wondering if I've completely misunderstood what Boost.Python is capable of; I was expecting something similar to Lua, where I could modify the C++ program via scripts.
Is this not possible? Or am I doing something very wrong?
Thank you in advance!
If you are embedding Python into your C++ program, it should be possible to access the instance from your script. Obviously, I don't have your full code, but have you tried something like this?
PyImport_AppendInittab("mandala", &initmandala);
Py_Initialize();
try {
object main_module = import("__main__");
object main_namespace = main_module.attr("__dict__");
main_namespace.attr("states") = ptr(&states);
object ignored = exec("print states.count()", main_namespace);
} catch(const error_already_set&) {
PyErr_Print();
}