I created a C program and choose to use python as my GUI creator
I wrote the python GUI and decided to create a struct
typedef struct GUIcommunicator
{
char action[MAX_SIZE];
char params[MAX_PARAMS][MAX_SIZE];
}GUIcommunicator;
I using this answer I will return the
struct from the python receive him in my C code
but I don't know how.
And I'd like to some help
(the function that sends the struct (the python code) already got the action and the params as string list )
If you are developing a linux based application, you can perform Inter-Process Communication using FIFO files
For windows based applications, you could use Named-pipes
Regardless of the direction of the call, you end up with a PyObject* in C that refers to a list of strings. Fetch each element, then fetch the string data pointer and size for a Python 2 str or (the UTF-8 encoding of) a Python 3 str. Check the size against MAX_SIZE (probably MAX_SIZE-1) and beware of embedded null characters, which your struct presumably cannot represent.
There are of course ways of accepting any iterable, converting elements to strings, etc., if you want that flexibility.
Related
Apologies if this is a stupid question which I suspect it may well be. I'm a Python user with little experience in C.
According to the official Python docs (v3.10.6):
PyObject *PyUnicode_FromStringAndSize(const char *u, Py_ssize_t size)
Return value: New reference. Part of the Stable ABI.
Create a Unicode object from the char buffer u. The bytes will be interpreted as being UTF-8 encoded. The buffer is copied into the new object. If the buffer is not NULL, the return value might be a shared object, i.e. modification of the data is not allowed. [...]
which has me slightly confused.
It says the data i.e. the buffer u is copied.
But then it says the data may be shared which seems to contradict the first statement.
My Question is:
What exactly do they mean? That the newly allocated copy of the data is shared? If so who with?
Also, coming from Python: Why do they make a point of warning against tampering with the data anyway? Is changing a Python-immutable object something routinely done in C?
Ultimately, all I need to know is what to do with u: Can/should I free it or has Python taken ownership?
Ultimately, all I need to know is what to do with u: Can/should I free it or has Python taken ownership?
You still own u. Python has no idea where u came from or how it should be freed. It could even be a local array. Python will not retain a pointer to u. Cleaning up u is still your responsibility.
What exactly do they mean? That the newly allocated copy of the data is shared? If so who with?
The returned string object may be shared with arbitrary other code. Python makes no promises about how that might happen, but in the current implementation, one way is that a single-character ASCII string will be drawn from a cached array of such strings:
/* ASCII is equivalent to the first 128 ordinals in Unicode. */
if (size == 1 && (unsigned char)s[0] < 128) {
if (consumed) {
*consumed = 1;
}
return get_latin1_char((unsigned char)s[0]);
}
Also, coming from Python: Why do they make a point of warning against tampering with the data anyway? Is changing a Python-immutable object something routinely done in C?
It is in fact fairly routine. Python-immutable objects have to be initialized somehow, and that means writing to their memory, in C. Immutability is an abstraction presented at the Python level, but the physical memory of an object is mutable. However, such mutation is only safe in very limited circumstances, and one of the requirements is that no other code should hold any references to the object.
Problem
I need to send 2 float from C++ to python, and vice-versa.
I am using the C++ library winsock2.h, and the function send() that requires the buffer to be char*.
I want to send, for example this string to python:
std::string string = "1 12.3"
My try
When using struct.unpack("..") python returns the wrong numbers.
I tried with many different values inside the string in unpack, but it never works.
How should I structure my unpack to work correctly? Thanks
When defining a variable type that will hold a string in Cython + Python 3, I can use (at least):
cdef char* mystring = "foo"
cdef str mystring = "foo"
cdef bytes mystring = "foo"
The documentation page on strings is unclear on this -- it mostly gives examples using char* and bytes, and frankly I'm having a lot of difficulty understanding it.
In my case the strings will be coming from a Python3 program and are assumed to be unicode. They will be used as dict keys and function arguments, but I will do no further manipulation on them. Needless to say I am trying to maximize speed.
This question suggests that under Python2.7 and without Unicode, typing as str makes string manipulation code run SLOWER than with no typing at all. (But that's not necessarily relevant here since I won't be doing much string manipulation.)
What are the advantages and disadvantages of each of these options?
If there is no further processing done on a particular type, it would be best and fastest to not type them at all, which means they are treated as a general purpose PyObject *.
The str type is a special case which means bytes on Python 2 and unicode on Python 3.
The str type is special in that it is the byte string in Python 2 and the Unicode string in Python 3
So code that types a string as str and handles it as unicode will break on python 2 where str means bytes.
Strings only need to be typed if they are to be converted to C char* or C++ std::string. There, you would use str to handle py2/py3 compatibility, along with helper functions to convert to/from bytes and unicode in order to be able to convert to either char* or std::string.
Typing of strings is for interoperability with C/C++, not for speed as such. Cython will auto-convert, without copying, a bytes string to a char* for example when it sees something like cdef char* c_string = b_string[:b_len] where b_string is a bytes type.
OTOH, if strings are typed without that type being used, Cython will do a conversion from object to bytes/unicode when it does not need to which leads to overhead.
This can be seen in the C code generated as Pyx_PyObject_AsString, Pyx_PyUnicode_FromString et al.
This is also true in general - the rule of thumb is if a specific type is not needed for further processing/conversion, best not to type it at all. Everything in python is an object so typing will convert from the general purpose PyObject* to something more specific.
Some quick testing revealed that for this particular case, only the str declaration worked -- all other options produced errors. Since the string is generated elsewhere in Python3, evidently the str type declaration is needed.
Whether it is faster not to make any declaration at all remains an open question.
I have a C pipe client (pulled directly from the CallNamedPipe example found here) that, if given the string "first", sends the following message to my Python pipeserver:
b'f\x00i\x00r\x00s\x00t\x00\x00\x00'
The struct documentation gives examples where I did both the packing and unpacking in Python. That means I know the format because I explicitly specified it when I called struct.pack.
Is there some way for me to either a) infer the format from the above output or b) set the format in C the same way I do in Python?
Here's the relevant client code:
LPTSTR lpszPipename = TEXT("\\\\.\\pipe\\testpipe");
LPTSTR lpszWrite = TEXT("first");
fSuccess = CallNamedPipe(
lpszPipename, // pipe name
lpszWrite, // message to server
...
>>> b'f\x00i\x00r\x00s\x00t\x00\x00\x00'.decode('utf-16le')
u'first\x00'
"The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)"
Your C code is not writing a struct to the pipe, it is writing a null-terminated string encoded as little-endian UTF-16 text, which is produced by the TEXT() macro when you compile your Windows program in Unicode mode for an Intel CPU. Python knows how to decode these strings without using the struct module. Try this:
null_terminated_unicode_string = data.decode('utf-16le')
unicode_string = null_terminated_unicode_string[:-1]
You can use decode('utf-16') if your python code is running on the same CPU architecture as the C program that writes the data. You might want to read up on python's unicode codecs.
EDIT: You can infer the type of that data by knowing how UTF-16 and Windows string macros work, but python cannot infer it. You could set the string encoding in C the same way you would in python if you wanted to write some code to do so, but it's probably not worth your time.
I often have to write code in other languages that interact with C structs. Most typically this involves writing Python code with the struct or ctypes modules.
So I'll have a .h file full of struct definitions, and I have to manually read through them and duplicate those definitions in my Python code. This is time consuming and error-prone, and it's difficult to keep the two definitions in sync when they change frequently.
Is there some tool or library in any language (doesn't have to be C or Python) which can take a .h file and produce a structured list of its structs and their fields? I'd love to be able to write a script to generate my automatically generate my struct definitions in Python, and I don't want to have to process arbitrary C code to do it. Regular expressions would work great about 90% of the time and then cause endless headaches for the remaining 10%.
If you compile your C code with debugging (-g), pahole (git) can give you the exact structure layouts being used.
$ pahole /bin/dd
…
struct option {
const char * name; /* 0 8 */
int has_arg; /* 8 4 */
/* XXX 4 bytes hole, try to pack */
int * flag; /* 16 8 */
int val; /* 24 4 */
/* size: 32, cachelines: 1, members: 4 */
/* sum members: 24, holes: 1, sum holes: 4 */
/* padding: 4 */
/* last cacheline: 32 bytes */
};
…
This should be quite a lot nicer to parse than straight C.
Regular expressions would work great about 90% of the time and then cause endless headaches for the remaining 10%.
The headaches happen in the cases where the C code contains syntax that you didn't think of when writing your regular expressions. Then you go back and realise that C can't really be parsed by regular expressions, and life becomes not fun.
Try turning it around: define your own simple format, which allows less tricks than C does, and generate both the C header file and the Python interface code from your file:
define socketopts
int16 port
int32 ipv4address
int32 flags
Then you can easily write some Python to convert this to:
typedef struct {
short port;
int ipv4address;
int flags;
} socketopts;
and also to emit a Python class which uses struct to pack/unpack three values (possibly two of them big-endian and the other native-endian, up to you).
Have a look at Swig or SIP that would generate interface code for you or use ctypes.
Have you looked at Swig?
I have quite successfully used GCCXML on fairly large projects. You get an XML representation of the C code (including structures) which you can post-process with some simple Python.
ctypes-codegen or ctypeslib (same thing, I think) will generate ctypes Structure definitions (also other things, I believe, but I only tried structs) by parsing header files using GCCXML. It's no longer supported, but will likely work in some cases.
One my friend for this tasks done C-parser which he use with cog.