def add(a,b):
for i in range(len(a)):
a[i] = a[i] + b
def main():
amounts = [100,200]
rate = 1
add(amounts,rate)
print amounts
main()
The function add does not have a return. I read that changes are available to only mutable objects like list. But why did the person omits the return? Either with or without return is fine. Why? This is so different from C++.
Thanks
But why did the person omits the
return? Either with or without return
is fine. Why? This is so different
from C++.
Not at all - it's identical to C++ to all intent and purposes! Just make, in the C++ version, a void add and pass its argument a, say a std::vector<int>, by reference -- to all intents and purposes, this is what this Python add is doing, seen in C++ terms.
In Python terms, when a function "falls off the end" that's exactly the same as if it executed return None at that point. It's better style in such cases (when a function always ends by "falling off the end") to avoid the redundant return None statement (don't waste pixels and screen space in redundant ornamentation of this kind).
add() mutates a instead of rebinding it, so the change shows up in the original object.
Everything is passed by reference in python, but integers, strings etc. are immutable so when you change it you create a new one which is bound to the local variable so the variable passed to the function isn't changed.
Lists and dicts are, however, mutable - so if you change them no new object is created and due to this the change also affects the variable in the caller's scope.
Consider the following C++ program:
#include <vector>
#include <iostream>
void add_val(std::vector<int> addTo, int addThis)
{
for(std::vector<int>::iterator it = addTo.begin(); it!=addTo.end(); ++it)
{
*it += addThis;
}
}
void add_ref(std::vector<int>& addTo, int addThis)
{
for(std::vector<int>::iterator it = addTo.begin(); it!=addTo.end(); ++it)
{
*it += addThis;
}
}
int main()
{
std::vector<int> myVector;
myVector.push_back(1);
myVector.push_back(2);
myVector.push_back(3);
add_val(myVector, 3);
std::cout<<"After add_val"<<std::endl;
for (std::vector<int>::iterator it = myVector.begin(); it!=myVector.end(); ++it)
{
std::cout<<*it<<" ";
}
std::cout<<std::endl;
add_ref(myVector, 3);
std::cout<<"After add_ref"<<std::endl;
for (std::vector<int>::iterator it = myVector.begin(); it!=myVector.end(); ++it)
{
std::cout<<*it<<" ";
}
std::cout<<std::endl;
return 0;
}
The program outputs:
After add_val
1 2 3
After add_ref
4 5 6
Passing the vector to add_val() results in the original vector remaining unchanged, since it is passed by value. Passing the vector to add_ref() however, causes the values inside the original vector to change, since it's passed by reference.
In Python everything is passed by reference. However, a lot of the builtin types (str, tuple, int, float, etc.) are immutable. This means that any operation you perform on these types results in a new variable being bound in the current scope with the new value. For mutable types (list, dict, etc.), you end up with exactly the same result as passing a parameter by reference in C++.
Related
Context:
I have written a Red Black tree implementation in C language. To allow it to use variable types, it only handles const void * elements, and initialisation of a tree must be given a comparison function with a signature int (*comp)(const void *, const void *). So far, so good, but I now try to use that C code to build an extension module for Python. It looks simple as first sight, because Python languages always pass references to objects which are received as pointers by C routines.
Problem:
Python objects come with rich comparison operators. That means that from a C extension module, comparing 2 arbitrary objects is trivial: just a matter of using int PyObject_RichCompareBool(PyObject *o1, PyObject *o2, int opid).
But the comparison may return -1 to indicate that the objects are not comparable. In Python or C++ it would be simple enough to throw an exception to signal an abnormal condition. Unfortunately C has no notion of exception, and I could not find a way using setjmp-longjmp because:
the environment buffer has do be known to both the englobing function and the internal one
I should free any allocated memory at longjmp time, when the internal function does not know what has been allocated
First idea:
A simple solution is to give a third parameter to the comparison function for it to signal an abnormal condition. But when the library is used in a plain C environment, that third parameter just does not make sense. I then remembered that in the 80', I had learned that in C language, parameters were passed in the stack in reversed order and unstacked by the caller to allow functions with a variable number of parameters. That means that provided the first 2 parameters are correct passing a third parameter to a function expecting 2 should be harmless.
Demo code:
#include <stdio.h>
// declares a type for the comparison functions
typedef int (*func)();
// A simple function for comparing integers - only 2 params
int f1(int a, int b) {
return a - b;
}
/* Inserts a value into an increasing array
* By convention 0 denotes the end of the array
* No size control implemented for brievety
* The comp function recieves a pointer to an int
* to be able to signal abnormal conditions
* */
int insert(int* arr, int val, func comp) {
int err = 0;
while ((0 != *arr) && (comp(*arr, val, &err) < 0)) { // 1
if (err) return 0;
++arr;
}
do {
int tmp = *arr;
*arr = val;
val = tmp;
} while (0 != *arr++);
return 1;
}
int main() {
func f = &f1;
// a simple test with 3 parameters
int cr = f(3, 1, 5); // 2
printf("%d\n", cr);
// demo usage of the insert function
int arr[10] = {0};
int data[] = { 1,5,3,2,4 };
for (int i = 0; i < sizeof(data) / sizeof(*data); i++) {
insert(arr, data[i], f1);
}
for (int i = 0; i < sizeof(data) / sizeof(*data); i++) {
printf("%d ", arr[i]);
}
return 0;
}
At (1) and (2) the 2 parameter function is called with 3 parameters. Of course, this code compiles without even a warning in Clang or MSVC, and runs fine giving the expected result.
Question:
While this code works fine on common implementations, I wonder whether actually passing a third parameter to a function expecting only two is really legit or does it invokes Undefined Behaviour?
Current research
Is it safe to invoke a C function with more parameters than it expects? : the accepted answer suggests that it should be safe when the C calling convention is used (which is my use case) while other answers show that the MSVC stdcall calling convention would not allow it
6.7.6.3 Function declarators (including prototypes) and 6.5.2.2 Function calls in draft n1570 for C11, but as English is not my first language, I could not understand where it was or not allowed
Remark:
The originality of this question is that it uses function pointers conversions.
I think it invokes Undefined Behavior.
From 6.5.2.2p6:
If the expression that denotes the called function has a type that does not include a prototype, the integer promotions are performed on each argument, and arguments that have type float are promoted to double. These are called the default argument promotions. If the number of arguments does not equal the number of parameters, the behavior is undefined.
The proper solution is redesign the Red Black tree implementation to allow passing a context as a third parameter.
int (*comp)(const void *, const void *, void *);
It is highly recommended to add a context argument to any function pointer type to allow emulate closures.
As a workaround, you could use a global variable.
static int err;
int f1(int a, int b) {
err = 0;
return a - b;
}
int insert(int* arr, int val, int comp(int,int)) {
err = 0;
while ((0 != *arr) && (comp(*arr, val) < 0)) { // 1
if (err) return 0;
++arr;
}
...
}
It is not the best solution because it is not re-entrant. Only a single instance of insert()/f1() can run at a time.
This is a complement to the accepted answer. The shown code uses function pointers to solve the compilation errors that would arise when calling a prototyped function with a wrong number of parameters.
But the draft n1570 for C11 says at 6.3.2.3 [Language/Conversions/Other operands/] Pointers ยง8:
... If a converted
pointer is used to call a function whose type is not compatible with the referenced type,
the behavior is undefined.
And it fully applies here because the referenced type is a function taking 2 parameters and the converted pointer type is a function taking 3 parameters. Per the accepted answer and 6.5.2.2p6 those two function type are not compatible, hence the code does invoke UB.
After finding that, I haved decided to give up with that way, and instead choosed to use wrapper functions that call the function passed to the library with their expected number of arguments to avoid UB.
I have a C function
int * myfunc()
{
int * ret = (int *) malloc(sizeof(int)*5);
...
return ret;
}
in python I can call it as
ret = lib.myfunc()
but I can't seem to figure out how to actually use ret in the python code (i.e. cast it to an int array of length 5.
I see lots of documentation (and questions here) about how to pass a python array into a C function, but not how one deals with an array returned from a C function.
the only thing I've figured out so far (which sort of works, but seems ugly)
buf = ffi.buffer(ret,ffi.sizeof("int")*5)
int_array = ffi.from_buffer("int *", buf)
is that what I'm supposed to do? or is there a better way?
In C, the type int * and the type int[5] are equivalent at runtime. That means that although you get a int * from calling the function, you can directly use it as if it were a int[5]. All the same operations work with the exception of len(). For example:
ret = lib.myfunc()
my_python_list = [ret[i] for i in range(5)]
lib.free(ret) # see below
As the function called malloc(), you probably need to call free() yourself, otherwise you have a memory leak. Declare it by adding the line void free(void *); to the cdef() earlier.
There are some more advanced functions whose usage is optional here: you could cast the type from int * to int[5] with p = ffi.cast("int[5]", p); or instead you could convert the C array into a Python list with an equivalent but slightly faster call my_python_list = ffi.unpack(ret, 5).
I am currently writing a simple function in C, which is structured in this way:
int *fillArray(int dim)
{
static int ar[dim];
// fill the array ar in some way
return ar;
}
It is usually said that using the static keyword in local function is discouraged. I was wondering if it was better to do in the classic way:
void fillArray(int *ar, int dim)
{
// fill the array ar in some way
}
As a further fact, consider that I later want to wrap the function in Python code, and the Python function should not take parameters.
int *fillArray(dim)
{
static int ar[dim];
// fill the array ar in some way
return ar;
}
Fill array using static does not makes much sense. Every time this function will be called, this will return same array (same address of the static array). Therefore, multiple calls to fillArray which returns a static variable, may in fact corrupt the previous use of the array. Also, you should ideally never return the address of a variable locally defined.
Also, second fillArray function makes much sense, as it can be actually reused.
As suggested by #alk the first solution does not even compile. Thus the second solution is mandatory, at least for VLA.
Your fillArray function is incorrect: the size of an object with static duration must be known at compile time. Furthermore, asa suggested by user3150716, returning the address of a local object with static storage would return the same array every time, which might not be the intent.
You should use a different approach:
write a function to allocate a new array
use a different function to initialize an array with its size passed as an argument.
Here is an example:
#include <stdlib.h>
int *fillArray(int *array, size_t size) {
if (array != NULL) {
for (size_t i = 0; i < size; i++)
array[i] = 0;
}
}
return array;
}
int *newArray(size_t size) {
return fillArray(malloc(size * sizeof(int)), size);
}
void freeArray(int *array) {
free(array);
}
Note that allocating and initializing the array to 0 can be performed in a single step with calloc(size, sizeof(int)).
I am writing a Python C extension. I am passing a Python dict to a C function. I am able to parse it, using the following code:
PyObject *large_dict = NULL;
if (!PyArg_ParseTuple(args, "O!", &PyDict_Type, &large_dict)) return NULL;
if (large_dict != NULL)
{
printf("Large Dictionary Not Null\n");
}
Here the statement "Large Dictionary Not Null" is printed, which means that the dictionary is parsed successfully.
Now I want to access the dictionary values by specifying keys, like we do in Python. For example, large_dict['k1'] would give value v1.
How can I access dictionary keys/values inside this C function?
You should go through the link, https://docs.python.org/2/c-api/dict.html
Excerpt given below,
PyObject* PyDict_GetItem(PyObject *p, PyObject *key)
Return value: Borrowed reference.
Return the object from dictionary p which has a key key. Return NULL if the key key is not present, but without setting an exception.
PyObject* PyDict_GetItemString(PyObject *p, const char *key)
Return value: Borrowed reference.
This is the same as PyDict_GetItem(), but key is specified as a char*, rather than a PyObject*.
PyObject* PyDict_Items(PyObject *p)
Return value: New reference.
Return a PyListObject containing all the items from the dictionary, as in the dictionary method dict.items().
PyObject* PyDict_Keys(PyObject *p)
Return value: New reference.
Return a PyListObject containing all the keys from the dictionary, as in the dictionary method dict.keys().
PyObject* PyDict_Values(PyObject *p)
Return value: New reference.
Return a PyListObject containing all the values from the dictionary p, as in the dictionary method dict.values().
Keep an eye on borrowed reference / new reference. It is little tricky while coding for Python extensions.
I'll reply for Python 3.7+, since that's the most likely use case these days (see reference). Converting from/to C types is dependent on the Python type of the keys/values, which you haven't specified. For the sake of illustrating an actual use-case, let's assume the keys and values are both positive integers. Then if you want to retrieve the value for k1=42 then I think you can do this:
unsigned int k1 = 42;
PyObject *py_k1 = PyLong_FromUnsignedLong(k1);
PyObject *py_v1 = PyDict_GetItem(large_dict, py_k1);
unsigned int v1 = PyLong_AsUnsignedLong(py_v1);
Breaking this down line by line, the first line defines the key as a plain C unsigned int. But we can't use a regular C type to index into a Python dict, therefore in the second line we define a pointer to a Python type that will point to the key. We then do the C equivalent of Python's large_dict[k1] via PyDict_GetItem, but using the pointer we created as the key and obtaining a pointer to a Python type as the value. Finally, if we want to access the value as a regular C type, we need to convert it in the last line, which is the reverse of what we did in the second line.
Note: PyDict_GetItem returns a borrowed reference, so we don't need to worry about reference counting.
I am currently translating a rospy IMU-driver to roscpp and have difficulites figuring out what this piece of code does and how I can translate it.
def ReqConfiguration(self):
"""Ask for the current configuration of the MT device.
Assume the device is in Config state."""
try:
masterID, period, skipfactor, _, _, _, date, time, num, deviceID,\
length, mode, settings =\
struct.unpack('!IHHHHI8s8s32x32xHIHHI8x', config)
except struct.error:
raise MTException("could not parse configuration.")
conf = {'output-mode': mode,
'output-settings': settings,
'length': length,
'period': period,
'skipfactor': skipfactor,
'Master device ID': masterID,
'date': date,
'time': time,
'number of devices': num,
'device ID': deviceID}
return conf
I have to admit that I never ever worked with neither ros nor python before.
This is no 1:1 code from the source, I removed the lines I think I know what they do, but especially the try-block is what I don't understand. I would really appreciate help, because I am under great preasure of time.
If someone is curious(context reasons): The files I have to translate are mtdevice.py , mtnode.py and mtdef.py and can be found googleing for the filesnames + the keyword ROS IMU Driver
Thanks a lot in advance.
This piece of code unpacks the fields of a C structure, namely masterID, period, skipfactor, _, _, _, date, time, num, deviceID, length, mode, settings, stores those in a Python dictionary and returns that dictionary as call result. The underscores are placeholders for the parts of the struct that aren't used.
See also: https://docs.python.org/2/library/struct.html, e.g. for a description of the format string ('!IHHHHI8s8s32x32xHIHHI8x') that tells the unpack function what the struct looks like.
The syntax a, b, c, d = f () means that the function returns a thing called a tuple in Python. By assigning a tuple to multiple variables, it's split into its fields.
Example:
t = (1, 2, 3, 4)
a, b, c, d = t
# At this point a == 1, b == 2, c == 3, d == 4
To replace this piece of code by C++ should not be too hard, since C++ has structs much like C. So the simplest C++ implementation of requestConfiguration would be to just return that struct. If you want to stay closer to the Python functionality, your function could put the fields of the struct into a C++ STL map and return that. The format string + the docs that the link points to, tell you what data types are in your struct and where.
Note that it's the second parameter of unpack that holds your data, the first parameter just contains information on the layout (format) of the second parameter, as explained in the link. The second parameter looks to Python as if it's a string, but it's actually a C struct. The first parameter tells Python where to find what in that struct.
So if you read the docs on format strings, you can find out the layout of your second parameter (C struct). But maybe you don't need to. It depends on the caller of your function. It may just expect the plain C struct.
From your added comments I understand that there's more code in your function than you show. The fields of the structs are assigned to attributes of a class.
If you know the field names of your C struct (config) then you can assign them directly to the attributes of your C++ class.
// Pointer 'this' isn't needed but inserted for clarity
this->mode = config.mode;
this->settings = config.settings;
this->length = config.length;
I've assumed that the field names of the config struct are indeed mode, settings, length etc. but you'd have to verify that. Probably the layout of this struct is declared in some C header file (or in the docs).
To do the same thing with C++, you'd declare a struct with the various parameters:
struct DeviceRecord {
uint32_t masterId;
uint16_t period, skipfactor, _a, _b;
uint32_t _c;
char date[8];
char time[8];
char padding[64];
uint16_t num;
uint32_t deviceID;
uint16_t length, mode;
uint32_t settings;
char padding[8];
};
(It's possible this struct is already declared somewhere; it might also use "unsigned int" instead of "uint32_t" and "unsigned short" instead of "uint16_t", and _a, _b, _c would probably have real names.)
Once you have your struct, the question is how to get the data. That depends on where the data is. If it's in a file, you'd do something like this:
DeviceRecord rec; // An instance of the struct, whatever it's called
std::ifstream fin("yourfile.txt", std::ios::binary);
fin.read(reinterpret_cast<char*>(&rec), sizeof(rec));
// Now you can access rec.masterID etc
On the other hand, if it's somewhere in memory (ie, you have a char* or void* to it), then you just need to cast it:
void* data_source = get_data(...); // You'd get this from somewhere
DeviceRecord* rec_ptr = reinterpret_cast<DeviceRecord*>(stat_source);
// Now you can access rec_ptr->masterID etc
If you have a std::vector, you can easily get such a pointer:
std::vector<uint8_t> data_source = get_data(...); // As above
DeviceRecord* rec_ptr = reinterpret_cast<DeviceRecord*>(data_source.data());
// Now you can access rec_ptr->masterID etc, provided data_source remains in scope. You should probably also avoid modifying data_source.
There's one more issue here. The data you've received is in big-endian, but unless you have a PowerPC or other unusual processor, you're probably on a little-endian machine. So you need to do a little byte-swapping before you access the data. You can use the following function to do this.
template<typename Int>
Int swap_int(Int n) {
if(sizeof(Int) == 2) {
union {char c[2]; Int i;} swapper;
swapper.i = n;
std::swap(swapper.c[0], swapper.c[1]);
n = swapper.i;
} else if(sizeof(Int) == 4) {
union {char c[4]; Int i;} swapper;
swapper.i = n;
std::swap(swapper.c[0], swapper.c[3]);
std::swap(swapper.c[1], swapper.c[2]);
n = swapper.i;
}
return n;
}
These return the swapped value rather than changing it in-place, so now you'd access your data with something like swap_int(rec->num). NB: The above byte-swapping code is untested; I'll try compiling it a bit later and fix it if necessary.
Without more information, I can't give you a definitive way of doing this, but perhaps this will be enough to help you work it out on your own.