We Keep Coding

Is it legal to use more parameters than expected when calling a function?

Context:
I have written a Red Black tree implementation in C language. To allow it to use variable types, it only handles const void * elements, and initialisation of a tree must be given a comparison function with a signature int (*comp)(const void *, const void *). So far, so good, but I now try to use that C code to build an extension module for Python. It looks simple as first sight, because Python languages always pass references to objects which are received as pointers by C routines.
Problem:
Python objects come with rich comparison operators. That means that from a C extension module, comparing 2 arbitrary objects is trivial: just a matter of using int PyObject_RichCompareBool(PyObject *o1, PyObject *o2, int opid).
But the comparison may return -1 to indicate that the objects are not comparable. In Python or C++ it would be simple enough to throw an exception to signal an abnormal condition. Unfortunately C has no notion of exception, and I could not find a way using setjmp-longjmp because:
the environment buffer has do be known to both the englobing function and the internal one
I should free any allocated memory at longjmp time, when the internal function does not know what has been allocated
First idea:
A simple solution is to give a third parameter to the comparison function for it to signal an abnormal condition. But when the library is used in a plain C environment, that third parameter just does not make sense. I then remembered that in the 80', I had learned that in C language, parameters were passed in the stack in reversed order and unstacked by the caller to allow functions with a variable number of parameters. That means that provided the first 2 parameters are correct passing a third parameter to a function expecting 2 should be harmless.
Demo code:
#include <stdio.h>
// declares a type for the comparison functions
typedef int (*func)();
// A simple function for comparing integers - only 2 params
int f1(int a, int b) {
return a - b;
}
/* Inserts a value into an increasing array
* By convention 0 denotes the end of the array
* No size control implemented for brievety
* The comp function recieves a pointer to an int
* to be able to signal abnormal conditions
* */
int insert(int* arr, int val, func comp) {
int err = 0;
while ((0 != *arr) && (comp(*arr, val, &err) < 0)) { // 1
if (err) return 0;
++arr;
}
do {
int tmp = *arr;
*arr = val;
val = tmp;
} while (0 != *arr++);
return 1;
}
int main() {
func f = &f1;
// a simple test with 3 parameters
int cr = f(3, 1, 5); // 2
printf("%d\n", cr);
// demo usage of the insert function
int arr[10] = {0};
int data[] = { 1,5,3,2,4 };
for (int i = 0; i < sizeof(data) / sizeof(*data); i++) {
insert(arr, data[i], f1);
}
for (int i = 0; i < sizeof(data) / sizeof(*data); i++) {
printf("%d ", arr[i]);
}
return 0;
}
At (1) and (2) the 2 parameter function is called with 3 parameters. Of course, this code compiles without even a warning in Clang or MSVC, and runs fine giving the expected result.
Question:
While this code works fine on common implementations, I wonder whether actually passing a third parameter to a function expecting only two is really legit or does it invokes Undefined Behaviour?
Current research
Is it safe to invoke a C function with more parameters than it expects? : the accepted answer suggests that it should be safe when the C calling convention is used (which is my use case) while other answers show that the MSVC stdcall calling convention would not allow it
6.7.6.3 Function declarators (including prototypes) and 6.5.2.2 Function calls in draft n1570 for C11, but as English is not my first language, I could not understand where it was or not allowed
Remark:
The originality of this question is that it uses function pointers conversions.

I think it invokes Undefined Behavior.
From 6.5.2.2p6:
If the expression that denotes the called function has a type that does not include a prototype, the integer promotions are performed on each argument, and arguments that have type float are promoted to double. These are called the default argument promotions. If the number of arguments does not equal the number of parameters, the behavior is undefined.
The proper solution is redesign the Red Black tree implementation to allow passing a context as a third parameter.
int (*comp)(const void *, const void *, void *);
It is highly recommended to add a context argument to any function pointer type to allow emulate closures.
As a workaround, you could use a global variable.
static int err;
int f1(int a, int b) {
err = 0;
return a - b;
}
int insert(int* arr, int val, int comp(int,int)) {
err = 0;
while ((0 != *arr) && (comp(*arr, val) < 0)) { // 1
if (err) return 0;
++arr;
}
...
}
It is not the best solution because it is not re-entrant. Only a single instance of insert()/f1() can run at a time.

This is a complement to the accepted answer. The shown code uses function pointers to solve the compilation errors that would arise when calling a prototyped function with a wrong number of parameters.
But the draft n1570 for C11 says at 6.3.2.3 [Language/Conversions/Other operands/] Pointers §8:
... If a converted
pointer is used to call a function whose type is not compatible with the referenced type,
the behavior is undefined.
And it fully applies here because the referenced type is a function taking 2 parameters and the converted pointer type is a function taking 3 parameters. Per the accepted answer and 6.5.2.2p6 those two function type are not compatible, hence the code does invoke UB.
After finding that, I haved decided to give up with that way, and instead choosed to use wrapper functions that call the function passed to the library with their expected number of arguments to avoid UB.

How can I use a python variable as an input / output for a C function using Cython?

So I have a C implementation of a hashmap where I can store heterogeneous data. The interface for it looks something like this:
error_e put(const char* key, const type_e type, const void* data);
error_e get(const char* key, const type_e type, void* data);
Where error_e is an enum defining all the various errors I can receive and type_e is an enum defining all the various types of data I can store.
Now I want to create a python wrapper for this using cython. This is what I have so far in my hashmap.pyx:
cdef extern from "hashmap.h":
cdef enum error_e:
# some enum values...
cdef enum type_e:
type_empty = 0,
type_int32 = 1,
type_float = 2,
type_string = 3
error_e put(const char* key, const type_e type, const void* value)
error_e get(const char* key, const type_e type, void* value)
class Hashmap:
def __init__(self):
self.type_dict = {
int: type_int32,
float: type_float,
str: type_string
}
def put(self, key, value):
if not type(value) in self.type_dict:
return unknown_type
# Not sure what to do here?
return put(key.encode('utf-8'), self.type_dict[type(value)], &value) # Obviously doesn't work
def get(self, key, val_type):
if not val_type in self.type_dict:
return unknown_type
# Also not really sure what to do here
res = get(key.encode('utf-8'), self.type_dict[val_type], # How can I get data out of here?)
return (res, data)
I'm having problems with getting a void pointer from the value that should be written / read from the hashmap from a python object. I've tried to use if else statements for the different types and converting the input to them, but I can't seem to be able to then get a pointer to the correct thing.
So this leads me to my questions:
1) How could I use the value of a python variable as an input to a C function that takes a void*?
2) How could I get data out of a C function and into a python variable if the function takes a void*?
edit 1:
Some additional information about how the C functions are supposed to be used:
The void* supplied to them is a pointer to a C object with the value you want to store / get. The type of the value is supplied using the type parameter. Currently only ints, floats and strings are supported.
I don't want to store python objects in the hashmap, nor do I want the C code to have any reference to python, since I want to store values using python, save the hashmap and then retrieve and use them using C.
edit 2:
Here's a small example of how you could add values to the hashmap in C:
int32_t some_int = 5;
put("int_key", type_int32, &some_int);
float some_float = 3.1416;
put("pi", type_float, &some_float);
char* some_string = "some text you want to store";
put("text", type_string, some_string);
And here how you'd get values out of the same one:
int32_t returned_int;
get("int_key", type_int32, &returned_int);
float returned_float;
get("pi", type_float, &returned_float);
char returned_string[30];
get("text", type_string, returned_string);
When storing or retrieving something from the hashmap, it will always either copy your data into its internal storage or copy the data from the storage to the supplied "buffer" location.

One thing we still don't know if how this library expects to be used from C. The Cython implementation will almost certainly look a lot like the C implementation and without that it's largely just a guess.
However, if I were doing this in C I think the structure I'd be using us a "Union" - you can define it in Cython as:
cdef union DataType:
int32_t as_int
float as_float
char* as_string
put would then look something like:
cdef DataType data
if isinstance(value, int):
data.as_int = value
# etc.
return put(key.encode('utf-8'), self.type_dict[type(value)], <void*>(&data)) # Obviously doesn't work
get is kind of the inverse:
cdef DataType data
res = get(key.encode('utf-8'), self.type_dict[val_type], <void*>(&data))
if something_to_do_with_val_type is int:
out = data.as_int
# etc.
Obviously the C string needs some extra thought because (as ever) something needs to own the memory. You really run the risk of ending up with a pointer to the internals of a Python object that no longer exists. I don't have enough information to even make an intelligent guess as to how you should handle that.

Python to C conversion, reading columns of csv into arrays

I am looking to read a csv file line by line and write each column to arrays in C. Essentially I want to convert the following Python code to C:
import csv
date = []
x = []
y = []
with open("file.csv") as old:
read = csv.reader(old,delimiter = ',')
for row in read:
date.append(row[0])
x.append(float(row[1]))
y.append(float(row[2]))
The csv file has 128 rows and three columns; date,x,y. My thoughts:
char Date[];
int Date[128], i;
for(i = 0; i < 128; i++)
{
Date[i] = x;
}
This is a simple example I have attempted to fill an array with values within a for loop. I want to know how I can modify this to fill arrays with each line of a csv file split by the ',' delimiter? I want to use the fscanf function but am unsure about how to incorporate it into the above setting?
Attempt:
FILE* f = fopen("file.csv", "r");
fscanf(f, "%char, %f %f", time, &value, &value);
Update:
The following code reads in a text file of my data and outputs to the screen:
#include <stdio.h>
int main(void)
{
char buff[128];
FILE * myfile;
myfile = fopen("File.txt","r");
while (!feof(myfile))
{
fgets(buff,128,myfile);
printf("%s",buff);
}
fclose(myfile);
return 0;
}
Instead of outputting to the screen, I want to store each column as an array. Any suggestions on how to do this?
Update 2.
I have updated the code as follows:
#include <stdio.h>
#include<string.h>
int main(void)
{
char buff[128];
char * entry;
FILE * myfile;
myfile = fopen("file.txt","r");
while(fgets(buff,128,myfile)){
puts(buffer);
entry = strtok(buff,",");
while(entry!=NULL)
{
printf("%s\n",entry) ;
entry = strtok(NULL,",");
}
}
return 0;
}
Final Update.
I have found an example that does something very similar and is much more intuitive for me (given my limited ability in C)
https://cboard.cprogramming.com/c-programming/139377-confused-parsing-csv-txt-file-strtok-fgets-sscanf.html
Updated code
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main()
{
char line[200];
int i = 0;
int x[50];
int y[50];
int z[50];
FILE *myFile;
myFile = fopen("file.txt", "rt");
while(fgets(line, sizeof line, myFile) != NULL)
{
if(sscanf(line, "%d,%d,%d", &x[i], &y[i],&z[i]) == 3)
{
++i;
}
}
//Close the file
fclose(myFile);
getch();
return 0;
}
This code works for me provided $x$, $y$ and $z$ are integers/floats. However, when $x$ is a date, I am unable to parse it. The date is of the form $Year-Month-day-Time$.
My attempt
I have tried changing the line
if(sscanf(line, "%d,%d,%d", &x[i], &y[i],&z[i]) == 3)
to
if(sscanf(line, "%d-%d-%d-%d",&year[i],&month[i], &day[i],&time[i], "%d,%d", &y[i],&z[i]) == 3)
and have declared new arrays
int year, int month, int day, int time
However, this approach gives garbage as the output. Any suggestions on how to modify this code to read and parse dates correctly?

You can use strtok to split a string by delimiter.
This describes the function. It includes a simple example.
Each call gives you a substring, ending where the delimiter was found. You can then copy that substring into a string in an array of strings (of max length, if you know the max length) (or you can have an array of pointers to strings, and allocate memory for each one before the copy to strlen(substring) + 1 (+1 for NULL terminator).
Note that strtok temporarily modifies the existing string, so you must either use it right away or copy it. If you just save a pointer to what strtok returns, when you finish the sequence of strtok calls the string will be restored to its original form and your "substrings" will not be what you expect. (See answer to this question for an explanation.)
And, please do not use !feof(myfile) to control the exit from your loop. See this post for an explanation. Since BradS gave an alternative in his comment, and it is not your main question, I won't repeat him here.
OK, looking at your sscanf approach and your question about dates:
I have used the various scanf and printf functions many times; I have never seen two different format strings in the same call, with parameters in the middle. From the function prototypes, this can't work. I think the only reason it compiles is because the variable parameter list in a variadic function does not have type checking. It will simply ignore these parameters: "%d,%d", &y[i],&z[i], because these are output parameters which do not correspond to anything in the format string. You can do something like:
if(sscanf(line, "%d-%d-%d-%d,%d,%d",&year[i],&month[i], &day[i],&time[i],&y[i],&z[i]) == 6)
This includes all the parameters in the format string.
Also, you mention creating arrays like this:
int year, int month, int day, int time
Those are not arrays. They are simple integers. You can declare an array like:
int year[MAX_SIZE];
I personally have found the scanf set of functions rather difficult to deal with; if the format string doesn't quite match the reality in the string you won't get what you expect/need. That is why I like strtok. But not everyone is comfortable with it; it is not the most obvious or intuitive interface.

How to write an LLDB synthetic provider for a shaped view of std::vector data

I am trying to create LLDB visualizers for classes in my project. The LLDB documentation is... sparse. I have an array class that stores the underlying data in a std::vector and has an extent array to describe the shape. It can also be reshaped later.
By default, the std::vector "data_" is always shown as a linear vector. I would like my provider to create a view hierarchy. In this example, the first level would be the child rows, each row expanding to a list of column values. Similar to viewing a static 2D array (i.e. double[3][2]). You can imagine extending this to N dimensions.
I can't seem to figure out how to use the lldb python object model to impose hierarchical structure onto the linear buffer in std::vector.
Nothing seems to be documented, and I have been guessing in the dark for about a week. Here is a simplified example array class that I would like to create a visualizer for.
Any help is greatly appreciated!
#include <vector>
#include <cassert>
template <typename T>
class myarray {
int extent_[2];
std::vector<T> data_;
public:
myarray(int r, int c, const T* data) {
extent_[0] = r;
extent_[1] = c;
data_.resize(r * c);
for(size_t i = 0; i < data_.size(); ++i) data_[i] = data[i];
}
void reshape(int r, int c) {
assert(r * c == data_.size());
extent_[0] = r;
extent_[1] = c;
}
};
int main(int argc, const char * argv[])
{
double initdata[6] = { 0, 1, 2, 3, 4, 5 };
myarray<double> mydata(3, 2, initdata);
mydata.reshape(1, 6);
return 0;
}
As requested: The output I would like to see for the first [3][2] example might look like the following. The first level of 3 children are "rows", with a summary string of the leading elements in the row. The idea is to get a 2D view of the matrix data. Then when a row is expanded, it would be viewed as an array of column values.
LLDB potential synthetic output:
mydata
[0]
[0] = 0 <-- expanded contents
[1] = 1
[1] = {2, 3} <-- summary string of row contents. First N elements, then ...
[2] = {4, 5}
The synthetic provider examples for a simple vector implement get_child_at_index something like this, where I determined the count, value_size, and value_type in the update() method:
def get_child_at_index(self,index):
logger = lldb.formatters.Logger.Logger()
logger >> "get_child_at_index: " + str(index)
if index < 0: return None;
if index >= self.count: return None;
try:
offset = index * self.value_size
return self.data.CreateChildAtOffset('['+str(index)+']',offset,self.value_type)
except:
return None
I think I can easily work this out if I could just figure out how to create an SBType to use in place of value_type when calling CreateChildAtOffset. I think I could then lay down any kind of structure that I like. However, with many shots in the dark, I couldn't figure out how to create an SBType object successfully.
Ideas? Does anyone know how to create an SBType from a string that I compose?

I am assuming you have already looked over: http://lldb.llvm.org/varformats.html
IIUC, what you want to do is display the elements of the vector in a more hierarchical format.
It's kind of an interesting task, one for which you're probably going to have to craft your own data types - something for which I don't think we have a whole lot of support in our public API currently.
As a workaround, you can of course run an expression that generates the struct you care about and hold on to it - however that is going to be slow.
In your example, what exactly is the view you'd like to get? That kind of by example information can actually be helpful in figuring out more details.
EDIT: Currently LLDB doesn't let you create new types through the public API. What you can do to get your hands on an SBType of your own making is use the expression parser, as in this example:
x = lldb.frame.EvaluateExpression("struct foo { int x; }; foo myfoo = {12}; myfoo")
data = lldb.SBData.CreateDataFromSInt32Array(lldb.eByteOrderLittle,8,[24])
x_type = x.GetType()
myOtherFoo = x.CreateValueFromData("myOtherFoo",data,x_type)
print myOtherFoo
OUTPUT: (foo) myOtherFoo = (x = 24)
This is going to be fairly slow, especially if you don't cache the foo type you need (which from your example seems to be a T[2] for your template argument T) - but until LLDB has SB API to create types through clang (like we do internally), this is your only approach

Not sure if this will help, but you can find existing types
target = lldb.debugger.GetSelectedTarget()
type_list = target.FindTypes('base::Value')
if you want to create your child with an existing type, that may help.

python function that modifies parameters

def add(a,b):
for i in range(len(a)):
a[i] = a[i] + b
def main():
amounts = [100,200]
rate = 1
add(amounts,rate)
print amounts
main()
The function add does not have a return. I read that changes are available to only mutable objects like list. But why did the person omits the return? Either with or without return is fine. Why? This is so different from C++.
Thanks

But why did the person omits the
return? Either with or without return
is fine. Why? This is so different
from C++.
Not at all - it's identical to C++ to all intent and purposes! Just make, in the C++ version, a void add and pass its argument a, say a std::vector<int>, by reference -- to all intents and purposes, this is what this Python add is doing, seen in C++ terms.
In Python terms, when a function "falls off the end" that's exactly the same as if it executed return None at that point. It's better style in such cases (when a function always ends by "falling off the end") to avoid the redundant return None statement (don't waste pixels and screen space in redundant ornamentation of this kind).

add() mutates a instead of rebinding it, so the change shows up in the original object.

Everything is passed by reference in python, but integers, strings etc. are immutable so when you change it you create a new one which is bound to the local variable so the variable passed to the function isn't changed.
Lists and dicts are, however, mutable - so if you change them no new object is created and due to this the change also affects the variable in the caller's scope.

Consider the following C++ program:
#include <vector>
#include <iostream>
void add_val(std::vector<int> addTo, int addThis)
{
for(std::vector<int>::iterator it = addTo.begin(); it!=addTo.end(); ++it)
{
*it += addThis;
}
}
void add_ref(std::vector<int>& addTo, int addThis)
{
for(std::vector<int>::iterator it = addTo.begin(); it!=addTo.end(); ++it)
{
*it += addThis;
}
}
int main()
{
std::vector<int> myVector;
myVector.push_back(1);
myVector.push_back(2);
myVector.push_back(3);
add_val(myVector, 3);
std::cout<<"After add_val"<<std::endl;
for (std::vector<int>::iterator it = myVector.begin(); it!=myVector.end(); ++it)
{
std::cout<<*it<<" ";
}
std::cout<<std::endl;
add_ref(myVector, 3);
std::cout<<"After add_ref"<<std::endl;
for (std::vector<int>::iterator it = myVector.begin(); it!=myVector.end(); ++it)
{
std::cout<<*it<<" ";
}
std::cout<<std::endl;
return 0;
}
The program outputs:
After add_val
1 2 3
After add_ref
4 5 6
Passing the vector to add_val() results in the original vector remaining unchanged, since it is passed by value. Passing the vector to add_ref() however, causes the values inside the original vector to change, since it's passed by reference.
In Python everything is passed by reference. However, a lot of the builtin types (str, tuple, int, float, etc.) are immutable. This means that any operation you perform on these types results in a new variable being bound in the current scope with the new value. For mutable types (list, dict, etc.), you end up with exactly the same result as passing a parameter by reference in C++.