If I have a function in Python like this:
def multiply(a, b)
return a * b
How can I call PyObject_CallObject when it will give me an error if I have more than two arguments? There may be a much better way of calling a function from C++ but I am very new to the Python/C API
From the documentation:
PyObject* PyObject_CallObject(PyObject *callable_object, PyObject *args)
Return value: New reference.
Call a callable Python object callable_object, with arguments given
by the tuple args. If no arguments are needed, then args may be NULL.
Returns the result of the call on success, or NULL on failure. This is
the equivalent of the Python expression callable_object(*args).
In other words:
You can pass more than one argument to the function by passing a single tuple containing the arguments. So you'd have to build a tuple containing x and y (i.e. (x, y)) and then pass the tuple as single parameter to PyObject_CallObject(multiply, the_tuple) this will be equivalent to multiply(x, y).
It does not represent the most general call. The most generic call is PyObject_Call which takes two arguments: a tuple of positional arguments and a dictionary of keyword arguments.
There are also the PyObject_CallFunction* functions that are similar to PyObject_CallObject but they avoid having to create the tuple and allow multiple parameters.
Related
I wanted to know how to work with an array as a functional argument in Python. I will show a short example:
def polynom(x, coeff_arr):
return coeff_arr[0]+ coeff_arr[1]+x +coeff_arr[2]*x**2
I obviously get the error that 2 positional arguments are needed but 4 were given when I try to run it, can anybody tell me how to do this accept just using (coeff_arr[i]) in the argument of the function?
Cheers
Your question is missing the code you use to call the function, but from the error I infer that you are calling it as polynom(x, coefficient1, coefficient2, coefficient3). Instead you need to either pass the coefficients as a list:
polynom(x, [coefficient1, coefficient2, coefficient3])
Or use the unpacking operator * to define the function as follows, which will take all positional arguments after x and put them into coeff_arr as a list:
def polynom(x, *coeff_arr):
(The unpacking operator can also be used in a function call, which will do the opposite of taking a list and passing its elements as positional arguments:
polynom(x, *[coefficient1, coefficient2, coefficient3])
is equivalent to
polynom(x, coefficient1, coefficient2, coefficient3)
)
At the end of the built-in list function descriptions is "/". What is it?
>>> l=[]
>>> help(l.insert)
Help on built-in function insert:
insert(index, object, /) method of builtins.list instance
Insert object before index.
>>> help(l.index)
Help on built-in function index:
index(value, start=0, stop=9223372036854775807, /) method of builtins.list instance
Return first index of value.
There are three kinds of parameters in Python:
Positional-only, which can only be set by positional arguments
"Regular", which can be set either by positional arguments or keyword arguments
Keyword-only, which can only be set by keyword arguments.
When defining a function, a / is used to separate positional-only parameters (at least one) on the left and the rest on the right. If there is no / in the parameter list, there are no positional-only parameters.
The syntax originated in the argument clinic, which is used to define functions for the CPython implementation. Its use appeared in help for such functions before it was added to the syntax of Python itself in PEP-570.
From Python documentation about help() :
Note that if a slash(/) appears in the parameter list of a function, when invoking help(), it means that the parameters prior to the slash are positional-only.
and from the FAQ entry on positional-only parameters :
What does the slash(/) in the parameter list of a function mean?
A slash in the argument list of a function denotes that the parameters prior to it are positional-only. Positional-only parameters are the ones without an externally-usable name. Upon calling a function that accepts positional-only parameters, arguments are mapped to parameters based solely on their position. For example, divmod() is a function that accepts positional-only parameters. Its documentation looks like this:
>>> help(divmod)
Help on built-in function divmod in module builtins:
>
> divmod(x, y, /)
> Return the tuple (x//y, x%y). Invariant: div*y + mod == x.
The slash at the end of the parameter list means that both parameters are positional-only. Thus, calling divmod() with keyword arguments would lead to an error:
>>> divmod(x=3, y=4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: divmod() takes no keyword arguments
So I have some funciton
def fun(x,y=None):
# Do some stuff
and I want to minimize this function over x, but sometimes I will want to have an additional argument y. (Essentially, I want y to always take its default value, except when I'm minimizing.)
Usually, if the function was just def fun(x,y): I'd be able to do scipy.optimize.minimize(x,args=(y)). But what do I do when y is an optional variable? putting args=(y=value) is giving me invalid syntax, (and, sure, it looks very wrong), but I'm not sure what the correct syntax would be.
I'm using Python 3.7 if that's relevant.
You can't. minimize only takes optional arguments without keywords for the function it calls. This is written in the documentation, since it describes the function to be called as fun(x, *args), without any optional arguments.
If your function is f(x, y=None, z=None) you would have to call scipy.optimize.minimize(f, x, (y, z)).
Instead I would construct a lambda that passes the arguments as keywords directly. Thus doing:
scipy.optimize.minimize(lambda x: f(x, z=0), x)
which makes it clear which optional arguments you are passing.
I recently attempted Googles foo.bar challenge. After my time was up I decided to try find a solution to the problem I couldn't do and found a solution here (includes the problem statement if you're interested). I'd previously been making a dictionary for every function I wanted to cache but it looks like in this solution any function/input can be cached using the same syntax.
Firstly I'm confused on how the code is even working, the *args variable isn't inputted as an argument (and prints to nothing). Heres an modified minimal example to illustrate my confusion:
mem = {}
def memoize(key, func, *args):
"""
Helper to memoize the output of a function
"""
print(args)
if key not in mem:
# store the output of the function in memory
mem[key] = func(*args)
return mem[key]
def example(n):
return memoize(
n,
lambda: longrun(n),
)
def example2(n):
return memoize(
n,
longrun(n),
)
def longrun(n):
for i in range(10000):
for j in range(100000):
2**10
return n
Here I use the same memoize function but with a print. The function example returns memoize(n, a lambda function,). The function longrun is just an identity function with lots of useless computation so it's easy to see if the cache is working (example(2) will take ~5 seconds the first time and be almost instant after).
Here are my confusions:
Why is the third argument of memoize empty? When args is printed in memoize it prints (). Yet somehow mem[key] stores func(*args) as func(key)?
Why does this behavior only work when using the lambda function (example will cache but example2 won't)? I thought lambda: longrun(n) is just a short way of giving as input a function which returns longrun(n).
As a bonus, does anyone know how you could memoize functions using a decorator?
Also I couldn't think of a more descriptive title, edits welcome. Thanks.
The notation *args stands for a variable number of positional arguments. For example, print can be used as print(1), print(1, 2), print(1, 2, 3) and so on. Similarly, **kwargs stands for a variable number of keyword arguments.
Note that the names args and kwargs are just a convention - it's the * and ** symbols that make them variadic.
Anyways, memoize uses this to accept basically any input to func. If the result of func isn't cached, it's called with the arguments. In a function call, *args is basically the reverse of *args in a function definition. For example, the following are equivalent:
# provide *args explicitly
print(1, 2, 3)
# unpack iterable to *args
arguments = 1, 2, 3
print(*arguments)
If args is empty, then calling print(*args) is the same as calling print() - no arguments are passed to it.
Functions and lambda functions are the same in python. It's simply a different notation for creating a function object.
The problem is that in example2, you are not passing a function. You call a function, then pass on its result. Instead, you have to pass on the function and its argument separately.
def example2(n):
return memoize(
n,
longrun, # no () means no call, just the function object
# all following parameters are put into *args
n
)
Now, some implementation details: why is args empty and why is there a separate key?
The empty args comes from your definition of the lambda. Let's write that as a function for clarity:
def example3(n):
def nonlambda():
return longrun(n)
return memoize(n, nonlambda)
Note how nonlambda takes no arguments. The parameter n is bound from the containing scope as a closure, bound from the containing scope. As such, you don't have to pass it to memoize - it is already bound inside the nonlambda. Thus, args is empty in memoize, even though longrun does receive a parameter, because the two don't interact directly.
Now, why is it mem[key] = f(*args), not mem[key] = f(key)? That's actually slightly the wrong question; the right question is "why isn't it mem[f, args] = f(*args)?".
Memoization works because the same input to the same function leads to the same output. That is, f, args identifies your output. Ideally, your key would be f, args as that's the only relevant information.
The problem is you need a way to look up f and args inside mem. If you ever tried putting a list inside a dict, you know there are some types which don't work in mappings (or any other suitable lookup structure, for that matter). So if you define key = f, args, you cannot memoize functions taking mutable/unhashable types. Python's functools.lru_cache actually has this limitation.
Defining an explicit key is one way of solving this problem. It has the advantage that the caller can select an appropriate key, for example taking n without any modifications. This offers the best optimization potential. However, it breaks easily - using just n misses out the actual function called. Memoizing a second function with the same input would break your cache.
There are alternative approaches, each with pros and cons. Common is the explicit conversion of types: list to tuple, set to frozenset, and so on. This is slow, but the most precise. Another approach is to just call str or repr as in key = repr((f, args, sorted(kwargs.items()))), but it relies on every value having a proper repr.
I have the following code, which results in this error:
TypeError('smallTask() takes exactly 1 argument (2 given)',)
#task
def master():
count = 0
obj = { 'var1':'val1', 'var2':'val2' }
while count < 10:
subtask('smallTask',obj).apply_async()
count += 1
#task(name='smallTask')
def smallTask(obj):
print obj
Passing a dictionary to a function, I imagine I need to use **kwargs but if I do that, I get the error that the function takes no arguments yet 2 have been supplied.
I assume the issue here is with either the decorator (have a basic understanding of this but not enough to solve the problem) or the subtask function in Celery.
I don't have enough python knowledge to really proceed..could anyone give me an idea of what's happening and how I can pass the smallTask function a dictionary?
You need to pass arguments for a subtask in the args keyword argument, which must be a tuple according to the celery.subtask() documentation:
subtask('smallTask', args=(obj,)).apply_async()
or use the Task.subtask() method on your smallTask task, but again pass the arguments as a tuple:
smallTask.subtask((obj,)).apply_async()
Alternatively, use star arguments with the Task.s() method:
smallTask.s(obj).apply_async()
The subtasks documentation you yourself linked to use a tuple in the examples; arguments and keyword arguments are two pieces of data that Celery has to store for you until it can run that task, then it'll apply those arguments and keyword arguments for you.
But the celery.subtask() function takes more than just the arguments and keyword arguments for your task; it also takes additional options. In order to work with arbitrary arguments (positional or keyword) to your task, and support other arguments that are not passed to your task, the function signature has no choice but to accept positional arguments as an explicit tuple, and keyword arguments as an explicit dictionary.
The Task.s() method does not accept any arguments other than what the task itself would accept, so it does support passing arguments as if you called the task directly. Internally, this uses catch-all arguments: Task.s(*args, **kwarg), and just passes the captured arguments as a tuple and dictionary on to Task.subtask().