I'm using ctypes to work with a library written in C. This C library allows me to register a callback function, which I'm implementing in Python.
Here is the callback function type, according to the ctypes API:
_command_callback = CFUNCTYPE(
UNCHECKED(c_int),
POINTER(vedis_context),
c_int,
POINTER(POINTER(vedis_value)))
Here is a decorator I've written to mark a function as a callback:
def wrap_callback(fn):
return _command_callback(fn)
To use this, I am able to simply write:
#wrap_callback
def my_callback(*args):
print args
return 1 # Needed by C library to indicate OK response.
c_library_func.register_callback(my_callback)
I can now invoke my callback (my_callback) from C and this works perfectly well.
The problem I'm encountering is that there will be some boilerplate behavior I would like to perform as part of these callbacks (such as returning a success flag, etc). To minimize boilerplate, I tried to write a decorator:
def wrap_callback(fn):
def inner(*args, **kwargs):
return fn(*args, **kwargs)
return _command_callback(inner)
Note that this is functionally equivalent to the previous example.
#wrap_callback
def my_callback(*args):
print args
return 1
When I attempt to invoke the callback using this approach, however, I receive the following exception, originating from _ctypes/callbacks.c:
Traceback (most recent call last):
File "_ctypes/callbacks.c", line 314, in 'calling callback function'
File "/home/charles/tmp/scrap/z1/src/vedis/vedis/core.py", line 28, in inner
return fn(*args, **kwargs)
SystemError: Objects/cellobject.c:24: bad argument to internal function
I am not sure what is going on here that would cause the first example to work but the second example to fail. Can anyone shed some light on this? Bonus points if you can help me find a way to decorate these callbacks so I can reduce boilerplate code!
Thanks to eryksyn, I was able to fix this issue. The fix looks like:
def wrap_callback(fn):
def inner(*args, **kwargs):
return fn(*args, **kwargs)
return _command_callback(inner), inner
def my_callback(*args):
print args
return 1
ctypes_cb, my_callback = wrap_callback(my_callback)
Related
I am trying to understand the examples for replacing the use of try-finally and flag variables in Python's documentation
According to the documentation instead of:
cleanup_needed = True
try:
result = perform_operation()
if result:
cleanup_needed = False
finally:
if cleanup_needed:
cleanup_resources()
we could use a small ExitStack-based helper class Callback like this (I added the perform_operation and cleanup_resources function):
from contextlib import ExitStack
class Callback(ExitStack):
def __init__(self, callback, /, *args, **kwds):
super(Callback, self).__init__()
self.callback(callback, *args, **kwds)
def cancel(self):
self.pop_all()
def perform_operation():
return False
def cleanup_resources():
print("Cleaning up resources")
with Callback(cleanup_resources) as cb:
result = perform_operation()
if result:
cb.cancel()
I think the code simulates the exceptional case, where the perform_operation() did not run smoothly and a cleanup is needed (perform_operation() returned False). The Callback class magically takes care of the running the cleanup_resources() function (I can't quite understand why, by the way).
Then I simulated the normal case, where everything runs smoothly and no cleanup is needed, I changed the code to make perform_operation() return True instead. In this case, however, the cleanup_resources function also runs and the code errors out:
$ python minimal.py
Cleaning up resources
Traceback (most recent call last):
File "minimal.py", line 26, in <module>
cb.cancel()
File "minimal.py", line 12, in cancel
self.pop_all()
File "C:\ProgramData\Anaconda3\envs\claw\lib\contextlib.py", line 390, in pop_all
new_stack = type(self)()
TypeError: __init__() missing 1 required positional argument: 'callback'
Can you explain what exactly is going on here and how this whole ExitStack and callack stuff works?
Code
def zappa_async(func):
print('here')
#wraps(func)
#task(capture_response=True)
def func_wrap_async(*args, **kwargs):
return func(*args, **kwargs)
def func_wrap_async_response_id(*args, **kwargs):
return func_wrap_async(*args, **kwargs).response_id
return func_wrap_async_response_id
Expected Behavior
Take a function and return a new function that is asynchronous and returns its response id
Actual Behavior
lambda throws module 'rap_stats.MapReduce' has no attribute 'func_wrap_async': AttributeError
Update
It works when I remove '#task' and '.response_id' but I need these in order for it to properly function asynchronously
The README says:
To capture responses, you must configure a async_response_table in zappa_settings.
This needs to be done.
Here is my code:
name = "Arthur"
message = "Hello!"
def decorate(func_to_decorate):
def wrap(*args, **kwargs):
print ("........................")
func_to_decorate(*args, **kwargs)
print ("........................")
return wrap
#decorate
def send_message(your_name="unassigned", your_message="blank"):
print(your_name)
print(your_message)
send_message(name, message)
My error is in line 20:
send_message(name, message)
TypeError: 'NoneType' object is not callable
My understanding is that the wrapper is "replacing" itself with the function immediately following the decorator. This seems work when I am not passing arguments to the function being decorated, but not with the decorator present.
There are two things wrong with your decorator.
First, there's an indentation mistake:
def decorate(func_to_decorate):
def wrap(*args, **kwargs):
print ("........................")
func_to_decorate(*args, **kwargs)
print ("........................")
return wrap
That return wrap is part of the wrap function body, not part of the decorate function body. So, decorate has no return statement, which means it returns None. Hence the error you see: the decorator is in fact "replacing" the wrapped function with the wrapper it returns—but that wrapper is None, so you end up trying to call None as a function.
And meanwhile, you seem to understand that wrap should return something, but that something definitely shouldn't be itself. Usually, what you want to return is the result of the wrapped function (or some post-processed version of that result). In your test, you're only trying to wrap up a function that's used only for side-effects, not to mention that you never get to call the wrapper because of your first problem, so you wouldn't notice this problem yet, but you still want to fix it.
So:
def decorate(func_to_decorate):
def wrap(*args, **kwargs):
print ("........................")
retval = func_to_decorate(*args, **kwargs)
print ("........................")
return retval
return wrap
I'm trying to invoke a python function from robotframework keyword. The python function has been decorated to be invoked using run_keyword from Builtin library. This is because robot logs appear well structured if library functions are invoked via run_keyword function from built in library. rather than invoked directly. However this is resulting in an infinite loop. Is there a solution to gracefully accomplish the goal?
robotkeyword :
do something
#creates a user by calling a function from python based library
create user
python function
#wrap_with_run_keyword
def create_user():
pass
def wrap_with_run_keyword(func):
def func_wrapper(*args, **kwargs):
return run_keyword(func, *args, **kwargs)
return func_wrapper
I couldn't solve the problem using partial application.
However, I broker the recursive loop by setting and unsetting an attribute as give below.
def wrap_with_run_keyword(func):
def func_wrapper(*args, **kwargs):
if not hasattr(func, 'second'):
setattr(func, "second", True)
return run_keyword(func, *args, **kwargs)
else:
delattr(func, "second")
return func(*args, **kwargs)
return func_wrapper
I have however run into another problem. I defined create_user as follows
def create_user(properties):
#some code
pass
On Calling this function in the way below
create_user("name=abc")
I'm getting the following error : got an unexpected keyword argument 'name'
I did run in the same issue, but solved it, only wondering if i can detect the caller...if the call is done from robotframework or by python in case that the call is done by the rf it should do only the second call
#wraps(function)
def wrapper(self, *args, **kwargs):
if not hasattr(function, 'second'):
setattr(function, 'second', True)
ar= list(args)
for key, value in kwargs.items():
ar.append(value)
return BuiltIn().run_keyword('Mylib.' + function.__name__, ar)
else:
delattr(function, 'second')
return function(self,*args[0])
return wrapper
Take a look at the partial class from the functools module. I think this might help you.
Or take a look at how decorators work in python.
I currently have the following code which uses a python library:
f = Foo(original_method, parameters)
I would like to augment original_method, and have a decorator add a few lines of code. Let's call the new decorated method decorated_method. Finally I would like to have something like this:
f = Foo(decorated_method(original_method), parameters)
My questions are: is this possible? how would the decorator look like?
I must say that I can't extend original_method, since it is part of an external library.
Edit: original_method is not executed, is only passed to Foo as a parameter. decorated_method function should do some logging and gather some statistics of the number of calls.
Later edit: the code in examples below works fine. I had a few additional problems because original_method had a few attributes, so this is the final code:
def decorated_method(method):
def _reporter(*args, **kwargs):
addmetric('apicall', method.__name__)
return method(*args, **kwargs)
_reporter.original_method_attribute = method.original_method_attribute
return _reporter
You don't mention what you want decorated_method to do, but this is certainly possible:
def decorated_method(f):
def _wrapped(*args, **kwargs):
print "About to call f!"
ret = f(*args, **kwargs)
print "Just got finished with f, ret = %r" % (ret,)
return ret
return _wrapped
This is just standard decorator structure: A decorator is a function which accepts a function and returns a function.
Absolutely:
def decorated_method(fn):
def inner_method(*args, **kwargs):
print("before calling")
result = fn(*args, **kwargs)
print("after calling")
return result
return inner_method
Once you've got this working, you should look at signature-preserving decorators.