Cite from docs:
...look at the **kwargs argument. All signals send keyword arguments,
and may change those keyword arguments at any time. In the case of
request_finished, it’s documented as sending no arguments, which means
we might be tempted to write our signal handling as
my_callback(sender).
This would be wrong – in fact, Django will throw an error if you do
so. That’s because at any point arguments could get added to the
signal and your receiver must be able to handle those new arguments.
I don't get it. Why 'arguments could be added at any time', isn't interfaces in programs exist to be constant and everybody should be aware of them? Or does these words mean, that every receiver must always fail silently? Because it is obvious if, sender would randomly change interface, receivers will fail and throw errors.
This would be wrong – in fact, Django will throw an error if you do
so.
Throwing errors is wrong when using signals or what did they meant?
Seems like that is just telling you to be sure you always include the **kwargs argument. So you should do that.
In Python and especially in Django, it is common to program API's (or more precisely, the functions that expose the API's) in a way that when given additional parameters, they can still operate instead of crashing.
In your particular situation- consider a signal handler like:
def (sender, param1, param2):
pass
Lets say you have some version X.Y of Django where this handler works perfectly. Then you realise that Django has been updated to X.Z and one thing in the changelog was that signals now get given a fourth keyword arg (param3).
Would you rather go through you entire codebase and change handlers to:
def handler(sender, param1, param2, param3):
pass
...or would it have been better to program all handlers like:
def handler(sender, param1, param2, **kwargs):
pass
?
This kind of design is also useful when your functions are supposed to relay the params to other functions:
def func(*args,**kwargs):
# do something
other_func(*args, **kwargs)
There is a caveat though: this kind of API design is reasonably when acting on params is voluntary. Consider a (naive) example like the following "sum" API.
def sum(a,b):
return a+b
Then in the next version the sum function suddenly starts to get more params: a,b,c. A function like the following would probably cause hard to track bugs:
def sum(a,b,**kwargs):
return a+b
Throwing errors is wrong when using signals or what did they meant?
They only mean that the **kwargs is required when you call signal receiver function. Throwing errors when using signals is not wrong a priori. In conclusion, remind to always include the kwargs when you need to define a receiver function, as the docs says:
def my_callback(sender, **kwargs):
print("Request finished!")
Related
I am using Python RPyC to communicate between two machines. Since the link may be prone to errors I would like to have a generic wrapper function which takes a remote function name plus that function's parameters as its input, does some status checking, calls the function with the parameters, does a little more status checking and then returns the result of the function call. The wrapper should have no knowledge of the function, its parameters/parameter types or the number of them, or the return value for that matter, the user has to get that right; it should just pass them transparently through.
I get the getattr(conn.root, function)() pattern to call the function but my Python expertise runs out at populating the parameters. I have read various posts on the use of *arg and **kwarg, in particular this one, which suggests that it is either difficult or impossible to do what I want to do. Is that correct and, if so, might there be a scheme which would work if I, say, ensured that all the function parameters were keyword parameters?
I do own both ends of this interface (the caller and the called) so I could arrange to dictionary-ise all the function parameters but I'd rather not make my API too peculiar if I could possibly avoid it.
Edit: the thing being called, at the remote end of the link, is a class with very ordinary methods, e.g.;
def exposed_a(self)
def exposed_b(self, thing1)
def exposed_c(self, thing1=None)
def exposed_d(self, thing1=DEFAULT_VALUE1, thing2=None)
def exposed_e(self, thing1, thing2, thing3=DEFAULT_VALUE1, thing4=None)
def exposed_f(self, thing1=None, thing2=None)
...where the types of each argument (and the return values) could be string, dict, number or list.
And it is indeed, trivial, my Goggle fu had simply failed me in finding the answer. In the hope of helping anyone else who is inexperienced in Python and is having a Google bad day:
One simply takes *arg and **kwarg as parameters and passes them directly on, with the asterisks attached. So in my case, to do my RPyC pass-through, where conn is the RPyC connection:
def my_passthru(conn, function_name, *function_args, **function_kwargs):
# Do a check of something or other here
return_value = getattr(conn.root, function_name)(*function_args, **function_kwargs)
# Do another check here
return return_value
Then, for example, a call to my exposed_e() method above might be:
return_value = my_passthru(conn, e, thing1, thing2, thing3)
(the exposed_ prefix being added automagically by RPyC in this case).
And of course one could put a try: / except ConnectionRefusedError: around the getattr() call in my_passthru() to generically catch the case where the connection has dropped underneath RPyC, which was my main purpose.
Given a simple Python function with an optional argument, like:
def wait(seconds=3):
time.sleep(seconds)
How do I create a function that calls this and passes on an optional argument? For example, this does NOT work:
def do_and_wait(message, seconds=None):
print(message)
wait(seconds)
Note: I want to be able to call wait from other functions with the optional seconds argument without having to know and copy the current default seconds value in the underlying wait function to every other function which calls it.
As above, if I call it with the optional argument, like do_and_wait(2) then it works, but trying to rely on wait's default, e.g. calling it like do_and_wait() causes a TypeError because inside wait seconds == None.
Is there a simple and clean way to make this work? I know I can abuse kwargs like this:
def do_and_wait(message, **kwargs):
print(message)
wait(**kwargs)
But that seems unclear to the reader and user of this function since there is no useful name on the argument.
Note: This is a stupidly simplified example.
I understand you've simplified your question, but I think you mean how one can call a function with optional arguments that could be None. Does the following work for you?
import time
def func1(mess, sec):
if sec != None:
time.sleep(sec)
print(mess)
func1('success', sec=None)
I don't think you've quite explained your problem completely because I don't expect an answer should be this simple, but I would just use the same default value (and data type) in do_and_wait() as wait() uses, like so:
def do_and_wait(message, seconds=3):
print(message)
wait(seconds)
After thinking a bit more, I came up with something like this; Han's answer suggested this and reminded me that I think PEP even suggests it somewhere. This especially avoids having to know and copy the default value into any function that calls wait and wants to support a variable value for seconds.
def wait(seconds=None):
time.sleep(seconds if seconds is not None else 3)
def do_and_wait(message, seconds=None):
print(message)
wait(seconds)
def something_else(callable, seconds=None):
callable()
wait(seconds)
I want to realize some sort oft client-server-connection using Python and are rather new to multiprocessing. Basically, I have a class 'Manager' that inherits from multiprocessing.Process and manages the connection from a client to different data sources. This process has some functions like 'get_value(key)' that should return the value of the key-data source. Now, as I want this to run asynchronized, I cannot simply call this function from my client process.
My idea so far would be that I connect the Client- and Manager-Processes using a Pipe and then send a message from the Client to the Manager to execute this function. I would realize this by sending a list through the pipe where the first element is the name of the function the remaining elements are the arguments of the actual function, e.g. ['get_value', 'datasource1']. The process then would receive this and send the return value through the pipe to the client. This would look something like this:
from multiprocessing import Process, Pipe
import time
class Manager(Process):
def __init__(self, connection):
super(Process, self).__init__()
self.connection = connection
def run(self):
while True:
if self.connection.poll():
msg = self.connection.recv()
self.call_function(msg[0], msg[:])
def call_function(self, name, *args):
print('Function Called with %s' % name)
return_val = getattr(self, name)(*args)
self.connection.send(return_val)
def get_value(self, key):
return 1.0
While I guess that this would work, I am not very happy with this solution. Especially the call-function-by-string-method seems very error-prone. Is there a more elegant way of requesting to execute a function in Python?
I think that your approach, all in all, is a good one (there are other ways to do the same thing, of course, but there is nothing wrong with your general approach).
That said, I would change the design slightly to add a "routing" component: think of some logic that somehow limits what "commands" can be sent by clients, and hooks between commands and "handlers" - that is functions that handle them. Basically think Web Framework routing (if you are familiar with the concept).
This is a good idea both in terms of flexibility of the design, in terms of error detection and in terms of security (you don't want clients to call ['__del__'] for example on your Manager.
At it's very basic form, a router can be a dictionary mapping commands to class methods:
class Manager(Process):
def __init__(self, connection):
super(Process, self).__init__()
self.connection = connection
self.routes = {'do_action': self._do_action,
'do_other_action': some_callable,
'ping': lambda args: args} # <- as long as it's callable and has the right signature...
def call_function(self, name, *args):
try:
handler = self.routes[name]
except KeyError:
return self._error_reply('{} is not a valid command'.format(name))
try:
return_val = handler(*args) # handler functions will need to throw something if arguments are wrong...
except ValueError as e:
return self._error_reply('Invalid command arguments: {}'.format(str(e)))
except Exception as e:
# This is your catch-all "internal server error" handler
return self._error_reply(str(e))
self.connection.send(return_val)
This is of course just an example of an approach. You will need to implement _error_reply() in whatever way works for you.
You can expand on it by creating a Router class and passing it as a dependency to Manager, making it even more flexible. You might also want to think about making your Manager a separate thing and not a subclass of Process (because you might want to run it regardless of whether it is in a subprocess - for example in testing).
BTW, there are frameworks for implementing such things with various degrees of complexity and flexibility (Thrift, ZeroMQ, ...), but if you want to do something simple and learn, doing it yourself is in my opinion a great choice.
I have a setting where event handlers are always functions taking a single event argument.
But more often than not, I find myself writing handlers that doesn´t use any of the event information. So I constantly write handlers of the form:
def handler(_):
#react
In order to discard the argument.
But I wish I didn´t have to, as sometimes I want to reuse handlers as general action functions that take no arguments, and sometimes I have existing zero-arguments functions that I want to use as handlers.
My current solution is wrapping the function using a lambda:
def handler():
#react
event.addHandler(lambda _:handler())
But that seems wrong for other reasons.
My intuitive understanding of a lambda is that it is first and foremost a description of a return value, and event handlers return nothing. I feel lambdas are meant to express pure functions, and here I´m using them only to cause side effects.
Another solution would be a general decorator discarding all arguments.
def discardArgs(func):
def f(*args):
return func()
return f
But I need this in a great many places, and it seems silly having to import such a utility to every script for something so simple.
Is there a particularly standard or "pythonic" way of wrapping a function to discard all arguments?
Use *args:
def handler(*args):
#react
Then handler can take 0 arguments, or any number of position arguments.
I'm currently writing an api library in python and I'm wondering if the following code is too unpythonic:
#accepts('video_id', 'reference_id', 'page_size', 'page_number',
'get_item_count', 'fields', 'video_fields', 'custom_fields',
'media_delivery', 'output')
#requires('video_id', 'reference_id')
def find_related_videos(self, **params):
return ItemCollection(read_request(params))
The accepts decorator throws an error if any kwargs not in the the list are passed to the method. It also does validation for certain keywords.
The requires decorator ensures that those keyword arguments are present.
Not having the keyword args in the method definition is bugging me. However, having to build the params dictionary manually for each method also seems annoying. Also, there is validation code that is the same for every instance of the video_fields argument passed to a method, so I can call that from the accepts decorator to avoid repeating myself.
Thoughts?
I'd definitely put the required fields in the method signature:
def find_related_videos(self, video_id, reference_id, **params):
params.update({'video_id': video_id, 'reference_id': reference_id})
return ItemCollection(read_request(params))
If possible, I'd even modify read_request to take keyword arguments as well:
def find_related_videos(self, video_id, reference_id, **params):
return ItemCollection(read_request(video_id=video_id, reference_id=reference_id, **params))
As far as acceptable params go, personally I wouldn't throw errors for invalid params -- I'd just ignore the ones I didn't need.
How about this solution?
def check_params(params):
# Do whatever check you want to do on params
return dict((k, v) for k, v in params
if v is not None and k != "self")
def find_related_videos(self, video_id, reference_id, page_size=None,
page_number=None, get_item_count=None, fields=None,
video_fields=None, custom_fields=None,
media_delivery=None, output=None):
params = check_prams(locals())
return ItemCollection(read_request(params))
This will leave checking for acceptable and required parameters to Python, while facitlitating custom tests in check_params().
I would do it without the decorators. Much easier for readers to follow the codes executation path without decorators:
required = set(['video_id', 'reference_id'])
acceptable = required.union(set(['page_size', 'page_number', 'get_item_count', 'fields', 'video_fields', 'custom_fields', 'media_delivery', 'output']))
def find_related_videos(self, **params):
if not (required.issubset(set(params.keys())) or set(params.keys()).issubset(acceptable)):
raise Exception("Some exception")
return ItemCollection(read_request(params))
This will check to see if all kwargs' keys are in the acceptable set and has at least the required args.
If you were to do this:
def find_related_videos(self, video_id, reference_id, ...)
then you could still use a decorator to check the validity of the arguments, and leave the interpreter to enforce the rest for you. There are some trade-offs though.
Using positional arguments you no longer allow the caller to specify the arguments in any order. If you are basically reading in a dictionary and passing it, or something like that, this could be a problem.
It's harder to write your #accepts (or #validate) decorator to check the validity of the arguments in the same general way. It can still be done by using the inspect.getargspec function, and I've done something similar before, but it'll take you some time to get it working. My decorator to automatically match HTTP GET parameters to function arguments is complicated, but it works.