I often see python code that takes default arguments and has special behaviour when they are not specified.
If I for example want behavior like this:
def getwrap(dict, key, default = ??):
if ???: # default is specified
return dict.get(key, default)
else:
return dict[key]
If I were to roll my own, I'd end up with something like:
class Ham:
__secret = object()
def Cheese(self, key, default = __secret):
if default is self.__secret:
return self.dict.get(key, default)
else:
return self.dict[key]
But I don't want to invent something silly when there certainly is a standard. What is the idiomatic way of doing this in Python?
I usually prefer
def getwrap(my_dict, my_key, default=None):
if default is None:
return my_dict[my_key]
else:
return my_dict.get(my_key, default)
but of course this assumes that None is never a valid default value.
You could do it based on *args and/or **kwargs.
Here's an alternate implementation of getwrap based on *args:
def getwrap(my_dict, my_key, *args):
if args:
return my_dict.get(my_key, args[0])
else:
return my_dict[my_key]
And here it is in action:
>>> a = {'foo': 1}
>>> getwrap(a, 'foo')
1
>>> getwrap(a, 'bar')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in getwrap
KeyError: 'bar'
>>> getwrap(a, 'bar', 'Nobody expects the Spanish Inquisition!')
'Nobody expects the Spanish Inquisition!'
Related
Take a look at the code below.
I want to use the value of a as the default value for argument c in the declaration of the classmethod my_method().
How can I do it? The example below fails.
>>> class X:
... a = 'hello'
... def __init__(self, b):
... self.b = b
... #classmethod
... def my_method(cls, c=X.a):
... print 'cls.a = {}'.format(cls.a)
... print 'c = {}'.format(c)
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in X
NameError: name 'X' is not defined
It really depends what you want, but one way to get what you seem to want is to do it like this.
class X:
a = 'hello'
def __init__(self, b):
self.b = b
#classmethod
def my_method(cls, c=None):
if c is None:
c = X.a
print('cls.a = {}'.format(cls.a))
print('c = {}'.format(c))
X.my_method()
Make the default None and check for None in the function body, assigning to c if it is found to be None. If you ever wanted None to be a valid argument for the function then it wouldn't work, but that doesn't seem likely.
This idiom is often used in python to avoid the pitfalls mutable default argument, but here it delays the assignment from X.a until after X has actually been defined.
It also means that if you change X.a then my_method will pick up the new value. You may or may not want that.
I would like to know if the following introspection is possible in cpython:
>>> def potato(x=69):
... if x == 69 and ???:
... print '69 was taken from argument defaults'
... if x == 69 and ???:
... print '69 was passed as positional arg'
... if x == 69 and ???:
... print '69 was passed as kwarg'
...
>>> potato()
69 was taken from argument defaults
>>> potato(69)
69 was passed as positional arg
>>> potato(x=69)
69 was passed as kwarg
I'm interested in both python2 and python3 answers, if they are different.
Any kind of blackmagic involving inspect, traceback, pdb, sys._getframe etc is permissable here. Of course, modifying the argspec of the function is not allowed.
It doesn't look like inspect can provide this information directly although frames have a string called code_context which gives you the source line at which the function was called. The problem is that one would have to rewrite a small parser to make sense of it.
Here is a simpler solution based on a wrapper around the function you want to examine. It doesn't change the arg spec and the arg validation isn't changed either :
import inspect
def track_args(func):
def tracker(*args, **kwargs):
r = func(*args, **kwargs)
for arg_num,arg_name in enumerate(inspect.getargspec(func)[0]):
if arg_name in kwargs:
print "%s was provided as keyword arg" % arg_name
elif arg_num < len(args):
print "%s was provided as positional arg" % arg_name
else:
print "%s was provided by default value" % arg_name
return r
return tracker
#track_args
def f(a, b, c=30):
print "I'm f()"
f(20, b=10)
f(20)
Result with valid arguments:
I'm f()
a was provided as positional arg
b was provided as keyword arg
c was provided by default value
Result with invalid arguments:
Traceback (most recent call last):
File "test.py", line 21, in <module>
f(20)
File "test.py", line 5, in tracker
r = func(*args, **kwargs)
TypeError: f() takes at least 2 arguments (1 given)
I am trying to identify a way to implement same concept as data annotation validation from .NET in Python. It would be something like the following:
class MyClass:
#Property
def Message(self):
return self._message
#Message.setter
#MaxValue(233)
def Message(self, value):
self._message = value
I have tried different approaches but with out success. I would like to get access to "value" argument in order to apply a specific validation on it.
Does This help you?
This way you can make python annotations with additional arguments
def MaxValue(maxValue):
def wrapFunction(function):
def replacedMaxValueFunction(self, value):
assert value <= maxValue
return function(self, value)
replacedMaxValueFunction.__name__ = function.__name__
return replacedMaxValueFunction
return wrapFunction
So now you can do this:
I do not know wether it is C# conform but hopefully it does the checking you desire.
>>> #MaxValue(123)
def f(self, value):
print(value)
>>> f(1, 123)
123
>>> f(1, 124)
Traceback (most recent call last):
File "<pyshell#9>", line 1, in <module>
f(1, 124)
File "<pyshell#1>", line 4, in replacedMaxValueFunction
assert value <= maxValue
AssertionError
I'm writing an api and was wondering what's the most pythonic way to do the following.
I'm writing a bunch of methods to do various web calls, the arguments mostly translate into post data keys and values.
The way I've been writing it so far is mostly like this;
def doSomething(self,param1,param2,param3):
payload={"param1":param1,
"param2":param2,
"param3":param3}
return self.request("do/something",payload)
This already has the draw back of having to repeat the parameter names which are subject to change, but this pattern isn't too bad.
The following case is what got me trying to think of a better way. In this case there are 4 optional arguments for the call
def doSomethingElse(self,param1,param2=None,param3=None,param4=None,param5=None):
payload= {"param1":param1}
if param2:
payload["param2"]= param2
if param3:
payload["param3"]= param3
# ... etc ...
self.request("do/something/else",payload)
My first thought was to do this:
def doSomethingElse(self,param1,**params):
payload = {"param1":param1}
payload.update(params)
self.request("do/something/else",payload)
or even:
def doSomethingElse(self,**payload):
self.request("do/something/else",payload)
Although the second one is nice and simple, the method can be called without the non-default argument. But in both cases I lose the method signature when using the api and the user won't know what the parameters are (I know I could write the expected signature in a docstring but I'd rather prevent misspelt keywords getting sent).
I'm thinking there must be a nice pythonic solution to do this, any ideas?
EDIT
I think a key point which I didn't make clear enough is that the arguments are getting sent in post data in a call, and I want to make sure only those keys can get sent, like in the first example of doSomethingElse, you can't send anything other than those 5 named parameters.
The Pythonic way is to name the parameters when you call the function, not in the function signature:
def doSomething(self, **kwargs):
self.request("do/something/else", kwargs)
doSomething(param1=3, param2='one', param3=4)
How about simply
def get_payload(ldict):
return {k:v for k,v in ldict.iteritems() if k != 'self' and v is not None}
class fred(object):
some_class_var = 17
def method(self, a, b=2):
payload = get_payload(locals())
print payload
which gives
>>> f = fred()
>>> f.method()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method() takes at least 2 arguments (1 given)
>>> f.method(2)
{'a': 2, 'b': 2}
>>> f.method(2, b=3)
{'a': 2, 'b': 3}
>>> f.method(5, b=None)
{'a': 5}
>>> f.method(2, b=3, c=19)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method() got an unexpected keyword argument 'c'
>>> help(f.method)
Help on method method in module __main__:
method(self, a, b=2) method of __main__.fred instance
which I think matches your criteria. The next step would be to use a decorator (probably with either wraps or the decorator module to preserve the signature) so that payload was computed and then passed, but I don't know if #payload would be all that much better than payload = get_payload(locals()). Note that using locals() this way, it needs to be done at the start.
I second the feeling that this isn't exactly the best way to prevent unwanted nuclear attacks, though.
Something like this, perhaps:
def doSomethingElse(self, param1, **params):
payload = {"param1": param1}
for name, value in params.items():
if value is not None:
payload[name] = value
self.request("do/something/else", payload)
If you have several such functions, you can do as following:
class Requester(object):
def __init__(self, tobecalled, *allowed):
self.tobecalled = tobecalled
self.allowed = set(allowed)
def __call__(self, otherobj, **k):
for kw in k.iterkeys():
if kw not in self.allowed:
raise ValueError("unknown argument(s) given: %s" % kw)
otherobj.request(self.tobecalled, **k)
def __get__(self, outside, outsideclass):
return lambda **k: self(outside, **k)
class Outside(object):
def request(self, method, **k):
print method, k
do_one_thing = Requester("do/one/thing", 'param1', 'param2')
do_nonsense = Requester("do/nonsense", 'param3')
simple = Requester("simple")
o = Outside()
o.do_one_thing(param1=1, param2=2)
o.do_nonsense(param3=12)
o.simple()
try: o.do_one_thing(rparam1=1, param2=2)
except ValueError, e: print e
try: o.do_nonsense(gparam3=12)
except ValueError, e: print e
try: o.simple(whatever=12)
except ValueError, e: print e
What happens here? We create a Requester object which plays the role of a method: if we put it in another class (here: Outside), it can be called in a way that it also gets a reference of an object which it is called on. What I call outside here is "the outer self", as I call it now. And then, it returns a lambda which calls the object itself, just like a function does. And there, the arguments are checked for validity, and if that passes, we do the call on the "outside"'s request() method.
I'm dreaming of a Python method with explicit keyword args:
def func(a=None, b=None, c=None):
for arg, val in magic_arg_dict.items(): # Where do I get the magic?
print '%s: %s' % (arg, val)
I want to get a dictionary of only those arguments the caller actually passed into the method, just like **kwargs, but I don't want the caller to be able to pass any old random args, unlike **kwargs.
>>> func(b=2)
b: 2
>>> func(a=3, c=5)
a: 3
c: 5
So: is there such an incantation? In my case, I happen to be able to compare each argument against its default to find the ones that are different, but this is kind of inelegant and gets tedious when you have nine arguments. For bonus points, provide an incantation that can tell me even when the caller passes in a keyword argument assigned its default value:
>>> func(a=None)
a: None
Tricksy!
Edit: The (lexical) function signature has to remain intact. It's part of a public API, and the primary worth of the explicit keyword args lies in their documentary value. Just to make things interesting. :)
I was inspired by lost-theory's decorator goodness, and after playing about with it for a bit came up with this:
def actual_kwargs():
"""
Decorator that provides the wrapped function with an attribute 'actual_kwargs'
containing just those keyword arguments actually passed in to the function.
"""
def decorator(function):
def inner(*args, **kwargs):
inner.actual_kwargs = kwargs
return function(*args, **kwargs)
return inner
return decorator
if __name__ == "__main__":
#actual_kwargs()
def func(msg, a=None, b=False, c='', d=0):
print msg
for arg, val in sorted(func.actual_kwargs.iteritems()):
print ' %s: %s' % (arg, val)
func("I'm only passing a", a='a')
func("Here's b and c", b=True, c='c')
func("All defaults", a=None, b=False, c='', d=0)
func("Nothin'")
try:
func("Invalid kwarg", e="bogon")
except TypeError, err:
print 'Invalid kwarg\n %s' % err
Which prints this:
I'm only passing a
a: a
Here's b and c
b: True
c: c
All defaults
a: None
b: False
c:
d: 0
Nothin'
Invalid kwarg
func() got an unexpected keyword argument 'e'
I'm happy with this. A more flexible approach is to pass the name of the attribute you want to use to the decorator, instead of hard-coding it to 'actual_kwargs', but this is the simplest approach that illustrates the solution.
Mmm, Python is tasty.
Here is the easiest and simplest way:
def func(a=None, b=None, c=None):
args = locals().copy()
print args
func(2, "egg")
This give the output: {'a': 2, 'c': None, 'b': 'egg'}.
The reason args should be a copy of the locals dictionary is that dictionaries are mutable, so if you created any local variables in this function args would contain all of the local variables and their values, not just the arguments.
More documentation on the built-in locals function here.
One possibility:
def f(**kw):
acceptable_names = set('a', 'b', 'c')
if not (set(kw) <= acceptable_names):
raise WhateverYouWantException(whatever)
...proceed...
IOW, it's very easy to check that the passed-in names are within the acceptable set and otherwise raise whatever you'd want Python to raise (TypeError, I guess;-). Pretty easy to turn into a decorator, btw.
Another possibility:
_sentinel = object():
def f(a=_sentinel, b=_sentinel, c=_sentinel):
...proceed with checks `is _sentinel`...
by making a unique object _sentinel you remove the risk that the caller might be accidentally passing None (or other non-unique default values the caller could possibly pass). This is all object() is good for, btw: an extremely-lightweight, unique sentinel that cannot possibly be accidentally confused with any other object (when you check with the is operator).
Either solution is preferable for slightly different problems.
How about using a decorator to validate the incoming kwargs?
def validate_kwargs(*keys):
def entangle(f):
def inner(*args, **kwargs):
for key in kwargs:
if not key in keys:
raise ValueError("Received bad kwarg: '%s', expected: %s" % (key, keys))
return f(*args, **kwargs)
return inner
return entangle
###
#validate_kwargs('a', 'b', 'c')
def func(**kwargs):
for arg,val in kwargs.items():
print arg, "->", val
func(b=2)
print '----'
func(a=3, c=5)
print '----'
func(d='not gonna work')
Gives this output:
b -> 2
----
a -> 3
c -> 5
----
Traceback (most recent call last):
File "kwargs.py", line 20, in <module>
func(d='not gonna work')
File "kwargs.py", line 6, in inner
raise ValueError("Received bad kwarg: '%s', expected: %s" % (key, keys))
ValueError: Received bad kwarg: 'd', expected: ('a', 'b', 'c')
This is easiest accomplished with a single instance of a sentry object:
# Top of module, does not need to be exposed in __all__
missing = {}
# Function prototype
def myFunc(a = missing, b = missing, c = missing):
if a is not missing:
# User specified argument a
if b is missing:
# User did not specify argument b
The nice thing about this approach is that, since we're using the "is" operator, the caller can pass an empty dict as the argument value, and we'll still pick up that they did not mean to pass it. We also avoid nasty decorators this way, and keep our code a little cleaner.
There's probably better ways to do this, but here's my take:
def CompareArgs(argdict, **kwargs):
if not set(argdict.keys()) <= set(kwargs.keys()):
# not <= may seem weird, but comparing sets sometimes gives weird results.
# set1 <= set2 means that all items in set 1 are present in set 2
raise ValueError("invalid args")
def foo(**kwargs):
# we declare foo's "standard" args to be a, b, c
CompareArgs(kwargs, a=None, b=None, c=None)
print "Inside foo"
if __name__ == "__main__":
foo(a=1)
foo(a=1, b=3)
foo(a=1, b=3, c=5)
foo(c=10)
foo(bar=6)
and its output:
Inside foo
Inside foo
Inside foo
Inside foo
Traceback (most recent call last):
File "a.py", line 18, in
foo(bar=6)
File "a.py", line 9, in foo
CompareArgs(kwargs, a=None, b=None, c=None)
File "a.py", line 5, in CompareArgs
raise ValueError("invalid args")
ValueError: invalid args
This could probably be converted to a decorator, but my decorators need work. Left as an exercise to the reader :P
Perhaps raise an error if they pass any *args?
def func(*args, **kwargs):
if args:
raise TypeError("no positional args allowed")
arg1 = kwargs.pop("arg1", "default")
if kwargs:
raise TypeError("unknown args " + str(kwargs.keys()))
It'd be simple to factor it into taking a list of varnames or a generic parsing function to use. It wouldn't be too hard to make this into a decorator (python 3.1), too:
def OnlyKwargs(func):
allowed = func.__code__.co_varnames
def wrap(*args, **kwargs):
assert not args
# or whatever logic you need wrt required args
assert sorted(allowed) == sorted(kwargs)
return func(**kwargs)
Note: i'm not sure how well this would work around already wrapped functions or functions that have *args or **kwargs already.
Magic is not the answer:
def funky(a=None, b=None, c=None):
for name, value in [('a', a), ('b', b), ('c', c)]:
print name, value