Related
Let's imagine I have a dict :
d = {'a': 3, 'b':4}
I want to create a function f that does the exact same thing than this function :
def f(x, a=d['a'], b=d['b']):
print(x, a, b)
(Not necessarily print, but do some stuff with the variable and calling directly from their name).
But I would like to create this function directly from the dict, that is to say, I would like to have something that look likes
def f(x, **d=d):
print(x, a, b)
and that behaves like the previously defined function. The idea is that I have a large dictionary that contains defaults values for arguments of my function, and I would like not to have to do
def f(a= d['a'], b = d['b'] ...)
I don't know if it's possible at all in python. Any insight is appreciated !
Edit : The idea is to be able to call f(5, a=3).
Edit2 : The question is not about passing arguments stored in a dict to a function but to define a function whose arguments names and defaults values are stored in a dict.
You cannot achieve this at function definition because Python determines the scope of a function statically. Although, it is possible to write a decorator to add in default keyword arguments.
from functools import wraps
def kwargs_decorator(dict_kwargs):
def wrapper(f):
#wraps(f)
def inner_wrapper(*args, **kwargs):
new_kwargs = {**dict_kwargs, **kwargs}
return f(*args, **new_kwargs)
return inner_wrapper
return wrapper
Usage
#kwargs_decorator({'bar': 1})
def foo(**kwargs):
print(kwargs['bar'])
foo() # prints 1
Or alternatively if you know the variable names but not their default values...
#kwargs_decorator({'bar': 1})
def foo(bar):
print(bar)
foo() # prints 1
Caveat
The above can be used, by example, to dynamically generate multiple functions with different default arguments. Although, if the parameters you want to pass are the same for every function, it would be simpler and more idiomatic to simply pass in a dict of parameters.
Python is designed such that the local variables of any function can be determined unambiguously by looking at the source code of the function. So your proposed syntax
def f(x, **d=d):
print(x, a, b)
is a nonstarter because there's nothing that indicates whether a and b are local to f or not; it depends on the runtime value of the dictionary, whose value could change across runs.
If you can resign yourself to explicitly listing the names of all of your parameters, you can automatically set their default values at runtime; this has already been well covered in other answers. Listing the parameter names is probably good documentation anyway.
If you really want to synthesize the whole parameter list at run time from the contents of d, you would have to build a string representation of the function definition and pass it to exec. This is how collections.namedtuple works, for example.
Variables in module and class scopes are looked up dynamically, so this is technically valid:
def f(x, **kwargs):
class C:
vars().update(kwargs) # don't do this, please
print(x, a, b)
But please don't do it except in an IOPCC entry.
try this:
# Store the default values in a dictionary
>>> defaults = {
... 'a': 1,
... 'b': 2,
... }
>>> def f(x, **kwa):
# Each time the function is called, merge the default values and the provided arguments
# For python >= 3.5:
args = {**defaults, **kwa}
# For python < 3.5:
# Each time the function is called, copy the default values
args = defaults.copy()
# Merge the provided arguments into the copied default values
args.update(kwa)
... print(args)
...
>>> f(1, f=2)
{'a': 1, 'b': 2, 'f': 2}
>>> f(1, f=2, b=8)
{'a': 1, 'b': 8, 'f': 2}
>>> f(5, a=3)
{'a': 3, 'b': 2}
Thanks Olvin Roght for pointing out how to nicely merge dictionaries in python >= 3.5
How about the **kwargs trick?
def function(arg0, **kwargs):
print("arg is", arg0, "a is", kwargs["a"], "b is", kwargs["b"])
d = {"a":1, "b":2}
function(0., **d)
outcome:
arg is 0.0 a is 1 b is 2
This question is very interesting, and it seemed different people have their different own guess about what the question really want.
I have my own too. Here is my code, which can express myself:
# python3 only
from collections import defaultdict
# only set once when function definition is executed
def kwdefault_decorator(default_dict):
def wrapper(f):
f.__kwdefaults__ = {}
f_code = f.__code__
po_arg_count = f_code.co_argcount
keys = f_code.co_varnames[po_arg_count : po_arg_count + f_code.co_kwonlyargcount]
for k in keys:
f.__kwdefaults__[k] = default_dict[k]
return f
return wrapper
default_dict = defaultdict(lambda: "default_value")
default_dict["a"] = "a"
default_dict["m"] = "m"
#kwdefault_decorator(default_dict)
def foo(x, *, a, b):
foo_local = "foo"
print(x, a, b, foo_local)
#kwdefault_decorator(default_dict)
def bar(x, *, m, n):
bar_local = "bar"
print(x, m, n, bar_local)
foo(1)
bar(1)
# only kw_arg permitted
foo(1, a=100, b=100)
bar(1, m=100, n=100)
output:
1 a default_value
1 m default_value
1 100 100
1 100 100
Posting this as an answer because it would be too long for a comment.
Be careful with this answer. If you try
#kwargs_decorator(a='a', b='b')
def f(x, a, b):
print(f'x = {x}')
print(f'a = {a}')
print(f'b = {b}')
f(1, 2)
it will issue an error:
TypeError: f() got multiple values for argument 'a'
because you are defining a as a positional argument (equal to 2).
I implemented a workaround, even though I'm not sure if this is the best solution:
def default_kwargs(**default):
from functools import wraps
def decorator(f):
#wraps(f)
def wrapper(*args, **kwargs):
from inspect import getfullargspec
f_args = getfullargspec(f)[0]
used_args = f_args[:len(args)]
final_kwargs = {
key: value
for key, value in {**default, **kwargs}.items()
if key not in used_args
}
return f(*args, **final_kwargs)
return wrapper
return decorator
In this solution, f_args is a list containing the names of all named positional arguments of f. Then used_args is the list of all parameters that have effectively been passed as positional arguments. Therefore final_kwargs is defined almost exactly like before, except that it checks if the argument (in the case above, a) was already passed as a positional argument.
For instance, this solution works beautifully with functions such as the following.
#default_kwargs(a='a', b='b', d='d')
def f(x, a, b, *args, c='c', d='not d', **kwargs):
print(f'x = {x}')
print(f'a = {a}')
print(f'b = {b}')
for idx, arg in enumerate(args):
print(f'arg{idx} = {arg}')
print(f'c = {c}')
for key, value in kwargs.items():
print(f'{key} = {value}')
f(1)
f(1, 2)
f(1, b=3)
f(1, 2, 3, 4)
f(1, 2, 3, 4, 5, c=6, g=7)
Note also that the default values passed in default_kwargs have higher precedence than the ones defined in f. For example, the default value for d in this case is actually 'd' (defined in default_kwargs), and not 'not d' (defined in f).
You can unpack values of dict:
from collections import OrderedDict
def f(x, a, b):
print(x, a, b)
d = OrderedDict({'a': 3, 'b':4})
f(10, *d.values())
UPD.
Yes, it's possible to implement this mad idea of modifying local scope by creating decorator which will return class with overriden __call__() and store your defaults in class scope, BUT IT'S MASSIVE OVERKILL.
Your problem is that you're trying to hide problems of your architecture behind those tricks. If you store your default values in dict, then access to them by key. If you want to use keywords - define class.
P.S. I still don't understand why this question collect so much upvotes.
Sure...
hope this helps
def funcc(x, **kwargs):
locals().update(kwargs)
print(x, a, b, c, d)
kwargs = {'a' : 1, 'b' : 2, 'c':1, 'd': 1}
x = 1
funcc(x, **kwargs)
Let's imagine I have a dict :
d = {'a': 3, 'b':4}
I want to create a function f that does the exact same thing than this function :
def f(x, a=d['a'], b=d['b']):
print(x, a, b)
(Not necessarily print, but do some stuff with the variable and calling directly from their name).
But I would like to create this function directly from the dict, that is to say, I would like to have something that look likes
def f(x, **d=d):
print(x, a, b)
and that behaves like the previously defined function. The idea is that I have a large dictionary that contains defaults values for arguments of my function, and I would like not to have to do
def f(a= d['a'], b = d['b'] ...)
I don't know if it's possible at all in python. Any insight is appreciated !
Edit : The idea is to be able to call f(5, a=3).
Edit2 : The question is not about passing arguments stored in a dict to a function but to define a function whose arguments names and defaults values are stored in a dict.
You cannot achieve this at function definition because Python determines the scope of a function statically. Although, it is possible to write a decorator to add in default keyword arguments.
from functools import wraps
def kwargs_decorator(dict_kwargs):
def wrapper(f):
#wraps(f)
def inner_wrapper(*args, **kwargs):
new_kwargs = {**dict_kwargs, **kwargs}
return f(*args, **new_kwargs)
return inner_wrapper
return wrapper
Usage
#kwargs_decorator({'bar': 1})
def foo(**kwargs):
print(kwargs['bar'])
foo() # prints 1
Or alternatively if you know the variable names but not their default values...
#kwargs_decorator({'bar': 1})
def foo(bar):
print(bar)
foo() # prints 1
Caveat
The above can be used, by example, to dynamically generate multiple functions with different default arguments. Although, if the parameters you want to pass are the same for every function, it would be simpler and more idiomatic to simply pass in a dict of parameters.
Python is designed such that the local variables of any function can be determined unambiguously by looking at the source code of the function. So your proposed syntax
def f(x, **d=d):
print(x, a, b)
is a nonstarter because there's nothing that indicates whether a and b are local to f or not; it depends on the runtime value of the dictionary, whose value could change across runs.
If you can resign yourself to explicitly listing the names of all of your parameters, you can automatically set their default values at runtime; this has already been well covered in other answers. Listing the parameter names is probably good documentation anyway.
If you really want to synthesize the whole parameter list at run time from the contents of d, you would have to build a string representation of the function definition and pass it to exec. This is how collections.namedtuple works, for example.
Variables in module and class scopes are looked up dynamically, so this is technically valid:
def f(x, **kwargs):
class C:
vars().update(kwargs) # don't do this, please
print(x, a, b)
But please don't do it except in an IOPCC entry.
try this:
# Store the default values in a dictionary
>>> defaults = {
... 'a': 1,
... 'b': 2,
... }
>>> def f(x, **kwa):
# Each time the function is called, merge the default values and the provided arguments
# For python >= 3.5:
args = {**defaults, **kwa}
# For python < 3.5:
# Each time the function is called, copy the default values
args = defaults.copy()
# Merge the provided arguments into the copied default values
args.update(kwa)
... print(args)
...
>>> f(1, f=2)
{'a': 1, 'b': 2, 'f': 2}
>>> f(1, f=2, b=8)
{'a': 1, 'b': 8, 'f': 2}
>>> f(5, a=3)
{'a': 3, 'b': 2}
Thanks Olvin Roght for pointing out how to nicely merge dictionaries in python >= 3.5
How about the **kwargs trick?
def function(arg0, **kwargs):
print("arg is", arg0, "a is", kwargs["a"], "b is", kwargs["b"])
d = {"a":1, "b":2}
function(0., **d)
outcome:
arg is 0.0 a is 1 b is 2
This question is very interesting, and it seemed different people have their different own guess about what the question really want.
I have my own too. Here is my code, which can express myself:
# python3 only
from collections import defaultdict
# only set once when function definition is executed
def kwdefault_decorator(default_dict):
def wrapper(f):
f.__kwdefaults__ = {}
f_code = f.__code__
po_arg_count = f_code.co_argcount
keys = f_code.co_varnames[po_arg_count : po_arg_count + f_code.co_kwonlyargcount]
for k in keys:
f.__kwdefaults__[k] = default_dict[k]
return f
return wrapper
default_dict = defaultdict(lambda: "default_value")
default_dict["a"] = "a"
default_dict["m"] = "m"
#kwdefault_decorator(default_dict)
def foo(x, *, a, b):
foo_local = "foo"
print(x, a, b, foo_local)
#kwdefault_decorator(default_dict)
def bar(x, *, m, n):
bar_local = "bar"
print(x, m, n, bar_local)
foo(1)
bar(1)
# only kw_arg permitted
foo(1, a=100, b=100)
bar(1, m=100, n=100)
output:
1 a default_value
1 m default_value
1 100 100
1 100 100
Posting this as an answer because it would be too long for a comment.
Be careful with this answer. If you try
#kwargs_decorator(a='a', b='b')
def f(x, a, b):
print(f'x = {x}')
print(f'a = {a}')
print(f'b = {b}')
f(1, 2)
it will issue an error:
TypeError: f() got multiple values for argument 'a'
because you are defining a as a positional argument (equal to 2).
I implemented a workaround, even though I'm not sure if this is the best solution:
def default_kwargs(**default):
from functools import wraps
def decorator(f):
#wraps(f)
def wrapper(*args, **kwargs):
from inspect import getfullargspec
f_args = getfullargspec(f)[0]
used_args = f_args[:len(args)]
final_kwargs = {
key: value
for key, value in {**default, **kwargs}.items()
if key not in used_args
}
return f(*args, **final_kwargs)
return wrapper
return decorator
In this solution, f_args is a list containing the names of all named positional arguments of f. Then used_args is the list of all parameters that have effectively been passed as positional arguments. Therefore final_kwargs is defined almost exactly like before, except that it checks if the argument (in the case above, a) was already passed as a positional argument.
For instance, this solution works beautifully with functions such as the following.
#default_kwargs(a='a', b='b', d='d')
def f(x, a, b, *args, c='c', d='not d', **kwargs):
print(f'x = {x}')
print(f'a = {a}')
print(f'b = {b}')
for idx, arg in enumerate(args):
print(f'arg{idx} = {arg}')
print(f'c = {c}')
for key, value in kwargs.items():
print(f'{key} = {value}')
f(1)
f(1, 2)
f(1, b=3)
f(1, 2, 3, 4)
f(1, 2, 3, 4, 5, c=6, g=7)
Note also that the default values passed in default_kwargs have higher precedence than the ones defined in f. For example, the default value for d in this case is actually 'd' (defined in default_kwargs), and not 'not d' (defined in f).
You can unpack values of dict:
from collections import OrderedDict
def f(x, a, b):
print(x, a, b)
d = OrderedDict({'a': 3, 'b':4})
f(10, *d.values())
UPD.
Yes, it's possible to implement this mad idea of modifying local scope by creating decorator which will return class with overriden __call__() and store your defaults in class scope, BUT IT'S MASSIVE OVERKILL.
Your problem is that you're trying to hide problems of your architecture behind those tricks. If you store your default values in dict, then access to them by key. If you want to use keywords - define class.
P.S. I still don't understand why this question collect so much upvotes.
Sure...
hope this helps
def funcc(x, **kwargs):
locals().update(kwargs)
print(x, a, b, c, d)
kwargs = {'a' : 1, 'b' : 2, 'c':1, 'd': 1}
x = 1
funcc(x, **kwargs)
I am trying to create a dictionary which maps strings to functions. The problem is, the functions can have different parameter lengths. Is there a way to handle this.
For ex:
myFuncDict
{
'A' : a ----> def a(param1, param2)
'B : b ----> def b(param1)
'C' : c ----> def c(param1, param2, param3)
}
I want to call functions like:
def test(k):
myFuncDict[k](params)
How can i achieve this?
kwargs or args is one way to go, but not sure how to handle the above using those without sending extra parameters.
Python actually makes this quite simple. You can simply unpack your container of arguments into the function call using the unpacking operator *. Here is an example:
def a(x):
print(x)
def b(x, y):
print x, y
dic = {'a': a, 'b': b}
def call_func(func, params):
dic[func](*params) # *params is the magic.
call_func('a', (1))
call_func('b', (1, 2))
In Python I can define a function as follows:
def func(kw1=None,kw2=None,**kwargs):
...
In this case, I can call func as:
func(kw1=3,kw2=4,who_knows_if_this_will_be_used=7,more_kwargs=Ellipsis)
I can also define a function as:
def func(arg1,arg2,*args):
...
which can be called as
func(3,4,additional,arguments,go,here,Ellipsis)
Finally, I can combine the two forms
def func(arg1,arg2,*args,**kwargs):
...
But, what does not work is calling:
func(arg1,arg2,*args,kw1=None,kw2=None,**kwargs): #SYNTAX ERROR (in Python 2 only, apparently this works in Python 3)
...
My original thought was that this was probably because a function
def func(arg1,arg2,*args,kw1=None):
...
can be called as
func(1,2,3) #kw1 will be assigned 3
So this would introduce some ambiguity as to whether 3 should be packed into args or kwargs. However, with Python 3, there is the ability to specify keyword only arguments:
def func(a,b,*,kw=None): # can be called as func(1,2), func(1,2,kw=3), but NOT func(1,2,3)
...
With this, it seems that there is no syntactic ambiguity with:
def func(a,b,*args,*,kw1=None,**kwargs):
...
However, this still brings up a syntax error (tested with Python3.2). Is there a reason for this that I am missing? And, is there a way to get the behavior I described above (Having *args with default arguments) -- I know I can simulate that behavior by manipulating the kwargs dictionary inside the function.
You can do that in Python 3.
def func(a,b,*args,kw1=None,**kwargs):
The bare * is only used when you want to specify keyword only arguments without accepting a variable number of positional arguments with *args. You don't use two *s.
To quote from the grammar, in Python 2, you have
parameter_list ::= (defparameter ",")*
( "*" identifier [, "**" identifier]
| "**" identifier
| defparameter [","] )
while in Python 3, you have
parameter_list ::= (defparameter ",")*
( "*" [parameter] ("," defparameter)*
[, "**" parameter]
| "**" parameter
| defparameter [","] )
which includes a provision for additional parameters after the * parameter.
UPDATE:
Latest Python 3 documentation here.
If you want to do a mixture of both remember that *args and **kwargs must be the last parameters specified.
def func(arg1,arg2,*args,kw1=None,kw2=None,**kwargs): #Invalid
def func(arg1,arg2,kw1=None,kw2=None,*args,**kwargs): #Valid
The comments seem to be based on mixing up how a function definition is constructed compared to how the arguments provided are assigned back to the parameters specified in the definition.
This is the definition of this function which has 6 parameters.
It is called by passing named and unnamed arguments to it in a function call.
For this example...
When an argument is named when calling the function it can be provided out of order.
arg1 and arg2 are mandatory parameters and if not passed to the function as named arguments, then they must be assigned in order from the provided unnamed arguments.
kw1 and kw2 have default values provided in the function definition so they are not mandatory, but if not provided for as named arguments they will take any available values from the remaining provided unnamed arguments.
Any unnamed arguments left over are provided to the function in an array called args
Any named arguments that do not have a corresponding parameter name in the function definition are provided to the function call in a dictionary called kwargs.
Clear and concise:
In Python 3.5 or greater:
def foo(a, b=3, *args, **kwargs):
defaultKwargs = { 'c': 10, 'd': 12 }
kwargs = { **defaultKwargs, **kwargs }
print(a, b, args, kwargs)
# Do something else
foo(1) # 1 3 () {'c': 10, 'd': 12}
foo(1, d=5) # 1 3 () {'c': 10, 'd': 5}
foo(1, 2, 4, d=5) # 1 2 (4,) {'c': 10, 'd': 5}
Note: you can use
In Python 2
kwargs = merge_two_dicts(defaultKwargs, kwargs)
In Python 3.5
kwargs = { **defaultKwargs, **kwargs }
In Python 3.9
kwargs = defaultKwargs | kwargs # NOTE: 3.9+ ONLY
If you are looking to do that in Python 2, I have found a workaround explained in this post, using a decorator.
This decorator assigns default kwarg values if they are not strictly defined.
from functools import wraps
def force_kwargs(**defaultKwargs):
def decorator(f):
#wraps(f)
def g(*args, **kwargs):
new_args = {}
new_kwargs = defaultKwargs
varnames = f.__code__.co_varnames
new_kwargs.update(kwargs)
for k, v in defaultKwargs.items():
if k in varnames:
i = varnames.index(k)
new_args[(i, k)] = new_kwargs.pop(k)
# Insert new_args into the correct position of the args.
full_args = list(args)
for i, k in sorted(new_args.keys()):
if i <= len(full_args):
full_args.insert(i, new_args.pop((i, k)))
else:
break
# re-insert the value as a key-value pair
for (i, k), val in new_args.items():
new_kwargs[k] = val
return f(*tuple(full_args), **new_kwargs)
return g
return decorator
Result
#force_kwargs(c=7, z=10)
def f(a, b='B', c='C', d='D', *args, **kw):
return a, b, c, d, args, kw
# a b c d args kwargs
f('r') # 'r', 'B', 7, 'D', (), {'z': 10}
f(1, 2, 3, 4, 5) # 1, 2, 7, 3, (4,5), {'z': 10}
f(1, 2, 3, b=0, c=9, f='F', z=5) # 1, 0, 9, 2, (3,), {'f': 'F', 'z': 5}
Variant
If you want to use the default values as written in the function definition, you could access the argument default values using f.func_defaults, which lists the default values. You would have to zip them with the end of the f.__code__.varnames to match these default values with the variable names.
I don't want to use *args or **kwargs since I can't change function declaration.
For example:
def foo( a, b, c ) """Lets say values passed to a, b and c are 1,2 and 3 respectively"""
...
...
""" I would like to generate an object preferably a dictionary such as {'a':1, 'b':2, 'c':3} """
...
...
Can anyone suggest a way to do this?
Thanks in advance.
If you can't change the function "declaration" (why not?) but you can change the contents of the function, then just create the dictionary as you want it:
def foo(a, b, c):
mydict = {'a': a, 'b': b, 'c': c}
If that doesn't work, I think you need a better explanation of what you want and what the constraints are in your case.
This is also going to give similar results in the above case (where you don't show any local variables other than the arguments), but be warned that you should not try to modify locals():
def foo(a, b, c):
mydict = locals()
#Rohit, we do not understand what you mean when you say "the function declaration". If you mean you don't want to change the API of the function (the documented way the function is called), perhaps because you have existing code already calling an existing function, then you can still use the **kwargs notation, and the callers will never know:
def foo(a, b, c):
return a + b + c
def foo(**kwargs):
total = 0
for x in ("a", "b", "c"):
assert x in kwargs
total += kwargs[x]
return total
def bar():
foo(3, 5, 7)
bar() cannot tell which version of foo() it is calling, and does not care.
Perhaps you are looking for a "wrapper" you can wrap around existing function objects, without changing the actual source code of the function object?
def make_wrapper(fn, *arg_names):
def wrapped_fn(*args):
mydict = dict(tup for tup in zip(arg_names, args))
print("TEST: mydict: %s" % str(mydict))
return fn(*args)
return wrapped_fn
def foo(a, b, c):
return a + b + c
foo = make_wrapper(foo, "a", "b", "c")
foo(3, 5, 7)
The new wrapped function gathers the arguments into mydict and prints mydict before calling the function.
By diligent searching of StackOverflow, I found out how to do this. You use the inspect module.
import inspect
def make_wrapper(fn):
arg_names = inspect.getargspec(fn)[0]
def wrapped_fn(*args, **kwargs):
# mydict now gets all expected positional arguments:
mydict = dict(tup for tup in zip(arg_names, args))
# special name "__args" gets list of all positional arguments
mydict["__args"] = args
# mydict now updated with all keyword arguments
mydict.update(kwargs)
# mydict now has full information on all arguments of any sort
print("TEST: mydict: %s" % str(mydict))
return fn(*args, **kwargs)
return wrapped_fn
def foo(a, b, c, *args, **kwargs):
# a, b, and c must be set; extra, unexpected args will go in args list
return a + b + c
foo = make_wrapper(foo)
foo(3, 5, 7, 1, 2)
# prints: TEST: mydict: {'a': 3, 'c': 7, 'b': 5, '__args': (3, 5, 7, 1, 2)}
# returns: 15
There you go, a perfect solution to the problem you stated. It is a wrapper, you don't need to pass in the arguments, and it should work for any function. If you need it to work with class objects or something you can read the docs for inspect and see how to do it.
Note, of course order is not preserved in dictionaries, so you may not see the exact order I saw when I tested this. But the same values should be in the dict.
def foo(a, b, c):
args = {"a": a, "b": b, "c": c}