I'm newbie in Python.
I'm using Python 3.7.7 and Tensorflow 2.1.0.
This is my code:
import tensorflow as tf
import tensorflow_datasets as tfds
d = {"name": "omniglot:3.0.0", "data_dir": "d:\\tmp"}
omniglot_builder = tfds.builder("omniglot:3.0.0", builder_init_kwargs=d)
omniglot_builder.download_and_prepare(download_dir="d:\\tmp")
But I get this error:
got an unexpected keyword argument 'builder_init_kwargs'
I want to set data_dir, but I don't know how to do it. I have tried to set download_dir in omniglot_builder.download_and_prepare(download_dir="d:\\tmp") but it stills download it to ~/tensorflow_datasets.
From Tensorflow documentation for tdfs.builder:
**builder_init_kwargs: dict of keyword arguments passed to the DatasetBuilder. These will override keyword arguments passed in name,
if any.
How can I set builder_init_kwargs parameter value?
Based on the docs, which say the tfds.builder method has type:
tfds.builder(
name, **builder_init_kwargs
)
You want to do this:
dict = {"name":"omniglot:3.0.0", "data_dir": "d:\\tmp"}
tfds.builder(**dict)
The ** syntax passes a variable as the kwargs, making the above code equivalent to:
tfds.builder(name="omniglot:3.0.0", data_dir="d:\\tmp")
To set a kwargs argument in python, you have to simply add the ** before the argument itself.
So, this would be your code:
import tensorflow as tf
import tensorflow_datasets as tfds
dict = {"name": "omniglot:3.0.0", "data_dir": "d:\\tmp"}
omniglot_builder = tfds.builder("omniglot:3.0.0", builder_init_kwargs=**dict)
omniglot_builder.download_and_prepare(download_dir="d:\\tmp")
Of course, I am just guessing, because I know what a kwargs argument is, but I am not familiar with tensorflow.
Hope this helps!
It seems you need a little help with argument packing and unpacking.
In the definition of a function or method, you specify the sequence of arguments that will be passed. If you want to have a variable number of input arguments, the mechanism is to "pack" them together into a list or directory. For example say you want to get the sum of all arguments given:
def get_sum(a, b): #only useful for two numbers
return a + b
def get_sum(a,b,c): #only useful for three numbers
return a + b
You would have to have a different definition for every possible number of input arguments. The solution to this is to use the packing operator to pack all arguments given into a list that can be iterated over
def get_sum(*list_of_inputs): # * will pack all subsequent positional arguments into a list
x = 0
for item in list_of_inputs:
x += item
return x
get_sum(1,2,3,4,5,6,7) #returns 28
get_sum() #returns 0
The same can be done for keyword arguments which get packed into a dictionary:
def foo(**keyword_args):
for k in keyword_args:
print(f'{k}: {keyword_args[k]}')
Now when you are using (calling) a function, sometimes you need to be able to "unpack" a list or a dictionary into the function call. The same operator is used to pack and unpack, so it looks very similar:
def foo(a,b,c):
print(f'{a} + {b} = {c}')
arguments = ['spam', 'eggs', 'delicious']
foo(*arguments) #unpack the list of arguments into their required positions
Now finally on to your specific case: the function you are trying to use defines **kwargs in its definition. This means that it will take any subsequent keyword arguments and pack them all up into a dictionary to be used inside the function definition. The practical meaning of this is that you can provide keyword arguments to the function that aren't specifically defined in the function signature (this is particularly common when the function is calling another function and passing along the arguments). If you have already packed up your arguments prior to calling the function, it is easy to unpack them using the same process as shown by Oli: tfds.builder(**dict)
Related
I wanted to know how to work with an array as a functional argument in Python. I will show a short example:
def polynom(x, coeff_arr):
return coeff_arr[0]+ coeff_arr[1]+x +coeff_arr[2]*x**2
I obviously get the error that 2 positional arguments are needed but 4 were given when I try to run it, can anybody tell me how to do this accept just using (coeff_arr[i]) in the argument of the function?
Cheers
Your question is missing the code you use to call the function, but from the error I infer that you are calling it as polynom(x, coefficient1, coefficient2, coefficient3). Instead you need to either pass the coefficients as a list:
polynom(x, [coefficient1, coefficient2, coefficient3])
Or use the unpacking operator * to define the function as follows, which will take all positional arguments after x and put them into coeff_arr as a list:
def polynom(x, *coeff_arr):
(The unpacking operator can also be used in a function call, which will do the opposite of taking a list and passing its elements as positional arguments:
polynom(x, *[coefficient1, coefficient2, coefficient3])
is equivalent to
polynom(x, coefficient1, coefficient2, coefficient3)
)
I am learning about functions in python and found many good tutorials and answers about functions and their types, but I am confused in some places. I have read the following:
If function has "=" then it's a keyword argument i.e (a,b=2)
If function does not have "=" then it's positional argument i.e (a,b)
My doubts :
What is the meaning of a required argument and an optional argument? Is the default argument also a keyword argument? (since because both contain "=")
Difference between the positional, keyword, optional, and required
arguments?
python official documentation says that there are two types of arguments. If so, then what are *args and **kargs (I know how they work but don't know what they are)
how *args and **kargs store values? I know how *args and
**kargs works but how do they store values? Does *args store values in a tuple and **kargs in the dictionary?
please explain in deep. I want to know about functions because I am a newbie :)
Thanks in advance
Default Values
Let's imagine a function,
def function(a, b, c):
print a
print b
print c
A positional argument is passed to the function in this way.
function("position1", "position2", "position3")
will print
position1
position2
position3
However, you could pass in a keyword argument as below,
function(c="1",a="2",b="3")
and the output will become:
2
3
1
The input is no longer based on the position of the argument, but now it is based on the keyword.
The reason that b is optional in (a,b=2) is because you are giving it a default value.
This means that if you only supply the function with one argument, it will be applied to a. A default value must be set in the function definition. This way when you omit the argument from the function call, the default will be applied to that variable. In this way it becomes 'optional' to pass in that variable.
For example:
def function(a, b=10, c=5):
print a
print b
print c
function(1)
and the output will become:
1
10
5
This is because you didn't give an argument for b or c so they used the default values. In this sense, b and c are optional because the function will not fail if you do not explicitly give them.
Variable length argument lists
The difference between *args and **kwargs is that a function like this:
def function(*args)
for argument in args:
print argument
can be called like this:
function(1,2,3,4,5,6,7,8)
and all of these arguments will be stored in a tuple called args. Keep in mind the variable name args can be replaced by any variable name, the required piece is the asterisk.
Whereas,
def function(**args):
keys = args.keys()
for key in keys:
if(key == 'somethingelse'):
print args[key]
expects to be called like this:
function(key1=1,key2=2,key3=3,somethingelse=4,doesnt=5,matter=6)
and all of these arguments will be stored in a dict called args. Keep in mind the variable name args can be replaced by any variable name, the required piece is the double asterisk.
In this way you will need to get the keys in some way:
keys = args.keys()
Optional arguments are those that have a default or those which are passed via *args and **kwargs.
Any argument can be a keyword argument, it just depends on how it's called.
these are used for passing variable numbers of args or keyword args
yes and yes
For more information see the tutorial:
https://docs.python.org/2/tutorial/controlflow.html#more-on-defining-functions
Okay this one is confusing. My old piece of code has something like
map(lambda x:x.func1(arg1), other_args_to_be_mapped)
now I would like to make arg1 -> *args
while other_args_to_be_mapped stays unchanged.
in func1, the length of arguments will be checked different operations. My questions are
1) which length will be checked? arg1 or other_args_to_be_mapped
2) in func1, how should I set up the default? It was like
def func1(arg1=something)
but now with potential multiple arguments, I don't know what to do with the initialization. I want to be able to do something like
def func1(args*=something, something_else)
Is that even possible?
If I understand your question correctly, you're looking for variable arguments. These can be mixed with fixed arguments, provided you obey a logical ordering (fixed arguments first, then keyword arguments or variable arguments).
For example, the following shows how map to a function that takes in one constant argument and one variable argument. If you would like different behaviour, please provide a concrete example of what you are trying to accomplish
import random
class Foo:
def get_variable_parameters(self):
return [1] if random.random() > .5 else [1,2]
def foo( self, arg, *args ):
print("Doing stuff with constant arg", arg)
if len(args) == 1:
print("Good",args)
else:
print("Bad",args)
list(map( lambda x : x.foo( 'Static Argument', *x.get_variable_parameters()), [Foo(),Foo(),Foo()] ))
We don't know how many arguments are going to be passed to foo (in this trivial case, it's one or two), but the "*" notation accepts any number of objects to be passed
Note I've encapsulated map in list so that it gets evaluated, as in python3 it is a generator. List comprehension may be more idiomatic in python. Also don't forget you can always use a simple for loop - an obfuscated or complex map call is far less pythonic than a clear (but several line) for-loop, imo.
If, rather, you're trying to combine multiple arguments in a map call, I would recommend using the same variable argument strategy with the zip function, e.g.,
def foo(a,*b): ...
map(lambda x : foo(x[0],*x[1]), zip(['a','b'],[ [1], [1,2] ]))
In this case, foo will get called first as foo('a',1), and then as foo('b',2,3)
For this function
def eat_dog(name, should_digest=True):
print "ate dog named %s. Digested, too? %" % (name, str(should_digest))
I want to, external to the function, read its arguments and any default values attached. So for this specific example, I want to know that name has no default value (i.e. that it is a required argument) and that True is the default value for should_digest.
I'm aware of inspect.getargspec(), which does give me information about arguments and default values, but I see no connection between the two:
ArgSpec(args=['name', 'should_digest'], varargs=None, keywords=None, defaults=(True,))
From this output how can I tell that True (in the defaults tuple) is the default value for should_digest?
Additionally, I'm aware of the "ask for forgiveness" model of approaching a problem, but unfortunately output from that error won't tell me the name of the missing argument:
>>> eat_dog()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: eat_dog() takes at least 1 argument (0 given)
To give context (why I want to do this), I'm exposing functions in a module over a JSON API. If the caller omits certain function arguments, I want to return a specific error that names the specific function argument that was omitted. If a client omits an argument, but there's a default provided in the function signature, I want to use that default.
Python3.x
In a python3.x world, you should probably use a Signature object:
import inspect
def get_default_args(func):
signature = inspect.signature(func)
return {
k: v.default
for k, v in signature.parameters.items()
if v.default is not inspect.Parameter.empty
}
Python2.x (old answer)
The args/defaults can be combined as:
import inspect
a = inspect.getargspec(eat_dog)
zip(a.args[-len(a.defaults):],a.defaults)
Here a.args[-len(a.defaults):] are the arguments with defaults values and obviously a.defaults are the corresponding default values.
You could even pass the output of zip to the dict constructor and create a mapping suitable for keyword unpacking.
looking at the docs, this solution will only work on python2.6 or newer since I assume that inspect.getargspec returns a named tuple. Earlier versions returned a regular tuple, but it would be very easy to modify accordingly. Here's a version which works with older (and newer) versions:
import inspect
def get_default_args(func):
"""
returns a dictionary of arg_name:default_values for the input function
"""
args, varargs, keywords, defaults = inspect.getargspec(func)
return dict(zip(args[-len(defaults):], defaults))
Come to think of it:
return dict(zip(reversed(args), reversed(defaults)))
would also work and may be more intuitive to some people.
Depending on exactly what you need, you might not need the inspect module since you can check the __defaults__ attribute of the function:
>>> eat_dog.__defaults__
(True,)
>>> eat_dog.__code__.co_argcount
2
>>> eat_dog.__code__.co_varnames
('name', 'should_digest')
>>>
>>> eat_dog.__kwdefaults__
>>> eat_dog.__code__.co_kwonlyargcount
0
You can use inspect module with its getargspec function:
inspect.getargspec(func)
Get the names and default values of a Python function’s arguments. A tuple of four things is returned: (args, varargs, keywords, defaults). args is a list of the argument names (it may contain nested lists). varargs and keywords are the names of the * and ** arguments or None. defaults is a tuple of default argument values or None if there are no default arguments; if this tuple has n elements, they correspond to the last n elements listed in args.
See mgilson's answer for exact code on how to retrieve argument names and their default values.
To those looking for a version to grab a specific default parameter with mgilson's answer.
value = signature(my_func).parameters['param_name'].default
Here's a full working version, done in Python 3.8.2
from inspect import signature
def my_func(a, b, c, param_name='apple'):
pass
value = signature(my_func).parameters['param_name'].default
print(value == 'apple') # True
to take care of keyword-only args (and because defaults and kwonlydefaults can be None):
spec = inspect.getfullargspec(func)
defaults = dict(zip(spec.args[::-1], (spec.defaults or ())[::-1]))
defaults.update(spec.kwonlydefaults or {})
You can get this via some of the __dunder__ vars as mentioned by other posts. Putting that into a simple helper function can get you a dictionary of default values.
.__code__.co_varnames: A tuple of all input variables
.__defaults__: A tuple of the default values
It is worth noting that this tuple only incudes the default provided variables which must always be positioned last in the function arguments
You can use these two items to match the last n variables in the .__code__.co_varnames with all the items in the .__defaults__
EDIT Thanks to #griloHBG - Added if statement to prevent exceptions when no defaults are specified.
def my_fn(a, b=2, c='a'):
pass
def get_defaults(fn):
if fn.__defaults__==None:
return {}
return dict(zip(
fn.__code__.co_varnames[-len(fn.__defaults__):],
fn.__defaults__
))
print(get_defaults(my_fn))
Should give:
{'b': 2, 'c': 'a'}
In python, all the arguments with default value come after the arguments without default value. So the mapping should start from the end till you exhaust the default value list. Hence the logic:
dict(zip(reversed(args), reversed(defaults)))
gives the correctly mapped defaults.
I have a Python script which creates a dictionary of its own functions and I'd like it to execute them by reading in a function name and arguments using YARP (knowledge of YARP is irrelevant to this question though).
I create a list of strings called "inc" which is populated by values coming into the program. The first item is a function name, and any other strings in the list are arguments. I create a dictionary called "methods" where the key is the function name and the value is a reference to the function object (using the inspect module). I store the return value of the function in a variable "result".
The snippet below shows a simplified version of what I'm using so far, which works fine, but can't handle functions with more than one argument. To circumvent this I use a list if a function needs more parameters:
if len(inc) == 1:
result = methods[inc[0]]() # call method with 0 arguments
elif len(inc) == 2:
result = methods[inc[0]](inc[1]) # call method passing a string
else:
args = []
result = methods(inc[0])(inc[1:]) # call method passing a list
Ideally, I'd like to change this so that my functions can have any number of arguments, but I can't figure out how I can do this. I'm new to Python and I have looked at the documentation and various websites - I just can't find a solution. I've tried things like creating a tuple of the arguments, but that doesn't work either as it ends up passing the whole tuple in as one parameter.
Is there a better solution to this problem, like creating some kind of object which represents a set of parameters and passing that into the function? Any suggestions would be greatly appreciated!
You should check out https://stackoverflow.com/a/3394898/1395668.
The magic you are looking for is the *. Apply this to your list and it unpacks the items into the argument fields of your function:
a = [ 1, 2, 3]
def myfunc(a, b, c):
return a + b + c
print myfunc(*a)
Check out ** for the same approach for dict
It's a bit strange to have this kind of mixed structure:
inc = [func_name, arg1, arg2, ...]
Wouldn't it be much more natural to have two separate bits of information?
func_name = ...
args = [arg1, arg2, ...]
The you could do
methods[func_name](*args)
(Usually, I wouldn't bind the functions name to a variable, but preferably the function itself.)