What I would really like to do is make a function with a default first argument
def do_thing(self, category:str='default', sub_category:str, argA:int|float, argB:int|float, argC:int|float)
...
obj.do_thing('cat1', 'sub1', 1, 2, 3)
obj.do_thing('sub2', 1, 2, 3) #In thise case 'default' is used for the category
But that isn't possible since all arguments with default values must come after non-default arguments.
I could technically move the category argument last and the default work work fine, but the category and sub_category arguments are closely related so I would like to keep them grouped together at the beginning of the line then followed by argA/B/C since it follows the format [selected thing, arguments to use on selected thing]
Using *args would also technically work and could check number of arguments and whether there are one or two strings at the beginning of the list.
But this seems much less clear what arguments are expected to be given and doesn't have type hinting or the arguments which are useful for IDE auto-suggestions.
def do_thing(self, *args)
Multiple Dispatch seemed like it did sort of what I wanted since it allows different versions of the function to be called based on the arguments that are used.
So I tried doing this
from multipledispatch import dispatch
#dispatch(str, str, object, object, object) #Use object to allow more than one type since float and int are both valid
def do_thing(self, category:str='default', sub_category:str, argA:int|float, argB:int|float, argC:int|float)
...
#dispatch(str, object, object, object)
def do_thing(self, sub_category:str, argA:int|float, argB:int|float, argC:int|float)
do_thing('default', sub_category, argA, argB, argC)
This mostly works if all the arguments are entered as positional arguments, however the users of this library regularly (but not always) call the function with argA-argC as keyword arguments, which doesn't work with multiple dispatch.
Is there any way to check if there are at least 2 positional argument and then only if the first two arguments of the function are strings run one function, else for any other combination, call the other version?
functools.singledispatchmethod almost does what I want to do, but in both cases, the first argument will always be a string, so I don't think that will work unless there's a way to tell it to look at the second argument instead of the first
Alternatively, is there a better way to go about doing what I was trying to do to begin with? Basically create a default value for the first argument?
I'm newbie in Python.
I'm using Python 3.7.7 and Tensorflow 2.1.0.
This is my code:
import tensorflow as tf
import tensorflow_datasets as tfds
d = {"name": "omniglot:3.0.0", "data_dir": "d:\\tmp"}
omniglot_builder = tfds.builder("omniglot:3.0.0", builder_init_kwargs=d)
omniglot_builder.download_and_prepare(download_dir="d:\\tmp")
But I get this error:
got an unexpected keyword argument 'builder_init_kwargs'
I want to set data_dir, but I don't know how to do it. I have tried to set download_dir in omniglot_builder.download_and_prepare(download_dir="d:\\tmp") but it stills download it to ~/tensorflow_datasets.
From Tensorflow documentation for tdfs.builder:
**builder_init_kwargs: dict of keyword arguments passed to the DatasetBuilder. These will override keyword arguments passed in name,
if any.
How can I set builder_init_kwargs parameter value?
Based on the docs, which say the tfds.builder method has type:
tfds.builder(
name, **builder_init_kwargs
)
You want to do this:
dict = {"name":"omniglot:3.0.0", "data_dir": "d:\\tmp"}
tfds.builder(**dict)
The ** syntax passes a variable as the kwargs, making the above code equivalent to:
tfds.builder(name="omniglot:3.0.0", data_dir="d:\\tmp")
To set a kwargs argument in python, you have to simply add the ** before the argument itself.
So, this would be your code:
import tensorflow as tf
import tensorflow_datasets as tfds
dict = {"name": "omniglot:3.0.0", "data_dir": "d:\\tmp"}
omniglot_builder = tfds.builder("omniglot:3.0.0", builder_init_kwargs=**dict)
omniglot_builder.download_and_prepare(download_dir="d:\\tmp")
Of course, I am just guessing, because I know what a kwargs argument is, but I am not familiar with tensorflow.
Hope this helps!
It seems you need a little help with argument packing and unpacking.
In the definition of a function or method, you specify the sequence of arguments that will be passed. If you want to have a variable number of input arguments, the mechanism is to "pack" them together into a list or directory. For example say you want to get the sum of all arguments given:
def get_sum(a, b): #only useful for two numbers
return a + b
def get_sum(a,b,c): #only useful for three numbers
return a + b
You would have to have a different definition for every possible number of input arguments. The solution to this is to use the packing operator to pack all arguments given into a list that can be iterated over
def get_sum(*list_of_inputs): # * will pack all subsequent positional arguments into a list
x = 0
for item in list_of_inputs:
x += item
return x
get_sum(1,2,3,4,5,6,7) #returns 28
get_sum() #returns 0
The same can be done for keyword arguments which get packed into a dictionary:
def foo(**keyword_args):
for k in keyword_args:
print(f'{k}: {keyword_args[k]}')
Now when you are using (calling) a function, sometimes you need to be able to "unpack" a list or a dictionary into the function call. The same operator is used to pack and unpack, so it looks very similar:
def foo(a,b,c):
print(f'{a} + {b} = {c}')
arguments = ['spam', 'eggs', 'delicious']
foo(*arguments) #unpack the list of arguments into their required positions
Now finally on to your specific case: the function you are trying to use defines **kwargs in its definition. This means that it will take any subsequent keyword arguments and pack them all up into a dictionary to be used inside the function definition. The practical meaning of this is that you can provide keyword arguments to the function that aren't specifically defined in the function signature (this is particularly common when the function is calling another function and passing along the arguments). If you have already packed up your arguments prior to calling the function, it is easy to unpack them using the same process as shown by Oli: tfds.builder(**dict)
Under normal circumstances one calls a function with its default arguments by omitting those arguments. However if I'm generating arguments on the fly, omitting one isn't always easy or elegant. Is there a way to use a function's default argument explicitly? That is, to pass an argument which points back to the default argument.
So something like this except with ~use default~ replaced with something intelligent.
def function(arg='default'):
print(arg)
arg_list= ['not_default', ~use default~ ]
for arg in arg_list:
function(arg=arg)
# output:
# not_default
# default
I don't know if it's even possible and given the term "default argument" all my searches come up with is coders first tutorial. If this functionality is not supported that's ok too, I'd just like to know.
Unfortunately there is no such feature in Python. There are hackarounds, but they're not very likable.
The simple and popular pattern is to move the default into the function body:
def function(arg=None):
if arg is None:
arg = 'default'
...
Now you can either omit the argument or pass arg=None explicitly to take on the default value.
There is no general purpose way to omit an argument; you can specialize to particular functions by explicitly passing the appropriate default value, but otherwise, your only option is to fail to pass the argument.
The closest you could come is to replace your individual values with tuples or dicts that omit the relevant argument, then unpack them at call time. So for your example, you'd change arglist's definition to:
arg_list = [{'arg': 'not_default'}, {}]
then use it like so:
for arg in arg_list:
function(**arg)
A slightly uglier approach is to use a sentinel when you don't want to pass the argument, use that in your arg_list, and test for it, e.g.:
USEDEFAULT = object()
arg_list = ['not_default', USEDEFAULT]
for arg in arg_list:
if arg is USEDEFAULT:
function()
else:
function(arg=arg)
Obviously a bit less clean, but possibly more appropriate for specific use cases.
For the following Python script:
from sys import argv
script, input_encoding, error = argv
def main(language_file, encoding, errors):
line = language_file.readline()
if line:
print_line(line, encoding, errors)
return main(language_file, encoding, errors)
def print_line(line, encoding, errors):
next_lang = line.strip()
raw_bytes = next_lang.encode(encoding, errors=errors)
cooked_string = raw_bytes.decode(encoding, errors=errors)
print(raw_bytes, "<===>", cooked_string)
languages = open("languages.txt", encoding="utf-8")
main(languages, input_encoding, error)
Looking at the main function I do not understand the following line:
print_line(line, encoding, errors)
Why are we calling the print_line function and passing arguments to it that are named exactly the same as its parameters?
print_line()
When I attempt to call the print_line() argument without passing arguments Python is outputting:
print_line() missing 3 required positional arguments: 'line',
'encoding', and 'errors'
OP: print_line() missing 3 required positional arguments: 'line', 'encoding', and 'errors'
The error is obvious since it is the way the function print_line() was defined.
Furthermore:
def print_line(line, encoding, errors):
print(line, encoding, errors)
line = 1
encoding = 2
errors = 3
print_line(errors, encoding, line)
OUTPUT:
3 2 1
Note: It is positional, not naming arguments
EDIT: 1
def abc(a,b,c=2):
return a+b+c
abc(1,2) #both positional argument and c is default
5
abc(2, b=3) # positional, named and again c is default
7
abc(a=2,b=4) # both named argument and c is default
8
EDIT 2:
OP: What is the purpose of a positional argument please?
Well ..
Short answer: A positional argument is any argument that's not supplied as a key=value pair.
To understand what that means, unfortunately, is somewhat involved.
The term "argument" is used somewhat imprecisely throughout the programming community and especially in Python documentation.
Technically arguments are what you pass into functions and parameters are what you define as the names/placeholders for those arguments.
So, when I define a function thus:
def foo(a,b):
return a+b
... and call it like so:
foo(1,3)
... then a and b are my parameters while 1 and 3 are arguments for a given call to that function.
Now this is a quibble. People will often refer to a and b as "arguments" to their function when they are actually the names (parameters) which will contain the arguments while the function is executing.
Now, with this point made, understand that Python supports four classes of parameters: "required positional" (the kind you've seen for just about any other programming language), "optional" ... or "defaulted" ... positional parameters with some default value specified, "star args" (similar to the VARARGS support in some other languages such as C/C++ and Java), and "kwargs."
The latter is almost unique to Python (though Ruby has very similar support).
So you can define a function with a parameter list like:
def bar(a, b, c=None, d=[], *e, **opts):
'''bar takes all sorts of arguments
'''
results = dict()
results['positional 1'] = a
results['positional 2'] = b
results['sum']=a+b
results['optional'] = list()
for each in e:
results['optional'].append(each)
if c is not None:
results['default over-ridden']=c
else:
results['default']='no arg supplied for parameter c'
d.append(results)
if 'verbose' in opts and opts['verbose']:
for k,v in results:
print '%s:%s' % (k, v)
return (results, d)
... this, rather contrived, example has a couple of normal, traditional positional parameters (a and b), and optional third and fourth parameters (one of which will default to the special Python singleton value None if bar() is called with only two arguments and the other which will default to a list) as well as an optional variable number of additional arguments (which are passed into our function through the parameter named "e") and, finally, bar() can accept any number of "keyword" arguments (passed to it as key-value pairs and referenced through the parameter "opts" (in my example).
There's a lot to digest there. First there are a and b. Those are just like you'd see in most programming languages. If you call this with only one argument you'll get an error because the function requires two arguments. Then there are c and d ... these parameters can be supplied with arguments ... but if the function is called with only the two required arguments than parameter c will refer to "None" and d will refer to the list ... which was instantiated at the time we defined the function!
Whoa! Re-read that last bit because it's a common source of confusion to people who make the mistake of defining functions with parameters that default to mutable types (lists, dictionaries, sets, or most instances of your custom defined classes).
When you're defining a function in Python the interpreter is executing code. It's executing the def statement and evaluating the arguments (which become your function's parameters). So a Python virtual machine instantiates a list (the [] --- empty list literal) as the function is defined. The parameter (e in my example) is now bound to that list, just as any Python "variable" (name) is bound to any other object. And the object to which it refers will be accessible any time bar() is called with three or fewer arguments.
Here's the tricky bit: any time you call bar with more than three arguments then the parameter e will be bound (for the duration of that particular call) to the fourth argument. The underlying list will be hidden for the duration of that call. Such the default list object is contained in a closure and would normally be completely inaccessible outside of the function. (In this example I've including a reference to it in my return statement, showing how we can export a reference to that enclosed object if we choose).
So, if I pass bar() four arguments then it will attempt to modify the object referred to through its "e" parameter using that object's 'append' method. (Obviously that will fail and raise an exception for any object that doesn't support an append method). Every time I call bar() with only three arguments then I'm modifying that enclosed list object within its closure.
This can be useful but is rarely what new programmers expect when they are learning Python. So, as a rule, don't define functions with mutable objects as defaults to your parameters. When you understand the semantics well enough then you know when to break that rule.
I used "None" as my default for the other parameter for a reason. It's often useful to use "None" as the default for any parameter for which you want to have a default value and, then, to explicitly test "is None" and supply your own default from within the body of your function. In this way you can distinguish between your default parameter value and any argument that was explicitly passed to you (and happened to match your default). (This will also prevent the case where you might inadvertently create a mutable closure as previously described. The assignments/bindings occurring in the body of your function will result in a new instantiation for every call that reaches that condition).
So we've covered the required positional parameters, and those which supply defaults (and are, thus, optionally positional parameters).
With parameter 'e' we see Python's support for variable numbers of arguments. Any arguments specified after the fourth will all be gathered up into a tuple and passed to us through our parameter 'e' ... except for any arguments of the form: this=that.
Which brings us, at last. to "opts." Conventionally Python programmers will refer to this parameter as "kwargs" (key/word arguments). I named it "opts" to emphasize that point that "kwargs" is only a conventional bit of terminology and, technically, a bit confusing since, as I've belabored throughout this text, it's a parameter which is referring to any keyword arguments which might have been passed as parameters through some function call.
It's possible to write all your functions such that they take NO positional arguments and are only defined with a "key/words arguments" parameter. You can then ensure that callers to your function must always spell out which argument is bound to what name every time that call on you. This can be handy if your function might have catastrophic consequences in any case where it was called with arguments in the wrong order.
I realize this is confusing ... and I definitely recommend that you play with different function definitions and a wide variety of calls to see how these interact.
I'll also note one additional "gotchya" that can bite you. Your callers CANNOT pass kwargs (opts) values with keys that have names conflicting with your parameter names. Try calling bar() with an argument list like: bar(1,2,3,4,a=99) and you'll get an exception like this: "TypeError: bar() got multiple values for keyword argument 'a'"
Python is parsing the arguments into parameters by managing a namespace (like a dictionary). Attempting to supply a key/word argument which any keys that match your parameter names creates an ambiguity ... and thus raises an exception.
I'll also add two additional notes to this already cumbersome answer.
When you're calling functions in Python you can pass arguments like so:
myargs=(1,2,3)
bar(*myargs)
... and this will be treated as though you'd called it with bar(1,2,3) --- or as if you make the following function call: apply(bar, myargs).
In fact it used to be that you had to use the apply() function in order to accomplish this layer of indirection (back when you could define functions with *foo parameters but not call them with *foo arguments).
(The addition of the *args syntax on function calls largely displaced the use of Python's apply() builtin function).
... and lastly it's possible to pass a dictionary of kwargs using this syntax:
mykwargs={'z':99, 'whatever':'yikes'}
bar(1,2,3, **mykwargs)
... or combined with my previous example:
bar(*myargs, **mykwargs)
So it's vital to understand this distinction between what * and ** mean when defining functions and what they mean when calling them. The meanings are complementary to one another and intuitive if you understand the distinction between arguments and parameters (despite how the term "arguments" is more commonly used).
Why are we calling the print_line function and passing arguments to it that are named exactly the same as it's parameters?
That is really just a coincidence. The following does exactly the same as your example:
from sys import argv
script, input_encoding, error = argv
def main(language_file, what_encoding_do_you_want, list_of_errors):
next_line = language_file.readline()
if next_line:
print_line(next_line, what_encoding_do_you_want, list_of_errors)
return main(language_file, what_encoding_do_you_want, list_of_errors)
def print_line(line, encoding, errors):
next_lang = line.strip()
raw_bytes = next_lang.encode(encoding, errors=errors)
cooked_string = raw_bytes.decode(encoding, errors=errors)
print(raw_bytes, "<===>", cooked_string)
languages = open("languages.txt", encoding="utf-8")
main(languages, input_encoding, error)
It all comes down to 'scope'. The function-definition of print_line declares that it has three (positional) arguments, which can be referred to inside the function as 'line', 'encoding' and 'errors'.
The function-call to print_line from main adds three arguments to the call that refer to existing variables or arguments at that point in code: line refers to the variable created there; encoding and encoding refer to the argument to main function itself.
The function requires three arguments, and normally, you provide them in the order specified.
Python allows you to pass them in any order you like, though:
print_line(encoding='ascii', line='hello', errors=None)
or even
my_dict = {
'line': 'privet, mir!',
'errors': None,
'encoding': 'utf-8'
}
print_line(**my_dict)
However, the function requires all these arguments; you have to pass them in somehow, or you get an error message, like you well noticed.
The variables' scope is local: Anything in the def is local to the function it defines, and separate from any other variables with the same name outside of that block.
A parameter can be made optional by defining a default value. Optional parameters must always come after mandatory parameters in Python.
def another_fun(confetti, fireworks, bassoons=None, candy='sugar fluff')
If you call another_fun with only two parameters, bassoons will default to None and candy will be set to the string 'sugar fluff'.
Why are we calling the print_line function
Well, to execute it obviously ;-)
and passing arguments to it
Because the function is defined to take arguments - it cannot do its job if you don't give what it needs to work on.
that are named exactly the same as its parameters?
This is actually totally irrelevant - it just happens that the variables in main() are named the same as the function's arguments (which makes sense since we're taling about the same things - a "line of text", an encoding name and a value that describes how to handle encoding errors), but you pass litteral (unnamed) values or variables with just any arbitrary name, it would work just the same.
A function is a (mostly) self-contained unit of work. It usually uses some internal ("local") variables which only it can sees and that only exists during the function's execution. It also usually takes arguments: values that must be passed by the caller, and are bound to the matching arguments names (in our case, the first value will be bound to the line name, the second to the encoding name etc). The local names (local variables and arguments) are totally unrelated to the name under which those values are known in the caller's scope (if even they are bound to names - as I said you can as well pass literal values or other anonymous objects)
When I attempt to call the print_line() argument without passing arguments Python is outputting "print_line() missing 3 required positional arguments: 'line', 'encoding', and 'errors'"
Yes, of course. The function needs three arguments, so you have to pass three arguments, plain and simple. The fact that you have three local variables by the same name in the caller scope (the main function) will not automagically "fill in" those arguments for you, and the print_line function knows absolutely nothing about it's caller's scope anyway.
Note that the terms "positional" and "named" arguments mostly to refer to how you pass the arguments themselves - by position (the default), which I already explained above, or by name (ie print_line(line="hello", errors="ignore", encoding="utf-8"), which allow you to pass the arguments in a different order, or from a dictionnary you've built dynamically etc etc, but you first need to understand the concepts of function, arguments and scope before going further...
I strongly suggest that you do the official tutorial, which does have a chapter on functions - it's mostly intended at peoples with already some programming experience (it's not a CS101 course) but it's still well explained.
For example, I'd like to do something like: greet(,'hola'), where greet is:
def greet(person='stranger', greeting='hello')
This would help greatly for testing while writing code
Upon calling a function you can use the variable names to make it even more clear what variable will assume which value. At the same time, if defaults are provided in the function definition, skipping variables when calling the function does not raise any errors. So, in short you can just do this:
def greet(person='stranger', greeting='hello')
print('{} {}'.format(greeting, person))
return
greet(greeting='hola') # same as greet(person='stranger', greeting='hola')
# returns 'hola stranger'
Note that, as I said above this would not work if for example your function definition was like this:
def greet(person, greeting)
print('{} {}'.format(greeting, person))
return
Since in this case, Python would complain saying that it does not know what to do with person; no default is supplied..
And by the way, the problem you are describing is most likely the very reason defaults are used in the first place
Without knowing the other parameters, and only knowing that the parameter you want to change is in second position you could use the inspect module to get function signature & associated default values.
Then make a copy of the default values list and change the one at the index you want:
import inspect
def greet(person='stranger', greeting='hello'):
print(person,greeting)
argspec = inspect.getargspec(greet)
defaults = list(argspec.defaults)
defaults[1] = "hola" # change second default parameter
greet(**dict(zip(argspec.args,defaults)))
Assuming that all parameters have default values (else it shifts the lists an that fails) that prints:
stranger hola