Imagine I have a function that looks like this :
myFunction(arg, arg, kwarg, arg, arg, kwarg, etc...):
Where arg is an *arg and kwarg is a *kwarg. Before now, my function looked like myFunction(*args): and I was using just a long list of *args and I would just pass in a big list like this
myFunction(*bigList):
The bigList looked like = [[1,2,3],[4,5,6],'hello',[1,3,5],[2,4,6],'world',etc...]
But, now I need to have a kwarg every third argument. So, in my mind, the list "looks" like this now:
newBigList = [[1,2,3],[4,5,6],word='hello',[1,3,5],[2,4,6],word='world',etc...]
So, there are two questions to make this work.
1) Can I construct a list with a string for a kwarg without the function reading it in as an actual argument? Could the word(s) in the newBigList be strings?
2) Can you alternate kwargs and args? I know that kwargs are usually done with dictionaries. Is it even possible to use both by alternating?
As always, if anyone knows a better way of doing this, I would be happy to change the way I'm going about it.
EDIT Here's the method. Its a matplotlib method that plots a polygon (or a bunch of polygons):
plot([x1], [y1], color=(RBG tuple), [x2], [y2], color=(RGB tuple), etc...)
Where [x1] is a list of x values for the first polygon, [y1] is a list of y values for the first polygon, and so on.
The problem is, to use RBG values for the color argument, I need to include the color keyword. To further complicate matters, I am generating a random tuple using the random.random() module.
So, I have this list of lists of x values for all the polygons, a list of lists of y values for all my polygons, and a list of tuples of random RBG colors. They look something like this:
x = [[1,2,3], [4,5,6], [7,8,9]]
y = [[0,9,8], [7,6,5], [4,3,2]]
colors = [(.45, .645, .875), (.456, .651, .194), (.813, .712, .989)]
So, there are three polygons to plot. What I had been doing before I could do keywords was zip them all up into one tuple and use it like this.
list_of_tuples = zip(x, y, colors)
denormalized = [x for tup in list_of_tuples for x in tup]
plot.plot(*denormalized)
But, now I need those keywords. And I'm definitely happy to provide more information if needed. Thanks
The function signature doesn't work the way you think it does. Keyword arguments to matplotlib's plot function apply to all the lines you specify:
If you make multiple lines with one plot command, the kwargs apply to all those lines, e.g.:
plot(x1, y1, x2, y2, antialised=False)
If you want to specify individual colors to each line, you need to turn them into format strings that you can pass as every third positional argument. Perhaps you can format them as HTML style hex codes: #RRGGBB
Or alternately, call plot once per line and pass your color tuple just once as a keyword argument.
Short answer: No.
Longer answer: Depends on exactly what you are trying to do. The Python interface cannot accept the signature you want, so what is the function, and what are you actually trying to do?
There are several reasons that prevents you to do what you are trying to do:
You can specify a keyword only once in a function call, hence color=something, ..., color=other raises an exception
You cannot mix keyword arguments and positionals, so x1, y1, color=something, x2 is an error.
Even if this worked as you expected, there's still matplotlibs documentation that states:
If you make multiple lines with one plot command, the kwargs apply to
all those lines
I.e. you cannot use color= for only one of the lines, or once for each line. It's a "global" property. You have to use the other ways of providing line colors if you want to specify a different color for each line.
I believe, by your question, that you do not have clear how positional and keyword arguments work so I'll try to give you a clue in this regard.
First of all, there are different kind of parameters. I shall introduce an example to explain the differences:
def a_function(pos_kw1, pos_kw2, *args, kw_only)
This function has:
Two parameters pos_kw1, pos_kw2 which can be assigned both by a positional argument or a keyword argument
A parameter *args that can be specified only with positional arguments
A parameter kw_only that can be specified only with a keyword argument
Note: default values have nothing to do with being keyword parameters. They simply make the parameter not required.
To understand the mechanics of argument passing you can think as (although it's not strictly true) if when python performs a function call (e.g.):
a_function(1, 2, *'abc', kw_only=7)
It first collects all positional arguments into a tuple. In the case above the resultant tuple would be pos_args = (1, 2, 'a', 'b', 'c'), then collects all keyword arguments into a dict, in this case kw_args = {'kw_only': 7}, afterwards, it calls the function doing:
a_function(*pos_args, **kw_args)
Note: since dicts are not ordered the order of the keywords doesn't matter.
In your question you wanted to do something like:
plot(x, y, color=X, x2, y2, color=Y, ...)
Since the call is actually using *pos_args and **kw_args the function:
Doesn't know that color=X was specified right after y.
Doesn't know that color=Y was specified right after y2.
Doesn't know that color=X was specified before color=Y.
Corollary: you cannot specify the same argument more than once since python has no way to know which occurrence should be assigned to which parameter. Also when defining the function you simply couldn't use two parameters with the same name.
(And no, python does not automatically build a list of values or similar. It simply raises an error.)
You can also think that python first expands *pos_args without taking keyword arguments into account, and after that it expands **kw_args. If you think in this terms you can clearly understand that a function call such as:
# naive intent: assign pos_kw1 via keyword and pos_kw2 via positional
# assuming python will skip positional that were already provided as keyword args
a_function(1, pos_kw1=2)
# or even:
a_function(pos_kw1=2, 1) # hoping order matters
doesn't have any sense because the 1 is assigned to pos_kw1 via positional arguments, and when expanding the keyword arguments it would be reassigned.
Explained in an other way, in the call a_function(*pos_args, **kw_args) the *pos_args is a simple tuple-unpacking operation, equivalent to:
pos_kw1, pos_kw2, *args = pos_args
(in python2 you cannot use the *, but that's how the *args parameters work more or less).
Tuple-unpacking doesn't skip elements: it simply assign to consecutive elements of the tuple, and so do function calls: there is no check if a positional argument was already passed via keyword and eventually it's skipped. They are simply assigned, blindly.
Due to these restrictions it doesn't make any sense to allow function calls where positionals appear after keyword arguments hence you cannot do something like:
plot(x, y, color=X, x2, ...)
Allowing such function calls would only trick people making them think that order matters for keywords or that arguments could be skipped when unpacking etc. so Python simply raises an error and avoids this kind of ambiguity.
Related
I'm new with Python language and a I'm a little bit frustrated.
Till today, I thought that passing parameter names in a function call was not mandatory. For example, if you have the following function:
def computeRectangleArea(width=7, height=8):
return width * height
I thought that you can call like this computeRectangleArea(width=7,height=8) only to make clearer the meaning of the parameters, but actually keywords of input arguments were not needed, so you can call the same function in this way also: computeRectangleArea(7, 8)
Today, while using openpyxl.styles.PatternFill(), I realized that fill_type keyword is a necessary when calling this function.
Suppose that you call the function in this way: openpyxl.styles.PatternFill('FFFFFF','FFFFFF','solid'), then the interpretation of the input parameter will be wrong.
I have some experience with OOP language (Java, C#) and these thing doesn't exist there.
It seems an inconsistent behaviour to me that some parameter names (like start_color and end_color in the example above) are optional, while others (like fill_type) must be specified before their values.
Can someone explain me why this apparently strange policy? In addition, I will be glad if someone can point me out some useful resource to understand the way it is implemented.
Positional and keyword parameters work just as they do in the languages you know better. You need to go to the documentation of the method you're using and look at the signature. For creating a PatternFill object, go to the class's __init__ method.
class PatternFill(Fill):
def __init__(self, patternType=None, fgColor=Color(), bgColor=Color(),
fill_type=None, start_color=None, end_color=None):
You may specify arguments without the keyword as long as you supply them all in order, without skipping any. For instance, your failing call can be legally given as:
PatternFill(None, 'FFFFFF', 'FFFFFF', 'solid')
These will match the first four parameters. Any time you supply an argument out of order, then you must supply the keyword for that argument and all later arguments in that invocation. For instance, with the above call, if you want to let they style default to None, then you must supply the keywords for the three arguments you do supply. If you simply omit the None, then the parser still tries to match them up sequentially from the front:
patternType <= 'FFFFFF'
fgColor <= 'FFFFFF'
bgColor <= 'solid'
... and your call fails to pass parsing.
Does that clear things up a little?
Can someone explain me why we need this "headache"…
For your specific example, it doesn't appear that there are any keyword-only parameters. Rather, you're trying to pass arguments for the first, second, and fourth parameters, without having to pass an argument for the one in between that you don't care about.
In other words, it's not a headache at all. It's a convenience (and sanity check) you could quite easily ignore—but probably don't want to.
Instead of this:
PatternFill('FFFFFF', 'FFFFFF', fill_type='solid')
… you could write this:
PatternFill('FFFFFF', 'FFFFFF', Color(), 'solid')
… but in order to know that's what you'd need to send, you need to read the source or docs to see the whole parameter list, and see what the default values are for the parameters you want to skip over, and explicitly add them to your call.
I doubt anyone would find that better.
Also, as multiple people pointed out in comments, this is pretty much exactly how named arguments work in C#.
And this class is, accidentally, a great example of why Python actually does allow keyword-only parameters, even though they aren't being used here.
The fact that you can write PatternFill('FFFFFF', 'FFFFFF', 'solid') and not get a TypeError for bad arguments to PatternFill, but instead a mysterious error about 'solid' not working as a color, is hardly a good thing. And (at least without type hinting annotations, which this type doesn't have) there's no way your IDE or any other tool could catch that mistake.
And, in fact, by not using keywords, you've even gotten the initial arguments wrong, without realizing it. You almost certainly wanted to do this:
PatternFile(None, 'FFFFFF', 'FFFFFF')
… but you got away with this without a visible error:
PatternFile('FFFFFF', 'FFFFFF')
… which means you're passing your foreground color as a pattern type and your background color as a foreground color and leaving the default background color.
That could be solved by making all or most parameters keyword-only. But without keyword-only params, the only option would be **kwargs, and that tradeoff is usually not worth it.
Quoting from the Rationale of PEP 3102, the proposal that added keyword-only parameters to the language:
There are often cases where it is desirable for a function to take a variable number of arguments. The Python language supports this using the 'varargs' syntax (*name), which specifies that any 'left over' arguments be passed into the varargs parameter as a tuple.
One limitation on this is that currently, all of the regular argument slots must be filled before the vararg slot can be.
This is not always desirable. One can easily envision a function which takes a variable number of arguments, but also takes one or more 'options' in the form of keyword arguments. Currently, the only way to do this is to define both a varargs argument, and a 'keywords' argument (**kwargs), and then manually extract the desired keywords from the dictionary.
If it isn't obvious why using *args and **kwargs isn't good enough:
The actual signature of the function is not visible when looking at the function definition in the source, or the inline help, or auto-generated docs.
The signature is also not available to dynamic reflective code using the inspect module or similar.
The signature is also not available to static reflective code—like that used by many IDEs to do completion and suggestions.
The implementation of the function is less clear, because at best it's half boilerplate for extracting and testing the parameters, and at worst the args and kwargs access are scattered throughout the body of the function.
For an example of what this feature allows, consider the builtin print function, which you can call like this:
print(x, y, z, sep=', ')
This works because print is defined like this:
print(*objects, sep=' ', end='\n', file=sys.stdout, flush=False):
If it weren't for keyword arguments, there'd be no way to pass that sep as something different from the actual values to print.
You could force the user to pass all of the objects in a tuple instead of as separate arguments, but that would be a lot less friendly—and even if you did that, there'd be no way to pass flush without passing values for all of sep, end, and file.
And, even with keyword arguments, if it weren't for keyword-only parameters, the function signature would have to look like this:
print(*objects, **kwargs):
… which would make it a lot harder to figure out what keyword arguments you could pass.
I'm working on a project that almost everywhere arguments are passed by key. There are functions with positional params only, with keyword (default value) params or mix of both. For example the following function:
def complete_task(activity_task, message=None, data=None):
pass
This function in the current code would be called like this:
complete_task(activity_task=activity_task, message="My massage", data=task_data)
For me there is no point to name arguments whose name is obvious by the context of the function execution / by the variable names. I would call it like this:
complete_task(activity_task, "My message", task_data)
In certain cases where it's not clear what the a call argument is from the context, or inferred from the variable names, I might do:
complete_task(activity_task, message="success", task_data=json_dump)
So this got me wondering if there is a convention or "pythonic" way to call functions with positional/keyword params, when there is no need to rearrange method arguments or use default values for some of the keyword params.
The usual rules of thumb I follow are:
Booleans, particularly boolean literals, should always be passed by keyword unless it is really obvious what they mean. This is important enough that I will often make booleans keyword-only when writing my own functions. If you have a boolean parameter, your function may want to be split into two smaller functions, particularly if it takes the overall structure of if boolean_parameter: do_something(); else: do_something_entirely_different().
If a function takes a lot of optional parameters (more than ~3 including required parameters), then the optionals should usually be passed by keyword. But if you have a lot of parameters, your function may want to be refactored into multiple smaller functions.
If a function takes multiple parameters of the same type, they probably want to be passed as keyword arguments unless order is completely obvious from context (e.g. src comes before dest).
Most of the time, keyword arguments are not wrong. If you have a case where positional arguments are confusing, you should use keyword arguments without a second thought. With the possible exception of simple one parameter functions, keyword arguments will not make your code any harder to read.
Python has 2 types of arguments1. positional and keyword (aka default). The waters get a little muddy because positional arguments can be called by keyword and keyword arguments can be called by position...
def foo(a, b=1):
print(a, b)
foo(1, 2)
foo(a=1, b=2)
With that said, I think that the names of the types of arguments should indicate how you should (typically) use them. Most of the time, I see positional arguments called by position and keyword arguments called by keyword. So, if you're looking for a general rule of thumb, I'd advise that you make the function call mimic the signature. In the case of our above foo function, I'd call it like this:
foo(1, b=2)
I think that one reason to follow this advice is because (most of the time), people expect keyword arguments to be passed via keyword. So it isn't uncommon for someone to later add a keyword:
def foo(a, aa='1', b=2):
print(a, aa, b)
If you were calling the function using only positional arguments, you'd now be passing a value to a different parameter than you were before. However, keyword arguments don't care what order you pass them, so you should still be all set.
So far, so good. But what rules should you use when you're creating a function? How do you know whether to make an argument a default argument or a positional argument? That's a reasonable question -- And it's hard to find a good rule of thumb. The rules of thumb I use are as follows:
Be consistent with the rest of the project -- It's hard to get it right if you're doing something different than the rest of the surrounding code.
Make an argument a default argument if (and only if) it is possible to supply a reasonable default. If the function will fail if the user doesn't supply a particular argument (because there is no good default), then it should be positional.
1Python3.x also has keyword only arguments. Those don't give you a choice, so I don't know that they add too much to the discussion here :-) -- Though I don't know that I've seen their use out in the wild too much.
I've encountered a problem in a project where it may be useful to be able to pass a large number (in the tens, not the hundreds) of arguments to a single "Write once, use many times" function in Python. The issue is, I'm not really sure what the best ay is to handle a large block of functions like that - just pass them all in as a single dictionary and unpack that dictionary inside the function, or is there a more efficient/pythonic way of achieving the same effect.
Depending on exactly what you are doing, you can pass in arbitrary parameters to Python functions in one of two standard ways. The first is to pass them as a tuple (i.e. based on location in the function call). The second is to pass them as key-value pairs, stored in a map in the function definition. If you wanted to be able to differentiate the arguments using keys, you would call the function using arguments of the form key=value and retrieve them from a map parameter (declared with ** prefix) in the function definition. This parameter is normally called kwargs by convention. The other way to pass an arbitrary number of parameters is to pass them as a tuple. Python will wrap the arguments in a tuple automatically if you declare it with the * prefix. This parameter is usually called args by convention. You can of course use both of these in some combination along with other named arguments as desired.
Let's say now I have a function:
def func(x, p): return p[0] * x ** 2 + p[1] * x + p[2]
And now, I can get the information about the function using inspect:
import inspect
args, varargs, varkw, defaults = inspect.getargspec(func)
But I only know I have two arguments, instead of the information on each argument (whether it's a scalar or something else).
Just making sure - theoretically, is there any way that I can know the minimum length of the tuple p used in the function?
Thank you!
You cannot enforce neither the type nor the value of the argument that is passed to your function.
The only thing you can do is annotate your functions in python 3: annotations, But even that doesn't prevent the user from passing in something invalid.
Note: actually you can enforce by checking directly in the function or with a decorator, but that doesn't help in your case.
The answer is no.
Firstly you can't assume the type (let alone the size of the argument).
Secondly, there is no way to tell the length, because it's supposed to be arbitrary and the function may do nothing about the input at all.
if you do want something similar, try to use *l for variable-length arguments. similarly there is **d for arbitrary map (named arguments).
Is there a good rule of thumb as to when you should prefer varargs function signatures in your API over passing an iterable to a function? ("varargs" being short for "variadic" or "variable-number-of-arguments"; i.e. *args)
For example, os.path.join has a vararg signature:
os.path.join(first_component, *rest) -> str
Whereas min allows either:
min(iterable[, key=func]) -> val
min(a, b, c, ...[, key=func]) -> val
Whereas any/all only permit an iterable:
any(iterable) -> bool
Consider using varargs when you expect your users to specify the list of arguments as code at the callsite or having a single value is the common case. When you expect your users to get the arguments from somewhere else, don't use varargs. When in doubt, err on the side of not using varargs.
Using your examples, the most common usecase for os.path.join is to have a path prefix and append a filename/relative path onto it, so the call usually looks like os.path.join(prefix, some_file). On the other hand, any() is usually used to process a list of data, when you know all the elements you don't use any([a,b,c]), you use a or b or c.
My rule of thumb is to use it when you might often switch between passing one and multiple parameters. Instead of having two functions (some GUI code for example):
def enable_tab(tab_name)
def enable_tabs(tabs_list)
or even worse, having just one function
def enable_tabs(tabs_list)
and using it as enable_tabls(['tab1']), I tend to use just: def enable_tabs(*tabs). Although, seeing something like enable_tabs('tab1') looks kind of wrong (because of the plural), I prefer it over the alternatives.
You should use it when your parameter list is variable.
Yeah, I know the answer is kinda daft, but it's true. Maybe your question was a bit diffuse. :-)
Default arguments, like min() above is more useful when you either want to different behaviours (like min() above) or when you simply don't want to force the caller to send in all parameters.
The *arg is for when you have a variable list of arguments of the same type. Joining is a typical example. You can replace it with an argument that takes a list as well.
**kw is for when you have many arguments of different types, where each argument also is connected to a name. A typical example is when you want a generic function for handling form submission or similar.
They are completely different interfaces.
In one case, you have one parameter, in the other you have many.
any(1, 2, 3)
TypeError: any() takes exactly one argument (3 given)
os.path.join("1", "2", "3")
'1\\2\\3'
It really depends on what you want to emphasize: any works over a list (well, sort of), while os.path.join works over a set of strings.
Therefore, in the first case you request a list; in the second, you request directly the strings.
In other terms, the expressiveness of the interface should be the main guideline for choosing the way parameters should be passed.