In Python, varargs collection seems to work quite differently from how sequence unpacking works in assignment statements. I'm trying to understand the reason for this potentially confusing difference. I'm sure there is a good reason, but what is it?
# Example 1 with assignment statement
a, *b, c = 1,2,3,4
print(b) # [2, 3]
# Example 1 with function call
def foo(a, *b, c):
print(b)
foo(1,2,3,4)
The function call results in the following error:
Traceback (most recent call last):
File "<pyshell#309>", line 1, in <module>
foo(1,2,3,4)
TypeError: foo() missing 1 required keyword-only argument: 'c'
Question 1: Why is b not assigned similar to how it is done in the assignment statement?
# Example 2 with function call
def foo(*x):
print(x)
print(type(x))
foo(1,2,3) # (1, 2, 3) <class 'tuple'>
# Example 2 with assignment statement
a, *b, c = 1,2,3,4,5
print(b) # [2, 3, 4]
print(type(b)) # <class 'list'>
Question 2: Why the difference in type (list vs tuple)?
When used in an assignment, Python will try to make *b match to whatever it needs in order to make the assignment work (this is new in Python 3, see PEP 3132). These are both valid:
a, *b, c = 1,4
print(b) # []
a, *b, c = 1,2,3,4,5
print(b) # [2, 3, 4]
When used in a function, if *b is the second parameter in the function definition, it will match with the second to last arguments in the function call, if there are any. It's used when you want your function to accept a variable number of parameters. Some examples:
def foo(a, *b):
print(b)
foo(1) # ()
foo(1,2,3,4,5) # (2,3,4,5)
I recommend you read the following:
PEP 3132 -- Extended Iterable Unpacking
Understanding the asterisk(*) of Python.
Unpacking Argument Lists.
Keyword Arguments
On the difference between lists and tuples, the big one is mutability. Lists are mutable, tuples aren't. That means that this works:
myList = [1, 2, 3]
myList[1] = 4
print(myList) # [1, 4, 3]
And this doesn't:
myTuple = (1, 2, 3)
myTuple[1] = 4 # TypeError: 'tuple' object does not support item assignment
The reason why b is a list in this case:
a, *b, c = 1,2,3,4,5
print(b) # [2, 3, 4]
And not a tuple (as is the case when using *args in a function), is because you'll probably want to do something with b after the assignment, so it's better to make it a list since lists are mutable. Making it a tuple instead of a list is one of the possible changes that were considered before this was accepted as a feature, as discussed in PEP 3132:
Make the starred target a tuple instead of a list. This would be
consistent with a function's *args, but make further processing of the
result harder.
Related
This question already has answers here:
What do the * (star) and ** (double star) operators mean in a function call?
(4 answers)
Closed 2 years ago.
I'm using itertools.chain to "flatten" a list of lists in this fashion:
uniqueCrossTabs = list(itertools.chain(*uniqueCrossTabs))
how is this different than saying:
uniqueCrossTabs = list(itertools.chain(uniqueCrossTabs))
* is the "splat" operator: It takes an iterable like a list as input, and expands it into actual positional arguments in the function call.
So if uniqueCrossTabs were [[1, 2], [3, 4]], then itertools.chain(*uniqueCrossTabs) is the same as saying itertools.chain([1, 2], [3, 4])
This is obviously different from passing in just uniqueCrossTabs. In your case, you have a list of lists that you wish to flatten; what itertools.chain() does is return an iterator over the concatenation of all the positional arguments you pass to it, where each positional argument is iterable in its own right.
In other words, you want to pass each list in uniqueCrossTabs as an argument to chain(), which will chain them together, but you don't have the lists in separate variables, so you use the * operator to expand the list of lists into several list arguments.
chain.from_iterable() is better-suited for this operation, as it assumes a single iterable of iterables to begin with. Your code then becomes simply:
uniqueCrossTabs = list(itertools.chain.from_iterable(uniqueCrossTabs))
It splits the sequence into separate arguments for the function call.
>>> def foo(a, b=None, c=None):
... print a, b, c
...
>>> foo([1, 2, 3])
[1, 2, 3] None None
>>> foo(*[1, 2, 3])
1 2 3
>>> def bar(*a):
... print a
...
>>> bar([1, 2, 3])
([1, 2, 3],)
>>> bar(*[1, 2, 3])
(1, 2, 3)
Just an alternative way of explaining the concept/using it.
import random
def arbitrary():
return [x for x in range(1, random.randint(3,10))]
a, b, *rest = arbitrary()
# a = 1
# b = 2
# rest = [3,4,5]
This question already has answers here:
What does ** (double star/asterisk) and * (star/asterisk) do for parameters?
(25 answers)
Closed 2 years ago.
Here is one easy math function in Jupyter using Python 3
def sum(*formulation):
ans = 0
for i in formulation:
ans += i
return ans
If I want to try this function, I write down like this:
sum(1,2,3,4)
The output will be
10
My question is what is * mean in sum(*formulation)?
Because if I don't use *, I get an error.
The "*" and then "**" notation are called "packing" and "unpacking". The main idea is that if you unpack objects, the they are removed from their list/dict and if you pack objects, then they are placed into a list/dict. For example,
x = [*[1,2,3],4]
print(x)
Here I have "unpacked" the [1,2,3] into the list for "x". Hence, x is now [1,2,3,4]. Here is another example,
d1 = {'x':7}
d2 = {'y':10}
d3 = {**d1,**d2}
Here I have dictionary "unpacked" the first two dictionaries into the third one. Here is another example:
def func(*args):
print(args)
func(1,2,3,4,5)
Here the 1,2,3,4,5 are not in a list, hence they will be "packed" into a list called args in the func.
That is called a starred expression. In the argument list of a function, this means that all other supplied positional arguments (that are not caught by preceding positional arguments) will be "packed" into the starred variable as a list.
So
def function(*arguments):
print(arguments)
function(1, 2, 3)
will return
[1, 2, 3]
Note that it has different behaviour in other contexts in which it is usually used to "unpack" lists or other iterables. The Searchwords for that would be "starred", "packing" and "unpacking".
A good mnemonic for unpacking is that they remove the list brackets
a, b, c = *[1, 2, 3] #equivalent to
a, b, c = 1, 2, 3
And for packing like a regex wildcard
def function(*arguments):
pass
def function(zero, or_, more, arguments):
pass
head, *everything_in_between, tail = [1, 2, 3, 4, 5, 6]
It means that the function takes zero or more arguments and the passed arguments would be collected in a list called formulation.
For example, when you call sum(1, 2, 3, 4), formation would end up being [1, 2, 3, 4].
Another similar but different usage of * that you might come across is when calling the function. Say you have a function defined as def add(a, b), and you have a list l = [1, 2], when you call add(*l) it means to unpack l and is equivalent to add(l[0], l[1]).
Let's say I create a list in the following way:
a = [[0, 1], [2, 3]]
If I execute the following for loop:
for b in a:
print(b)
I get
[0, 1]
[2, 3]
which makes sense to me because I am looking for elements of a which, in that case, are sub-lists. However, if I execute:
for b, c in a:
print(b, c)
I get
0 1
2 3
I would have expected to get
[0, 1] [2, 3]
Is there an obvious reason why Python is unpacking the list this way? I mean, why Python would suddenly give me elements of sub-elements of my list when I am requesting objects in the list?
You're unpacking to the integers here:
>>> for b, c in a:
... print(b, c)
... print(type(b), type(c))
...
0 1
<class 'int'> <class 'int'>
2 3
<class 'int'> <class 'int'>
Put another way, compare the following two:
>>> b, c = a; print(b, c)
[0, 1] [2, 3]
>>> b, c = a[0]; print(b,c)
0 1
When you specified:
for b, c in a:
print(b, c)
You could read that as: for each element in a, unpack into the tuple "b, c"
It's helpful to understand the Assignment Statement being used here. Relevant part here:
If the target list is a single target with no trailing comma,
optionally in parentheses, the object is assigned to that target.
Else: The object must be an iterable with the same number of items as
there are targets in the target list, and the items are assigned, from
left to right, to the corresponding targets.
The for loop essentially boils down to this:
for (object) in (iterable):
Now let's see what objects are in your iterable:
[0, 1] # object1: list
[2, 3] # object2: list
But when you use the Assignment Statement, the "Else" clause applies.
for (obj1, obj2 of object) in (iterable):
Which treats the object as an inner iterable, and unpack each to the inner objects, i.e. the ints within:
0 1 # obj1 of object1: int, obj2 of object1: int
2 3 # obj1 of object2: int, obj2 of object2: int
What you're experiencing is unpacking assignment, which assigns the elements of an iterable to multiple variables. To get your expected behavior, do the following:
b, c = a
print(b, c)
for b, c in a: iterates through a the same way as for b in a:, but since each element is a 2-element iterable, its own elements get unpacked to b and c. See what happens when you do this:
a = [[0, 1], [2, 3, 4]]
for b, c in a:
print(b, c)
You would get an error, because a[1] has more elements than there are variables to assign to.
This question already has answers here:
How are Python in-place operator functions different than the standard operator functions?
(2 answers)
Closed 5 years ago.
Please look into the code below
def double(arg):
print("Before: ", arg)
arg = arg * 2
print("After: ", arg)
I was studying Head first Python, and I came to this section where they were discussing about pass by value and pass by reference.
If we invoke above function with a list as argument such as:
num = [1,2,3]
double(num)
print(num)
The output is :-
Before: [1, 2, 3]
After: [1, 2, 3, 1, 2, 3]
[1, 2, 3]
Which seems fine, considering the fact that arg in function double is a new object reference. So the value of num did not change.
But if I use compound operators instead of assignment operators, things work differently as shown:
def double(arg):
print("Before: ", arg)
arg *= 2
print("After: ", arg)
num = [1,2,3]
double(num)
print(num)
The output that I get for this is:
Before: [1, 2, 3]
After: [1, 2, 3, 1, 2, 3]
[1, 2, 3, 1, 2, 3]
Why does this happen? I used to think a*=2 and a = a*2 are same. But what's going on in here?
Thanks
This is a difference between mutable and immutable objects. A mutable object can implement obj *= something by actually modifying the object in place; an immutable object can only return a new object with the updated value (in which case the result is identical to obj = obj * something). The compound assignment statements can handle either case, it's entirely up to the object's implementation.
a *= 2 modifies the structure itself (a) whereas a = a*2 reassigns a as a new variable.
(see this question)
This question already has answers here:
Why does += behave unexpectedly on lists?
(9 answers)
Closed 9 years ago.
>>> c = [1, 2, 3]
>>> print(c, id(c))
[1, 2, 3] 43955984
>>> c += c
>>> print(c, id(c))
[1, 2, 3, 1, 2, 3] 43955984
>>> del c
>>> c = [1, 2, 3]
>>> print(c, id(c))
[1, 2, 3] 44023976
>>> c = c + c
>>> print(c, id(c))
[1, 2, 3, 1, 2, 3] 26564048
What's the difference? are += and + not supposed to be merely syntactic sugar?
docs explain it very well, I think:
__iadd__(), etc.
These methods are called to implement the augmented arithmetic assignments (+=, -=, *=, /=, //=, %=, **=, <<=, >>=, &=, ^=, |=). These methods should attempt to do the operation in-place (modifying self) and return the result (which could be, but does not have to be, self). If a specific method is not defined, the augmented assignment falls back to the normal methods. For instance, to execute the statement x += y, where x is an instance of a class that has an __iadd__() method, x.__iadd__(y) is called.
+= are designed to implement in-place modification. in case of simple addition, new object created and it's labelled using already-used name (c).
Also, you'd notice that such behaviour of += operator only possible because of mutable nature of lists. Integers - an immutable type - won't produce the same result:
>>> c = 3
>>> print(c, id(c))
3 505389080
>>> c += c
>>> print(c, id(c))
6 505389128
They are not same
c += c append a copy of content of c to c itself
c = c + c create new object with c + c
For
foo = []
foo+=foo is syntactic sugar for foo.extend(foo) (and not foo = foo + foo)
In the first case, you're just appending members of a list into another (and not creating a new one).
The id changes in the second case because a new list is created by adding two lists. It's incidental that both are the same and the result is being bound to the same identifier than once pointed to them.
If you rephrase this question with different lists (and not c itself), it will probably become clearer.
The += operator appends the second list to the first, but the modification is in-place, so the ID remains the same.
When you use + , a new list is created, and the final "c" is a new list, so it has a different ID.
The end result is the same for both operations though.