Related
Suppose I have function with list parameter, and inside its body I want to modify passed list, by copying elements of an array to the list:
def function1 (list_arg):
a = function2() #function2 returns an array of numbers
list_arg = list(a)
list1 = [0] * 5
function1(list1)
list1
[0,0,0,0,0]
When doing it like this, it doesn't work. After executing function1(list1), list1 remains unchanged. So, how to make function1 return list1 with the same elements (numbers) as array a?
If you assign something to the variable list_arg, it will from then on point to the new value. The value it pointed to before that assignment (your original list) will stay unchanged.
If you, instead, assign something to elements of that list, this will change the original list:
list_arg[:] = list(a)
This will make your code work as you wanted it.
But keep in mind that in-place changes are hard to understand and probably can confuse the next developer who has to maintain your code.
What I think you are asking is why after calling f(a), when f re-assigns the a you passed, a is still the "old" a you passed.
The reason for this is how Python treats variables and pass them to functions. They are passed by reference, but the reference is passed by value (meaning that a copy is created). This means that the reference you have inside f is actually a copy of the reference you passed. This again implies that if you reassign the variable inside the function. It is a local variable existing only inside the function; re-assigning it won't change anything in outside scopes.
Now, if you rather than reassigning the local variable/reference inside f (which won't work, since it's a copy) perform mutable operations on it, such as append(), the list you pass will have changed after f is done.
See also the question How do I pass a variable by reference? which treats the problem and possible solutions in further detail.
TL;DR: Reassigning a variable inside a function won't change the variable you passed as an argument outside the function. Performing mutable operations on the variable, however, will change it.
You can operate on the list to change its values (eg, append something to it, or set its values) but changes will be reflected outside of the function only if you operate on the reference to the passed in object:
def function1 (list_arg):
list_arg.append(5)
If you have questions when doing this, print out the ids:
def function1 (list_arg):
print 1, id(list_arg)
list_arg[:] = ["a", "b", "c"]
print 2, id(list_arg)
list_arg = range(10)
print 3, id(list_arg)
x = [1,2,3]
function1(x)
print x
prints:
1 4348413856
2 4348413856
3 4348411984
['a', 'b', 'c']
That is, x is changed in place, but assigning to the function's local variable list_arg has no impact on x, because is then just assigns a different object to list_arg.
You're changing a reference to a local variable. When you pass in list_arg this way:
def function1 (list_arg):
list_arg is a reference to an underlying list object. When you do this:
list_arg = list(a)
You're changing what list_arg means within the function. Since the function exits right after that, list_arg = list(a) has no effect.
If you want to actually change the reference to the list you have to do assign it to the result of the function.
def function1 ():
'a = some array'
return list(a)
list1 = [0] * 5
list1 = function1()
Or you could modify the contents of the list without changing the reference.
def function1(list_arg):
del list_arg[:] # Clears the array
'a = some array'
list_arg.extend(a)
When you define a function in Python with an array parameter, what is the scope of that parameter?
This example is taken from the Python tutorial:
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
Prints:
[1]
[1, 2]
[1, 2, 3]
I'm not quite sure if I understand what's happening here. Does this mean that the scope of the array is outside of the function? Why does the array remember its values from call to call? Coming from other languages, I would expect this behavior only if the variable was static. Otherwise it seems it should be reset each time. And actually, when I tried the following:
def f(a):
L = []
L.append(a)
return L
I got the behavior I expected (the array was reset on each call).
So it seems to me that I just need the line def f(a, L=[]): explained - what is the scope of the L variable?
The scope is as you would expect.
The perhaps surprising thing is that the default value is only calculated once and reused, so each time you call the function you get the same list, not a new list initialized to [].
The list is stored in f.__defaults__ (or f.func_defaults in Python 2.)
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
print f.__defaults__
f.__defaults__ = (['foo'],) # Don't do this!
print f(4)
Result:
[1]
[1, 2]
[1, 2, 3]
([1, 2, 3],)
['foo', 4]
The scope of the L variable is behaving as you expect.
The "problem" is with the list you're creating with []. Python does not create a new list each time you call the function. L gets assigned the same list each time you call which is why the function "remembers" previous calls.
So in effect this is what you have:
mylist = []
def f(a, L=mylist):
L.append(a)
return L
The Python Tutorial puts it this way:
The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes.
and suggests the following way to code the expected behaviour:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
There's even less "magic" than you might suspect. This is equivalent to
m = []
def f(a, L=m):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
m is only created once.
Say you have the following code:
def func(a=[]):
a.append(1)
print("A:", a)
func()
func()
func()
You can use python's indentation to help you understand what's going on. Everything that is flush to the left margin is executed when the file gets executed. Everything that's indented is compiled into a code object which gets executed when func() is called. So the function is defined and its default arguments set once, when the program gets executed (because the def statement is flush left).
What it does with the default arguments is an interesting issue. In python 3, it puts most of the information about a function in two places: func.__code__ and func.__defaults__. In python 2, func.__code__ was func.func_code func.__defaults__ was func.func_defaults. Later versions of python 2, including 2.6 have both sets of names, to aid the transition from python 2 to python 3. I will use the more modern __code__ and __defaults__. If you're stuck on an older python, the concepts are the same; just the names differ.
The default values are stored in func.__defaults__, and retrieved each time the function is called.
Thus when you define the function above, the body of the function gets compiled and stored in variables under __code__, to be executed later, and the default arguments get stored in __defaults__. When you call the function, it uses the values in __defaults__. If those values get modified for any reason, it only has the modified version available to use.
Play around defining different functions in the interactive interpreter, and see what you can figure out about how python creates and uses functions.
The explaination is given in answers to this question. To sum it up here:
Functions in Python are a kind of object. Because they are a kind of object, they act like objects when instantiated. A function, if defined with a mutable attribute as a default argument, is exactly the same as a class with a static attribute that is a mutable list.
Lennart Regebro has a good explanation and the answer to the question by Roberto Liffredo is excellent.
To adapt Lennart's answer ... if I have a BananaBunch class:
class BananaBunch:
bananas = []
def addBanana(self, banana):
self.bananas.append(banana)
bunch = BananaBunch()
>>> bunch
<__main__.BananaBunch instance at 0x011A7FA8>
>>> bunch.addBanana(1)
>>> bunch.bananas
[1]
>>> for i in range(6):
bunch.addBanana("Banana #" + i)
>>> for i in range(6):
bunch.addBanana("Banana #" + str(i))
>>> bunch.bananas
[1, 'Banana #0', 'Banana #1', 'Banana #2', 'Banana #3', 'Banana #4', 'Banana #5']
// And for review ...
//If I then add something to the BananaBunch class ...
>>> BananaBunch.bananas.append("A mutated banana")
//My own bunch is suddenly corrupted. :-)
>>> bunch.bananas
[1, 'Banana #0', 'Banana #1', 'Banana #2', 'Banana #3', 'Banana #4', 'Banana #5', 'A mutated banana']
How does this apply to functions? Functions in Python are objects. This bears repeating. Functions in Python are objects.
So when you create a function, you are creating an object. When you give a function a mutable default value, you are populating that object's attribute with a mutable value, and every time you call that function you are operating on the same attribute. So if you are using a mutable call (like append), then you are modifying the same object, just as if you were adding bananas to the bunch object.
The "problem" here is that L=[] is only evaluated once, that is, when the file is compiled. Python steps through each line of the file and compiles it. By the time it reaches the def with the default parameter, it creates that list instance once.
If you put L = [] inside the function code, the instance is not created at "compile time" (actually compile time can also be called part of the run time) because Python compiles the function's code but does not call it. So you will get a new list instance because the creation is done every time you call the function (instead of once during compilation).
A solution for that problem is not to use mutable objects as default parameters, or only fixed instances like None:
def f(a, L = None):
if l == None:
l = []
L.append(a)
return L
Note that in both cases you described, the scope of L is the function scope.
You have to keep in mind that python is an interpreted language. What is happening here is when the function "f" is defined, it creates the list and assigns it to the default parameter "L" of function "f". Later, when you call this function, the same list is used as the default parameter. In short, the code on the "def" line, only gets executed once when the function is defined. This is a common python pitfall, of which I have fallen in myself.
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
There have been suggestions for idioms in other answers here to fix this issue. The one I would suggest is as follows:
def f(a, L=None):
L = L or []
L.append(a)
return L
This uses the or short circuit to either take the "L" that was passed, or create a new list.
The answer to your scope question is the "L" only has a scope within the function "f", but because the default parameters are only assigned once to a single list instead of every time you call the function it behaves as if the default parameter "L" has a global scope.
I think that the variable length argument turns into a tuple when it enters the edit_list function, so I changed it from a Tuple to a List to edit it. When it returns, I assume it is still treated as a Tuple and therefore no changes to the argument values can be returned?
If so, how would I go about editing the contents of a list that is used in a variable length argument context?
def main():
x = ['hi','hello','world',1,2]
edit_list(*x)
print(x)
#why can't I change the list index 1 value to '2' and then
#return the modified arg list to main and then print it out?
def edit_list(*args):
args = list(args)
print(args)
args[1] = 2
return args
if __name__ == '__main__' : main()
You would need to pass in the list directly, instead of unpacking the list using edit_list(*x).
def edit_list(my_list):
my_list[1] = 2
def main():
x = [ ... ]
edit_list(x)
print(x)
To understand the mechanism of this, you should try to be familiar with the concept "mutable" and "immutable". (of course, only if you want to be better at Python).
Take your code as an example, if the element you passed in is a list, and you are changing the element of that list, you will get a different result.
def main():
x = ['hi',['hello'],'world',1,2]
edit_list(*x)
print(x)
# After you run edit_list, the original x will be changed
def edit_list(*args):
args = list(args)
print(args)
args[1][0] = 2
return args
In python, objects are either "mutable" or "immutable". lists for example, are mutable. integers, strings are immutable. When you pass a mutable object to a function, you pass in it's reference so when you edit it(not assigning another object to the variable) you will edit the original object. However, if it's immutable, when you pass in the object to the function, it will generate a copy.
Also use your code as an example. You are effectively doing
edit_list("hi", "hello", "world", 1, 2)
All the arguments are immutable, so you copied each of them and give them to the function. Therefore, when you are inside the function, you already have different objects to the original. There's nothing you can do to change the original list.
However, if you pass them in as a list
edit_list(x)
Because x is a mutable object, you pass the reference, or the original object in the function. So when you edit the list lst[1] = 2, the original one will change. However, if you do something like lst = [1, 2, 3], the original x won't be changed, because you just created another object and assigned to the unrelated variable lst.
I would like to create instances of a class containing a list that's empty by default; instead of later setting this list to the final full list I would like to successively add items to it. Here's a piece of sample code illustrating this:
#!/usr/bin/python
class test:
def __init__(self, lst=[], intg=0):
self.lista = lst
self.integer = intg
name_dict = {}
counter = 0
for name in ('Anne', 'Leo', 'Suzy'):
counter += 1
name_dict[name] = test()
name_dict[name].integer += 1
name_dict[name].lista.append(counter)
print name, name_dict[name].integer, name_dict[name].lista
When I ran the above program I expected to get
Anne 1 [1]
Leo 1 [2]
Suzy 1 [3]
as I assumed lista to always be initialised to an empty list.
What I got instead was this:
Anne 1 [1]
Leo 1 [1, 2]
Suzy 1 [1, 2, 3]
If I replace self.lista = lst by self.lista = [] it works fine, just like when I add the line name_dict[name].lista = [] to the for loop.
Why is it that the contents of the previous objects' lists are retained, yet their values of integer aren't? I am rather new to Python, so it would be great if someone could point out to me where my thoughts/assumptions have gone astray.
Thanks a lot in advance for your replies.
It is a very bad idea to use a mutable object as a default value, as you do here:
def __init__(self, lst=[], intg=0):
# ...
Change it to this:
def __init__(self, lst=None, intg=0):
if lst is None:
lst = []
# ...
The reason that your version doesn't work is that the empty list is created just once when the function is defined, not every time the function is called.
In some Python implementations you can see the value of the default values of the function by inspecting the value of func_defaults:
print test.__init__.func_defaults
name_dict[name] = test()
# ...
Output:
([],)
Anne 1 [1]
([1],)
Leo 1 [1, 2]
([1, 2],)
Suzy 1 [1, 2, 3]
The problem lies in this line:
def __init__(self, lst=[], intg=0):
You shouldn't use a list as a default argument. The first time __init__ is called without lst specified the Python interpreter will define an empty list []. Subsequent calls to the function will operate on the same list if lst is not specified, without declaring a new list. This causes weird problems.
You should instead use a default value of None and add a check at the beginning of the function:
def __init__(self, lst=None, intg=0):
if lst is None:
lst = []
See this post for further details. Quoting the post:
Default arguments are evaluated at
function definition time, so they're
persistent across calls. This has some
interesting (and confusing) side
effects. An example:
>>> def foo(d=[]):
... d.append('a')
... return d
If
you've not tried this before, you
probably expect foo to always return
['a']: it should start with an empty
list, append 'a' to it, and return.
Here's what it actually does:
>>> foo() ['a']
>>> foo() ['a', 'a']
>>> foo() ['a', 'a', 'a']
This is because the default value for d is
allocated when the function is
created, not when it's called. Each
time the function is called, the value
is still hanging around from the last
call. This gets even weirder if you
throw threads into the mix. If two
different threads are executing the
function at the same time, and one of
them changes a default argument, they
both will see the change.
Of course, all of this is only true if
the default argument's value is a
mutable type. If we change foo to be
defined as
>>> def foo2(d=0):
... d += 1
... return d
then it will always return 1.
(The difference here is that in foo2,
the variable d is being reassigned,
while in foo its value was being
changed.)
The problem lies with the default constructor argument. You should read this question to find answer to your question: “Least Astonishment” in Python: The Mutable Default Argument
Python evaluates the default arguments once at function definition:
def init_a():
"""Initialize a."""
print("init_a")
return 1
def test(a_parameter=init_a()):
"""A test function."""
print("Execute test")
print("whatever")
test()
test()
gives
init_a
whatever
Execute test
Execute test
So your list gets defined once. You use the same list as before. This is why you should use the pattern
def a_function(a_parameter=None): # Create 'None' once
if a_parameter is None:
a_parameter = [] # Create a new list - each time
I need to write a function that cuts the first 2 items from the list that is passed as argument and change it. Not return the modified list
So if a call:
fruits = ['apple','orange','grape','banana']
print fruits # prints ['apple','orange','grape','banana']
f1(fruits)
print fruits # prints ['grape','banana']
I tried:
def f1(a):
a[:] = a[2:] # throws TypeError: 'str' object does not support item assignment
def f1(a):
a = a[2:] # doesn't change the value of fruits
Neither f1(*a) seems to work. I searched StackOverflow but I can't seem to find an answer.
Please don't tell me to return the value of a from f1.
The slice operator ( [x:y] ) creates another list, what you have to do instead is to delete the elements that are already in the list like this:
def f1(a):
del a[0:2]
Try the below function! You have to mutate a itself, not creating another list.
def f(a):
del a[0]
del a[0]
You could use pop method of the list to change input list inplace:
def f(a):
a.pop(0)
a.pop(0)