We Keep Coding

Why does this an internal dictionary variable in my function still still contain key-value pairs from a previous function call? [duplicate]

When you define a function in Python with an array parameter, what is the scope of that parameter?
This example is taken from the Python tutorial:
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
Prints:
[1]
[1, 2]
[1, 2, 3]
I'm not quite sure if I understand what's happening here. Does this mean that the scope of the array is outside of the function? Why does the array remember its values from call to call? Coming from other languages, I would expect this behavior only if the variable was static. Otherwise it seems it should be reset each time. And actually, when I tried the following:
def f(a):
L = []
L.append(a)
return L
I got the behavior I expected (the array was reset on each call).
So it seems to me that I just need the line def f(a, L=[]): explained - what is the scope of the L variable?

The scope is as you would expect.
The perhaps surprising thing is that the default value is only calculated once and reused, so each time you call the function you get the same list, not a new list initialized to [].
The list is stored in f.__defaults__ (or f.func_defaults in Python 2.)
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
print f.__defaults__
f.__defaults__ = (['foo'],) # Don't do this!
print f(4)
Result:
[1]
[1, 2]
[1, 2, 3]
([1, 2, 3],)
['foo', 4]

The scope of the L variable is behaving as you expect.
The "problem" is with the list you're creating with []. Python does not create a new list each time you call the function. L gets assigned the same list each time you call which is why the function "remembers" previous calls.
So in effect this is what you have:
mylist = []
def f(a, L=mylist):
L.append(a)
return L
The Python Tutorial puts it this way:
The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes.
and suggests the following way to code the expected behaviour:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L

There's even less "magic" than you might suspect. This is equivalent to
m = []
def f(a, L=m):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
m is only created once.

Say you have the following code:
def func(a=[]):
a.append(1)
print("A:", a)
func()
func()
func()
You can use python's indentation to help you understand what's going on. Everything that is flush to the left margin is executed when the file gets executed. Everything that's indented is compiled into a code object which gets executed when func() is called. So the function is defined and its default arguments set once, when the program gets executed (because the def statement is flush left).
What it does with the default arguments is an interesting issue. In python 3, it puts most of the information about a function in two places: func.__code__ and func.__defaults__. In python 2, func.__code__ was func.func_code func.__defaults__ was func.func_defaults. Later versions of python 2, including 2.6 have both sets of names, to aid the transition from python 2 to python 3. I will use the more modern __code__ and __defaults__. If you're stuck on an older python, the concepts are the same; just the names differ.
The default values are stored in func.__defaults__, and retrieved each time the function is called.
Thus when you define the function above, the body of the function gets compiled and stored in variables under __code__, to be executed later, and the default arguments get stored in __defaults__. When you call the function, it uses the values in __defaults__. If those values get modified for any reason, it only has the modified version available to use.
Play around defining different functions in the interactive interpreter, and see what you can figure out about how python creates and uses functions.

The explaination is given in answers to this question. To sum it up here:
Functions in Python are a kind of object. Because they are a kind of object, they act like objects when instantiated. A function, if defined with a mutable attribute as a default argument, is exactly the same as a class with a static attribute that is a mutable list.
Lennart Regebro has a good explanation and the answer to the question by Roberto Liffredo is excellent.
To adapt Lennart's answer ... if I have a BananaBunch class:
class BananaBunch:
bananas = []
def addBanana(self, banana):
self.bananas.append(banana)
bunch = BananaBunch()
>>> bunch
<__main__.BananaBunch instance at 0x011A7FA8>
>>> bunch.addBanana(1)
>>> bunch.bananas
[1]
>>> for i in range(6):
bunch.addBanana("Banana #" + i)
>>> for i in range(6):
bunch.addBanana("Banana #" + str(i))
>>> bunch.bananas
[1, 'Banana #0', 'Banana #1', 'Banana #2', 'Banana #3', 'Banana #4', 'Banana #5']
// And for review ...
//If I then add something to the BananaBunch class ...
>>> BananaBunch.bananas.append("A mutated banana")
//My own bunch is suddenly corrupted. :-)
>>> bunch.bananas
[1, 'Banana #0', 'Banana #1', 'Banana #2', 'Banana #3', 'Banana #4', 'Banana #5', 'A mutated banana']
How does this apply to functions? Functions in Python are objects. This bears repeating. Functions in Python are objects.
So when you create a function, you are creating an object. When you give a function a mutable default value, you are populating that object's attribute with a mutable value, and every time you call that function you are operating on the same attribute. So if you are using a mutable call (like append), then you are modifying the same object, just as if you were adding bananas to the bunch object.

The "problem" here is that L=[] is only evaluated once, that is, when the file is compiled. Python steps through each line of the file and compiles it. By the time it reaches the def with the default parameter, it creates that list instance once.
If you put L = [] inside the function code, the instance is not created at "compile time" (actually compile time can also be called part of the run time) because Python compiles the function's code but does not call it. So you will get a new list instance because the creation is done every time you call the function (instead of once during compilation).
A solution for that problem is not to use mutable objects as default parameters, or only fixed instances like None:
def f(a, L = None):
if l == None:
l = []
L.append(a)
return L
Note that in both cases you described, the scope of L is the function scope.

You have to keep in mind that python is an interpreted language. What is happening here is when the function "f" is defined, it creates the list and assigns it to the default parameter "L" of function "f". Later, when you call this function, the same list is used as the default parameter. In short, the code on the "def" line, only gets executed once when the function is defined. This is a common python pitfall, of which I have fallen in myself.
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
There have been suggestions for idioms in other answers here to fix this issue. The one I would suggest is as follows:
def f(a, L=None):
L = L or []
L.append(a)
return L
This uses the or short circuit to either take the "L" that was passed, or create a new list.
The answer to your scope question is the "L" only has a scope within the function "f", but because the default parameters are only assigned once to a single list instead of every time you call the function it behaves as if the default parameter "L" has a global scope.

inhibit sharing default arg b/w successive calls

Python Tutorial -
Important warning: The default value is evaluated only once
...
If you don’t want the default to be
shared between subsequent calls, you
can write the function like this
instead:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
I still was expecting:
print f(1) # [1]
print f(2) # [1,2]
print f(3) # [1,2,3]
I reason:
The default value (L=None) was executed for f(1) which helped L point to a new empty list in the fn body. However on successive calls, L=None was not executed; so L still points to the list which already has 1 in it now, and subsequent calls are simply appending more elements to it thereby sharing L.
Where am I thinking incorrectly?
UPDATE
def f(a, L=[]):
L.append(a)
return L
Does L here point to an empty list created in heap or stack?

L is the name of an argument, but it is also a local variable. Rebinding it rebinds the local variable, but does not change the default argument.
UPDATE EDIT:
Python doesn't have "heap" and "stack" in the same manner as C; all it has are objects, and references to those objects. Your function call returns a reference to the same list that was created as the default value for the L argument of the f function, and any operation that mutates it will mutate the value of the default argument.

The default value is evaluated only once
means that if you do def foo(a, b=len([1,2,3])) b will be set to 3 and it will not make any function call to len if you call foo.
The = operator assigns an object to a name. It doesn't change the previous object.

If you run the functions below, it will help you to see how it works.
def f(a, L=[]):
print("id default: ", id(L))
L.append(a)
print("id used: ", id(L)
return L
Notice 1 object address, we are using only the default list object and we are changing it.
def f(a, L=[]):
print("id default: ", id(L))
if L == []:
L = []
L.append(a)
print("id used: ", id(L))
return L
Notice 2 different object addresses, when you assign L=[] again in the function, you are using a different list object and not the default list object, and that is why the default list object doesn't change.
def f(a, L=None):
print("id default", id(L))
if L is None:
L = []
L.append(a)
print("id used: ", id(L))
return L
This function is basically the same as the one above it, same idea, but it uses a None object instead of an empty list object.

Python list should be empty on class instance initialisation, but it's not. Why?

I would like to create instances of a class containing a list that's empty by default; instead of later setting this list to the final full list I would like to successively add items to it. Here's a piece of sample code illustrating this:
#!/usr/bin/python
class test:
def __init__(self, lst=[], intg=0):
self.lista = lst
self.integer = intg
name_dict = {}
counter = 0
for name in ('Anne', 'Leo', 'Suzy'):
counter += 1
name_dict[name] = test()
name_dict[name].integer += 1
name_dict[name].lista.append(counter)
print name, name_dict[name].integer, name_dict[name].lista
When I ran the above program I expected to get
Anne 1 [1]
Leo 1 [2]
Suzy 1 [3]
as I assumed lista to always be initialised to an empty list.
What I got instead was this:
Anne 1 [1]
Leo 1 [1, 2]
Suzy 1 [1, 2, 3]
If I replace self.lista = lst by self.lista = [] it works fine, just like when I add the line name_dict[name].lista = [] to the for loop.
Why is it that the contents of the previous objects' lists are retained, yet their values of integer aren't? I am rather new to Python, so it would be great if someone could point out to me where my thoughts/assumptions have gone astray.
Thanks a lot in advance for your replies.

It is a very bad idea to use a mutable object as a default value, as you do here:
def __init__(self, lst=[], intg=0):
# ...
Change it to this:
def __init__(self, lst=None, intg=0):
if lst is None:
lst = []
# ...
The reason that your version doesn't work is that the empty list is created just once when the function is defined, not every time the function is called.
In some Python implementations you can see the value of the default values of the function by inspecting the value of func_defaults:
print test.__init__.func_defaults
name_dict[name] = test()
# ...
Output:
([],)
Anne 1 [1]
([1],)
Leo 1 [1, 2]
([1, 2],)
Suzy 1 [1, 2, 3]

The problem lies in this line:
def __init__(self, lst=[], intg=0):
You shouldn't use a list as a default argument. The first time __init__ is called without lst specified the Python interpreter will define an empty list []. Subsequent calls to the function will operate on the same list if lst is not specified, without declaring a new list. This causes weird problems.
You should instead use a default value of None and add a check at the beginning of the function:
def __init__(self, lst=None, intg=0):
if lst is None:
lst = []
See this post for further details. Quoting the post:
Default arguments are evaluated at
function definition time, so they're
persistent across calls. This has some
interesting (and confusing) side
effects. An example:
>>> def foo(d=[]):
... d.append('a')
... return d
If
you've not tried this before, you
probably expect foo to always return
['a']: it should start with an empty
list, append 'a' to it, and return.
Here's what it actually does:
>>> foo() ['a']
>>> foo() ['a', 'a']
>>> foo() ['a', 'a', 'a']
This is because the default value for d is
allocated when the function is
created, not when it's called. Each
time the function is called, the value
is still hanging around from the last
call. This gets even weirder if you
throw threads into the mix. If two
different threads are executing the
function at the same time, and one of
them changes a default argument, they
both will see the change.
Of course, all of this is only true if
the default argument's value is a
mutable type. If we change foo to be
defined as
>>> def foo2(d=0):
... d += 1
... return d
then it will always return 1.
(The difference here is that in foo2,
the variable d is being reassigned,
while in foo its value was being
changed.)

The problem lies with the default constructor argument. You should read this question to find answer to your question: “Least Astonishment” in Python: The Mutable Default Argument

Python evaluates the default arguments once at function definition:
def init_a():
"""Initialize a."""
print("init_a")
return 1
def test(a_parameter=init_a()):
"""A test function."""
print("Execute test")
print("whatever")
test()
test()
gives
init_a
whatever
Execute test
Execute test
So your list gets defined once. You use the same list as before. This is why you should use the pattern
def a_function(a_parameter=None): # Create 'None' once
if a_parameter is None:
a_parameter = [] # Create a new list - each time

What are the implications of using mutable types as default arguments in Python? [duplicate]

This question already has answers here:
Closed 12 years ago.
Possible Duplicates:
Why the “mutable default argument fix” syntax is so ugly, asks python newbie
least astonishment in python: the mutable default argument
Here is an example.
def list_as_default(arg = []):
pass

From: http://www.network-theory.co.uk/docs/pytut/DefaultArgumentValues.html
The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. For example, the following function accumulates the arguments passed to it on subsequent calls:
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
This will print
[1]
[1, 2]
[1, 2, 3]
If you don't want the default to be shared between subsequent calls, you can write the function like this instead:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L

Python instances and attributes: is this a bug or i got it totally wrong?

Suppose you have something like this:
class intlist:
def __init__(self,l = []):
self.l = l
def add(self,a):
self.l.append(a)
def appender(a):
obj = intlist()
obj.add(a)
print obj.l
if __name__ == "__main__":
for i in range(5):
appender(i)
A function creates an instance of intlist and calls on this fresh instance the method append on the instance attribute l.
How comes the output of this code is:
[0]
[0, 1]
[0, 1, 2]
[0, 1, 2, 3]
[0, 1, 2, 3, 4]
?
If i switch
obj = intlist()
with
obj = intlist(l=[])
I get the desired output
[0]
[1]
[2]
[3]
[4]
Why this happens?
Thanks

Ah, you've hit one of the common Python gotchas: default values are computed once, then re-used. So, every time __init__ is called, the same list is being used.
This is the Pythonic way of doing what you want:
def __init__(self, l=None):
self.l = [] if l is None else l
For a bit more information, check out the Python docs (especially about three paragraphs after that heading).
Edit: There is a much better description in another answer.

When you set the default value of l=[] in __init__, you're actually using the same list each time. Instead, you could try something like:
class intlist:
def __init__(self, l=None):
if l is None:
self.l = []
else:
self.l = l

The issue is that when you are saying
def __init__(self,l = []):
You are telling Python to use the same list, [], for each invocation of the constructor. So each time obj = intlist() is called the same list is appended to.
What you should do instead is set l to a default value of None, which is a scalar (so your code will work as expected if it is used multiple times). Then, if l is None, initialize a new class member as []. Otherwise just assign the member variable to l.

obj = intlist() calls your __init__() function which uses the same array for every instance of the class.
obj = intlist(l=[]) creates a new array for every instance.

For more information I suggest reading this: http://effbot.org/zone/default-values.htm

The behavior occurs because all calls to your __init__ method share the same default list.
Try:
class intlist:
def __init__(self, l):
self.l = l if (l is not None) else []
def add(self,a):
self.l.append(a)
EDIT: Use is not, per SilentGhost

Be careful with default parameters of types like lists and dicts. Each instance of intlist gets that same list object from the default parameter.