We Keep Coding

Local and Global lists in Python

Please help me as I am new to python
when I call this function the original value of list1 changes
def mystery(list1):
list1[0] , list1[1] = list1[1], list1[0]
list1 = [7,82,44,23,11]
mystery(list1)
print(list1) #prints [82, 7, 44, 23, 11]
how it can change the value of global list1
if I change my function to
def mystery(list1):
list1 = list1 + list1[2:5]
then I am getting the global value of list1 and not the updated one.

lists are mutable objects in python, so if you are passing to a function a list all the changes that you make to the list inside your function will be reflected everywhere/globally
If you pass a mutable object into a method, the method gets a reference to that same object and you can mutate it to your heart's delight, but if you rebind the reference in the method, the outer scope will know nothing about it, and after you're done, the outer reference will still point at the original object.
If you pass an immutable object to a method, you still can't rebind the outer reference, and you can't even mutate the object.[more details here]

The reason list1 is changing is because you're actually modifying the list object inside of the function.
It really has nothing todo with global / local variables, if you renamed the parameter inside your function to something else, and still passed in list1, list1 would be modified.
If you're wanting to return a new list, you need to first create a copy of the list, there's many ways to do this. list(list1) is one way. Then return the list at the end of the function.
If I understand your queston, you want to actually append some more values to the passed in list, use list1.append(...) to add to the end of the list.
And since you're modifying the list itself, it'll change in the larger scope.
But it's still not using the global scope, as you're just using a var with the same name in the local scope.

Maybe just return the list?
def mystery(list1):
return list1 + list1[2:5]
list1 = [7,82,44,23,11]
list1 = mystery(list1)
print(list1)

In python, assigning a value to a variable like a = 5 means you are creating an object in the memory for the value 5. Now a is a link to that object holding 5 (in Layman's terms). If you change a now, say a = "something else", this creates a new object for the string "something else" in somewhere else in the memory and now a is pointing to that. This is why you get to change the data type of python variables.
When you pass something to a python function, that link is the one that is passed on. So when you call mystery(list1), original link to the list1 is passed. Therefore changing an element in the list1 means you are assigning a new value to that particular element in the original list1. No matter you are inside or outside of the function mystery() in this case since you will be using the original link you created to access the element inside list1 to change it. The change happens inside the list1; list1 didn't get reassigned.
However when you do list1 = "something new" inside your function mystery(), you are creating a new variable list1 inside the function which is local to the function. You are not changing the original list1.

Passing mutable objects into a function and then modifying the object inside the function will have the same effect as modifying the object directly.
list1 = [1,2,3]
def func(list1):
list1[0] = 5
>>>list1
[5,2,3]
This is effectively same as directly running list1[0] = 5
However if you pass an immutable object into a function such as tuple, It will not support item assignement TypeError: 'tuple' object does not support item assignment. So you need to build a new immutable object and return it.
>>> tup1 = (1,2,3) # doing tup1[0] = 5 will cause TypeError
>>> tup2 = tup1 + (5,)
>>> tup2
(1, 2, 3, 5)
Put it in function,
>>> def func2(tup1):
return tup1 + (5,)
>>> func2(tup1=(1,2,3))
(1, 2, 3, 5)

Python Modify List outside scope of a class

I'm trying to call a function defined outside of class scope in Python, and it modifies the passed value, without any references to the passed variable, but somehow the original variable in class scope gets modified.
I have used python extensively for scriting but haven't really done much with Classes. I looked at this post, which explains a lot but doesn't really talk about it. This variable is surely out of scope, how's it getting mutated?
def doSomethingOutOfScope(arr):
spv = arr[0]
arr[0] = 'S'
class Solution:
def doSomethingInScope(self, passed_arr):
print(passed_arr) # prints [0, 1, 2]
doSomethingOutOfScope(passed_arr) # passed_arr shouldn't get modified
print(passed_arr) # prints ['S', 1, 2] - Why ?
s = Solution()
s.doSomethingInScope([0, 1, 2])

In Python, everything is an object and every object is passed in reference. Therefore basically any changes you directly made on the passed object besides reassigning is always reflected on the object itself in real time. The key thing to note is the mutability of the object, i.e. whether the object is mutable pass its initial assignment.
For example, str and int objects are immutable. Note there are never any methods under str and int that changes the object directly:
s = 'foo' # str type
s.upper() # FOO
print(s)
# foo
Note how str.upper() method doesn't change s itself. in order to make s into "FOO", you need to reassign s:
s = s.upper()
print(s)
# FOO
However by reassigning the object (obj = ...) it changes the object reference, where id(s) will now be different. So if a str was passed like this:
def func(string):
string = string.upper()
func(s)
s will remain unchanged because at the point of reassignment string = ..., string will no longer have the same object reference of s.
However, with mutable objects, as long as you don't reassign, the object itself will take the changes:
l = list('abcde')
def func(lst, value):
lst.append(value)
# no reassignment
func(l, 'f')
print(l)
# ['a', 'b', 'c', 'd', 'e', 'f']
That is because the object reference (id(l)) remains the same within the function.
All that said, in your case, if you wanted to do something with the object but not mutate it, you want to either pass a copy of the object (which will have a different id), or create a copy of the passed object in the local scope:
# pass a copy of the object:
def func(lst):
lst.append('x')
return lst
func(lst[:])
# OR, create a local object
def out_of_scope(lst):
temp_lst = lst[:]
temp_lst.append('x')
return temp_lst
In this particular case, [:] returns a full slice of the list as a different object.
For a bit more information, here's a relevant read.

Pythons list are actually mutable and are passed by reference

global variable with setdefault in python

I'm a bit confused about the behavior of global variables with the use if setdefault() in python:
Please find sample code for how this are referenced/resolved using setdefault, can somebody help me clarify what is going on in here?
Main file where variables are resoved :
#main
from test import *
fill_func()
print my_list
print my_dict
Test file where variables are assigned values :
#test
my_list = []
my_dict = {}
def fill_func():
my_list = [1,2,3]
print my_list
my_dict.setdefault(0,[]).append('zero')
print my_dict
Output :
[1, 2, 3]
{0: ['zero']}
[]
{0: ['zero']}
I'm not able to understand why the list(my_list) variable shows empty when called from main.py, whereas my_dict shows data fine?
Any help is appreciated. TIA!
#
Sample test file 2
#test
my_list = []
my_dict = {}
def fill_func():
def fill_list():
global my_list
my_list = [1,2,3]
print my_list
my_dict.setdefault(0,[]).append('zero')
print my_dict
fill_list()
Ouput :
{0: ['zero']}
[1, 2, 3]
[]
{0: ['zero']}
Can someone please throw some light on the second test file, please bear with me, trying to understand the basics :)
TIA!

my_list is defined as a local variable inside the fill_func; this is shadowing the other my_list defined in the global scope.
therefore, your code first calls the fill_func that prints the local my_list, then the default dict. It then exits the function and prints the outer scope my_list, and the dict (which was not shadowed)

You are creating a local variable with the same name as the global variable. Just add global my_list at the top of the function.

That's because, as you've rightly indicated, it has to do with scope. my_list and my_dict are global to test.py, which have to accessed using the global qualifier. That is, your code should be:
# Edited to address comments (see explanation below)
def fill_func():
global my_list # this is necessary since you're updating the values
global my_dict
my_list.extend([1,2,3])
print my_list
my_dict.setdefault(0,[]).append('zero')
print my_dict
EDIT:
To get the both the list and the dictionary to update, one has to extend the list, and modify the dictionary (as you've done) - i.e. actually change its value. Assigning it a value using the assignment operator only changes what the variable refers to and not the value itself. This is why it doesn't update outside of the local function scope. And, this is also why it updates when we use other methods to modify the contents of those variables.
The problem is when a function body is parsed all the variables used
in either normal assignments or augmented assigments are considered as
local variables, so when the function gets called Python will not look
for those variables in global scope hence it will raise an error.
Hence you need to specify those variables as global to tell Python to
look for them in global scope.
Another alternative is to use list.extend()
(From here: https://stackoverflow.com/a/23436510/866930 . An additional reference that's useful on this is: https://stackoverflow.com/a/31437415/866930)
You can always access a global variable as long as you don't have a
local variable of the same name. You only need the global statement
when you are going to change what object a variable name refers to.
Compare to this version:
def fill_func():
global my_list
global my_dict
my_list = [1,2,3]
print my_list # prints `[1, 2, 3]` here and `[]` in main
my_dict = {1: 'a', 2: 'b'}
print my_dict # prints `{1: 'a', 2: 'b'}` here, and `{}` in main
Without the use of global, Python would think that the variable is local to the scope of the code element where it was defined, hence global tells the interpreter to look within the globals symbol table (these contents can be accessed via globals())

you need to declare in your function that you want to use the global variable.
You can do that by
def fill_func():
global my_list
global my_dict
my_list = [1,2,3]
print my_list
my_dict.setdefault(0,[]).append('zero')
print my_dict
Please note that normally it is preferred to use only CAPITAL LETTERS for globals to avoid confusion

Why are lists linked in Python in a persistent way?

A variable is set. Another variable is set to the first. The first changes value. The second does not. This has been the nature of programming since the dawn of time.
>>> a = 1
>>> b = a
>>> b = b - 1
>>> b
0
>>> a
1
I then extend this to Python lists. A list is declared and appended. Another list is declared to be equal to the first. The values in the second list change. Mysteriously, the values in the first list, though not acted upon directly, also change.
>>> alist = list()
>>> blist = list()
>>> alist.append(1)
>>> alist.append(2)
>>> alist
[1, 2]
>>> blist
[]
>>> blist = alist
>>> alist.remove(1)
>>> alist
[2]
>>> blist
[2]
>>>
Why is this?
And how do I prevent this from happening -- I want alist to be unfazed by changes to blist (immutable, if you will)?

Python variables are actually not variables but references to objects (similar to pointers in C). There is a very good explanation of that for beginners in http://foobarnbaz.com/2012/07/08/understanding-python-variables/
One way to convince yourself about this is to try this:
a=[1,2,3]
b=a
id(a)
68617320
id(b)
68617320
id returns the memory address of the given object. Since both are the same for both lists it means that changing one affects the other, because they are, in fact, the same thing.

Variable binding in Python works this way: you assign an object to a variable.
a = 4
b = a
Both point to 4.
b = 9
Now b points to somewhere else.
Exactly the same happens with lists:
a = []
b = a
b = [9]
Now, b has a new value, while a has the old one.
Till now, everything is clear and you have the same behaviour with mutable and immutable objects.
Now comes your misunderstanding: it is about modifying objects.
lists are mutable, so if you mutate a list, the modifications are visible via all variables ("name bindings") which exist:
a = []
b = a # the same list
c = [] # another empty one
a.append(3)
print a, b, c # a as well as b = [3], c = [] as it is a different one
d = a[:] # copy it completely
b.append(9)
# now a = b = [3, 9], c = [], d = [3], a copy of the old a resp. b

What is happening is that you create another reference to the same list when you do:
blist = alist
Thus, blist referes to the same list that alist does. Thus, any modifications to that single list will affect both alist and blist.
If you want to copy the entire list, and not just create a reference, you can do this:
blist = alist[:]
In fact, you can check the references yourself using id():
>>> alist = [1,2]
>>> blist = []
>>> id(alist)
411260888
>>> id(blist)
413871960
>>> blist = alist
>>> id(blist)
411260888
>>> blist = alist[:]
>>> id(blist)
407838672
This is a relevant quote from the Python docs.:
Assignment statements in Python do not copy objects, they create bindings between a target and an object. For collections that are mutable or contain mutable items, a copy is sometimes needed so one can change one copy without changing the other.

Based on this post:
Python passes references-to-objects by value (like Java), and
everything in Python is an object. This sounds simple, but then you
will notice that some data types seem to exhibit pass-by-value
characteristics, while others seem to act like pass-by-reference...
what's the deal?
It is important to understand mutable and immutable objects. Some
objects, like strings, tuples, and numbers, are immutable. Altering
them inside a function/method will create a new instance and the
original instance outside the function/method is not changed. Other
objects, like lists and dictionaries are mutable, which means you can
change the object in-place. Therefore, altering an object inside a
function/method will also change the original object outside.
So in your example you are making the variable bList and aList point to the same object. Therefore when you remove an element from either bList or aList it is reflected in the object that they both point to.

The short answer two your question "Why is this?": Because in Python integers are immutable, while lists are mutable.
You were looking for an official reference in the Python docs. Have a look at this section:
http://docs.python.org/2/reference/simple_stmts.html#assignment-statements
Quote from the latter:
Assignment statements are used to (re)bind names to values and to
modify attributes or items of mutable objects
I really like this sentence, have never seen it before. It answers your question precisely.
A good recent write-up about this topic is http://nedbatchelder.com/text/names.html, which has already been mentioned in one of the comments.

The immutable object in python

I see a article about the immutable object.
It says when:
variable = immutable
As assign the immutable to a variable.
for example
a = b # b is a immutable
It says in this case a refers to a copy of b, not reference to b.
If b is mutable, the a wiil be a reference to b
so:
a = 10
b = a
a =20
print (b) #b still is 10
but in this case:
a = 10
b = 10
a is b # return True
print id(10)
print id(a)
print id(b) # id(a) == id(b) == id(10)
if a is the copy of 10, and b is also the copy of 10, why id(a) == id(b) == id(10)?

"Simple" immutable literals (and in particular, integers between -1 and 255) are interned, which means that even when bound to different names, they will still be the same object.
>>> a = 'foo'
>>> b = 'foo'
>>> a is b
True

While that article may be correct for some languages, it's wrong for Python.
When you do any normal assignment in Python:
some_name = some_name_or_object
You aren't making a copy of anything. You're just pointing the name at the object on the right side of the assignment.
Mutability is irrelevant.
More specifically, the reason:
a = 10
b = 10
a is b
is True, is that 10 is interned -- meaning Python keeps one 10 in memory, and anything that is set to 10 points to that same 10.
If you do
a = object()
b = object()
a is b
You'll get False, but
a = object()
b = a
a is b
will still be True.

Because interning has already been explained, I'll only address the mutable/immutable stuff:
As assign the immutable to a variable.
When talking about what is actually happening, I wouldn't choose this wording.
We have objects (stuff that lives in memory) and means to access those objects: names (or variables), these are "bound" to an object in reference. (You could say the point to the objects)
The names/variables are independent of each other, they can happen to be bound to the same object, or to different ones. Relocating one such variable doesn't affect any others.
There is no such thing as passing by value or passing by reference. In Python, you always pass/assign "by object". When assigning or passing a variable to a function, Python never creates a copy, it always passes/assigns the very same object you already have.
Now, when you try to modify an immutable object, what happens? As already said, the object is immutable, so what happens instead is the following: Python creates a modified copy.
As for your example:
a = 10
b = a
a =20
print (b) #b still is 10
This is not related to mutability. On the first line, you bind the int object with the value 10 to the name a. On the second line, you bind the object referred to by a to the name b.
On the third line, you bind the int object with the value 20 to the name a, that does not change what the name b is bound to!
It says in this case a refers to a copy of b, not reference to b. If b
is mutable, the a wiil be a reference to b
As already mentioned before, there is no such thing as references in Python. Names in Python are bound to objects. Different names (or variables) can be bound to the very same object, but there is no connection between the different names themselves. When you modify things, you modify objects, that's why all other names that are bound to that object "see the changes", well they're bound to the same object that you've modified, right?
If you bind a name to a different object, that's just what happens. There's no magic done to the other names, they stay just the way they are.
As for the example with lists:
In [1]: smalllist = [0, 1, 2]
In [2]: biglist = [smalllist]
In [3]: biglist
Out[3]: [[0, 1, 2]]
Instead of In[1] and In[2], I might have written:
In [1]: biglist = [[0, 1, 2]]
In [2]: smalllist = biglist[0]
This is equivalent.
The important thing to see here, is that biglist is a list with one item. This one item is, of course, an object. The fact that it is a list does not conjure up some magic, it's just a simple object that happens to be a list, that we have attached to the name smalllist.
So, accessing biglist[i] is exactly the same as accessing smalllist, because they are the same object. We never made a copy, we passed the object.
In [14]: smalllist is biglist[0]
Out[14]: True
Because lists are mutable, we can change smallist, and see the change reflected in biglist. Why? Because we actually modified the object referred to by smallist. We still have the same object (apart from the fact that it's changed). But biglist will "see" that change because as its first item, it references that very same object.
In [4]: smalllist[0] = 3
In [5]: biglist
Out[5]: [[3, 1, 2]]
The same is true when we "double" the list:
In [11]: biglist *= 2
In [12]: biglist
Out[12]: [[0, 1, 2], [0, 1, 2]]
What happens is this: We have a list: [object1, object2, object3] (this is a general example)
What we get is: [object1, object2, object3, object1, object2, object3]: It will just insert (i.e. modify "biglist") all of the items at the end of the list. Again, we insert objects, we do not magically create copies.
So when we now change an item inside the first item of biglist:
In [20]: biglist[0][0]=3
In [21]: biglist
Out[21]: [[3, 1, 2], [3, 1, 2]]
We could also just have changed smalllist, because for all intents and purposes, biglist could be represented as: [smalllist, smalllist] -- it contains the very same object twice.