We Keep Coding

Python Modify List outside scope of a class

I'm trying to call a function defined outside of class scope in Python, and it modifies the passed value, without any references to the passed variable, but somehow the original variable in class scope gets modified.
I have used python extensively for scriting but haven't really done much with Classes. I looked at this post, which explains a lot but doesn't really talk about it. This variable is surely out of scope, how's it getting mutated?
def doSomethingOutOfScope(arr):
spv = arr[0]
arr[0] = 'S'
class Solution:
def doSomethingInScope(self, passed_arr):
print(passed_arr) # prints [0, 1, 2]
doSomethingOutOfScope(passed_arr) # passed_arr shouldn't get modified
print(passed_arr) # prints ['S', 1, 2] - Why ?
s = Solution()
s.doSomethingInScope([0, 1, 2])

In Python, everything is an object and every object is passed in reference. Therefore basically any changes you directly made on the passed object besides reassigning is always reflected on the object itself in real time. The key thing to note is the mutability of the object, i.e. whether the object is mutable pass its initial assignment.
For example, str and int objects are immutable. Note there are never any methods under str and int that changes the object directly:
s = 'foo' # str type
s.upper() # FOO
print(s)
# foo
Note how str.upper() method doesn't change s itself. in order to make s into "FOO", you need to reassign s:
s = s.upper()
print(s)
# FOO
However by reassigning the object (obj = ...) it changes the object reference, where id(s) will now be different. So if a str was passed like this:
def func(string):
string = string.upper()
func(s)
s will remain unchanged because at the point of reassignment string = ..., string will no longer have the same object reference of s.
However, with mutable objects, as long as you don't reassign, the object itself will take the changes:
l = list('abcde')
def func(lst, value):
lst.append(value)
# no reassignment
func(l, 'f')
print(l)
# ['a', 'b', 'c', 'd', 'e', 'f']
That is because the object reference (id(l)) remains the same within the function.
All that said, in your case, if you wanted to do something with the object but not mutate it, you want to either pass a copy of the object (which will have a different id), or create a copy of the passed object in the local scope:
# pass a copy of the object:
def func(lst):
lst.append('x')
return lst
func(lst[:])
# OR, create a local object
def out_of_scope(lst):
temp_lst = lst[:]
temp_lst.append('x')
return temp_lst
In this particular case, [:] returns a full slice of the list as a different object.
For a bit more information, here's a relevant read.

Pythons list are actually mutable and are passed by reference

global variable with setdefault in python

I'm a bit confused about the behavior of global variables with the use if setdefault() in python:
Please find sample code for how this are referenced/resolved using setdefault, can somebody help me clarify what is going on in here?
Main file where variables are resoved :
#main
from test import *
fill_func()
print my_list
print my_dict
Test file where variables are assigned values :
#test
my_list = []
my_dict = {}
def fill_func():
my_list = [1,2,3]
print my_list
my_dict.setdefault(0,[]).append('zero')
print my_dict
Output :
[1, 2, 3]
{0: ['zero']}
[]
{0: ['zero']}
I'm not able to understand why the list(my_list) variable shows empty when called from main.py, whereas my_dict shows data fine?
Any help is appreciated. TIA!
#
Sample test file 2
#test
my_list = []
my_dict = {}
def fill_func():
def fill_list():
global my_list
my_list = [1,2,3]
print my_list
my_dict.setdefault(0,[]).append('zero')
print my_dict
fill_list()
Ouput :
{0: ['zero']}
[1, 2, 3]
[]
{0: ['zero']}
Can someone please throw some light on the second test file, please bear with me, trying to understand the basics :)
TIA!

my_list is defined as a local variable inside the fill_func; this is shadowing the other my_list defined in the global scope.
therefore, your code first calls the fill_func that prints the local my_list, then the default dict. It then exits the function and prints the outer scope my_list, and the dict (which was not shadowed)

You are creating a local variable with the same name as the global variable. Just add global my_list at the top of the function.

That's because, as you've rightly indicated, it has to do with scope. my_list and my_dict are global to test.py, which have to accessed using the global qualifier. That is, your code should be:
# Edited to address comments (see explanation below)
def fill_func():
global my_list # this is necessary since you're updating the values
global my_dict
my_list.extend([1,2,3])
print my_list
my_dict.setdefault(0,[]).append('zero')
print my_dict
EDIT:
To get the both the list and the dictionary to update, one has to extend the list, and modify the dictionary (as you've done) - i.e. actually change its value. Assigning it a value using the assignment operator only changes what the variable refers to and not the value itself. This is why it doesn't update outside of the local function scope. And, this is also why it updates when we use other methods to modify the contents of those variables.
The problem is when a function body is parsed all the variables used
in either normal assignments or augmented assigments are considered as
local variables, so when the function gets called Python will not look
for those variables in global scope hence it will raise an error.
Hence you need to specify those variables as global to tell Python to
look for them in global scope.
Another alternative is to use list.extend()
(From here: https://stackoverflow.com/a/23436510/866930 . An additional reference that's useful on this is: https://stackoverflow.com/a/31437415/866930)
You can always access a global variable as long as you don't have a
local variable of the same name. You only need the global statement
when you are going to change what object a variable name refers to.
Compare to this version:
def fill_func():
global my_list
global my_dict
my_list = [1,2,3]
print my_list # prints `[1, 2, 3]` here and `[]` in main
my_dict = {1: 'a', 2: 'b'}
print my_dict # prints `{1: 'a', 2: 'b'}` here, and `{}` in main
Without the use of global, Python would think that the variable is local to the scope of the code element where it was defined, hence global tells the interpreter to look within the globals symbol table (these contents can be accessed via globals())

you need to declare in your function that you want to use the global variable.
You can do that by
def fill_func():
global my_list
global my_dict
my_list = [1,2,3]
print my_list
my_dict.setdefault(0,[]).append('zero')
print my_dict
Please note that normally it is preferred to use only CAPITAL LETTERS for globals to avoid confusion

How do I execute a function inside a namespace dict?

Say I have the following dict:
In [6]: scope
Out[6]: {'bar': <function bar>, 'foo': <function foo>}
And foo and bar are:
def foo():
return 5
def bar(x):
return foo() + x
I want to run bar(1), but it will need to find foo(). Is there any way to run bar() in the scope namespace so it finds foo()?
I don't know precisely which function in scope bar will need, so I need a general method of running bar in the scope namespace. I don't have the source code for and cannot modify either function to accept a dict.
It seems like functions have a __closure__ attribute, but it's immutable. There is also a __globals__ attribute, but that just points to globals(). I've seen some answers on SO that update locals() but I'd like to leave locals() untouched.
I tried an eval in scope, but get a NameError for foo:
In [12]: eval(scope['bar'](1), scope)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-12-f6305634f1da> in <module>()
----> 1 eval(scope['bar'](1), scope)
<string> in bar(x)
NameError: global name 'foo' is not defined
Is there some simple way I'm missing?

One approach is to create new functions with a shared globals dictionary:
from types import FunctionType
def scopify(**kwargs):
scoped = dict()
for name, value in kwargs.items():
if isinstance(value, FunctionType):
scoped[name] = FunctionType(value.__code__, scoped, name)
else:
scoped[name] = value
return scoped
scope = scopify(
foo = lambda: baz,
bar = lambda x: foo() + x,
baz = 5,
)
>>> scope['foo']
5
>>> scope['bar'](10)
15
>>> scope['baz'] = 100
>>> scope['bar'](10)
110

Using eval() will work, however there are two problems with the way you tried to use it. For starters, its first argument, expression, should be a string. Secondly, since you're referencing scope in this expression and passing it to eval() as the globals namespace argument, you'll need to add it to itself.
This illustrates what I mean:
def foo():
return 5
def bar(x):
return foo() + x
scope = {'scope': {'bar': bar, 'foo':foo}} # define self-referential scope
expression = "scope['bar'](42)" # explicit reference to `scope`
print('eval({!r}, scope) returns {}'.format(expression, eval(expression, scope)))
However, in this specific case, it would be simpler to just let eval() directly look-up the value of bar in the scope dictionary namespace itself (instead of via scope['scope']['bar']):
scope = {'bar': bar, 'foo':foo}
expression = 'bar(42)'
print('eval({!r}, scope) returns {}'.format(expression, eval(expression, scope)))

python modify a dictionary inside a method

Is it possible to modify values for a dictionary inside a function without passing the dictionary as a parameter.
I would like not to return a dictionary, but only to modify its values.

That's possible, but not necessarily advisable, i can't imagine why you wouldn't like to pass or return the dictionary, if you merely don't want to return the dictionary, but could pass it, you can modify it to reflect in the original dictionary without having to return it, for example:
dict = {'1':'one','2':'two'}
def foo(d):
d['1'] = 'ONE'
print dict['1'] # prints 'one' original value
foo(dict)
print dict['1'] # prints 'ONE' ie, modification reflects in original value
# so no need to return it
However, if you absolutely cannot pass it for whatever reason, you can use a global dictionary as follows:
global dict # declare dictionary as global
dict = {'1':'one','2':'two'} # give initial value to dict
def foo():
global dict # bind dict to the one in global scope
dict['1'] = 'ONE'
print dict['1'] # prints 'one'
foo(dict)
print dict['1'] # prints 'ONE'
I'd recommend the first method demonstrated in the first code block, but feel free to use the second if absolutely necessary.
Enjoy :)

Yes you can, dictionary is an mutable object so they can be modified within functions, but it must be defined before you actually call the function.
To change the value of global variable pointing to an immutable object you must use the global statement.
>>> def func():
... dic['a']+=1
...
>>> dic = {'a':1} #dict defined before function call
>>> func()
>>> dic
{'a': 2}
For immutable objects:
>>> foo = 1
>>> def func():
... global foo
... foo += 3 #now the global variable foo actually points to a new value 4
...
>>> func()
>>> foo
4

What's the difference between globals(), locals(), and vars()?

What is the difference between globals(), locals(), and vars()? What do they return? Are updates to the results useful?

Each of these return a dictionary:
globals() always returns the dictionary of the module namespace
locals() always returns a dictionary of the current namespace
vars() returns either a dictionary of the current namespace (if called with no argument) or the dictionary of the argument.
locals and vars could use some more explanation. If locals() is called inside a function, it updates a dict with the values of the current local variable namespace (plus any closure variables) as of that moment and returns it. Multiple calls to locals() in the same stack frame return the same dict each time - it's attached to the stack frame object as its f_locals attribute. The dict's contents are updated on each locals() call and each f_locals attribute access, but only on such calls or attribute accesses. It does not automatically update when variables are assigned, and assigning entries in the dict will not assign the corresponding local variables:
import inspect
def f():
x = 1
l = locals()
print(l)
locals()
print(l)
x = 2
print(x, l['x'])
l['x'] = 3
print(x, l['x'])
inspect.currentframe().f_locals
print(x, l['x'])
f()
gives us:
{'x': 1}
{'x': 1, 'l': {...}}
2 1
2 3
2 2
The first print(l) only shows an 'x' entry, because the assignment to l happens after the locals() call. The second print(l), after calling locals() again, shows an l entry, even though we didn't save the return value. The third and fourth prints show that assigning variables doesn't update l and vice versa, but after we access f_locals, local variables are copied into locals() again.
Two notes:
This behavior is CPython specific -- other Pythons may allow the updates to make it back to the local namespace automatically.
In CPython 2.x it is possible to make this work by putting an exec "pass" line in the function. This switches the function to an older, slower execution mode that uses the locals() dict as the canonical representation of local variables.
If locals() is called outside a function it returns the actual dictionary that is the current namespace. Further changes to the namespace are reflected in the dictionary, and changes to the dictionary are reflected in the namespace:
class Test(object):
a = 'one'
b = 'two'
huh = locals()
c = 'three'
huh['d'] = 'four'
print huh
gives us:
{
'a': 'one',
'b': 'two',
'c': 'three',
'd': 'four',
'huh': {...},
'__module__': '__main__',
}
So far, everything I've said about locals() is also true for vars()... here's the difference: vars() accepts a single object as its argument, and if you give it an object it returns the __dict__ of that object. For a typical object, its __dict__ is where most of its attribute data is stored. This includes class variables and module globals:
class Test(object):
a = 'one'
b = 'two'
def frobber(self):
print self.c
t = Test()
huh = vars(t)
huh['c'] = 'three'
t.frobber()
which gives us:
three
Note that a function's __dict__ is its attribute namespace, not local variables. It wouldn't make sense for a function's __dict__ to store local variables, since recursion and multithreading mean there can be multiple calls to a function at the same time, each with their own locals:
def f(outer):
if outer:
f(False)
print('Outer call locals:', locals())
print('f.__dict__:', f.__dict__)
else:
print('Inner call locals:', locals())
print('f.__dict__:', f.__dict__)
f.x = 3
f(True)
which gives us:
Inner call locals: {'outer': False}
f.__dict__: {'x': 3}
Outer call locals: {'outer': True}
f.__dict__: {'x': 3}
Here, f calls itself recursively, so the inner and outer calls overlap. Each one sees its own local variables when it calls locals(), but both calls see the same f.__dict__, and f.__dict__ doesn't have any local variables in it.