I usually don't think too hard about variable scope in python, but I wanted to see if there's a clean explanation for this. Given two files called main.py and utils.py:
utils.py
def run():
print(L)
main.py
import utils
def run():
print(L)
if __name__ == '__main__':
L = [1,2]
run()
utils.run()
The first run() call in main.py runs fine despite L not being fed into run(), and the utils.run() call raises a NameError. Is L a global variable available to all functions defined in main.py?
If I imported utils with from utils import * instead of import utils, would that change anything?
It's module-level scope. A global variable defined in a module is available to all functions defined in the same module (if it's not overriden). Functions in another module don't have access to another module's variables unless they import them.
About "If I imported utils with from utils import * instead of import utils, would that change anything?":
No. The scope is determined at parsing time.
Check
this
for more information.
Notably:
It is important to realize that scopes are determined textually: the global
scope of a function defined in a module is that module’s namespace, no matter
from where or by what alias the function is called. On the other hand, the
actual search for names is done dynamically, at run time [...]
So the global scopes of both functions for variables defined in a module are the modules they're defined in. For one, its module also later has a definition for a global variable it uses, but not the other module, and when it's time to check for a variable when a function is run, each checks their own module's variables definitions, one finds it, the other does not.
See Python's FAQ. Their implementation of scope is a compromise between convenience and the dangers of globals.
Variables are treated as globals if they're only referenced by a function, and need to be explicitly declared as globals (e.g. global foo ) inside of the function body if you want to edit them. If you edit run() to try and change the value of L, you'll get an error.
What's happening here is that your Python code imports utils, and then runs run(). This function sees that you're looking for a variable named "L," and checks your global namespace.
Related
I would like to clarify, how globals from different modules are scoped?
I failed to find relevant documentation for this matter, thus I am relying on observation, but I would like to have some more insight, if my findings are pure implementation coincidence, or if I can trust them?
Testcase:
module_1.py:
global_var1 = 'm1_v1'
global_var2 = 'm1_v2'
def f():
print('f: {}'.format(global_var1))
print('f: {}'.format(global_var2))
module_2.py:
import module_1
global_var1 = 'm2_v1'
def g():
print('g: {}'.format(global_var1))
print('g: {}'.format(global_var2))
module_1.f()
g()
$ python3 module_2.py
f: m1_v1
f: m1_v2
g: m2_v1
Traceback (most recent call last):
File "module_2.py", line 11, in <module>
g()
File "module_2.py", line 7, in g
print('g: {}'.format(global_var2))
NameError: name 'global_var2' is not defined
Conclusion:
Thus, my conclusion is that a function will use the globals in this order:
the module where the function is used
the module where the function is defined (EDIT: this is the only answer!)
Globals are not bleeding through from imported modules.
EDIT: Functions imported from other modules bring their module globals like a closure.
Question:
I would like to see some comprehensive documentation on this matter (which I failed to find...).
While testing this out is nice, I have no idea, if this is a freak coincidence, and this behavior should never be trusted, or this is the expected way?
Also, what if a function is somehow imported through a 3rd module? what if the function is a class method? Etc.etc.
If you can't point me to a documentation, but you know a guideline "for sure", I am interested, as well.
I won't go in lengths why and how I want to use this, I am primarily interested to better understand the workings of python. However, if you do know a better solution, given the information at hand, I am interested to see that, as well - it will NOT be accepted as an answer, though.
In case there is a difference between python2 and python3, my main interest is python3, but it is nice to have the info on python2, as well.
Each module has its own global scope. Let's look at the second module:
import module_1
global_var1 = 'm2_v1'
def g():
print('g: {}'.format(global_var1))
print('g: {}'.format(global_var2))
module_1.f()
g()
Since global_var1 and global_var2 aren't local variables, they are looked up in the global scope. Since module_2.global_var2 isn't defined, you get a NameError.
You would either need to create module_2.global_var2 with something like from module_1 import global_var2, or change the definition of g to
def g():
print('g: {}'.format(global_var1))
print('g: {}'.format(module_1.global_var2))
import module_1 only adds one name to the global scope of module_2, that being module_1, which refers to the module object created by the import statement. from module_1 import global_var2 also only adds one name, global_var2, which is initialized to the value of module1.global_var2 (though with the side effect of creating a new module). In this case, module1.global_var2 and module2.globar_var2 are two distinct objects, not two names for the same object.
TLDR: Python uses lexical/static scoping since Python 2.1, including Python 3. (See name resolution and PEP 227)
Lexical scoping means functions only have access to the enclosing scopes in which they are defined, up to their module’s global namespace. Only the builtins are visible in all modules. The global namespace of modules is not shared, and importing or calling a function in another module does not give it access to that namespace.
Some peculiarities worth pointing out:
Local/function scopes are defined when the function is parsed. Names are either local, closures/nonlocal, or global.
Global/module scopes are mutable. Names can be added, removed or changed at any time.
Classes introduce short-lived scopes that are not visible to functions or other classes defined inside them.
When we import a module in a Python script, does this copy all the required code into the script, or does it just let the script know where to find it?
What happens if we don't use the module then in the code, does it get optimized out somehow, like in C/C++?
None of those things are the case.
An import does two things. First, if the requested module has not previously been loaded, the import loads the module. This mostly boils down to creating a new global scope and executing the module's code in that scope to initialize the module. The new global scope is used as the module's attributes, as well as for global variable lookup for any code in the module.
Second, the import binds whatever names were requested. import whatever binds the whatever name to the whatever module object. import whatever.thing also binds the whatever name to the whatever module object. from whatever import somefunc looks up the somefunc attribute on the whatever module object and binds the somefunc name to whatever the attribute lookup finds.
Unused imports cannot be optimized out, because both the module loading and the name binding have effects that some other code might be relying on.
Is it possible to import global variables into an object's instance namespace?
In a structure like this:
./mypackage/module1.py
./config.py
./script1.py
If config.py has:
#config
MYSETTING=1
and script1.py has:
#SCRIPT1
from config import *
from mypackage.mymodule import MyClass as MC
obj=MC()
module1.py:
#Myclass
class MyClass(object):
def test(self):
print MYSETTING
will give error NameError: global name "MYSETTING" is not defined
Is there some way I can import variables from config.py into MyClass namespace as globals without doing enumerated "global x" on each of them?
This problem is a little confusing. There are three parts.
config: contains variables you want.
script: contains your logical code and variables imported from config.
module: contains functions' definition.
When you call MC.test, it will find MYSETTING in module's namespace, but what you want is to find in script's namespace. So in test function, what you real want is to access parent frame's global namespace.
In this case, you need to use inspect built-in module.
import inspect
print(inspect.current_frame().f_back.f_globals["MYSETTING"])
inspect.current_frame() will return current frame, and f_back refers to parent frame which is the frame calling this function, then f_globals refers to frame's globals()
I have a module with the name my_module.py
Inside of this module there is a function my_function:
def myFunction():
print my_variable
Apparently when this functions is called it prints my_variable which is not instantiated anywhere yet. So, calling myFunction() from inside of the module itself will crash the execution.
Now, aside from my_module.py I have another script with the name my_app.py residing in the same folder.
Inside of my_app.py I am importing my_module.py and instantiating my_variable under its namespace. After my_variable is instantiated I am calling my_module.myFunction() which picks up my_variable and prints its context out:
import module
module.my_variable = 'this variable is instantiated inside of another script'
module.myFunction()
While this approach works I wonder if it is designed properly. Is there other way to instantiate a variable outside the imported module to be used by this imported module?
import module
module.my_variable = 'this variable is instantiated inside of another script'
module.myFunction()
While this approach works I wonder if it is designed properly.
No, this is not designed properly. One proper way is to pass the value to the function explicitly.
Is there other way to instantiate a variable outside the imported module to be used by this imported module?
Just have another module were you declare this variable(s). For example my_vars.py:
my_variable = 'this variable is instantiated inside of another script'
Then in my_module.py:
import my_vars
def myFunction():
print my_vars.my_variable
I'm not sure what you're trying to achieve but it's generally best practice to not mutate "global" variables. Every time you'd want to use my_function() in your code you'd have to explicitly change my_variable first, which can trigger side effects in your code if other functions/methods are depending on it. The best way would be to rewrite my_function() so that it accepts my_variable as an argument
I put a method in a file mymodule.py:
def do_something():
global a
a=1
If I try
>>> execfile('mymodule.py')
>>> do_something()
>>> print a
I get "1" as I expect. But if I import the module instead,
>>> from mymodule import *
and then run do_something(), then the python session knows nothing about the variable "a".
Can anyone explain the difference to me? Thanks.
execfile without globals, locals argument, It executes the file content in the current namespace. (the same namespace that call the execfile)
While, import execute the specified module in a separated namespace, and define the mymodule in the local namespace.
In the second part where you import mymodule, the reason why it isn't showing up is that a is global to the namespace of mymodule as done that way.
Try:
print mymodule.a
This prints:
1
As expected.
As per the Python documentation:
The global statement is a declaration which holds for the entire
current code block. It means that the listed identifiers are to be
interpreted as globals. It would be impossible to assign to a global
variable without global, although free variables may refer to globals
without being declared global.
Names listed in a global statement must not be used in the same code
block textually preceding that global statement.
Names listed in a global statement must not be defined as formal
parameters or in a for loop control target, class definition, function
definition, or import statement.