I'm writing a module manager for a program that I am writing now, and I wanted to store the module names in a dictionary then reference them and call them from globals().
module_number = 5
module_names = ["", "quiz", "scores", "gender", "help", "exit"]
I would then expect that I can call
globals()[module_names[module_number]]()
Which would call exit() and close the script, instead I get the resulting error:
Traceback (most recent call last): File "a2.py", line 103, in start() File "a2.py", line 44, in start menu() File "a2.py", line 36, in menu call_module(choice) File "a2.py", line 50, in call_module globals()converter[int(module_number)]
KeyError: 'exit'
exit isn't in globals() because it isn't a global, it's a builtin.
In Python, the "global" namespace is per-module, not system-wide. There's a special "builtins" module that holds the things that are truly system-wide, like the normal builtin functions and a few special things like exit.
You can access this module with import builtins.
The way the interpreter accesses this module is a little funky. Global lookup works roughly like this:
def get_global(global_namespace, name):
try:
return global_namespace[name]
except KeyError:
pass
try:
builtins = global_namespace['__builtins__']
except KeyError:
raise NameError(name)
if isinstance(builtins, types.ModuleType):
builtins = builtins.__dict__
try:
return builtins[name]
except KeyError:
raise NameError(name)
There's special code in places like exec, and the internal code for building function objects, that makes sure that if you override the normal globals dictionary, __builtins__ gets copied over (unless you explicitly tell it not to). When the import system builds a module object out of your module source (or compiled .pyc), it calls exec, so every module's globals ends up with the right __builtins__.
Most things in the builtins module are there because they're compiled into it (as you'd expect from the name); for CPython, you can see the source in Python/bltinmodule.c.
But notice that exit isn't there. In fact, it's injected into the builtins module by the site module, which is imported (unless you disable it) as part of the normal startup sequence. You can see the code that does this in Lib/site.py and Lib/_sitebuiltins.py. And the exit constant says it's injected this way.
So, when you type exit in your code, or at the interactive prompt, it's found in globals()['__builtins__']['exit'] or globals()['__builtins__'].__dict__['exit'].
But if you want to access it manually, you're better off doing an import builtins and accessed it as builtins.exit.
Although really, you rarely want to access builtins.exit anyway; if you want to exit programmatically, call sys.exit, which is a normal function. builtins.exit is a special Quitter object that's made specifically for interactive use. (It has a repr that gives a helpful message if you forget the parentheses, and some extra code to make it play nice with IDLE.)
In fact, the docs on the exit constant explicitly say that it's:
… useful for the interactive interpreter shell and should not be used in programs.
Related
I have 2 files, the first is named function_call_test.py and contains the following code;
from Strategy_File import strategy
def function_1():
print('This works')
strategy()
The second file is called Strategy_File.py and contains the following code;
def strategy():
print('got here')
function_1()
When running the first script I get 'NameError: name 'function_1' is not defined'.
I thought that when you imported a function that it was added to the importing modules namespace. If that is the case why can't strategy() see function_1()?
Just as importantly, how do I make this work. The above is for demo purposes only, I have reasons for wanting strategy() to be in a separate module.
Python 3.6, Windows 7-64, Visual Studio 2019 and IDLE
Python is statically scoped. Lookup for a free variable (such as function_1) proceeds though the scopes where strategy is defined, not where it is called. As strategy is defined in the global scope of the module Strategy_File, that means looking for Strategy_File.function_1, and that function is not defined.
If you want strategy to call something in the current global scope, you need to define it to accept a callable argument, and pass the desired function when you call strategy.
# Strategy_File.py
# f is not a free variable here; it's a local variable
# initialized when strategy is called.
def strategy(f):
print('got here')
f()
and
# function_call_test.py
from Strategy_File import strategy
def function_1():
print('This works')
# Assign f = function_1 in the body of strategy
strategy(function_1)
You have to import every name into the file where it is used. So you need to modify Strategy_File.py to this:
from function_call_test import function_1
def strategy():
print('got here')
function_1()
But now you encounter a new problem: circular imports. Python won't allow this. So you will have to figure out a different way to organize your functions.
This error caused by Strategy_File.py is missing definition of function_1().
adding this line in Strategy_File.py on the top will helps.
edit: circular imports will not help. sorry for the misinformation.
This question already has answers here:
Is there an easy way to pickle a python function (or otherwise serialize its code)?
(12 answers)
Closed 2 years ago.
I am trying to run the following code:
import pickle
def foo():
print("i am foo")
pickle_foo = pickle.dumps(foo)
def foo():
print("i am the new foo")
fkt = pickle.loads(pickle_foo)
return fkt()
foo()
The expected behavior would be:
the new defined function "foo" is called
in the new function the old function gets unpickeled and then called
output:
i am the new foo
i am foo
What actually happens is:
the new function foo gets called, and then recursively calls itself until a Recursion Error gets thrown:
RecursionError: maximum recursion depth exceeded while calling a Python object
The error does not occur, when the two functions are named differently, but that would be very unpractical for my project.
Could anyone explain, why this behavior occurs and how to avoid it (without changing the function names)?
The pickle module pickles functions based on their fully-qualified name reference. This means that if your function is redefined somewhere in code, and then you unpickle a pickled reference to it, calling it will result in a call to the new definition.
From the Python docs on pickle:
Note that functions (built-in and user-defined) are pickled by “fully
qualified” name reference, not by value. 2 This means that only the
function name is pickled, along with the name of the module the
function is defined in. Neither the function’s code, nor any of its
function attributes are pickled. Thus the defining module must be
importable in the unpickling environment, and the module must contain
the named object, otherwise an exception will be raised.
What you can do however, is use inspect.getsource() to retrieve the source code for your function, and pickle that. This requires that your code be available as source somewhere on the file system, so compiled C code imported, or other outside sources (interpreter input, dynamically loaded modules) will not work.
When you unpickle it, you can use exec to convert it into a function and execute it.
Note: this will redefine foo every time, so calls to foo cannot be guaranteed to have the same effect.
Note 2: exec is unsafe and usually unsuitable for code that will be interacting with external sources. Make sure you protect calls to exec from potential external attacks that attempt to execute arbitrary code.
I create a LazyLoader class which downloads files from S3 to the local file system, but only if they were not downloaded before. This class deletes the stuff it downloaded once it gets destroyed:
def __del__(self):
"""Remove downloaded files when this object is deleted."""
for downloaded_file in self.loaded_data.values():
os.remove(downloaded_file)
The tests pass, but after pytest tells me that the tests passed I get:
Exception ignored in: <bound method LazyLoader.__del__ of LazyLoader({})>
Traceback (most recent call last):
File "my_lazy_loader.py", line 47, in __del__
TypeError: 'NoneType' object is not callable
Line 47 is os.remove(downloaded_file). So os.remove is None evaluates to True. Why? How can I fix that?
If I move the code in __del__ to a method clean(), I don't have that problem.
https://docs.python.org/3/reference/datamodel.html#object.del
'del()' can be executed during interpreter shutdown. As a consequence, the global variables it needs to access (including other modules) may already have been deleted or set to None. Python guarantees that globals whose name begins with a single underscore are deleted from their module before other globals are deleted; if no other references to such globals exist, this may help in assuring that imported modules are still available at the time when the del() method is called.
If it's just unit tests, use tearDown method.
If the problem occurs when running your application and you want to do a cleanup at the end of the program, consider using atexit.register.
You might also use tempfile module for temporary files automatically removed when closed.
If your use-case allows that, turn LazyLoader into a context manager.
If none of the above applies, simply call clean() explicitly. It will follow explicit is better than implicit rule.
I have a project in which I want to repeatedly change code in a class and then run other modules to test the changes (verification..). Currently, after each edit I have to reload the code, the testing modules which run it, and then run the test. I want to reduce this cycle to one line, moreover, I will later want to test different classes, so I want to be able to receive the name of the tested class as a parameter - meaning I need dynamic imports.
I wrote a function for clean imports of any module, it seems to work:
def build_module_clean(module_string,attr_strings):
module = import_module(module_string)
module = reload(module)
for f in attr_strings:
globals()[f]=getattr(module,f)
Now, in the name of cleanliness, I want to keep this function in a wrapper module (which will contain the one-liner I want to rebuild and test all the code each time), and run it from the various modules, i.e. among the import statements of my ModelChecker module I would place the line
from wrapper import build_module_clean
build_module_clean('test_class_module',['test_class_name'])
however, when I do this, it seems the test class is added to the globals in the wrapper module, but not in the ModelChecker module (attempting to access globals()['test_class_name'] in ModelChecker gives a key error). I have tried passing globals or globals() as further parameters to build_module_clean, but globals is a function (so the test module is still loaded to the wrapper globals), and passing and then using globals() gives the error
TypeError: 'builtin_function_or_method' object does not support item assignment
So I need some way to edit one module's globals() from another module.
Alternatively, (ideally?) I would like to import the test_class module in the wrapper, in a manner that would make it visible to all the modules that use it (e.g. ModelChecker). How can I do that?
Your function should look like:
def build_module_clean(globals, module_string, attr_strings):
module = import_module(module_string)
module = reload(module)
globals[module_string] = module
for f in attr_strings:
globals[f] = getattr(module, f)
and call it like so:
build_module_clean(globals(), 'test_class_module', ['test_class_name'])
Explanation:
Calling globals() in the function call (build_module_clean(globals()...) grabs the module's __dict__ while still in the correct module and passes that to your function.
The function is then able to (re)assign the names to the newly-loaded module and it's current attributes.
Note that I also (re)assigned the newly-loaded module itself to the globals (you may not want that part).
I have a problem similar to the first problem in this question, which as far as I can see went unanswered.
I have a file "config.py" which contains a lot of parameters to be used by a class (this config.py file will change), however I can't get these to propagate into the class via execfile.
In an example piece of code:
class Class():
def __init__(self):
execfile("config.py")
print x
# config.py
x = "foo"
>>> t = Class()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in __init__
NameError: global name 'x' is not defined
Any help welcome, or any better methods of retrieving parameters from a file to be used in a class.
Many Thanks.
I don't get what you're trying to do (but i don't like it, and this is just me) but to fix your problem do (test in python2.6):
class Class():
def __init__(self):
execfile('config.py', locals()) # Not recommanded, maybe you want globals().
print x
But from the doc:
Note
The default locals act as described
for function locals() below:
modifications to the default locals
dictionary should not be attempted.
Pass an explicit locals dictionary if
you need to see effects of the code on
locals after function execfile()
returns. execfile() cannot be used
reliably to modify a function’s
locals.
and about :
Any help welcome, or any better
methods of retrieving parameters from
a file to be used in a class.
You can use import.
Even though it might be convenient to keep configuration settings in a Python file I would recommend against it. I think it opens up a whole set of problems that you don't really want to have do deal with. Anything could be placed in your configuration file, including malicious code.
I would use either the json module or the ConfigParser module to hold my configuration.
If you have trouble choosing between those two I would recommend the json module. Json is a simple yet flexible format for structured data.