I've been looking at dynamic evaluation of Python code, and come across the eval() and compile() functions, and the exec statement.
Can someone please explain the difference between eval and exec, and how the different modes of compile() fit in?
The short answer, or TL;DR
Basically, eval is used to evaluate a single dynamically generated Python expression, and exec is used to execute dynamically generated Python code only for its side effects.
eval and exec have these two differences:
eval accepts only a single expression, exec can take a code block that has Python statements: loops, try: except:, class and function/method definitions and so on.
An expression in Python is whatever you can have as the value in a variable assignment:
a_variable = (anything you can put within these parentheses is an expression)
eval returns the value of the given expression, whereas exec ignores the return value from its code, and always returns None (in Python 2 it is a statement and cannot be used as an expression, so it really does not return anything).
In versions 1.0 - 2.7, exec was a statement, because CPython needed to produce a different kind of code object for functions that used exec for its side effects inside the function.
In Python 3, exec is a function; its use has no effect on the compiled bytecode of the function where it is used.
Thus basically:
>>> a = 5
>>> eval('37 + a') # it is an expression
42
>>> exec('37 + a') # it is an expression statement; value is ignored (None is returned)
>>> exec('a = 47') # modify a global variable as a side effect
>>> a
47
>>> eval('a = 47') # you cannot evaluate a statement
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
a = 47
^
SyntaxError: invalid syntax
The compile in 'exec' mode compiles any number of statements into a bytecode that implicitly always returns None, whereas in 'eval' mode it compiles a single expression into bytecode that returns the value of that expression.
>>> eval(compile('42', '<string>', 'exec')) # code returns None
>>> eval(compile('42', '<string>', 'eval')) # code returns 42
42
>>> exec(compile('42', '<string>', 'eval')) # code returns 42,
>>> # but ignored by exec
In the 'eval' mode (and thus with the eval function if a string is passed in), the compile raises an exception if the source code contains statements or anything else beyond a single expression:
>>> compile('for i in range(3): print(i)', '<string>', 'eval')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
for i in range(3): print(i)
^
SyntaxError: invalid syntax
Actually the statement "eval accepts only a single expression" applies only when a string (which contains Python source code) is passed to eval. Then it is internally compiled to bytecode using compile(source, '<string>', 'eval') This is where the difference really comes from.
If a code object (which contains Python bytecode) is passed to exec or eval, they behave identically, excepting for the fact that exec ignores the return value, still returning None always. So it is possible use eval to execute something that has statements, if you just compiled it into bytecode before instead of passing it as a string:
>>> eval(compile('if 1: print("Hello")', '<string>', 'exec'))
Hello
>>>
works without problems, even though the compiled code contains statements. It still returns None, because that is the return value of the code object returned from compile.
In the 'eval' mode (and thus with the eval function if a string is passed in), the compile raises an exception if the source code contains statements or anything else beyond a single expression:
>>> compile('for i in range(3): print(i)', '<string>'. 'eval')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
for i in range(3): print(i)
^
SyntaxError: invalid syntax
The longer answer, a.k.a the gory details
exec and eval
The exec function (which was a statement in Python 2) is used for executing a dynamically created statement or program:
>>> program = '''
for i in range(3):
print("Python is cool")
'''
>>> exec(program)
Python is cool
Python is cool
Python is cool
>>>
The eval function does the same for a single expression, and returns the value of the expression:
>>> a = 2
>>> my_calculation = '42 * a'
>>> result = eval(my_calculation)
>>> result
84
exec and eval both accept the program/expression to be run either as a str, unicode or bytes object containing source code, or as a code object which contains Python bytecode.
If a str/unicode/bytes containing source code was passed to exec, it behaves equivalently to:
exec(compile(source, '<string>', 'exec'))
and eval similarly behaves equivalent to:
eval(compile(source, '<string>', 'eval'))
Since all expressions can be used as statements in Python (these are called the Expr nodes in the Python abstract grammar; the opposite is not true), you can always use exec if you do not need the return value. That is to say, you can use either eval('my_func(42)') or exec('my_func(42)'), the difference being that eval returns the value returned by my_func, and exec discards it:
>>> def my_func(arg):
... print("Called with %d" % arg)
... return arg * 2
...
>>> exec('my_func(42)')
Called with 42
>>> eval('my_func(42)')
Called with 42
84
>>>
Of the 2, only exec accepts source code that contains statements, like def, for, while, import, or class, the assignment statement (a.k.a a = 42), or entire programs:
>>> exec('for i in range(3): print(i)')
0
1
2
>>> eval('for i in range(3): print(i)')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
for i in range(3): print(i)
^
SyntaxError: invalid syntax
Both exec and eval accept 2 additional positional arguments - globals and locals - which are the global and local variable scopes that the code sees. These default to the globals() and locals() within the scope that called exec or eval, but any dictionary can be used for globals and any mapping for locals (including dict of course). These can be used not only to restrict/modify the variables that the code sees, but are often also used for capturing the variables that the executed code creates:
>>> g = dict()
>>> l = dict()
>>> exec('global a; a, b = 123, 42', g, l)
>>> g['a']
123
>>> l
{'b': 42}
(If you display the value of the entire g, it would be much longer, because exec and eval add the built-ins module as __builtins__ to the globals automatically if it is missing).
In Python 2, the official syntax for the exec statement is actually exec code in globals, locals, as in
>>> exec 'global a; a, b = 123, 42' in g, l
However the alternate syntax exec(code, globals, locals) has always been accepted too (see below).
compile
The compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1) built-in can be used to speed up repeated invocations of the same code with exec or eval by compiling the source into a code object beforehand. The mode parameter controls the kind of code fragment the compile function accepts and the kind of bytecode it produces. The choices are 'eval', 'exec' and 'single':
'eval' mode expects a single expression, and will produce bytecode that when run will return the value of that expression:
>>> dis.dis(compile('a + b', '<string>', 'eval'))
1 0 LOAD_NAME 0 (a)
3 LOAD_NAME 1 (b)
6 BINARY_ADD
7 RETURN_VALUE
'exec' accepts any kinds of python constructs from single expressions to whole modules of code, and executes them as if they were module top-level statements. The code object returns None:
>>> dis.dis(compile('a + b', '<string>', 'exec'))
1 0 LOAD_NAME 0 (a)
3 LOAD_NAME 1 (b)
6 BINARY_ADD
7 POP_TOP <- discard result
8 LOAD_CONST 0 (None) <- load None on stack
11 RETURN_VALUE <- return top of stack
'single' is a limited form of 'exec' which accepts a source code containing a single statement (or multiple statements separated by ;) if the last statement is an expression statement, the resulting bytecode also prints the repr of the value of that expression to the standard output(!).
An if-elif-else chain, a loop with else, and try with its except, else and finally blocks is considered a single statement.
A source fragment containing 2 top-level statements is an error for the 'single', except in Python 2 there is a bug that sometimes allows multiple toplevel statements in the code; only the first is compiled; the rest are ignored:
In Python 2.7.8:
>>> exec(compile('a = 5\na = 6', '<string>', 'single'))
>>> a
5
And in Python 3.4.2:
>>> exec(compile('a = 5\na = 6', '<string>', 'single'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
a = 5
^
SyntaxError: multiple statements found while compiling a single statement
This is very useful for making interactive Python shells. However, the value of the expression is not returned, even if you eval the resulting code.
Thus greatest distinction of exec and eval actually comes from the compile function and its modes.
In addition to compiling source code to bytecode, compile supports compiling abstract syntax trees (parse trees of Python code) into code objects; and source code into abstract syntax trees (the ast.parse is written in Python and just calls compile(source, filename, mode, PyCF_ONLY_AST)); these are used for example for modifying source code on the fly, and also for dynamic code creation, as it is often easier to handle the code as a tree of nodes instead of lines of text in complex cases.
While eval only allows you to evaluate a string that contains a single expression, you can eval a whole statement, or even a whole module that has been compiled into bytecode; that is, with Python 2, print is a statement, and cannot be evalled directly:
>>> eval('for i in range(3): print("Python is cool")')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
for i in range(3): print("Python is cool")
^
SyntaxError: invalid syntax
compile it with 'exec' mode into a code object and you can eval it; the eval function will return None.
>>> code = compile('for i in range(3): print("Python is cool")',
'foo.py', 'exec')
>>> eval(code)
Python is cool
Python is cool
Python is cool
If one looks into eval and exec source code in CPython 3, this is very evident; they both call PyEval_EvalCode with same arguments, the only difference being that exec explicitly returns None.
Syntax differences of exec between Python 2 and Python 3
One of the major differences in Python 2 is that exec is a statement and eval is a built-in function (both are built-in functions in Python 3).
It is a well-known fact that the official syntax of exec in Python 2 is exec code [in globals[, locals]].
Unlike majority of the Python 2-to-3 porting guides seem to suggest, the exec statement in CPython 2 can be also used with syntax that looks exactly like the exec function invocation in Python 3. The reason is that Python 0.9.9 had the exec(code, globals, locals) built-in function! And that built-in function was replaced with exec statement somewhere before Python 1.0 release.
Since it was desirable to not break backwards compatibility with Python 0.9.9, Guido van Rossum added a compatibility hack in 1993: if the code was a tuple of length 2 or 3, and globals and locals were not passed into the exec statement otherwise, the code would be interpreted as if the 2nd and 3rd element of the tuple were the globals and locals respectively. The compatibility hack was not mentioned even in Python 1.4 documentation (the earliest available version online); and thus was not known to many writers of the porting guides and tools, until it was documented again in November 2012:
The first expression may also be a tuple of length 2 or 3. In this case, the optional parts must be omitted. The form exec(expr, globals) is equivalent to exec expr in globals, while the form exec(expr, globals, locals) is equivalent to exec expr in globals, locals. The tuple form of exec provides compatibility with Python 3, where exec is a function rather than a statement.
Yes, in CPython 2.7 that it is handily referred to as being a forward-compatibility option (why confuse people over that there is a backward compatibility option at all),
when it actually had been there for backward-compatibility for two decades.
Thus while exec is a statement in Python 1 and Python 2, and a built-in function in Python 3 and Python 0.9.9,
>>> exec("print(a)", globals(), {'a': 42})
42
has had identical behaviour in possibly every widely released Python version ever; and works in Jython 2.5.2, PyPy 2.3.1 (Python 2.7.6) and IronPython 2.6.1 too (kudos to them following the undocumented behaviour of CPython closely).
What you cannot do in Pythons 1.0 - 2.7 with its compatibility hack, is to store the return value of exec into a variable:
Python 2.7.11+ (default, Apr 17 2016, 14:00:29)
[GCC 5.3.1 20160413] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = exec('print(42)')
File "<stdin>", line 1
a = exec('print(42)')
^
SyntaxError: invalid syntax
(which wouldn't be useful in Python 3 either, as exec always returns None), or pass a reference to exec:
>>> call_later(exec, 'print(42)', delay=1000)
File "<stdin>", line 1
call_later(exec, 'print(42)', delay=1000)
^
SyntaxError: invalid syntax
Which a pattern that someone might actually have used, though unlikely;
Or use it in a list comprehension:
>>> [exec(i) for i in ['print(42)', 'print(foo)']
File "<stdin>", line 1
[exec(i) for i in ['print(42)', 'print(foo)']
^
SyntaxError: invalid syntax
which is abuse of list comprehensions (use a for loop instead!).
exec is not an expression: a statement in Python 2.x, and a function in Python 3.x. It compiles and immediately evaluates a statement or set of statement contained in a string. Example:
exec('print(5)') # prints 5.
# exec 'print 5' if you use Python 2.x, nor the exec neither the print is a function there
exec('print(5)\nprint(6)') # prints 5{newline}6.
exec('if True: print(6)') # prints 6.
exec('5') # does nothing and returns nothing.
eval is a built-in function (not a statement), which evaluates an expression and returns the value that expression produces. Example:
x = eval('5') # x <- 5
x = eval('%d + 6' % x) # x <- 11
x = eval('abs(%d)' % -100) # x <- 100
x = eval('x = 5') # INVALID; assignment is not an expression.
x = eval('if 1: x = 4') # INVALID; if is a statement, not an expression.
compile is a lower level version of exec and eval. It does not execute or evaluate your statements or expressions, but returns a code object that can do it. The modes are as follows:
compile(string, '', 'eval') returns the code object that would have been executed had you done eval(string). Note that you cannot use statements in this mode; only a (single) expression is valid.
compile(string, '', 'exec') returns the code object that would have been executed had you done exec(string). You can use any number of statements here.
compile(string, '', 'single') is like the exec mode but expects exactly one expression/statement, eg compile('a=1 if 1 else 3', 'myf', mode='single')
exec is for statement and does not return anything.
eval is for expression and returns value of expression.
expression means "something" while statement means "do something".
I am currently developing an automated function tester in Python.
The purpose of this application is to automatically test if functions are returning an expected return type based on their defined hints.
Currently I have two test functions (one which fails and one which passes), along with the rest of my code in one file. My code utilizes the globals() command in order to scan the Python file for all existing functions and to isolate user-made functions and exclude the default ones.
This initial iteration works well. Now I am trying to import the function and use it from another .py file.
When I run it in the other .py file it still returns results for the functions from the original file instead of the new test-cases in the new file.
Original File - The Main Application
from math import floor
import random
#declaring test variables
test_string = 'test_string'
test_float = float(random.random() * 10)
test_int = int(floor(random.random() * 10))
#Currently supported test types (input and return)
supported_types = ['int', 'float', 'str']
autotest_result = {}
def int_ret(number: int) -> str:
string = "cactusmonster"
return string
def false_test(number: int) -> str:
floating = 3.2222
return floating
def test_typematching():
for name in list(globals()):
if not name.startswith('__'):
try:
return_type = str((globals()[name].__annotations__)['return'])
autotest_result.update({name: return_type.replace("<class '", "").replace("'>", "")})
except:
continue
for func in autotest_result:
if autotest_result[func] != None:
this_func = globals()[func].__annotations__
for arg in this_func:
if arg != 'return':
input_type = str(this_func[arg]).replace("<class '", "").replace("'>", "")
for available in supported_types:
if available == input_type:
func_return = globals()[func]("test_" + input_type)
func_return = globals()[func]("test_" + input_type)
actual_return_type = str(type(func_return)).replace("<class '", "").replace("'>", "")
if actual_return_type == autotest_result[func]:
autotest_result[func] = 'Passed'
else:
autotest_result[func] = 'Failed'
return autotest_result
Test File - Where I Am Importing The "test_typematching()" Function
from auto_test import test_typematching
print(test_typematching())
def int_ret_newfile(number: int) -> str:
string="cactusmonster"
# print(string)
# return type(number)
return string
Regardless if I run my main "auto_test.py" file or the "tester.py" file, I still get the following output:
{'int_ret': 'Passed', 'false_test': 'Failed'}
I am guessing this means that even when I am running the function from auto_test.py on my tester.py file it still just scans itself. I would like it to scan the file where the function is currently being called. For example, I expect it to test the int_ret_newfile function of tester.py.
Any advice or help would be much appreciated.
globals() is a bit of a misnomer. It gets the calling module's __dict__. (Python's true "global" namespace is actually builtins.)
How can globals() get its caller's __dict__ when it's defined in the builtins module? Here's a clue:
PyObject *
PyEval_GetGlobals(void)
{
PyThreadState *tstate = _PyThreadState_GET();
PyFrameObject *current_frame = _PyEval_GetFrame(tstate);
if (current_frame == NULL) {
return NULL;
}
assert(current_frame->f_globals != NULL);
return current_frame->f_globals;
}
globals() is one of those builtins that's implemented in C (in CPython), but you get the gist. It reads the frame globals from the current stack frame, so in Python,
import inspect
inspect.currentframe().f_globals
would do the same thing as globals(). But you can't just put this in a function and expect it to work the same way, because calling it would add a stack frame, and that frame's globals depends on the function's .__globals__ attribute, which is set to the .__dict__ of the module that defined it. You want the caller's frame.
def myglobals():
"""Behaves like the builtin globals(), but written in Python!"""
return inspect.currentframe().f_back.f_globals
You could do the same thing in test_typematching. But walking up the stack to the previous frame like that is a weird thing to do. It can be surprising and brittle. It amounts to passing the caller's frame as an implicit hidden argument, something that normally is not supposed to matter. Consider what happens if you wrap it in a decorator. Now which stack frame are you getting the globals from?
So really, you should be passing in globals() as an explicit argument to test_typematching(), like test_typematching(globals()). A defined and documented parameter would be much less confusing than implicit introspection. "Explicit is better than implicit".
Still, Python's standard library does do this kind of thing occasionally, with globals() itself being a notable example. And exec() can use the current namespace if you don't give it a different one. It's also how super() can now work without arguments in Python 3. So stack frame inspection does have precedent for this kind of use case.
I want to evaluate a lambda expression using the built-in eval function, with a variable y defined in the 'locals' argument. Sadly the result function doesn't work:
>>>x = eval('lambda: print(y)',{},{'y':2})
>>>x()
Traceback (most recent call last):
File "<pyshell#75>", line 1, in <module>
x()
File "<string>", line 1, in <lambda>
NameError: name 'y' is not defined
But with y defined in the 'globals' argument, it does work:
>>> x = eval('lambda: print(y)', {'y': 2},{})
>>> x()
2
As I understand, the lambda expression should have captured the whole current frame including all the variables defined in the globals and locals arguments.
So why does Python behave like this?
Quite simply: passing a populated locals directory doesn't change the way python parses a function code and decides which names are locals and which are globals.
Local names are arguments names and names that are bound within the function's body and not explicitely declared globals or non-locals. Here, y is not an argument, and is not bound within the function's body (which is impossible in a lambda anyway) so it is marked by the compiler as global.
Now those global and local environment are those used to evaluate the expression itself (here the full 'lambda: print(y)' expression), not "the local environment for the lambda's body", so even if there was a way to make y local to the function's body (hint: there's one - but it won't solve your problem) this name would still not be "automagically" set to the the 'y' value in the locals dict passed to eval.
BUT this is actually not a problem - the function created by eval("lambda: y", {"y":42}) captures the globals dict passed to eval and uses it in place of the module/script/whatever globals, so it will work as expected:
Python 3.4.3 (default, Nov 28 2017, 16:41:13)
[GCC 4.8.4] on linux
>>> f = eval("lambda: y+2", {'y':2}, {})
>>> f()
4
>>> y = 42
>>> f()
4
Now you have the explanation, I whish to add that eval() is very dangerous and most often than not there's a much better solution, depending on the exact problem you're trying to solve. You didn't provide any context so it's impossible to tell more but I have to say I actually have a hard time thinking of a concrete use-case for such a thing as f = eval("lambda: whatever").
x = eval('lambda: print(y)',{},{'y':2}) is not equal to this line x = eval('lambda: print(y)', {'y': 2},{}) in first part change params order and it should work
This is my first question so please be nice :) I am rather new to Python but I am very experienced in other programming languages (e.g. C++).
UPDATE 2 - Solution Found
Thanks everyone for helping :) As the solution is "hidden" in comments I will repost it here.
Instead of
file_symbols = {}
the variable local_symbol must initially be added to the file_symbols dictionary:
file_symbols = { "local_symbol" : local_symbol }
For anyone reading this: all variable / class names posted here are not to be understood as actual useful names as these examples are synthetic in nature ;)
Well... now I have to figure out the FULL meaning of:
exec compiled_code in file_symbols
So far I thought it would do nothing more than updating the dictionary file_symbols with the symbols found in compiled_code.
But it actually does a bit more as it seems! :)
UPDATE 1
Ok, my sample project below seems to be too simple to show the actual problem. Anyway, thanks for your already provided support! :)
In fact I want to first compile multiple *.py files which need access to a local symbol (class instance). All symbols coming from these compiled files shall be collected and then be used as an environment for OTHER code objects.
So I really need to do this
(note the following code shows the concept, not actual executable code):
class Functions:
(...)
global_symbols = {}
local_symbol = Functions()
# note that the external files need to access local_symbol to call its functions!
for file in external_files:
code = file.load()
compiled_code = compile(code, "<string>", "exec")
file_symbols = {}
exec compiled_code in file_symbols
global_symbols.update(file_symbols)
some_code = another_file.load()
compiled_code = compile(some_code, "<string>", "exec")
exec(compiled_code, global_symbols)
In this example the line
exec compiled_code in file_symbols
produces a NameError() - because there is no way they could access local_symbol as it is not defined anywhere in the external files although it shall be used!
So the question is how to provide access to local_symbol for the external_files so that they can call the instance's functions??
My import hook solution that some of you regard as "hack" was the only working solution so far. I would love to use a simpler one if there is any!
Thanks again :)
My initial question was this:
So here we go. What I intend to do is advanced stuff and I did not find a solution to my problem neither here nor anywhere else.
Assume the following code in Python (2.6.x / 2.7.x):
class Functions:
def __init__(self):
(...)
def func_1(...):
(...)
def func_2(...):
(...)
(...)
def func_n(...):
(...)
functions = Functions()
code = loadSomeFile(...)
compiled_code = compile(code, "<string>", "exec")
(...)
global_env = {}
local_env = {"func":functions}
exec(compiled_code, global_env, local_env)
where code in the example above is loaded from a file with a content that might look like this:
import something
(...)
def aFunction(...):
a = func.func_1(...)
b = func.func_2(...)
return a * b
(...)
aFunction()
Please note that (...) in the code above means that there might be more code that I left out for the sake of simplicity.
The problem I encounter in my example is that the compiler raises an error for this line:
compiled_code = compile(code, "<string>", "exec")
I will get this error: NameError("global name 'func' is not defined")
This error is totally understandable as the compiler can't bind to any global symbol with the name "func". But I still want to compile the code this way.
So the obvious question is:
How can I define global symbols that can be used by the compiler for compile() statements so that the compiler will bind any "unknown" symbol to an object of my choice?
In my example I would like to define a global symbol func that is bound to an instance of class Functions so that the compiler will find this symbol when compiling code which makes use of func as seen in the example above.
So how can this be accomplished?
Important:
Please note that I am aware that directly executing the code using exec(...) would fix the compilation problem because the dict local_env in the example above would provide the symbol that is required for successful execution. HOWEVER I can't do it this way because the code that shall be compiled is not small at all. It might consist of hundreds of code lines and this code is also not executed only once but many many times.
So for performance reasons I really need to compile the code rather than directly executing it.
Thanks for helping me :)
Don't provide separate globals and locals dicts to exec. That causes the executed code to behave as if it's embedded in a class definition. That means that any variable lookups in functions defined in the executed code bypass locals.
>>> exec("""
... def f():
... print a
... f()""", {}, {"a": 3})
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
File "<string>", line 4, in <module>
File "<string>", line 3, in f
NameError: global name 'a' is not defined
>>> exec("""
... def f():
... print a
... f()""", {"a": 3})
3
Just pass a globals dict.
The simple explanation is that you pass in func in the local namespace. aFunction does not have access to the locals you passed in (it has its own locals). All aFunction has access to is its own locals and its module's globals. func is in neither of these and so the function call fails.
Most normal modules work with their globals and locals as the same namespace (you can check this out yourself assert globals() is locals()). This is why you can define things at the module level and have them available to any defined functions (all names in a module are automatically global).
To make this work you need to make locals and globals the same dict, or just not pass locals at all. If you don't want globals to be mutated, then just copy globals and then add func to it.
src = """
def aFunction():
a = func.func_1()
b = func.func_2()
return a + b
value = aFunction()
"""
class Functions:
def func_1(self):
return "first"
def func_2(self):
return "second"
functions = Functions()
compiled_code = compile(src, "<string>", "exec")
global_env = {}
local_env = {"func":functions}
namespace = dict(global_env)
namespace.update(local_env)
exec(compiled_code, namespace)
print(namespace["value"])
it is an interesting question, thanks for posting it. I've been taking a look at how to change the globals table in compile time. Apparently, you can call the __import__() function directly and:
pass your globals in order to determine how to interpret the name in a package context.
Source: Package documentation
Well, obviously it works by simply adding (and potentially removing - if using multiple "func" instances) the instance to sys.modules:
sys.modules["func"] = functions
(...)
compiled_code = compile(code, "<string>", "exec")
The complete and working solution including an importer hook (which intercepts "import func" lines in source code files) would look like this:
import sys
class Functions:
def __init__(self):
(...)
def func_1(...):
(...)
def func_2(...):
(...)
(...)
def func_n(...):
(...)
functions = Functions()
code = loadSomeFile(...)
hook_name = "func"
class ImporterHook:
def __init__(self, path):
if path != hook_name:
raise ImportError()
def find_module(self, fullname, path=None):
return self
def load_module(self, path):
if sys.modules.has_key(path):
return sys.modules[path]
else:
sys.modules[path] = functions
return functions
sys.path_hooks.append(ImporterHook)
sys.path.insert(0, hook_name)
compiled_code = compile(code, "<string>", "exec")
(...)
exec(compiled_code)
Not so hard as it seems :) For more information see here:
https://www.python.org/dev/peps/pep-0302/#specification-part-2-registering-hooks
and here:
https://pymotw.com/2/sys/imports.html
THANKS :)
I'm writing Python 3 code and for some reason I want to run everything just in memory and save no files on disk. I managed to solve almost all of my problems so far by reading answers here, but I'm stuck on these lines:
>>> code = compile(source, filename, 'exec')
>>> exec code in module.__dict__
I don't really understand what the second line does, since I have "in" connected with loops and testing whether something is in some set or not which is not this case.
So, what does the second line do? And what is its Python 3 equivalent since in py3 is exec function, not keyword?
exec code in module.__dict__
means execute the commands in the file or string called 'code', taking global and local variables referred to in 'code' from module.__dict__ and storing local and global variables created in 'code' into the dictionary module.__dict__
See http://docs.python.org/reference/simple_stmts.html#exec
Eg:
In [51]: mydict={}
In [52]: exec "val1=100" in mydict
In [53]: mydict['val1']
Out[53]: 100
Eg2:
In [54]: mydict={}
In [55]: mydict['val2']=200
In [56]: exec "val1=val2" in mydict
In [57]: mydict.keys()
Out[57]: ['__builtins__', 'val2', 'val1']
In [58]: mydict['val2']
Out[58]: 200
In [59]: mydict['val1']
Out[59]: 200
The in keyword specifies a dictionary to use for the global and local namespaces. From the python 2 documentation for exec:
In all cases, if the optional parts are omitted, the code is executed in the current scope. If only the first expression after in is specified, it should be a dictionary, which will be used for both the global and the local variables.
The python 3 equivalent is thus exec(code, module.__dict__).
In Python 3 that exec line would translate to:
exec(code, module.__dict__)
Excerpts from the Python 3 help files:
exec(object[, globals[, locals]])
In all cases, if the optional parts are omitted, the code is executed
in the current scope. If only globals is provided, it must be a
dictionary, which will be used for both the global and the local
variables. If globals and locals are given, they are used for the
global and local variables, respectively. If provided, locals can be
any mapping object.