I sometimes use embed at a certain point in a script to quickly flesh out some local functionality. Minimal example:
#!/usr/bin/env python
# ...
import IPython
IPython.embed()
Developing a local function often requires a new import. However, importing a module in the IPython session does not seem to work, when used in a function. For instance:
In [1]: import os
In [2]: def local_func(): return os.path.sep
In [3]: local_func()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-3-f0e5d4635432> in <module>()
----> 1 local_func()
<ipython-input-2-c530ce486a2b> in local_func()
----> 1 def local_func(): return os.path.sep
NameError: global name 'os' is not defined
This is rather confusing, especially since I can even use tab completion to write os.path.sep.
I noticed that the problem is even more fundamental: In general, functions created in the IPython embed session do not close over variables from the embed scope. For instance, this fails as well:
In [4]: x = 0
In [5]: def local_func(): return x
In [6]: local_func()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-6-f0e5d4635432> in <module>()
----> 1 local_func()
<ipython-input-5-2116e9532e5c> in local_func()
----> 1 def local_func(): return x
NameError: global name 'x' is not defined
Module names are probably just the most common thing to "close over"...
Is there any solution to this problem?
Update: The problem not only applies for closures, but also nested list comprehensions.
Disclaimer: I'll post an (unsatisfactory) answer to the question myself -- still hoping for a better solution though.
I also had the same problem. I used this trick to deal with the case when the embed() is called outside a function, so that globals() and locals() should be the same dictionary.
The simplest way is to call the following function after ipython is launched
ipy = get_ipython()
setattr(ipy.__class__, 'user_global_ns', property(lambda self: self.user_ns))
Another way is to subclass InteractiveShellEmbed
class InteractiveShellEmbedEnhanced(InteractiveShellEmbed):
#property
def user_global_ns(self):
if getattr(self, 'embedded_outside_func', False):
return self.user_ns
else:
return self.user_module.__dict__
def init_frame(self, frame):
if frame.f_code.co_name == '<module>':
self.embedded_outside_func = True
else:
self.embedded_outside_func = False
and modify slightly the code of IPython.terminal.embed.embed() so that in it all InteractiveShellEmbed is changed to InteractiveShellEmbedEnhanced and call shell.init_frame(frame) after the line shell = InteractiveShellEmbed.instance(...).
This is based on the following observations:
In an ipython session, we always have id(globals()) == id(ipy.user_module.__dict__) == id(ipy.user_global_ns) (user_global_ns is a class property of the super class of InteractiveShellEmbed, which returns ipy.user_module.__dict__)
Also we have id(locals()) == id(ipy.user_ns)
For normal ipython session, id(locals()) == id(globals())
user_global_ns (a property) and user_ns (a dict) define the execution context
In embedded ipython, ipy.user_module and ipy.user_ns are set in function ipy.__call__() and passed to ipy.mainloop(). They are not the same object since ipy.user_ns is constructed inside the functions.
If you are to launch ipython outside a function (like in a script), then it is safe to assume the globals() should be identical to locals().
With this setup, the following code should work while not working using the default embedded shell:
a=3
(lambda :a)() # default behavior: name 'a' is not defined
import time
(lambda: time.time())() # default behavior: name 'time' is not defined
(default behavior is due to a and time are not added to globals() and ipython does not make closures for local functions (the lambdas defined above) and insists to look up the variables in global scope. search closure in this page)
Update: Again only a work-around, but somewhat simpler: globals().update(locals())
I don't have a general solution, but at least a work-around: After defining a local function, it is possible to add the locals() of the session to the func_globals of the function just defined, e.g.:
In [1]: import os
In [2]: def local_func(): return os.path.sep
In [3]: local_func.func_globals.update(locals())
In [4]: local_func()
Out[4]: '/'
However, one should be aware that this is only a "manual closure" and will not work as a regular closure in cases like this:
In [1]: x = 1
In [2]: def local_func(): return x
In [3]: local_func.func_globals.update(locals())
In [4]: local_func()
Out[4]: 1
In [5]: x = 42
In [6]: local_func() # true closure would return 42
Out[6]: 1
In [7]: local_func.func_globals.update(locals()) # but need to update again
In [8]: local_func()
Out[8]: 42
At least it can solve the notorious global name '...' is not defined problem for imports.
This "bug" of IPython.embed() is still there today. But from the comment of developer, we could instead use:
from IPython import start_ipython
... # some random code
start_ipython(argv=[], user_ns = locals())
Or more aggressively, replace the user_ns = locals() to user_ns = locals() | globals()
Related
Let's say I have two modules:
a.py
value = 3
def x()
return value
b.py
from a import x
value = 4
My goal is to use the functionality of a.x in b, but change the value returned by the function. Specifically, value will be looked up with a as the source of global names even when I run b.x(). I am basically trying to create a copy of the function object in b.x that is identical to a.x but uses b to get its globals. Is there a reasonably straightforward way to do that?
Here is an example:
import a, b
print(a.x(), b.x())
The result is currently 3 3, but I want it to be 3 4.
I have come up with two convoluted methods that work, but I am not happy with either one:
Re-define x in module b using copy-and paste. The real function is much more complex than shown, so this doesn't sit right with me.
Define a parameter that can be passed in to x and just use the module's value:
def x(value):
return value
This adds a burden on the user that I want to avoid, and does not really solve the problem.
Is there a way to modify where the function gets its globals somehow?
I've come up with a solution through a mixture of guess-and-check and research. You can do pretty much exactly what I proposed in the question: copy a function object and replace its __globals__ attribute.
I am using Python 3, so here is a modified version of the answer to the question linked above, with an added option to override the globals:
import copy
import types
import functools
def copy_func(f, globals=None, module=None):
"""Based on https://stackoverflow.com/a/13503277/2988730 (#unutbu)"""
if globals is None:
globals = f.__globals__
g = types.FunctionType(f.__code__, globals, name=f.__name__,
argdefs=f.__defaults__, closure=f.__closure__)
g = functools.update_wrapper(g, f)
if module is not None:
g.__module__ = module
g.__kwdefaults__ = copy.copy(f.__kwdefaults__)
return g
b.py
from a import x
value = 4
x = copy_func(x, globals(), __name__)
The __globals__ attribute is read-only, which is why it must be passed to the constructor of FunctionType. The __globals__ reference of an existing function object can not be changed.
Postscript
I've used this enough times now that it's implemented in a utility library I wrote and maintain called haggis. See haggis.objects.copy_func.
So I found a way to (sort of) do this, although I don't think it entirely solves your problems. Using inspect, you can access the global variables of the file calling your function. So if you set up your files like so:
a.py
import inspect
value = 3
def a():
return inspect.stack()[1][0].f_globals['value']
b.py
from a import a
value = 5
print(a())
The output is 5, instead of 3. However, if you imported both of these into a third file, it would look for the globals of the third file. Just wanted to share this snippet however.
I had the same problem. But then I remembered eval was a thing.
Here's a much shorter version(if you don't need arguments):
b.py:
from a import x as xx
# Define globals for the function here
glob = {'value': 4}
def x():
return eval(xx.__code__, glob)
Hopefully after 2 years it'll still be helpful
I want to modify a global variable from a function in Python 2.7
x = 0
def func():
global x
x = 2
If I load this code in the interpreter, and then run func(), x remains 0. How do I modify the value of x from within a function?
EDIT: Here's a screenshot of the interpreter and source code. I'm not sure why it works for others and not for me.
http://img18.imageshack.us/img18/9567/screenshotfrom201304222.png
This is a very interesting situation. When I ran your code from an interpreter with from mytest import * I encountered the same issue:
>>> from mytest import *
>>> x
0
>>> func()
>>> x
0
However, when I just did import mytest and ran it from there:
>>> import mytest
>>> mytest.x
0
>>> mytest.func()
>>> mytest.x
2
It turned out fine! The reason, I believe, comes from a line in http://docs.python.org/2/reference/simple_stmts.html#the-global-statement:
Names listed in a global statement must not be defined as formal
parameters or in a for loop control target, class definition, function
definition, or import statement.
Looks like because it is a parameter in your import statement (by importing all), global is having trouble with it. Do you need to import *, or can you simply import the module whole?
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
declaring a global dynamic variable in python
>>> def f():
global cat
exec 'cat'+'="meow"'
return
>>> f()
>>> cat
Traceback (most recent call last):
File "<pyshell#23>", line 1, in <module>
cat
NameError: name 'cat' is not defined
This is just a stripped down example of the issue I've come across. In my actual script, I need various instances of a class be created and named, hence the exec statement.
Just calling
exec 'cat'+'="meow"'
directly in the shell works fine, but as soon as it's packed in a function, it doesn't seem to work anymore.
I still don't understand why you are using exec, its a bad design choice and alternatives are usually easier, for example instead of global and then something else you could simply do this
ns = {}
def f():
ns["cat"] = "miow"
print ns
Now isn't that cleaner?
looks like the exec ignores the global, the documentation is a bit vague. but this works:
>>> def f():
... global cat
... exec 'global cat; cat'+'="meow"'
...
>>>
>>> f()
>>> cat
'meow'
Certain list comprehensions don't work properly when I embed IPython 0.10 as per the instructions. What's going on with my global namespace?
$ python
>>> import IPython.Shell
>>> IPython.Shell.IPShellEmbed()()
In [1]: def bar(): pass
...:
In [2]: list(bar() for i in range(10))
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
/tmp/<ipython console>
/tmp/<ipython console> in <generator expression>([outmost-iterable])
NameError: global name 'bar' is not defined
List comprehensions are fine, this works:
[bar() for i in range(10)]
It's generator expressions (which is what you passed to that list() call) that are not fine:
gexpr = (bar() for i in range(10))
list(gexpr)
The difference: items in the list comprehension are evaluated at definition time. Items in the generator expression are evaluated when next() is called (e.g. through iteration when you pass it to list()), so it must keep a reference to the scope where it is defined. That scope reference seems to be incorrectly handled; most probably that's simply an IPython bug.
Seems to work, but IPython thinks it's the main program. So after instantiating IPShell, a crash shows "whoops, IPython crashed".
import IPython.Shell
ipshell = IPython.Shell.IPShell(argv=[], user_ns={'root':root})
ipshell.mainloop()
For this situation, I've found the following updates the scope so that bar() can be found:
globals().update(locals())
I'm having a bit of trouble understanding what's going wrong with the following function:
def ness():
pie='yum'
vars()[pie]=4
print vars()[pie]
print yum
So When I run that I get this result:
>>> ness()
4
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in ness
NameError: global name 'yum' is not defined
If I don't write it as a function and just type it in on the command line one line at a time it works fine, like so:
>>> pie='yum'
>>> vars()[pie]=4
>>> print vars()[pie]
4
>>> print yum
4
>>>
Edit:
Suppose I wanted to make things a bit more complicated than this and instead of setting yum to a value and printing that value, I define some functions, and want to call one of them based on some input:
def ness(choo):
dic={}
dessert=()
dnum=[10,100]
desserts='pie'
dic[dessert]=str(desserts[bisect(dnum,choo)])
vars()[dic[dessert]]()
def p():
print 'ummmm ummm'
def i():
print 'hooo aaaaa'
def e():
print 'woooo'
So when I call ness I get a key error:
>>> ness(3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in ness
KeyError: 'p'
Now I know I can do things like this with some elif statements, but I'm wondering if this would work too, and if using bisect like this would be more efficient (say if i need to check 1000 values of choo) than using elifs.
Thanks much for the assistance.
vars() within a function gives you the local namespace, just like locals() -- see the docs. Outside of a function (e.g. at the prompt) locals() (and vars() of course) gives you the module's global namespace, just like globals(). As the docs say, trying to assign to a function's local variable through locals() (or equivalently, vars() inside a function) is not supported in Python. If you want to assign to a global variable, as you do when you're at the prompt (or otherwise outside of a function), use globals() instead of vars() (maybe not the cleanest approach -- global variables are understandably frowned upon -- but it does work).
There is way to do it with exec
>>> def ness():
... pie='yum'
... exec pie+"=4"
... print vars()[pie]
... print yum
...
>>>
>>> ness()
4
4
But Instead of doing that, using a new dict is better and safe
>>> def ness():
... dic={}
... pie='yum'
... dic[pie]=4
... print dic[pie]
... print dic['yum']
...
>>> ness()
4
4
>>>
It's not safe to modify the dict returned by vars()
vars([object])ΒΆ
Without an argument, act like locals().
With a module, class or class instance object as argument (or
anything else that has a dict
attribute), return that attribute.
Note
The returned dictionary should not be modified: the effects on the
corresponding symbol table are
undefined.
Your second example is a special case. vars() is equivalent to globals() in the global namespace, and the dict returned by globals() behaves as you would expect ( but is frowned upon )
>>> id(vars()),id(globals())
(3085426868L, 3085426868L)
vars() is equivalent to locals(), which in the case of the function is the local variables in its scope and at in the interactive interpreter at the scope you have it, vars() is globals(). locals() is for reading only; the effects of trying to change it are undefined (and in practice, just doesn't work). globals() can be modified, but you still should never directly put anything in the dict it returns.
[Edit: I must be wrong here, since the 'exec' example works.]
As everyone points out, it's a bad idea to modify vars(). You can understand the error, though, by realizing that python in some sense doesn't "see" that "yum" is a local. "print yum" is still resolved as a global reference; this happens before any code is executed.
It's the same reason you get an UnboundLocalError from:
>>> y = 100
>>> def foo(x):
... if x == 1:
... y = 10
... print y
...
>>> foo(1)
10
>>> foo(2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in foo
UnboundLocalError: local variable 'y' referenced before assignment