How to export a variable from PDB? - python

Imagine the following scenario: a script is started from the IPython shell and at a break point the python debugger is called. Using the PDB commands one can analyze the code and variables at this point. But often it turns out that the values of the variables call for a deeper research.
Is it possible to export the value of a variable to the IPython shell?
My specific use case:
I struggle with a quite huge numpy array which does not seem to have the correct values. I know that I can run any python commands from the python debugger, but it would be helpful to save the values of the variable at different break points and to use all of them at IPython shell. I am imaging something like
ipdb> global var1; var1 = var
ipdb> continue
...
ipdb> global var2; var2 = var
ipdb> continue
...
In [2]: abs(var1 - var2) # do some interesting calculations with IPython

You can use globals():
ipdb>__name__
'my_module'
ipdb> get_var = 'a value'
ipdb> globals()['myvar'] = get_var
ipdb> q
In [11]: my_module.myvar
Out[11]: 'a value'
This assumes the break point is set in my_module.py, so we are editing the globals of the module my_module.

Not a pretty solution, but working:
ipdb> import cPickle; f=open('/tmp/dump1','w+'); cPickle.dump(var,f); f.close()
...
ipdb> import cPickle; f=open('/tmp/dump2','w+'); cPickle.dump(var,f); f.close()
then
In [2]: var1 = cPickle.load(open('/tmp/dump1'))
In [3]: var2 = cPickle.load(open('/tmp/dump2'))

You need to distinguish different globals().
For example, suppose we have a module: mymodule.py
foo = 100
def test():
bar = 200
return bar
We run it under the control of pdb.
>>> import pdb
>>> import mymodule
>>> foobar = 300
>>> pdb.run('mymodule.test()')
> <string>(1)<module>()
(Pdb) print foobar
300
(Pdb) print foo
*** NameError: name 'foo' is not defined
(Pdb) global foobar2; foobar2 = 301
(Pdb) print foobar2
301
At the beginning, namely, before executing test(), the environment in pdb is your current globals(). Thus foobar is defined, while foo is not defined.
Then we execute test() and stop at the end of bar = 200
-> bar = 200
(Pdb) print bar
200
(Pdb) print foo
100
(Pdb) print foobar
*** NameError: name 'foobar' is not defined
(Pdb) global foo2; foo2 = 101
(Pdb) print foo2
101
(Pdb) c
>>>
The environment in pdb has been changed. It uses mymodule's globals() in test(). Thus 'foobaris not defined. whilefoo` is defined.
We have exported two variables foobar2 and foo2. But they live in different scopes.
>>> foobar2
301
>>> mymodule.foobar2
Traceback (most recent call last):
File "<pyshell#16>", line 1, in <module>
mymodule.foobar2
AttributeError: 'module' object has no attribute 'foobar2'
>>> mymodule.foo2
101
>>> foo2
Traceback (most recent call last):
File "<pyshell#18>", line 1, in <module>
foo2
NameError: name 'foo2' is not defined
You have already found the solution. But it works slightly differently.

Related

Python Import Star Creating Hidden Namespace?

I recently ran into some unusual behavior.
foo.py
a = 0
def bar():
print (a)
Console:
>>> import foo
>>> foo.bar()
0
>>> foo.a = 10
>>> foo.bar()
10
Console:
>>> from foo import *
>>> bar()
0
>>> a
0
>>> a = 10
>>> a
10
>>> bar()
0
I'm inferring that import * is actually creating two copies of a - one in the global namespace and one inside the foo module which cannot be accessed. Is this behavior explained/documented anywhere? I'm having trouble figuring out what to search for.
This seems like a notable and unexpected consequence of import * but for some reason I've never seen it brought up before.
There is no such thing as a hidden namespace in Python and the described behaviour is the normal and expected one.
You should read https://docs.python.org/3/tutorial/modules.html#more-on-modules in order to understand better how the globals do do work.

Using not defined variable in function body [closed]

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 9 years ago.
There is this code:
def f():
x = m
m = 2
def g():
x = m
f() # UnboundLocalError: local variable 'm' referenced before assignment
g() # NameError: global name 'm' is not defined
In both function bodies there is used variable m which is not defined when used but the error messages are different. Do Python know what variables are defined in function before using them (like in function f)? Why the error messages are different?
If there is an assignment to a variable anywhere in a function, then it is considered a local variable everywhere in that function. This means that for the function f(), even though the assignment to m happens after the attempt to access m, the line x = m will only look for the name m within the local scope. This is why the error message for f() refers to m as a local variable.
In the function g() there is no assigment to m, so the line x = m will look for m using the order described here:
the innermost scope, which is searched first, contains the local
names
the scopes of any enclosing functions, which are searched starting with the nearest enclosing scope, contains non-local, but also non-global names
the next-to-last scope contains the current module’s global names
the outermost scope (searched last) is the namespace containing built-in names
The error message for g(), "global name 'm' is not defined", refers to the global scope because that is the last location that was searched (except built-in, but it would be confusing to have a message like "the name 'm' was not found in the built-in namespace").
Note that you can use the global or nonlocal statements to change this behavior (nonlocal only exists in Python 3.x).
Python checks it as soon as you call it.
When importing, and typing directly into the interpreter, it only cares if you broke any syntax rules. it doesnt care about locals or globals at this level.
>>> def foo():
... print locals()
... bar = 34
... print locals()
... DIP = SET
...
>>>
>>> foo()
{}
{'bar': 34}
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in foo
NameError: global name 'SET' is not defined
it runs from top to bottom, and checks locals() and globals() if it sees that variable then its okay and does whatever with it.
it even works with definitions and sub definitions.. or anything else you are assigning
>>> def foo():
... bar()
... def bar():
... print("never gonna give you up")
...
>>>
>>> foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in foo
UnboundLocalError: local variable 'bar' referenced before assignment
Yes. If you assign to a variable at any point in a function (without using the global keyword), Python treats all references to that name in the function as local.
When you execute a function, you actually invoke __call()__ on a function object.
The function object is created in the scripts global namespace when the script is parsed.
Created, but not executed.
As part of the parsing process, the objects namespace is computed. So the interpreter can actually know which variable exists and when.
def f():
x = m
m = 2
When the above function parsed python thinks that m is local variable as it finds m = 2, so when the function is actually called x = m will raise error as m is not defined yet in local scope.
def g():
x = m
In this python thinks that m is going to be some value from the global scope, it searches the global namespace first and then built-ins, but when m is not found anywhere the error is raised.
>>> m = 1
>>> def g():
x = m
>>> g() #works fine because `m` is found in global scope
>>> def g():
x = sum
>>> g() # sum is found in built-ins
To modify a global variable use global:
>>> m = 1
>>> def g():
global m
m += 1
...
>>> g()
>>> m
2
You need to use global m inside the function

Set the name of a python object/variable with a string

Most people will probably say this is a bad idea. I want to use the content of a string as the name of a variable. I want to accomplish the dreaded:
s = 'x'
x = 1
where the variable name x comes (somehow?) from the string s.
To answer the "why?", say I have a global default value for x that I want the option of overriding in a function. But:
x = 0
def f(**kw):
print x
f(x=1)
prints 0 not 1. If I could use the strings in kw.keys() to reassign x (or any other globally set variables) then I'd be happy.
I realize that this works for reassigning x in f:
x = 0
def f(x=x):
print x
f(x=1)
But I want to do this for cases where there are MANY variables in my namespace that I might at some point want to override without rewriting every function definition in my module.
Check out exec
>>> print x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
>>> s = 'x'
>>> exec(s + " = 1")
>>> print x
1
See also: How can I assign the value of a variable using eval in python?
After a little experimentation, this also seems to work:
>>> print x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
>>> s = 'x'
>>> globals()[s] = 1
>>> print x
1
You should reconsider using the global keyword (http://docs.python.org/2.7/reference/simple_stmts.html#the-global-statement)
I recommend taking a look at Use of "global" keyword in Python as well.
Also, as promanow suggested, using a global mutable object might be a better approach.
However, keep in mind that having large amounts of code that might modify or depend on a mutating global is very scary idea, from a maintenance perspective. Tread carefully.
Assignment via function arguments is not obvious and therefore is not considered Pythonic (Python is not C). See import this.
The cheap way that I've done flat configuration is through a global partial dict:
In [1]: from functools import partial
In [2]: config = partial(dict)
In [3]: def f(x=None):
...: print x or config.x
...:
In [4]: config.x = 'foo'
In [5]: f()
foo
Something to this effect is obvious, readable and therefore much more maintainable.
But I want to do this for cases where there are MANY variables in my namespace that I might at some point want to override without rewriting every function definition in my module.
It sounds like you want a class:
class Foo(object):
x = 0
def f(self):
print self.x
my_foo = Foo()
my_foo.x = 1
my_foo.f()
I use the following for creating dynamically custom modeling databses for targets of interest from one global database:
DBs = ["Target_1_DB","Target_2_DB","Target_3_DB"]
for db in DBs:
print(db)
exec_str = "Modeling_DB_Base = " + db
exec(exec_str)
#s = "Modeling_DB_Base"
#globals()[s] = db
print(Modeling_DB_Base.shape)
using the commented-out globals() results in error message: AttributeError: 'str' object has no attribute 'shape'
You can use the following solution:
s = 'x'
locals()[s] = 1

Python 3.2.2 Error Local Variable

gCharlie = 0
gJeff = 0
def Bob ():
Charlie = gCharlie
Jeff = gJeff
Number = int(input("Hello and Welcome to Charlie's Number Guessing Game. Enter a nonnegative number from 0 to 10 not counting 0. "))
This code gives me this error in Python 3.2:
UnboundLocalError: local variable 'gCharlie' referenced before assignment
What does this local variable error mean?
inside the Scope of your function you must have reassigned gJeff and gCharlie, which created them as new local variables. To tell python that you're using the globals, change the top of your function to look like this.
def Bob():
global gCharlie
global gJeff
Charlie=gCharlie
without telling python that you're using the globals, it tries to reference local gCharlie and gJeff variables, which as it says, have not been assigned at that point in your function. The reason people are getting it to work is because they're using only the code you've posted. You can reference globals without explicitly saying so ONLY if you don't change their values within the function referencing them.
As a rule python searches in this order for a variable name: local scope, any def it is nested inside of, global, built ins. Bbecause gJeff and gCharlie are local variables in your function, it stops there, unless you tell it otherwise.
If you want to see this in action try to look at this
x=5
def useX():
x=0 #this is a local variable
print "in function: ", x
def main():
print "in main(1): ", x
useX()
print "in main(2): ", x
main()
this will output
in main(1): 5
in function: 0
in main(2): 5
because within the function, x is created as a new local variable. Adding a global x statement to the useX function would change the last line to print "0" instead of "5"
It might be due to there is gCharlie = inside a function (note: the first letter is g).
Use parameters instead of globals:
def bob(charlie=0, jeff=0):
number = int(input("..."))
# ...
bob(charlie=3)
It means that you're assigning to gCharlie in the part of the function you didn't show, and so the Python compiler has marked it as a local variable. As such, you're accessing it before it exists. Use nonlocal or global to solve.
Two previous answers here are correct, but both are a bit unclear. I'll show with some examples:
The code you show will work fine:
>>> gCharlie = "Global!"
>>>
>>> def foo():
... print(gCharlie)
...
>>> foo()
Global!
>>> print(gCharlie)
Global!
So that's not the problem at all. However, you can't assign global variables inside a function:
>>> gCharlie = "Global!"
>>> def foo():
... gCharlie = "Local!"
... print(gCharlie)
...
>>> foo()
Local!
>>> print(gCharlie)
Global!
As you see, the global variable gCharlie did not change. This is because you did not modify it, you created a new local variable, with the same name. And this is the cause of the error:
>>> gCharlie = "Global!"
>>> def foo():
... oldCharlie = gCharlie
... gCharlie = "Local!"
... print(gCharlie)
...
>>> foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in foo
UnboundLocalError: local variable 'gCharlie' referenced before assignment
The hint is in the error. It says local variable gCharlie. The problem is not the inability to access the global gCharlie, but that the local one hasn't been created yet.
The fix is to specify that you don't want to create a local variable, but that you want to modify the global one. You do this with the global keyword.
>>> gCharlie = "Global!"
>>> def foo():
... global gCharlie
... oldCharlie = gCharlie
... gCharlie = "Local!"
... print(gCharlie)
...
>>> foo()
Local!
>>> print(gCharlie)
Local!
As you see now, you modified the global variable.
That said, global variables are usually a bad idea. Avoid them. Try to pass in the variables as parameters instead.

Python - Assign global variable to function return requires function to be global?

So, I'm confused. I have a module containing some function that I use in another module. Imported like so:
from <module> import *
Inside my module, there exist functions whose purpose is to set global variables in the main program.
main.py:
from functions import *
bar = 20
print bar
changeBar()
print bar
functions.py:
def changeBarHelper(variable):
variable = variable * 2
return variable
def changeBar():
global bar
bar = changeBarHelper(bar)
Now, this is a simplification, but it is the least code that yields the same result:
Traceback (most recent call last):
File "/path/main.py", line 5, in
changeBar()
File "/path/functions.py", line 7, in changeBar
bar = changeBarHelper(bar)
NameError: global name 'bar' is not defined
Doing an import * in the way that you've done it is a one way process. You've imported a bunch of names, much the same way as you'd do:
from mymodule import foo, bar, baz, arr, tee, eff, emm
So they are all just assigned to names in the global scope of the module where the import is done. What this does not do is connect the global namespaces of these two modules. global means module-global, not global-to-all-modules. So every module might have its own fubar global variable, and assigning to one won't assign to every module.
If you want to access a name from another module, you must import it. So in this example:
def foo(var1, var2):
global bar
from mainmodule import fubar
bar = fubar(var1)
By doing the import inside the function itself, you can avoid circular imports.
Well, I can't comment on any of the posts here and this solution isn't working. I would like to clarify this a bit.
There are two modules here:
main.py:
from functions import *
bar = 20
print bar
changeBar()
print bar
functions.py:
def changeBarHelper(variable):
variable = variable * 2
return variable
def changeBar():
global bar
bar = changeBarHelper(bar)
Now, this is a simplification, but it is the least code that yields the same result:
Traceback (most recent call last):
File "/path/main.py", line 5, in
changeBar()
File "/path/functions.py", line 7, in changeBar
bar = changeBarHelper(bar)
NameError: global name 'bar' is not defined
Now, the solution given isn't working for me, and this is a problem I would really like a solution to.
def changeBar()
needs to be:
def changeBar(bar)
that way when you call changeBar(bar) in main.py
functions.py works on bar variable from main.py
sooo...
from functions import *
bar = 20
print bar
bar = changeBar(bar)
print bar
def changeBarHelper(variable):
variable = variable * 2
return variable
def changeBar(bar):
bar = changeBarHelper(bar)
return bar

Categories