Is there a way to view a list of all of my variables in python while the program is running without setting breakpoints? Printing is too messy because I have a lot of variables that are constantly changing.
Thanks
Maybe inspect helps you, but you have to filter the information then.
Usage like:
> import inspect
> a = 5
> f = inspect.currentframe()
> print f.f_locals
...
...
'a': 5
...
Maybe it's worth to mention that you cannot iterate over the resulting dictionary in a for loop because asignment to a variable would change that dictionary. You have to iterate only over the keys (at least that's what I just found out).
Example:
for v in f.f_locals.keys():
if not v.startswith("_"):
print v
Look at the first line: simply writing for v in f.f_locals would not succeed.
If you are running on Pydev (python extension for eclipse), you can easily watch your variables, however you'll need to set a breakpoint initially, then only step into/over your code.
Grab a debugger: http://winpdb.org/ and then you can watch them and see them in there; however if you're just looking for a function to print all the variables defined in the local scope, just calls locals().
If your question is referring to python as a language (as opposed to "python as an ecosystem of applications and utilities"), AFAIK, the answer is "no" (not out-of-the-box anyhow). Of course there are a number of IDE's that allow you to use debug to see the values of variables [see Sumit answer, for example].
However if during development you wanted a "live monitor" of a number of variables, you could use the logging module, and define your own Handler class so as to redirect the information "live" somewhere. Setting the message to log.debug level would make trivial to activate/deactivate this feature by simply changing the minimum log level of your application.
HTH!
Related
I'm using pyinstaller to distribute my code as executable within my team as most of them are not coding/scripting people and do not have Python Interpreter installed.
For some advanced usage of my tool, I want to make it possible for the user to implement a small custom function to adjust functionality slightly (for the few experienced people). Hence I want to let them input a python file which defines a function with a fixed name and a string as return.
Is that possible?
I mean the py-file could be drag/dropped for example, and I'd tell them that their user-defined function needs to have a certain name, e.g. "analyze()" - is it now possible to import that from the drag/dropped pythonfile within my PyInstaller Script and use it as this?
I know, it certainly will not be safe/secure and they could do evil things, delete files and so one... But that are things which we don#t care at this point, please no discussions about it. Thanks!
To answer my own question: yes it does actually work to import a module/function from a given path/pythonfile at runtime (that I knew already) even in PyInstaller (that was new for me).
I used this for my Py2.7 program:
f = r'C:\path\to\userdefined\filewithfunction.py'
if os.path.exists(f):
import imp
userdefined = imp.load_source('', f) # Only Python 2.x, for 3.x see: https://stackoverflow.com/a/67692/701049
print userdefined # just a debugging print
userdefined.imported() # here you should use try/catch; or check whether the function with the desired name really exists in the object "userdefined". This is only a small demo as example how to import, so didnt do it here.
filewithfunction.py:
--------------------
def imported():
print 'yes it worked :-)'
As written in the comments of the example code, you'll need a slightly different approach in Python 3.x. See this link: https://stackoverflow.com/a/67692/701049
There're many kinds of Python REPL, like the default REPL, ptpython, ipython, bpython, etc. Is there a way to inspect what current REPL is when I'm already in it?
A little background:
As you may have heard, I made pdir2 to generate pretty dir() printing. A challenge I'm facing is to make it compatible with those third-party REPLs, but first I need to know which REPL the program is running in.
Ok, finally found a simple but super reliable way: checking sys.modules.
A function you could copy and use.
import sys
def get_repl_type():
if any('ptpython' in key for key in sys.modules):
return 'PTPYTHON'
if any('bpython' in key for key in sys.modules):
return 'BPYTHON'
try:
__IPYTHON__
return 'IPYTHON'
except NameError:
return 'PYTHON'
Probably the best you can do is to look at sys.stdin and stdout and compare their types.
Maybe there are also ways for each interpreter to hook in custom completions or formatters.
You can try to find information from the call stack.
Those fancy REPLs use their startup script to initialize.
It's possible to run one REPL in another, so you need to traverse call stack from top to bottom until find a frame from REPL init script.
I am looking for a way to quickly print a variable name and value while rapidly developing/debugging a small Python script on a Unix command line/ssh session.
It seems like a very common requirement and it seems wasteful (on keystrokes and time/energy) to duplicate the variable_names on every line which prints or logs its value. i.e. rather than
print 'my_variable_name:', my_variable_name
I want to be able to do the following for str, int, list, dict:
log(my_variable_name)
log(i)
log(my_string)
log(my_list)
and get the following output:
my_variable_name:some string
i:10
my_string:a string of words
my_list:[1, 2, 3]
Ideally the output would also log the function name.
I have seen some solutions attempting to use locals, globals, frames etc., But I have not yet seen something that works for ints, strings, lists, and works inside functions too.
Thanks!
Sorry to Necro this ancient thread, but this was surprisingly difficult to find a good answer for.
Using the '=' sign after the variable achieves this. For instance:
import pathlib as pl
import logging
logging.basicConfig(level=logging.DEBUG)
data_root = pl.Path("D:\Code_Data_Dev\Data\CSV_Workspace_Data")
logging.debug(f'{data_root=}')
This outputs
DEBUG:root:data_root=WindowsPath('D:/Code_Data_Dev/Data/CSV_Workspace_Data')
If the tool you need is only for developing and debugging, there's a useful package called q.
It has been submitted to pypi, it can be installed with pip install q or easy_install q.
import q; q(foo)
# use #q to trace a function's arguments and return value
#q
def bar():
...
# to start an interactive console at any point in your code:
q.d()
The results are output to the file /tmp/q (or any customized paths) by default, so they won't be mixed with stdout and normal logs. You can check the output with tail -f /tmp/q. The output is highlighted with different colors.
The author introduced his library in a lightning talk of PyconUS 2013. The video is here, begins at 25:15.
Here is another evil hack:
import inspect
def log(a):
call_line = inspect.stack()[1][4][0].strip()
assert call_line.strip().startswith("log(")
call_line = call_line[len("log("):][:-1]
print "Log: %s = %s" % (call_line, a)
b=3
log(b)
This obviously would need some range checks, better parsing, etc.
Also works only if the calls is made always in the same way and has probably more - unknown to me - assumptions...
I don't know any way to simply get the name of a variable of a string.
you can however get the list of argument of the current fonction log.
import inspect
def log(a):
frame = inspect.currentframe()
args, _, _, values = inspect.getargvalues(frame)
print "%s:%s" % (args[0], values[args[0]])
I am writing a Python interpreter and want to redirect the function's return values to stdout, like the Python Interpreter in Interactive Mode. Within this mode, when the user calls a function, its return value is printed on the screen. The same occurs with expressions.
E.g.
>>> foo()
'Foo return value'
>>> 2+4
6
>>> print('Hello!')
'Hello!'
Changing the sys.stdout only affects the print function. How do I redirect the other expressions to stdout?
Thank you
First, the interactive mode does not print the return value from any function called. Instead, it prints the result of whatever expression the user typed in. If that's not a function call, it still gets printed. If it has 3 function calls in it, it still prints one result, not 3 lines. And so on.
So, trying to redirect function return values to stdout is the wrong thing to do.
What the interactive interpreter does is something sort of like this:
line = raw_input(sys.ps1)
_ = eval(line)
if _ is not None:
print repr(_)
(You may notice that you can change sys.ps1 from the interactive prompt to change what the prompt looks like, access _ to get the last value, etc.)
However, that's not what it really does. And that's not how you should go about this yourself either. If you try, you'll have to deal with complexities like keeping your own globals separate from the user's, handling statements as well as expressions, handling multi-line statements and expressions (doing raw_input(sys.ps2) is easy, but how do you know when to do that?), interacting properly with readline and rlcomplete, etc.
There's a section of the documentation called Custom Python Interpreters which explains the easy way to do this:
The modules described in this chapter allow writing interfaces similar to Python’s interactive interpreter. If you want a Python interpreter that supports some special feature in addition to the Python language, you should look at the code module.
And code:
… provides facilities to implement read-eval-print loops in Python. Two classes and convenience functions are included which can be used to build applications which provide an interactive interpreter prompt.
The idea is that you let Python do all the hard stuff, up to whatever level you want to take over, and then you just write the part on top of that.
You may want to look at the source for IDLE, ipython, bpython, etc. for ideas.
Instead of using exec() to run the user input, try eval():
retval = eval(user_input)
sys.stdout.write(repr(retval) + "\n")
I love being able to modify the arguments the get sent to a function, using settrace, like :
import sys
def trace_func(frame,event,arg):
value = frame.f_locals["a"]
if value % 2 == 0:
value += 1
frame.f_locals["a"] = value
def f(a):
print a
if __name__ == "__main__":
sys.settrace(trace_func)
for i in range(0,5):
f(i)
And this will print:
1
1
3
3
5
What other cool stuff can you do using settrace?
I would strongly recommend against abusing settrace. I'm assuming you understand this stuff, but others coming along later may not. There are a few reasons:
Settrace is a very blunt tool. The OP's example is a simple one, but there's practically no way to extend it for use in a real system.
It's mysterious. Anyone coming to look at your code would be completely stumped why it was doing what it was doing.
It's slow. Invoking a Python function for every line of Python executed is going to slow down your program by many multiples.
It's usually unnecessary. The original example here could have been accomplished in a few other ways (modify the function, wrap the function in a decorator, call it via another function, etc), any of which would have been better than settrace.
It's hard to get right. In the original example, if you had not called f directly, but instead called g which called f, your trace function wouldn't have done its job, because you returned None from the trace function, so it's only invoked once and then forgotten.
It will keep other tools from working. This program will not be debuggable (because debuggers use settrace), it will not be traceable, it will not be possible to measure its code coverage, etc. Part of this is due to lack of foresight on the part of the Python implementors: they gave us settrace but no gettrace, so it's difficult to have two trace functions that work together.
Trace functions make for cool hacks. It's fun to be able to abuse it, but please don't use it for real stuff. If I sound hectoring, I apologize, but this has been done in real code, and it's a pain. For example, DecoratorTools uses a trace function to perform the magic feat of making this syntax work in Python 2.3:
# Method decorator example
from peak.util.decorators import decorate
class Demo1(object):
decorate(classmethod) # equivalent to #classmethod
def example(cls):
print "hello from", cls
A neat hack, but unfortunately, it meant that any code that used DecoratorTools wouldn't work with coverage.py (or debuggers, I guess). Not a good tradeoff if you ask me. I changed coverage.py to provide a mode that lets it work with DecoratorTools, but I wish I hadn't had to.
Even code in the standard library sometimes gets this stuff wrong. Pyexpat decided to be different than every other extension module, and invoke the trace function as if it were Python code. Too bad they did a bad job of it.
</rant>
I made a module called pycallgraph which generates call graphs using sys.settrace().
Of course, code coverage is accomplished with the trace function. One cool thing we haven't had before is branch coverage measurement, and that's coming along nicely, about to be released in an alpha version of coverage.py.
So for example, consider this function:
def foo(x):
if x:
y = 10
return y
if you test it with this call:
assert foo(1) == 10
then statement coverage will tell you that all the lines of the function were executed. But of course, there's a simple problem in that function: calling it with 0 raises a UnboundLocalError.
Branch measurement would tell you that there's a branch in the code that isn't fully exercised, because only one leg of the branch is ever taken.
For example, get the memory consumption of Python code line-by-line: http://pypi.python.org/pypi/memory_profiler
One latest project that uses settrace heavily is PySnooper
It helps new programmers to trace/log/monitor their program output. Cheers!
I don't have an exhaustively comprehensive answer but one thing I did with it, with the help of another user on SO, was create a program that generates the trace tables of other Python programs.
The python debugger Pdb uses sys.settrace to analyse lines to debug.
Here's an c optimization/extension for pdb that also uses sys.settrace
https://bitbucket.org/jagguli/cpdb