Tracing the execution of a Python program line/statement by line programmatically - python

What would be the easiest way to execute a Python script and, for each executed statement, pass the line number or the line itself to a callback function? For example
A piece of code to be executed
for i in range(5):
z = i**2
and a callback
def callback(line):
print line
The output would be:
for i in range(5):
z = i**2
for i in range(5):
z = i**2
for i in range(5):
z = i**2
...
etc
Another way of stating this is that I want to know the piece of code that would be next to execute if I were running the script stepping through a debugger.. I've seen this question about tracing but I'm interested in doing more than tracing the number of times executed in the callback function.
A debugger is going to be helpful, but only if I can run it non-interactive mode with the ability to call back to the python code

Look at the sys.settrace function. This allows you to specify a tracing function which is executed for every line of code. It was, I think, specifically implemented to create pdb so it's at the right level of abstraction you seem to want. Implementing what you want will not be trivial but I think it's the best place to start.
For an production program that uses this, look at coverage.py.

Accepting Noufal's answer since it put me on the right path, but this helped as well
http://www.dalkescientific.com/writings/diary/archive/2005/04/20/tracing_python_code.html

The April's fools module goto is a working example of tracing implemented in Python, so although it's doubtlessly slower (than the C implementation in coverage.py), it's rather easy to understand. http://entrian.com/goto/

Related

Confusion about returning values when files finish getting executed

So I have 2 files, fish_life_simulator.py and menu.py. fish_life_simulator.py is the main file and executes other files like menu.py depending on what happens. So here is the code and how it should work:
import os
os.chdir(os.path.dirname(__file__))
result = exec(open(r'menu.py', encoding='utf-8').read())
print(result)
So at first when the code arrives to result = exec(open(r'menu.py', encoding='utf-8').read()) it executes menu.py and all is fine, but it could stop for several reasons:
The player exit the game
The player entered settings
The player pressed play
So what I decided to do, is when menu.py will stop running it will return a value, like 1, 2 or 3, so I tried several methods that have been included in here:
Best way to return a value from a python script
like using return or sys.exit("some value here"), but even though I did the part inside of menu.py, neither of them worked, as when I tried return, result from result = exec(open(r'menu.py', encoding='utf-8').read()) always was None for some reason and when I tried sys.exit(1) for example, result didn't get printed at all, so I was just wandering if it was something I was missing inside of fish_life_simulator.py, because the part with sending the value should be fine, but the part of receiving it is problematic.
Just define a function in menu.py:
def do_stuff_in_menu():
...
return result
and in fish_life_simulator.py you just call that function:
import menu
result = menu.do_stuff_in_menu()
print(result)
I agree with everyone who says exec() is not the best way to do this, however, since that's not your question, here's an answer for you.
The exec() function always returns None (see docs). If you need the return code, you could use os.system() or one of the various methods from the subprocess library. Unlike exec(), however, both of these alternatives would create a child process.
That said, I personally would not use any of those methods, but would instead modify menu.py to allow you to import it. It's much more natural and direct.

How do I run a function from an input before the code for the function has been read?

I am writing a game, similar to mastermind, and I want a choice bewteen an easy or hard version. I'm not sure how to do this as I need the question before the actual game starts but then there's an error because the function is being called to run before it has been assigned.
def difficulty():
difficulty = input("would you like to the easy or hard version?")
if difficulty == ("easy"):
easy()
elif difficulty == ("hard"):
hard()
difficulty()
This is the start then after is the function with the harder game code then the easier game code. I am trying to run the easy if they request easy and vice versa but the easy () and hard() don't run the code as it isn't assigned yet. I think this is because python reads the code from top to bottom and stops when it finds an error but not sure.
I have never used this before so I apologise if things are unclear or I have done some things wrong.
I am also relatively new to python.
If anybody could help me I would greatly apprectiate it.
Python is quite smart when it comes to identifying functions inside a module. For instance you could do this:
def x():
y()
def y():
print("Y")
x()
and it would execute correctly.
You are right about the execution of a code block that happens from top to bottom, as well as the definitions of those functions will also be constructed top to button, but executed afterwards.
I see some issues in your code.
you do difficulty = input("would you like to the easy or hard version?") but at the same time you have a function called def difficulty. There is a conflict there, try to rename that variable.
you don't need to do ("easy"), it's overkill, you can compare directly to "easy".

Segmentation fault when initializing array

I am getting a segmentation fault when initializing an array.
I have a callback function from when an RFID tag gets read
IDS = []
def readTag(e):
epc = str(e.epc, 'utf-8')
if not epc in IDS:
now = datetime.datetime.now().strftime('%m/%d/%Y %H:%M:%S')
IDS.append([epc, now, "name.instrument"])
and a main function from which it's called
def main():
for x in vals:
IDS.append([vals[0], vals[1], vals[2]])
for x in IDS:
print(x[0])
r = mercury.Reader("tmr:///dev/ttyUSB0", baudrate=9600)
r.set_region("NA")
r.start_reading(readTag, on_time=1500)
input("press any key to stop reading: ")
r.stop_reading()
The error occurs because of the line IDS.append([epc, now, "name.instrument"]). I know because when I replace it with a print call instead the program will run just fine. I've tried using different types for the array objects (integers), creating an array of the same objects outside of the append function, etc. For some reason just creating an array inside the "readTag" function causes the segmentation fault like row = [1,2,3]
Does anyone know what causes this error and how I can fix it? Also just to be a little more specific, the readTag function will work fine for the first two (only ever two) calls, but then it crashes and the Reader object that has the start_reading() function is from the mercury-api
This looks like a scoping issue to me; the mercury library doesn't have permission to access your list's memory address, so when it invokes your callback function readTag(e) a segfault occurs. I don't think that the behavior that you want is supported by that library
To extend Michael's answer, this appears to be an issue with scoping and the API you're using. In general pure-Python doesn't seg-fault. Or at least, it shouldn't seg-fault unless there's a bug in the interpreter, or some extension that you're using. That's not to say pure-Python won't break, it's just that a genuine seg-fault indicates the problem is probably the result of something messy outside of your code.
I'm assuming you're using this Python API.
In that case, the README.md mentions that the Reader.start_reader() method you're using is "asynchronous". Meaning it invokes a new thread or process and returns immediately and then the background thread continues to call your callback each time something is scanned.
I don't really know enough about the nitty gritty of CPython to say exactly what going on, but you've declared IDS = [] as a global variable and it seems like the background thread is running the callback with a different context to the main program. So when it attempts to access IDS it's reading memory it doesn't own, hence the seg-fault.
Because of how restrictive the callback is and the apparent lack of a buffer, this might be an oversight on the behalf of the developer. If you really need asynchronous reads it's worth sending them an issue report.
Otherwise, considering you're just waiting for input you probably don't need the asynchronous reads, and you could use the synchronous Reader.read() method inside your own busy loop instead with something like:
try:
while True:
readTags(r.read(timeout=10))
except KeyboardInterrupt: ## break loop on SIGINT (Ctrl-C)
pass
Note that r.read() returns a list of tags rather than just one, so you'd need to modify your callback slightly, and if you're writing more than just a quick script you probably want to use threads to interrupt the loop properly as SIGINT is pretty hacky.

Running lines of code inside code for new thread instead of specifying a target function to run

I've tried a lot of searching but didn't really know how to word my problem, so it may be that there's a solution that I couldn't find because I didn't know how to search for it.
I have a single line of code that I'd like to run in a separate thread. So far I have the impression that to create a new thread you have to put the code you want to run inside its own function and then call that function using the target argument when starting the thread:
threading.Thread(target = functionName).start()
This is fine and I have it working like this, however because I'm only running a single line of code it seems a bit pointless to have it in its own function, and I'd like to get rid of this. I want to create the thread and effectively specify the actual line of code itself as the target instead of the function name.
I can do this in C#:
new Thread(delegate()
{
// any amount of code goes here and it will be run in its own thread
}).Start();
But is there any way I can do this in Python?
Using python you can make inline functions using lambda
threading.Thread(target = lambda: print('hi')).start()
In this case, note that 'hi' won't show in the terminal as it's in another thread

How to tell the Python debugger to finish the program?

I have recently come across a VERY cool Python module called pdb. For those that are not familiar with it, it is super easy to use and gives you access to pretty much anything within scope at the time. All you have to do to use it is import pdb and put this line in your program where you want to set the breakpoint:
pdb.set_trace()
It works very much like gdb, and I wouldnt be surprised if it was built on top to some extent. Anyway, what I would like to know:
Say I have stopped at my first breakpoint, evaluated some things, and now I want to finish my program. How can I tell the debugger to finish the program, WITHOUT stopping at any more breakpoints? There are some commands, like continue, step, and next, but none of these seem to run the rest of the program uninterrupted. Anyone have some experience with this or am I asking for something that doesnt exist? Thanks!
I would just override pdb.set_trace function, delete all breakpoints and continue
pdb.set_trace = lambda : 0
The good thing is that you can do monkey patching in the debugger.
vikasdhi#redpanda:~$ cat ~/tmp/test.py
for i in range(1000):
import pdb
pdb.set_trace()
vikasdhi#redpanda:~$ python ~/tmp/test.py
> /home/vikasdhi/tmp/test.py(1)<module>()
-> for i in range(1000):
it stopped for the first time
(Pdb) c
> /home/vikasdhi/tmp/test.py(1)<module>()
-> for i in range(1000):
(Pdb) c
> /home/vikasdhi/tmp/test.py(1)<module>()
-> for i in range(1000):
when i want to skip everything i just replace the function
(Pdb) pdb.set_trace = lambda : 0
(Pdb) c
vikasdhi#redpanda:~$
the command is cl or clear.
cl(ear) [filename:lineno | bpnumber [bpnumber ...]]
With a filename:lineno argument, clear all the breakpoints at this line. With a space separated list of breakpoint numbers, clear those breakpoints. Without argument, clear all breaks (but first ask confirmation).

Categories