I have a couple of functions written in a single Python file. They perform a sequence of steps on a file-based dataset.
My workflow:
After I finished coding a part of the function's body, I run the function to see how it goes.
It may break at a certain point.
I fix the code and re-run the function.
The problem is that when I re-run the function, it will execute the lines that were already completed successfully. Yet I want to be able to start not from the beginning but rather from an arbitrary point. This is because the whole function runs for several minutes and it would be a waste time to wait for it to complete.
I could implement "checks" to see whether this operation is required (e.g., don't create a file if it already exists), but this would imply adding a lot of new validation code (e.g., make sure that the existing file does contain the content needed); in reality, my function will be run on a dataset in known format and the whole function should be executed.
The most obvious solution is to comment out the parts that were executed successfully, but it's a hustle and I got tired of commenting and uncommenting parts as I move forwards and the function gets larger.
Are there are any better approaches than commenting out lines for ignoring certain part of the function's body when executing?
I am on Wing IDE if this has something to do with the debugging tricks in an IDE itself.
Wing can move the program counter to a different line in the function via the right-click popup menu, but you'd need to do this every time you run the function. I think a better approach is to refactor the function into smaller functions -- then you can comment out or conditionalize only the function calls. You could also write tests that call some of the functions and not others.
Related
So I have a script called controller.py that has commands that interface with a front end on a website. I am looking for a way that when a command is run, for example:
if command == 'F':
drivingSpeed = drivingSpeedActuallyUsed
for motorIndex in range(2):
runMotor(motorIndex, forward[motorIndex])
time.sleep(straightDelay)
in controller.py, it changes a line in another script called send_video.py for a few seconds then change back to the original
overlayCommand = '-vf transpose=2,transpose=2,dynoverlay=overlayfile=/home/pi/runmyrobot/images/hud.png:check_interval=500'
I need the line in send_video.py to change the directory of the image. Another issue I am having is that the send_video.py only updates on reboot, I need to tell it to update automatically.
Changing a line in code automatically is very unusual and far from what we call "best practice". This is called self-modifying code and was the topic of research done several decades ago. Since then it has been dropped almost completely and is now regarded an ugly hack in all but the very strangest circumstances.
What is typically done to achieve your goal is to pass parameters and use them in the other code.
Your wording is a bit unclear (I don't understand that part about updating a file, for example), so I'm not sure if I address your issue correctly, but I'll give it a try in a general approach:
When your code in controller.py calls the code in send_video.py it will call a function. To this function is should pass an argument which is the line. The function in send_video.py should use the passed parameter to determine which line it should use (the default one or the special one).
Feel free to improve your question by showing us more of your code or ask further questions if this does not help enough.
I am trying to find a way that I can have a program step through Python code line by line and do something with the results of each line. In effect a debugger that could be controlled programmatically rather than manually. pdb would be exactly what I am looking for if it returned its output after each step as a string and I could then call pdb again to pickup where I left off. However, instead it outputs to stdout and I have to manually input "step" via the keyboard.
Things I have tried:
I am able to redirect pdb's stdout. I could redirect it to a second
Python program which would then process it. However, I cannot
figure out how to have the second Python program tell pdb to
step.
Related to the previous one, if I could get pdb to step all the way
through to the end (perhaps I could figure out something to spoof a
keyboard repeatedly entering "step"?) and redirect the output to a
file, I could then write another program that acted like it was
stepping through the program when it was actually just reading the
file line by line.
I could use exec to manually run lines of Python code. However,
since I would be looking at one line at a time, I would need to
manually detect and handle things like conditionals, loops, and
function calls which quickly gets very complicated.
I read some posts that say that pdb is implemented using
sys.settrace. If nothing else works I should be able to recreate
the behavior I need using this.
Is there any established/straight forward way to implement the behavior that I am looking for?
sys.settrace() is the fundamental building block for stepping through Python code. pdb is implemented entirely in Python, so you can just look at the module to see how it does things. It also has various public functions/methods for stepping under program control, read the library reference for your version of Python for details.
I read some posts that say that pdb is implemented using sys.settrace.
If nothing else works I should be able to recreate the behavior I need
using this.
Don't view this as a last resort. I think it's the best approach for what you want to accomplish.
I'm working on a python project that spans across multiple files and directories. Here's my workflow:
Run main python script
Main script calls some functions in other files
Functions in other files/directories execute
In the middle of execution, there is a bug in one of the functions, but I find the bug only after the main script finishes. Sometimes, there may not be a bug, but rather some parameter that needs tweaking.
I go back and fix the bug/make the necessary tweaks and re-run the main program and this time it executes fine.
Obviously, this workflow is terribly inefficient as considerable amount of code (that runs prior to the buggy function) gets re-executed. What would be ideal is to run the program in ipython and after discovering the issue and making the necessary changes, restart from the place where the buggy function executions starts and not from the beginning. I'm not sure how to achieve this and any help would be much appreciated.
I know how to rerun lines from ipython history (%rerun) and how to ensure autoreload of changed files in ipython, but in this case, I can't really type out the lines of code into ipython. Writing unit tests may not always be feasible, so I need an alternate solution. My use case is something similar to setting a "breakpoint" and then re-executing code past the breakpoint multiple times so as to avoid re-executing the code prior to the breakpoint more than once, while ensuring that all the necessary variables (until that stage) are correctly populated. One final condition is that I may not be able to use an IDE and vim is the only editor available across all the environments I work with.
You could start writing test cases for every function and try to debug each function separately instead of the whole program/script.
There is a python unittest-module: https://docs.python.org/3.4/library/unittest.html and a lot of tutorials like (just an example): http://docs.python-guide.org/en/latest/writing/tests/
It seems annoying writing tests but thinking about test cases gives deeper understanding of "How should the function behave if...".
You could use the code module to include "breakpoints" in your code
import code
# ... later in your program add
code.interact(local=locals()) # enter python interpreter at this point (ctrl+D to continue execution)
I am implementing a workflow management system, where the workflow developer overloads a little process function and inherits from a Workflow class. The class offers a method named add_component in order to add a component to the workflow (a component is the execution of a software or can be more complex).
My Workflow class in order to display status needs to know what components have been added to the workflow. To do so I tried 2 things:
execute the process function 2 times, the first time allow to gather all components required, the second one is for the real execution. The problem is, if the workflow developer do something else than adding components (add element in a databases, create a file) this will be done twice!
parse the python code of the function to extract only the add_component lines, this works but if some components are in a if / else statement and the component should not be executed, the component apears in the monitoring!
I'm wondering if there is other solution (I thought about making my workflow being an XML or something to parse easier but this is less flexible).
You cannot know what a program does without "executing" it (could be in some context where you mock things you don't want to be modified but it look like shooting at a moving target).
If you do a handmade parsing there will always be some issues you miss.
You should break the code in two functions :
a first one where the code can only add_component(s) without any side
effects, but with the possibility to run real code to check the
environment etc. to know which components to add.
a second one that
can have side effects and rely on the added components.
Using an XML (or any static format) is similar except :
you are certain there are no side effects (don't need to rely on the programmer respecting the documentation)
much less flexibility but be sure you need it.
I'm using python scripts to execute simple but long measurements. I as wondering if (and how) it's possible to edit a running script.
An example:
Let's assume I made an error in the last lines of a running script.These lines have not yet been executed. Now I'd like to fix it without restarting the script. What should I do?
Edit:
One Idea I had was loading each line of the script in a list. Then pop the first one. Feed it to an interpreter instance. Wait for it to complete and pop the next one. This way I could modify the list.
I guess I can't be the first one thinking about it. Someone must have implemented something like this before and I don't wan't to reinvent the weel. I one of you knows about a project please let me know.
I am afraid there's no easy way to arbitrarily modify a running Python script.
One approach is to test the script on a small amount of data first. This way you'll reduce the likelihood of discovering bugs when running on the actual, large, dataset.
Another possibility is to make the script periodically save its state to disk, so that it can be restarted from where it left off, rather than from the beginning.