I am using the Spyder IDE, version 3.3.1, and I would like to debug only a specific line of Python code that's about 1,100 lines into the program. I don't want to run all the code above line 1,100. Running those 1,100 lines would take time since it would execute various large database queries, and be unnecessary for my purposes.
In Visual Basic (VBA) for Excel, when debugging, I can drag the yellow arrow to any line and run the code line-by-line from that point forward. Is there something similar in Python (specifically in the Spyder IDE), or do I have to set break points and run the entire code?
I suppose one option is to move this specific line of code to the top of the file (i.e. line 1, not line 1,100), so that it runs first when debugging. Maybe I've answered my own question, but I'm still curious whether there's a way to start debugging from a certain line.
My options seem limited to the following, judging by Spyder's debug menu:
Run current line (i.e. starting at line 1)
Step into function or method of current line
Run until current function or method returns
Continue execution until next breakpoint
I don't want to reorganize all my code, since this is just an ad hoc/one-time analysis. For context, I am a business analyst and I am just doing some data exploration/correlation/regression analysis. I am a functional programmer whose background is in business/finance, not computer science.
It seems the easiest/quickest solution is to move the specific line of code to the top of the file (i.e. line 1, not line 1,100), so that it runs first when debugging.
Related
I'm debugging python code and I would find it very helpful if I can continue the execution of the program (just like using c) BUT break at the first executed line if a new file is being run. Is it possible?
The reason I want to do that is I want to skip a bunch of site packages and see exactly where the entrypoint to my own code is.
I am using Python 3.6.3.
A problem remains in my script, which is fully operational.
Main modules are pandas and xslxwriter (with easygui for GUI).
From a single master file (Excel 2010), this script can generate dozens of Excel files (with xlsxwriter), each of them can contain hundreds of columns of data (depending of parameters of the master file).
Indentation, logic and results are OK.
But the last Excel file is not committed to disk, and I have to restart Python to get it.
For example, if one run produces 100 files, only 99 will be written on disk. The last one is calculated, but not visible.
If Python is not restarted, this file is written to disk at the beginning of a next run of the script.
I identified maybe a flush problem, and tried some solutions, but this problem still remains.
Are there some tricks to force the buffer? I am not allowed to modify the environment variables on my professional computer.
Thank you for your help and your time :)
Thank you Mark Tolonen
You were right : the file was not closed properly, and it was because I made a mistake.
My code was a bit difficult to summarize, and I could not post a résumé of my script.
First, a keyword continue (for the main loop) was bad-indented and I replaced it at the right place.
But just before this keyword, I was closing the xlsxwriter file with: workbook.close (only one in the script exists for the main loop).
But this was not mentioned as an error at run-time.
Each xlsxwriter file was committed to disk except the last one, as mentioned in my question above.
I then reviewed the documentation at https://xlsxwriter.readthedocs.io/workbook.html, and I noticed that parenthesis were missing when worbook is closed.
After correction by adding the missing parenthesis: workbook.close(), all is fine now :)
I would like to share this information, because some may have meet the same problem.
Than you also to progmatico for your information on flush properties.
Greetings from Paris, France :)
I am using VBA-macros in order to automate serveral data processing steps in excel, such as data reduction and visualization. But since excel has no appropriate fit for my purposes I use a python script using scipy's least squares cubic b-spline function. The in and output is done via .txt files. Since I adjusted the script from a manual script I got from a friend.
VBA calls Python
Call common.callLSQCBSpline(targetrng, ThisWorkbook) #calls python which works
Call common.callLoadFitTxt(targetrng, ThisWorkbook) # loads pythons output
Now the funny business:
This works in debug mode but does not when running "full speed". The solution to the problem is simply wait for the directory were the .txt is written to refresh and to load the current and not the previous output file. My solution currently looks like this:
Call common.callLSQCBSpline(targetrng, ThisWorkbook)
Application.Wait (Now + 0.00005)
Call common.callLoadFitTxt(targetrng, ThisWorkbook)
This is slow and anoying but works. Is there a way to speed this up? The python script works fine and writes the output.txt file properly. VBA just needs a second or two before it can load it. The txts are very small under 1 kB.
Thanks in advance!
I am currently running evaluations with multiple parameter configurations in a medium sized project.
I set certain parameters and change some code parts and run the main file with python.
Since the execution will take several hours, after starting it I make changes to some files (comment out some lines and change parameter) and start it again in a new tmux session.
While doing this, I observed behaviour, where the first execution will use configuration options of the second execution, so it seems like python was not done parsing the code files or maybe lazy loads them.
Therefore I wonder how python loads modules / code files and if changing them after I started the execution will have an impact on the execution?
Suppose that I have put a Python script to run. Let's say while it is running, I open the source code and change the value of a variable to different value. Now, in another terminal if I start running the latest source code, what happens to the previous run that is progress?
Will it be get affected because of this latest change that I did while I was running it?
The thing is that I want to do parallel runs of the program for different values of a particular variable. Any better way to do this?
Python compiles your source into bytecode and runs that bytecode. Once the source file has been read it is no longer needed to run that bytecode. Changes to the source file won't then affect already running code.
However, if an exception is raised and Python tries to format a traceback for display, it'll reach back to the source code, mapping line markers in the bytecode back to source lines. If the source file changed after compilation, this could mean the wrong lines are being displayed. That can create confusion.
You can easily give your program command line parameters to vary how it behaves. Take a look at the sys.argv list, and perhaps the argparse module for more complex command-line option handling. That way your code remains stable and flexible.
Python typically compiles the source code to a *.pyc file. Changing the value in the script usually won't affect the value already in memory.
The better way to do this is take an argument from argv
python your_script.py value
You can access it with
import sys
sys.argv[1] #this is the 'value' from the command line