Lets say you have a large file containing LETTER,NUMBER comma-delimited tokens. You want to write a program that reads from standard input and prints out NUMBER+1 for each line. Very trivial program, I understand. However, here is the constraint -- you can only read from this standard in pipe one-time AND you have to start out with programming an empty file.
So for example:
cat FILE.csv | python empty_program.py
This should pop up an interactive session which allows you to write what ever code you want. Since empty_program.py has not called stdin.readline(), the stdin buffer is appropriately in-tact.
Is something like this possible?
One example of something that can sort of do this is the Excel VBA debugger/IDE. It allows you to pause execution -- add new lines to the programs source code and continue exeuction.
cat FILE.csv | python empty_program.py
Well, python will try to read "empty_program.py", and fail to find anything in it, assuming there is file, and then exit. If the file doesn't exist, you get an error. I tested it [you should have been able to do that as well, doesn't take that much effort - probably a lot less than it took to go to SO and write the question].
So, my next thought was to use an interactive python process, but since you are feeding things through stdin, it won't work - I didn't have a good csv file, so I did "cat somefile.c|python", and that falls over at "int main()" with "invalid syntax". I'm surprised it got as far as that, but I guess that's because #include's are seen as comments.
Most interactive programming languages read from stdin, so you can't really do what you are describing with any of them.
I'm far from sure why you'd want to either. If your first program can produce the relevant program code, why would you not just put it in a file and let python read that file... Rather than jump through hoops? Note that an IDE is not the same as a command line program. I'm pretty sure that if you work hard enough at something, you can write a C program that acesses the Eclipse IDE with Python plugins. But that's really doing things the hard way. Why anyone would WANT to spend that much effort to achieve so little, I don't see.
Sorry, but I don't really see the point of what you are trying to do - I'm sure you have some good idea in there, but I'm sure the implementation details need to be worked on.
Related
I am working with multiprocessing and I want to ask if there is some option to create new process with new terminal window in Ubuntu
I have 3 processes starting simultaneously, but I want results from them in separated terminals for each
Thanks
No. Sadly there is no* way to do what you want within python, as python has no control over the terminal it is running in.
What I think you want though is to separate the messages from your different processes, so you can see what's going on. What I sometimes do for this (in testing only!) is to have each process log to a different file, and then watch those three files in three terminal windows. You can do this with watch or even a simple while loop in bash:
watch -n 3 "cat /my/output/file" # or:
while true; do cat /my/output/file; sleep 3; done
of course you can replace cat with something more useful, perhaps tail. Or you can just open the output files in a text editor which has an auto-revert facility (e.g. Emacs with M-x auto-revert-mode). This does exactly the same thing internally---poll the file for changes and update if need be.
I also really suggest you use logging inside your code, and give each paralleled function it's own logger (with the name derived from the parameters of the function). (This can be easier with a small class rather than a function). That way later on you can send all your output to a file, and if something goes wrong you can easily find out which run failed and extract information from only that run (with grep!). I use this approach in paralleled fuzzy matching code (actually for matching music libraries) and it's invaluable when you need to dig into how some strange result occurred.
*Okay, I'm sure there's some horrible way to control some particular terminal and output to it, but that's not what you meant.
To introduce my question let get an example.
I have a file code.py. Let's say I run it, python code.py which takes some time. Python starts a process, call it p0.
Obviously changing code.py during p0 won't change its execution, because the code, code.py is loaded at the begining.
My point is then: is there a way, knowing p0's pid to see/print the python code currently used by p0 i.e. code.py's "version" when p0 started?
I'm quite curious about how difficult this could be, and, as python isn't a compiled language, I think there may be the original python source, somewhere. Plus, it's quite hacky, but this could help me when I'm not 100% sure how was the code when I started a looooong process (then modified the file, yes that's not a really good pratice, must say).
Any clue?
thx folks
pltrdy
I am trying to find a way that I can have a program step through Python code line by line and do something with the results of each line. In effect a debugger that could be controlled programmatically rather than manually. pdb would be exactly what I am looking for if it returned its output after each step as a string and I could then call pdb again to pickup where I left off. However, instead it outputs to stdout and I have to manually input "step" via the keyboard.
Things I have tried:
I am able to redirect pdb's stdout. I could redirect it to a second
Python program which would then process it. However, I cannot
figure out how to have the second Python program tell pdb to
step.
Related to the previous one, if I could get pdb to step all the way
through to the end (perhaps I could figure out something to spoof a
keyboard repeatedly entering "step"?) and redirect the output to a
file, I could then write another program that acted like it was
stepping through the program when it was actually just reading the
file line by line.
I could use exec to manually run lines of Python code. However,
since I would be looking at one line at a time, I would need to
manually detect and handle things like conditionals, loops, and
function calls which quickly gets very complicated.
I read some posts that say that pdb is implemented using
sys.settrace. If nothing else works I should be able to recreate
the behavior I need using this.
Is there any established/straight forward way to implement the behavior that I am looking for?
sys.settrace() is the fundamental building block for stepping through Python code. pdb is implemented entirely in Python, so you can just look at the module to see how it does things. It also has various public functions/methods for stepping under program control, read the library reference for your version of Python for details.
I read some posts that say that pdb is implemented using sys.settrace.
If nothing else works I should be able to recreate the behavior I need
using this.
Don't view this as a last resort. I think it's the best approach for what you want to accomplish.
I am new to Python, have some experience in MatLab and r. My question is: Is it possible to run part of the code in .py block by block (line by line)?
In r or Matlab, I can first have some data and variables loaded in the memory first. Then experimentally I can run a line or two to try out the syntax... this is particularly useful for new learners I believe. I know there is something called the iPython which can execute Python code line by line however this is not what I am after. Thanks.
Since ipython has already been discounted, I'm not sure this answer will be better. But I will tell you the two things that I do.
I drop into the debugger at the point where I want to "try out" something, so the code will run up to that point, and then drop me into the debugger. You do this simply by inserting this code at that point:
import pdb; pdb.set_trace()
Once you've done what needs to be done, you can either press q to quit, or c to continue running the process.
I use the -i option to python. This enters interactive mode at the end of your python code. This is useful if you want to set up a bunch of data structures, and try out some code on it, instead of typing all of it into a python shell first. (that might be why you rejected ipython?)
I think what you need is a debugger.
You can use the pydev plugin for Eclipse which has a debugger.
Another option is pdb as already suggested but it's not very easy to use.
I'm experimenting with debugging my python from the raw pdb program rather than running pdb though emacs (which tracks the current line with a marker in the text display of the code). It's slightly annoying that the list command l in pdb only displays a few lines of code, I would rather have it fill my terminal with all the code of the current function up to the current line.
I know I can do this manually by looking at the line numbers are typing l 50,100 (where 100 is the current line) but this is time consuming and I'd like to set this up to work automatically.
I wonder if there is a way to define a pdb command to do this? I'm guessing it would need to (1) access the number of the current line, N; then (2) execute "l N-50, N". I've searched around a lot but can't find anyone who has done this before. Perhaps there is a way to access the pdb module's own internals to get the line number?
(Or a roundabout way would be to write something that calls list once, parses the output to extract the current line, then executes a new list command, I wonder if anyone has done this already? Is this how IDEs manage to get the current line information from pdb or are they using its internals I wonder?)
You could probably extend the PDB class, but the extension interface isn't documented very well.
I'd instead recommend using pdb++ and its sticky feature which pretty much does what you want, if I understood your use case correctly.