Please help me to write simple console application in python. It should redirect all input to system shell (bash or windows cmd or powershell) and give all their output to the screen.
Simply I can say run terminal from python application.
The next code works with some strange behavior: first 3 times after any key pressed it outputs (executes?) some previous commands (may be from cache)
#!/bin/python3
import subprocess
import sys
proc = subprocess.Popen(['bash'])
while True:
buff = sys.stdin.readline()
stdoutdata, stderrdata = proc.communicate(buff)
if( stdoutdata ):
print( stdoutdata )
else:
print('n')
break
I think you need
proc = subprocess.Popen(['bash'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
From the docs:
PIPE indicates that a new pipe to the child should be created.
DEVNULL indicates that the special file os.devnull will be used. With
the default settings of None, no redirection will occur; the child’s
file handles will be inherited from the parent.
I don't think you want your bash to be connected to your parent process's stdin directly. That would explain wierdness.
Related
I try to interact to python interpreter using subprocess module like this :
import subprocess
def start(executable_file):
return subprocess.Popen(
executable_file,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
def read(process):
return process.stdout.readline().decode("utf-8").strip()
def write(process, message):
process.stdin.write(f"{message.strip()}\n".encode("utf-8"))
process.stdin.flush()
def terminate(process):
process.stdin.close()
process.terminate()
process.wait(timeout=0.2)
process = start("python")
while True:
write(process, input())
print(read(process))
terminate(process)
But it seems it's locked a deadlock.
If anyone knows how to interact with python with python code and recover stdout, stderr with stream mode.
You need to use communicate() rather than read() and write() with subprocesses else it will lead to deadlock.
See red warning breakout towards middle of this page.
I try to make a backup script in python and start, stop a service with popen...
Stopping the service is working, but unfortunatly starting the service works, but blocks the rest of the execution, the scripts stays there, why ?
Seems to be somehow linked with the httpd service... :-(
the program config element is like "service;httpd;start" or "/etc/init.d/myprog;start"
class execute(actions):
def __init__(self,config,section,logger):
self.name="execute"
actions.__init__(self,config,section,logger)
def process(self):
try:
program=self.config.get(self.section,"program").split(";")
self.logger.debug("program=%s" % program)
p = subprocess.Popen(program, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
if stdout:
self.logger.info(stdout)
if stderr:
self.logger.error(stderr)
return p.returncode
except Exception:
self.logger.exception(Exception)
You have to open a stdin as a pipe as well, and then close it (if you use read() and write() instead of communicate()).
p = subprocess.Popen(..., stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p.stdin.close()
print "Stdout:", p.stdout.read()
print "Stderr:", p.stderr.read()
If it doesn't work, and you don't really need any checks, just close all pipes after call to Popen, what will cause program execution and detachment
from pipes.
Warning: This will make program run as a daemon if it doesn't terminate on its own.
After doing this you may call wait() to see whether it'll block as well. And use exitcodes to check for eventual errors.
There are not much of them. Just service started or not. Sometimes even it returns that service is running, but service crashes.
To check whether service script is still running, but without blocking, use:
if p.poll()==None: print "Still running"
Else, poll() returns the exit code.
This works neatly for starting and stopping a service:
from subprocess import Popen, PIPE
service = "brltty"
p = Popen(["service", service, "start"], stdin=PIPE, stdout=PIPE, stderr=PIPE)
# Note: using sequence uses shell=0
stdout, stderr = p.communicate()
print "Stdout:", stdout
print "Stderr:", stderr
Don't forget to change start to stop :D :D :D
The call to p.communicate() waits for the process to terminate.
Refer to: subprocess documentation
Interact with process: Send data to stdin. Read data from stdout and
stderr, until end-of-file is reached. Wait for process to terminate.
The optional input argument should be a string to be sent to the child
process, or None, if no data should be sent to the child.
You can try to use p.poll() instead. This method doesn't wait for a process to terminate.
I have a dtrace snippet run via python script and the dtrace snippet is such that it generates data when CTRL-C is issued to it. So I had a signal_handler defined in the python script to catch CTRL-C from user and relay this to the dtrace invocation done via subprocess.Popen but I am unable to get any output in my log file. Here is the script:
Proc = []
signal_posted = False
def signal_handler(sig, frame):
print("Got CTRL-C!")
global signal_posted
signal_posted = True
global Proc
Proc.send_signal(signal.SIGINT) #Signal posting from handler
def execute_hotkernel():
#
# Generate the .out output file
#
fileout = "hotkernel.out"
fileo = open(fileout, "w+")
global Proc
Proc = subprocess.Popen(['/usr/sbin/dtrace', '-n', dtrace_script], stdout = fileo)
while Proc.poll() is None:
time.sleep(0.5)
def main():
signal.signal(signal.SIGINT, signal_handler) # Change our signal handler
execute_hotkernel()
if __name__ == '__main__':
main()
Since I have a file hotkernel.out set in subprocess.Popen command for stdout I was expecting the output from dtrace to be redirected to hotkernel.out on doing a CTRL-C but it is empty. What is missing here?
I have a similar issue.
In my case, it's a shell script that runs until you hit Control-C, and then prints out summary information. When I run this using subprocess.Popen, whether using a PIPE or a file object for stdout, I either don't get the information (with a file object) or it hangs when I try to run stdout.readline().
I finally tried running the subprocess from the interpreter and discovered I could get the last line of output after the SIGINT with a PIPE if I call stdout.readline() (where it hangs) and hit Control-C (in the interpreter), and then call stdout.readline() again.
I do not know how to emulate this in script, for a file output or for a PIPE. I did not try the file output in the interpreter.
EDIT:
I finally got back to this and determined, it's actually pretty easy to emulate outside of python and really has nothing to do with python.
/some_cmd_that_ends_on_sigint
(enter control-c)
*data from stdout in event handler*
Works
/some_cmd_that_ends_on_sigint | tee some.log
(enter control-c)
*Nothing sent to stdout in event handler prints to the screen or the log*
Where's my log?
I ended up just adding a file stream in the event handler (in the some_cmd_that_ends_on_sigint source) that writes the data to a (possibly secondary) log. Works, if a bit awkward. You get the data on the screen if running without any piping, but I can also read it when piped or from python from the secondary log.
I am making a simple IDE with a text box and a run button. This asks the user to enter the filename, writes the code written to a file and runs that file.
I want to show whatever is outputted from the console, such as a print, input, etc.. like IDE's do.
Is this possible?
Here is my code:
from Tkinter import *
import tkFileDialog
import ScrolledText
import subprocess
filepath=""
def run():
global filepath
print "<<<<<<=====-------------Restart-------------=====>>>>>>"
py=code.get(1.0,END)
if filepath=="":
filepath=tkFileDialog.asksaveasfilename()
if ".py" not in filepath:
filepath=filepath+".py"
script=open(filepath, "w")
script.write(py)
script.close()
p = subprocess.Popen(['python', filepath],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
for line in iter(p.stdout.readline, ''):
print line
print "<<<<<<=====-------------EXECUTION FINISHED-------------=====>>>>>>"
root = Tk()
code=ScrolledText.ScrolledText(root)
code.pack()
run=Button(root,text="Run", command=run)
run.pack()
root.mainloop()
Yes, just use the subprocess module.
Getting output in one go
import subprocess
output = subprocess.check_output(['python', filepath])
If you want to capture the standard error out of the called process as well as standard out use this instead:
output = subprocess.check_output(['python', filepath], stderr=subprocess.STDOUT)
Or if you want to capture them separately:
p = subprocess.Popen(['python', filepath],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
out, err = p.communicate()
Getting the output as it's produced
This gives you the combined stdout and strerr output, line by line:
p = subprocess.Popen(['python', filepath],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
for line in iter(p.stdout.readline, ''):
# process the line
Keeping the UI responsive
If you run the above code on the same thread as your GUI, you are essentially blocking the Tk's event loop from running while you are waiting for each line. This means that though you are getting each line in real-time, and writing it to your GUI, it won't update the display until the event loop gets to run again, and process all your calls to Tk.
You need to run the subprocess code on a new thread, and in your GUI thread, periodically check for new input.
I've done an example based on yours for you, you can find it here:
http://pastebin.com/FRFpaeJ2
I have a script that has been working properly for the past 3 months. The Server went down last Monday and since then my script stopped working. The script hangs at coords = p.communicate()[0].split().
Here's a part of the script:
class SelectByLatLon(GridSelector):
def __init__(self, from_lat, to_lat, from_lon, to_lon):
self.from_lat = from_lat
self.to_lat = to_lat
self.from_lon = from_lon
self.to_lon = to_lon
def get_selection(self, file):
p = subprocess.Popen(
[
os.path.join(module_root, 'bin/points_from_latlon.tcl'),
file,
str(self.from_lat), str(self.to_lat), str(self.from_lon), str(self.to_lon)
],
stdout = subprocess.PIPE
)
coords = p.communicate()[0].split()
return ZGridSelection(int(coords[0]), int(coords[1]), int(coords[2]), int(coords[3]))
When I run the script on another server everything works just fine.
Can I use something else instead of p.communicate()[0].split() ?
You might have previously run your server without daemonization i.e., you had functional stdin, stdout, stderr streams. To fix, you could redirect the streams to DEVNULL for the subprocess:
import os
from subprocess import Popen, PIPE
DEVNULL = os.open(os.devnull, os.O_RDWR)
p = Popen(tcl_cmd, stdin=DEVNULL, stdout=PIPE, stderr=DEVNULL, close_fds=True)
os.close(DEVNULL)
.communicate() may wait for EOF on stdout even if tcl_cmd already exited: the tcl script might have spawned a child process that inherited the standard streams and outlived its parent.
If you know that you don't need any stdout after the tcl_cmd exits then you could kill the whole process tree when you detect that tcl_cmd is done.
You might need start_new_session=True analog to be able to kill the whole process tree:
import os
import signal
from threading import Timer
def kill_tree_on_exit(p):
p.wait() # wait for tcl_cmd to exit
os.killpg(p.pid, signal.SIGTERM)
t = Timer(0, kill_tree_on_exit, [p])
t.start()
coords = p.communicate()[0].split()
t.cancel()
See How to terminate a python subprocess launched with shell=True