Well the first problem I run into is that I have no idea how to respond to a command prompt.
bat_location = "F:/SteamLibrary/steamapps/common/Terraria"
os.chdir(bat_location)
os.system("TerrariaServer.exe -steam -lobby friends -config serverconfig.txt")
all of this works, but then when I want to respond to the command prompt which asks me which world I want to run ( the worlds are indexed by numbers from 1 - n (amount of worlds)) I dont know how to respond to it.
I've looked all over google but the code doesnt seem to work.
So basically what I need is when a cmd asks me for example :
Choose World:
I want to automatically respond with the number 10.
os.system("10")
this doesnt seem to do anything, I've also tried a lot with subprocesses but im clearly lost.
Any help is apriciated!
EDIT NR.1 :
Welp, now I've tried this :
bat_location = r'F:\SteamLibrary\steamapps\common\Terraria'
with Popen('TerrariaServer.exe -steam -lobby friends -config serverconfig.txt',
cwd=f'{bat_location}', stdin=PIPE, shell=True) as proc:
proc.stdin.write(b'10\n')
and all it does, im guessing by the response, it just loops around and around.
EDIT NR.2 :
I WILL CLOSE THIS AND START A NEW THREAD, SINCE MY PROBLEM COMPLETELY DERIVED FROM THE ORIGINAL.
From your last few comments, I realized the problem you were having with Popen. When you pass stdout=PIPE and stderr=PIPE, the outputs of the process are captured by the pipes, so you won't see them unless you read from the pipes.
So here's a simple example that you should be able to work with:
import subprocess
from subprocess import PIPE
from textwrap import dedent
with open('tmp.py', 'w') as f:
f.write(dedent("""
print(input())
print(input())
"""))
with subprocess.Popen(['python3', 'tmp.py'], stdin=PIPE) as proc:
proc.stdin.write(b'Hello, world!\n') # write to the process' input
proc.stdin.write(b'Good bye, world!\n') # write to the process' input
If you want to read the data from the function in Python, you can use stdout=PIPE, then use proc.stdout.read and the like, but you may have to be careful about how you get data from the blocking read functions.
Related
I have a python 3 script that accepts input from the user, this input is piped into a subprocess, where a shell has been spawned. Originally I was going to put this code together with a socket, to be able to make my own non-serious remote administration tool, however this proves too hard for my level currently. Code:
import subprocess
p1 = subprocess.Popen(["/bin/sh"], stderr = subprocess.PIPE, stdin = subprocess.PIPE, stdout = subprocess.PIPE, encoding = "utf-8")
command = input("Command: ")
p1.stdin.write(command)
p1.stdout.read()
Problem: Nothing gets printed out. I have searched endless hours
online for a reason, over multiple days, but all of them don't seem to
work, and/or advise using communicate() which is something I do not
want to do. When thinking ahead if I am able to implement this with a
socket, I can't have the process closing after each command. I have
also tried flushes everywhere, before write, inbetween the read, after
the read, pretty much everywhere you can think of. It should be simple
enough, without me having to look deep into the io module or the rules
of buffering (too late now). I have been struggling with this for days
on end.
You have a couple of issues, I will try to draw the path, I hope I am not misleading you.
First of all, disclaimer: executing user input is inherently unsafe. If that's not an issue, let's keep on your problem.
I would start by doing
p1 = subprocess.Popen(["/bin/sh"], stderr=subprocess.PIPE, stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True, bufsize=1)
Having a line-buffered PIPE is a good idea, in this scenario.
Then, remember that you have to press enter after the command, so:
command = input("Command: ") + "\n"
The write is correct,
p1.stdin.write(command)
But the, the read is VERY dangerous and you will trivially become deadlocked. Because the stdout is open (without an EOF or anything like that) you should read conservatively, and still, you will have problems on that.
A first idea would be to read lines:
p1.stdout.readline()
But, if you don't know how many lines you can read... then that's a problem. In fact, that specific problem has already been asked.
If you want to experiment, just open an interactive python interpreter, send a "ls\n" command and perform readline(). It will work, until you read the last line, and then... it will wait forever. That is bad behaviour. So you will need to solve that.
I'm trying to run a shell program through python. I need to run a command, then while it's still running and waiting for input to continue, I need to take the output received by the program, and process that data as a string. Then I need to parse some data into that program, and simulate an enter pressing.
What would be the best way to achieve this?
subprocess.Popen will work for this, but to read and then write and then read again you can't use communicate (because this will cause the process to end).
Instead, you'll need to work with the process's output pipe (process.stdout below). This is tricky to get right, because reading on the process's stdout is blocking, so you sort of need to know when to stop trying to read (or know how much output the process is going to produce).
In this example, the subprocess is a shell script that writes a line of output, and then echoes whatever you give it until it reads EOF.
import subprocess
COMMAND_LINE = 'echo "Hello World!" ; cat'
process = subprocess.Popen(COMMAND_LINE, shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
s = process.stdout.readline().strip()
print(s)
s2 = process.communicate(s)[0]
print(s2)
Gives:
Hello World!
Hello World!
For more complicated cases, you might think about looking at something like pexpect.
Use subprocess.Popen to run your shell application and use communicate to interact with it.
I need to run a external exe file inside a python script. I need two things out of this.
Get whatever the exe outputs to the stdout (stderr).
exe stops executing only after I press the enter Key. I can't change this behavior. I need the script the pass the enter Key input after it gets the output from the previous step.
This is what I have done so far and I am not sure how to go after this.
import subprocess
first = subprocess.Popen(["myexe.exe"],shell=True,stdout=subprocess.PIPE)
from subprocess import Popen, PIPE, STDOUT
first = Popen(['myexe.exe'], stdout=PIPE, stderr=STDOUT, stdin=PIPE)
while first.poll() is None:
data = first.stdout.read()
if b'press enter to' in data:
first.stdin.write(b'\n')
first.stdin.close()
first.stdout.close()
This pipes stdin as well, do not forget to close your open file handles (stdin and stdout are also file handles in a sense).
Also avoid shell=True if at all possible, I use it a lot my self but best practices say you shouldn't.
I assumed python 3 here and stdin and stdout assumes bytes data as input and output.
first.poll() will poll for a exit code of your exe, if none is given it means it's still running.
Some other tips
one tedious thing to do can be to pass arguments to Popen, one neat thing to do is:
import shlex
Popen(shlex.split(cmd_str), shell=False)
It preserves space separated inputs with quotes around them, for instance python myscript.py debug "pass this parameter somewhere" would result in three parameters from sys.argv, ['myscript.py', 'debug', 'pass this parameter somewhere'] - might be useful in the future when working with Popen
Another thing that would be good is to check if there's output in stdout before reading from it, otherwise it might hang the application. To do this you could use select.
Or you could use pexpect which is often used with SSH since it lives in another user space than your application when it asks for input, you need to either fork your exe manually and read from that specific pid with os.read() or use pexpect.
I need to open an R script and supply it with input formulated by a separate python script. The subprocess module seems to be a good way to do this.
I have encountered some puzzling results though, namely that I can apparently write once and only once via p.stdin. Here is what I have so far:
from subprocess import Popen, PIPE, STDOUT
p = Popen(['r --no-save'],stdin=PIPE,stdout=PIPE,stderr=PIPE,shell=True)
p.stdin.write("source('myrscript.R')\n")
p.stdin.write('myfirstinput')
What happens when I run this code is that the first instance of stdin.write() performs as expected (and opens my R script), but the second line does nothing, and the subprocess (really, the R script) exits with an error, indicating that the subprocessed received no input where input was expected and therefore terminated.
N.B. - In a perfect world, I would just interact directly through R, but this particular script requires complex inputs that cannot be entered directly for practical purposes. Also, rpy / rpy2 is not an option, because end-users of this script will not necessarily have access to that module or its dependencies. rscript is also not an option (for many reasons, but mainly because of variability in the end-users R configurations).
Finally, p.communicate is not an option, because apparently that will close the process after writing and I need to keep it open.
Thanks in advance
What you need is to call .communicate():
from subprocess import Popen, PIPE, STDOUT
p = Popen(
['r', '--nosave'],
stdin=PIPE,
stdout=PIPE,
stderr=PIPE)
p.stdin.write("source('myrscript.R')\n")
p.stdin.write('myfirstinput\n')
p.stdin.write('q\n')
stdout, stderr = p.communicate()
print '---STDOUT---'
print stdout
print '---STDERR---'
print stderr
print '---'
Discussion
I don't use the shell=True and it seems working with my fake R script since I don't have R install in my system. You might or might not need it.
I prefer breaking the command line up into a list of string as shown, but a single string such as r --nosave will work as well; just don't do them both at the same time.
Don't forget that stdin.write() does not write the new line character \n, you have to supply that yourself.
Update
My first attempt was off the mark, I hope this second attempt gets closer. As J.F. Sebastian suggested, you might want to use pexpect:
import pexpect
import sys
if __name__ == '__main__':
prompt = '> ' # Don't know what the R prompt looks like
lines = ['one', 'two', 'three']
r = pexpect.spawn('r --no-save', logfile=sys.stdout)
for line in lines:
r.expect(prompt)
r.sendline(line)
# If you want to interact with your script, use these two lines
# Otherwise, comment them out
r.logfile = None # Turn off logging to sys.stdout
r.interact()
Discussion
You might need to install pexpect. I did it with pip install pexpect
If you don't want to interact with the system, comment out the last two line, but make sure to send some signal for the R script to exit.
spawn() returns a spawn object, see doc here.
I'm having troubles getting this to work. Basically I have a python program that expect some data in stdin, that is reading it as sys.stdin.readlines() I have tested this and it is working without problems with things like echo "" | myprogram.py
I have a second program that using the subprocess module calls on the first program with the following code
proc = subprocess.Popen(final_shell_cmd,
stderr=subprocess.PIPE, stdout=subprocess.PIPE,
shell=False), env=shell_env)
f = ' '.join(shell_cmd_args)
#f.append('\4')
return proc.communicate(f)
The second program is a daemon and i have discovered that the second program works well as long as I hit ctrl-d after calling it from the first program.
So it seems there is something wrong with subprocess not closing the file and my first program expecting more input when nothing more should be sending.
anyone has any idea how I can get this working?
The main problem here is that "shell_cmd_args" may contain passwords and other sensitive information that we do not want to pass in as the command name as it will show in tools like "ps".
You want to redirect the subprocess's stdin, so you need stdin=subprocess.PIPE.
You should not need to write Control-D ('\4') to the file object. Control-D tells the shell to close the standard input that's connected to the program. The program doesn't see a Control-D character in that context.