I am trying to make a python program communicate with a minecraft server I am hosting, but the problem I am finding is when the console output from the server, it freezes the program. I am using Popen and PIPE from subprocess, and whenever I use process.stdout.readline() it prints all the lines and when it is done, it freezes the program, and then I am not able to execute any more commands because of that.
def start(cmd):
try:
process = Popen(cmd, stdin=PIPE, stdout=PIPE)
except Exception as e:
print(e)
return process
start('java -Xmx1024M -Xms1024M -jar server.jar nogui') # runs the minecraft server with 8GB of RAM
while True:
print(process.stdout.readline()) # prints out every line in the server console
readmail() # the program checks email for commands, and enters the command into the console using stdin
I have tried this:
def start(cmd):
try:
process = Popen(['python', '-u', cmd], stdin=PIPE, stdout=PIPE)
except Exception as e:
print(e)
return process
It fixes the issue with process.stdout.readline(), but it gives me an error because the command is meant to be called in cmd and not with python. Does anyone know how to fix the issue with readline()?
Related
I have a python test that startes a tcp server as subprocess.
def test_client(self):
if sys.platform.startswith('linux'):
proc = subprocess.Popen([f'{self.bin_output_path}/CaptureUnitHalServer'], stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
else:
proc = subprocess.Popen([f'{self.bin_output_path}/CaptureUnitHalServer'], stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
hal_access = HalAccess(port=12309)
hal = CaptureUnitHal(hal_access)
response = hal.test_scratch(TestScratchReq([1]))
assert TestScratchCnf(verdict=True, return_value=5) == response
print(f"\nRESPONSE: {response}\n")
# The first proc.communicate waits for the subprocess to finish. As the server runs forever a TimeoutExpired
# error is thrown the second proc.communicate in the exception handler get stdout and stderr from the server
try:
outs, errs = proc.communicate(timeout=2)
print(f'{outs.decode()}\n')
except subprocess.TimeoutExpired:
proc.kill()
outs, errs = proc.communicate()
print("CaptureUnitHalServer stdout:")
print(f'{outs.decode()}\n')
print("CaptureUnitHalServer stderr:")
print(f'{errs.decode()}\n')
The hal is a simple tcp client that sends a test request (TestScratchReq) to it and receives the response.
In linux this works perfectly fine. But when I run this in windows The code blocks forever at the line response = hal.test_scratch(TestScratchReq([1])).
This line calls
def send_and_receive(self, request: str) -> str:
self._send(request)
return self._receive()
and blocks in the socket.recv() call in self._receive()
data = self._socket.recv(1024).decode(encoding=self.MESSAGE_ENCODING) # blocking
So it seems like the server is not started properly as a subprocess in windows when calling subprocess.Popen().
The following command shows the port as listening however:
Get-NetTCPConnection -State Listen | grep 12309
0.0.0.0 12309 0.0.0.0 0 Listen
I have a second implementation that is also working on windows:
def test_client(self):
daemon = Thread(target=self.server_thread, daemon=True, name='HalServer')
daemon.start()
hal_access = HalAccess(port=12309)
hal = CaptureUnitHal(hal_access)
response = hal.test_scratch(TestScratchReq([1]))
print(response)
assert TestScratchCnf(verdict=True, return_value=5) == response
daemon.join() # wait for daemon timeout
def server_thread(self):
if sys.platform.startswith('linux'):
result = subprocess.run([f'{self.bin_output_path}/CaptureUnitHalServer'], timeout=5, stdout=subprocess.PIPE)
else:
result = subprocess.run([f'{self.bin_output_path}/CaptureUnitHalServer'], timeout=5, creationflags=subprocess.CREATE_NEW_CONSOLE)
print(result.stdout.decode())
print(result.stderr.decode())
pass
But it seems overcomplicated to me to have another Thread just to start the blocking subprocess.run call that throws a TimeoutExpired error after 5 seconds. A problem with this solution is also that I don't get the stdout and stderr of the subprocess respectively the following two lines of code don't print anything as the Thread is killed by the exception before it reaches these lines.
print(result.stdout.decode())
print(result.stderr.decode())
EDIT:
My question is: Why is the windows version in the first version of the code blocking? How does subprocess.Popen() differ between linux and windows? Is the subprocess not started properly in the windows case?
Hi I am trying to create a function which remotely executes my packet sniffing script on my raspberry pi using paramiko and ssh.
def startPacketReceiver():
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(AutoAddPolicy())
ssh.connect(RECV_IP_ADDRESS, username="pi", password="raspberry")
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command("sudo gcc Code/test.c && sudo ./a.out")
print("Done")
The test.c file is the packet sniffing script. It will only terminate with a CTRL-C (or equivalent method). It does not terminate naturally/eventually.
I want to be able to start the receiver and then quit the receiver e.g:
startPacketReceiver()
...
stopPacketReceiver()
Currently when I run the python script I never get the "Done" print message, meaning that the program is hung on the exec_command and will not continue until it is terminated.
Additional Info
The test.c file loops infinitely, essentially:
while(1)
{
saddr_size = sizeof saddr;
//Receive a packet
data_size = recvfrom(sock_raw , buffer , 65536 , 0 , &saddr , (socklen_t*)&saddr_size);
//fprintf(stderr,"%d",data_size);
if(data_size <0 )
{
fprintf(stderr,"Failed to get packet\n");
printf("Recvfrom error , failed to get packets\n");
return 1;
}
//Now process the packet
ProcessPacket(buffer , data_size);
}
and so to stop it you must CTRL-C it.
You need to send your password to the sudo command. Please enable tty mode for executing your command by passing get_pty = True argument to exec_command function call. And then you need to pass your password through ssh_stdin file interface.
def startPacketReceiver():
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(AutoAddPolicy())
ssh.connect(RECV_IP_ADDRESS, username="pi", password="raspberry")
ssh.stdin, ssh_stdout, ssh_stderr = ssh.exec_command("gcc Code/test.c && sudo ./a.out", get_pty=True)
print("raspberry", ssh_stdin) # Your password for sudo command
print("Done")
return ssh, ssh_stdin, ssh_stdout, ssh_stderr
And then you can write your stopPacketReceiver to send Ctrl-C signal.
def stopPacketReceiver(ssh, ssh_stdin, ssh_stdout, ssh_stderr):
print('\x03', file=ssh_stdin) # Send Ctrl-C signal
print(ssh_stdout.read()) #print the stdout
print(ssh_stderr.read())
I suggest taking a look at daemon(3)
Then you can capture SIGNALS or reopen standard input.
Depends on what you would like to do with your python script.
(If you don't want to use any python any python library other than sys and os)
EDIT:
I am pretty sure when the python script will terminate the ssh connection will obviously close and so any program running on that tty by the user who initiated the connection will be terminated.
For that reason your c program needs to be daemonized and may need
to have its uid and/or euid changed.
Despite that i tried to reproduce your code and I ran into a similar problem: the python script was running the command then print "Done"
but as I tried to read stdout, the entire script was pause.
I think it was waiting for the script return status.
So i did the following changes:
try:
port = '22'
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect('<host_name>', port=22, username='<username>',
password='<password>')
chan = client.get_transport().open_session()
chan.get_pty()
chan.exec_command("cd <script_directory>;\
; gcc test.c; ./a.out")
while True:
print(chan.recv(1024))
finally:
client.close()
The while loop here is to get output of the c program.
But if you close the python script the c program will follow.
I did not dug to much into this library.
EDIT 2:
Look up nohup
if you don't want to use de daemon approach.
I used the following Python script to run an executable remotely and I can send input data through stdin.
from pypsexec.client import Client
c = Client("my_ip", username="myusername", password="mypassword")
c.connect()
try:
c.create_service()
stdin b'first-input\nsecond-input\n'
stdout, stderr, rc = c.run_executable("app.exe", stdin=stdin)
print(stdout)
finally:
c.remove_service()
c.disconnect()
I need to send my data after executing my application.
I have a problem with terminating processes on timeout. Basically, I am running one linux command in for loop (same command for a list of files):
for target in targets:
try:
result = subprocess.run(['filerunner', 'work', '-target=' + target],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=600)
logging.info(result.stdout.decode('utf-8')) # logging out to log file
logging.info(result.stderr.decode('utf-8')) # logging err to log file
except subprocess.TimeoutExpired:
logging.info('Operation failed due to process timeout (10 minutes).')
# result.kill() tried with this to kill process manually
time.sleep(1)
pass
I tried few things, but processes are not being killed after timeout expired. How can I do it?
Trying to send 'powershell' command through telnet (from linux to windows) and fail on Timeout.
other commands i send through telnet, such as 'dir' command are ok.
this is part of the code i'm using:
p = host.pobject()
p.cmd = cmd
child = self.connection or self.OpenTelnetConnection()
t = stopwatch.Timer()
try:
child.sendline('{0}\r'.format(cmd))
child.expect(self.prompt, timeout=timeout)
# output = child.before
output = child.after
if stdout:
sys.stdout.write(child.after)
sys.stdout.flush()
child.sendline('echo %errorlevel%\r')
child.expect(self.prompt)
p.rc = int(child.after.split("\r\n")[1])
p.runtime = t.stop()
if p.rc:
p.stderr = output.split("\r\n")[1:-1]
else:
p.stdout = output.split("\r\n")[1:-1]
return p
except Exception, e:
self.report.Error("Failed to run command {0}. {1}".format(cmd, e),
exception=["TestFailure"], testName="WindowsHost")
The solution i found is to send the powershell command as 1st argument.
for example if i want to send 'host' command to the powershell i'll send:
'powershell host'