We Keep Coding

How to write a code to get another python script output that require an input

I want to give score to several python scripts written by different people and i want to automate the answer check since the given question is same. So we send an input to another python file, we get the output (the terminal/console output) then we compare it, all that within a python file (Like hackerrank, uri, or another competitive programming website)
For example the problem is to multiply the given input by 2. Then i have one python script answer_check.py to automate answer checking, and i have another python script which is one of the answer a.py.
a.py:
a= int(input('input a: '))
res= a*2
print(res)
answer_check.py:
# Some code to send input to a.py
# Then some code to get the console output from a given input
if given_output==desired_output:
score= 100
What i have tried:
I have read some other stackoverflow post that related to this problem but it is kinda different because either they don't have input() in the answer file they want to check, or they do input via sys.args .
I have tried pexpect but but apparently it doesn't apply to windows os
I have tried wexpect it is like pexpect but for windows, but i have an installation problem with pywin32
I tried runpy but we have to input manually
I tried subprocess module
from subprocess import Popen, PIPE
p = Popen("python a.py", stdin=PIPE, stdout=PIPE, shell=False)
out = p.communicate(input='1', timeout=5)
print(out)
But it give me this error
File "a.py", line 1, in <module>
a= input('input a: ')
EOFError: EOF when reading a line
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='cp1252'>
OSError: [Errno 22] Invalid argument
If you know please answer even though it is on another language :)

subprocess.Popen.communicate docs claims that
Interact with process: Send data to stdin. Read data from stdout and
stderr, until end-of-file is reached. Wait for process to terminate
and set the returncode attribute. The optional input argument should
be data to be sent to the child process, or None, if no data should be
sent to the child. If streams were opened in text mode, input must be
a string. Otherwise, it must be bytes.
So you should provide bytes, not str, that is your example should be altered to
from subprocess import Popen, PIPE
p = Popen("python a.py", stdin=PIPE, stdout=PIPE, shell=False)
out = p.communicate(input=b'1', timeout=5)
print(out)
If you need to prepare input from str use .encode() for example
from subprocess import Popen, PIPE
myinput = '1'
p = Popen("python a.py", stdin=PIPE, stdout=PIPE, shell=False)
out = p.communicate(input=myinput.encode(), timeout=5)
print(out)

A much better design is to refactor the code to not require interactive I/O.
a.py:
def make_res(a):
return a*2
def main():
a = input('input a: ')
res = make_res(a)
print(res)
if __name__ == "__main__":
main()
answer_check.py:
from .a import make_res
if make_res(value) == desired_output:
score = 100

After a bit of testing myself, you could use subprocess to invoke the script to be tested from the test script and then import the script being tested as a module and call the variables. I set it up like this
import importlib
import subprocess
script='a'
subprocess.call(script+'.py', shell=True)
script=importlib.import_module(script, package=None)
a=script.a
res=script.res
if a*2 == res:
score=100
And then the tested script needs a small correction to make the input actually be a number, so just put int() around the input like this
a= int(input('input a: '))
res= a*2

Missing: timeout=None
import pexpect
child = pexpect.spawn('python3 test.py')
fout = open('mylog.txt','wb')
child.logfile = fout
child.expect('input a: ', timeout=None)
child.sendline('2')
child.expect('22', timeout=None)

Start background process in python via subprocess and write output to file

Dear stackoverflow users,
I'm looking for a solution for a probably quite easy problem. I want to automate some quantum chemical calculations and ran into a small problem.
Normally you start your quantum chemical programm (in my case it's called orca) with your input file (*.inp) on a remote server as a background process and pipe the output into an outputfile (*.out) via
nohup orca H2.inp >& H2.out &
or something similar.
Now I wanted to use a python script (with some templating) to write the input file automatically. At the end the script should start the calculation in a way that I could log out of the server without stopping orca. I tried that with
subprocess.run(["orca", input_file], stdout=output_file)
but so far it did not work. How do I "emulate" the command given at the top with the subprocess module?
Regards
Update
I have one file that is called H2.xyz. The script reads and splits the filename by the point and creates an input file name H2.inp and the output should be written into the file H2.out.
Update 2
The input file is derived from the *xyz file via
xyzfile = str(sys.argv[1])
input_file = xyzfile.split(".")[0] + ".inp"
output_file = xyzfile.split(".")[0] + ".out"
and is created within the script via templating. In the end I want to run the script in the following way:
python3 script.py H2_0_1.xyz

Why not simply:
subprocess.Popen(f'orca {input_file} >& {output_file}',
shell=True, stdin=None, stdout=None, stderr=None, close_fds=True)
More info:
Run Process and Don't Wait

For me (Windows, Python 2.7) the method call works very fine like this:
with open('H2.out', 'a') as out :
subprocess.call(['orca', infile], stdout=out,
stderr=out,
shell=True) # Yes, I know. But It's Windows.
On Linux you maybe do not need shell=True for a list of arguments.

Is the usage of subprocess important? If not, you could use os.system.
The Python call would get really short, in your case
os.system("nohup orca H2.inp >& H2.out &")
should do the trick.

I had the same problem not long ago.
Here is my solution:
commandLineCode = "nohup orca H2.inp >& H2.out &"
try:
proc = subprocess.Popen(commandLineCode,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
cwd = workingDir)
except OSError:
print("Windows Error occured")
print(traceback.format_exc())
timeoutInSeconds = 100
try:
outs, errs = proc.communicate(timeout = timeoutInSeconds)
except subprocess.TimeoutExpired:
print("timeout")
proc.kill()
outs, errs = proc.communicate()
stdoutDecode = outs.decode("utf-8")
stderrDecode = errs.decode("utf-8")
for line in stdoutDecode.splitlines():
# write line to outputFile
if stderrDecode:
for line in stderrDecode.splitlines():
# write line to error log
The OSError exception is pretty important since you never now what your OS might do wrong.
For more on the communicate() command which actually starts the process read:
https://docs.python.org/3/library/subprocess.html#subprocess.Popen.communicate

Python subprocess stdout iterator always blocks

I want to capture the output while printing it, but I'm blocking forever without reading even a single line. What's going on? I'm using Python2.
Generator script:
#!/usr/bin/env python2.7
import random
import time
while True:
print(random.random())
time.sleep(1)
Sample generator output:
$ ./generator.py
0.334835137212
0.896609571236
0.833267988558
0.55456332113
^CTraceback (most recent call last):
Reader script:
import subprocess
cmd = ['./generator.py']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
for line in p.stdout:
print(line)
print('Looping')
p.wait()
I've also tried:
import subprocess
import sys
cmd = ['./generator.py']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
while True:
line = p.stdout.readline()
print(line)
print('Looping')
p.wait()
...and:
import sys
import subprocess
import select
import time
cmd = ['./generator.py']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
s = select.poll()
s.register(p.stdout, select.POLLIN)
while True:
if s.poll(1):
line = p.stdout.read()
else:
p.poll()
if p.returncode is not None:
break
print('Looping')
time.sleep(1)
p.wait()

As #dhke mentioned, one of the issues is implicit output-buffering in the producer. If you have the ability to change the producer, and you're willing to, and the production is done by calls to the print-function then just add "flush=True" as an argument to the print function. You can also fall-back to doing a sys.stdout.flush() at key points in the producer.
The second problem appears to be iterating over sys.stdout. This never seems to work for a long-running process. The second and third methods

I'm dealing with a similar problem. This is the workaround im currently using to prevent buffering.
proc = subprocess.Popen(['stdbuf', '-o0'] + cmd, stdout=subprocess.PIPE)
The disadvantage of this methood is that it relys on an external Linux command to solve the problem. Have a look in the comments here for a different and native python approach to get rid of the PIPE buffering. Many thanks to #9000 for suggesting both solutions to me.

Understanding subprocess module in python when running program from python script

I usually run a program from my OpenSuse linux terminal by typing ./run file_name. This will bring up a series of options that I can choose from by typing a numeric value 0-9 and hitting return on my keyboard. Now I want to do this from a python script automatically. My example below is not working, but I can't understand where I'm failing and how to debug:
import subprocess
p = subprocess.Popen(["/path/to/program/run", file_name], stdin = subprocess.PIPE,stdout=subprocess.PIPE,shell=False)
print "Hello"
out, err = p.communicate(input='0\r\n')
print out
print err
for line in p.stdout.readlines():
print line
The output of this program is just
>> Hello
>>
i.e. then it seems to freeze (I have no idea whats actually happening!) I would have expected to see what I see when I run ./run file_name
and hit 0 and then return directly in my terminal, but I assure you this is not the case.
What can I do to debug my code?
Edit 1: as suggested in comments
import subprocess
fileName = 'test_profile'
p = subprocess.Popen(["/path/to/program/run", fileName], stdin = subprocess.PIPE,stdout=subprocess.PIPE,shell=False)
print "Hello"
for line in iter(p.stdout.readline,""):
print line
will indeed return the stdout of my program!

communicate waits for the completion of the program. For example:
import subprocess
p = subprocess.Popen(["cut", "-c2"], stdin=subprocess.PIPE, stdout=subprocess.PIPE,shell=False)
out, err = p.communicate(input='abc')
print("Result: '{}'".format(out.strip()))
# Result: 'b'
It sounds like you have a more interactive script, in which case you probably should try out pexpect
import pexpect
child = pexpect.spawn('cut -c2')
child.sendline('abc')
child.readline() # repeat what was typed
print(child.readline()) # prints 'b'

Linux, Write output to the file and terminate it using Python

There are various topics available on this very topic, "How to write output to the text file".
But my issue is different because the output to the command in question is continous.
What I want to do is, write the Output of the command cgps -s to the file aaa.txt
here is the code,
import signal
import os
import subprocess
p = subprocess.Popen(["cgps", "-s", ">> aaa.txt"], stdout=subprocess.PIPE,shell=True, preexec_fn=os.setsid)
os.killpg(p.pid, signal.SIGTERM)
The code doesn't work at all and no file is created with the name aaa.txt
When I execute this command through terminal,
cgps -s > aaa.txt
Then I have to press CTRL+C to terminate the output from being written on the output file because the output is continuous.
Is there any way to just capture one output and write it to the file and terminate it using Python or using Command line ?

So you are trying to create a pipe. Try this:
import subprocess
p = subprocess.Popen(["cgps", "-s"], stdout=subprocess.PIPE)
with open("aaa.txt", "w") as f:
while True:
line = p.stdout.readline()
if not line:
break
if some_condition(): # <-- check whether you have enough output
p.terminate()
break
f.writeline(line)
p.wait()