this is my first question, I hope I'm doing this right.
let's say I have these this file:
"simple.py":
a=raw_input("your name?")
print "Hello",a
but with a different script, I want to execute "simple.py" many time and giving the input automatically, that would work like:
"everyone.py"
run simple.py input=Alice
run simple.py input=Bob
...
to get
"Hello Alice"
"Hello Bob"
...
I know it's possible to make "everyone.py" run "simple.py" by using os.system, but is there any practical way to do something like this? And what if the first script asks for input several times?
It's important that I CANNOT EDIT SIMPLE.PY, only the other file
Thanks in advance :)
For a case as simple as simple.py, you should be able to use subprocess.Popen:
import subprocess
child = subprocess.Popen(['python', 'simple.py'], stdin=subprocess.PIPE)
child.communicate('Alice')
For more complex cases, the Pexpect module may be useful. It helps automate interaction with normally-interactive programs by providing more convenient, robust interfaces to send input, wait for prompts, and read output. It should work in cases where Popen doesn't work or is more annoying.
import pexpect
child = pexpect.spawn('python simple.py')
child.expect("your name?")
child.sendline('Alice')
import sys
print "Hello",sys.argv[1]
running C:\Python26\python.exe simple.py Alice would produce
Hello Alice
There's a good example on how to get input from the system into a python application here:
http://www.tutorialspoint.com/python/python_command_line_arguments.htm
Since you didn't mention that you can not modify simple.py then you would need to automaticly input things into the raw_input() and the fastest way to do this is simply to pipe in data into the script as "input":
C:> echo "Alice" | run simple.py
Unfortunately neither of the answers above worked for me so I came up with a third solution for others to try.
To send inputs from one python file to another (python version 3.7), I used three files.
File for running the subprocess
File for outputs (very simple)
File that needs the inputs
Here are the three files in the same order as above.
You don't need to print out the output, but I'll include the terminal output below the file examples.
The subprocess file:
from subprocess import Popen,PIPE
p1 = Popen(["python","output_file.py"], stdout=PIPE)
p2 = Popen(["python", "input_file.py"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()
output = p2.communicate()[0]
print(output)
The output file is very simple and there may be a way to work around it. Nevertheless, here is my version:
print(1)
print(2)
print('My String')
The input file requires type casting for numbers.
i = input('Enter a number: ')
j = input('Enter another: ')
k = int(i) + int(j)
print(k)
l = input('Tell me something. ')
print(l)
Here is the terminal output:
b'Enter a number: Enter another: 3\r\nTell me something. My String!\r\n'
Related
Using Python 3, I want to execute an external program, interact with it by providing some text into standard input, and then print the result.
As an example, I created the following external program, called test.py:
print('Test Program')
print('1 First option, 2 Second Option')
choice = input()
if choice == '1':
second_param = input('Insert second param: ')
result = choice + ' ' + second_param
print(result)
If I run this program directly, it works as expected. If I provide the input 1 and then 2, the result is 1 2.
I want to run this program in another script and interact with it to print the same result.
After reading the documentation for subprocess, and checking out similar questions on SO, I ended up with the following:
EXTERNAL_PROG = 'test.py'
p = Popen(['py', EXTERNAL_PROG], stdout=PIPE, stdin=PIPE, shell=True)
print(p.stdout.readline().decode('utf-8'))
print(p.stdout.readline().decode('utf-8'))
p.stdin.write(b'1\n')
p.stdin.write(b'2\n')
print(p.stdout.readline().decode('utf-8'))
However, when I run the code, the program freezes after printing 1 First option, 2 Second Option, and I need to restart my shell. This is probably caused by the fact that subprocess.stdout.readline() expects to find a newline character, and the prompt for the second param doesn’t contain one.
I found 2 SO questions that talk about something similar but I couldn’t get it to work.
Here, the answer recommends using the pexpect module. I tried to adapt the code to my situation but it didn’t work.
Here, the suggestion is to use -u, but adding it didn’t change anything.
I know that a solution can be found by modifying test.py, but this is not possible in my case since I need to use another external program and this is just a minimal example based on it.
If you have fixed input to your program (means input not changing at run time) then this solution can be relevant.
Answer
First create file.
Input file. name it input.txt and put 1 2 in it
command = "python test.py < input.txt > output.txt 2>&1"
# now run this command
os.system(command)
When you run this, you will find output.txt in same directory. If your program is executed successfully then output.txt contains output of code test.py but if your code gives any error then error is in output.txt.
Answer As You Want
main.py become
import sys
from subprocess import PIPE, Popen
EXTERNAL_PROG = 'test.py'
p = Popen(['python3', EXTERNAL_PROG], stdout=PIPE, stdin=PIPE, stderr=PIPE)
print(p.stdout.readline())
print(p.stdout.readline())
p.stdin.write(b'1\n')
p.stdin.write(b'2\n')
p.stdin.flush()
print(p.stdout.readline())
print(p.stdout.readline())
I encountered a problem in work. Here it is.
I have several scripts(mostly are shell scripts) to execute, and I want to write a python script to run them automatically. One of these shell scripts needs interactive input during it's execution. What troubled me is that I can't find a way to read its input prompt, so I can't decide what to enter to continue.
I simplified the problem to something like this:
There is a script named mediator.py, which run greeter.sh inside. The mediator takes greeter's input prompt and print it to the user, then gets user's input and pass it to greeter. The mediator needs to act exactly the same as the greeter from user's point of view.
Here is greeter.sh:
#! /bin/bash
echo "Please enter your name: " # <- I want 'mediator.py' to read this prompt and show it to me, and then get what I input, then pass my input to 'greeter.sh'
read name
echo "Hello, " $name
I want to do this in the following order:
The user (that's me) run mediator.py
The mediator run greeter.sh inside
The mediator get the input prompt of greeter, and output it on the screen.(At this time, the greeter is waiting for user's input. This is the main problem I stuck with)
The user input a string (for example, 'Mike'), mediator get the string 'Mike' and transmit it to greeter
The greeter get the name 'Mike', and print a greeting
The mediator get the greeting, and output it on the screen.
I searched for some solution and determined to use Popen function in subprocess module with stdout of sub-process directed to PIPE, it's something like this:
sb = subprocess.Popen(['sh', 'greeter.sh'], stdout = subprocess.PIPE, stdin = stdout, stderr = stdout)
but I can't solve the main problem in step 3 above. Can anyone give me some advice for help? Thanks very much!
You make it much more complicated (and brittle) than it has to be. Instead of coding everything at the top-level and try to use subprocess or whatever to use your scripts as if they where functions, just write modules and functions and use them from your main script.
Here's an example with all contained in the script itself, but you can split it into distinct modules if you need to share some functions between different scripts
# main.py
def ask_name():
return raw_input("Please enter your name: ")
def greet(name):
return "Hello, {} name !\n".format(name)
def main():
name = ask_name()
print greet(name)
if __name__ == "__main__":
main()
I am trying to learn how to write a script control.py, that runs another script test.py in a loop for a certain number of times, in each run, reads its output and halts it if some predefined output is printed (e.g. the text 'stop now'), and the loop continues its iteration (once test.py has finished, either on its own, or by force). So something along the lines:
for i in range(n):
os.system('test.py someargument')
if output == 'stop now': #stop the current test.py process and continue with next iteration
#output here is supposed to contain what test.py prints
The problem with the above is that, it does not check the output of test.py as it is running, instead it waits until test.py process is finished on its own, right?
Basically trying to learn how I can use a python script to control another one, as it is running. (e.g. having access to what it prints and so on).
Finally, is it possible to run test.py in a new terminal (i.e. not in control.py's terminal) and still achieve the above goals?
An attempt:
test.py is this:
from itertools import permutations
import random as random
perms = [''.join(p) for p in permutations('stop')]
for i in range(1000000):
rand_ind = random.randrange(0,len(perms))
print perms[rand_ind]
And control.py is this: (following Marc's suggestion)
import subprocess
command = ["python", "test.py"]
n = 10
for i in range(n):
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while True:
output = p.stdout.readline().strip()
print output
#if output == '' and p.poll() is not None:
# break
if output == 'stop':
print 'sucess'
p.kill()
break
#Do whatever you want
#rc = p.poll() #Exit Code
You can use subprocess module or also the os.popen
os.popen(command[, mode[, bufsize]])
Open a pipe to or from command. The return value is an open file object connected to the pipe, which can be read or written depending on whether mode is 'r' (default) or 'w'.
With subprocess I would suggest
subprocess.call(['python.exe', command])
or the subprocess.Popen --> that is similar to os.popen (for instance)
With popen you can read the connected object/file and check whether "Stop now" is there.
The os.system is not deprecated and you can use as well (but you won't get a object from that), you can just check if return at the end of execution.
From subprocess.call you can run it in a new terminal or if you want to call multiple times ONLY the test.py --> than you can put your script in a def main() and run the main as much as you want till the "Stop now" is generated.
Hope this solve your query :-) otherwise comment again.
Looking at what you wrote above you can also redirect the output to a file directly from the OS call --> os.system(test.py *args >> /tmp/mickey.txt) then you can check at each round the file.
As said the popen is an object file that you can access.
What you are hinting at in your comment to Marc Cabos' answer is Threading
There are several ways Python can use the functionality of other files. If the content of test.py can be encapsulated in a function or class, then you can import the relevant parts into your program, giving you greater access to the runnings of that code.
As described in other answers you can use the stdout of a script, running it in a subprocess. This could give you separate terminal outputs as you require.
However if you want to run the test.py concurrently and access variables as they are changed then you need to consider threading.
Yes you can use Python to control another program using stdin/stdout, but when using another process output often there is a problem of buffering, in other words the other process doesn't really output anything until it's done.
There are even cases in which the output is buffered or not depending on if the program is started from a terminal or not.
If you are the author of both programs then probably is better using another interprocess channel where the flushing is explicitly controlled by the code, like sockets.
You can use the "subprocess" library for that.
import subprocess
command = ["python", "test.py", "someargument"]
for i in range(n):
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while True:
output = p.stdout.readline()
if output == '' and p.poll() is not None:
break
if output == 'stop now':
#Do whatever you want
rc = p.poll() #Exit Code
I am trying to get a basic idea of how the os.system module can be used to execute 7zip commands. For now I don't want to complicate things with Popen or subprocess. I have installed 7zip and copied the 7zip.exe into my users folder. I just want to extract my test file install.zip. However using the code below causes the shell to appear briefly before exiting and no unzip has occurred. Please could you tell me why?
def main():
try:
os.system(r"C:\Users\Oulton\ 7z e C:\Users\Oulton\install.zip")
except:
time.sleep(3)
traceback.print_exc
if __name__ == "__main__":
main()
Many Thanks
There are several problems with the following line:
os.system("C:\Users\Oulton\ 7z e C:\Users\Oulton\install.zip ")
Since your string contains backslashes, you should use a raw string:
os.system(r"C:\Users\Oulton\7z -e C:\Users\Oulton\install.zip")
(note the r before the first double quote.)
I've also removed the extraneous spaces. The first one (before the 7z) was definitely problematic.
Also note that the traceback.print_exc does not call the function. You need to add parentheses: traceback.print_exc().
Finally, it is recommended that in new code the subprocess module is used in preference to os.system().
Can be done using sub process module:
import subprocess
beforezip = D:\kr\file #full location
afterzip = filename.zip
Unzipped_file = "7z a \"%s\" \"%s\"" %( afterzip, beforezip )
retV = subprocess.Popen(cmdExtractISO, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
outData = retV.stdout.readlines();
How do I perform logging of all activities that are done by a Python script and all scripts that are called from it?
I had several Bash scripts but now wrote a Python script which call all of these Bash scripts. I would like to have all output produced from these scripts stored in some file.
The script is interactive Python script, i.e contains raw_input lines, so I couldn't do like 'python script.py | tee log.txt' for overall the Python script since for some reasons questions are not seen on the screen.
Here is an excerpt from the script which calls one of the shell scripts.
cmd = "somescript.sh"
try:
retvalue = subprocess.check_call(cmd, shell=True)
except subprocess.CalledProcessError:
print ("script command has been failed")
sys.exit("exit from script")
What do you think could be done here?
Edit
Two subquestions based on Alex's answer:
How to make the answers on the questions stored in the output file as well? For example on line ok = raw_input(prompt) the user will be asked for the question and I would like to the answer logged as well.
I read about Popen and communicate and didn't use since it buffers the data in memory. Here the amount of output is big and I need to care about standard-error with standard-output as well. Do you know if this is possible to handle with Popen and communicate method as well?
Making Python's own prints go to both the terminal and a file is not hard:
>>> import sys
>>> class tee(object):
... def __init__(self, fn='/tmp/foo.txt'):
... self.o = sys.stdout
... self.f = open(fn, 'w')
... def write(self, s):
... self.o.write(s)
... self.f.write(s)
...
>>> sys.stdout = tee()
>>> print('hello world!')
hello world!
>>>
$ cat /tmp/foo.txt
hello world!
This should work both in Python 2 and Python 3.
To similarly direct the output from subcommands, don't use
retvalue = subprocess.check_call(cmd, shell=True)
which lets cmd's output go to its regular "standard output", but rather grab and re-emit it yourself, as follows:
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
so, se = p.communicate()
print(so)
retvalue = p.returncode
assuming you don't care about standard-error (only standard-output) and the amount of output from cmd is reasonably small (since .communicate buffers that data in memory) -- it's easy to tweak if either assumption doesn't correspond to what you exactly want.
Edit: the OP has now clarified the specs in a long comment to this answer:
How to make the answers on the
questions stored in the output file
as well? For example on line ok =
raw_input(prompt) the user will be
asked for the question and I would
like to the answer logged as well.
Use a function such as:
def echoed_input(prompt):
response = raw_input(prompt)
sys.stdout.f.write(response)
return response
instead of just raw_input in your application code (of course, this is written specifically to cooperate with the tee class I showed above).
I read about Popen and communicate
and didn't use since it buffers the
data in memory. Here amount of output
is big and I need to care about
standard-error with standard-output
as well. Do you know if this is
possible to handle with Popen and
communicate method as well?
communicate is fine as long as you don't get more output (and standard-error) than comfortably fits in memory, say a few gigabytes at most depending on the kind of machine you're using.
If this hypothesis is met, just recode the above as, instead:
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
so, se = p.communicate()
print(so)
retvalue = p.returncode
i.e., just redirect the subcommand's stderr to get mixed into its stdout.
If you DO have to worry about gigabytes (or whatever) coming at you, then
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
for line in p.stdout:
sys.stdout.write(p)
p.wait()
retvalue = p.returncode
(which gets and emits one line at a time) may be preferable (this depends on cmd not expecting anything from its standard input, of course... because, if it is expecting anything, it's not going to get it, and the problem starts to become challenging;-).
Python has a tracing module: trace. Usage: python -m trace --trace file.py
If you want to capture the output of any script, then on a *nix-y system you can redirect stdout and stderr to a file:
./script.py >> /tmp/outputs.txt 2>> /tmp/outputs.txt
If you want everything done by the scripts, not just what they print, then the python trace module won't trace things done by external scripts that your python executes. The only thing that can trace every action done by a program would be something like DTrace, if you are lucky enough to have a system that supports it. (OS X Instruments are based on DTrace)