I am using Python 2.7.3 in Ubuntu 12.04 OS. I have an external program say 'xyz' whose input is a single file and two files say 'abc.dat' and 'gef.dat' are its output.
When I used os.system or subprocess.check_output or os.popen none of them printed the output files in the working directory.
I need these output files for further calculations.
Plus I've to keep calling the 'xyz' program 'n' times and have to keep getting the output 'abc.dat' and 'gef.dat' every time from it. Please help.
Thank you
I can not comment on your question because my reputation is too low.
If you use os.system or subprocess.check_output or os.popen, you will just get the standard output of your xyz program (if it is printing something in the screen). To see the files in some directory, you can use os.listdir(). Then you can use these files in your script afterwards. It may also be worth using subprocess.check_call.
There may be other better and more efficient solutions.
First, run the program which you invoked in python script directly and see if it generates those two files.
Assume it does, the problem is in your python script. Try using subprocess.Popen then call communicate().
Here's an example:
from subprocess import Popen
p = Popen(["xyz",])
p.communicate()
communicate waits for process to terminate. you should be able to get output files when executing code after p.communicate().
Thank you for answering my question but the answer to my question is this -
import subprocess
subprocess.call("/path/to/software/xyz abc.dat", shell=True)
which gave me the desired the output.
I tried the subprocess-related commands but they returned error " No such file or directory". The 'shell=True' worked like a charm.
Thank you all again for taking your time to answer my question.
Related
I'm new in this world of python. Recently I have been asked to make an interface between XFoil (an aerodynamics program) and python. After researching a little bit, I found the subprocess module. As the documentation says it's used to "Spawn new processes, connect to their input/output/error pipes, and obtain their return codes."
The problem is that I need some output archives that XFoil creates while its running. If I close the program, the archives are accesible, but if I try to open or read them while the subprocess is still opened it gives me the following error (Although I can see the archive in the folder):
OSError: save not found.
Here the code:
import subprocess
import numpy as np
import os
process = subprocess.Popen(['<xfoil_path>'], stdin=subprocess.PIPE, universal_newlines=True, creationflags = subprocess.CREATE_NEW_PROCESS_GROUP)
airfoil_path = '<path to airfoil>'
process.stdin.write(f'\nload\n{airfoil_path}')
process.stdin.write('\n\n\noper\nalfa\n2\ncpwr\nsave\n')
process.stdin.tell()
print(os.listdir())
c = np.loadtxt('save', skiprows=1)
print(c)
process.stdin.write('\n\n\noper\nalfa\n3\ncpwr\nsave2\n')
stdin.tell is used to get this output archives, but they are not accesible.
Someone knows why this could be happening?
Why do you imagine process.stdin.tell() should "get this output archives"? It retrieves the file pointer's position.
I'm imagining that the actual problem here is that the subprocess doesn't write the files immediately. Maybe just time.sleep(1) before you try to open them, or figure out a way for it to tell you when it's done writing (some OSes let you tell whether another process has a file open for writing, but I have no idea whether this is possible on Windows, let alone reliable).
Sleeping for an arbitrary time is obviously not very robust; you can't predict how long it takes for the subprocess to write out the files. But if that solves your immediate problem, at least you have some idea of what caused it and maybe where to look next.
As an aside, maybe look into the input= keyword parameter for subprocess.run(). If having the subprocess run in parallel is not crucial, that might be more pleasant as well as more robust.
(Converted into an answer from a comment thread.)
I'm writing a code to read serial input. Once the serial input has been read, I have to add a time stamp below it and then the output from a certain software. To get the output from the software, I want python to write a certain command to the terminal, and then read the output that comes on the terminal. Could you suggest how do I go about doing the last step: namely, writing to the terminal then reading the output? I'm a beginner in python, so please excuse me if this sounds trivial.
To run a command and get the returned output you can use the subprocess module's check_output function.
import subprocess
output = subprocess.check_output("ls -a", shell=True)
That will return the current directory contents in MacOS/Linux and store the output for you to read from later in your program. The "shell=True" allows you to execute a command as a string "ls -a". If you do not use "shell=True" you will pass the command as a list of each part of the command, example subprocess.check_output(["ls", "-a"]). Subprocess is a great module included with Python that allows a lot of command line execution.
So with subprocess you should be able to call another program, code, command, etc.by using a shell command.
You would need to have python implemented into the software.
Also, I believe this is a task for GCSE Computing this year as I was privileged enough to choose what test we are doing and there was a question about serial numbers.
There is a program written and compiled in C, with typical data input from a Unix shell; on the other hand, I'm using Windows.
I need to send input to this program from the output of my own code written in Python.
What is the best way to go about doing this? I've read about pexpect, but not sure really how to implement it; can anyone explain the best way to go about this?
i recommend you use the python subprocess module.
it is the replacement of the os.popen() function call, and it allows to execute a program while interacting with its standard input/output/error streams through pipes.
example use:
import subprocess
process = subprocess.Popen("test.exe", stdin=subprocess.PIPE, stdout=subprocess.PIPE)
input,output = process.stdin,process.stdout
input.write("hello world !")
print(output.read().decode('latin1'))
input.close()
output.close()
status = process.wait()
If you don't need to deal with responding to interactive questions and prompts, don't bother with pexpect, just use subprocess.communicate, as suggested by Adrien Plisson.
However, if you do need pexpect, the best way to get started is to look through the examples on its home page, then start reading the documentation once you've got a handle on what exactly you need to do.
I am using below code for running unix commands:
cmd = 'ls -l'
(status,output) = commands.getstatusoutput(cmd)
print output
But the problem is that it shows output only after the command completed, but i want to see the output printed as the execution progresses.
ls -l is just dummy command, i am using some complex command in actual program.
Thanks!!
Since this is homework, here's what to do instead of the full solution:
Use the subprocess.Popen class to call the executable. Note that the constructor takes a named stdout argument, and take a look at subprocess.PIPE.
Read from the Popen object's STDOUT pipe in a separate thread to avoid dead locks. See the threading module.
Wait until the subprocess has finished (see Popen.wait).
Wait until the thread has finished processing the output (see Thread.join). Note that this may very well happen after the subprocess has finished.
If you need more help please describe your precise problem.
Unless there are simpler ways in Python which I'm not aware of, I believe you'll have to dig into the slightly more complex os.fork and os.pipe functions.
Basically, the idea is to fork your process, have the child execute your command, while having its standard output redirected to a pipe which will be read by the parent. You'll easily find examples of this kind of pattern.
Most programs will use block buffered output if they are not connected to a tty, so you need to run the program connected to a pty; the easiest way is to use pexpect:
for line in pexpect.spawn('command arg1 arg2'):
print line
I'm trying to parse some data and make graphs with python and there's an odd issue coming up. A call to os.system() seems to get lost somewhere.
The following three lines:
os.system('echo foo bar')
os.system('gnuplot test.gnuplot')
os.system('gnuplot --version')
Should print:
foo bar
Warning: empty x range [2012:2012], adjusting to [1991.88:2032.12]
gnuplot 4.4 patchlevel 2
But the only significant command in the middle seems to get dropped. The script still runs the echo and version check, and running gnuplot by itself (the gnuplot shell) works too, but there is no warning and no file output from gnuplot.
Why is this command dropped, and why completely silently?
In case it's helpful, the invocation should start gnuplot, it should open a couple of files (the instructions and a data file indicated therein) and write out to an SVG file. I tried deleting the target file so it wouldn't have to overwrite, but to no avail.
This is python 3.2 on Ubuntu Natty x86_64 virtual machine with the 2.6.38-8-virtual kernel.
Is the warning printed to stderr, and that is intercepted somehow?
Try using subprocess instead, for example using
subprocess.check_output(cmd, stderr=subprocess.STDOUT)
and checking the output.
(or plaing subprocess.call might work better than os.system)
So, it turned out the issue was something I failed to mention. Earlier in the script test.gnuplot and test.data were written, but I neglected to call the file objects' close() and verify that they got closed (still don't know how to do that last part so for now it cycles for a bit). So there was some unexpected behaviour going on there causing gnuplot to see two unreadable files, take no action, produce no output, and return 0.
I guess nobody gets points for this one.
Edit: I finally figured it out with the help of strace. Don't know how I did things before I learned how to use it.
don't use os.system. Use subprocess module.
os.system documentation says:
The subprocess module provides more powerful facilities for spawning
new processes and retrieving their results; using that module is
preferable to using this function.
Try this:
subprocess.check_call(['gnuplot', 'test.gnuplot'])