We Keep Coding

Print bash history using python

I have to print bash history using subprocess package.
import subprocess
co = subprocess.Popen(['history'], stdout = subprocess.PIPE)
History = co.stdout.read()
print("----------History----------" + "\n" + History)
but they prompt an error
Traceback (most recent call last):
File "test.py", line 4, in <module>
co = subprocess.Popen(['history'], stdout = subprocess.PIPE)
File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Normally, you would need to add shell=True argument to your Popen call:
co = subprocess.Popen(['history'], shell=True, stdout = subprocess.PIPE)
Or to manually specify the shell you want to call.
co = subprocess.Popen(['/bin/bash', '-c', 'history'], stdout = subprocess.PIPE)
Unfortunately, in this particular case it won't help, because bash has empty history when used non-interactively.
A working solution would be to read ${HOME}/.bash_history manually.

Kit is correct, reading ~/.bash_history may be a better option:
from os.path import join, expanduser
with open(join(expanduser('~'), '.bash_history'), 'r') as f:
for line in f:
print(line)

subprocess error on windows while listing an s3 bucket

I am trying to list the contents of an s3 bucket using Python 2.7.13. This does not work:
>>> args
['aws', 's3', 'ls', 's3://mybucket']
>>> p = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Error is:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\subprocess.py", line 711, in __init__
errread, errwrite)
File "C:\Python27\lib\subprocess.py", line 959, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
Why is this so ?
But this works:
>>> p = subprocess.Popen(args, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Why is shell=False failing but shell=True working ?

You are totally right to try to use subprocess with shell=False. It is the best way to ensure portability, and is probably faster to launch.
In your case, your arguments look okay (no redirection, no pipe, multiple command):
['aws', 's3', 'ls', 's3://mybucket']
so the only thing that prevents it to work with shell=False is the fact that aws is not really an executable, but rather a file whose extension (not shown here) is contained in PATHEXT
On my machine PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.PY
Means that any aws.js, aws.bat, ... file can be executed. But you need the shell for that.
To locate your program, type where aws in a command prompt, you'll get the full path & extension of the command.
If you don't want to use shell=True, there's an alternative which amounts to the same thing
args = ['cmd','/c','aws', 's3', 'ls', 's3://mybucket']
Since you're already running a cmd you don't need shell=True

Reading and writing from and to different files in Python

I am trying to read the commands from "command.txt" file and want to redirect the output of this commands to "output.txt", contents of command.txt
ps -a
free
So far I came up with this code which for certain reason is not good and fails to execute.
import os
import sys
import subprocess
with open('output.txt', 'w') as out_file, open('command.txt', 'r') as in_file:
for line in in_file:
output = subprocess.Popen(line, stdout=subprocess.PIPE)
print output
out_file.write(output)
I am getting the below error:
Error:
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7/Users/PythonTutorials/subprocess1.py
Traceback (most recent call last):
File "/Users/shandeepkm/PythonTutorials/subprocess1.py", line 9, in <module>
output = subprocess.Popen(line, stdout=subprocess.PIPE)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 709, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1326, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Process finished with exit code 1
Could anyone please suggest appropriate python code for this task.

I see two errors.
First, you have "command.txt" as the first line in your file. That definitely won't execute as a subprocess.
Also, your line
out_file.write(output)
needs to be tabbed under the for loop.
Now for the updated question:
The Popen constructor needs to take an array for the args. So instead of
'ps -a'
you need to pass
['ps', '-a']
Also, what gets returned from Popen isn't text. So altogether you need:
args = shlex.split(line)
output = subprocess.Popen(args, stdout=subprocess.PIPE).stdout.read()

Solution 1: Execute line by line
You can redirect to a file using the stdout parameter of Popen:
import subprocess
import shlex
with open('output.txt', 'wb') as outfile, open('command.txt') as infile:
for line in infile:
command = shlex.split(line)
if not command:
continue # Skip blank lines
try:
process = subprocess.Popen(command, stdout=outfile)
process.wait()
except OSError:
outfile.write('COMMAND ERROR: {}'.format(line))
In the code above, you redirect the output by pointing stdout to the output file's handle, no printing is needed. The code also guard against bad commands
Solution 2: Call the shell
If you are running under Linux or Mac, the following solution is simpler: by calling bash to execute the whole command.txt file and record the stdout and stderr. This should work under windows with cmd -c, but I don't have a Windows machine to try.
import subprocess
with open('output.txt', 'wb') as outfile:
command = ['bash', 'command.txt']
process = subprocess.Popen(command, stdout=outfile, stderr=outfile)
process.wait()

error while running Popen

I am running into following error when trying to run a command using Popen,what is wrong here?
cmd = "export COMMANDER_SERVER=commander.company.com"
Pipe = Popen(cmd, stdout=PIPE, stderr=PIPE)
(output, error) = Pipe.communicate()
Error:-
Traceback (most recent call last):
File "test_ectool.py", line 26, in <module>
main()
File "test_ectool.py", line 13, in main
Pipe = Popen(cmd, stdout=PIPE, stderr=PIPE)
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

You need to separate the arguments from the command and give a list to Popen.
As Kenster's comment said, even if your command worked, you would only succeed in modifying an environmental variable inside a sub-shell not the main shell.
You will not be able run run export this way, because it is not a program. It is a bash built-in.
Here is an example of a command that does work, with the correct semantics.
from subprocess import Popen,PIPE
cmd = "echo COMMANDER_SERVER=commander.company.com"
Pipe = Popen(cmd.split(), stdout=PIPE, stderr=PIPE)
(output, error) = Pipe.communicate()
print output

merlin2011's answer is incorrect regarding the command string for Popen (point #1). From the Python docs:
args should be a sequence of program arguments or else a single string.
As other people have stated, the environment variable will not be saved. For that, you need to use os.environ['VARNAME'] = value. If you want to use other bash builtins, then you must pass shell=True as an argument to Popen.

checking the return code using python (MAC)

I have written a script that checks if an SVN Repo is up and running, the result is based on the return value.
import subprocess
url = " validurl"
def check_svn_status():
subprocess.call(['svn info'+url],shell=True)
def get_status():
subprocess.call('echo $?',shell=True)
def main():
check_svn_status()
get_status()
if __name__ == '__main__':
main()
The problem I'm facing is that if I change the url to something that does't exist I still get the return value as 0, but if I were to run this outside the script, i.e go to the terminal type svn info wrong url and then do a echo $? I get a return value of 1. But I can't re-create this in the python. Any guidelines ?
TraceBack after updating
Traceback (most recent call last):
File "svn_status.py", line 21, in <module>
main()
File "svn_status.py", line 15, in main
check_svn_status()
File "svn_status.py", line 8, in check_svn_status
p = sp.Popen(['svn info'], stdout=sp.PIPE, stderr=sp.PIPE)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 672, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1202, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or director
y

Why your approach does not work:
You invoke two independent subshells. The second shell does not know of the first shell and therefore does not have any information about the returncode of the process that was executed in the first shell.
Solution:
Use the subprocess module, spawn your subprocess directly (not through a subshell) and retrieve the returncode. Help yourself by reading the documentation of the module: http://docs.python.org/library/subprocess.html
There are several ways to achieve your goal. One simple way could be:
import subprocess as sp
p = sp.Popen(['command', 'arg1', 'arg2'], stdout=sp.PIPE, stderr=sp.PIPE)
stdout, stderr = p.communicate()
returncode = p.returncode
This way, you don't go through a subshell (shell=False by default), which is the recommended approach for various reasons. You directly catch the returncode of the spawned subprocess and you have full access to the subprocess' standard output and standard error.

subprocess.call returns the retcode, just store the result of your subprocess.call(['svn info'+url],shell=True)
http://docs.python.org/library/subprocess.html