We Keep Coding

Python's sh module - is it at all possible for a script to request input?

Using Python's sh, I am running 3rd party shell script that requests my input (not that it matters much, but to be precise, I'm running an Ansible2 playbook with the --step option)
As an oversimplification of what is happening, I built a simple bash script that requests an input. I believe that if make this simple example work I can make the original case work too.
So please consider this bash script hello.sh:
#!/bin/bash
echo "Please input your name and press Enter:"
read name
echo "Hello $name"
I can run it from python using sh module, but it fails to receive my input...
import errno
import sh
cmd = sh.Command('./hello.sh')
for line in cmd(_iter=True, _iter_noblock=True):
if line == errno.EWOULDBLOCK:
pass
else:
print(line)
How could I make this work?

After following this tutorial, this works for my use case:
#!/usr/bin/env python3
import errno
import sh
import sys
def sh_interact(char, stdin):
global aggregated
sys.stdout.write(char)
sys.stdout.flush()
aggregated += char
if aggregated.endswith(":"):
val = input()
stdin.put(val + "\n")
cmd = sh.Command('./hello.sh')
aggregated = ""
cmd(_out=sh_interact, _out_bufsize=0)
For example, the output is:
$ ./testinput.py
Please input your name and press Enter:arod
Hello arod

There are two ways to solve this:
Using _in:
using _in, we can pass a list which can be taken as input in the python script
cmd = sh.Command('./read.sh')
stdin = ['hello']
for line in cmd(_iter=True, _iter_noblock=True, _in=stdin):
if line == errno.EWOULDBLOCK:
pass
else:
print(line)
Using command line args if you are willing to modify the script.

Python If Statement Error, in bash_profile function

I am very new to python and am trying to use it to parse a file with JSON in it within my bash_profile script, and return the value. I can't seem to get the if statement working. It will read the contents of the file, but I recieve this error, with an arrow under the if statement.
File "<string>", line 1
import sys,json;data=json.loads(sys.stdin.read());if data['123456789']:print...
^
SyntaxError: invalid syntax
Contents of File:
{"123456789":"Well Hello"}
Function that searches file:
function find_it {
file=~/.pt_branches
CURRENT_CONTENTS=`cat $file`
echo $CURRENT_CONTENTS | python -c "import sys,json;data=json.loads(sys.stdin.read());if data['$1']:print data['$1'];else:print '';"
}
Command
find_it "123456789"
If I were to remove the if statement and just return the value, it does work, but I want it to be able to return and empty string if it does not exist.
echo $CURRENT_CONTENTS | python -c "import sys,json;data=json.loads(sys.stdin.read());print data['$1'];"
Any ideas on what I am doing wrong?!?

Code golf!
"import sys, json; data=json.loads(sys.stdin.read()); print data.get('$1') or ''"

Do this:
echo $CURRENT_CONTENTS | python -c "
import sys
import json
data = json.loads(sys.stdin.read())
if data['$1']:
print data['$1']
else:
print ''"

commands.getstatusoutput fails saying sh: Syntax error: ";" unexpected

I am running mongoimport command using python commands module as
status = utilities.execute(mongoimport)
in utilities.py
def execute(command):
if not command:
return (-1, 'command can not be empty or null')
return commands.getstatusoutput(command)
When I run this, I see error as
sh: Syntax error: ";" unexpected
I see that documentation says :
commands.getstatusoutput(cmd)
Execute the string cmd in a shell with os.popen() and return a 2-tuple (status, output). cmd is actually run as { cmd ; } 2>&1, so that the returned output will contain output or error messages
How can I fix this to be able to run this command?

Use the subprocess module
from subprocess import check_output
output = check_output(["ls", "-l"])
This will raise an error if the command fails - no need to check for empty string. If you are really sure that you want to pass stuff through the shell then call like this
output = check_output("ls -l", shell=True)
Just note that passing stuff through the shell is an excellent vector for security problems.

Malformed environment variables detection in python

I am trying to source a bash script containing some environment variables in python. I followed one other thread to do it. But, there seems that one of the variable is malformed, as can be seen in the given snippet.
COLORTERM=gnome-terminal
mc=() { . /usr/share/mc/mc-wrapper.sh
}
_=/usr/bin/env
I am using the following code to set up the current environment.
import os
import pprint
import subprocess
command = ['bash', '-c', 'source init_env && env']
proc = subprocess.Popen(command, stdout = subprocess.PIPE)
for line in proc.stdout:
(key, _, value) = line.partition("=")
os.environ[key] = value
proc.communicate()
'
If I change the above code a little like putting a condition:
for line in proc.stdout:
(key, _, value) = line.partition("=")
if not value:
continue
os.environ[key] = value
then things are working but the environment is corrupted because of one missing bracket as can be seen from the snippet of environment variable that the bracket is appearing on new line. Because of this corruption, If I run some other command like
os.system("ls -l")
it gives me the following error
sh: mc: line 1: syntax error: unexpected end of file
sh: error importing function definition for `mc'
What could be the possible solutions for this problem?
Thanks alot

Probably the best way to do this is to create a separate program that writes out the environment variables in a way that is easily and unambiguously processed by your own program; then call that program instead of env. Using the standard pickle module, that separate program can be as simple as this:
import os
import sys
import pickle
pickle.dump(os.environ, sys.stdout)
which you can either save into its own .py file, or else put directly in a Bash command:
python -c 'import os, sys, pickle; pickle.dump(os.environ, sys.stdout)'
In either case, you can process its output like this:
import os
import pprint
import subprocess
import pickle
command = [
'bash',
'-c',
'source init_env && ' +
'python -c "import os, sys, pickle; ' +
'pickle.dump(os.environ, sys.stdout)"'
]
proc = subprocess.Popen(command, stdout = subprocess.PIPE)
for k, v in pickle.load(proc.stdout).iteritems():
os.environ[k] = v
proc.communicate()

Python Script execute commands in Terminal [duplicate]

This question already has answers here:
Running Bash commands in Python
(11 answers)
Closed 9 months ago.
I read this somewhere a while ago but cant seem to find it. I am trying to find a command that will execute commands in the terminal and then output the result.
For example: the script will be:
command 'ls -l'
It will out the result of running that command in the terminal

There are several ways to do this:
A simple way is using the os module:
import os
os.system("ls -l")
More complex things can be achieved with the subprocess module:
for example:
import subprocess
test = subprocess.Popen(["ping","-W","2","-c", "1", "192.168.1.70"], stdout=subprocess.PIPE)
output = test.communicate()[0]

I prefer usage of subprocess module:
from subprocess import call
call(["ls", "-l"])
Reason is that if you want to pass some variable in the script this gives very easy way for example take the following part of the code
abc = a.c
call(["vim", abc])

import os
os.system("echo 'hello world'")
This should work. I do not know how to print the output into the python Shell.

Custom standard input for python subprocess
In fact any question on subprocess will be a good read
https://stackoverflow.com/questions/tagged/subprocess

for python3 use subprocess
import subprocess
s = subprocess.getstatusoutput(f'ps -ef | grep python3')
print(s)
You can also check for errors:
import subprocess
s = subprocess.getstatusoutput('ls')
if s[0] == 0:
print(s[1])
else:
print('Custom Error {}'.format(s[1]))
# >>> Applications
# >>> Desktop
# >>> Documents
# >>> Downloads
# >>> Library
# >>> Movies
# >>> Music
# >>> Pictures
import subprocess
s = subprocess.getstatusoutput('lr')
if s[0] == 0:
print(s[1])
else:
print('Custom Error: {}'.format(s[1]))
# >>> Custom Error: /bin/sh: lr: command not found

You should also look into commands.getstatusoutput
This returns a tuple of length 2..
The first is the return integer (0 - when the commands is successful)
second is the whole output as will be shown in the terminal.
For ls
import commands
s = commands.getstatusoutput('ls')
print s
>> (0, 'file_1\nfile_2\nfile_3')
s[1].split("\n")
>> ['file_1', 'file_2', 'file_3']

In python3 the standard way is to use subprocess.run
res = subprocess.run(['ls', '-l'], capture_output=True)
print(res.stdout)

The os.popen() is pretty simply to use, but it has been deprecated since Python 2.6.
You should use the subprocess module instead.
Read here: reading a os.popen(command) into a string

Jupyter
In a jupyter notebook you can use the magic function !
!echo "execute a command"
files = !ls -a /data/dir/ #get the output into a variable
ipython
To execute this as a .py script you would need to use ipython
files = get_ipython().getoutput('ls -a /data/dir/')
execute script
$ ipython my_script.py

You could import the 'os' module and use it like this :
import os
os.system('#DesiredAction')

Running: subprocess.run
Output: subprocess.PIPE
Error: raise RuntimeError
#! /usr/bin/env python3
import subprocess
def runCommand (command):
output=subprocess.run(
command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
if output.returncode != 0:
raise RuntimeError(
output.stderr.decode("utf-8"))
return output
output = runCommand ([command, arguments])
print (output.stdout.decode("utf-8"))