We Keep Coding

Returning status of grep command executed remotely via ssh embedded in expect command inside python

I have a below python script where in I am executing a remote SSH command using expect. Here, even the target file contains the string "error" or not, the exit code is returned as success always as the ssh connectivity is what only checked. How can I get the status of the grep command?
#! /usr/bin/python
import subprocess
def execute(cmd):
proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
_output, _error = proc.communicate()
_code = proc.returncode
return _output, _error, _code
host = "localhost"
passwd = "kube"
cmd="/usr/bin/expect -c 'spawn ssh "+host+" \"cat /home/kube/f1 | grep -qi error\"; expect \"password:\" { send \""+passwd+"\r\"} ;interact' "
_output, _error, return_code = execute(cmd)
print cmd + "\n" + _output + "\n" + _error
if (return_code == 0):
print "no error"
else:
print "contains error"

Option 1:
Let the remote command output something which indicates success/failure for you. E.g.:
ssh user#host "/some/command | grep -q some-string && echo MAGIC-SUCCESS || echo MAGIC-FAILURE"
And in Python you can get the output and parse it.
Option 2:
According to man expect:
wait [args]
[...] wait normally returns a list of four integers. The first integer is the pid of the process that was waited upon. The second integer is the corresponding spawn id. The third integer is -1 if an operating system error occurred, or 0 otherwise. If the third integer was 0, the fourth integer is the status returned by the spawned process. If the third integer was -1, the fourth integer is the value of errno set by the operating system. [...]
So your Expect code can check the wait result and then exit with different values and Python code can get the exit status.
For Expect part it's like this:
spawn ...
...
expect eof
set result [wait]
exit [lindex $result 3]

ssh not recognized as a command when executed from python using subprocess?

This is my code -
import subprocess
import sys
HOST="xyz3511.uhc.com"
# Ports are handled in ~/.ssh/config since we use OpenSSH
COMMAND="uptime"
ssh = subprocess.Popen(["ssh", "%s" % HOST, COMMAND],
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
result = ssh.stdout.readlines()
if result == []:
error = ssh.stderr.readlines()
print (sys.stderr, "ERROR: %s" % error)
else:
print (result)
and this is the error I'm getting-
ERROR:
[b"'ssh' is not recognized as an internal or external command,\r\n",
b'operable program or batch file.\r\n'].
Not sure what I'm doing wrong over here. Also, I haven't mentioned any port. All I want is to use subprocess and connect to remote server, execute a simple command like ls. Python version is 3.x.

Apparently this happens in python3.
Workaround found at this link:
https://gist.github.com/bortzmeyer/1284249
system32 = os.path.join(os.environ['SystemRoot'], 'SysNative' if platform.architecture()[0] == '32bit' else 'System32')
ssh_path = os.path.join(system32, 'OpenSSH/ssh.exe')
out1, err1 = Popen([ssh_path, "pi#%s"%self.host, "%s"%cmd],
shell=False,
stdout=PIPE,
stderr=PIPE).communicate()

subprocess.Popen is not able to execute tshark command in Linux in Python

I am trying to execute a tshark command to get some output for a validation and using subprocess.Popen to get this work done, but i am seeing sometimes subprocess.Popen is not able to execute the command. Below is a small function of my code:
import subprocess
import logging
def fetch_avps(request_name, logger, tcpdump, port, session_id):
out_list = []
if request_name == 'CCR':
com_sessn_filter = """tshark -r "%s" -odiameter.tcp.ports:"%s" -R 'diameter.cmd.code == 272 and diameter.flags.request==1 and !tcp.analysis.retransmission and diameter.flags.T == 0' -Tpdml -Tfields -ediameter.Session-Id -ediameter.CC-Request-Type -ediameter.User-Name -ediameter.Subscription-Id-Data -ediameter.Value-Digits | grep "%s" | cut -f 1-6 --output-delimiter=':'""" %(tcpdump, port, session_id)
elif request_name == 'CCA':
com_sessn_filter = """tshark -r "%s" -odiameter.tcp.ports:"%s" -R 'diameter.cmd.code == 272 and diameter.flags.request==0 and !tcp.analysis.retransmission and diameter.flags.T == 0' -Tpdml -Tfields -ediameter.Session-Id -ediameter.CC-Request-Type -ediameter.Result-Code -ediameter.Validity-Time -ediameter.Value-Digits -ediameter.Unit-Quota-Threshold | grep "%s" | cut -f 1-6 --output-delimiter=':'""" %(tcpdump, port, session_id)
p = subprocess.Popen(com_sessn_filter, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
out = p.stdout.read()
command_out_list = (out.strip().split("\n"))
sys.stdout.flush()
for i in range(len(command_out_list)):
out_list.append(command_out_list[i].split(":"))
if out_list[0][0] == '':
logger.error("Failed to execute Tshark command")
logger.debug("Failed to execute Tshark command \"%s\" for Session-Id \"%s\"" %(com_sessn_filter, session_id))
return 0
For example in the above code if I have 20 Sessions in a loop then subprocess.Popen might fail to execute around 12-13 times. Any help will be very useful.
Below is the stderr i am getting whenever it fails to execute.
(process:11306): GLib-ERROR **: /build/buildd/glib2.0-2.32.4/./glib/gmem.c:165: failed to allocate 4048572208 bytes Trace/breakpoint trap (core dumped)

How can I make my custom shell work with ssh?

I'm making a custom shell in Python for a very limited user on a server, who is logged in via ssh with a public key authentication. They need to be able to run ls, find -type d, and cat in specific directories with certain limitations. This works fine if you run something like ssh user#server -i keyfile, because you see the interactive prompt, and can run those commands. However, something like ssh user#server -i keyfile "ls /var/log" doesn't. ssh simply hangs, with no response. By using the -v switch I've found that the connection is succeeding, so the problem is in my shell. I'm also fairly certain that the script isn't even being started, since print sys.argv at the beginning of the program does nothing. Here's the code:
#!/usr/bin/env python
import subprocess
import re
import os
with open(os.devnull, 'w') as devnull:
proc = lambda x: subprocess.Popen(x, stdout=subprocess.PIPE, stderr=devnull)
while True:
try:
s = raw_input('> ')
except:
break
try:
cmd = re.split(r'\s+', s)
if len(cmd) != 2:
print 'Not permitted.'
continue
if cmd[0].lower() == 'l':
# Snip: verify directory
cmd = proc(['ls', cmd[1]])
print cmd.stdout.read()
elif cmd[0].lower() == 'r':
# Snip: verify directory
cmd = proc(['cat', cmd[1]])
print cmd.stdout.read()
elif cmd[0].lower() == 'll':
# Snip: verify directory
cmd = proc(['find', cmd[1], '-type', 'd'])
print cmd.stdout.read()
else:
print 'Not permitted.'
except OSError:
print 'Unknown error.'
And here's the relevant line from ~/.ssh/authorized_keys:
command="/path/to/shell $SSH_ORIGINAL_COMMAND" ssh-rsa [base-64-encoded-key] user#host
How can I make the shell script when the command is passed on the command line so it can be used in scripts without starting an interactive shell?

The problem with ssh not responding is related to the fact that ssh user#host cmd does not open a terminal for the command being run. Try calling ssh user#host -t cmd.
However, even if you pass the -t option, you'd still have another problem with your script: it only works interactively and totally ignores the $SSH_ORIGINAL_PROGRAM being passed. A naive solution would be to check sys.argv and if its bigger than 1 you don't loop forever, and instead only execute whatever command you have in it.

Python 2.6 - passing | char into os.system in for loop

I have a list of IP's that I want to run a whois (using the linux tool whois) against and only see the Country option.
Here is my script:
import os
import time
iplist = open('ips.txt').readlines()
for i in iplist:
time.sleep(2)
print "Country: IP {0}".format(i)
print os.system("whois -h whois.arin.net + {0} | grep Country ".format(i))
So I want to display what IP is being ran, then I just want to see the Country info using grep. I see this error when I run it and the grep is not ran:
sh: -c: line 1: syntax error near unexpected token `|'
sh: -c: line 1: ` | grep Country '
this code below works so it must be an issue with my for loop:
print os.system("whois -h whois.arin.net + {0} | grep Country ".format('8.8.8.8'))
What am I doing wrong? Thank you!!!!

You're not stripping trailing newlines from the lines you read from the file. As a result, you are passing to os.system a string like "whois -h whois.arin.net + a.b.c.d\n | grep Country". The shell parses the string as two commands and complains of "unexpected token |" at the beginning of the second one. This explains why there is no error when you use a hand-made string such as "8.8.8.8".
Add i = i.strip() after the sleep, and the problem will go away.

user4815162342 is correct about the issue you are having, but might I suggest you replace os.system with subprocess.Popen? Capturing the output from the system call is not intuitive.. should you want to result to go anywhere but your screen, you'll likely going to have issues
from subprocess import Popen, PIPE
server = 'whois.arin.net'
def find_country(ip):
proc = Popen(['whois', '-h', server, ip], stdout = PIPE, stderr = PIPE)
stdout, stderr = proc.communicate()
if stderr:
raise Exception("Error with `whois` subprocess: " + stderr)
for line in stdout.split('\n'):
if line.startswith('Country:'):
return line.split(':')[1].strip() # Good place for regex
for ip in [i.strip() for i in open('ips.txt').readlines()]:
print find_country(ip)
Python is awesome at string handling- there should be no reason to create a grep subprocess to pattern match the output of a separate subprocess.

Try sh:
import os
import time
import re
import sh
iplist = open('ips.txt').readlines()
for i in iplist:
time.sleep(2)
print "Country: IP {0}".format(i)
print sh.grep(sh.whois(i, h="whois.arin.net"), "Country")