We Keep Coding

Attempt to generate email with new lines from Python through Bash mailx command

Where I work, we have servers that are pre-configured for the use of the bash mail command to send attachments and messages. I'm working on a notification script that will monitor server activity and generate an email if it detects an issue. I'm using the subprocess.call function in order to send a bash command.
I am successful in sending messages, but in the body portion of the email, it is stringing each notification line together rather than putting each notification on a separate line. I have tried to append each line within the string with "\n" and "\r\n". I have to use double backslashes as python will interpret this as literal new lines when it sends the echo command. I also passed the command "shopt -s xpg_echo" before using the echo with pipe to mail using the double backspaces but this also had no effect. I also tried using echo without the "-e" option and this had no effect either.
The trick is that I need python to send the new line to bash and then somehow get bash to interpret this as a new line using echo piped through to mail. Here is a sample of the code:
import os
import shutil
import sys
import time
import re
import subprocess
import smtplib
serviceports["SCP Test"] = ["22"]
serviceports["Webtier"] = ["9282"]
bashCommand = "netstat -an | grep LISTEN | grep -v LISTENING"
netstat_results = subprocess.check_output(bashCommand, shell=True)
netstat_results = str(netstat_results)
#Iterate through all ports for each service and assign down ports to variable
for servicename, ports in serviceports.items():
for ind_port in ports:
ind_port_chk = ":" + ind_port
count = sum(1 for _ in re.finditer(r'\b%s\b' % re.escape(ind_port_chk), netstat_results))
if count == 0:
warning = servicename + " on port " + ind_port + " is currently down!"
report.append(warning)
for warning in report:
message = message + warning + "\\\n"
fromaddr=serveridsimp + "#xxxxx.com"
toaddr='email#xxxxx.com'
subject="Testing..."
body=message
cmd= cmd='echo -e '+body+' | mail -s '+subject+' -r '+fromaddr+' '+toaddr
send=subprocess.call(cmd,shell=True)
The code runs a netstat command and assigns it to a string. The code will then iterate through the specified ports and search for where that port doesn't exist in the netstat string (netstat_results). It then will create a list object (warning) containing all the ports not located in netstat_results and then append each line adding \n to a string called "message". It then sends an echo piped to the xmail command to generate an email to be sent containing all the ports not found. What happens currently is that I will get an email saying something like this:
SCP Test on port 22 is currently down!nOHS Webtier on port 9282 is currently down!n etc...
I want it to put each message on a new line like so:
SCP Test on port 22 is currently down!
Webtier on port 9282 is currently down!
I am trying to avoid writing the output to a file and then using bash to read it back into the mail command. Is this possible without having to create a file?

I was finally able to fix the issue by changing the command sent to bash and character being appended to the following:
message = message + warning + "\n"
cmd= cmd='echo -e '+'"'+body+'"'+'|awk \'{ print $0" " }\''+' | mail -s '+'"'+subject+'"'+' -r '+fromaddr+' '+toaddr

Parse Linux Command with Python

Here's a quick snippet of my code using pexpect:
child.expect('tc#')
child.sendline('ps -o args | grep lp_ | grep -v grep | sort -n')
child.expect('tc#')
print(child.before)
child.sendline('exit')
and then the output:
user#myhost:~/Python$ python tctest.py
tc-hostname:~$ ps -o args | grep lp_ | grep -v grep | sort -n
/usr/local/bin/lp_server -n 5964 -d /dev/usb/lp1
/usr/local/bin/lp_server -n 5965 -d /dev/usb/lp0
{lp_supervisor} /bin/sh /usr/local/lp/lp_supervisor /dev/usb/lp0 SERIAL#1 /var/run/lp/lp_pid/usb_lp0
{lp_supervisor} /bin/sh /usr/local/lp/lp_supervisor /dev/usb/lp1 SERIAL#2 /var/run/lp/lp_pid/usb_lp1
user#myhost:~$
There's 4 lines of output. The first two lines show with printer port the usb device is assigned to (EX: first line shows port 5964 is assigned to lp1)
The 3rd and 4th lines show which device serial number is assigned to which usb port. (EX: SERIAL#1 is assigned to lp0)
I need to somehow parse that output so I can do the following:
If SERIAL#1 is not assigned to 5964:
run some command
else:
do something else
If SERIAL#2 is not assigned to 5965:
run some command
else:
do something else
I'm not sure how to manipulate that output so I can get the desired variables. Any help is appreciated.

You can extract port and serial information from pexpect data using re.findall and do something like this
import re
data = child.before
ports = re.findall(r'lp_server -n (\d+)', data)
# ['5964', '5965']
serials = re.findall(r'(SERIAL#\d+)', data)
# ['SERIAL#1', 'SERIAL#2']
list(zip(ports, serials))
# [('5964', 'SERIAL#1'), ('5965', 'SERIAL#2')]
for serial, port in zip(ports, serials):
# Check if serial and port matches expectation

Another way of doing it is by using dictionaries to build relationships between device serial numbers and printer ports:
inString = """/usr/local/bin/lp_server -n 5964 -d /dev/usb/lp1
/usr/local/bin/lp_server -n 5965 -d /dev/usb/lp0
{lp_supervisor} /bin/sh /usr/local/lp/lp_supervisor /dev/usb/lp0 SERIAL#1 /var/run/lp/lp_pid/usb_lp0
{lp_supervisor} /bin/sh /usr/local/lp/lp_supervisor /dev/usb/lp1 SERIAL#2 /var/run/lp/lp_pid/usb_lp1"""
inString = inString.split("\n")
matches = dict()
serials = dict()
for i in range(len(inString[:2])):
lp = inString[i][-3:]
printerPort = int(inString[i].split("-n ")[1][:4])
matches.update({lp:printerPort})
for i in range(2,len(inString)):
t = inString[i].split(" ")
lp = t[3][-3:]
serial = t[4]
serials.update({serial:lp})
finalLookup = dict((k,matches[v]) for k,v in serials.items())
print(finalLookup)
Output:
{'SERIAL#1': 5965, 'SERIAL#2': 5964}
Then you can do:
if not finalLookup['SERIAL#1'] == 5964:
run some command
else:
do something else
if not finalLookup['SERIAL#2'] == 5965:
run some command
else:
do something else

How to get "Client name:" of a perforce workspace using "p4 info" from python script?

I have a scenario where i need to have perforce workspace Client name information. Trying out with script as follows:
import sys
import subprocess
cmd = "p4 info | grep" + """ "Client name" """ +""
print ("p4 command is: ",cmd)
p4 = subprocess.call(cmd)
print( p4)
Where i get out put as :
p4 command is: p4 info | grep "Client name"
Usage: info [-s]
Unexpected arguments.
1
I tried doing as follows too but did not worked:
import sys
import subprocess
p4_info = subprocess.call([sys.executable, "p4 info | grep "Client name""])
I actually want to fetch only Client name information. So, suggest me how i may have that. If there is another way of doing it please suggest. The script will be executed within the workspace and shall return the only Client name:

Do:
p4 -Ztag -F %clientName% info

Bash command only executes the first time around

I have the following nmap command:
nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5
This produces a list of ip addresses which I'm trying to pass to the following python script:
#!/usr/bin/python
# Python tool to check a range of hosts for SMTP servers that respond to VRFY requests
import socket
import sys
from socket import error as socket_error
# Read the username file
with open(sys.argv[1]) as f:
usernames = f.read().splitlines()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host_ip = sys.argv[2]
print("****************************")
print("Results for: " + host_ip)
try:
c = s.connect((host_ip,25))
banner=s.recv(1024)
#Send VRFY requests and print result
for user in usernames:
s.send('VRFY ' + user + '\r\n')
result = s.recv(1024)
print(result)
print("****************************")
#Close Socket
s.close()
#If error is thrown
except socket_error as serr:
print("\nNo SMTP verify for " +host_ip)
print("****************************")
I've tried to do this with the following command, however it's only running the script over the first ip that it finds:
./smtp_verify.py users.txt $(nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5)
I've also tried to do this with:
for $ip in (nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5); do ./smtp_verify.py users.txt $ip done
However I receive a syntax error for it which suggests to me I can't pass pipes this way?
bash: syntax error near unexpected token `('

Do not consciously use for loop for parsing command output, see DontReadLinesWithFor, rather use a Process-Subtitution syntax with a while loop
#!/bin/bash
while IFS= read -r line; do
./smtp_verify.py users.txt "$line"
done< <(nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5)
And for the error you are likely seeing, you are NOT using command-substitution $(..) syntax properly to run the piped commands, the commands should have been enclosed around () with a $ before it. Something like,
#!/bin/bash
for ip in $(nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5); do
./smtp_verify.py users.txt "$ip"
done
And remember to always double-quote shell variables to avoid Word Splitting done by the shell.

Python 2.6 - passing | char into os.system in for loop

I have a list of IP's that I want to run a whois (using the linux tool whois) against and only see the Country option.
Here is my script:
import os
import time
iplist = open('ips.txt').readlines()
for i in iplist:
time.sleep(2)
print "Country: IP {0}".format(i)
print os.system("whois -h whois.arin.net + {0} | grep Country ".format(i))
So I want to display what IP is being ran, then I just want to see the Country info using grep. I see this error when I run it and the grep is not ran:
sh: -c: line 1: syntax error near unexpected token `|'
sh: -c: line 1: ` | grep Country '
this code below works so it must be an issue with my for loop:
print os.system("whois -h whois.arin.net + {0} | grep Country ".format('8.8.8.8'))
What am I doing wrong? Thank you!!!!

You're not stripping trailing newlines from the lines you read from the file. As a result, you are passing to os.system a string like "whois -h whois.arin.net + a.b.c.d\n | grep Country". The shell parses the string as two commands and complains of "unexpected token |" at the beginning of the second one. This explains why there is no error when you use a hand-made string such as "8.8.8.8".
Add i = i.strip() after the sleep, and the problem will go away.

user4815162342 is correct about the issue you are having, but might I suggest you replace os.system with subprocess.Popen? Capturing the output from the system call is not intuitive.. should you want to result to go anywhere but your screen, you'll likely going to have issues
from subprocess import Popen, PIPE
server = 'whois.arin.net'
def find_country(ip):
proc = Popen(['whois', '-h', server, ip], stdout = PIPE, stderr = PIPE)
stdout, stderr = proc.communicate()
if stderr:
raise Exception("Error with `whois` subprocess: " + stderr)
for line in stdout.split('\n'):
if line.startswith('Country:'):
return line.split(':')[1].strip() # Good place for regex
for ip in [i.strip() for i in open('ips.txt').readlines()]:
print find_country(ip)
Python is awesome at string handling- there should be no reason to create a grep subprocess to pattern match the output of a separate subprocess.

Try sh:
import os
import time
import re
import sh
iplist = open('ips.txt').readlines()
for i in iplist:
time.sleep(2)
print "Country: IP {0}".format(i)
print sh.grep(sh.whois(i, h="whois.arin.net"), "Country")