We Keep Coding

How to format plain Null ('\0') character for query string in python

For an macOS system uninstall command, need to insert and format the command with plain null ('\0') character.
However, python formatter convert that null character string into it's own hex format '\x00' which is in-correct for the requirement
How do I format following uninstall command in python
Command = 'pkgutil --only-dirs --files <your-package-id> | tail -r | tr '\n' '\0' | xargs -n 1 -0 sudo rmdir'
Tried following method but not working as expected
PACKAGE_ID = 'com.abcd.bdr'
TR_COMMAND = 'tr \'\n\' \'\0\''
command = u'pkgutil --only-dirs --files {} | tail -r | {} | xargs -n 1 -0 sudo rm -rif'.format(PACKAGE_ID, TR_COMMAND)
prog = Popen(command, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)
_outs, _errs = prog.communicate(b'')
Python converts the Null character '\0' string to '\x00' which is not correct for me
Expected result was :
Command = 'pkgutil --only-dirs --files <your-package-id> | tail -r | tr '\n' **'\0'** | xargs -n 1 -0 sudo rmdir'
But python complier gives :
Command = 'pkgutil --only-dirs --files <your-package-id> | tail -r | tr '\n' '**\x00'** | xargs -n 1 -0 sudo rmdir'

How to execute python subprocess.Popen with many Arguments?

I need to execute the same command on a local and remote server. So I'm using subprocess.Popen to execute, and local command work as expected, but when I execute on remote it gives me some error like command not found. I appreciate your support as I am new to this.
Local execution function
def topic_Offset_lz(self):
CMD = "/dsapps/admin/edp/scripts/edp-admin.sh kafka-topic offset %s -e %s | grep -v Getting |grep -v Verifying | egrep -v '^[[:space:]]*$|^#' | awk -F\: '{print $3}'|sed '%sq;d'" % (self.topic,self.envr,self.partition)
t_out_lz, t_error_lz = subprocess.Popen(CMD, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
return t_out_lz
Remote server execution
def topic_offset_sl(self):
CMD = "/dsapps/admin/edp/scripts/edp-admin.sh kafka-topic offset %s -e %s | grep -v Getting |grep -v Verifying | egrep -v '^[[:space:]]*$|^#' | awk -F\: '{print $3}'|sed '%sq;d'" % (self.topic, self.envr, self.partition)
t_out_sl, t_error_sl = subprocess.Popen(["ssh", "-q", CMD], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
return t_error_sl
Error I'm getting for the remote execution
Landing Zone Offset: 0
SoftLayer Zone Offset: /bin/sh: ^# |sed 1: command not found
/bin/sh: d: command not found

I came up with below solution, might be there will easy way rather than this.
def topic_offset_sl(self):
CMD_SL1 = "ssh -q %s '/dsapps/admin/edp/scripts/edp-admin.sh kafka-topic offset %s -e %s'" % (KEY_SERVER,self.topic, self.envr)
CMD_SL2 = "| grep -v Getting |grep -v Verifying | egrep -v '^[[:space:]]*$|^#' | awk -F\: '{print $3}'|sed '%sq;d'" % (self.partition)
t_out_sl, t_error_sl = subprocess.Popen(CMD_SL1 + CMD_SL2 , stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
return t_out_sl

The ssh command passes its argument vector as a single command line string, not an array. To do this, it simply concatenates the arguments, without performing shell quoting:
$ ssh target "python -c 'import sys;print(sys.argv)'" 1 2 3
['-c', '1', '2', '3']
$ ssh target "python -c 'import sys;print(sys.argv)'" "1 2 3"
['-c', '1', '2', '3']
If there was proper shell quoting, the distinction between 1 2 3 and "1 2 3" would have been preserved, and the first argument would not need double-quoting.
Anyway, in your case, the following might work:
def topic_offset_sl(self):
CMD = "ssh -q " + pipes.quote("/dsapps/admin/edp/scripts/edp-admin.sh"
+ " kafka-topic offset %s -e %s" % (self.topic, self.envr)) \
+ "grep -v Getting |grep -v Verifying | egrep -v '^[[:space:]]*$|^#'"
+ " | awk -F\: '{print $3}'|sed '%sq;d'" % self.partition
t_out_sl, t_error_sl = subprocess.Popen(CMD], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
return t_error_sl
This assumes you only want to run the /dsapps/admin/edp/scripts/edp-admin.sh script remotely and not the rest.
Note that the way you use string splicing to construct command lines likely introduces shell command injection vulnerabilities (both locally and on the remote server).

Bash command only executes the first time around

I have the following nmap command:
nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5
This produces a list of ip addresses which I'm trying to pass to the following python script:
#!/usr/bin/python
# Python tool to check a range of hosts for SMTP servers that respond to VRFY requests
import socket
import sys
from socket import error as socket_error
# Read the username file
with open(sys.argv[1]) as f:
usernames = f.read().splitlines()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host_ip = sys.argv[2]
print("****************************")
print("Results for: " + host_ip)
try:
c = s.connect((host_ip,25))
banner=s.recv(1024)
#Send VRFY requests and print result
for user in usernames:
s.send('VRFY ' + user + '\r\n')
result = s.recv(1024)
print(result)
print("****************************")
#Close Socket
s.close()
#If error is thrown
except socket_error as serr:
print("\nNo SMTP verify for " +host_ip)
print("****************************")
I've tried to do this with the following command, however it's only running the script over the first ip that it finds:
./smtp_verify.py users.txt $(nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5)
I've also tried to do this with:
for $ip in (nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5); do ./smtp_verify.py users.txt $ip done
However I receive a syntax error for it which suggests to me I can't pass pipes this way?
bash: syntax error near unexpected token `('

Do not consciously use for loop for parsing command output, see DontReadLinesWithFor, rather use a Process-Subtitution syntax with a while loop
#!/bin/bash
while IFS= read -r line; do
./smtp_verify.py users.txt "$line"
done< <(nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5)
And for the error you are likely seeing, you are NOT using command-substitution $(..) syntax properly to run the piped commands, the commands should have been enclosed around () with a $ before it. Something like,
#!/bin/bash
for ip in $(nmap -n -p 25 10.11.1.1-254 --open | grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\.[0-9]\{1,3\}' | cut -d" " -f5); do
./smtp_verify.py users.txt "$ip"
done
And remember to always double-quote shell variables to avoid Word Splitting done by the shell.

Execute a bash command with parameter in Python

This is a bash command that I run in python and get the expected result:
count = subprocess.Popen("ps -ef | grep app | wc -l", stdout=subprocess.PIPE, shell=True)
but when I'd like to pass an argument (count in this case) cannot figure out how to do it.
I tried:
pid = subprocess.call("ps -ef | grep app | awk -v n=' + str(count), 'NR==n | awk \'{print $2}\'", shell=True)
and
args = shlex.split('ps -ef | grep app | awk -v n=' + str(count), 'NR==n | awk \'{print $2}\'')
pid = subprocess.Popen(args, stdout=subprocess.PIPE, shell=True)
among other attempts, from various posts here, but still cannot make it.

You're mixing opening and closing quotations and you pass a colon by mistake on your other attempts among other things.
Try this for a fix:
pid = subprocess.call("ps -ef | grep app | awk -v n=" + str(count) + " NR==n | awk '{print $2}'", shell=True)
You opened the command parameter with " and there for you need to close it before you do + str() with a " and not a '. Further more i swapped the , 'NR= with + "NR= since you want to append more to your command and not pass a argument to subprocess.call().
As pointed out in the comments, there's no point in splitting the command with shlex since piping commands isn't implemented in subprocess, I would however like to point out that using shell=True is usually not recommended because for instance one of the examples given here.

An other vay is using format:
pid = subprocess.call("ps -ef | grep app | awk -v n={} NR==n | awk '{{print $2}}'".format(str(count)), shell=True)

Your Awk pipeline could be simplified a great deal - if the goal is to print the last match, ps -ef | awk '/app/ { p=$2 } END { print p }' does that. But many times, running Awk from Python is just silly, and performing the filtering in Python is convenient and easy, as well as obviously more efficient (you save not only the Awk process, but also the pesky shell=True).
for p in subprocess.check_output(['ps', '-ef']).split('\n'):
if 'app' in p:
pid = p.split()[1]

Handle result of os.system

I'm using python to script a functional script and I can't handler the result of this command line:
os.system("ps aux -u %s | grep %s | grep -v 'grep' | awk '{print $2}'" % (username, process_name)
It shows me pids but I can't use it as List.
If I test:
pids = os.system("ps aux -u %s | grep %s | grep -v 'grep' | awk '{print $2}'" % (username, process_name)
print type(pids)
#Results
29719
30205
31037
31612
<type 'int'>
Why is pids an int? How can I handle this result as List?
Stranger part:
print type(os.system("ps aux -u %s | grep %s | grep -v 'grep' | awk '{print $2}'" % (username, process_name))
There is nothing. Not any type written on my console..

os.system does not capture the output of the command it runs. To do so you need to use subprocess.
from subprocess import check_output
out = check_output("your command goes here", shell=true)
The above will work in Python 2.7. For older Pythons, use:
import subprocess
p = subprocess.Popen("your command goes here", stdout=subprocess.PIPE, shell=True)
out, err = p.communicate()

os module documentation
os.system(command)
Execute the command (a string) in a subshell. This is implemented by calling the Standard C function system(), and has the same limitations. Changes to sys.stdin, etc. are not reflected in the environment of the executed command.
On Unix, the return value is the exit status of the process encoded in the format specified for wait(). Note that POSIX does not specify the meaning of the return value of the C system() function, so the return value of the Python function is system-dependent.
If you want access to the output of the command, use the subprocess module instead, e.g. check_output:
subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)
Run command with arguments and return its output as a byte string.