We Keep Coding

How to suppress stdout & stderr until success in python? [duplicate]

This question already has answers here:
Redirect subprocess stderr to stdout
(2 answers)
Closed 2 years ago.
I want to repeat block of codes until successful output but want to display only successful messages.
while i < 6:
try:
sys.tracebacklimit = 0 #this line seems not work
gluster_volume_names = []
gstatus_output = subprocess.check_output('gstatus -a -o json ', shell=True).decode()
date, time, json_part = gstatus_output.split(maxsplit=2)
gluster_info = json.loads(json_part)
volume_list = gluster_info["volume_summary"]
....
....
break
except:
i += 1
continue
But I don't know how to suppress these output below. (unsuccessful run) They are not the outcome I want. The block of code eventually ran successfully after less than 5 tries then exit.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gstatus-0.66-py3.6.egg/EGG-INFO/scripts/gstatus", line 143, in main
File "/usr/local/lib/python3.6/site-packages/gstatus-0.66-py3.6.egg/gstatus/libgluster/cluster.py", line 543, in update_state
gstatus.libutils.excepts.GlusterFailedVolume: Unable to query volume 'BLAH'
Possible cause: cluster is currently reconverging after a nodehas entered a disconnected state.
Response: Rerun gstatus or issue a peer status command to confirm
Please help!

Instead of using subprocess.check_output, you should use the standard subprocess.run method and pipe the standard error to /dev/null. Use the following instead:
gstatus_output = subprocess.run('gstatus -a -o json ', shell=True, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL).stdout.decode()

If you just want to suppress the errors in the console on a linux-system you can try:
python yourCode.py 2>/dev/null
With this you can also suppress stdout:
python yourCode.py 1>/dev/null

One possibility is to redirect standard output and standard error to a string. After execution, you can choose whether to print the results of the string, or discard it and try again.
from contextlib import redirect_stdout, redirect_stderr
import io
f = io.StringIO()
with redirect_stdout(f):
with redirect_stderr(f):
.... whatever you want ....
s = f.getvalue()

I am trying to write python script which ping n number of server and store result in text file same as ex ping 8.8.8.8 >>c:\result.txt

What I tried as below which giving me error message.
What I want:
I actually want to ping N number of servers and traceroute it and result should be saved in a text file. No matter if it store in any other format too but it should be easy to understand and read.
Issue:
Error related to Popen is not getting resolved however if you aware of any other method, please welcome to that too. Please help. Thanks in advance.
Note: I am using Windows 10
import subprocess
with open('ip-source.txt') as file:
IP = file.read()
IP = IP.splitlines()
for ip in IP:
with open('output.txt','ab') as out:
out.write(subprocess.Popen("ping " + ip))
===================== RESTART: F:/PingandTracert/Ping.py =====================
Traceback (most recent call last):
File "F:/PingandTracert/Ping.py", line 12, in <module>
out.write(subprocess.Popen("ping " + ip))
TypeError: a bytes-like object is required, not 'Popen'

The below is what you could use to run a ping test, I am using the subprocess module.
You can write the result to a csv file or a text file, depends on how you would like to have it.
import subprocess
def pingTest(host):
process = subprocess.Popen(["ping", "-n", "1",host], stdout=subprocess.PIPE,stderr=subprocess.PIPE)
streamdata = process.communicate()[0]
if not 'Reply from {}'.format(host) in str(streamdata):
return "Ping Failed"
else:
return "Ping Successful"
print(pingTest("ip"))

call IPtables command within python script

I am trying to call the following command in my python script. I am trying to insert the rule into IP tables. I am using sub process to call it and inserting variables into it where needed, but am getting a large error. Any suggestions?
iptables = subprocess.call('iptables -I FORWARD -eth 0 -m '+protocol+' -t'+protocol+'--dport '+port+'-j DNAT --to-destination'+ipAddress)
Error:
Traceback (most recent call last):
File "./port_forward.py", line 42, in <module>
iptables = subprocess.call('iptables -I FORWARD -i eth0 -m '+protocol+' -t'+protocol+'--dport '+port+'-j DNAT --to-destination'+ipAddress)
File "/usr/lib/python2.7/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Your problem is very common for Python beginners. Instead of formatting the string command,
you are trying to build a complex string by concatenating many strings and variables. Instead, use a
a string format, it will allow you to test your command and make your code more readable and flexible.
Your original string lacks spaces between the options and the arguments, e.g. --to-destination1.2.3.4.
Hence, you should format your string (This works also for python 2.7):
opts = {'iptables': '/sbin/iptables', 'protocol': 'tcp', 'port': 80, 'ipAddress': '0.0.0.0'}
ipcmd = '{iptables} -I FORWARD -eth 0 -m {protocol} -t {protocol} \
--dport {port} -j DNAT --to-destination {ipAddress}'.format(**opts)
if DEBUG:
print ipcmd
iptables = subprocess.call(ipcmd)
This is much easier to modify later, and also, when you do more Python programming, you will see that it is more readable.
Also, to call IPTables, you should be root, as stated in the comments:
In the beginning of your script add:
import sys
import os
if not os.getuid() == 0:
print "You must be root to change IPTables."
sys.exit(2)
update after seeing your error trace:
You are trying to call a command iptables but it is not in your path.
You should call the full path of iptables , e.g. /sbin/iptables

I wrote a simple firewall the same way and realized, "why not just write it in bash?". Anyway I discovered the python-iptables library and rewrote my code using that. I recommend checking it out. I think it will give you a more robust and structured way of writing iptables rules.

Your command is full of syntax errors due to missing spaces, as shown below:
iptables = subprocess.call(
'iptables -I FORWARD -eth 0 -m '
+ protocol
+ ' -t'+protocol
^---here
+ '--dport '
^-- here
+ port
+ '-j DNAT --to-destination'
^--here
+ ipAddress)
^---here
As generated, your iptables line will look like
-ttcp--dport 80-j DNAT --to-destination1.2.3.4
-ttcp--dport is parsed as a SINGLE argument, ditto for 80-j and --to-destination1.2.3.4

Just pass the argument shell=True along with the command.
iptables = subprocess.call('iptables -I FORWARD -eth 0 -m '+protocol+' -t'+protocol+'--dport '+port+'-j DNAT --to-destination'+ipAddress, shell=True)

Python 2.6 - passing | char into os.system in for loop

I have a list of IP's that I want to run a whois (using the linux tool whois) against and only see the Country option.
Here is my script:
import os
import time
iplist = open('ips.txt').readlines()
for i in iplist:
time.sleep(2)
print "Country: IP {0}".format(i)
print os.system("whois -h whois.arin.net + {0} | grep Country ".format(i))
So I want to display what IP is being ran, then I just want to see the Country info using grep. I see this error when I run it and the grep is not ran:
sh: -c: line 1: syntax error near unexpected token `|'
sh: -c: line 1: ` | grep Country '
this code below works so it must be an issue with my for loop:
print os.system("whois -h whois.arin.net + {0} | grep Country ".format('8.8.8.8'))
What am I doing wrong? Thank you!!!!

You're not stripping trailing newlines from the lines you read from the file. As a result, you are passing to os.system a string like "whois -h whois.arin.net + a.b.c.d\n | grep Country". The shell parses the string as two commands and complains of "unexpected token |" at the beginning of the second one. This explains why there is no error when you use a hand-made string such as "8.8.8.8".
Add i = i.strip() after the sleep, and the problem will go away.

user4815162342 is correct about the issue you are having, but might I suggest you replace os.system with subprocess.Popen? Capturing the output from the system call is not intuitive.. should you want to result to go anywhere but your screen, you'll likely going to have issues
from subprocess import Popen, PIPE
server = 'whois.arin.net'
def find_country(ip):
proc = Popen(['whois', '-h', server, ip], stdout = PIPE, stderr = PIPE)
stdout, stderr = proc.communicate()
if stderr:
raise Exception("Error with `whois` subprocess: " + stderr)
for line in stdout.split('\n'):
if line.startswith('Country:'):
return line.split(':')[1].strip() # Good place for regex
for ip in [i.strip() for i in open('ips.txt').readlines()]:
print find_country(ip)
Python is awesome at string handling- there should be no reason to create a grep subprocess to pattern match the output of a separate subprocess.

Try sh:
import os
import time
import re
import sh
iplist = open('ips.txt').readlines()
for i in iplist:
time.sleep(2)
print "Country: IP {0}".format(i)
print sh.grep(sh.whois(i, h="whois.arin.net"), "Country")

Run shell command with input redirections from python 2.4?

What I'd like to achieve is the launch of the following shell command:
mysql -h hostAddress -u userName -p userPassword
databaseName < fileName
From within a python 2.4 script with something not unlike:
cmd = ["mysql", "-h", ip, "-u", mysqlUser, dbName, "<", file]
subprocess.call(cmd)
This pukes due to the use of the redirect symbol (I believe) - mysql doesn't receive the input file.
I've also tried:
subprocess.call(cmd, stdin=subprocess.PIPE)
no go there ether
Can someone specify the syntax to make a shell call such that I can feed in a file redirection ?
Thanks in advance.

You have to feed the file into mysql stdin by yourself. This should do it.
import subprocess
...
filename = ...
cmd = ["mysql", "-h", ip, "-u", mysqlUser, dbName]
f = open(filename)
subprocess.call(cmd, stdin=f)

The symbol < has this meaning (i. e. reading a file to stdin) only in shell. In Python you should use either of the following:
1) Read file contents in your process and push it to stdin of the child process:
fd = open(filename, 'rb')
try:
subprocess.call(cmd, stdin=fd)
finally:
fd.close()
2) Read file contents via shell (as you mentioned), but redirect stdin of your process accordingly:
# In file myprocess.py
subprocess.call(cmd, stdin=subprocess.PIPE)
# In shell command line
$ python myprocess.py < filename

As Andrey correctly noticed, the < redirection operator is interpreted by shell. Hence another possible solution:
import os
os.system("mysql -h " + ip + " -u " + mysqlUser + " " + dbName)
It works because os.system passes its argument to the shell.
Note that I assumed that all used variables come from a trusted source, otherwise you need to validate them in order to prevent arbitrary code execution. Also those variables should not contain whitespace (default IFS value) or shell special characters.