I'm trying to write a Python script that reads lines of strings from a file and executes a bash-shell command with each line as parameter:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
lines = [line.strip() for line in open('/tmp/i586rpm.txt')]
for rpm in lines:
try:
s = os.system("rpm -qip") % lines
except Exception:
print "Something wrong"
But I always get an error. I think that there is something wrong with % lines.
Any ideas?
Yes, you're not building the command string properly. Try:
s = os.system("rpm -qip \"%s\"" % rpm)
There are a few things that could be improved:
Uses os.system instead of subprocess.Popen (for more information, please have look at the documentation)
Iterates over lines, but it doesn't use the iteration variable (rpm)
Attemps to format the output string from os.system with lines as a parameter.
My advice is to use this code:
for line in lines:
p = subprocess.Popen('rpm -qip {0}'.format(line),
shell=True,
stdout=subprocess.PIPE)
stdout, stderr = p.communicate()
instead of
for rpm in lines:
s = os.system("rpm -qip") % lines
do this
for rpm in lines:
s = os.system('rpm -qip "%s"' % rpm)
EDIT correct the code
Related
I am having this problem where I can print out the powershell code output with the print() function, but when I try to do the same, except this time I write the output to a file, the only thing that is written in the file is "0", why would the printed output be different from when I write the same exact code, except that I this time "print" it to a text file.
I want the text file to contain exactly what the print function prints to the terminal, why isn't it working, and how can I get it to work??
Here are some pictures and the code:
import os
import time
def monitorprocess(process):
run = True
time_q = float(input("How many minutes before each check? "))
while run:
timespan = os.system(f'powershell New-TimeSpan -Start(Get-process {process}).StartTime')
try:
open(f'powershellpython\{process}.txt','x')
except:
pass
with open(f'powershellpython\{process}.txt',"w") as file:
file.write(str(timespan))
print(timespan)
time.sleep(time_q*60)
def processes():
process = input("What is the name of your process, if you are unsure, type 'get-process', and if you want to use ID (this works with multiple processes with the same name) type ID: \n")
if process == "get-process":
print(os.system("powershell get-process"))
process = input("What is the name of your process, if you are unsure, type 'get-process', and find your process: \n")
else:
monitorprocess(process)
processes()
And there is some more output with the print, that being "hours" and "days", but that does not really matter in this context.
I can't test it with powershell because I don't use Windows but to catch output you should use other methods in subprocess
ie. subprocess.check_output()
import subprocess
output = subprocess.check_output(cmd, shell=True)
with open('output.txt', 'w') as file:
file.write(output.decode())
ie. subprocess.run()
import subprocess
from subprocess import PIPE
output = subprocess.run(cmd, shell=True, stdout=PIPE).stdout
with open('output.txt', 'w') as file:
file.write(output.decode())
Probably you could even redirect run() directly to file using stdout=
with open('output.txt', 'w') as file:
subprocess.run(cmd, shell=True, stdout=file)
Using os.system() you can catch only return code (error code) and you could only do python script.py > output.txt to get text in file output.txt
What you see on screen can be produced by PowerShell.
Try
timespan = os.system(f'powershell New-TimeSpan -Start(Get-process {process}).StartTime | Format-List | Out-String')
This now will not return a TimeSpan object, but rather a multiline string meant to display the properties of the object on screen.
I have the below print statement
print(subprocess.check_output(['bash', '-c', shell_script], stdin=open('/etc/fstab', 'r')))
that prints few lines like '/xyz NOT mounted' and I would like to store it into a variable just like how the lambda function does it.
out = lambda: print("/xyz NOT mounted")
out()
I would then need to use that variable to write an if statement as in
if out contains any string, then print contains string, else contains nothing
Could you please try following, tested and written in Python2.7
#!/usr/bin/python
import re
out="/xyz NOT mounted"
if re.search(r'[a-zA-Z]+', out):
print "String found."
else:
print "String NOT found."
When value of variable out is having string values it prints String found.
Now lets test it with NULL value of variable val here.
cat script.py
#!/usr/bin/python
import re
out=""
if re.search(r'[a-zA-Z]+', out):
print "String found."
else:
print "String NOT found."
When we run above then it gives as String NOT found.
can you not just save the variable and then print it?
x = subprocess.check_output()
print(x)
#and then...
if x == ???:
#do something
Saving the variable and printing it seems a lot easier than trying to grab hold of the print statements' output.
Here, an example to execute a command using subprocess and store and output/buffer data in a variable.
Ex:
Command_Input = subprocess.Popen("Your Command",stdin=process_output.stdout, stdout=subprocess.PIPE, shell=True)
Command_Output = Command_Input.communicate()[0]
Print(Command_Output)
I cannot run your code on my machine because I miss the shell_script part.
For similar cases I use subprocess.PIPE and collect the output lines in a list that you can iterate and check later:
import subprocess
def bash_command(cmd):
sp = subprocess.Popen(['/bin/bash', '-c', cmd], stdout=subprocess.PIPE)
return sp.stdout.readlines()
result = bash_command('ls') # Insert here your bash command
for line in result:
if "NOT mounted" in line:
print("Hey I have a line containing NOT mounted")
else:
print("Hey I have a line not containing NOT mounted")
I'm trying to write a simple script that takes a list of words I've created in a text file on linux and runs it through a program that checks the word against a steganography extractor.
The program (steghide) uses the following syntax in the command:
steghide --extract -p {password here} -sf {filename here}
I've been able to call the file and set up a for loop for the words in the list, but can not find a way to input the word from that iteration into each command.
Here's how I've been trying to work it.
import os
import sys
script, filename = argv
filename2 = sys.open(filename)
for word in filename2:
os.system('steghide --extract -p [want to put something here] -sf stegfilename')
I'm on a controlled box and can't download anything beyond what I already have. Any help is appreciated.
Update:
I got it to work. But now I'm trying to get it to exit out if it finds the correct answer. I am just having a hard time getting Python to read the output. Here's what I have so far.
`import subprocess
from sys import argv
script, filename = argv
passes = filename
with open(passes) as f:
for line in f:
proc = subprocess.popen(['steghide', '--extract', '-p' line.strip(), '-sf', 'stegged file name'],stdout = subprocess.PIPE)
stdout = proc.communicate()[0]
output = proc.stdout.readline()
if 'wrote' in output:
print 'YOU FOUND IT!'
break
else:
print line`
This is a good time to learn string formating options in Python. They let you insert values dynamically into a string. An example:
"This {0} is an example".format([1,2,3])
>>>> "This [1,2,3] is an example"
In this particular case, you want to do
value = 'foo' # the item you want to insert - replace with list, number, etc.
...
for word in filename2:
os.system('steghide --extract -p {0} -sf stegfilename'.format(value))
This will insert the value into your string, and then call steghide on that string.
Use the subprocess module instead; it gives you more options/control and os.system() is deprecated.
import subproces
with open(filename, "r") as f:
# assumes that you have one word per line
for line in f:
subprocess.call(['steghide', '--extract', '-p', line.strip(), '-sf', stegfilename])
# or if you want the output of running Niels Provos' cool, old tool :)
cmd_output = subprocess.check_output(['steghide', '--extract', '-p', line.strip(), '-sf', stegfilename])
Use subprocess.check_call passing a list of args:
from subprocess import check_call
for word in map(str.rstrip, filename2):
check_call(['steghide', "--extract", "-p", word, "-sf", "stegfilename'"])
String formatting is the subject here.
import os
import sys
script, filename = argv
filename2 = sys.open(filename)
for word in filename2:
os.system('steghide --extract -p ' +[want to put something here]+ '-sf stegfilename')
To join all elements of a list in a string, try python join:
' '.join(your_variable)
Example:
var = ['t', 't', 't', 't', 't', 't']
joined = ' '.join(var)
print(joined)
print(type(joined))
t t t t t t
class 'str'
I am writing something for static analysis of source code in different languages. As anything has to be open source and callable from command line I now have downloaded one tool per language. So I decided to write a python script listing all source files in a project folder and calling the respective tool.
So part of my code looks like this:
import os
import sys
import subprocess
from subprocess import call
from pylint.lint import Run as pylint
class Analyser:
def __init__(self, source=os.getcwd(), logfilename=None):
# doing initialization stuff
self.logfilename = logfilename or 'CodeAnalysisReport.log'
self.listFiles()
self.analyseFiles()
def listFiles(self):
# lists all source files in the specified directory
def analyseFiles(self):
self.analysePythons()
self.analyseCpps()
self.analyseJss()
self.analyseJavas()
self.analyseCs()
if __name__ == '__main__':
Analyser()
Let's have at a look at the C++ files part (I use Cppcheck to analyse those):
def analyseCpps(self):
for sourcefile in self.files['.cc'] + self.files['.cpp']:
print '\n'*2, '*'*70, '\n', sourcefile
call(['C:\\CodeAnalysis\\cppcheck\\cppcheck', '--enable=all', sourcefile])
The console output for one of the files (it's just a random downloaded file) is:
**********************************************************************
C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc
Checking C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc...
[C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc:18]: (style) The scope of the variable 'oldi' can be reduced.
[C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc:43]: (style) The scope of the variable 'lastbit' can be reduced.
[C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc:44]: (style) The scope of the variable 'two_to_power_i' can be reduced.
(information) Cppcheck cannot find all the include files (use --check-config for details)
Line 1 and 2 coming from my script, lines 3 to 7 coming from Cppcheck.
And this is what I want to save to my log file, for all the other files too. Everything in one single file.
Of course I have searched SO and found some methods. But none is working completely.
First try:
Adding sys.stdout = open(self.logfilename, 'w') to my constructor. This makes line 1 and 2 of the above showed output be written to my log file. The rest is still shown on console.
Second try:
Additionaly, in analyseCpps I use:
call(['C:\CodeAnalysis\cppcheck\cppcheck', '--enable=all', sourcefile], stdout=sys.stdout)
This makes my log file to be:
Checking C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc...
**********************************************************************
C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc
and the console output is:
[C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc:18]: (style) The scope of the variable 'oldi' can be reduced.
[C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc:43]: (style) The scope of the variable 'lastbit' can be reduced.
[C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc:44]: (style) The scope of the variable 'two_to_power_i' can be reduced.
Not what I want.
Third try:
Using Popen with pipe. sys.stdout is back to default.
As preliminary work analyseCpps now is:
for sourcefile in self.files['.cc'] + self.files['.cpp']:
print '\n'*2, '*'*70, '\n', sourcefile
p = subprocess.Popen(['C:\\CodeAnalysis\\cppcheck\\cppcheck', '--enable=all', sourcefile], stdout=subprocess.PIPE)
p.stdout.read()
p.stdout.read() shows only the last line of my desired output (line 7 in code box 3)
Fourth try:
Using subprocess.Popen(['C:\CodeAnalysis\cppcheck\cppcheck', '--enable=all', sourcefile], stdout=open(self.logfilename, 'a+')) just writes the one line Checking C:\CodeAnalysis\testproject\cpp\BiggestUnInt.cc... to my logfile, the rest is shown on the console.
Fifth try:
Instead of subprocess.Popen I use os.system, so my calling command is:
os.system('C:\CodeAnalysis\cppcheck\cppcheck --enable=all %s >> %s' % (sourcefile, self.logfilename))
This results in the same log file as my fourth try. If I type the same command directly in the windows console the result is the same. So I guess it it is not exactly a python problem but still:
If it is on the console there must be a way to put it in a file. Any ideas?
E D I T
Foolish me. I'm still a noob so I forgot about the stderr. That's where the decisive messages are going to.
So now I have:
def analyseCpps(self):
for sourcefile in self.files['.cc'] + self.files['.cpp']:
p = subprocess.Popen(['C:\\CodeAnalysis\\cppcheck\\cppcheck', '--enable=all', sourcefile], stderr=subprocess.PIPE)
with open(self.logfilename, 'a+') as logfile:
logfile.write('%s\n%s\n' % ('*'*70, sourcefile))
for line in p.stderr.readlines():
logfile.write('%s\n' % line.strip())
and it's working fine.
ANOTHER EDIT
according to Didier's answer:
with sys.stdout = open(self.logfilename, 'w', 0) in my constructor:
def analyseCpps(self):
for sourcefile in self.files['.cc'] + self.files['.cpp']:
print '\n'*2, '*'*70, '\n', sourcefile
p = subprocess.Popen(['C:\\CodeAnalysis\\cppcheck\\cppcheck', '--enable=all', sourcefile], stdout=sys.stdout, stderr=sys.stdout)
There are several problems:
you should redirect both stdout and stderr
you should use unbuffered files if you want to mix normal print and the output of launched commands.
Something like this:
import sys, subprocess
# Note the 0 here (unbuffered file)
sys.stdout = open("mylog","w",0)
print "Hello"
print "-----"
subprocess.call(["./prog"],stdout=sys.stdout, stderr=sys.stdout)
print "-----"
subprocess.call(["./prog"],stdout=sys.stdout, stderr=sys.stdout)
print "-----"
print "End"
You need to redirect stderr too, you can use STDOUT or pass the file object to stderr=:
from subprocess import check_call,STDOUT
with open("log.txt","w") as f:
for sourcefile in self.files['.cc'] + self.files['.cpp']:
check_call(['C:\\CodeAnalysis\\cppcheck\\cppcheck', '--enable=all', sourcefile],
stdout=f, stderr=STDOUT)
Try to redirect stdout and stderr to a logfile:
import subprocess
def analyseCpps(self):
with open("logfile.txt", "w") as logfile:
for sourcefile in self.files['.cc'] + self.files['.cpp']:
print '\n'*2, '*'*70, '\n', sourcefile
call(['C:\\CodeAnalysis\\cppcheck\\cppcheck',
'--enable=all', sourcefile], stdout=logfile,
stderr=subprocess.STDOUT)
In this example the filename is hardcoded, but you should be able to change that easily (to your self.logfilename or similar).
I am having a difficult time understanding how to get python to call a system command with the subprocess.Popen function.
the_file = ('logs/consolidated.log.gz')
webstuff = subprocess.Popen(['/usr/bin/zgrep', '/meatsauce/', the_file ],stdout=subprocess.PIPE)
for line in webstuff.stdout:
print line
Trying to get python to build another file with my search string.
The problem is in how you're constructing your arguments. The way you have it now, you're running:
/usr/bin/zgrep /meatsauce/ logs/consolidated.log.gz
Note the space between /meatsauce/ and logs...
To do what I think you're intending, use os.path.join a la:
import os
the_file = 'logs/consolidated.log.gz'
webstuff = subprocess.Popen(['/usr/bin/zgrep', os.path.join('/meatsauce/', the_file)],stdout=subprocess.PIPE) % dpt_search
for line in webstuff.stdout:
print line
Not exactly sure about your question, but the following snippet will call zgrep with two arguments, a searchterm and a filename - and print the result (stdout) line by line:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import subprocess
# filename and searchterm
fn, term = 'access_log.gz', 'hello'
p = subprocess.Popen(['/usr/bin/zgrep', term, fn], stdout=subprocess.PIPE)
for line in p.stdout:
print line
In the code you posted the string interpolation (% dpt_search) does not work out, since there is not pure string in front of the modulo sign - in fact it should fail with something like:
TypeError: "unsupported operand type(s) for %: 'Popen' and 'str'"
the_file = ('webalizerlogs/consolidated.log.gz')
output_f = open('output.txt','w')
webstuff = subprocess.Popen(['/usr/bin/zgrep', dpt_search, the_file ],stdout=output_f)
I think you are simply trying to grep the a content in the a file. Is it?
import os
import subprocess
the_file = os.path.join(os.getcwd(),'logs/consolidated.log.gz')
proc = subprocess.Popen(['/usr/bin/zgrep', dpt_search, the_file], stdout=subprocess.PIPE)
out, err = proc.communicate()
with open('resultoutput','w') as f:
f.write(out)
subprocess.call(['/usr/bin/zip',os.path.join(os.getcwd(),'resultoutput'])
Check the docs as well.