I've got a Docker Service Log that takes in NiFi Actions and I want to capture only Log Entries that include "Successfully sent" and "Failed to process session" (and nothing more). They should be captured in a directory called "nifi_logs" in the present working directory. I need to do all of this using Python.
This is what I got so far:
docker_log = 'docker service logs nifi | grep -e "Successfully sent" -e "Failed to process session" >> $PWD/nifi_logs/nifi1.log'
subprocess.Popen(docker_log, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
I believe subprocess.Popen() is having difficulty with the double quotes used in the grep, as nifi1.log is completely empty. If the first command looks like the following:
docker_log = 'docker service logs nifi | grep session >> $PWD/nifi_logs/nifi1.log'
The Python code works just fine and captures all log entries with "session" in nifi1.log. As I explained above though, I need to grep for 2 kinds of Log Entires and both include multiple words, meaning I need to use quotes.
If I were to just run this command on the Terminal without Python:
docker service logs nifi | grep -e "Successfully sent" -e "Failed to process session" >> $PWD/nifi_logs/nifi1.log
The log generates the entries just fine, so I know the Docker Service command is written correctly.
I've tried switching the single and double quotes around, I've tried using \" instead of " within the single quotes ... nifi1.log continues to be empty.
I also tried using os.system() instead of subprocess.Popen(), but I run into the same problem (and I believe os.system() is somewhat deprecated).
Any ideas what I'd need to do to change what docker_log equals so that it will properly grep for the 2 search criteria? So you're aware: this question is not asking HOW I generate the log entries (I know what Docker Services I'm looking for, they generate properly), just what I need to do to get Python Subprocess Popen to accept a command with quotes in it.
Thank you for your assistance #David. Looking at your example, I found a solution: I removed stdout=subprocess.PIPE from subprocess.Popen and now it accepts double quotes just fine!
docker_log = 'docker service logs nifi | grep -e "Successfully sent" -e "Failed to process session" >> $PWD/nifi_logs/nifi1.log'
subprocess.Popen(docker_log, shell=True, stderr=subprocess.STDOUT)
Related
I have found a way to get the info on the latest Git tag from the remote repository:
git ls-remote --tags --refs --sort="version:refname" git://github.com/git/git.git | awk -F/ 'END{print$NF}'
This works fine from the command line.
How would I get this info from the python script?
I tried os.system(command), but it doesn't seem like a good choice?
You can use subprocess.check_output and do what your awk incantation does in Python (namely print the last line's last /-separated segment).
Using the list form of arguments for subprocess.check_output ensures you don't need to think about shell escaping (in fact, the shell isn't even involved).
import subprocess
repo_url = "https://github.com/git/git.git"
output_lines = subprocess.check_output(
[
"git",
"ls-remote",
"--tags",
"--refs",
"--sort=version:refname",
repo_url,
],
encoding="utf-8",
).splitlines()
last_line_ref = output_lines[-1].rpartition("/")[-1]
print(last_line_ref)
I am trying to trigger the python script or shell script whenever a desktop notification has arrived using dbus-monitor
I am using the command in this way
dbus-monitor "interface='org.freedesktop.Notifications'" | grep --line-buffered "string" | xargs -I '{}' python3 ./test.py {}
after that, I am trying to send the desktop notification from another terminal using
-> notify-send "hello" "world"
the output for the above custom notification is
string "notify-send"
string ""
string "hello"
string "world "
string "urgency"
string "notify-send"
string ""
string "hello"
string "world "
string "urgency"
but if my output of this command is 10 lines, then the python script is getting called for every line.
but my expectation is to call the python script once for every notification and then get all the output in a single line as a param for the python script.
It is wise to take advantage of systemd integration with dbus.
Using systemd integration the programmer has better controls/sensors over the dbus integration. Also can take advantage on systemd loging/monitors mechanisms.
There is a good article here about systemd dbus with python..
Also there is very related answer to your question in this answer. as well.
How do I find the result of a find, in this script for sending by email ??
I've tried this way
How do I put the result within the email.
The result of find
mailx -s "result of find " support#systems.com
You need to "pipe" it.
find /bin | mailx -s "result of find in /bin" support#systems.com
Pipes allow you to transfer the output of a shell command into input of another command. They are very convenient and widely use in the linux/unix systems.
You should look into it.
Something like this would meet your needs:
find /path_to_examine -type f -print | mailx -s "Find Results" support#systems.com
mailx expects its input from STDIN; either read from the initiating terminal (and terminated with a control-D sequence); or redirected; or piped. The example chosen uses a pipe. You could also redirect an already created file, thusly:
mailx -s "Find Results" support#systems.com < myresults
I'm trying to create a python script on linux that does a 'whois' command on every connected/connecting IP Address that is parsed from the 'netstat' command.
I am get an error saying "sh: 1: Syntax error: Unterminated quoted string"
and the whois usage options posted below that.
Can anyone explain to me what's wrong the script? I believe it's something to do with the for loop and the way it executes the whois command I just cant seem to find a solution. Below is the script in question:
#!/usr/bin/python
from os import system
answer = [system("netstat -alpntu46 |grep 'ESTABLISHED\|SYN_RECV' | awk '{print $5 }' |cut -d: -f1'")]
for i in answer:
system('whois')
EDIT So my original problem is completely fixed, I'm getting no errors. However, now all the script does is list the IP Addresses and underneath that it lists the whois usage examples:
-h HOST, --host HOST connect to server HOST
-p PORT, --port PORT connect to PORT
-H hide legal disclaimers
--verbose explain what is being done
--help display this help and exit
--version output version information and exit"
So it seems to be running the answer variable but not being able to run the whois command on each address.
Your command string (inside de system() command) has one ' more than needed (at the end of the string). Here it is corrected:
#!/usr/bin/python
from os import system
answer = [system("netstat -alpntu46 |grep 'ESTABLISHED\|SYN_RECV' | awk '{print $5 }' |cut -d: -f1")]
for i in answer:
system('whois')
EDIT (your second question):
When you do for i in answer in python you are looping through all items in your answer, that is correct, however for each IP address you are looping on you are executing only a 'whois' command, without passing any parameters. You should add the parameter to the string, as in:
for i in answer:
system('whois %s' % i)
that is assuming the variable i holds the ip string.
Please check the edit on my first answer (posting this just so you get notified.)
I am trying to run a python script, and I need to Rsh a command from the script, the command I want to run is : df -Pk|grep "sd\|md"|gawk '{print $2}'
and I do it as -
cmd2='df -Pk|grep \\\"sd\|md\\\"|gawk \'{print $2}\''
process = subprocess.Popen(['rsh',ip,cmd2],stdout=subprocess.PIPE)
output = process.communicate()[0]
However when I do run the script,I get nothing in output.
I am new to python and as far as I know, its a problem with the escape characters.
Any help would be great.
Note:
I have to use Rsh only and cannot use ssh
Thanks
Enclose the command, with proper shell quoting, in triple quote marks:
"""df -Pk|grep "sd\|md"|gawk '{print $2}'"""
See also the Python tutorial's bit on strings.