I am sorry for my bad english.
I search solution for my question some day,but not found.
this is my question:
i have some manager shell script in server-A.
i use
ssh username#other_server_ip < shell_script.sh
is run OK.
i want to do this in python.
i was test this:
1\paramiko, 'exec_command (str)' is only run ONE command.and i use stdin to invoke_shell,not ok
2\pexect,sendline() is only ONE command.
Please help me,thanks!
(some AIX not support sft,so i not want use sftp script to other server. )
the shell script like this:
#!/bin/sh
if [ $# -lt 1 ]
os=`uname`
if [ "$os" = "linux" ] || [ "$os" = "Linux" ]
then
temp=`df -P $diskname| tail -1`
if [ "$temp" = "" ]
then
echo "error!t=$diskname not found"
exit 0
fi
# diskutil=`echo $temp|awk '{printf("%s",$5)}'|awk '{gsub("%",""); print $0}'`
disk_size=`echo $temp | awk '{print $2}'`
disk_size_mb=`expr $disk_size / 1024`
disk_size=`echo | awk '{ printf("%.2f",(c1/1024.0)) }' c1=$disk_size_mb`
disk_size="${disk_size}"
elif [ "$os" = "SunOS" ]
then
temp=`df -k $diskname | tail -1`
....
elif [ "$os" = "AIX" ] || [ "$os" = "aix" ]
then
temp=`df -k $diskname |tail -1|sed -e "s/%//g"`
....
else
echo "error!!=Unsupported platform: $os"
exit
fi
echo "Total Size=$disk_size_mb"
If just triggering the script from Python is enough, use subprocess.call from the standard library. From looking at your script, I guess it's being run on a number of remote hosts, probably in parallel. You might want to have a look at the excellent fabric module. It's a wrapper around paramiko which greatly facillitates running commands both locally and remotely and transfering files up and down.
Related
I know it is most silly question for openface But I am new to openface and I am not abe to figure this out
align = openface.AlignDlib()
what do we need to pass in the arguements . In openface docoumentation it is written
align = openface.AlignDlib(args.dlibFacePredictor)
and it says in its docoumentation , argument is a dtring which is path to dlib's .
I dont know what that means its my first programm for openface
The string is expected to lead to some pretrained model (residing on your harddisk). Some dlib-models are listed here (official developer).
In terms of openface, the code shows:
mkdir -p dlib
if [ ! -f dlib/shape_predictor_68_face_landmarks.dat ]; then
printf "\n\n====================================================\n"
printf "Downloading dlib's public domain face landmarks model.\n"
printf "Reference: https://github.com/davisking/dlib-models\n\n"
printf "This will incur about 60MB of network traffic for the compressed\n"
printf "models that will decompress to about 100MB on disk.\n"
printf "====================================================\n\n"
wget -nv \
http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2 \
-O dlib/shape_predictor_68_face_landmarks.dat.bz2
[ $? -eq 0 ] || die "+ Error in wget."
bunzip2 dlib/shape_predictor_68_face_landmarks.dat.bz2
[ $? -eq 0 ] || die "+ Error using bunzip2."
fi
Meaning: it's shape_predictor_68_face_landmarks.dat (part of above list).
You can look into the code for additional understanding, e.g. here:
def __init__(self, inputDir, outputDir, verbose):
self.inputDir = inputDir
self.dlibFacePredictor = os.path.join(
dlibModelDir, "shape_predictor_68_face_landmarks.dat")
I am running this command through on bash console through iTerm
{ cd /usr/local/path/to/code; echo "hi1"; sudo chmod 777 /tmp/dissolve.log; echo "hi2"; python someapp/runner.py dissolve; echo "hi3"; } > /tmp/dissolve.log &
on tailing the file i get :
tail: /tmp/dissolve.log: file truncated
hi1
hi2
I am not able to figure out why i am not getting output of file python someapp/runner.py dissolve, when i do cmd + c, the expected output is appearing on tail log.
code snippet from runner.py:
if __name__ == '__main__':
program_name = sys.argv[1]
if program_name == 'dissolve':
obj = SomeClass() # this is properly imported
obj.some_function() # this has lot of `print` statements, which i intened to catch in '/tmp/dissolve.log'
Is the initial print inside some_function() passing the values some where other than the /tmp/dissolve.log?
Any suggestion why this could be happening?
This seems like a buffering issue, as you are sending the output to a file. You can force line buffering with stdbuf, like this:
{ cd /usr/local/path/to/code;
echo "hi1";
sudo chmod 777 /tmp/dissolve.log;
echo "hi2";
stdbuf -oL python someapp/runner.py dissolve;
echo "hi3"; } > /tmp/dissolve.log &
Going through the answers at superuser.
I'm trying to modify this to listen for multiple strings and echo custom messages such as ; 'Your server started successfully' etc
I'm also trying to tack it to another command i.e. pip
wait_str() {
local file="$1"; shift
local search_term="Successfully installed"; shift
local search_term2='Exception'
local wait_time="${1:-5m}"; shift # 5 minutes as default timeout
(timeout $wait_time tail -F -n0 "$file" &) | grep -q "$search_term" && echo 'Custom success message' && return 0 || || grep -q "$search_term2" && echo 'Custom success message' && return 0
echo "Timeout of $wait_time reached. Unable to find '$search_term' or '$search_term2' in '$file'"
return 1
}
The usage I have in mind is:
pip install -r requirements.txt > /var/log/pip/dump.log && wait_str /var/log/pip/dump.log
To clarify, I'd like to get wait_str to stop tailing when pip exits, whether successfully or not.
Following is general answer and tail could be replaced by any command that result in stream of lines.
IF different string needs different actions then use following:
tail -f var/log/pip/dump.log |awk '/condition1/ {action for condition-1} /condition-2/ {action for condition-2} .....'
If multiple conditions need same action them ,separate them using OR operator :
tail -f var/log/pip/dump.log |awk '/condition-1/ || /condition-2/ || /condition-n/ {take this action}'
Based on comments : Single awk can do this.
tail -f /path/to/file |awk '/Exception/{ print "Worked"} /compiler/{ print "worked"}'
or
tail -f /path/to/file | awk '/Exception/||/compiler/{ print "worked"}'
OR Exit if match is found
tail -f logfile |awk '/Exception/||/compiler/{ print "worked";exit}'
I am using Python to execute an external program by simply using Python's "os" library:
os.system("./script1") # script1 generates several log files and the most important for me is "info.log" !
os.system("./script2") # script2 uses the output from script1
The problem is that those scripts are in 50000 element "for loop" and "script1" needs at least 2 min to finish its job (it has fixed time duration)
In the first 1-2 sec I can find out whether I need the output data or not by looking in to the "info.log" file. However, as "script1" is an already compiled program and I can't modify it, I have to wait until it finishes.
I was thinking about a method in Bash that allows me to run two processes at the same time:
one is to start ./script1 and the other is to monitor any change in the "info.log" file...
If "info.log" has been updated or changed in size, then the second script should terminate both processes.
Something like:
os.system("./mercury6 1 > ./energy.log 2>/dev/null & sleep 2 & if [ $(stat -c %s ./info.log) -ge 1371]; then kill %1; else echo 'OK'; fi")
– which does not work...
Please, if someone knows a method, let me know!
I would suggest using subprocess, with a combination of this answer describing how to kill a bash script along with some method of monitoring the file, something like watchdog, or a simple polling loop.
import os
import time
os.system("script1")
while True: # Now this would run in 10 sec cycles and checks the output of the logs
try: # And you don't have the wait the whole 2 mins to script1 to be ready
o = open("info.log", "r") # or add os.system("script2") here
# Add here more logs
break
except:
print "waiting script1 to be ready.."
time.sleep(10) # Modify (seconds) to be sufficient to script1 to be ready
continue
You can try this one. Tell me if script1 pauses on runtime so we could try to configure it further.
#!/bin/bash
[ -n "$BASH_VERSION" ] || {
echo "You need Bash to run this script."
exit 1
}
set +o monitor ## Disable job control.
INFO_LOG='./info.log'
if [[ ! -f $INFO_LOG ]]; then
# We must create the file.
: > "$INFO_LOG" || {
echo "Unable to create $INFO_LOG."
exit 1
}
fi
read FIRSTTIMESTAMP < <(stat -c '%s' "$INFO_LOG") && [[ $FIRSTTIMESTAMP == +([[:digit:]]) ]] || {
echo "Unable to get stat of $INFO_LOG."
exit 1
}
# Run the first process with the script.
./script1 &
SCRIPT1_PID=$!
disown "$SCRIPT1_PID"
# Run the second process.
(
function kill_processes {
echo "Killing both processes."
kill "$SCRIPT1_PID"
exit 1
}
[[ -f $INFO_LOG ]] || {
echo "File has been deleted."
kill_processes
}
read NEWTIMESTAMP < <(stat -c '%s' "$INFO_LOG") && [[ $NEWTIMESTAMP == +([[:digit:]]) ]] || {
echo "Unable to get new timestamp of $INFO_LOG."
kill_processes
}
[[ NEWTIMESTAMP -ne FIRSTTIMESTAMP ]] || {
echo "$INFO_LOG has changed."
kill_processes
}
sleep 4s
) &
disown "$!"
You might also want to check some other similar solutions here: linux script with netcat stops working after x hours
(
function kill_processes {
echo "Killing both processes."
kill "$mercury6_PID"
exit 1
}
while IFS= read -r LINE; do
[[ "${LINE}" == "ejected" ]] && { ## Perhaps it should be == *ejected* ?
echo "$INFO_LOG has changed."
kill_processes
}
done < <(tail -f "$INFO_LOG")
) &
disown "$!"
It has been done ! not so elegant but it works! this is the code:
#!/bin/bash
[ -n "$BASH_VERSION" ] || {
echo "You need Bash to run this script."
exit 1
}
# set +o monitor ## Disable job control.
INFO_LOG="./info.out"
# Run the first process with the script.
./mercury6 1 > ./energy.log 2>/dev/null &
mercury6_PID=$!
disown "$mercury6_PID"
# Run the second process.
(
sleep 5
function kill_processes {
echo "Killing both processes."
kill "$mercury6_PID"
exit 1
}
while IFS= read -r LINE; do
[[ "${LINE}" == "ejected" ]] || [[ "${LINE}" == "complete." ]] && {
echo "$INFO_LOG has changed."
killall tail
kill_processes
}
done < <(tail -f "$INFO_LOG")
)
#&
#disown "$!"
If you have any recommendations you are very welcome! BTW I will try to make it with os.fork() in python as well.
Thanks again!
See here (Google is your friend). You can make the processes background tasks with the & sign, then use wait to have the parent process wait for its children.
I have a python script that i want to run in the background on startup. This is the script:
#!/usr/bin/python
from Adafruit_CharLCD import Adafruit_CharLCD
from subprocess import *
from time import sleep, strftime
from datetime import datetime
from datetime import timedelta
from os import system
from os import getloadavg
from glob import glob
#Variables
lcd = Adafruit_CharLCD() #Stores LCD object
cmdIP = "ip addr show eth0 | grep inet | awk '{print $2}' | cut -d/ -f1" #Current IP
cmdHD = "df -h / | awk '{print $5}'" # Available hd space
cmdSD = "df -h /dev/sda1 | awk '{print $5}'" # Available sd space
cmdRam = "free -h"
temp = 0
#Run shell command
def run_cmd(cmd):
p = Popen(cmd, shell=True, stdout=PIPE)
output = p.communicate()[0]
return output
#Initalises temp device
def initialise_temp():
#Initialise
system("sudo modprobe w1-gpio")
system("sudo modprobe w1-therm")
#Find device
devicedir = glob("/sys/bus/w1/devices/28-*")
device = devicedir[0]+"/w1_slave"
return device
#Gets temp
def get_temp(device):
f = open (device, 'r')
sensor = f.readlines()
f.close()
#parse results from the file
crc=sensor[0].split()[-1]
temp=float(sensor[1].split()[-1].strip('t='))
temp_C=(temp/1000.000)
temp_F = ( temp_C * 9.0 / 5.0 ) + 32
#output
return temp_C
#Gets time
def get_time():
return datetime.now().strftime('%b %d %H:%M:%S\n')
#Gets uptime
def get_uptime():
with open('/proc/uptime', 'r') as f:
seconds = float(f.readline().split()[0])
array = str(timedelta(seconds = seconds)).split('.')
string = array[0].split(' ')
totalString = string[0] + ":" + string[2]
return totalString
#Gets average load
def get_load():
array = getloadavg()
average = 0
for i in array:
average += i
average = average / 3
average = average * 100
average = "%.2f" % average
return str(average + "%")
#def get_ram():
def get_ram():
ram = run_cmd(cmdRam)
strippedRam = ram.replace("\n"," ");
splitRam = strippedRam.split(' ')
totalRam = int(splitRam[52].rstrip("M"))
usedRam = int(splitRam[59].rstrip("M"))
percentage = "%.2f" % ((float(usedRam) / float(totalRam)) * 100)
return percentage + "%"
#Gets the SD usage
def get_sd():
sd = run_cmd(cmdSD)
strippedSD = sd.lstrip("Use%\n")
return strippedSD
#Gets the HD usage
def get_hd():
hd = run_cmd(cmdSD)
strippedHD = hd.lstrip("Use%\n")
return strippedHD
def scroll():
while(1):
lcd.scrollDisplayLeft()
sleep(0.5)
#Uptime and IP
def screen1():
uptime = get_uptime()
lcd.message('Uptime %s\n' % (uptime))
ipaddr = run_cmd(cmdIP)
lcd.message('IP %s' % (ipaddr))
#Ram and load
def screen2():
ram = get_ram()
lcd.message('Ram Used %s\n' % (ram))
load = get_load()
lcd.message('Avg Load %s' % (load))
#Temp and time
def screen3():
time = get_time();
lcd.message('%s\n' % (time))
lcd.message('Temp %s' % (temp))
#HD and SD usage
def screen4():
sd = get_sd()
lcd.message('SD Used %s\n' % (sd))
hd = get_hd()
lcd.message('HD Used %s' % (hd))
#Pause and clear
def screenPause(time):
sleep(time)
#In here to reduce lag
global temp
temp = str(get_temp(device));
lcd.clear()
###########################################################################################################
#Initialise
lcd.begin(16,2)
device = initialise_temp()
lcd.clear()
#Testing
#Main loop
while(1):
screen1()
screenPause(5)
screen2()
screenPause(5)
screen3()
screenPause(5)
screen4()
screenPause(5)
I know i probably havnt done things the write way but its the first attempt.
My startup script is in /etc/init.d This is the script:
#! /bin/sh
### BEGIN INIT INFO
# Provides: LCD looping
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: LCD daemon
# Description: This file should be used to construct scripts to be
# placed in /etc/init.d.
### END INIT INFO
# Author: Foo Bar <foobar#baz.org>
#
# Please remove the "Author" lines above and replace them
# with your own name if you copy and modify this script.
# Do NOT "set -e"
# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="Loops the LCD screen through LCD.py"
NAME=startup.py
DAEMON=/home/pi/Programming/LCD/startup.py
DAEMON_ARGS=""
PIDFILE=/var/run/daemonLCD.pid
SCRIPTNAME=/etc/init.d/daemonLCD
# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0
# Read configuration variable file if it is present
[ -r /etc/default/daemonLCD ] && . /etc/default/daemonLCD
# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions
#
# Function that starts the daemon/service
#
do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
|| return 1
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
$DAEMON_ARGS \
|| return 2
# Add code here, if necessary, that waits for the process to be ready
# to handle requests from services started subsequently which depend
# on this one. As a last resort, sleep for some time.
}
#
# Function that stops the daemon/service
#
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
RETVAL="$?"
[ "$RETVAL" = 2 ] && return 2
# Wait for children to finish too if this is a daemon that forks
# and if the daemon is only ever run from this initscript.
# If the above conditions are not satisfied then add some other code
# that waits for the process to drop all resources that could be
# needed by services started subsequently. A last resort is to
# sleep for some time.
start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
[ "$?" = 2 ] && return 2
# Many daemons don't delete their pidfiles when they exit.
rm -f $PIDFILE
return "$RETVAL"
#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
#
# If the daemon can reload its configuration without
# restarting (for example, when it is sent a SIGHUP),
# then implement that here.
#
start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
return 0
}
case "$1" in
start)
[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
do_start
case "$?" in
0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
esac
;;
stop)
[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
do_stop
case "$?" in
0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
esac
;;
status)
status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
;;
#reload|force-reload)
#
# If do_reload() is not implemented then leave this commented out
# and leave 'force-reload' as an alias for 'restart'.
#
#log_daemon_msg "Reloading $DESC" "$NAME"
#do_reload
#log_end_msg $?
#;;
restart|force-reload)
#
# If the "reload" option is implemented then remove the
# 'force-reload' alias
#
log_daemon_msg "Restarting $DESC" "$NAME"
do_stop
case "$?" in
0|1)
do_start
case "$?" in
0) log_end_msg 0 ;;
1) log_end_msg 1 ;; # Old process is still running
*) log_end_msg 1 ;; # Failed to start
esac
;;
*)
# Failed to stop
log_end_msg 1
;;
esac
;;
*)
#echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
exit 3
;;
esac
:
Im think i have missed something as when i type daemonLCD start it says command not found.
Any input would be great.
Thanks
Assuming you may want to manage more than one daemon in the future, let me recommend Supervisord. It's much simpler than writing and managing your own init.d scripts.
For example, starting your script would be as easy as including this in the conf:
[program:myscript]
command=/usr/bin/python /path/to/myscript.py
I use an init.d script available here. Rename it to supervisord and copy it to your /etc/init.d/ then run:
sudo update-rc.d supervisord defaults
I believe that init script has supervisord run as root as default. You can have it drop to run as another user if you like. I'm not if children run as root or not, although I'd assume not. Go ahead and check, but if they don't you can stick a sudo before the python command in your supervisord.conf where you call the script.
It that doesn't run, (or if you want supervisord to run as a non-root but still want your script run as root) you can allow for anyone (or a group of users) to run the python script as root (although you should make quite certain that this script cannot be edited by anyone other than root).
edit your sudoers file with "sudo visudo" and add the following to the end:
USERS ALL=(ALL) NOPASSWD: /path/to/myscript.py
Then make sure you have a shebang at the beginning of your python script and change the command to omit the python call, i.e:
[program:myscript]
command=sudo /path/to/myscript.py
Here's a good blog post which deals with this question: Getting a Python script to run in the background (as a service) on boot
Use daemontools from djb. It is a lot easier than the other answers provided. For starters you can install daemon tools with apt-get so you do not need to worry about grabbing an unknown script from a gist and you get updates through debian like normal. daemontools also takes care of restarting the service if it dies and provides for logging. There is a description of daemontools and debian here:
http://blog.rtwilson.com/how-to-set-up-a-simple-service-to-run-in-the-background-on-a-linux-machine-using-daemontools/
djb's page aout daemontools:
http://cr.yp.to/daemontools.html
This is a classic mistake new Unix/Linux users make. /etc/init.d isn't in your path which is why you can't just run daemonLCD. Try using the full path (/etc/init.d/daemonLCD start) or prepending ./ (./daemonLCD start).
The script needs to be executable for either of the above to work.
thanks for the code above. I've been using it to figure out how to set up a daemon on a linux machine.
With some tweaking I could get it to work quite well.
But something puzzled me. And that was checking if the process was running, by checking the exists of the /var/run/myfile.pid
That's just the pidfile - NOT the process, right?
Take a look at /lib/lsb/init-functions.status_of_proc
status_of_proc () {
local pidfile daemon name status OPTIND
pidfile=
OPTIND=1
while getopts p: opt ; do
case "$opt" in
p) pidfile="$OPTARG";;
esac
done
shift $(($OPTIND - 1))
if [ -n "$pidfile" ]; then
pidfile="-p $pidfile"
fi
daemon="$1"
name="$2"
status="0"
pidofproc $pidfile $daemon >/dev/null || status="$?"
if [ "$status" = 0 ]; then
log_success_msg "$name is running"
return 0
elif [ "$status" = 4 ]; then
log_failure_msg "could not access PID file for $name"
return $status
else
log_failure_msg "$name is not running"
return $status
fi
}
That's only dealing with the success or failure of accessing the PID file.
Now, I'm building this daemon to go on a small device. I've discovered it's using BusyBox and I don't have init-functions :-(
But I do have pidof.
So I added
log_success_msg "pidof $NAME is $(pidof -x $NAME)" >> $LOGFILE
log_success_msg "PIDFILE of $NAME is" >> $LOGFILE
sed -n '1p' < $PIDFILE >> $LOGFILE
and checked $LOGFILE and lo and behold the numbers are different.
I did pstree -s -p on both numbers and
the pidof number spits out a very short tree, so it's for the root level process
but the $PIDFILE number vomits out branch after branch, so I don't think pstree can find the process.
Yes, the do_stop in Joseph Baldwin Roberts's code will kill both processes. But if the process is killed in another way e.g. kill -9 12345, the $PIDFILE is still there. So, the daemon will falsely believe the process is already running an refuse to start.