We Keep Coding

daemonizing a python script: What is better? start-stop-daemon or python-daemon

I like to start a python program as a daemon. I checked a bit the internet and came up with two solutions.
The first one is to use start-stop-daemon on something like this
#!/bin/sh
start() {
start-stop-daemon -S -q -m -b -p /var/run/sample.pid --exec /usr/bin/python /bin/sample.py && echo "OK" || echo "Failed"
}
stop() {
start-stop-daemon -K -q -p /var/run/sample.pid
}
case "${1}" in
start)
start
echo "sample service started"
;;
stop)
stop
echo "sample service stopped"
;;
restart)
stop
start
echo "sample service restarted"
;;
*)
echo "usage: $0 {start|stop|restart}"
;;
esac
The second one is to use python-daemon
so I guess we have an extra python file - lets name it "sampleDaemon.py"
import daemon
import sample
with daemon.DaemonContext():
sample()
And a bash script (I am not sure about that - I guess this way):
#!/bin/sh
/usr/bin/python /bin/sampleDeamon.py
What is the best way to do it? What are the limitations? The python-deamon documentation is not that big(or I didn't found it).

Linux start-stop-daemon directory error calling shell/python script

I am just getting acquainted with Linux and I cannot seem to get the start-stop-daemon to run a python script due to directory issues. In a linux file structure I have the files:
~/test.txt
THIS LINE IS A TEST
~/test.py
#!/usr/bin/python
import time
with open("test.txt") as f:
while True:
try:
print("Hello World")
print(f.readline())
time.sleep(2)
except KeyboardInterrupt:
f.close()
break
~/test.sh
#!/bin/bash
echo "SHELL SCRIPT SUCCESS" > /var/log/test.log
cd ~/
./test.py > /var/log/test.log
Upon calling sudo bash ~/test.sh from any directory the test.log is populated as expected with the stdout originating from test.py. For some reason, starting the following start-stop-daemon service script WILL generate a test.log but does NOT populate it with the stdout:
/etc/init.d/test
#!/bin/sh
### BEGIN INIT INFO
# Provides: Python test script
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Prints out daemonized argument
# Description: Creates output of argument
### END INIT INFO
DAEMON_DIR=/home/alex
DAEMON=$DAEMON_DIR/test.sh
DAEMON_NAME=test
DAEMON_OPTS="hello"
DAEMON_USER=root
PYTHON=/usr/bin/python
PIDFILE=/var/run/$DAEMON_NAME.pid
. /lib/lsb/init-functions
do_start () {
log_daemon_msg "Starting system $DAEMON_NAME daemon"
#start-stop-daemon --start --background --pidfile $PIDFILE --make-pidfile --user $DAEMON_USER --exec $PYTHON --startas $DAEMON
start-stop-daemon --start --background --pidfile $PIDFILE --make-pidfile --chuid $DAEMON_USER --startas /bin/bash /home/alex/test.sh
log_end_msg $?
}
do_stop () {
log_daemon_msg "Stopping system $DAEMON_NAME daemon"
start-stop-daemon --stop --pidfile $PIDFILE --retry 10
log_end_msg $?
}
case "$1" in
start|stop)
do_${1}
;;
restart|reload|force-reload)
do_stop
do_start
;;
status)
status_of_proc "$DAEMON_NAME" "$DAEMON" && exit 0 || exit $?
;;
*)
echo "Usage: /etc/init.d/$DAEMON_NAME {start|stop|restart|status}"
exit 1
;;
esac
exit 0
Is this a directory issue that can be addressed within the start-stop-daemon?
Alternatively I'd be open to other methods of script servicing that can persist through a cold boot (i.e. no cron jobs)

Try calling cd using an absolute path, for example /home/alexjg/ instead of ~/; the reason it was broken before is that in your example you're using sudo which keeps the home directory of the user running it. However when you're calling the bash script from init it will use root's home directory instead which doesn't contain test.py.
The file is created because the redirection is still succeeding; however because starting Python failed there was no output.

Python script on boot Linux

I would like to launch my python program (Graphical User Interface) on startup in Linux (Raspbian on a Raspberry PI).
I've made an initscript to launch my Python program, and I put it in the etc/init.d map.
I enabled it with the update-rc.d command. It all works fine.
But my Python script won't start with the following code in the initscript:
#!/bin/bash
### BEGIN INIT INFO
# Provides: GUI
# Required-Start:
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: This is a test daemon
# Description: This is a test daemon
# This provides example about how to
# write a Init script.
### END INIT INFO
case $1 in
start)
python3 /home/pi/Desktop/GUI/GUI.py
;;
stop)
# Stop the daemon.
if [ -e $PIDFILE ]; then
status_of_proc -p $PIDFILE $DAEMON "Stoppping the $NAME process" && status="0" || status="$?"
if [ "$status" = 0 ]; then
start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
/bin/rm -rf $PIDFILE
fi
else
log_daemon_msg "$NAME process is not running"
log_end_msg 0
fi
;;
restart)
# Restart the daemon.
$0 stop && sleep 2 && $0 start
;;
status)
# Check the status of the process.
if [ -e $PIDFILE ]; then
status_of_proc -p $PIDFILE $DAEMON "$NAME process" && exit 0 || exit $?
else
log_daemon_msg "$NAME Process is not running"
log_end_msg 0
fi
;;
reload)
# Reload the process. Basically sending some signal to a daemon to reload
# it configurations.
if [ -e $PIDFILE ]; then
start-stop-daemon --stop --signal USR1 --quiet --pidfile $PIDFILE --name $NAME
log_success_msg "$NAME process reloaded successfully"
else
log_failure_msg "$PIDFILE does not exists"
fi
;;
*)
# For invalid arguments, print the usage message.
echo "Usage: $0 {start|stop|restart|reload|status}"
exit 2
;;
esac

The problem is that when the init script run, there is not graphical interface available. Instead of using a init script, try configuring your application to run on X startup.
First add the command line to start your GUI app to ~/.xinitrc
# ~/.xinitrc
exec python3 /home/pi/Desktop/GUI/GUI.py
And then start the X server
startx

how to add getopt options in a bash script

I have written a bash script that consists of multiple Unix commands and Python scripts. The goal is to make a pipeline for detecting long non coding RNA from a certain input. Ultimately I would like to turn this into an 'app' and host it on some bioinformatics website. One problem I am facing is using getopt tools in bash. I couldn't find a good tutorial that I understand clearly. In addition any other comments related to the code is appreciated.
#!/bin/bash
if [ "$1" == "-h" ]
then
echo "Usage: sh $0 cuffcompare_output reference_genome blast_file"
exit
else
wget https://github.com/TransDecoder/TransDecoder/archive/2.0.1.tar.gz && tar xvf 2.0.1 && rm -r 2.0.1
makeblastdb -in $3 -dbtype nucl -out $3.blast.out
grep '"u"' $1 | \
gffread -w transcripts_u.fa -g $2 - && \
python2.7 get_gene_length_filter.py transcripts_u.fa transcripts_u_filter.fa && \
TransDecoder-2.0.1/TransDecoder.LongOrfs -t transcripts_u_filter.fa
sed 's/ .*//' transcripts_u_filter.fa | grep ">" | sed 's/>//' > transcripts_u_filter.fa.genes
cd transcripts_u_filter.fa.transdecoder_dir
sed 's/|.*//' longest_orfs.cds | grep ">" | sed 's/>//' | uniq > longest_orfs.cds.genes
grep -v -f longest_orfs.cds.genes ../transcripts_u_filter.fa.genes > longest_orfs.cds.genes.not.genes
sed 's/^/>/' longest_orfs.cds.genes.not.genes > temp && mv temp longest_orfs.cds.genes.not.genes
python ../extract_sequences.py longest_orfs.cds.genes.not.genes ../transcripts_u_filter.fa longest_orfs.cds.genes.not.genes.fa
blastn -query longest_orfs.cds.genes.not.genes.fa -db ../$3.blast.out -out longest_orfs.cds.genes.not.genes.fa.blast.out -outfmt 6
python ../filter_sequences.py longest_orfs.cds.genes.not.genes.fa.blast.out longest_orfs.cds.genes.not.genes.fa.blast.out.filtered
grep -v -f longest_orfs.cds.genes.not.genes.fa.blast.out.filtered longest_orfs.cds.genes.not.genes.fa > lincRNA_final.fa
fi
Here is how I run it:
sh test.sh cuffcompare_out_annot_no_annot.combined.gtf /mydata/db/Brapa_sequence_v1.2.fa TE_RNA_transcripts.fa

If you wanted the call to be :
test -c cuffcompare_output -r reference_genome -b blast_file
You would have something like :
#!/bin/bash
while getopts ":b:c:hr:" opt; do
case $opt in
b)
blastfile=$OPTARG
;;
c)
comparefilefile=$OPTARG
;;
h)
echo "USAGE : test -c cuffcompare_output -r reference_genome -b blast_file"
;;
r)
referencegenome=$OPTARG
;;
\?)
echo "Invalid option: -$OPTARG" >&2
exit 1
;;
:)
echo "Option -$OPTARG requires an argument." >&2
exit 1
;;
esac
done
In the string ":b:c:hr:",
- the first ":" tells getopts that we'll handle any errors,
- subsequent letters are the allowable flags. If the letter is followed by a ':', then getopts will expect that flag to take an argument, and supply that argument as $OPTARG

shell start / stop for python script

I have a simple python script i need to start and stop and i need to use a start.sh and stop.sh script to do it.
I have start.sh:
#!/bin/sh
script='/path/to/my/script.py'
echo 'starting $script with nohup'
nohup /usr/bin/python $script &
and stop.sh
#!/bin/sh
PID=$(ps aux | grep "/path/to/my/script.py" | awk '{print $2}')
echo "killing $PID"
kill -15 $PID
I'm mainly concerned with the stop.sh script. I think that's an appropriate way to find the pid but i wouldn't bet much on it. start.sh successfully starts it. when i run stop.sh, i can no longer find the process by "ps aux | grep 'myscript.py'" but the console outputs:
killing 25052
25058
./stop.sh: 5: kill: No such process
so it seems like it works AND gives an error of sorts with "No such process".
Is this actually an error? Am I approaching this in a sane way? Are there other things I should be paying attention to?
EDIT - I actually ended up with something like this:
start.sh
#!/bin/bash
ENVT=$1
COMPONENTS=$2
TARGETS=("/home/user/project/modules/script1.py" "/home/user/project/modules/script2.py")
for target in "${TARGETS[#]}"
do
PID=$(ps aux | grep -v grep | grep $target | awk '{print $2}')
echo $PID
if [[ -z "$PID" ]]
then
echo "starting $target with nohup for env't: $ENVT"
nohup python $target $ENVT $COMPONENTS &
fi
done
stop.sh
#!/bin/bash
ENVT=$1
TARGETS=("/home/user/project/modules/script1.py" "/home/user/project/modules/script2.py")
for target in "${TARGETS[#]}"
do
pkill -f $target
echo "killing process $target"
done

It is because ps aux |grep SOMETHING also finds the grep SOMETHING process, because SOMETHING matches. After the execution the grep is finished, so it cannot find it.
Add a line: ps aux | grep -v grep | grep YOURSCRIPT
Where -v means exclude. More in man grep.

The "correct" approach would probably be to have your script write its pid to a file in /var/run, and clear it out when you kill the script. If changing the script is not an option, have a look at start-stop-daemon.
If you want to continue with the grep-like approach, have a look at proctools. They're built in on most GNU/Linux machines and readily available on BSD including OS X:
pkill -f /path/to/my/script.py

init-type scripts are useful for this. This is very similar to one I use. You store the pid in a file, and when you want to check if it's running, look into the /proc filesystem.
#!/bin/bash
script_home=/path/to/my
script_name="$script_home/script.py"
pid_file="$script_home/script.pid"
# returns a boolean and optionally the pid
running() {
local status=false
if [[ -f $pid_file ]]; then
# check to see it corresponds to the running script
local pid=$(< "$pid_file")
local cmdline=/proc/$pid/cmdline
# you may need to adjust the regexp in the grep command
if [[ -f $cmdline ]] && grep -q "$script_name" $cmdline; then
status="true $pid"
fi
fi
echo $status
}
start() {
echo "starting $script_name"
nohup "$script_name" &
echo $! > "$pid_file"
}
stop() {
# `kill -0 pid` returns successfully if the pid is running, but does not
# actually kill it.
kill -0 $1 && kill $1
rm "$pid_file"
echo "stopped"
}
read running pid < <(running)
case $1 in
start)
if $running; then
echo "$script_name is already running with PID $pid"
else
start
fi
;;
stop)
stop $pid
;;
restart)
stop $pid
start
;;
status)
if $running; then
echo "$script_name is running with PID $pid"
else
echo "$script_name is not running"
fi
;;
*) echo "usage: $0 <start|stop|restart|status>"
exit
;;
esac

ps aux | grep "/path/to/my/script.py"
will return both the pid for the instance of script.py and also for this instance of grep. That'll probably be why you're getting a no such process: by the time you get around to killing the grep, it's already dead.

I don't have a unix box on at the moment, so i can't test this, but it should be fairly simple to get the idea.
start.sh:
if [ -e ./temp ]
then
pid=`cat temp`
echo "Process already exists; $pid"
else
script='/path/to/my/script.py'
echo 'starting $script with nohup'
nohup /usr/bin/python $script &
echo $! > temp
fi
stop.sh:
if [ -e ./temp ]
then
pid=`cat temp`
echo "killing $pid"
kill -15 $PID
rm temp
else
echo "Process not started"
fi
Try this out.