We Keep Coding

systemd service failed to start bash script

I am running a bash script as systemd service but it is giving me this error
Failed at step EXEC spawning /home/pipeline/entity-extraction/start_consumer.sh: Permission denied
Feb 8 11:59:58 irum systemd[1]: ee-consumer.service: main process exited, code=exited, status=203/EXEC
Feb 8 11:59:58 irum systemd[1]: Unit ee-consumer.service entered failed state.
My bash scrip is running 2 Python scripts and it runs fine when I run it from terminal as
sudo bash start_consumer.sh
start_consumer.sh
while true
do
echo "starting FIRST Consumer.py : $(date +"%T")"
python3 /home/irum/Desktop/Marketsyc/Consumer.py &
pid=$!
echo "pid:$pid"
sleep 60
echo "starting SECOND Consumer.py : $(date +"%T")"
python3 /home/irum/Desktop/Marketsyc/Consumer.py &
new_pid=$!
echo "new_pid:$new_pid"
# Here I want to kill FIRST Consumer.py
echo "killing first consumer"
kill "$pid"
sleep 60
# Here I want to kill SECOND Consumer.py
echo "killing second consumer"
kill "$new_pid"
done
code of my systemd service ee-consumer.service
[Unit]
Description=Entity extraction - consumer
After=default.target
[Service]
Type=simple
Restart=always
User=pipeline
ExecStart=/home/pipeline/entity-extraction/start_consumer.sh
how can I resolve this issue ?

You have to set the shebang line and permission to the script, for systemd to execute.
Add #!/bin/bash to the start of the bash script. And do the following,
chmod 755 /home/pipeline/entity-extraction/start_consumer.sh

how to run gunicorn with django as non-root

I have a django application and I use gunicorn to run it. My script to start gunicorn looks like this:
django_path=/path/to/your/manage.py
settingsfile=my_name
workers=2
cd $django_path
exec gunicorn --env DJANGO_SETTINGS_MODULE=app.$settingsfile app.wsgi --workers=$workers &
this works when I execute it. However, when I look at my database in my projectfolder (cd /path/to/your/manage.py && ll) I get this:
-rw-r--r-- 1 root root 55K Dec 2 13:33 db.sqlite3
Which means I need root permisson to do anyhting on the databse (for example do a createuser). So I looked around on Stackoverflow and tried a couple of things:
I had the whole script at the top of /etc/init.d/rc.local
Then I put the script as a script file gunicorn_script.sh put in /etc/init.d, did a /usr/sbin/update-rc.d -f gunicorn_script.sh defaults
Lastly, I tried to put this command at the top of the rc.local file: su debian -c '/etc/init.d/gunicorn_script.sh start' to execute the gunicorn_script as a debian user
All of them started my app but the problem with the database remains (only root rights).
So how do I run that script as a non root user?

I have a script in my project's folder which I use to run gunicorn. Here is a header:
#!/bin/bash
CUR_DIR=$(dirname $(readlink -f $0))
WORK_DIR=$CUR_DIR
USER=myusername
PYTHON=/usr/bin/python3
GUNICORN=/usr/local/bin/gunicorn
sudo -u $USER sh -c "cd $WORK_DIR; $PYTHON -W ignore $GUNICORN -c $WORK_DIR/config/gunicorn/gunicorn.conf.py --chdir $WORK_DIR myappname.wsgi:application
Updated:
Put the code below to the file /etc/init.d/myservice, make the root owner and give +x permissions for the owner.
#!/bin/bash
#chkconfig: 345 95 50
#description: Starts myservice
if [ -z "$1" ]; then
echo "`basename $0` {start|stop}"
exit
fi
case "$1" in
start)
sh /path/to/run_script.sh start &
;;
stop)
sh /path/to/run_script.sh stop
;;
esac
Now you can use sudo service myservice start
I am sorry, I am not familiar with systemd yet, but with it it can be even easier.

Ok, so I found out that db.sqlite3 will be create in django through the makemigrations and migrate commands which I ran from root.
Hence, the problems with the permissions. I switched to debian and ran the commands from there et voila:
-rw-r--r-- 1 debian debian 55K Dec 2 13:33 db.sqlite3

Raspbian python script at boot keeps failing

Good Day All,
I am trying to run a python script at boot using Rasbian on a Raspberry PI.
I have added the following into crontabt -e:
#reboot /usr/bin/python /home/pi/Midori_Monitor/Main_Midori.py
That did not work.
I then tried adding it to rc.local:
# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
printf "My IP address is %s\n" "$_IP"
fi
/usr/bin/python /home/pi/Midori_Monitor/Main_Midori.py
exit 0
That did not seem to work either. I tried adding the user and group (pi:pi) to both and that did not help.
I don't see a log for the crontab however syslog offers:
Feb 21 13:44:57 raspberrypi /USR/SBIN/CRON[1991]: (pi) CMD (pi:pi /usr/bin/python /home/pi/Midori_Monitor/Main_Midori.py)
Feb 21 13:48:22 raspberrypi /USR/SBIN/CRON[1964]: (pi) CMD (pi:pi /usr/bin/python /home/pi/Midori_Monitor/Main_Midori.py)
Feb 21 13:48:24 raspberrypi /USR/SBIN/CRON[2008]: (pi) CMD (pi:pi /usr/bin/python /home/pi/Midori_Monitor/Main_Midori.py)
Feb 24 07:22:18 raspberrypi /USR/SBIN/CRON[1983]: (pi) CMD (/usr/bin/python /home/pi/Midori_Monitor/Main_Midori.py & )
Feb 24 07:28:13 raspberrypi /USR/SBIN/CRON[1983]: (pi) CMD (/usr/bin/python /home/pi/Midori_Monitor/Main_Midori.py & )
Feb 24 07:34:04 raspberrypi /USR/SBIN/CRON[1993]: (pi) CMD (/usr/bin/python /home/pi/Midori_Monitor/Main_Midori.py )
Any suggestions?

SOLVED - I think the core being here that my script is required to run in a GUI environment.
I found this solution here: https://raspberrypi.stackexchange.com/questions/8734/execute-script-on-start-up
If you want a script to run when you boot into the LXDE environment, you could take a look at this Raspberry Pi forum post:
Navigate to:
etc/xdg/lxsession/LXDE
Open the autostart file in that folder:
sudo nano autostart
Add
#python /path/to/script on a new line.
If you want to run something like a python script, put something like #python mypython.py on a new line. Running a script file would be #./superscript, but for some reason the script runs in an infinite loop (perhaps this will stop that).
Save and exit:
Ctrl+X, Y, Enter
Restart your Raspberry Pi into the LXDE environment.

You added the code after exit 0, so it's never executed!
Use this rc.local:
# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
printf "My IP address is %s\n" "$_IP"
fi
/usr/bin/python /home/pi/Midori_Monitor/Main_Midori.py
exit 0

BASH - crontab is always running except the python script

When i run this script from shell /var/tmp/server_always_alive.sh manually has no problem works. But when i let it run with crontab it never running even all the logics are correct.
How can i make the python server.py run via this crontab?
sun#sun-Inspiron-One-2320:~$ uname -a
Linux sun-Inspiron-One-2320 3.5.0-17-generic #28-Ubuntu SMP Tue Oct 9 19:31:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
/var/tmp/server_always_alive.sh:
#!/bin/bash
echo "test 1"
echo "test 2"
# 58888 TCP port is server port of server.py, if its not running server.py has to execute auto
main=$(export DISPLAY=:0.0 && lsof -i tcp:58888 | grep LISTEN | awk '{print $2}')
if [ -z "$main" ]; then
export DISPLAY=:0.0 && python /var/tmp/python/server.py &
sleep 2
break
fi
echo "test 3"
echo "all runs except python server.py"
crontab :
* * * * * /var/tmp/server_always_alive.sh &

DISPLAY=:0.0 indicates your python 'server' is connecting to an X server. Why?
Cron won't have the necessary X "cookie". , and almost certainly won't be running as the same user as the X server.
edit: I'll take you at your word that you're running as the correct user.
edit: If you really need to run a graphical program from cron, try
xhost +si:localuser:`whoami`

For reference to alternative workout.
Step 1: Put python script on this following script= save it in /var/tmp/main.sh
A) NON GUI BASED
#!/bin/sh
script='/my/python/script/is/here/ok.py'
/usr/bin/python $script &
B) GUI (GTK/TK etc)
#!/bin/sh
script='/my/python/script/is/here/ok.py'
export DISPLAY=:0.0 && /usr/bin/python $script &
Step 2: now make a file in /etc/init.d/scriptname_what_ever_feed_i_name with following (copy paste)
#! /bin/sh
PATH=/bin:/usr/bin:/sbin:/usr/sbin
DAEMON=/home/CHANGE _ ____ HERE ______ to the Step 1 file name
PIDFILE=/var/run/scriptname.pid
test -x $DAEMON || exit 0
. /lib/lsb/init-functions
case "$1" in
start)
log_daemon_msg "Starting feedparser"
start_daemon -p $PIDFILE $DAEMON
log_end_msg $?
;;
stop)
log_daemon_msg "Stopping feedparser"
killproc -p $PIDFILE $DAEMON
PID=`ps x |grep feed | head -1 | awk '{print $1}'`
kill -9 $PID
log_end_msg $?
;;
force-reload|restart)
$0 stop
$0 start
;;
status)
status_of_proc -p $PIDFILE $DAEMON atd && exit 0 || exit $?
;;
*)
echo "Usage: /etc/init.d/atd {start|stop|restart|force-reload|status}"
exit 1
;;
esac
exit 0
Step 3: make it executeable chmod +x /etc/init.d/scriptname_what_ever_feed_i_name and chmod -R 777 /etc/init.d/scriptname_what_ever_feed_i_name so that as any user you can execute it without sudo.
Step 4: for example:
/etc/init.d/scriptname_what_ever_feed_i_name restart
or
* * * * * /etc/init.d/scriptname_what_ever_feed_i_name restart
WORKING - and much better/safer.
ps aux | grep python
root 5026 0.5 0.3 170464 19336 pts/0 S 07:40 0:00 /usr/bin/python /var/tmp/python/server.py
Now you can start and stop your python script using the command /etc/init.d/scriptname start or stop manually or cron etc

Run python script as daemon at boot time (Ubuntu)

I've created small web server using werkzeug and I'm able to run it in usual python way with python my_server.py. Pages load, everything works fine. Now I want to start it when my pc boots. What's the easiest way to do that? I've been struggling with upstart but it doesn't seem to "live in a background" cuz after I execute start my_server I immediately receive kernel: [ 8799.793942] init: my_server main process (7274) terminated with status 1
my_server.py:
...
if __name__ == '__main__':
from werkzeug.serving import run_simple
app = create_app()
run_simple('0.0.0.0', 4000, app)
upstart configuration file my_server.conf:
description "My service"
author "Some Dude <blah#foo.com>"
start on runlevel [2345]
stop on runlevel [016]
exec /path/to/my_server.py
start on startup
Any Ideas how to make it work? Or any other better way to daemonize the script?
Update:
I believe the problem lies within my_server.py. It doesn't seem to initiate the webserver (method run_simple()) in the first place. What steps should be taken to make .py file be run by task handler such as upstart?
Place shebang as first line #!/usr/bin/env python
Allow execution permissions chmod 755
Start the daemon with superuser rights (to be absolutely sure no permission restrictions prevents it from starting)
Make sure all python libraries are there!
Something else?
Solved:
The problem was with missing python dependencies. When starting the script through task manager (e.g. upstart or start-stop-daemon) no errors are thrown. Need to be absolutely sure that pythonpath contains everything you need.

In addition to gg.kaspersky method, you could also turn your script into a "service", so that you can start or stop it using:
$ sudo service myserver start
* Starting system myserver.py Daemon [ OK ]
$ sudo service myserver status
* /path/to/myserver.py is running
$ sudo service myserver stop
* Stopping system myserver.py Daemon [ OK ]
and define it as a startup service using:
$ sudo update-rc.d myserver defaults
To do this, you must create this file and save it in /etc/init.d/.
#!/bin/sh -e
DAEMON="/path/to/myserver.py"
DAEMONUSER="myuser"
DAEMON_NAME="myserver.py"
PATH="/sbin:/bin:/usr/sbin:/usr/bin"
test -x $DAEMON || exit 0
. /lib/lsb/init-functions
d_start () {
log_daemon_msg "Starting system $DAEMON_NAME Daemon"
start-stop-daemon --background --name $DAEMON_NAME --start --user $DAEMONUSER --exec $DAEMON
log_end_msg $?
}
d_stop () {
log_daemon_msg "Stopping system $DAEMON_NAME Daemon"
start-stop-daemon --name $DAEMON_NAME --stop --retry 5 --name $DAEMON_NAME
log_end_msg $?
}
case "$1" in
start|stop)
d_${1}
;;
restart|reload|force-reload)
d_stop
d_start
;;
force-stop)
d_stop
killall -q $DAEMON_NAME || true
sleep 2
killall -q -9 $DAEMON_NAME || true
;;
status)
status_of_proc "$DAEMON_NAME" "$DAEMON" "system-wide $DAEMON_NAME" && exit 0 || exit $?
;;
*)
echo "Usage: /etc/init.d/$DAEMON_NAME {start|stop|force-stop|restart|reload|force-reload|status}"
exit 1
;;
esac
exit 0
In this example, I assume you have a shebang like #!/usr/bin/python at the head of your python file, so that you can execute it directly.
Last but not least, do not forget to give execution rights to your python server and to the service script :
$ sudo chmod 755 /etc/init.d/myserver
$ sudo chmod 755 /path/to/mserver.py
Here's the page where I learned this originally (french).
Cheers.

One simple way to do is using crontab:
$ crontab -e
A crontab file will appear for editing, write the line at the end:
#reboot python myserver.py
and quit. Now, after each reboot, the cron daemon will run your myserver python script.

If you have supervisor service that starts at boot, write a supervisor service is much, much simpler.
You can even set autorestart if your program fails.