We Keep Coding

Batch process !ERRORLEVEL! is not working when running multiple commands in parallel

I am trying to create a batch script which runs 2 python scripts in parallel and return an errorcode if in either one (or both) of the scripts is an error. In case both script have run correctly the next python script should be ran.
When I use the script below the processes are ran in parallel but the !errorlevel! is always 0, even if there is an error in both of the scripts.
In case I run only one of the python scripts with the start /wait command or if I run the 2 scripts sequentially using && instead of |, the !errorlevel! is returned correctly.
How can these scripts be run in parallel while still output the correct errorlevel?
setlocal enableDelayedExpansion
start /wait python apirequest_0001.py | start /wait python apirequest_0002.py
echo !errorlevel!
if !errorlevel! neq 0 (goto end )
start /wait python join_data.py

A couple points here...
The start command is used to run a process in parallel, but start /wait switch cancel such a feature and waits for the process to end. In this way, using start /wait is the same as no use start command at all...
However, in your example both commands are connected via a pipeline, so they are effectively executed in parallel (because the pipe, not the start command).
The simplest way to get the ERRORLEVEL value of two (or more) parallel processes is via an auxiliary file:
del error.aux 2> NUL
(python apirequest_0001.py & if errorlevel 1 echo X > error.aux) | (python apirequest_0002.py & if errorlevel 1 echo X > error.aux)
if exist error.aux goto end
python join_data.py
Although a "cleaner" way to do this is via this method
del error.aux 2> NUL
(
start "" "python apirequest_0001.py & if errorlevel 1 echo X > error.aux"
start "" "python apirequest_0002.py & if errorlevel 1 echo X > error.aux"
) | pause
if exist error.aux goto end
python join_data.py
Note: In this solution there is a possibility that both commands ends at the same time and try to create the file at "same time". However, this is not a problem in the case, because anyway the file will be created...

Why does my daemon-like Python script run multiple times?

I start a Python script from a bash script in /etc/init.d using sudo /etc/init.d/fanservice.sh start (or stop)
It starts and stops as expected, and starts on boot. The Python script works well.
However, in top or htop I see that the script is running multiple times.
What might be causing this, and how do I stop it?
Other than looking around for other triggers (I've checked there are no old references to the script in crontab) I'm not sure where else to look.
This is /etc/init.d/fanservice.sh
I run the Python script as root to allow access to GPIO.
#!/bin/sh
### BEGIN INIT INFO
# Provides: myservice
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Put a short description of the service here
# Description: Put a long description of the service here
### END INIT INFO
# Change the next 3 lines to suit where you install your script and what you want to call it
DIR=/home/pi
DAEMON=$DIR/fanservice.py
DAEMON_NAME=fanservice
# Add any command line options for your daemon here
DAEMON_OPTS=""
# This next line determines what user the script runs as.
# Root generally not recommended but necessary if you are using the Raspberry Pi GPIO from Python.
DAEMON_USER=root
# The process ID of the script when it runs is stored here:
PIDFILE=/var/run/$DAEMON_NAME.pid
. /lib/lsb/init-functions
do_start () {
log_daemon_msg "Starting system $DAEMON_NAME daemon"
start-stop-daemon --start --background --pidfile $PIDFILE --make-pidfile --user $DAEMON_USER --chuid $DAEMON_USER --startas $DAEMON -- $DAEMON_OPTS
log_end_msg $?
}
do_stop () {
log_daemon_msg "Stopping system $DAEMON_NAME daemon"
start-stop-daemon --stop --pidfile $PIDFILE --retry 10
log_end_msg $?
}
case "$1" in
start|stop)
do_${1}
;;
restart|reload|force-reload)
do_stop
do_start
;;
status)
status_of_proc "$DAEMON_NAME" "$DAEMON" && exit 0 || exit $?
;;
*)
echo "Usage: /etc/init.d/$DAEMON_NAME {start|stop|restart|status}"
exit 1
;;
esac
exit 0
The Python script uses wiringPi, GPIO, tach and Paho.mqtt to read a temperature, set a fan speed, read the speed from the tach then send an MQTT message to a broker. All that is contained in a while True: loop with a time.sleep(58) at the end, so what I'm expecting is for the 'service' to start and loop every 58 seconds.
It does this, but every minute or so a new instance appears in top.
If I examine /var/run/fanservice.pid where the script PID is stored, I see it has the PID of the first instance.
For example, after 15 or 20 minutes htop will show the script running with a PID that matches that stored in PID file, and that process will have a time of XX minutes and seconds since boot. Then there'll be XX duplicate processes with their own PIDs, all with a time of 00:00.00, and the number of them will correspond (I think) with the number of minutes since boot.
Any tips very welcome. The init.d script is one I found on the internet, can't remember where - but I think I've probably broken that somehow! That said, I moved to running these tasks in a timer loop rather than from cron because the Python script wasn't exiting after being called by cron. I also have other scripts running in this way that need a faster loop than the minimum 1 minute from cron, so I'd like to learn how to make this work properly.
Thank you!

How can I start a bash script 20 seconds after startup?

I got a bash script that stops a program, mounts the pi and starts the program again. I would like to start it on startup, but after the program itself started. So my idea was to simply wait some time (20 or 30 seconds) and start the script then (task.sh). Any idea how I can do that? Or any other idea how to solve this? (let the script wait for the program to start won't work i guess, cause then the script would restart after the program is restarted, right?)
Thanks and Greetings,
Elias

Thanks for all the help, but I solved it myself following this tutorial:
create a new file in /etc/init.d/, I'll call it example in this.
So:
sudo nano /etc/init.d/example
This will be a file that will be executed after raspberry pi startup.
The code for this file is the following:
#!/bin/sh
### BEGIN INIT INFO
# Provides: Für welches Programm ist das Script?
# Required-Start:
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Kurze Beschreibung
# Description: Längere Beschreibung
### END INIT INFO
# Actions
case "$1" in
start)
# START
;;
stop)
# STOP
;;
restart)
# RESTART
;;
esac
exit 0
Because I wanted this script to start /home/pi/task.sh at startup and reboot, I simply put /home/pi/task.sh before the ;; after # START and # RESTART
If you've done that, save it and exit it. Then type
sudo chmod +x /etc/init.d/example
to make the script executable. The last step is to define the runlevels:
sudo update-rc.d example defaults
After that you can reboot and see if it works.
I hope that made it clear for everyone :)
(And just for me: https://jankarres.de/2014/07/raspberry-pi-autostart-von-programmen-einrichten/ :D)

Assuming that you are using a Debian / Debian derivative distro (Ubuntu / Mint / Etc) here's how to achieve the result you look for.
Create your script in the path you've suggested using any text editor (here I use the simple nano):
nano /home/pi/task.sh
Paste into your task.sh:
sleep 40
/home/pi/pi_video_looper/disable.sh
mount -a
/home/pi/pi_video_looper/install.sh
Make the script executable:
chmod +x /home/pi/task.sh
Make sure the script works running it:
/home/pi/task.sh
Once you're sure that the script works fine edit your rc.local:
sudo nano /etc/rc.local
A key concept here is that whatever you put in rc.local will be executed with root permissions.
For this reason there is no need to use sudo.
Add before exit 0 the following:
/home/pi/task.sh
Reboot and test

ubuntu ec2 - run python script at startup with arguments

I have a python script i'd like to start on startup on an ubuntu ec2 instance but im running into troubles.
The script runs in a loop and takes care or exiting when its ready so i shouldn't need to start or stop it after its running.
I've read and tried a lot of approaches with various degrees of success and honestly im confused about whats the best approach. I've tried putting a shell script that starts the python script in /etc/init.d, making it executable and doing update-rc.d to try to get it to run but its failed at every stage.
here's the contents of the script ive tried:
#!/bin/bash
cd ~/Dropbox/Render\ Farm\ 1/appleseed/bin
while :
do
python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/
done
i then did
sudo chmod +x /etc/init.d/script_name
sudo sudo update-rc.d /etc/init.d/script_name defaults
This doesn't seem to run on startup and i cant see why, if i run the command manually it works as expected.
I also tried adding a line to rc.local to start the script but that doesn't seem to work either
Can anybody share what they have found is the simplest way to run a python script in the background with arguments on startup of an ec2 instance.
UPDATE: ----------------------
I've since moved this code to a file called /home/ubuntu/bin/watch_folder_start
#!/bin/bash
cd /home/ubuntu/Dropbox/Render\ Farm\ 1/appleseed/bin
while :
do
python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/
done
and changed my rc.local file to this:
nohup /home/ubuntu/bin/watch_folder_start &
exit 0
Which works when i manually run rc.local but wont fire on startup, i did chmod +x rc.local but that didn't change anything,

Your /etc/init.d/script_name is missing the plumbing that update-rc.d and so on use, and won't properly handle stop, start, and other init-variety commands, so...
For initial experimentation, take advantage of the /etc/init.d/rc.local script (which should be linked to by default from /etc/rc2/S99rc.local). The gets you out of having to worry about the init.d conventions and just add things to /etc/rc.local before the exit 0 at its end.
Additionally, that ~ isn't going to be defined, you'll need to use a full pathname - and furthermore the script will run as root. We'll address how to avoid this if desired in a bit. In any of these, you'll need to replace "whoeveryouare" with something more useful. Also be warned that you may need to prefix the python command with a su command and some arguments to get the process to run with the user id you might need.
You might try (in /etc/rc.local):
( if cd '/home/whoeveryouare/Dropbox/Render Farm 1/appleseed/bin' ; then
while : ; do
# This loop should respawn watchfolder18.py if it dies, but
# ideally one should fix watchfolder18.py and remove this loop.
python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/
done
else
echo warning: could not find watchfolder 1>&2
fi
) &
You could also put all that in a script and just call it from /etc/rc.local.
The first pass is roughly what you had, but if we assume that watchfolder18.py will arrange to avoid dying we can cut it down to:
( cd '/home/whoeveryouare/Dropbox/Render Farm 1/appleseed/bin' \
&& exec python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/ ) &
These aren't all that pretty, but it should let you get your daemon sorted out so you can debug it and so on, then come back to making a proper /etc/init.d or /etc/init script later. Something like this might work in /etc/init/watchfolder.conf, but I'm not yet facile enough to claim this is anything other than a rough stab at it:
# watchfolder - spawner for watchfolder18.py
description "watchfolder program"
start on runlevel [2345]
stop on runlevel [!2345]
script
if cd '/home/whoeveryouare/Dropbox/Render Farm 1/appleseed/bin' ; then
exec python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/0
fi
end script

I found that the best solution in the end was to use 'upstart' and create a file in etc/init called myfile.conf that contained the following
description "watch folder service"
author "Jonathan Topf"
start on startup
stop on shutdown
# Automatically Respawn:
respawn
respawn limit 99 5
script
HOST=`hostname`
chdir /home/ubuntu/Dropbox/Render\ Farm\ 1/appleseed/bin
exec /usr/bin/python ./watchfolder.py -t ./appleseed.cli -u $HOST ../../data/ >> /home/ubuntu/bin/ec2_server.log 2>&1
echo "watch_folder started"
end script
More info on using the upstart system here
http://upstart.ubuntu.com/
https://help.ubuntu.com/community/UbuntuBootupHowto
http://blog.joshsoftware.com/2012/02/14/upstart-scripts-in-ubuntu/

Python Service / Daemon

I'm writing a Python script that I need to have start at boot (it should continue running forever once it is started). I would like to be able to manage the service via running a command like:
sudo service my-service (and either start, stop, restart, etc.)
I have been reading a lot into it and would really like to finish my project. I saw this but would need help implementing it if it were to work Linux Start-up Script. I'm not even sure where to start, I don't really know how to program in bash but I'm open to all solutions. Thank you ahead of time, and I appreciate all responses!

Take a look at zdaemon. It provides a simple way to daemonize a Python process.
Then you can write an init.d script for it - based on your OS.
Or you can use a tool like Upstart, supervisord to control the daemon.
My init.d script (on centos 5.8) looks like this:
. /etc/rc.d/init.d/functions
. /etc/sysconfig/network
APP_PATH=/path/to/your/app
PYTHON=/usr/local/bin/python
USER=user
start() {
cd $APP_PATH
zdaemon -C app.zdconf start
}
stop() {
cd $APP_PATH
zdaemon -C app.zdconf stop
}
check_status() {
cd $APP_PATH
zdaemon -C app.zdconf status
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
check_status
;;
restart)
stop
start
;;
*)
esac
exit 0
Where app.zdconf is the zdaemon configuration file.