I have a Python script which process data off of a HTTP data stream and I need this script to in theory be running at all times and forever unless I kill it off manually to update it and run it again.
I was wondering what the best practice to do this was on a Unix (Ubuntu in particular) so that even if Terminal is closed etc the script continues to run in the background unless the process or server are shut down?
Thanks
From your question you are implying that you are going to start the script from your terminal and not by any of Linux's startup script management methods like systemd or upstart or init.d scripts.
If you intend to start your script from terminal, and you want it to continue to run after you close your terminal, you need to do two things
1. Make it run in the background by appending '&' after your script.
2. When you close the terminal, the shell associated to it sends HUP signal to all the processes before it dying. To ignore the HUP signal and continue to run in the background you need to start your script with 'nohup'.
tl;dr
Run your script this way:
$ nohup python mypythonscript.py &
Adding your script to rc.local would work, but 'best practice' in my opinion would be to use Upstart. See this post:
Daemon vs Upstart for python script
It's an infinite loop. You can launch the script at startup and it will run forever until you kill the process itself or shut the computer down.
#!/usr/bin/python
# -*- coding: utf-8 -*-
import time
while True:
print 'Hello'
time.sleep(2) # 2 second delay
I am not sure how "best practice" this is but you could do:
Add the program to:
/etc/rc.d/rc.local
This will have the program run at startup.
If you add an '&' to the end of the line it will be run in the background.
If you dont want to run the program manually (not at startup) you could switch to another tty by pressing ctrl + alt + f1, (this opens tty1 and will work with f1 - f6) then run the command. This terminal you do not have to have open in a window so you dont have to worry about it getting closed. To return to the desktop use ctrl + alt + f7.
Related
I've created a script for my school project that works with data. I'm quite new to working remotely on a server, so this might seem like a dumb question, but how do I execute my script named
stats.py
so that it continues executing even after I log off PuTTy? The script file is located on the server. It has to work with a lot of data, so I don't want to just try something and then few days later find out that it has exited right after I logged off.
Thank you for any help!
There are many ways you can run a python program after you disconnect from an SSH session.
1) Tmux or Screen
Tmux is a "terminal multiplexer" which enables a number of terminals to be accessed by a single one.
You start by sshing as you do, run it by typing tmux and executing it. Once you are done you can disconnect from putty and when you login back you can relog to the tmux session you left
Screen also does that you just type screen instead of tmux
2) nohup
"nohup is a POSIX command to ignore the HUP signal. The HUP signal is, by convention, the way a terminal warns dependent processes of logout."
You can run it by typing nohup <pythonprogram> &
I have read that upstart is obsolete in favor of systemd for raspberry pi 3.
My question is how do I run a python script :
a) forever unless I manually kill it
b) can restart if it dies due to some exception or stop running automatically without any human intervention
my python script itself is already using modules like schedule and while True loops to keep running certain jobs every few seconds.
I am just worried that it will die/stop (which it did) after some indeterminate amount of time.
If it stops, all I want is for it to restart.
Currently, I run the script by double clicking it to open in Python IDLE (2.7) and then run module.
What is the best way to run and open a python script and let it run continuously non-stop and then have it auto restart when it dies / stops for whatever reason?
See this picture where it suddenly stops by itself at 5 plus am
I think you should take a look at Python Supervisor. Supervisor will manage the restart in the event of a crash or even machine re-starts.
http://supervisord.org/
An easier method might be the handle the failure within your script. If it is failing due to some exception, wrap the offending code in a try:except block and handle it gracefully within the script.
That said, this post has the information you need to use systemd to execute a BASH script:
https://unix.stackexchange.com/questions/47695/how-to-write-startup-script-for-systemd
Within your script, you can easily run a python script and catch its return value (when it returns failure in your case) and react appropriately.
Something like this:
#!/bin/bash
python ~/path/to/my/script/myScript.py
if [ $? -ne 0 ] ; then #handle the failure here.
If that won't work either, you can create a script whose sole job is to call the other script and handle its failures, and use systemd to call that script.
I've a script to run on boot and I'd like to use the keyboard to interact with the script. I've successful set this up to run in crontab; however, the script runs in the background and I can't use the keyboard to interact with the script. Here's a simplified example of the script:
def write_to_txt(item_to_write):
with open("my_txt_file.txt", "a") as myfile:
myfile.write('\n'+str(item_to_write))
while True:
keys_to_enter = raw_input()
write_to_txt(keys_to_enter)
Please could someone point me in the right direction?
I found out how to run the script on boot and allow the keyboard to interact with the program. To the ~/.bashrc file, I appended:
sudo python /home/pi/example.py
If I understand correctly you want your program to attach its stdin to tty1? I.e. the terminal which you see on screen if you have a display hooked up - this is where by default keyboard input would end up if X windows is not installed or the tty is not switched with Ctrl+Alt+Fx?
Is moving the ownership of the background script process to the shell on tty1 an option? If so, the easiest may be to auto-login the Pi (or the user will need to login with the keyboard on startup). Then auto-start the program on tty1 so its stdin/stdout is tied to tty1.
To achieve the latter, I think you can put its invocation into one of the bash startup scripts, something like what is suggested here: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=7192
You can run a script in foreground at boot by adding a line to /etc/rc.local
This works in my experience, in particular if the Raspberry pi is configured to wait for network to be available when booting
I have one main program "main.py" that may freeze occasionally. Whenever I detect this happens, I want to have a separate program "watch.py" get my Linux machine to restart. These scripts start at bootup automatically since I edited /etc/rc.local.
Right now /etc/rc.local looks like this -
python watch.py &
python main.py &
This should let both programs run simultaneously. When I notice main.py has frozen, I'll give a signal manually to watch.py (using a remote TCP connection) to restart. What should my python code or shell script be to actually restart the system when "watch.py" receives a signal?
Possible answers could be writing some python code to restart a Linux machine, writing some python code to exit with a certain argument, and upon noticing the argument, execute "sudo reboot".
I am running a python process (through Django's manage.py) that will take about 6 days to complete. I would start it through an SSH tunnel, but that would require me to keep my Terminal window running and not let my computer fall asleep so the connection stays intact.
While the process is running, it reports the percentage on a single line (by replacing the line over and over again).
How can I start the process and be able to disconnect from SSH, but still be able to check on its progress when I reconnect?
use GNU Screen, your life will be so much more beautiful when you could load/reload your working session of terminals anytime you want --- they never die, and yet they ain't no zombies.
This is not really a Python question, rather an OS question.
So what you can do in linux is run your process with no hang up.
tmp.py
import time
for i in range(10):
print i
time.sleep(1)
No you can run it like so:
sudo nohup python -u tmp.py >> output &
The output will be placed in this file.
Then you can watch the output comming to the file by doing
tail -f output