I need to run a python code that takes several hours and my computer disconnects from the ssh after a certain amount of inactive time.
I have tried python test.py > output.txt & but my output file is empty. However, the python code "test" is still running after I log off and log back in to the ssh. I also tried python -u test.py > output.txt & which does write to the output.txt but it does not continue after the ssh connection is lost.
I am very new to Linux so I do not know very many commands. I need the simplest/easiest to understand method.
Thanks!
You can use screen, as Robin Krahl recommended, or you can just run your command with nohup, which suppresses the SIGHUP (hangup) signal from your SSH session disconnecting.
nohup "python -u test.py > output.txt" &
screen is the tool you want to use.
As others have said, screen is what you want. tmux is newer? tool that does the same thing, and byobu is an easy-to-use wrapper for both.
Related
So, I have a python script which outputs some data into terminal from time to time. Im trying to run in on the Ubuntu VPS even after I close the SSH connection and still keep the logs somewhere.
Im saving the logs by using:
python3 my_script.py >>file.txt
and it works perfect, however when I try to run this process using
nohup python3 my_script.py >>file.txt &
so it runs in the background and after the ssh connection is closed it seems to save only the first log outputted from my_script.py. I've also tried running this in crontab but the result is similar - only the first log is saved.
Any tips? What am I doing wrong?
I could not understand what you mean "the first log". Maybe the first line of logs?
To run something in the background when SSH connection is closed, I prefer Linux screen, a terminal simulation tool that help you run your command in a sub-process. With it, you could choose to view your output any time in the foreground, or leave your process run in the background.
Usage (short)
screen is not included in most Linux distributions. Install it (Ubuntu):
$ sudo apt-get install screen
Run your script in the foreground:
$ screen python3 my_script.py
You'll see it running. Now detach from this screen: Press keys Ctrl-A followed by Ctrl-D. You'll be back to your shell where you run previous screen command. If you need to switch back to the running context, use screen -r command.
This tool supports multiple parallel running process too.
Something weird
I've tried to redirect stdout or stderr to a file with > or >> symbol. It turned out in failure. I am not an expert of this either, and maybe you need to see its manual page. However, I tend to directly write to a file in Python scripts, with some essential output lines on the console.
I've created a script for my school project that works with data. I'm quite new to working remotely on a server, so this might seem like a dumb question, but how do I execute my script named
stats.py
so that it continues executing even after I log off PuTTy? The script file is located on the server. It has to work with a lot of data, so I don't want to just try something and then few days later find out that it has exited right after I logged off.
Thank you for any help!
There are many ways you can run a python program after you disconnect from an SSH session.
1) Tmux or Screen
Tmux is a "terminal multiplexer" which enables a number of terminals to be accessed by a single one.
You start by sshing as you do, run it by typing tmux and executing it. Once you are done you can disconnect from putty and when you login back you can relog to the tmux session you left
Screen also does that you just type screen instead of tmux
2) nohup
"nohup is a POSIX command to ignore the HUP signal. The HUP signal is, by convention, the way a terminal warns dependent processes of logout."
You can run it by typing nohup <pythonprogram> &
I am trying to run this Python Script on a Digital Ocean (DO) Droplet (Ubuntu 16):
https://github.com/instabot-py/instabot.py
Although I can run python3 example.py the process stops if I kill the Terminal windown I used to SSH into the DO Droplet.
Is there a way to run this Python script and disconnect from the DO Droplet with the Python script still running?
Use nohup to run program in background
nohup python3 example.py > /dev/null 2>&1 &
Or you can use screen to do this:
$ screen
$ python3 example.py
To detach the current session, you can press Ctrl+A and then press D.
To re-attach previous session, use command `screen -r`
The screen command is more powerful than nohup.
You can use screen in linux to continue executing the process in background even after you disconnect from server.
Similarly tmux can be used.
The answer to this question is most probably what you are looking for.
You can try these python library to achieve this
http://www.paramiko.org/
https://pypi.python.org/pypi/spur
Or you can also use tmux to run command on remote server and your tmux seisson will run in the background even if you quit the shell. You can join that tmux session even after days and continue executing command
We use the following format to execute remote script
ssh -t -t user#host.com 'nohup /path/to/scrip.py'
I have set up Flask on my Rapsberry Pi and I am using it for the sole purpose of acting as a server for an xml file which I created with a Python script to pass data to an iPad app (iRule).
My RPI is set up as headless and my access is with Windows 10 using PuTTY, WinSCP and TightVNC Viewer.
I run the server by opening a terminal window and the following command:
sudo python app1c.py
This sets up the server and I can access my xml file quite well. However, when I turn off the Windows machine and the PuTTY session, the Flask server shuts down!
How can I set it up so that the Flask server continues even when the Windows machine is turned off?
I read in the Flask documentation:
While lightweight and easy to use, Flask’s built-in server is not suitable for production as it doesn’t scale well and by default serves only one request at a time.
Then they go on to give examples of how to deploy your Flask application to a WSGI server! Is this necessary given the simple application I am dealing with?
Use:
$ sudo nohup python app1c.py > log.txt 2>&1 &
nohup allows to run command/process or shell script that can continue running in the background after you log out from a shell.
> log.txt: it forword the output to this file.
2>&1: move all the stderr to stdout.
The final & allows you to run a command/process in background on the current shell.
Install Node package forever at here https://www.npmjs.com/package/forever
Then use
forever start -c python your_script.py
to start your script in the background. Later you can use
forever stop your_script.py
to stop the script
You have multiple options:
Easy: deattach the process with &, for example:
$ sudo python app1c.py &
Medium: install tmux with apt-get install tmux
launch tmux and start your app as before and detach with CTRL+B.
Complexer:
Read run your flask script with a wsgi server - uwsgi, gunicorn, nginx.
Been stressing lately so I decide to go deep.
pm2 start app.py --interpreter python3
Use PM2 for things like this. I also use it for NodeJs app and a Python app on a single server.
Use:
$sudo python app1c.py >> log.txt 2>&1 &
">> log.txt" pushes all your stdout inside the log.txt file (You may check the application logs in it)
"2>&1" pushes all the stderr inside the log.txt file (This would push all the error logs inside log.txt)
"&" at the end makes it run in the background.
You would get the process id immediately after executing this command with which you can monitor or verify it.
$sudo ps -ef | grep <process-id>
Hope it helps..!!
You can always use nohup to run any scripts as background process.
nohup python script.py
This will run your script in background and also have its logs appended in nohup.out file which will be located in the directory script.py is store.
Make sure, you close the terminal and not press Ctrl + C. This will allow it to run in background even when you log out.
To stop it from running , ssh in to the pi again and run ps -ef |grep nohup and kill -9 XXXXX
where XXXX is the pid you will get ps command.
I've always found a detached screen process to be best for use cases such as these.
Run:
screen -m -d sudo python app1c.py
I was trying to run my flask app for testing in my GitHub CI and the step where I was running the app was getting stuck for ever. The reason was that it was never releasing the command line
Best solution I found was a combination of two other responses in here:
nohup python script.py &
I want to run a python script on my server (that python script has GUI). But I want to start it from ssh. Something like this:
ssh me#server -i my_key "nohup python script.py"
... > let the script run forever
BUT it complains "unable to access video driver" since it is trying to use my ssh terminal as output.
Can I somehow make my commands output run on server machine and not to my terminal... Basically something like "wake-on-lan functionality" -> tell the server you want something and he will do everything using its own system (not sending any output back)
What about
ssh me#server -i my_key "nohup python script.py >/dev/null 2>&1"
You can use redirection to some remote logfile instead of /dev/null of course.
? :)
EDIT: GUI applications on X usually use $DISPLAY variable to know where they should be displayed. Moreover, X11 display servers use authorization to permit or disallow applications connecting to its display. Commands
export DISPLAY=:0 && xhost +
may be helpful for you.
Isn't it possible for you to rather use python ssh extension instead of calling external application?
It would:
run as one process
guarantee that invocation will be the same among all possible system
lose the overhead from "execution"
send everything trough ssh (you won't have to worry about input like "; possibly local executed command)
If not, go with what Piotr Wades suggested.