We Keep Coding

How to keep AWS Lightsail virtual server running [duplicate]

I have set up Flask on my Rapsberry Pi and I am using it for the sole purpose of acting as a server for an xml file which I created with a Python script to pass data to an iPad app (iRule).
My RPI is set up as headless and my access is with Windows 10 using PuTTY, WinSCP and TightVNC Viewer.
I run the server by opening a terminal window and the following command:
sudo python app1c.py
This sets up the server and I can access my xml file quite well. However, when I turn off the Windows machine and the PuTTY session, the Flask server shuts down!
How can I set it up so that the Flask server continues even when the Windows machine is turned off?
I read in the Flask documentation:
While lightweight and easy to use, Flask’s built-in server is not suitable for production as it doesn’t scale well and by default serves only one request at a time.
Then they go on to give examples of how to deploy your Flask application to a WSGI server! Is this necessary given the simple application I am dealing with?

Use:
$ sudo nohup python app1c.py > log.txt 2>&1 &
nohup allows to run command/process or shell script that can continue running in the background after you log out from a shell.
> log.txt: it forword the output to this file.
2>&1: move all the stderr to stdout.
The final & allows you to run a command/process in background on the current shell.

Install Node package forever at here https://www.npmjs.com/package/forever
Then use
forever start -c python your_script.py
to start your script in the background. Later you can use
forever stop your_script.py
to stop the script

You have multiple options:
Easy: deattach the process with &, for example:
$ sudo python app1c.py &
Medium: install tmux with apt-get install tmux
launch tmux and start your app as before and detach with CTRL+B.
Complexer:
Read run your flask script with a wsgi server - uwsgi, gunicorn, nginx.

Been stressing lately so I decide to go deep.
pm2 start app.py --interpreter python3
Use PM2 for things like this. I also use it for NodeJs app and a Python app on a single server.

Use:
$sudo python app1c.py >> log.txt 2>&1 &
">> log.txt" pushes all your stdout inside the log.txt file (You may check the application logs in it)
"2>&1" pushes all the stderr inside the log.txt file (This would push all the error logs inside log.txt)
"&" at the end makes it run in the background.
You would get the process id immediately after executing this command with which you can monitor or verify it.
$sudo ps -ef | grep <process-id>
Hope it helps..!!

You can always use nohup to run any scripts as background process.
nohup python script.py
This will run your script in background and also have its logs appended in nohup.out file which will be located in the directory script.py is store.
Make sure, you close the terminal and not press Ctrl + C. This will allow it to run in background even when you log out.
To stop it from running , ssh in to the pi again and run ps -ef |grep nohup and kill -9 XXXXX
where XXXX is the pid you will get ps command.

I've always found a detached screen process to be best for use cases such as these.
Run:
screen -m -d sudo python app1c.py

I was trying to run my flask app for testing in my GitHub CI and the step where I was running the app was getting stuck for ever. The reason was that it was never releasing the command line
Best solution I found was a combination of two other responses in here:
nohup python script.py &

How to escape a python interactive shell

I am trying to escape a python interactive shell within an ssh server without closing the ssh connection, using exit(), quit() ctrl D closes the ssh connection

I'm assuming your connection is to a Linux/Unix server. If it's Windows, this won't help.
If you only close the python interpreter (ctrl-c for instance) it shouldn't close the SSH connection, since the python interpreter is running on top of the unix shell, which you are actually connected to.
The best way (or at least the easiest) to keep your SSH connection and keep any program running after you leave is to use a tool like tmux or screen (if your linux machine does not have tmux installed).
In order to do so, you can either start your program with $ screen python or start screen before you run anything, and it will start a screen session with bash running.
Then you can safely close the ssh connection, and, when you ssh back into the machine, use screen -r to return to where you leave.

You can easily import the pty module
import pty
and then spawn a new bash shell "/bin/bash" with the module using
pty.spawn("/bin/bash")
Note: "bash" in this context can be changed depending on what shell is avaliable on the server which could be "sh", "dash" or any other type of unix compatible shell

ssh -t
-t flag forces use of ptty. Control sequences will be sent directly to remote process

How to run Flask Server in the background

I have set up Flask on my Rapsberry Pi and I am using it for the sole purpose of acting as a server for an xml file which I created with a Python script to pass data to an iPad app (iRule).
My RPI is set up as headless and my access is with Windows 10 using PuTTY, WinSCP and TightVNC Viewer.
I run the server by opening a terminal window and the following command:
sudo python app1c.py
This sets up the server and I can access my xml file quite well. However, when I turn off the Windows machine and the PuTTY session, the Flask server shuts down!
How can I set it up so that the Flask server continues even when the Windows machine is turned off?
I read in the Flask documentation:
While lightweight and easy to use, Flask’s built-in server is not suitable for production as it doesn’t scale well and by default serves only one request at a time.
Then they go on to give examples of how to deploy your Flask application to a WSGI server! Is this necessary given the simple application I am dealing with?

Use:
$ sudo nohup python app1c.py > log.txt 2>&1 &
nohup allows to run command/process or shell script that can continue running in the background after you log out from a shell.
> log.txt: it forword the output to this file.
2>&1: move all the stderr to stdout.
The final & allows you to run a command/process in background on the current shell.

Install Node package forever at here https://www.npmjs.com/package/forever
Then use
forever start -c python your_script.py
to start your script in the background. Later you can use
forever stop your_script.py
to stop the script

You have multiple options:
Easy: deattach the process with &, for example:
$ sudo python app1c.py &
Medium: install tmux with apt-get install tmux
launch tmux and start your app as before and detach with CTRL+B.
Complexer:
Read run your flask script with a wsgi server - uwsgi, gunicorn, nginx.

Been stressing lately so I decide to go deep.
pm2 start app.py --interpreter python3
Use PM2 for things like this. I also use it for NodeJs app and a Python app on a single server.

Use:
$sudo python app1c.py >> log.txt 2>&1 &
">> log.txt" pushes all your stdout inside the log.txt file (You may check the application logs in it)
"2>&1" pushes all the stderr inside the log.txt file (This would push all the error logs inside log.txt)
"&" at the end makes it run in the background.
You would get the process id immediately after executing this command with which you can monitor or verify it.
$sudo ps -ef | grep <process-id>
Hope it helps..!!

You can always use nohup to run any scripts as background process.
nohup python script.py
This will run your script in background and also have its logs appended in nohup.out file which will be located in the directory script.py is store.
Make sure, you close the terminal and not press Ctrl + C. This will allow it to run in background even when you log out.
To stop it from running , ssh in to the pi again and run ps -ef |grep nohup and kill -9 XXXXX
where XXXX is the pid you will get ps command.

I've always found a detached screen process to be best for use cases such as these.
Run:
screen -m -d sudo python app1c.py

I was trying to run my flask app for testing in my GitHub CI and the step where I was running the app was getting stuck for ever. The reason was that it was never releasing the command line
Best solution I found was a combination of two other responses in here:
nohup python script.py &

How do I detach python script run from Pycharm so it keeps appending to files?

I am working on a remote system and I use Pycharm remote window to edit and run my scripts.
I login using
ssh -Y myName#myMachine
Then I run Pycharm from terminal.
I want to run my scripts from Pycharm in such a way, that if I close it (and perhaps even logout from ssh session), the processes will still run.
I have tried to exit Pycharm using option "Detach without terminating process". This results in the python process showing on the list of:
ps -all
however, it stops writing to a file. When pycharm is open the process normally writes to a file every few seconds. When detached from pycharm it shows on the list of processes (after logout and login again it shows in ps -x with unknown tty), however it stops working in the sense that it no longer appends any output to files that it normally should.
What can be causing it? How can I fix this?

There are several approaches:
Use terminal emulators as tmux and screen:
tmux - The tmux is a terminal multiplexer that enables a number of terminals to be created, accessed, and controlled from a single screen. A tmux session can be detached from a screen and continue running in the background, then later reattached. Like Screen tool, you can also use tmux to detach from an SSH session without quitting the remote jobs.
After installing tmux, start the tmux session using command:
$ tmux
Now, start your task or job. Then safely detach from the tmux session without exiting the remote jobs by pressing "CTRL-b" followed by "d". This will detach your tmux session but will leave you’re doing in that session running in the background. That means all remotes will be running even if you’re disconnected from the session.
To list the available sessions, run:
$ tmux ls
You can re-attach to the tmux session using the respective session ID as shown below:
$ tmux attach -t < session ID >
For more details, refer man pages.
$ man tmux
screen - The screen tool, a full-screen window manager with VT100/ANSI terminal emulation, allows you to safely detach from the SSH session without exiting the remote job. It will be helpful for those who are working with multiple remote servers.
After installing screen on your remote systems, start the screen session:
$ screen
The screen session has been started now. Now run whatever job or task you wanted to do on your remote system, Then you can exit from the screen session by pressing “Ctrl-A” followed by “d“, After detaching from the screen session, you can log out from the remote system. The remote job will keep running in the server.
To list the screen sessions, run:
$ screen -ls
You can re-attach to the screen session using the respective session ID as shown below:
$ screen -r < session ID >
For more details, refer man pages.
$ man screen
Use detached commands executed in the background ([read also][6]):
nohup - stands for No hangup, is yet another command line utility to help you run Linux commands even after you’re disconnected from the SSH sessions.
The usage is absolutely easy. After logging into your remote system, all you have to do is:
$ nohup < command > &
Now, you can exit from SSH session. The remote job will keep running.
To list the running jobs, run:
$ jobs -l
For more details, refer man pages.
$ man nohup
disown - Disown, removes the job from the process job list of the system, so the process is shielded from being killed during session disconnection as it won’t receive SIGHUP by the shell when you logout.
After logging into your remote system, run you command with "&":
$ < command > &
Then list the running jobs, using:
$ jobs -l
Then run disown with the process ID as shown below:
$ disown -h < PID >
You can now disconnect from the server
For more details, refer man pages.
$ man nohup
setsid - setsid allocates a new process group to the process being executed and hence, the process created is totally in a newly allocated process group and can execute safely without fear of being killed even after session logout.
After logging into your remote system, run:
$ setsid < command >
For more details, refer man pages.
$ man nohup

Running python code in background

I need to run a python code that takes several hours and my computer disconnects from the ssh after a certain amount of inactive time.
I have tried python test.py > output.txt & but my output file is empty. However, the python code "test" is still running after I log off and log back in to the ssh. I also tried python -u test.py > output.txt & which does write to the output.txt but it does not continue after the ssh connection is lost.
I am very new to Linux so I do not know very many commands. I need the simplest/easiest to understand method.
Thanks!

You can use screen, as Robin Krahl recommended, or you can just run your command with nohup, which suppresses the SIGHUP (hangup) signal from your SSH session disconnecting.
nohup "python -u test.py > output.txt" &

screen is the tool you want to use.

As others have said, screen is what you want. tmux is newer? tool that does the same thing, and byobu is an easy-to-use wrapper for both.