What is the correct way to have python schedule (by daniel bader) persistently run. I currently run the job by having a terminal open, connected to a VM where the scripts actually run. There I run python "scheduler.py" - where scheduler.py has all the jobs.
But when the connection closes, or I close the terminal, the scheduler stops.
Any easy solutions to fix this?
You have a couple options here. You are starting the process in your ssh session, but then killing the ssh session, which then kills the process.
One way to handle this, would be to have the VM run the script on startup. You could set the script as a service, so even if it goes down for some reason it will come back up. Read into init.rc for info on how launch a script at boot on linux. I'm not well-versed in Windows any more but I believe there is a way to do the same.
Another option is to keep the session open by connecting to it with screen or tmux. This article explains the problem some and gives you a few different ways to work around the issue: https://www.tecmint.com/keep-remote-ssh-sessions-running-after-disconnection/
I've created a script for my school project that works with data. I'm quite new to working remotely on a server, so this might seem like a dumb question, but how do I execute my script named
stats.py
so that it continues executing even after I log off PuTTy? The script file is located on the server. It has to work with a lot of data, so I don't want to just try something and then few days later find out that it has exited right after I logged off.
Thank you for any help!
There are many ways you can run a python program after you disconnect from an SSH session.
1) Tmux or Screen
Tmux is a "terminal multiplexer" which enables a number of terminals to be accessed by a single one.
You start by sshing as you do, run it by typing tmux and executing it. Once you are done you can disconnect from putty and when you login back you can relog to the tmux session you left
Screen also does that you just type screen instead of tmux
2) nohup
"nohup is a POSIX command to ignore the HUP signal. The HUP signal is, by convention, the way a terminal warns dependent processes of logout."
You can run it by typing nohup <pythonprogram> &
In my Python socket program, I sometimes need to interrupt it with Ctrl-C. When I do this, it does close the connection using socket.close().
However, when I try to reopen it I have to wait what seems like a minute before I can connect again. How does one correctly close a socket? Or is this intended?
Yes, it is intended. Here you can read detailed explanation. It is possible to override this behavior by setting SO_REUSEADDR option on a socket. For example:
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
$ ps -fA | grep python
501 81211 12368 0 10:11PM ttys000 0:03.12
python -m SimpleHTTPServer
$ kill 81211
This happens because you trying to run service at the same port and there is an already running application.
it can happen because your service is not stopped in the process stack. you just have to kill those processes.
There is no need to install anything here is the one line command to kill all running python processes.
for Linux based OS:
Bash:
kill -9 $(ps -A | grep python | awk '{print $1}')
Fish:
kill -9 (ps -A | grep python | awk '{print $1}')
If you use a TCPServer, UDPServer or their subclasses in the socketserver module, you can set this class variable (before instantiating a server):
socketserver.TCPServer.allow_reuse_address = True
(via SocketServer.ThreadingTCPServer - Cannot bind to address after program restart )
This causes the init (constructor) to:
if self.allow_reuse_address:
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
Nothing worked for me except running a subprocess with this command, before calling HTTPServer(('', 443), myHandler):
kill -9 $(lsof -ti tcp:443)
Of course this is only for linux-like OS!
A simple solution that worked for me is to close the Terminal and restart it.
For Linux,
ps aux | grep python
This will show you the error. The process number (eg.35225) containing your python file is the error.
Now,
sudo kill -9 35225
This will kill the error process and your problem will be solved.
First of all find the python process ID using this command
ps -fA | grep python
You will get a pid number by naming of your python process on second column
Then kill the process using this command
kill -9 pid
run the command
fuser -k (port_number_you_are _trying_to_access)/TCP
example for flask: fuser -k 5000/tcp
Also, remember this error arises when you interput by ctrl+z. so to terminate use ctrl+c
I faced similar error at odoo server and resolved that with these simple following steps:
Paste following code in terminal
ps -fA | grep python
You will get a pid number. Now copy the pid number from second column of terminal output.
Then write as below
kill -9 pid
The terminal will restart and then the command
flask run
Will work fine!
Thank you
Do nothing just wait for a couple of minutes and it will get resolved. It happens due to the slow termination of some processes, and that's why it's not even showing in the running processes list.
I had the same problem (Err98 Address already in use) on a Raspberry Pi running python for a EV charging manager for a Tesla Wall Connector. The software had previously been fine but it stopped interrogating the solar inverter one day and I spent days thinking it was something I'd done in python. Turns out the root cause was the Wifi modem assigning a new dynamic IP to the solar inverter as as result of introducing a new smart TV into my home. I changed the python code to reflect the new IP address that I found from the wifi modem and bingo, the issue was fixed.
Got this error after I ran my code while programming a Pico W via Thonny. At the command line just do a socket.reset() to clear the issue.
>>> socket.reset()
The cleanest way to make the socket immediately reusable is to follow the recommendation to first shutdown the client end (socket) of a connection, and make sure the server's end shuts down last (through exception handling if needed).
This might well mean that the server end runs forever.
This is not a problem if that "forever" loop pauses execution, e.g. read from socket.
How you "break" that "forever" loop is up to you as server admin, as long as there are no clients (apart from obvious system level exceptions)
I tried the following code to settle the issue:
sudo lsof -t -i tcp:8000 | xargs kill -9
sudo pkill -9 python
try this command
From my home pc using putty, I ssh'ed into a remote server, and I ran a python program that takes hours to complete, and as it runs it prints stuff. Now after a while, my internet disconnected, and I had to close and re-open putty and ssh back in. If I type 'top' I can see the python program running in the background with its PID number. Is there a command I can use to basically re-open that process and see it printing its stuff again?
Thanks
As noted, best practice is to use screen or tmux (before starting the program, so you do not need to ask this question).
But you can also attach to a running process with a debugger such as gdb (alluded to here as ddd, a wrapper for gdb), as well as with strace (see this question). That's better than nothing - but gdb and strace would not give you the program's command-line again (though this question suggests a way). At least strace could give you some clues of what the program was attempting to print.
Things to try:
nohup, or
screen
I am running a python process (through Django's manage.py) that will take about 6 days to complete. I would start it through an SSH tunnel, but that would require me to keep my Terminal window running and not let my computer fall asleep so the connection stays intact.
While the process is running, it reports the percentage on a single line (by replacing the line over and over again).
How can I start the process and be able to disconnect from SSH, but still be able to check on its progress when I reconnect?
use GNU Screen, your life will be so much more beautiful when you could load/reload your working session of terminals anytime you want --- they never die, and yet they ain't no zombies.
This is not really a Python question, rather an OS question.
So what you can do in linux is run your process with no hang up.
tmp.py
import time
for i in range(10):
print i
time.sleep(1)
No you can run it like so:
sudo nohup python -u tmp.py >> output &
The output will be placed in this file.
Then you can watch the output comming to the file by doing
tail -f output