I write a program my_test.py to get data from web and store to mysql.
But the program my_test.py collapses a lot (my bad programming skill...) and I try to monitor its status and restart it when it collapses.
I use subprocess modular with the following codes.
import subprocess
import time
p = subprocess.Popen(['python.exe', r'D:\my_test.py'], shell=True)
while True:
try:
stopped = p.poll()
except:
stopped = True
if stopped:
p = subprocess.Popen(['python.exe', r'D:\my_test.py'], shell=True)
time.sleep(60)
But when my_test.py collapses, a windows warning window jumps out to alert me that my_test.py is down and which action I will choose: stop, debug ...
Something like that.
And my_test.py seems frozen by the alert windows and the codes above can't restart it successfully.
Only when I manually close the window by choose 'close', it will restart again.
It there any solution to this problem such that my codes can successfully restart my_test.py when it breaks down?
Sorry for the inconvinience brought by my poor English and thank in advance for your kind advices.
There are two parts in your question:
what to do with the debug dialog. You could try this: How do I disable the 'Debug / Close Application' dialog on Windows Vista?
how to restart the script automatically
How to restart
The priority order:
fix my_test.py, to avoid crashing due to known issues
use a supervisor program to run your script such as upstart or supervisord -- they can restart it automatically if it crashes
write your own supervisor program with its own bugs that you have to maintain
It is best to limit yourself to options 1 and/or 2 if you can find an already-written supervisor program that works on Windows (upstart and supervisord do not work on Windows).
Your current supervisor script could be improved to avoid waiting a minute before restarting the program after it crashes if it has been running more than a minute already:
#!/usr/bin/env python3
import sys
import subprocess
import time
try:
from time import monotonic as timer
except ImportError:
from time import time as timer # time() can be set back
while True:
earliest_next_start = timer() + 60
subprocess.call([sys.executable, r'D:\my_test.py'])
while timer() < earliest_next_start:
time.sleep(max(0, earliest_next_start - timer()))
Related
I need to login to IBM i System using Python without entering the username and password manually.
I used py3270 library but it is not able to detect the Emulator wc3270. The emulator I use has .hod extension and opens with IBM i Launcher.
Can anyone help me with this? what could be the possible solution for this?
os.system() is a blocking statement. That is, it blocks, or stops further Python code from being executed until whatever os.system() is doing has completed. This problem needs us to spawn a separate thread, so that the Windows process executing the ACS software runs at the same time the rest of the Python code runs. subprocess is one Python library that can handle this.
Here is some code that opens an ACS 5250 terminal window and pushes the user and password onto that window. There's no error checking, and there are some setup details that my system assumes about ACS which your system may not.
# the various print() statements are for looking behind the scenes
import sys
import time
import subprocess
from pywinauto.application import Application
import pywinauto.keyboard as keyboard
userid = sys.argv[1]
password = sys.argv[2]
print("Starting ACS")
cmd = r"C:\Users\Public\IBM\ClientSolutions\Start_Programs\Windows_x86-64\acslaunch_win-64.exe"
system = r'/system="your system name or IP goes here"'
# Popen requires the command to be separate from each of the parameters, so an array
result = subprocess.Popen([cmd, r"/plugin=5250",system], shell=True)
print(result)
# wait at least long enough for Windows to get past the splash screen
print("ACS starting - pausing")
time.sleep(5)
print("connecting to Windows process")
ACS = Application().connect(path=cmd)
print(ACS)
# debugging
windows = ACS.windows()
print(windows)
dialog = ACS['Signon to IBM i']
print(dialog)
print("sending keystrokes")
keyboard.send_keys(userid)
keyboard.send_keys("{TAB}")
keyboard.send_keys(password)
keyboard.send_keys("{ENTER}")
print('Done.')
Currently, I am facing the same issue. I was able to run the IBMi (ACS), however, once it run, my python script stop functioning as if the app is preventing the python from being running. In generally speaking, the app seems to not detecting the script.But once I closed the app, my python script continue to work.. I put some indication e.g timesleep, however as i mentioned earlier, it only continue to that line of code once IBM is closed. There will be few lines to be added to move the selection to 5250 and inject the credential.
*I tried with pyautogui, still facing the same issue. so now i tried pywinauto import keyboard .
#Variables
dir = sys.argv[1]
username = sys.argv[2]
password = sys.argv[3]
x = dir.split("\\")
print(x[-1])
command = "cd \ && cd Users/Public/Desktop && " + '"' + x[-1] + '"'
print(command)
os.system(command)
------ FROM THIS LINE OF CODE ONWARDS, IT STOPPED RUNNING ONCE IBM IS LAUNCHED ---
print('TIME START')
time.sleep(5)
print('TIME END')
keyboard.send_keys(username)
keyboard.send_keys(password)
keyboard.send_keys("{ENTER}")
print('Done.')
Appreciate your help to look into this matter. Thanks
I need to login to IBM i System using Python without entering the username and password manually.
I used py3270 library but it is not able to detect the Emulator wc3270. The emulator I use has .hod extension and opens with IBM i Launcher.
Can anyone help me with this? what could be the possible solution for this?
os.system() is a blocking statement. That is, it blocks, or stops further Python code from being executed until whatever os.system() is doing has completed. This problem needs us to spawn a separate thread, so that the Windows process executing the ACS software runs at the same time the rest of the Python code runs. subprocess is one Python library that can handle this.
Here is some code that opens an ACS 5250 terminal window and pushes the user and password onto that window. There's no error checking, and there are some setup details that my system assumes about ACS which your system may not.
# the various print() statements are for looking behind the scenes
import sys
import time
import subprocess
from pywinauto.application import Application
import pywinauto.keyboard as keyboard
userid = sys.argv[1]
password = sys.argv[2]
print("Starting ACS")
cmd = r"C:\Users\Public\IBM\ClientSolutions\Start_Programs\Windows_x86-64\acslaunch_win-64.exe"
system = r'/system="your system name or IP goes here"'
# Popen requires the command to be separate from each of the parameters, so an array
result = subprocess.Popen([cmd, r"/plugin=5250",system], shell=True)
print(result)
# wait at least long enough for Windows to get past the splash screen
print("ACS starting - pausing")
time.sleep(5)
print("connecting to Windows process")
ACS = Application().connect(path=cmd)
print(ACS)
# debugging
windows = ACS.windows()
print(windows)
dialog = ACS['Signon to IBM i']
print(dialog)
print("sending keystrokes")
keyboard.send_keys(userid)
keyboard.send_keys("{TAB}")
keyboard.send_keys(password)
keyboard.send_keys("{ENTER}")
print('Done.')
Currently, I am facing the same issue. I was able to run the IBMi (ACS), however, once it run, my python script stop functioning as if the app is preventing the python from being running. In generally speaking, the app seems to not detecting the script.But once I closed the app, my python script continue to work.. I put some indication e.g timesleep, however as i mentioned earlier, it only continue to that line of code once IBM is closed. There will be few lines to be added to move the selection to 5250 and inject the credential.
*I tried with pyautogui, still facing the same issue. so now i tried pywinauto import keyboard .
#Variables
dir = sys.argv[1]
username = sys.argv[2]
password = sys.argv[3]
x = dir.split("\\")
print(x[-1])
command = "cd \ && cd Users/Public/Desktop && " + '"' + x[-1] + '"'
print(command)
os.system(command)
------ FROM THIS LINE OF CODE ONWARDS, IT STOPPED RUNNING ONCE IBM IS LAUNCHED ---
print('TIME START')
time.sleep(5)
print('TIME END')
keyboard.send_keys(username)
keyboard.send_keys(password)
keyboard.send_keys("{ENTER}")
print('Done.')
Appreciate your help to look into this matter. Thanks
I have some script in Python, which does some work. I want to re-run this script automatically. Also, I want to relaunch it on any crashes/freezes.
I can do something like this:
while True:
try:
main()
except Exception:
os.execv(sys.executable, ['python'] + sys.argv)
But, for unknown reason, this still crashes or freezes one time in few days. So I see crash, write "Python main.py" in cmd and it started, so I don't know why os.execv don't do this work by self. I guess it's because this code is part of this app. So, I prefer some script/app, which will control relaunch in external way. I hope it will be more stable.
So this script should work in this way:
Start any script
Check that process of this script is working, for example check some file time change and control it by process name|ID|etc.
When it dissapears from process list, launch it again
When file changed more than 5 minutes ago, stop process, wait few sec, launch it again.
In general: be cross-platform (Linux/Windows)
not important log all crashes.
I can do this by self (right now working on it), but I'm pretty sure something like this must already be done by somebody, I just can't find it in Google\Github.
UPDATE: added code from the #hansaplast answer to GitHub. Also added some changes to it: relauncher. Feel free to copy/use it.
As it needs to work both in windows and on linux I don't know a way to do that with standard tools, so here's a DIY solution:
from subprocess import Popen
import os
import time
# change into scripts directory
abspath = os.path.abspath(__file__)
dname = os.path.dirname(abspath)
os.chdir(dname)
while True:
p = Popen(['python', 'my_script.py', 'arg1', 'arg2'])
time.sleep(20) # give the program some time to write into logfile
while True:
if p.poll() != None:
print('crashed or regularly terminated')
break
file_age_in_s = time.time() - os.path.getmtime('output.log')
if file_age_in_s > 60:
print('frozen, killing process')
p.kill()
break
time.sleep(1)
print('restarting..')
Explanation:
time.sleep(20): give script 20 seconds to write into the log file
poll(): regularly check if script died (either crashed or regularly terminated, you can check the return value of poll() to differentiate that)
getmtime(): regularly check output.log and check if that was changed the past 60 seconds
time.sleep(1): between every check wait for 1s as otherwise it would eat up too many system resources
The script assumes that the check-script and the run-script are in the same directory. If that is not the case, change the lines beneath "change into scripts directory"
I personally like supervisor daemon, but it has two issues here:
It is only for unix systems
It restarts app only on crashes, not freezes.
But it has simple XML-RPC API, so It makes your job to write an freeze-watchdog app simplier. You could just start your process under supervisor and restart it via supervisor API when you see it freezes.
You could install it via apt install supervisor on ubuntu and write config like this:
[program:main]
user=vladimir
command=python3 /var/local/main/main.py
process_name=%(program_name)s
directory=/var/local/main
autostart=true
autorestart=true
I have a Python script that runs all day long checking time every 60 seconds so it can start/end tasks (other python scripts) at specific periods of the day.
This script is running almost all ok. Tasks are starting at the right time and being open over a new cmd window so the main script can keep running and sampling the time. The only problem is that it just won't kill the tasks.
import os
import time
import signal
import subprocess
import ctypes
freq = 60 # sampling frequency in seconds
while True:
print 'Sampling time...'
now = int(time.time())
#initialize the task.. lets say 8:30am
if ( time.strftime("%H:%M", time.localtime(now)) == '08:30'):
# The following method is used so python opens another cmd window and keeps original script running and sampling time
pro = subprocess.Popen(["start", "cmd", "/k", "python python-task.py"], shell=True)
# kill process attempts.. lets say 11:40am
if ( time.strftime("%H:%M", time.localtime(now)) == '11:40'):
pro.kill() #not working - nothing happens
pro.terminate() #not working - nothing happens
os.kill(pro.pid, signal.SIGINT) #not working - windows error 5 access denied
# Kill the process using ctypes - not working - nothing happens
ctypes.windll.kernel32.TerminateProcess(int(pro._handle), -1)
# Kill process using windows taskkill - nothing happens
os.popen('TASKKILL /PID '+str(pro.pid)+' /F')
time.sleep(freq)
Important Note: the task script python-task.py will run indefinitely. That's exactly why I need to be able to "force" kill it at a certain time while it still running.
Any clues? What am I doing wrong? How to kill it?
You're killing the shell that spawns your sub-process, not your sub-process.
Edit: From the documentation:
The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
Warning
Passing shell=True can be a security hazard if combined with untrusted input. See the warning under Frequently Used Arguments for details.
So, instead of passing a single string, pass each argument separately in the list, and eschew using the shell. You probably want to use the same executable for the child as for the parent, so it's usually something like:
pro = subprocess.Popen([sys.executable, "python-task.py"])
I'm writing a python script with an infinite while loop that I am running over ssh. I would like the script to terminate when someone kills ssh. For example:
The script (script.py):
while True:
# do something
Will be run as:
ssh foo ./script.py
When I kill the ssh process, I would like the script on the other end to stop running.
I have tried looking for a closed stdout:
while not sys.stdout.closed:
# do something
but this didn't work.
How do I achieve this?
Edit:
The remote machine is a Mac which opens the program in a csh:
502 29352 ?? 0:00.01 tcsh -c python test.py
502 29354 ?? 0:00.04 python test.py
I'm opening the ssh process from a python script like so:
p = Popen(['ssh','foo','./script.py'],stdout=PIPE)
while True:
line = p.stdout.readline()
# etc
EDIT
Proposed Solutions:
Run the script with while os.getppid() != 1
This seems to work on Linux systems, but does not work when the remote machine is running OSX. The problem is that the command is launched in a csh (see above) and so the csh has its parent process id set to 1, but not the script.
Periodically log to stderr
This works, but the script is also run locally, and I don't want to print a heartbeat to stderr.
Run the script in a pseduo tty with ssh -tt.
This does work, but has some weird consequences. Consider the following:
remote_script:
#!/usr/bin/env python
import os
import time
import sys
while True:
print time.time()
sys.stdout.flush()
time.sleep(1)
local_script:
#!/usr/bin/env python
from subprocess import Popen, PIPE
import time
p = Popen(['ssh','-tt','user#foo','remote_script'],stdout=PIPE)
while True:
line = p.stdout.readline().strip()
if line:
print line
else:
break
time.sleep(10)
First of all, the output is really weird, it seems to keep adding tabs or something:
[user#local ~]$ local_script
1393608642.7
1393608643.71
1393608644.71
Connection to foo closed.
Second of all, the program does not quit the first time it receives a SIGINT, i.e. I have to hit Ctrl-C twice in order to kill the local_script.
Okay, I have a solution for you
When the ssh connection closes, the parent process id will change from the pid of the ssh-deamon (the fork that handles your connection) to 1.
Thus the following solves your problem.
#!/usr/local/bin/python
from time import sleep
import os
#os.getppid() returns parent pid
while (os.getppid() != 1):
sleep(1)
pass
Can you confirm this is working in your end too :)
edit
I saw you update.
This is not tested, but to get this idea working on OSX, you may be able to detect if the process of the csh changes. The code below only illustrates an idea and has not been tested. That said i think it would work, but it would not be the most elegant solution. If a cross platform solution using signals could be found, it would be preferred.
def do_stuff():
sleep(1)
if sys.platform == 'darwin':
tcsh_pid = os.getppid()
sshfork_pid = psutil.Process(tcsh_pid).ppid
while (sshfork_pid == psutil.Process(tcsh_pid).ppid)
do_stuff()
elif sys.platform == 'linux':
while (os.getppid() != 1):
sleep(1)
else:
raise Exception("platform not supported")
sys.exit(1)
Have you tried
ssh -tt foo ./script.py
When the terminal connection is lost, the application is supposed to receive SIGHUP signal, so all you have to do is to register a special handler using signal module.
import signal
def MyHandler(self, signum, stackFrame):
errorMessage = "I was stopped by %s" % signum
raise Exception(errorMessage)
# somewhere in the beginning of the __main__:
# registering the handler
signal.signal(signal.SIGHUP, MyHandler)
Note that most likely you'll have to handle some other signals. You can do it in absolutely the same way.
I'd suggest periodically logging to stderr.
This will cause an exception to occur when you no longer have a stderr to write to.
The running script is a child pid of the terminal session. If you close the SSH session properly it will terminate the process. But, another method of going about this is to connect your while loop to another factor and disconnect it from your SSH session.
You can have your script controlled by cron to execute regularly. You can have the while loop have a counter. You can have a sleep command in the loop to control execution. Pretty much anything other than having it connected to your SSH session is valid.
To do this you could use exec & to disconnect instances from your loop.