I have a python script that collect data from a database every minutes by timestamp.
Every minutes this script collect data from a given table in DB by that match the current time with a dely of 1 minutes:
For example at ' 216-04-12 14:53 ' the script will look for data
that match ' 216-04-12 14:52 ' in the DB and so on...
Now I want the script to save the last timestamp it collected from the data base before exiting and that for any type of exit (keyboard interrupt, system errors, data base points of failure etc.)
Is there a simple way to do what I want knowing that I can't modify the dataBase
Python's atexit module may help you here. You can import atexit and register functions to run when the program exits.
See this post, namely:
import atexit
def exit_handler():
print 'My application is ending!'
atexit.register(exit_handler)
That will work in most exit situations.
Another, more robust answer from that same thread:
def main():
try:
execute_app() # Stuff that happens before program exits
finally:
handle_cleanup() # Stuff that happens when program exits
if __name__=='__main__':
main()
The above is a bit more reliable...
The MOST reliable way would be to write your output every minute, each time overwriting the previous output. That way no matter whether your exit cleanup fires or not, you still have your data from the minute the program exited.
You could use atexit.register() from module atexit to register cleanup functions. If you register functions f, g, h in that order. At program exit these will be executed in the reverse order, h, g, f.
But one thing to note: These functions will be invoked upon normal program exit. Normal means exits handled by python. So won't work in weird cases.
Related
I am using Slurm job manager on an HPC cluster. Sometimes there are situations, when a job is canceled due to time limit and I would like to finish my program gracefully.
As far as I understand, the process of cancellation occurs in two stages exactly for a software developer to be able to finish the program gracefully:
srun: Job step aborted: Waiting up to 62 seconds for job step to finish.
slurmstepd: error: *** JOB 18522559 ON ncm0317 CANCELLED AT 2020-12-14T19:42:43 DUE TO TIME LIMIT ***
You can see that I am given 62 seconds to finish the job the way I want it to finish (by saving some files, etc.).
Question: how to do this? I understand that first some Unix signal is sent to my job and I need to respond to it correctly. However, I cannot find in the Slurm documentation any information on what this signal is. Besides, I do not exactly how to handle it in Python, probably, through exception handling.
In Slurm, you can decide which signal is sent at which moment before your job hits the time limit.
From the sbatch man page:
--signal=[[R][B]:]<sig_num>[#<sig_time>]
When a job is within sig_time seconds of its end time, send it the signal sig_num.
So set
#SBATCH --signal=B:TERM#05:00
to get Slurm to signal the job with SIGTERM 5 minutes before the allocation ends. Note that depending on how you start your job, you might need to remove the B: part.
In your Python script, use the signal package. You need to define a "signal handler", a function that will be called when the signal is receive, and "register" that function for a specific signal. As that function is disrupting the normal flow when called , you need to keep it short and simple to avoid unwanted side effects, especially with multithreaded code.
A typical scheme in a Slurm environment is to have a script skeleton like this:
#! /bin/env python
import signal, os, sys
# Global Boolean variable that indicates that a signal has been received
interrupted = False
# Global Boolean variable that indicates then natural end of the computations
converged = False
# Definition of the signal handler. All it does is flip the 'interrupted' variable
def signal_handler(signum, frame):
global interrupted
interrupted = True
# Register the signal handler
signal.signal(signal.SIGTERM, signal_handler)
try:
# Try to recover a state file with the relevant variables stored
# from previous stop if any
with open('state', 'r') as file:
vars = file.read()
except:
# Otherwise bootstrap (start from scratch)
vars = init_computation()
while not interrupted and not converged:
do_computation_iteration()
# Save current state
if interrupted:
with open('state', 'w') as file:
file.write(vars)
sys.exit(99)
sys.exit(0)
This first tries to restart computations left by a previous run of the job, and otherwise bootstraps it. If it was interrupted, it lets the current loop iteration finish properly, and then saves the needed variables to disk. It then exits with the 99 return code. This allows, if Slurm is configured for it, to requeue the job automatically for further iteration.
If slurm is not configured for it, you can do it manually in the submission script like this:
python myscript.py || scontrol requeue $SLURM_JOB_ID
In most programming languages, Unix signals are captured using a callback. Python is no exception. To catch Unix signals using Python, just use the signal package.
For example, to gracefully exit:
import signal, sys
def terminate_signal(signalnum, handler):
print ('Terminate the process')
# save results, whatever...
sys.exit()
# initialize signal with a callback
signal.signal(signal.SIGTERM, terminate_signal)
while True:
pass # work
List of possible signals. SIGTERM is the one used to "politely ask a program to terminate".
I have 2 separate scripts working with the same variables.
To be more precise, one code edits the variables and the other one uses them (It would be nice if it could edit them too but not absolutely necessary.)
This is what i am currently doing:
When code 1 edits a variable it dumps it into a json file.
Code 2 repeatedly opens the json file to get the variables.
This method is really not elegant and the while loop is really slow.
How can i share variables across scripts?
My first scripts gets data from a midi controller and sends web-requests.
My second script is for LED strips (those run thanks to the same midi controller). Both script run in a "while true" loop.
I can't simply put them in the same script since every webrequest would slow the LEDs down. I am currently just sharing the variables via a json file.
If enough people ask for it i will post the whole code but i have been told not to do this
Considering the information you provided, meaning...
Both script run in a "while true" loop.
I can't simply put them in the same script since every webrequest would slow the LEDs down.
To me, you have 2 choices :
Use a client/server model. You have 2 machines. One acts as the server, and the second as the client. The server has a script with an infinite loop that consistently updates the data, and you would have an API that would just read and expose the current state of your file/database to the client. The client would be on another machine, and as I understand it, it would simply request the current data, and process it.
Make a single multiprocessing script. Each script would run on a separate 'thread' and would manage its own memory. As you also want to share variables between your two programs, you could pass as argument an object that would be shared between both your programs. See this resource to help you.
Note that there are more solutions to this. For instance, you're using a JSON file that you are consistently opening and closing (that is probably what takes the most time in your program). You could use a real Database that could handle being opened only once, and processed many times, while still being updated.
a Manager from multiprocessing lets you do this sort thing pretty easily
first I simplify your "midi controller and sends web-request" code down to something that just sleeps for random amounts of time and updates a variable in a managed dictionary:
from time import sleep
from random import random
def slow_fn(d):
i = 0
while True:
sleep(random() ** 2)
i += 1
d['value'] = i
next we simplify the "LED strip" control down to something that just prints to the screen:
from time import perf_counter
def fast_fn(d):
last = perf_counter()
while True:
sleep(0.05)
value = d.get('value')
now = perf_counter()
print(f'fast {value} {(now - last) * 1000:.2f}ms')
last = now
you can then run these functions in separate processes:
import multiprocessing as mp
with mp.Manager() as manager:
d = manager.dict()
procs = []
for fn in [slow_fn, fast_fn]:
p = mp.Process(target=fn, args=[d])
procs.append(p)
p.start()
for p in procs:
p.join()
the "fast" output happens regularly with no obvious visual pauses
I know there are plenty of "how to run a script from inside another script" answers, but I cannot seem to find how to queue one to start after the complete termination of the first. All of the variables need to be freed and the files closed and script dead before the second runs (or if it starts running within the live parent it needs to persist after the termination of the parent because that is when the file is accessible.)
I know it would be easier to not do this but I would like to know how.
I saw wait and polling suggested on other similar-ish questions but am new to either and not sure how they fit with my specific issue.
addition: Basically I need a parent script to spawn a child that lives on after the parent terminates/dies. But I cannot kill the parent myself or switch to the child. it has to end natrually as there are some more moving parts. Otherwise the below answer would be great!
What you are looking to do is to replace the running process with a new process, which is exactly what os.execl and the set of similar functions (os.execv, etc.) do.
This might be slightly more than is needed, but you might be able to use atexit:
import atexit
import signal
import time
from subprocess import check_output
def last_thing(*args, **kwargs):
def run_subprocess(*sig_args, **sig_kwargs):
print(sig_args, sig_kwargs)
print(check_output(['echo'] + list(args) + [k + v for k,v in kwargs.items()]))
return run_subprocess
def main_stuff():
atexit.register(last_thing('1', hello='world'))
signal.signal(signal.SIGINT, last_thing('2', hello='world'))
time.sleep(2)
main_stuff()
just a normal run of the script after 2 seconds:
03:51:30 (venv_ok) jmunsch#pop-os ±|PM-1371-jm-4 ✗|→ python test.py
() {}
b'1 helloworld\n'
exiting the script early by pressing ctrl + c before the 2 seconds is up:
03:51:37 (venv_ok) jmunsch#pop-os ±|PM-1371-jm-4 ✗|→ python test.py
^C(2, <frame object at 0x1483e98>) {}
b'2 helloworld\n'
() {}
b'1 helloworld\n'
I'm quite new at python and for a while I try to fight specific problem. I have function to listen and print radio frames.To do that I'm using NRF24 Lib and whole function is so easy. The point is that I run this function and from time to time I need to terminate it and again run. So in code it looks like
def recv():
radio.openWritingPipe(pipes[0])
radio.openReadingPipe(1, pipes[1])
radio.startListening()
radio.stopListening()
radio.printDetails()
radio.startListening()
while True:
pipe = [0]
while not radio.available(pipe):
time.sleep(10000/1000000.0)
recv_buffer = []
radio.read(recv_buffer)
print(recv_buffer)
I run this function from a server side and now I want to stop it and run again? There is it posible ? why I just cant recv.kill()? I read about threading, multiprocessing but all this didn't give me proper result.
How I run it:
from multiprocessing import Process
def request_handler(api: str, arg: dict) -> dict:
process_radio = Process(target=recv())
if api == 'start_radio':
process_radio.start()
...
elif api == 'stop_radio':
process_radio.terminate():
...
...
There is no way to stop a Python thread "from the outside." If the thread goes into a wait state (e.g. not running because it's waiting for radio.recv() to complete) there's nothing you can do.
Inside a single process the threads are autonomous, and the best you can do it so set a flag for the thread to action (by terminating) when it examines it.
As you have already discovered, it appears, you can terminate a subprocess, but you then have the issue of how the processes communicate with each other.
Your code and the test with it don't really give enough information (there appear to be several NRF24 implementations in Python) to debug the issues you report.
I'm trying to a program that executes a piece of code in such a way that the user can stop its execution at any time without stopping the main program. I thought I could do this using threading.Thread, but then I ran the following code in IDLE (Python 3.3):
from threading import *
import math
def f():
eval("math.factorial(1000000000)")
t = Thread(target = f)
t.start()
The last line doesn't return: I eventually restarted the shell. Is this a consequence of the Global Interpreter Lock, or am I doing something wrong? I didn't see anything specific to this problem in the threading documentation (http://docs.python.org/3/library/threading.html)
I tried to do the same thing using a process:
from multiprocessing import *
import math
def f():
eval("math.factorial(1000000000)")
p = Process(target = f)
p.start()
p.is_alive()
The last line returns False, even though I ran it only a few seconds after I started the process! Based on my processor usage, I am forced to conclude that the process never started in the first place. Can somebody please explain what I am doing wrong here?
Thread.start() never returns! Could this have something to do with the C implementation of the math library?
As #eryksun pointed out in the comment: math.factorial() is implemented as a C function that doesn't release GIL so no other Python code may run until it returns.
Note: multiprocessing version should work as is: each Python process has its own GIL.
factorial(1000000000) has hundreds millions of digits. Try import time; time.sleep(10) as dummy calculation instead.
If you have issues with multithreaded code in IDLE then try the same code from the command line, to make sure that the error persists.
If p.is_alive() returns False after p.start() is already called then it might mean that there is an error in f() function e.g., MemoryError.
On my machine, p.is_alive() returns True and one of cpus is at 100% if I paste your code from the question into Python shell.
Unrelated: remove wildcard imports such as from multiprocessing import *. They may shadow other names in your code so that you can't be sure what a given name means e.g., threading could define eval function (it doesn't but it could) with a similar but different semantics that might break your code silently.
I want my program to be able to handle ridiculous inputs from the user gracefully
If you pass user input directly to eval() then the user can do anything.
Is there any way to get a process to print, say, an error message without constructing a pipe or other similar structure?
It is an ordinary Python code:
print(message) # works
The difference is that if several processes run print() then the output might be garbled. You could use a lock to synchronize print() calls.