I have a script using DaemonRunner to create a daemon process with a pid file. The problem is that if someone tried to start it without stopping the currently running process, it will silently fail. What's the best way to detect an existing process and alert the user to stop it first? Is it as easy as checking the pidfile?
My code is similar to this example:
#!/usr/bin/python
import time
from daemon import runner
class App():
def __init__(self):
self.stdin_path = '/dev/null'
self.stdout_path = '/dev/tty'
self.stderr_path = '/dev/tty'
self.pidfile_path = '/tmp/foo.pid'
self.pidfile_timeout = 5
def run(self):
while True:
print("Howdy! Gig'em! Whoop!")
time.sleep(10)
app = App()
daemon_runner = runner.DaemonRunner(app)
daemon_runner.do_action()
To see my actual code, look at investor.py in:
https://github.com/jgillick/LendingClubAutoInvestor
since DaemonRunner handles its own lockfile, it's more wisely to refer to that one, to be sure you can't mess up. Maybe this block can help you with that:
Add
from lockfile import LockTimeout
to the beginning of the script and surround daemon_runner.doaction() like this
try:
daemon_runner.do_action()
except LockTimeout:
print "Error: couldn't aquire lock"
#you can exit here or try something else
This is the solution that I decided to use:
lockfile = runner.make_pidlockfile('/tmp/myapp.pid', 1)
if lockfile.is_locked():
print 'It looks like a daemon is already running!'
exit()
app = App()
daemon_runner = runner.DaemonRunner(app)
daemon_runner.do_action()
Is this a best practice or is there a better way?
Related
I am looking to build a daemon process that does some tasks when given some input. 99% of the time it lays silent on the background doing nothing and the tasks are short and few in number. How would I build the interface between two applications one of which constructs the task and the daemon that executes it?
I was thinking that the daemon might have a folder which i periodically checks. If there are some files in there, it reads it and follows instructions from there.
Would that work well or is there a better way?
EDIT: added example daemon code.
#!/usr/bin/python
import time
from daemon import runner
class Daemon():
def __init__(self):
self.stdin_path = '/dev/null'
self.stdout_path = '/dev/tty'
self.stderr_path = '/dev/tty'
self.pidfile_path = '/tmp/foo.pid'
self.pidfile_timeout = 5
self.task_dir = os.path.expanduser("~/.todo/daemon_tasks/")
def run(self):
while not time.sleep(1):
if len(os.listdir(self.task_dir)) == 0:
for task in os.listdir(self.task_dir):
self.process_task(task)
def process_task(self, task):
# input: filename
# output: void
# takes task and executes it according to instructions in the file
pass
if __name__ == '__main__':
app = Daemon()
daemon_runner = runner.DaemonRunner(app)
daemon_runner.do_action()
I would look into unix sockets of FIFOs as an option. This eliminates need for polling of some directory. Some SO link for help How to create special files of type socket?
I have yet another question about Python multiprocessing.
I have a module that creates a Process and just runs in a while True loop.
This module is meant to be enabled/disabled from another Python module.
That other module will import the first one once and is also run as a process.
How would I better implement this?
so for a reference:
#foo.py
def foo():
while True:
if enabled:
#do something
p = Process(target=foo)
p.start()
and imagine second module to be something like that:
#bar.py
import foo, time
def bar():
while True:
foo.enable()
time.sleep(10)
foo.disable()
Process(target=bar).start()
Constantly running a process checking for condition inside a loop seems like a waste, but I would gladly accept the solution that just lets me set the enabled value from outside.
Ideally I would prefer to be able to terminate and restart the process, again from outside of this module.
From my understanding, I would use a Queue to pass commands to the Process. If it is indeed just that, can someone show me how to set it up in a way that I can add something to the queue from a different module.
Can this even be easily done with Python or is it time to abandon hope and switch to something like C or Java
I purposed in comment two different approches :
using a shared variable from multiprocessing.Value
pause / resume the process with signals
Control by sharing a variable
def target_process_1(run_statement):
while True:
if run_statement.value:
print "I'm running !"
time.sleep(1)
def target_process_2(run_statement):
time.sleep(3)
print "Stoping"
run_statement.value = False
time.sleep(3)
print "Resuming"
run_statement.value = True
if __name__ == "__main__":
run_statement = Value("i", 1)
process_1 = Process(target=target_process_1, args=(run_statement,))
process_2 = Process(target=target_process_2, args=(run_statement,))
process_1.start()
process_2.start()
time.sleep(8)
process_1.terminate()
process_2.terminate()
Control by sending a signal
from multiprocessing import Process
import time
import os, signal
def target_process_1():
while True:
print "Running !"
time.sleep(1)
def target_process_2(target_pid):
time.sleep(3)
os.kill(target_pid, signal.SIGSTOP)
time.sleep(3)
os.kill(target_pid, signal.SIGCONT)
if __name__ == "__main__":
process_1 = Process(target=target_process_1)
process_1.start()
process_2 = Process(target=target_process_2, args=(process_1.pid,))
process_2.start()
time.sleep(8)
process_1.terminate()
process_2.terminate()
Side note: if possible do not run a while True.
EDIT: if you want to manage your process in two different files, supposing you want to use a control by sharing a variable, this is a way to do.
# file foo.py
from multiprocessing import Value, Process
import time
__all__ = ['start', 'stop', 'pause', 'resume']
_statement = None
_process = None
def _target(run_statement):
""" Target of the foo's process """
while True:
if run_statement.value:
print "I'm running !"
time.sleep(1)
def start():
global _process, _statement
_statement = Value("i", 1)
_process = Process(target=_target, args=(_statement,))
_process.start()
def stop():
global _process, _statement
_process.terminate()
_statement, _process = None, _process
def enable():
_statement.value = True
def disable():
_statement.value = False
I have a couple of different scripts that require opening a MongoDB instance that go something like this:
mongod = Popen(
["mongod", "--dbpath", '/path/to/db'],
)
#Do some stuff
mongod.terminate()
And this works great when the code I'm executing works, but while I'm tinkering, errors inevitably arise. Then the Mongod instance remains running, and the next time I attempt to run the script, it detects that and doesn't open a new one.
I can terminate the process from the command line, but this is somewhat tedious. Or I can wrap everything in a try loop, but for some of the scripts, I have to do this a bunch, since every function depends on every other one. Is there a more elegant way to force close the process even in the event of an error somewhere else in the code?
EDIT: Did some testing based on tdelaney's comment, it looks like when I run these scripts in Sublime text and en error is generated, the script doesn't actually finish - it hits the error and then waits with the mongod instance open... i think. Once I kill the process in the terminal, sublime text tells me "finished in X seconds with exit code1"
EDIT2: On Kirby's suggestion, tried:
def testing():
mongod = Popen(
["mongod", "--dbpath", '/Users/KBLaptop/computation/db/'],
)
#Stuff that generates error
mongod.terminate()
def cleanup():
for proc in subprocess._active[:]:
try: proc.terminate()
except: pass
atexit.register(cleanup)
testing()
The error in testing() seems to prevent anything from continuing, so the atexit never registers and the process keeps running. Am I missing something obvious?
If you're running under CPython, you can cheat and take advantage of Python's destructors:
class PopenWrapper(object):
def __del__(self):
if self._child_created:
self.terminate()
This is slightly ucky, though. My preference would be to atexit:
import atexit
mongod = Popen(...)
def cleanup():
for proc in subprocess._active[:]:
try: proc.terminate()
except: pass
atexit.register(cleanup)
Still slightly hack-ish, though.
EDIT: Try this:
from subprocess import Popen
import atexit
started = []
def auto_popen(*args, **kw):
p = Popen(*args, **kw)
started.append(p)
return p
def testing():
mongod = auto_popen(['blah blah'], shell=True)
assert 0
#Stuff that generates error
mongod.terminate()
def cleanup():
for proc in started:
if proc.poll() is None:
try: proc.kill()
except: pass
atexit.register(cleanup)
testing()
My problem is with os.system. Until it finishes formatting, GUI freezes, and I can't fix it.
class ImageDialog(QtGui.QMainWindow):
def __init__(self):
QtGui.QDialog.__init__(self)
self.ui = uic.loadUi("Recursos/main.ui",self)
self.connect(self.ui.proteger_Button, QtCore.SIGNAL("clicked()"),self,
QtCore.SLOT("protec()"))
#QtCore.pyqtSlot()
def protec(self):
self.USB = "G:"
comando = "format %s /fs:ntfs /q /v:test /y" %(self.USB)
os.system(comando)`
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
window = ImageDialog()
window.show()
sys.exit(app.exec_())
The simplest fix in your case is to add:
from threading import Thread
to your imports list, and then:
#QtCore.pyqtSlot()
def protec(self):
self.USB = "G:"
comando = "format %s /fs:ntfs /q /v:test /y" %(self.USB)
t = Thread(target = lambda: os.system(comando))
t.start()
This way the GUI thread will not get stuck waiting for the external process to finish.
You could probably remember the t's value and manage it in some way, so the number of threads running won't increase without limit, in case the external commands will hang, or run for a long time.
Note that it is not a good way to handle external processes. subprocess module is recommended for this.
What I'm trying to do is to write a script which would open an application only in process list. Meaning it would be "hidden". I don't even know if its possible in python.
If its not possible, I would settle for even a function that would allow for a program to be opened with python in a minimized state maybe something like this:
import subprocess
def startProgram():
subprocess.Hide(subprocess.Popen('C:\test.exe')) # I know this is wrong but you get the idea...
startProgram()
Someone suggested to use win32com.client but the thing is that the program that i want to launch doesn't have a COM server registered under the name.
Any ideas?
It's easy :)
Python Popen Accept STARTUPINFO Structure...
About STARTUPINFO Structure: https://msdn.microsoft.com/en-us/library/windows/desktop/ms686331(v=vs.85).aspx
Run Hidden:
import subprocess
def startProgram():
SW_HIDE = 0
info = subprocess.STARTUPINFO()
info.dwFlags = subprocess.STARTF_USESHOWWINDOW
info.wShowWindow = SW_HIDE
subprocess.Popen(r'C:\test.exe', startupinfo=info)
startProgram()
Run Minimized:
import subprocess
def startProgram():
SW_MINIMIZE = 6
info = subprocess.STARTUPINFO()
info.dwFlags = subprocess.STARTF_USESHOWWINDOW
info.wShowWindow = SW_MINIMIZE
subprocess.Popen(r'C:\test.exe', startupinfo=info)
startProgram()
You should use win32api and hide your window e.g. using win32gui.EnumWindows you can enumerate all top windows and hide your window
Here is a small example, you may do something like this:
import subprocess
import win32gui
import time
proc = subprocess.Popen(["notepad.exe"])
# lets wait a bit to app to start
time.sleep(3)
def enumWindowFunc(hwnd, windowList):
""" win32gui.EnumWindows() callback """
text = win32gui.GetWindowText(hwnd)
className = win32gui.GetClassName(hwnd)
#print hwnd, text, className
if text.find("Notepad") >= 0:
windowList.append((hwnd, text, className))
myWindows = []
# enumerate thru all top windows and get windows which are ours
win32gui.EnumWindows(enumWindowFunc, myWindows)
# now hide my windows, we can actually check process info from GetWindowThreadProcessId
# http://msdn.microsoft.com/en-us/library/ms633522(VS.85).aspx
for hwnd, text, className in myWindows:
win32gui.ShowWindow(hwnd, False)
# as our notepad is now hidden
# you will have to kill notepad in taskmanager to get past next line
proc.wait()
print "finished."
What is the purpose?
if you want a hidden(no window) process working in background, best way would be to write a windows service and start/stop it using usual window service mechanism. Windows service can be easily written in python e.g. here is part of my own service (it will not run without some modifications)
import os
import time
import traceback
import pythoncom
import win32serviceutil
import win32service
import win32event
import servicemanager
import jagteraho
class JagteRahoService (win32serviceutil.ServiceFramework):
_svc_name_ = "JagteRaho"
_svc_display_name_ = "JagteRaho (KeepAlive) Service"
_svc_description_ = "Used for keeping important services e.g. broadband connection up"
def __init__(self,args):
win32serviceutil.ServiceFramework.__init__(self,args)
self.stop = False
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
self.log('stopping')
self.stop = True
def log(self, msg):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_,msg))
def SvcDoRun(self):
self.log('folder %s'%os.getcwd())
self.ReportServiceStatus(win32service.SERVICE_RUNNING)
self.start()
def shouldStop(self):
return self.stop
def start(self):
try:
configFile = os.path.join(jagteraho.getAppFolder(), "jagteraho.cfg")
jagteraho.start_config(configFile, self.shouldStop)
except Exception,e:
self.log(" stopped due to eror %s [%s]" % (e, traceback.format_exc()))
self.ReportServiceStatus(win32service.SERVICE_STOPPED)
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(AppServerSvc)
and you can install it by
python svc_jagteraho.py--startup auto install
and run it by
python python svc_jagteraho.py start
I will be also be seen in services list e.g. services.msc will show it and you can start/stop it else you can use commandline
sc stop jagteraho
Run Hidden:
from subprocess_maximize import Popen
Popen("notepad.exe",show='hidden', priority=0)
Before the code above, use the following command:
pip install subprocess-maximize
If what is appearing is a terminal, redirect the process's stdout.