Related
I am sketching the architecture for a set of programs that share various interrelated objects stored in a database. I want one of the programs to act as a service which provides a higher level interface for operations on these objects, and the other programs to access the objects through that service.
I am currently aiming for Python and the Django framework as the technologies to implement that service with. I'm pretty sure I figure how to daemonize the Python program in Linux. However, it is an optional spec item that the system should support Windows. I have little experience with Windows programming and no experience at all with Windows services.
Is it possible to run a Python programs as a Windows service (i. e. run it automatically without user login)? I won't necessarily have to implement this part, but I need a rough idea how it would be done in order to decide whether to design along these lines.
Edit: Thanks for all the answers so far, they are quite comprehensive. I would like to know one more thing: How is Windows aware of my service? Can I manage it with the native Windows utilities? What is the equivalent of putting a start/stop script in /etc/init.d?
Yes you can. I do it using the pythoncom libraries that come included with ActivePython or can be installed with pywin32 (Python for Windows extensions).
This is a basic skeleton for a simple service:
import win32serviceutil
import win32service
import win32event
import servicemanager
import socket
class AppServerSvc (win32serviceutil.ServiceFramework):
_svc_name_ = "TestService"
_svc_display_name_ = "Test Service"
def __init__(self,args):
win32serviceutil.ServiceFramework.__init__(self,args)
self.hWaitStop = win32event.CreateEvent(None,0,0,None)
socket.setdefaulttimeout(60)
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
def SvcDoRun(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_,''))
self.main()
def main(self):
pass
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(AppServerSvc)
Your code would go in the main() method—usually with some kind of infinite loop that might be interrupted by checking a flag, which you set in the SvcStop method
The simplest way is to use the: NSSM - the Non-Sucking Service Manager. Just download and unzip to a location of your choosing. It's a self-contained utility, around 300KB (much less than installing the entire pywin32 suite just for this purpose) and no "installation" is needed. The zip contains a 64-bit and a 32-bit version of the utility. Either should work well on current systems (you can use the 32-bit version to manage services on 64-bit systems).
GUI approach
1 - install the python program as a service. Open a Win prompt as admin
c:\>nssm.exe install WinService
2 - On NSSM´s GUI console:
path: C:\Python27\Python27.exe
Startup directory: C:\Python27
Arguments: c:\WinService.py
3 - check the created services on services.msc
Scripting approach (no GUI)
This is handy if your service should be part of an automated, non-interactive procedure, that may be beyond your control, such as a batch or installer script. It is assumed that the commands are executed with administrative privileges.
For convenience the commands are described here by simply referring to the utility as nssm.exe. It is advisable, however, to refer to it more explicitly in scripting with its full path c:\path\to\nssm.exe, since it's a self-contained executable that may be located in a private path that the system is not aware of.
1. Install the service
You must specify a name for the service, the path to the proper Python executable, and the path to the script:
nssm.exe install ProjectService "c:\path\to\python.exe" "c:\path\to\project\app\main.py"
More explicitly:
nssm.exe install ProjectService
nssm.exe set ProjectService Application "c:\path\to\python.exe"
nssm.exe set ProjectService AppParameters "c:\path\to\project\app\main.py"
Alternatively you may want your Python app to be started as a Python module. One easy approach is to tell nssm that it needs to change to the proper starting directory, as you would do yourself when launching from a command shell:
nssm.exe install ProjectService "c:\path\to\python.exe" "-m app.main"
nssm.exe set ProjectService AppDirectory "c:\path\to\project"
This approach works well with virtual environments and self-contained (embedded) Python installs. Just make sure to have properly resolved any path issues in those environments with the usual methods. nssm has a way to set environment variables (e.g. PYTHONPATH) if needed, and can also launch batch scripts.
2. To start the service
nssm.exe start ProjectService
3. To stop the service
nssm.exe stop ProjectService
4. To remove the service, specify the confirm parameter to skip the interactive confirmation.
nssm.exe remove ProjectService confirm
Although I upvoted the chosen answer a couple of weeks back, in the meantime I struggled a lot more with this topic. It feels like having a special Python installation and using special modules to run a script as a service is simply the wrong way. What about portability and such?
I stumbled across the wonderful Non-sucking Service Manager, which made it really simple and sane to deal with Windows Services. I figured since I could pass options to an installed service, I could just as well select my Python executable and pass my script as an option.
I have not yet tried this solution, but I will do so right now and update this post along the process. I am also interested in using virtualenvs on Windows, so I might come up with a tutorial sooner or later and link to it here.
The simplest way to achieve this is to use native command sc.exe:
sc create PythonApp binPath= "C:\Python34\Python.exe --C:\tmp\pythonscript.py"
References:
https://technet.microsoft.com/en-us/library/cc990289(v=ws.11).aspx
When creating a service with sc.exe how to pass in context parameters?
There are a couple alternatives for installing as a service virtually any Windows executable.
Method 1: Use instsrv and srvany from rktools.exe
For Windows Home Server or Windows Server 2003 (works with WinXP too), the Windows Server 2003 Resource Kit Tools comes with utilities that can be used in tandem for this, called instsrv.exe and srvany.exe. See this Microsoft KB article KB137890 for details on how to use these utils.
For Windows Home Server, there is a great user friendly wrapper for these utilities named aptly "Any Service Installer".
Method 2: Use ServiceInstaller for Windows NT
There is another alternative using ServiceInstaller for Windows NT (download-able here) with python instructions available. Contrary to the name, it works with both Windows 2000 and Windows XP as well. Here are some instructions for how to install a python script as a service.
Installing a Python script
Run ServiceInstaller to create a new
service. (In this example, it is
assumed that python is installed at
c:\python25)
Service Name : PythonTest
Display Name : PythonTest
Startup : Manual (or whatever you like)
Dependencies : (Leave blank or fill to fit your needs)
Executable : c:\python25\python.exe
Arguments : c:\path_to_your_python_script\test.py
Working Directory : c:\path_to_your_python_script
After installing, open the Control
Panel's Services applet, select and
start the PythonTest service.
After my initial answer, I noticed there were closely related Q&A already posted on SO. See also:
Can I run a Python script as a service (in Windows)? How?
How do I make Windows aware of a service I have written in Python?
Step by step explanation how to make it work :
1- First create a python file according to the basic skeleton mentioned above. And save it to a path for example : "c:\PythonFiles\AppServerSvc.py"
import win32serviceutil
import win32service
import win32event
import servicemanager
import socket
class AppServerSvc (win32serviceutil.ServiceFramework):
_svc_name_ = "TestService"
_svc_display_name_ = "Test Service"
def __init__(self,args):
win32serviceutil.ServiceFramework.__init__(self,args)
self.hWaitStop = win32event.CreateEvent(None,0,0,None)
socket.setdefaulttimeout(60)
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
def SvcDoRun(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_,''))
self.main()
def main(self):
# Your business logic or call to any class should be here
# this time it creates a text.txt and writes Test Service in a daily manner
f = open('C:\\test.txt', 'a')
rc = None
while rc != win32event.WAIT_OBJECT_0:
f.write('Test Service \n')
f.flush()
# block for 24*60*60 seconds and wait for a stop event
# it is used for a one-day loop
rc = win32event.WaitForSingleObject(self.hWaitStop, 24 * 60 * 60 * 1000)
f.write('shut down \n')
f.close()
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(AppServerSvc)
2 - On this step we should register our service.
Run command prompt as administrator and type as:
sc create TestService binpath= "C:\Python36\Python.exe c:\PythonFiles\AppServerSvc.py" DisplayName= "TestService" start= auto
the first argument of binpath is the path of python.exe
second argument of binpath is the path of your python file that we created already
Don't miss that you should put one space after every "=" sign.
Then if everything is ok, you should see
[SC] CreateService SUCCESS
Now your python service is installed as windows service now. You can see it in Service Manager and registry under :
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TestService
3- Ok now. You can start your service on service manager.
You can execute every python file that provides this service skeleton.
pysc: Service Control Manager on Python
Example script to run as a service taken from pythonhosted.org:
from xmlrpc.server import SimpleXMLRPCServer
from pysc import event_stop
class TestServer:
def echo(self, msg):
return msg
if __name__ == '__main__':
server = SimpleXMLRPCServer(('127.0.0.1', 9001))
#event_stop
def stop():
server.server_close()
server.register_instance(TestServer())
server.serve_forever()
Create and start service
import os
import sys
from xmlrpc.client import ServerProxy
import pysc
if __name__ == '__main__':
service_name = 'test_xmlrpc_server'
script_path = os.path.join(
os.path.dirname(__file__), 'xmlrpc_server.py'
)
pysc.create(
service_name=service_name,
cmd=[sys.executable, script_path]
)
pysc.start(service_name)
client = ServerProxy('http://127.0.0.1:9001')
print(client.echo('test scm'))
Stop and delete service
import pysc
service_name = 'test_xmlrpc_server'
pysc.stop(service_name)
pysc.delete(service_name)
pip install pysc
nssm in python 3+
(I converted my .py file to .exe with pyinstaller)
nssm:
as said before
run nssm install {ServiceName}
On NSSM´s console:
path: path\to\your\program.exe
Startup directory: path\to\your\ #same as the path but without your program.exe
Arguments: empty
If you don't want to convert your project to .exe
create a .bat file with python {{your python.py file name}}
and set the path to the .bat file
I started hosting as a service with pywin32.
Everything was well but I met the problem that service was not able to start within 30 seconds (default timeout for Windows) on system startup. It was critical for me because Windows startup took place simultaneous on several virtual machines hosted on one physical machine, and IO load was huge.
Error messages were:
Error 1053: The service did not respond to the start or control request in a timely fashion.
Error 7009: Timeout (30000 milliseconds) waiting for the <ServiceName> service to connect.
I fought a lot with pywin, but ended up with using NSSM as it was proposed in this answer. It was very easy to migrate to it.
A complete pywin32 example using loop or subthread
After working on this on and off for a few days, here is the answer I would have wished to find, using pywin32 to keep it nice and self contained.
This is complete working code for one loop-based and one thread-based solution.
It may work on both python 2 and 3, although I've only tested the latest version on 2.7 and Win7. The loop should be good for polling code, and the tread should work with more server-like code. It seems to work nicely with the waitress wsgi server that does not have a standard way to shut down gracefully.
I would also like to note that there seems to be loads of examples out there, like this that are almost useful, but in reality misleading, because they have cut and pasted other examples blindly. I could be wrong. but why create an event if you never wait for it?
That said I still feel I'm on somewhat shaky ground here, especially with regards to how clean the exit from the thread version is, but at least I believe there are nothing misleading here.
To run simply copy the code to a file and follow the instructions.
update:
Use a simple flag to terminate thread. The important bit is that "thread done" prints.
For a more elaborate example exiting from an uncooperative server thread see my post about the waitress wsgi server.
# uncomment mainthread() or mainloop() call below
# run without parameters to see HandleCommandLine options
# install service with "install" and remove with "remove"
# run with "debug" to see print statements
# with "start" and "stop" watch for files to appear
# check Windows EventViever for log messages
import socket
import sys
import threading
import time
from random import randint
from os import path
import servicemanager
import win32event
import win32service
import win32serviceutil
# see http://timgolden.me.uk/pywin32-docs/contents.html for details
def dummytask_once(msg='once'):
fn = path.join(path.dirname(__file__),
'%s_%s.txt' % (msg, randint(1, 10000)))
with open(fn, 'w') as fh:
print(fn)
fh.write('')
def dummytask_loop():
global do_run
while do_run:
dummytask_once(msg='loop')
time.sleep(3)
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
global do_run
do_run = True
print('thread start\n')
dummytask_loop()
print('thread done\n')
def exit(self):
global do_run
do_run = False
class SMWinservice(win32serviceutil.ServiceFramework):
_svc_name_ = 'PyWinSvc'
_svc_display_name_ = 'Python Windows Service'
_svc_description_ = 'An example of a windows service in Python'
#classmethod
def parse_command_line(cls):
win32serviceutil.HandleCommandLine(cls)
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.stopEvt = win32event.CreateEvent(None, 0, 0, None) # create generic event
socket.setdefaulttimeout(60)
def SvcStop(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STOPPED,
(self._svc_name_, ''))
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.stopEvt) # raise event
def SvcDoRun(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_, ''))
# UNCOMMENT ONE OF THESE
# self.mainthread()
# self.mainloop()
# Wait for stopEvt indefinitely after starting thread.
def mainthread(self):
print('main start')
self.server = MyThread()
self.server.start()
print('wait for win32event')
win32event.WaitForSingleObject(self.stopEvt, win32event.INFINITE)
self.server.exit()
print('wait for thread')
self.server.join()
print('main done')
# Wait for stopEvt event in loop.
def mainloop(self):
print('loop start')
rc = None
while rc != win32event.WAIT_OBJECT_0:
dummytask_once()
rc = win32event.WaitForSingleObject(self.stopEvt, 3000)
print('loop done')
if __name__ == '__main__':
SMWinservice.parse_command_line()
This answer is plagiarizer from several sources on StackOverflow - most of them above, but I've forgotten the others - sorry. It's simple and scripts run "as is". For releases you test you script, then copy it to the server and Stop/Start the associated service. And it should work for all scripting languages (Python, Perl, node.js), plus batch scripts such as GitBash, PowerShell, even old DOS bat scripts.
pyGlue is the glue that sits between Windows Services and your script.
'''
A script to create a Windows Service, which, when started, will run an executable with the specified parameters.
Optionally, you can also specify a startup directory
To use this script you MUST define (in class Service)
1. A name for your service (short - preferably no spaces)
2. A display name for your service (the name visibile in Windows Services)
3. A description for your service (long details visible when you inspect the service in Windows Services)
4. The full path of the executable (usually C:/Python38/python.exe or C:WINDOWS/System32/WindowsPowerShell/v1.0/powershell.exe
5. The script which Python or PowerShell will run(or specify None if your executable is standalone - in which case you don't need pyGlue)
6. The startup directory (or specify None)
7. Any parameters for your script (or for your executable if you have no script)
NOTE: This does not make a portable script.
The associated '_svc_name.exe' in the dist folder will only work if the executable,
(and any optional startup directory) actually exist in those locations on the target system
Usage: 'pyGlue.exe [options] install|update|remove|start [...]|stop|restart [...]|debug [...]'
Options for 'install' and 'update' commands only:
--username domain\\username : The Username the service is to run under
--password password : The password for the username
--startup [manual|auto|disabled|delayed] : How the service starts, default = manual
--interactive : Allow the service to interact with the desktop.
--perfmonini file: .ini file to use for registering performance monitor data
--perfmondll file: .dll file to use when querying the service for performance data, default = perfmondata.dll
Options for 'start' and 'stop' commands only:
--wait seconds: Wait for the service to actually start or stop.
If you specify --wait with the 'stop' option, the service and all dependent services will be stopped,
each waiting the specified period.
'''
# Import all the modules that make life easy
import servicemanager
import socket
import sys
import win32event
import win32service
import win32serviceutil
import win32evtlogutil
import os
from logging import Formatter, Handler
import logging
import subprocess
# Define the win32api class
class Service (win32serviceutil.ServiceFramework):
# The following variable are edited by the build.sh script
_svc_name_ = "TestService"
_svc_display_name_ = "Test Service"
_svc_description_ = "Test Running Python Scripts as a Service"
service_exe = 'c:/Python27/python.exe'
service_script = None
service_params = []
service_startDir = None
# Initialize the service
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
self.configure_logging()
socket.setdefaulttimeout(60)
# Configure logging to the WINDOWS Event logs
def configure_logging(self):
self.formatter = Formatter('%(message)s')
self.handler = logHandler()
self.handler.setFormatter(self.formatter)
self.logger = logging.getLogger()
self.logger.addHandler(self.handler)
self.logger.setLevel(logging.INFO)
# Stop the service
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
# Run the service
def SvcDoRun(self):
self.main()
# This is the service
def main(self):
# Log that we are starting
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE, servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_, ''))
# Fire off the real process that does the real work
logging.info('%s - about to call Popen() to run %s %s %s', self._svc_name_, self.service_exe, self.service_script, self.service_params)
self.process = subprocess.Popen([self.service_exe, self.service_script] + self.service_params, shell=False, cwd=self.service_startDir)
logging.info('%s - started process %d', self._svc_name_, self.process.pid)
# Wait until WINDOWS kills us - retrigger the wait for stop every 60 seconds
rc = None
while rc != win32event.WAIT_OBJECT_0:
rc = win32event.WaitForSingleObject(self.hWaitStop, (1 * 60 * 1000))
# Shut down the real process and exit
logging.info('%s - is terminating process %d', self._svc_name_, self.process.pid)
self.process.terminate()
logging.info('%s - is exiting', self._svc_name_)
class logHandler(Handler):
'''
Emit a log record to the WINDOWS Event log
'''
def emit(self, record):
servicemanager.LogInfoMsg(record.getMessage())
# The main code
if __name__ == '__main__':
'''
Create a Windows Service, which, when started, will run an executable with the specified parameters.
'''
# Check that configuration contains valid values just in case this service has accidentally
# been moved to a server where things are in different places
if not os.path.isfile(Service.service_exe):
print('Executable file({!s}) does not exist'.format(Service.service_exe), file=sys.stderr)
sys.exit(0)
if not os.access(Service.service_exe, os.X_OK):
print('Executable file({!s}) is not executable'.format(Service.service_exe), file=sys.stderr)
sys.exit(0)
# Check that any optional startup directory exists
if (Service.service_startDir is not None) and (not os.path.isdir(Service.service_startDir)):
print('Start up directory({!s}) does not exist'.format(Service.service_startDir), file=sys.stderr)
sys.exit(0)
if len(sys.argv) == 1:
servicemanager.Initialize()
servicemanager.PrepareToHostSingle(Service)
servicemanager.StartServiceCtrlDispatcher()
else:
# install/update/remove/start/stop/restart or debug the service
# One of those command line options must be specified
win32serviceutil.HandleCommandLine(Service)
Now there's a bit of editing and you don't want all your services called 'pyGlue'. So there's a script (build.sh) to plug in the bits and create a customized 'pyGlue' and create an '.exe'. It is this '.exe' which gets installed as a Windows Service. Once installed you can set it to run automatically.
#!/bin/sh
# This script build a Windows Service that will install/start/stop/remove a service that runs a script
# That is, executes Python to run a Python script, or PowerShell to run a PowerShell script, etc
if [ $# -lt 6 ]; then
echo "Usage: build.sh Name Display Description Executable Script StartupDir [Params]..."
exit 0
fi
name=$1
display=$2
desc=$3
exe=$4
script=$5
startDir=$6
shift; shift; shift; shift; shift; shift
params=
while [ $# -gt 0 ]; do
if [ "${params}" != "" ]; then
params="${params}, "
fi
params="${params}'$1'"
shift
done
cat pyGlue.py | sed -e "s/pyGlue/${name}/g" | \
sed -e "/_svc_name_ =/s?=.*?= '${name}'?" | \
sed -e "/_svc_display_name_ =/s?=.*?= '${display}'?" | \
sed -e "/_svc_description_ =/s?=.*?= '${desc}'?" | \
sed -e "/service_exe =/s?=.*?= '$exe'?" | \
sed -e "/service_script =/s?=.*?= '$script'?" | \
sed -e "/service_params =/s?=.*?= [${params}]?" | \
sed -e "/service_startDir =/s?=.*?= '${startDir}'?" > ${name}.py
cxfreeze ${name}.py --include-modules=win32timezone
Installation - copy the '.exe' the server and the script to the specified folder. Run the '.exe', as Administrator, with the 'install' option. Open Windows Services, as Adminstrator, and start you service. For upgrade, just copy the new version of the script and Stop/Start the service.
Now every server is different - different installations of Python, different folder structures. I maintain a folder for every server, with a copy of pyGlue.py and build.sh. And I create a 'serverBuild.sh' script for rebuilding all the service on that server.
# A script to build all the script based Services on this PC
sh build.sh AutoCode 'AutoCode Medical Documents' 'Autocode Medical Documents to SNOMED_CT and AIHW codes' C:/Python38/python.exe autocode.py C:/Users/russell/Documents/autocoding -S -T
The accepted answer using win32serviceutil works but is complicated and makes debugging and changes harder. It is far easier to use NSSM (the Non-Sucking Service Manager). You write and comfortably debug a normal python program and when it finally works you use NSSM to install it as a service in less than a minute:
From an elevated (admin) command prompt you run nssm.exe install NameOfYourService and you fill-in these options:
path: (the path to python.exe e.g. C:\Python27\Python.exe)
Arguments: (the path to your python script, e.g. c:\path\to\program.py)
By the way, if your program prints useful messages that you want to keep in a log file NSSM can also handle this and a lot more for you.
This doesn't answer the original question, but might help other people that want to automatically start a Python script at Windows startup:
Have a look at the Windows Task Scheduler instead, it is way easier if you just want to start a script after boot without all the service functionality of a Windows Service.
Create a new task, select "At startup" as trigger, "Start program" as action with "C:\Python39\python.exe" as the program (or wherever your python.exe is) and the full path to your script ("C:...\my_dir\xyz.py") as argument (you can use " if the path contains spaces).
You can also select the path of your script (without the .py file, e.g. "C:...\my_dir") for "start in" if you use relative paths in your script, e.g. for logging.
https://www.chrisumbel.com/article/windows_services_in_python
Follow up the PySvc.py
changing the dll folder
I know this is old but I was stuck on this forever. For me, this specific problem was solved by copying this file - pywintypes36.dll
From -> Python36\Lib\site-packages\pywin32_system32
To -> Python36\Lib\site-packages\win32
setx /M PATH "%PATH%;C:\Users\user\AppData\Local\Programs\Python\Python38-32;C:\Users\user\AppData\Local\Programs\Python\Python38-32\Scripts;C:\Users\user\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\pywin32_system32;C:\Users\user\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\win32
changing the path to python folder by
cd C:\Users\user\AppData\Local\Programs\Python\Python38-32
NET START PySvc
NET STOP PySvc
Im pretty new to programming and Python and i have crated an API using FastAPI that i need to run on Windows, i want to run it as a service. The server will start and run fine, but when i try to stop it Windows throws an error and i think its because im not "stopping" uvicorn and i cant figure out how to. I've been googling and tried to run Uvicorn in a Daemon thread because i read that daemon threads will shutdown when the main thread exits, but that did not help either.
Im converting the app to exe using cx_Freeze. Could anyone point me in the right direction?
Im using this Template for cx_Freeze: https://github.com/marcelotduarte/cx_Freeze/tree/main/cx_Freeze/samples/service
Im on Windows 10, Python 3.9.1, Uvicorn 0.13.3, cx_Freeze 6.5
ServiceHandler.py
import os
import sys
import cx_Logging
import api
import logging
import threading
from uvicorn import Config, Server
logging.basicConfig(
filename = os.path.join(os.path.dirname(sys.executable), "log.txt"),
level = logging.DEBUG,
format = '[API] %(levelname)-7.7s %(message)s'
)
class Handler:
# no parameters are permitted; all configuration should be placed in the
# configuration file and handled in the initialize() method
def __init__(self):
self.stopEvent = threading.Event()
self.stopRequestedEvent = threading.Event()
# called when the service is starting
def initialize(self, configFileName):
self.directory = os.path.dirname(sys.executable)
cx_Logging.StartLogging(os.path.join(self.directory, "testing.log"), cx_Logging.DEBUG)
#pass
# called when the service is starting immediately after initialize()
# use this to perform the work of the service; don't forget to set or check
# for the stop event or the service GUI will not respond to requests to
# stop the service
def run(self):
cx_Logging.Debug("stdout=%r", sys.stdout)
sys.stdout = open(os.path.join(self.directory, "stdout.log"), "a")
sys.stderr = open(os.path.join(self.directory, "stderr.log"), "a")
self.main()
self.stopRequestedEvent.wait()
self.stopEvent.set()
# called when the service is being stopped by the service manager GUI
def stop(self):
try:
logging.debug("Stopping Service")
self.stopRequestedEvent.set()
self.stopEvent.wait()
# How to stop the server???
except Exception as e:
logging.error(e)
def main(self):
try:
self.config = Config(app=api.app, host="0.0.0.0", port=8004, reload=False)
self.app_server = Server(self.config)
self.app_server.install_signal_handlers = lambda: None # Need this line, or the server wont start
self.app_server.run()
except Exception as e:
logging.error(e)
setup.py
from cx_Freeze import setup, Executable
options = {
"build_exe": {
"packages": ["uvicorn", "fastapi", "pydantic", "threading"],
"includes": ["ServiceHandler", "cx_Logging", "ipaddress", "colorsys"],
"excludes": ["tkinter"],
}
}
executables = [
Executable(
"Config.py",
base="Win32Service",
target_name="api.exe",
)
]
setup(
name="TestService",
version="0.1",
description="Sample Windows serice",
executables=executables,
options=options,
)
I've also read this; https://github.com/encode/uvicorn/issues/742
But my knowledge is limited, so i dont quite understand how to implement it in my app?
So i got the service working.
Uvicorn/FastAPI will now start and stop as a Windows service, i dont know if there are any potential drawbacks, but here is the working code;
import os
import sys
import cx_Logging
import api
import logging
import threading
from uvicorn import Config, Server
logging.basicConfig(
filename = os.path.join(os.path.dirname(sys.executable), "log.txt"),
level = logging.DEBUG,
format = '[API] %(levelname)-7.7s %(message)s'
)
#My class for creating and running Uvicorn in a thread
class AppServer:
def __init__(self, app, host: str = "0.0.0.0", port: int = 8004, reload: bool = False):
self.config = Config(app=app, host=host, port=port, reload=reload)
self.server = Server(self.config)
self.server.install_signal_handlers = lambda: None # Need this line, or the server wont start
self.proc = None
def run(self):
self.server.run()
def start(self):
self.proc = threading.Thread(target=self.run, name="Test", args=())
self.proc.setDaemon(True)
self.proc.start()
def stop(self):
if self.proc:
self.proc.join(0.25)
class Handler:
# no parameters are permitted; all configuration should be placed in the
# configuration file and handled in the initialize() method
def __init__(self):
self.stopEvent = threading.Event()
self.stopRequestedEvent = threading.Event()
# called when the service is starting
def initialize(self, configFileName):
self.directory = os.path.dirname(sys.executable)
cx_Logging.StartLogging(os.path.join(self.directory, "testing.log"), cx_Logging.DEBUG)
#pass
# called when the service is starting immediately after initialize()
# use this to perform the work of the service; don't forget to set or check
# for the stop event or the service GUI will not respond to requests to
# stop the service
def run(self):
cx_Logging.Debug("stdout=%r", sys.stdout)
sys.stdout = open(os.path.join(self.directory, "stdout.log"), "a")
sys.stderr = open(os.path.join(self.directory, "stderr.log"), "a")
self.main()
self.stopRequestedEvent.wait()
self.stopEvent.set()
# called when the service is being stopped by the service manager GUI
def stop(self):
try:
logging.debug("Stopping Service")
self.server.stop()
self.stopRequestedEvent.set()
self.stopEvent.wait()
# How to stop the server???
except Exception as e:
logging.error(e)
def main(self):
try:
logging.debug("Starting server,,,")
self.server = AppServer(app=api.app)
self.server.start()
except Exception as e:
logging.error(e)
I got the answer here;
How to use FastAPI and uvicorn.run without blocking the thread?
I'm trying to create a Windows Service to launch Celery. I have come across an article that does it using Task Scheduler. However it seems to launch numerous celery instances and keeps eating up memory till the machine dies. Is there any way to launch it as a Windows service?
I got the answer from another website. Celeryd (daemon service for Celery) runs as a paster application, searching for 'Paster Windows Service' lead me here. It describes how to run a Pylons application as a Windows Service. Being new to paster framework and hosting python web services, it didn't cross my mind to check it at first. But that solution works for Celery with a slight change here and there in the script.
I've modified the script to make it easier for modifying Celery settings. The essential changes are:
Create an INI file with the settings for Celery service (shown below)
Create a python script to create a Windows service.
INI file settings (celeryd.ini):
[celery:service]
service_name = CeleryService
service_display_name = Celery Service
service_description = WSCGI Windows Celery Service
service_logfile = celeryd.log
Python script to create Windows Service (CeleryService.py):
"""
The most basic (working) Windows service possible.
Requires Mark Hammond's pywin32 package.
Most of the code was taken from a CherryPy 2.2 example of how to set up a service
"""
import pkg_resources
import win32serviceutil
from paste.script.serve import ServeCommand as Server
import os, sys
import ConfigParser
import win32service
import win32event
SCRIPT_DIR = os.path.abspath(os.path.dirname(__file__))
INI_FILE = 'celeryd.ini'
SERV_SECTION = 'celery:service'
SERV_NAME = 'service_name'
SERV_DISPLAY_NAME = 'service_display_name'
SERV_DESC = 'service_description'
SERV_LOG_FILE = 'service_logfile'
SERV_APPLICATION = 'celeryd'
SERV_LOG_FILE_VAR = 'CELERYD_LOG_FILE'
# Default Values
SERV_NAME_DEFAULT = 'CeleryService'
SERV_DISPLAY_NAME_DEFAULT = 'Celery Service'
SERV_DESC_DEFAULT = 'WSCGI Windows Celery Service'
SERV_LOG_FILE_DEFAULT = r'D:\logs\celery.log'
class DefaultSettings(object):
def __init__(self):
if SCRIPT_DIR:
os.chdir(SCRIPT_DIR)
# find the ini file
self.ini = os.path.join(SCRIPT_DIR,INI_FILE)
# create a config parser opject and populate it with the ini file
c = ConfigParser.SafeConfigParser()
c.read(self.ini)
self.c = c
def getDefaults(self):
'''
Check for and get the default settings
'''
if (
(not self.c.has_section(SERV_SECTION)) or
(not self.c.has_option(SERV_SECTION, SERV_NAME)) or
(not self.c.has_option(SERV_SECTION, SERV_DISPLAY_NAME)) or
(not self.c.has_option(SERV_SECTION, SERV_DESC)) or
(not self.c.has_option(SERV_SECTION, SERV_LOG_FILE))
):
print 'setting defaults'
self.setDefaults()
service_name = self.c.get(SERV_SECTION, SERV_NAME)
service_display_name = self.c.get(SERV_SECTION, SERV_DISPLAY_NAME)
service_description = self.c.get(SERV_SECTION, SERV_DESC)
iniFile = self.ini
service_logfile = self.c.get(SERV_SECTION, SERV_LOG_FILE)
return service_name, service_display_name, service_description, iniFile, service_logfile
def setDefaults(self):
'''
set and add the default setting to the ini file
'''
if not self.c.has_section(SERV_SECTION):
self.c.add_section(SERV_SECTION)
self.c.set(SERV_SECTION, SERV_NAME, SERV_NAME_DEFAULT)
self.c.set(SERV_SECTION, SERV_DISPLAY_NAME, SERV_DISPLAY_NAME_DEFAULT)
self.c.set(SERV_SECTION, SERV_DESC, SERV_DESC_DEFAULT)
self.c.set(SERV_SECTION, SERV_LOG_FILE, SERV_LOG_FILE_DEFAULT)
cfg = file(self.ini, 'wr')
self.c.write(cfg)
cfg.close()
print '''
you must set the celery:service section service_name, service_display_name,
and service_description options to define the service
in the %s file
''' % self.ini
sys.exit()
class CeleryService(win32serviceutil.ServiceFramework):
"""NT Service."""
d = DefaultSettings()
service_name, service_display_name, service_description, iniFile, logFile = d.getDefaults()
_svc_name_ = service_name
_svc_display_name_ = service_display_name
_svc_description_ = service_description
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
# create an event that SvcDoRun can wait on and SvcStop
# can set.
self.stop_event = win32event.CreateEvent(None, 0, 0, None)
def SvcDoRun(self):
os.chdir(SCRIPT_DIR)
s = Server(SERV_APPLICATION)
os.environ[SERV_LOG_FILE_VAR] = self.logFile
s.run([self.iniFile])
win32event.WaitForSingleObject(self.stop_event, win32event.INFINITE)
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
#win32event.SetEvent(self.stop_event)
self.ReportServiceStatus(win32service.SERVICE_STOPPED)
sys.exit()
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(CeleryService)
To install the service run python CeleryService.py install and then python CeleryService.py start to start the service. NOTE: These commands should be run in command-line with administrator rights.
If the service needs to be removed, run python CeleryService.py remove.
I was trying to host Celery as part of enhancing my RhodeCode installation. This solution seems to work. Hope this will help someone.
The accepted answer does not apply for running celery with a Django application. But it inspired me to come up with a solution for running celery as a Windows service with Django. Note that the following is for Django projects only. It may work with other applications with some modifications.
The following discussion assumes Python >= 3.6 and RabbitMQ are already installed, and rabbitmq-server is running on localhost.
Create a file celery_service.py (or whatever you like) inside your Django project's top level folder, same level as manage.py, with the following content:
'''Usage : python celery_service.py install (start / stop / remove)
Run celery as a Windows service
'''
import win32service
import win32serviceutil
import win32api
import win32con
import win32event
import subprocess
import sys
import os
from pathlib import Path
import shlex
import logging
import time
# The directory for celery.log and celery_service.log
# Default: the directory of this script
INSTDIR = Path(__file__).parent
# The path of python Scripts
# Usually it is in path_to/venv/Scripts.
# If it is already in system PATH, then it can be set as ''
PYTHONSCRIPTPATH = INSTDIR / 'venvcelery/Scripts'
# The directory name of django project
# Note: it is the directory at the same level of manage.py
# not the parent directory
PROJECTDIR = 'proj'
logging.basicConfig(
filename = INSTDIR / 'celery_service.log',
level = logging.DEBUG,
format = '[%(asctime)-15s: %(levelname)-7.7s] %(message)s'
)
class CeleryService(win32serviceutil.ServiceFramework):
_svc_name_ = "Celery"
_svc_display_name_ = "Celery Distributed Task Queue Service"
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
def SvcStop(self):
logging.info('Stopping {name} service ...'.format(name=self._svc_name_))
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
self.ReportServiceStatus(win32service.SERVICE_STOPPED)
sys.exit()
def SvcDoRun(self):
logging.info('Starting {name} service ...'.format(name=self._svc_name_))
os.chdir(INSTDIR) # so that proj worker can be found
logging.info('cwd: ' + os.getcwd())
self.ReportServiceStatus(win32service.SERVICE_RUNNING)
command = '"{celery_path}" -A {proj_dir} worker -f "{log_path}" -l info -P eventlet'.format(
celery_path=PYTHONSCRIPTPATH / 'celery.exe',
proj_dir=PROJECTDIR,
log_path=INSTDIR / 'celery.log')
logging.info('command: ' + command)
args = shlex.split(command)
proc = subprocess.Popen(args)
logging.info('pid: {pid}'.format(pid=proc.pid))
self.timeout = 3000
while True:
rc = win32event.WaitForSingleObject(self.hWaitStop, self.timeout)
if rc == win32event.WAIT_OBJECT_0:
# stop signal encountered
# terminate process 'proc'
PROCESS_TERMINATE = 1
handle = win32api.OpenProcess(PROCESS_TERMINATE, False, proc.pid)
win32api.TerminateProcess(handle, -1)
win32api.CloseHandle(handle)
break
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(CeleryService)
Before the script can be run, you need to
Optionally create a python virtual environment e.g. 'venvcelery'.
Install the following requirements:
django>=2.0.0
sqlalchemy>=1.0.14
celery>=4.3.0,<5.0
pywin32>=227
eventlet>=0.25
Fix pywin32 pywintypes36.dll location. ref
Correctly set PYTHONSCRIPTPATH and PROJECTDIR in celery_service.py
PYTHONSCRIPTPATH is usually the "Scripts" folder under your python's installation path or current virtual environment
PROJECTDIR is the directory name of the Django project.
It is the directory at the same level of manage.py, not the parent directory.
Now you can install / start / stop / remove the service with:
python celery_service.py install
python celery_service.py start
python celery_service.py stop
python celery_service.py remove
I created a demo Django project with celery running as a Windows service:
https://github.com/azalea/django_celery_windows_service
In case you are interested in a running example.
Note: this is an updated version assuming Python >= 3.6, Django 2.2 and Celery 4.
An older version with Python 2.7, Django 1.6 and Celery 3 can be viewed in edit history.
#azalea 's answer helped me a lot, but one thing I like to highlight here is, the service (celery_service.py) needs to be installed with your user/password, otherwise, when you run subprocess.Popen(args) in SvcDoRun() function, nothing will happen as there will be a permission issue. To set the user/password, you can choose one of two methods:
Using command line:
python33 .\celeryService1.py --username .\USERNAME --password PASSWORD
Go to Computer Management(local) > Services and Applications > Services, find your server (in #azalea's example, it is "Celery Distributed Task Queue Service"), and right click to open Properties page, input "This account" in Log On tab
A good project here but didn't succeed to use it :
Link to the GitHub of the django-windows-tools.
It gave me a timeout at the last command line. Doesn't have enough time to search why.
The package allow settings FastCGI, Celery and Static files of a Django project on IIS.
Thanks to Azalea as this led me to the path in being able to create 2 windows services with Celery 4 on Windows.
One to be able to start/stop multiple workers T
Second to be able to start/stop the beat service and tidy up the pid using Celery 4.
Only caveat in this that I do not have a solution for is you can not restart the workers as you need to ensure the spawned processes for multiple are stopped before starting back up.
Workers.py:
'''Usage : python celery_service.py install (start / stop / remove)
Run celery as a Windows service
'''
import win32service
import win32serviceutil
import win32api
import win32con
import win32event
import subprocess
import sys
import os
import shlex
import logging
import time
# The directory for celery_worker.log and celery_worker_service.log
# Default: the directory of this script
INSTDIR = 'X:\Application\Project'
LOGDIR = 'X:\Application\LogFiles'
# The path of python Scripts
# Usually it is in PYTHON_INSTALL_DIR/Scripts. e.g.
# r'C:\Python27\Scripts'
# If it is already in system PATH, then it can be set as ''
PYTHONSCRIPTPATH = 'C:\Python36\Scripts'
# The directory name of django project
# Note: it is the directory at the same level of manage.py
# not the parent directory
PROJECTDIR = 'Project'
logging.basicConfig(
filename = os.path.join(LOGDIR, 'celery_worker_service.log'),
level = logging.DEBUG,
format = '[%(asctime)-15s: %(levelname)-7.7s] %(message)s'
)
class CeleryService(win32serviceutil.ServiceFramework):
_svc_name_ = "CeleryWorkers"
_svc_display_name_ = "CeleryWorkers"
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
def SvcStop(self):
logging.info('Stopping {name} service ...'.format(name=self._svc_name_))
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
logging.info('Stopped1 {name} service ...'.format(name=self._svc_name_))
logging.info('Stopped3 {name} service ...'.format(name=self._svc_name_))
command = '"{celery_path}" -A {proj_dir} --workdir=X:/Application/Project control shutdown --timeout=10'.format(
celery_path=os.path.join(PYTHONSCRIPTPATH, 'celery.exe'),
proj_dir=PROJECTDIR,
log_path=os.path.join(LOGDIR,'celery_worker.log'))
logging.info('command: ' + command)
args = shlex.split(command)
proc = subprocess.Popen(args)
logging.info('Stopped celery shutdown ...')
self.ReportServiceStatus(win32service.SERVICE_STOPPED)
logging.info('Stopped2 {name} service ...'.format(name=self._svc_name_))
sys.exit()
def SvcDoRun(self):
logging.info('Starting {name} service ...'.format(name=self._svc_name_))
os.chdir(INSTDIR) # so that proj worker can be found
logging.info('cwd: ' + os.getcwd())
self.ReportServiceStatus(win32service.SERVICE_RUNNING)
command = '"{celery_path}" -A {proj_dir} -c 8 worker --workdir=X:/Application/Project --pidfile=celeryservice.pid -f "{log_path}" -l info'.format(
celery_path=os.path.join(PYTHONSCRIPTPATH, 'celery.exe'),
proj_dir=PROJECTDIR,
log_path=os.path.join(LOGDIR,'celery_worker.log'))
logging.info('command: ' + command)
args = shlex.split(command)
proc = subprocess.Popen(args)
logging.info('pid: {pid}'.format(pid=proc.pid))
self.timeout = 3000
while True:
rc = win32event.WaitForSingleObject(self.hWaitStop, self.timeout)
if rc == win32event.WAIT_OBJECT_0:
# stop signal encountered
# terminate process 'proc'
PROCESS_TERMINATE = 1
handle = win32api.OpenProcess(PROCESS_TERMINATE, False, proc.pid)
win32api.TerminateProcess(handle, -1)
win32api.CloseHandle(handle)
break
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(CeleryService)
Beatservice.py:
'''Usage : python celery_service.py install (start / stop / remove)
Run celery as a Windows service
'''
import win32service
import win32serviceutil
import win32api
import win32con
import win32event
import subprocess
import sys
import os
import shlex
import logging
import time
import signal
# The directory for celery_beat.log and celery_beat_service.log
# Default: the directory of this script
INSTDIR = os.path.dirname(os.path.realpath(__file__))
LOGPATH = 'X:\Application\Logs'
# The path of python Scripts
# Usually it is in PYTHON_INSTALL_DIR/Scripts. e.g.
# r'C:\Python27\Scripts'
# If it is already in system PATH, then it can be set as ''
PYTHONSCRIPTPATH = 'C:\Python36\Scripts'
# The directory name of django project
# Note: it is the directory at the same level of manage.py
# not the parent directory
PROJECTDIR = 'PROJECT'
logging.basicConfig(
filename = os.path.join(LOGPATH, 'celery_beat_service.log'),
level = logging.DEBUG,
format = '[%(asctime)-15s: %(levelname)-7.7s] %(message)s'
)
class CeleryService(win32serviceutil.ServiceFramework):
_svc_name_ = "CeleryBeat"
_svc_display_name_ = "CeleryBeat"
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
def SvcStop(self):
logging.info('Stopping 1 {name} service ...'.format(name=self._svc_name_))
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
pidno = open("X:\Aplication\Project\celerybeat.pid", "r")
_pid_id_ = pidid=pidno.read()
pidno.close()
logging.info(_pid_id_)
logging.info('taskkill /F /PID {pidid} ..'.format(pidid=_pid_id_))
cmdcom = 'taskkill /F /PID {pidid}'.format(pidid=_pid_id_)
logging.info(cmdcom)
killargs = shlex.split(cmdcom)
process = subprocess.Popen(killargs)
output, error = process.communicate()
logging.info(output)
logging.info('Stopping 2 {name} service ...'.format(name=self._svc_name_))
os.remove("X:\Application\PROJECT\celerybeat.pid")
logging.info('X:\Application\PROJECT\celerybeat.pid file removed')
self.ReportServiceStatus(win32service.SERVICE_STOPPED)
sys.exit()
def SvcDoRun(self):
logging.info('Starting {name} service ...'.format(name=self._svc_name_))
os.chdir(INSTDIR) # so that proj worker can be found
logging.info('cwd: ' + os.getcwd())
self.ReportServiceStatus(win32service.SERVICE_RUNNING)
command = '"{celery_path}" -A {proj_dir} beat --workdir=X:/Application/Project -f X:/Application/logs/beat.log -l info'.format(
celery_path=os.path.join(PYTHONSCRIPTPATH, 'celery.exe'),
proj_dir=PROJECTDIR,
log_path=os.path.join(LOGPATH,'celery_beat.log'))
logging.info('command: ' + command)
args = shlex.split(command)
proc = subprocess.Popen(args)
logging.info('pid: {pid}'.format(pid=proc.pid))
self.timeout = 3000
while True:
rc = win32event.WaitForSingleObject(self.hWaitStop, self.timeout)
if rc == win32event.WAIT_OBJECT_0:
# stop signal encountered
# terminate process 'proc'
PROCESS_TERMINATE = 1
handle = win32api.OpenProcess(PROCESS_TERMINATE, False, proc.pid)
win32api.TerminateProcess(handle, -1)
win32api.CloseHandle(handle)
break
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(CeleryService)
I have a minimal python win32 service service.py that does nothing special:
import win32serviceutil
import win32service
import win32event
class SmallestPythonService(win32serviceutil.ServiceFramework):
_svc_name_ = "SmallestPythonService"
_svc_display_name_ = "display service"
# _svc_description_='ddd'
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
def SvcDoRun(self):
win32event.WaitForSingleObject(self.hWaitStop, win32event.INFINITE)
if __name__=='__main__':
win32serviceutil.HandleCommandLine(SmallestPythonService)
When I run:
service.py install
service.py start
it works fine but when I compile the service.py file with py2exe to service.exe and run the following:
service.exe install
service.exe start [or trying to restart the service from the Services.msc]
I get this message:
Could not start the service name service on Local Computer.
Error 1053: The service did not respond to the start or control request in a timely fashion
How can I resolve this problem?
Also here the distutil code:
from distutils.core import setup
import py2exe
py2exe_options = {"includes": ['decimal'],'bundle_files': 1}
setup(console=[{"script":'Service.py'}],
options={"py2exe": py2exe_options},
zipfile = None,
},
)
Replace your: setup(console=[{"script":'Service.py'}] with setup(service=[{"script":'Service.py'}]. Instead of console use service.
try this setup:
py2exe_options = {"includes": ['decimal'],'bundle_files': 1}
setup(
service=[{'modules':'Service.py','cmdline_style':'pywin32','description':'your service description'}],
options={'py2exe':py2exe_options},
zipfile=None)
A quick google came up with this: http://islascruz.org/html/index.php?gadget=StaticPage&action=Page&id=6
It has Italian comments, but I can help you translate some stuff if you don't know Italian.
To truly debug your problem, I guess we will need to see your setup.py distutils script...
You probably might be missing the right PATH for finding all the DLLs required by the service. Usually the service gets installed as a 'LocalSystem' service so you need to add the PATH to the System (and not to User).
Try adding c:\python27 (or whatever the path to your python dlls is) to the SYSTEM PATH, restart the computer and check if it now starts fine.
I am sketching the architecture for a set of programs that share various interrelated objects stored in a database. I want one of the programs to act as a service which provides a higher level interface for operations on these objects, and the other programs to access the objects through that service.
I am currently aiming for Python and the Django framework as the technologies to implement that service with. I'm pretty sure I figure how to daemonize the Python program in Linux. However, it is an optional spec item that the system should support Windows. I have little experience with Windows programming and no experience at all with Windows services.
Is it possible to run a Python programs as a Windows service (i. e. run it automatically without user login)? I won't necessarily have to implement this part, but I need a rough idea how it would be done in order to decide whether to design along these lines.
Edit: Thanks for all the answers so far, they are quite comprehensive. I would like to know one more thing: How is Windows aware of my service? Can I manage it with the native Windows utilities? What is the equivalent of putting a start/stop script in /etc/init.d?
Yes you can. I do it using the pythoncom libraries that come included with ActivePython or can be installed with pywin32 (Python for Windows extensions).
This is a basic skeleton for a simple service:
import win32serviceutil
import win32service
import win32event
import servicemanager
import socket
class AppServerSvc (win32serviceutil.ServiceFramework):
_svc_name_ = "TestService"
_svc_display_name_ = "Test Service"
def __init__(self,args):
win32serviceutil.ServiceFramework.__init__(self,args)
self.hWaitStop = win32event.CreateEvent(None,0,0,None)
socket.setdefaulttimeout(60)
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
def SvcDoRun(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_,''))
self.main()
def main(self):
pass
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(AppServerSvc)
Your code would go in the main() method—usually with some kind of infinite loop that might be interrupted by checking a flag, which you set in the SvcStop method
The simplest way is to use the: NSSM - the Non-Sucking Service Manager. Just download and unzip to a location of your choosing. It's a self-contained utility, around 300KB (much less than installing the entire pywin32 suite just for this purpose) and no "installation" is needed. The zip contains a 64-bit and a 32-bit version of the utility. Either should work well on current systems (you can use the 32-bit version to manage services on 64-bit systems).
GUI approach
1 - install the python program as a service. Open a Win prompt as admin
c:\>nssm.exe install WinService
2 - On NSSM´s GUI console:
path: C:\Python27\Python27.exe
Startup directory: C:\Python27
Arguments: c:\WinService.py
3 - check the created services on services.msc
Scripting approach (no GUI)
This is handy if your service should be part of an automated, non-interactive procedure, that may be beyond your control, such as a batch or installer script. It is assumed that the commands are executed with administrative privileges.
For convenience the commands are described here by simply referring to the utility as nssm.exe. It is advisable, however, to refer to it more explicitly in scripting with its full path c:\path\to\nssm.exe, since it's a self-contained executable that may be located in a private path that the system is not aware of.
1. Install the service
You must specify a name for the service, the path to the proper Python executable, and the path to the script:
nssm.exe install ProjectService "c:\path\to\python.exe" "c:\path\to\project\app\main.py"
More explicitly:
nssm.exe install ProjectService
nssm.exe set ProjectService Application "c:\path\to\python.exe"
nssm.exe set ProjectService AppParameters "c:\path\to\project\app\main.py"
Alternatively you may want your Python app to be started as a Python module. One easy approach is to tell nssm that it needs to change to the proper starting directory, as you would do yourself when launching from a command shell:
nssm.exe install ProjectService "c:\path\to\python.exe" "-m app.main"
nssm.exe set ProjectService AppDirectory "c:\path\to\project"
This approach works well with virtual environments and self-contained (embedded) Python installs. Just make sure to have properly resolved any path issues in those environments with the usual methods. nssm has a way to set environment variables (e.g. PYTHONPATH) if needed, and can also launch batch scripts.
2. To start the service
nssm.exe start ProjectService
3. To stop the service
nssm.exe stop ProjectService
4. To remove the service, specify the confirm parameter to skip the interactive confirmation.
nssm.exe remove ProjectService confirm
Although I upvoted the chosen answer a couple of weeks back, in the meantime I struggled a lot more with this topic. It feels like having a special Python installation and using special modules to run a script as a service is simply the wrong way. What about portability and such?
I stumbled across the wonderful Non-sucking Service Manager, which made it really simple and sane to deal with Windows Services. I figured since I could pass options to an installed service, I could just as well select my Python executable and pass my script as an option.
I have not yet tried this solution, but I will do so right now and update this post along the process. I am also interested in using virtualenvs on Windows, so I might come up with a tutorial sooner or later and link to it here.
The simplest way to achieve this is to use native command sc.exe:
sc create PythonApp binPath= "C:\Python34\Python.exe --C:\tmp\pythonscript.py"
References:
https://technet.microsoft.com/en-us/library/cc990289(v=ws.11).aspx
When creating a service with sc.exe how to pass in context parameters?
There are a couple alternatives for installing as a service virtually any Windows executable.
Method 1: Use instsrv and srvany from rktools.exe
For Windows Home Server or Windows Server 2003 (works with WinXP too), the Windows Server 2003 Resource Kit Tools comes with utilities that can be used in tandem for this, called instsrv.exe and srvany.exe. See this Microsoft KB article KB137890 for details on how to use these utils.
For Windows Home Server, there is a great user friendly wrapper for these utilities named aptly "Any Service Installer".
Method 2: Use ServiceInstaller for Windows NT
There is another alternative using ServiceInstaller for Windows NT (download-able here) with python instructions available. Contrary to the name, it works with both Windows 2000 and Windows XP as well. Here are some instructions for how to install a python script as a service.
Installing a Python script
Run ServiceInstaller to create a new
service. (In this example, it is
assumed that python is installed at
c:\python25)
Service Name : PythonTest
Display Name : PythonTest
Startup : Manual (or whatever you like)
Dependencies : (Leave blank or fill to fit your needs)
Executable : c:\python25\python.exe
Arguments : c:\path_to_your_python_script\test.py
Working Directory : c:\path_to_your_python_script
After installing, open the Control
Panel's Services applet, select and
start the PythonTest service.
After my initial answer, I noticed there were closely related Q&A already posted on SO. See also:
Can I run a Python script as a service (in Windows)? How?
How do I make Windows aware of a service I have written in Python?
Step by step explanation how to make it work :
1- First create a python file according to the basic skeleton mentioned above. And save it to a path for example : "c:\PythonFiles\AppServerSvc.py"
import win32serviceutil
import win32service
import win32event
import servicemanager
import socket
class AppServerSvc (win32serviceutil.ServiceFramework):
_svc_name_ = "TestService"
_svc_display_name_ = "Test Service"
def __init__(self,args):
win32serviceutil.ServiceFramework.__init__(self,args)
self.hWaitStop = win32event.CreateEvent(None,0,0,None)
socket.setdefaulttimeout(60)
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
def SvcDoRun(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_,''))
self.main()
def main(self):
# Your business logic or call to any class should be here
# this time it creates a text.txt and writes Test Service in a daily manner
f = open('C:\\test.txt', 'a')
rc = None
while rc != win32event.WAIT_OBJECT_0:
f.write('Test Service \n')
f.flush()
# block for 24*60*60 seconds and wait for a stop event
# it is used for a one-day loop
rc = win32event.WaitForSingleObject(self.hWaitStop, 24 * 60 * 60 * 1000)
f.write('shut down \n')
f.close()
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(AppServerSvc)
2 - On this step we should register our service.
Run command prompt as administrator and type as:
sc create TestService binpath= "C:\Python36\Python.exe c:\PythonFiles\AppServerSvc.py" DisplayName= "TestService" start= auto
the first argument of binpath is the path of python.exe
second argument of binpath is the path of your python file that we created already
Don't miss that you should put one space after every "=" sign.
Then if everything is ok, you should see
[SC] CreateService SUCCESS
Now your python service is installed as windows service now. You can see it in Service Manager and registry under :
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TestService
3- Ok now. You can start your service on service manager.
You can execute every python file that provides this service skeleton.
pysc: Service Control Manager on Python
Example script to run as a service taken from pythonhosted.org:
from xmlrpc.server import SimpleXMLRPCServer
from pysc import event_stop
class TestServer:
def echo(self, msg):
return msg
if __name__ == '__main__':
server = SimpleXMLRPCServer(('127.0.0.1', 9001))
#event_stop
def stop():
server.server_close()
server.register_instance(TestServer())
server.serve_forever()
Create and start service
import os
import sys
from xmlrpc.client import ServerProxy
import pysc
if __name__ == '__main__':
service_name = 'test_xmlrpc_server'
script_path = os.path.join(
os.path.dirname(__file__), 'xmlrpc_server.py'
)
pysc.create(
service_name=service_name,
cmd=[sys.executable, script_path]
)
pysc.start(service_name)
client = ServerProxy('http://127.0.0.1:9001')
print(client.echo('test scm'))
Stop and delete service
import pysc
service_name = 'test_xmlrpc_server'
pysc.stop(service_name)
pysc.delete(service_name)
pip install pysc
nssm in python 3+
(I converted my .py file to .exe with pyinstaller)
nssm:
as said before
run nssm install {ServiceName}
On NSSM´s console:
path: path\to\your\program.exe
Startup directory: path\to\your\ #same as the path but without your program.exe
Arguments: empty
If you don't want to convert your project to .exe
create a .bat file with python {{your python.py file name}}
and set the path to the .bat file
I started hosting as a service with pywin32.
Everything was well but I met the problem that service was not able to start within 30 seconds (default timeout for Windows) on system startup. It was critical for me because Windows startup took place simultaneous on several virtual machines hosted on one physical machine, and IO load was huge.
Error messages were:
Error 1053: The service did not respond to the start or control request in a timely fashion.
Error 7009: Timeout (30000 milliseconds) waiting for the <ServiceName> service to connect.
I fought a lot with pywin, but ended up with using NSSM as it was proposed in this answer. It was very easy to migrate to it.
A complete pywin32 example using loop or subthread
After working on this on and off for a few days, here is the answer I would have wished to find, using pywin32 to keep it nice and self contained.
This is complete working code for one loop-based and one thread-based solution.
It may work on both python 2 and 3, although I've only tested the latest version on 2.7 and Win7. The loop should be good for polling code, and the tread should work with more server-like code. It seems to work nicely with the waitress wsgi server that does not have a standard way to shut down gracefully.
I would also like to note that there seems to be loads of examples out there, like this that are almost useful, but in reality misleading, because they have cut and pasted other examples blindly. I could be wrong. but why create an event if you never wait for it?
That said I still feel I'm on somewhat shaky ground here, especially with regards to how clean the exit from the thread version is, but at least I believe there are nothing misleading here.
To run simply copy the code to a file and follow the instructions.
update:
Use a simple flag to terminate thread. The important bit is that "thread done" prints.
For a more elaborate example exiting from an uncooperative server thread see my post about the waitress wsgi server.
# uncomment mainthread() or mainloop() call below
# run without parameters to see HandleCommandLine options
# install service with "install" and remove with "remove"
# run with "debug" to see print statements
# with "start" and "stop" watch for files to appear
# check Windows EventViever for log messages
import socket
import sys
import threading
import time
from random import randint
from os import path
import servicemanager
import win32event
import win32service
import win32serviceutil
# see http://timgolden.me.uk/pywin32-docs/contents.html for details
def dummytask_once(msg='once'):
fn = path.join(path.dirname(__file__),
'%s_%s.txt' % (msg, randint(1, 10000)))
with open(fn, 'w') as fh:
print(fn)
fh.write('')
def dummytask_loop():
global do_run
while do_run:
dummytask_once(msg='loop')
time.sleep(3)
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
global do_run
do_run = True
print('thread start\n')
dummytask_loop()
print('thread done\n')
def exit(self):
global do_run
do_run = False
class SMWinservice(win32serviceutil.ServiceFramework):
_svc_name_ = 'PyWinSvc'
_svc_display_name_ = 'Python Windows Service'
_svc_description_ = 'An example of a windows service in Python'
#classmethod
def parse_command_line(cls):
win32serviceutil.HandleCommandLine(cls)
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.stopEvt = win32event.CreateEvent(None, 0, 0, None) # create generic event
socket.setdefaulttimeout(60)
def SvcStop(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STOPPED,
(self._svc_name_, ''))
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.stopEvt) # raise event
def SvcDoRun(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_, ''))
# UNCOMMENT ONE OF THESE
# self.mainthread()
# self.mainloop()
# Wait for stopEvt indefinitely after starting thread.
def mainthread(self):
print('main start')
self.server = MyThread()
self.server.start()
print('wait for win32event')
win32event.WaitForSingleObject(self.stopEvt, win32event.INFINITE)
self.server.exit()
print('wait for thread')
self.server.join()
print('main done')
# Wait for stopEvt event in loop.
def mainloop(self):
print('loop start')
rc = None
while rc != win32event.WAIT_OBJECT_0:
dummytask_once()
rc = win32event.WaitForSingleObject(self.stopEvt, 3000)
print('loop done')
if __name__ == '__main__':
SMWinservice.parse_command_line()
This answer is plagiarizer from several sources on StackOverflow - most of them above, but I've forgotten the others - sorry. It's simple and scripts run "as is". For releases you test you script, then copy it to the server and Stop/Start the associated service. And it should work for all scripting languages (Python, Perl, node.js), plus batch scripts such as GitBash, PowerShell, even old DOS bat scripts.
pyGlue is the glue that sits between Windows Services and your script.
'''
A script to create a Windows Service, which, when started, will run an executable with the specified parameters.
Optionally, you can also specify a startup directory
To use this script you MUST define (in class Service)
1. A name for your service (short - preferably no spaces)
2. A display name for your service (the name visibile in Windows Services)
3. A description for your service (long details visible when you inspect the service in Windows Services)
4. The full path of the executable (usually C:/Python38/python.exe or C:WINDOWS/System32/WindowsPowerShell/v1.0/powershell.exe
5. The script which Python or PowerShell will run(or specify None if your executable is standalone - in which case you don't need pyGlue)
6. The startup directory (or specify None)
7. Any parameters for your script (or for your executable if you have no script)
NOTE: This does not make a portable script.
The associated '_svc_name.exe' in the dist folder will only work if the executable,
(and any optional startup directory) actually exist in those locations on the target system
Usage: 'pyGlue.exe [options] install|update|remove|start [...]|stop|restart [...]|debug [...]'
Options for 'install' and 'update' commands only:
--username domain\\username : The Username the service is to run under
--password password : The password for the username
--startup [manual|auto|disabled|delayed] : How the service starts, default = manual
--interactive : Allow the service to interact with the desktop.
--perfmonini file: .ini file to use for registering performance monitor data
--perfmondll file: .dll file to use when querying the service for performance data, default = perfmondata.dll
Options for 'start' and 'stop' commands only:
--wait seconds: Wait for the service to actually start or stop.
If you specify --wait with the 'stop' option, the service and all dependent services will be stopped,
each waiting the specified period.
'''
# Import all the modules that make life easy
import servicemanager
import socket
import sys
import win32event
import win32service
import win32serviceutil
import win32evtlogutil
import os
from logging import Formatter, Handler
import logging
import subprocess
# Define the win32api class
class Service (win32serviceutil.ServiceFramework):
# The following variable are edited by the build.sh script
_svc_name_ = "TestService"
_svc_display_name_ = "Test Service"
_svc_description_ = "Test Running Python Scripts as a Service"
service_exe = 'c:/Python27/python.exe'
service_script = None
service_params = []
service_startDir = None
# Initialize the service
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
self.configure_logging()
socket.setdefaulttimeout(60)
# Configure logging to the WINDOWS Event logs
def configure_logging(self):
self.formatter = Formatter('%(message)s')
self.handler = logHandler()
self.handler.setFormatter(self.formatter)
self.logger = logging.getLogger()
self.logger.addHandler(self.handler)
self.logger.setLevel(logging.INFO)
# Stop the service
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
# Run the service
def SvcDoRun(self):
self.main()
# This is the service
def main(self):
# Log that we are starting
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE, servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_, ''))
# Fire off the real process that does the real work
logging.info('%s - about to call Popen() to run %s %s %s', self._svc_name_, self.service_exe, self.service_script, self.service_params)
self.process = subprocess.Popen([self.service_exe, self.service_script] + self.service_params, shell=False, cwd=self.service_startDir)
logging.info('%s - started process %d', self._svc_name_, self.process.pid)
# Wait until WINDOWS kills us - retrigger the wait for stop every 60 seconds
rc = None
while rc != win32event.WAIT_OBJECT_0:
rc = win32event.WaitForSingleObject(self.hWaitStop, (1 * 60 * 1000))
# Shut down the real process and exit
logging.info('%s - is terminating process %d', self._svc_name_, self.process.pid)
self.process.terminate()
logging.info('%s - is exiting', self._svc_name_)
class logHandler(Handler):
'''
Emit a log record to the WINDOWS Event log
'''
def emit(self, record):
servicemanager.LogInfoMsg(record.getMessage())
# The main code
if __name__ == '__main__':
'''
Create a Windows Service, which, when started, will run an executable with the specified parameters.
'''
# Check that configuration contains valid values just in case this service has accidentally
# been moved to a server where things are in different places
if not os.path.isfile(Service.service_exe):
print('Executable file({!s}) does not exist'.format(Service.service_exe), file=sys.stderr)
sys.exit(0)
if not os.access(Service.service_exe, os.X_OK):
print('Executable file({!s}) is not executable'.format(Service.service_exe), file=sys.stderr)
sys.exit(0)
# Check that any optional startup directory exists
if (Service.service_startDir is not None) and (not os.path.isdir(Service.service_startDir)):
print('Start up directory({!s}) does not exist'.format(Service.service_startDir), file=sys.stderr)
sys.exit(0)
if len(sys.argv) == 1:
servicemanager.Initialize()
servicemanager.PrepareToHostSingle(Service)
servicemanager.StartServiceCtrlDispatcher()
else:
# install/update/remove/start/stop/restart or debug the service
# One of those command line options must be specified
win32serviceutil.HandleCommandLine(Service)
Now there's a bit of editing and you don't want all your services called 'pyGlue'. So there's a script (build.sh) to plug in the bits and create a customized 'pyGlue' and create an '.exe'. It is this '.exe' which gets installed as a Windows Service. Once installed you can set it to run automatically.
#!/bin/sh
# This script build a Windows Service that will install/start/stop/remove a service that runs a script
# That is, executes Python to run a Python script, or PowerShell to run a PowerShell script, etc
if [ $# -lt 6 ]; then
echo "Usage: build.sh Name Display Description Executable Script StartupDir [Params]..."
exit 0
fi
name=$1
display=$2
desc=$3
exe=$4
script=$5
startDir=$6
shift; shift; shift; shift; shift; shift
params=
while [ $# -gt 0 ]; do
if [ "${params}" != "" ]; then
params="${params}, "
fi
params="${params}'$1'"
shift
done
cat pyGlue.py | sed -e "s/pyGlue/${name}/g" | \
sed -e "/_svc_name_ =/s?=.*?= '${name}'?" | \
sed -e "/_svc_display_name_ =/s?=.*?= '${display}'?" | \
sed -e "/_svc_description_ =/s?=.*?= '${desc}'?" | \
sed -e "/service_exe =/s?=.*?= '$exe'?" | \
sed -e "/service_script =/s?=.*?= '$script'?" | \
sed -e "/service_params =/s?=.*?= [${params}]?" | \
sed -e "/service_startDir =/s?=.*?= '${startDir}'?" > ${name}.py
cxfreeze ${name}.py --include-modules=win32timezone
Installation - copy the '.exe' the server and the script to the specified folder. Run the '.exe', as Administrator, with the 'install' option. Open Windows Services, as Adminstrator, and start you service. For upgrade, just copy the new version of the script and Stop/Start the service.
Now every server is different - different installations of Python, different folder structures. I maintain a folder for every server, with a copy of pyGlue.py and build.sh. And I create a 'serverBuild.sh' script for rebuilding all the service on that server.
# A script to build all the script based Services on this PC
sh build.sh AutoCode 'AutoCode Medical Documents' 'Autocode Medical Documents to SNOMED_CT and AIHW codes' C:/Python38/python.exe autocode.py C:/Users/russell/Documents/autocoding -S -T
The accepted answer using win32serviceutil works but is complicated and makes debugging and changes harder. It is far easier to use NSSM (the Non-Sucking Service Manager). You write and comfortably debug a normal python program and when it finally works you use NSSM to install it as a service in less than a minute:
From an elevated (admin) command prompt you run nssm.exe install NameOfYourService and you fill-in these options:
path: (the path to python.exe e.g. C:\Python27\Python.exe)
Arguments: (the path to your python script, e.g. c:\path\to\program.py)
By the way, if your program prints useful messages that you want to keep in a log file NSSM can also handle this and a lot more for you.
This doesn't answer the original question, but might help other people that want to automatically start a Python script at Windows startup:
Have a look at the Windows Task Scheduler instead, it is way easier if you just want to start a script after boot without all the service functionality of a Windows Service.
Create a new task, select "At startup" as trigger, "Start program" as action with "C:\Python39\python.exe" as the program (or wherever your python.exe is) and the full path to your script ("C:...\my_dir\xyz.py") as argument (you can use " if the path contains spaces).
You can also select the path of your script (without the .py file, e.g. "C:...\my_dir") for "start in" if you use relative paths in your script, e.g. for logging.
https://www.chrisumbel.com/article/windows_services_in_python
Follow up the PySvc.py
changing the dll folder
I know this is old but I was stuck on this forever. For me, this specific problem was solved by copying this file - pywintypes36.dll
From -> Python36\Lib\site-packages\pywin32_system32
To -> Python36\Lib\site-packages\win32
setx /M PATH "%PATH%;C:\Users\user\AppData\Local\Programs\Python\Python38-32;C:\Users\user\AppData\Local\Programs\Python\Python38-32\Scripts;C:\Users\user\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\pywin32_system32;C:\Users\user\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\win32
changing the path to python folder by
cd C:\Users\user\AppData\Local\Programs\Python\Python38-32
NET START PySvc
NET STOP PySvc