Using the following example I can get a basic web server running but my problem is that the handle_request() blocks the do_something_else() until a request comes in. Is there any way around this to have the web server do other back ground tasks?
def run_while_true(server_class=BaseHTTPServer.HTTPServer,
handler_class=BaseHTTPServer.BaseHTTPRequestHandler):
server_address = ('', 8000)
httpd = server_class(server_address, handler_class)
while keep_running():
httpd.handle_request()
do_something_else()
You can use multiple threads of execution through the Python threading module. An example is below:
import threading
# ... your code here...
def run_while_true(server_class=BaseHTTPServer.HTTPServer,
handler_class=BaseHTTPServer.BaseHTTPRequestHandler):
server_address = ('', 8000)
httpd = server_class(server_address, handler_class)
while keep_running():
httpd.handle_request()
if __name__ == '__main__':
background_thread = threading.Thread(target=do_something_else)
background_thread.start()
# ... web server start code here...
background_thread.join()
This will cause a thread which executes do_something_else() to start before your web server. When the server shuts down, the join() call ensures do_something_else finishes before the program exits.
You should have a thread that handles http requests, and a thread that does do_something_else().
Related
I have a scenario where I need to first respond with HTTP 200 to a server request (due to a time limit) and then continue processing with the actual work.
I also can not use threads, processes, tasks, queues or any other method that would allow me to do this by starting a parallel "process".
My approach is to use the build in "Simple HTTP" server and I am looking for a way to force the server to respond with HTTP 200 and then be able to continue processing.
The current code will receive a POST request and print its content after a 3 seconds. I put a placeholder where I would like to send the response.
from http.server import BaseHTTPRequestHandler, HTTPServer
import time
class MyWebServer(BaseHTTPRequestHandler):
def do_POST(self):
content_length = int(self.headers['Content-Length'])
post_data = self.rfile.read(content_length)
self.send_response_only(200)
self.end_headers()
# force server to send request ???
time.sleep(3)
print(post_data)
def run(server_class=HTTPServer, handler_class=MyWebServer, port=8000):
server_address = ('', port)
httpd = server_class(server_address, handler_class)
print('Starting httpd...')
httpd.serve_forever()
if __name__ == "__main__":
run()
I figured out a workaround solution. You can force the server to send a 200 OK and continue processing after with these two commands:
self.finish()
self.connection.close()
This solution is from this SO question: SimpleHTTPRequestHandler close connection before returning from do_POST method
However, this will apparently close the internal IO buffer that the server uses and it won't be able to server any additional requests after that.
To avoid running into an exception it works to terminate the program (which works for me). However this is just a workaround and I would still be looking for a solution that allows the server to keep processing new requests.
from http.server import BaseHTTPRequestHandler, HTTPServer
import time
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self):
content_length = int(self.headers['Content-Length'])
post_data = self.rfile.read(content_length)
self.send_response_only(200)
self.end_headers()
self.finish()
self.connection.close()
time.sleep(3)
print(post_data)
quit()
def run(server_class=HTTPServer, handler_class=MyHandler, port=8000):
server_address = ('', port)
httpd = server_class(server_address, handler_class)
print('Starting httpd...')
httpd.serve_forever()
if __name__ == "__main__":
run()
I am attempting to start a simple HTTP web server in python and then ping it with the selenium driver. I can get the web server to start but it "hangs" after the server starts even though I have started it in a new thread.
from socket import *
from selenium import webdriver
import SimpleHTTPServer
import SocketServer
import thread
def create_server():
port = 8000
handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", port), handler)
print("serving at port:" + str(port))
httpd.serve_forever()
thread.start_new_thread(create_server())
print("Server has started. Continuing..")
browser = webdriver.Firefox()
browser.get("http://localhost:8000")
assert "<title>" in browser.page_source
thread.exit()
The server starts but the script execution stops after the server has started. The code after I start the thread is never executed.
How do I get the server to start and then have the code continue execution?
Start your thread with function create_server (without calling it ()):
thread.start_new_thread(create_server, tuple())
If you call create_server(), it will stop at httpd.serve_forever().
For Python 3 you can use this:
import threading
threading.Thread(target=create_server).start()
Found the following chat server demo on Twisted's website:
factory = protocol.ServerFactory()
factory.protocol = SimpleLogger
factory.clients = []
application = service.Application("charServer")
internet.TCPServer(9999, factory).setServiceParent(application)
It didn't work in my project. It does not wait until the client connects to the server, but runs through the code and exits immediately. How can I fix this?
The lines above set up the connections to listen, but then immediately exit. You need to add something along the lines of:
if __name__ == '__main__':
from twisted.internet import reactor
reactor.run()
I use http webserver python script:
class PiFaceWebHandler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
[....]
if __name__ == "__main__":
# get the port
if len(sys.argv) > 1:
port = int(sys.argv[1])
else:
port = DEFAULT_PORT
# set up PiFace Digital
PiFaceWebHandler.pifacedigital = pifacedigitalio.PiFaceDigital()
print("Starting simple PiFace web control at:\n\n"
"\thttp://{addr}:{port}\n\n"
"Change the output_port with:\n\n"
"\thttp://{addr}:{port}?output_port=0xAA\n"
.format(addr=get_my_ip(), port=port))
# run the server
server_address = ('', port)
try:
httpd = http.server.HTTPServer(server_address, PiFaceWebHandler)
httpd.serve_forever()
except KeyboardInterrupt:
print('^C received, shutting down server')
httpd.socket.close()
It's working fine, but i want the script (or another one, ) check some I/O continuously, in a while loop, ie.
And sometimes this I/O could change state with http request.
Currently, I/O changes state on http request, but i don't find the tips to change them on external trigger (another input ie).
How can i do? Where can i code the loop test?
do I make myself clear?
Thanks,
I'm trying to run a django development server from within a Kivy application. This did work out quite well so far.
Now i want to allow the user to continue working with the program while the server is running. My idea was to create a multiprocessing.Process for the httpd.serve_forever() to avoid a complete lock of the main program. Did work well. This is the code in my internal_django module:
import multiprocessing
import os
import time
from wsgiref.simple_server import make_server
def django_wsgi_application():
PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
settings_module = "djangosettings"#%s.djangosettings" % PROJECT_ROOT.split(os.sep)[-1]
os.environ.update({"DJANGO_SETTINGS_MODULE":settings_module})
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
return application
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
return cls._instances[cls]
class DjangoServer():
__metaclass__ = Singleton
def start(self):
self.httpd = make_server('', 8000, django_wsgi_application())
self.server = multiprocessing.Process(target=self.httpd.serve_forever)
self.server.start()
print "Now serving on port 8000..."
print "Server Process PID = %s" %self.server.pid
def stop(self):
print("shutdown initiated")
print "Server Process PID = %s" %self.server.pid
while self.server.is_alive():
self.server.terminate()
print("Server should have shut down")
time.sleep(1)
print("Server is_alive: %s" %self.server.is_alive())
self.server.join()
print("server process joined")
if __name__ == "__main__":
server = DjangoServer()
server.start()
time.sleep(3)
server.stop()
When i run this code, everything works as expected. This is what is being put out in the console:
Now serving on port 8000...
Server Process PID = 1406
shutdown initiated
Server Process PID = 1406
Server should have shut down
Server is_alive: False
server process joined
Next step was to provide a way to stop the server from within the Kivy application. For that i just wanted to use my DjangoServer class as i did before:
from internal_django import DjangoServer
class StartScreen(Screen):
def start_server(self):
server = DjangoServer()
server.start()
class StopScreen(Screen):
def stop_server(self):
server = DjangoServer()
server.stop()
But when doing so, the process once started never quits. My first idea was that the Singleton did not work as expected, and that i try to quit the wrong process. but as you can see in the output, the PID's are identical. The server receives the terminate command, but just continues to work. This is what the console looks like:
Now serving on port 8000...
Server Process PID = 1406
shutdown initiated
Server Process PID = 1406
Server should have shut down
Server should have shut down
Server should have shut down
Server should have shut down
Server should have shut down
Server should have shut down
Server should have shut down
Server should have shut down
(and so on, until i manually kill the server process)
Am i using multiprocessing in a completely wrong way? Is Kivy somehow interfering with the process?
I think the problems here might be two:
A signal handler is intercepting the TERM request sent by Process.terminate() and ignores it. To verify that simply use the signal.getsignal(signal.SIGTERM) from within the new process and print the results. To circumvent such issue you can reset the default behavior with signal.signal(signal.SIGTERM, signal.SIG_DFL), nevertheless keep in mind that there might be a reason why SIGTERM is silenced by the frameworks (I'm not familiar neither with Django nor with Kivy).
If you're using Python 2 you must consider that the interpreter does not process signals if it's blocked on a synchronization primitive from threading library (Locks, Semaphores..) or on a native C call. The serve_forever() function might fall in these cases (use the force of the source!). Quick check could be trying to run the code on Python 3 and see whether it works or not.
A quick and dirty solution consists in waiting a small amount of time and send a SIGKILL if the process is still alive.
import os
import signal
process.terminate()
process.join(1)
if process.is_alive() and os.name != 'nt':
try:
os.kill(process.pid, signal.SIGKILL)
process.join()
except OSError:
return # process might have died while checking it
On windows you cannot kill a process in such simple way that's why I test the os.name.
It's a pretty raw approach so I'd rather recommend to find the cause of the issue.
What happens if you call terminate(), then join() and skip the while loop? Also, I shuffle the code a little and factor some code into _create_server(). Please let me know if this works out for you.
class DjangoServer():
__metaclass__ = Singleton
def _create_server(self):
httpd = make_server('', 8000, django_wsgi_application())
print "Now serving on port {}...".format(httpd.server_port)
httpd.serve_forever()
def start(self):
self.server = multiprocessing.Process(target=self._create_server)
self.server.start()
print "Server Process PID = %s" %self.server.pid
def stop(self):
print("shutdown initiated")
print "Server Process PID = %s" %self.server.pid
self.server.terminate()
self.server.join()
print("server process terminated")