We Keep Coding

Respond HTTP 200 and continue processing

I have a scenario where I need to first respond with HTTP 200 to a server request (due to a time limit) and then continue processing with the actual work.
I also can not use threads, processes, tasks, queues or any other method that would allow me to do this by starting a parallel "process".
My approach is to use the build in "Simple HTTP" server and I am looking for a way to force the server to respond with HTTP 200 and then be able to continue processing.
The current code will receive a POST request and print its content after a 3 seconds. I put a placeholder where I would like to send the response.
from http.server import BaseHTTPRequestHandler, HTTPServer
import time
class MyWebServer(BaseHTTPRequestHandler):
def do_POST(self):
content_length = int(self.headers['Content-Length'])
post_data = self.rfile.read(content_length)
self.send_response_only(200)
self.end_headers()
# force server to send request ???
time.sleep(3)
print(post_data)
def run(server_class=HTTPServer, handler_class=MyWebServer, port=8000):
server_address = ('', port)
httpd = server_class(server_address, handler_class)
print('Starting httpd...')
httpd.serve_forever()
if __name__ == "__main__":
run()

I figured out a workaround solution. You can force the server to send a 200 OK and continue processing after with these two commands:
self.finish()
self.connection.close()
This solution is from this SO question: SimpleHTTPRequestHandler close connection before returning from do_POST method
However, this will apparently close the internal IO buffer that the server uses and it won't be able to server any additional requests after that.
To avoid running into an exception it works to terminate the program (which works for me). However this is just a workaround and I would still be looking for a solution that allows the server to keep processing new requests.
from http.server import BaseHTTPRequestHandler, HTTPServer
import time
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self):
content_length = int(self.headers['Content-Length'])
post_data = self.rfile.read(content_length)
self.send_response_only(200)
self.end_headers()
self.finish()
self.connection.close()
time.sleep(3)
print(post_data)
quit()
def run(server_class=HTTPServer, handler_class=MyHandler, port=8000):
server_address = ('', port)
httpd = server_class(server_address, handler_class)
print('Starting httpd...')
httpd.serve_forever()
if __name__ == "__main__":
run()

HTTPS connection Python still loading the content until it's KeyboardInterrupted

Can anyone tell me what's the solution for this?
When I run it and load it from the browser... It's only loading and never displaying the "Hello Word!" text.
But the text will appear in the browser after I shutdown the server by triggering the KeyboardInterrupt.
PS: SSL is enabled in python 2.6 interpreter on Linux. Also, it's not working in Windows 7.
Here's the code:
#!/usr/bin/python
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer
import ssl
import sys
PORT_NUMBER = int(sys.argv[1])
#This class will handles any incoming request from the browser
class myHandler(BaseHTTPRequestHandler):
#Handler for the GET requests
def do_GET(self):
print(self.requestline)
#print(self.rfile.read(content_length))
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
# Send the html message
self.wfile.write("Hello World !".encode())
return
try:
#Create a web server and define the handler to manage the
#incoming request
server = HTTPServer(('', PORT_NUMBER), myHandler)
server.socket = ssl.wrap_socket(server.socket, certfile='cert.pem',keyfile='key.pem', server_side=True)
print 'Started httpserver on port ' , PORT_NUMBER
#Wait forever for incoming htto requests
server.serve_forever()
except KeyboardInterrupt:
print '^C received, shutting down the web server'
server.socket.close()
in order to run this in Python 2.x, command: python this_code.py [port]
Example:
python this_code.py 8080
Then navigate to the browser with the address: https://localhost:8080/
If I remove this line, it'll work but it's just running under HTTP protocol and not in HTTPS (which I'm intended to run in):
server.socket = ssl.wrap_socket(server.socket, certfile='cert.pem',keyfile='key.pem', server_side=True)

How to serve no more that one HTTP request in python?

I have a simple HTTP server setup like this one. It processes a slow 40 second request to open and then close gates (real metallic gates). If second HTTP query is made during execution of the first one, it is placed in queue and then executed after first run. I don't need this behavior, I need to reply with error if gate open/close procedure is in progress now.
How can I do that? There's a parameter 'request_queue_size' - but I'm not sure how to set it.

You need to follow a different strategy designing your server service. You need to keep the state of the door either in memory or in a database. Then, each time you receive a request to do something on the door, you check the current state of the door in your persistence, and then you execute the action if it is possible to do on the current state, otherwise you return an error. Also, don't forget to update the state of the door once an action completes.

'request_queue_size' seems to have no effect.
The solution was to make server multithreaded, and implement locking variable 'busy':
from socketserver import ThreadingMixIn
from http.server import BaseHTTPRequestHandler, HTTPServer
import time
from gpiozero import DigitalOutputDevice
import logging
from time import sleep
logging.basicConfig(format='%(asctime)s %(levelname)s:%(message)s', level=logging.INFO)
hostName = ''
hostPort = 9001
busy = False
class ThreadingServer(ThreadingMixIn, HTTPServer):
pass
class MyServer(BaseHTTPRequestHandler):
def do_GET(self):
global busy
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write(bytes("Hello!<br>", "utf-8"))
if self.path == '/gates':
if not busy:
busy = True
relay = DigitalOutputDevice(17) # Initialize GPIO 17
relay.on()
logging.info('Cycle started')
self.wfile.write(bytes("Cycle started<br>", "utf-8"))
sleep(2)
relay.close()
sleep(20)
relay = DigitalOutputDevice(17)
relay.on()
sleep(2)
relay.close()
logging.info('Cycle finished')
self.wfile.write(bytes("Cycle finished", "utf-8"))
busy = False
else:
# self.wfile.write(bytes("Busy now!<br>", "utf-8"))
self.send_error(503)
myServer = ThreadingServer((hostName, hostPort), MyServer)
print(time.asctime(), "Server Starts - %s:%s" % (hostName, hostPort))
try:
myServer.serve_forever()
except KeyboardInterrupt:
pass
myServer.server_close()
print(time.asctime(), "Server Stops - %s:%s" % (hostName, hostPort))

In general, the idea you're looking for is called request throttling. There are lots of implementations of this kind of thing which shouldn't be hard to dig up out there on the Web: here's one for Flask, my microframework of choice - https://flask-limiter.readthedocs.io/en/stable/
Quick usage example:
#app.route("/open_gate")
#limiter.limit("1 per minute")
def slow():
gate_robot.open_gate()
return

Websockets with Tornado: Get access from the "outside" to send messages to clients

I'm starting to get into WebSockets as way to push data from a server to connected clients. Since I use python to program any kind of logic, I looked at Tornado so far. The snippet below shows the most basic example one can find everywhere on the Web:
import tornado.httpserver
import tornado.websocket
import tornado.ioloop
import tornado.web
class WSHandler(tornado.websocket.WebSocketHandler):
def open(self):
print 'new connection'
self.write_message("Hello World")
def on_message(self, message):
print 'message received %s' % message
self.write_message('ECHO: ' + message)
def on_close(self):
print 'connection closed'
application = tornado.web.Application([
(r'/ws', WSHandler),
])
if __name__ == "__main__":
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().start()
As it is, this works as intended. However, I can't get my head around how can get this "integrated" into the rest of my application. In the example above, the WebSocket only sends something to the clients as a reply to a client's message. How can I access the WebSocket from the "outside"? For example, to notify all currently connected clients that some kind event has occured -- and this event is NOT any kind of message from a client. Ideally, I would like to write somewhere in my code something like:
websocket_server.send_to_all_clients("Good news everyone...")
How can I do this? Or do I have a complete misundersanding on how WebSockets (or Tornado) are supposed to work. Thanks!

You need to keep track of all the clients that connect. So:
clients = []
def send_to_all_clients(message):
for client in clients:
client.write_message(message)
class WSHandler(tornado.websocket.WebSocketHandler):
def open(self):
send_to_all_clients("new client")
clients.append(self)
def on_close(self):
clients.remove(self)
send_to_all_clients("removing client")
def on_message(self, message):
for client in clients:
if client != self:
client.write_message('ECHO: ' + message)

This is building on Hans Then's example. Hopefully it helps you understand how you can have your server initiate communication with your clients without the clients triggering the interaction.
Here's the server:
#!/usr/bin/python
import datetime
import tornado.httpserver
import tornado.websocket
import tornado.ioloop
import tornado.web
class WSHandler(tornado.websocket.WebSocketHandler):
clients = []
def open(self):
print 'new connection'
self.write_message("Hello World")
WSHandler.clients.append(self)
def on_message(self, message):
print 'message received %s' % message
self.write_message('ECHO: ' + message)
def on_close(self):
print 'connection closed'
WSHandler.clients.remove(self)
#classmethod
def write_to_clients(cls):
print "Writing to clients"
for client in cls.clients:
client.write_message("Hi there!")
application = tornado.web.Application([
(r'/ws', WSHandler),
])
if __name__ == "__main__":
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().add_timeout(datetime.timedelta(seconds=15), WSHandler.write_to_clients)
tornado.ioloop.IOLoop.instance().start()
I made the client list a class variable, rather than global. I actually wouldn't mind using a global variable for this, but since you were concerned about it, here's an alternative approach.
And here's a sample client:
#!/usr/bin/python
import tornado.websocket
from tornado import gen
#gen.coroutine
def test_ws():
client = yield tornado.websocket.websocket_connect("ws://localhost:8888/ws")
client.write_message("Testing from client")
msg = yield client.read_message()
print("msg is %s" % msg)
msg = yield client.read_message()
print("msg is %s" % msg)
msg = yield client.read_message()
print("msg is %s" % msg)
client.close()
if __name__ == "__main__":
tornado.ioloop.IOLoop.instance().run_sync(test_ws)
You can then run the server, and have two instances of the test client connect. When you do, the server prints this:
bennu#daveadmin:~$ ./torn.py
new connection
message received Testing from client
new connection
message received Testing from client
<15 second delay>
Writing to clients
connection closed
connection closed
The first client prints this:
bennu#daveadmin:~$ ./web_client.py
msg is Hello World
msg is ECHO: Testing from client
< 15 second delay>
msg is Hi there! 0
And the second prints this:
bennu#daveadmin:~$ ./web_client.py
msg is Hello World
msg is ECHO: Testing from client
< 15 second delay>
msg is Hi there! 1
For the purposes of the example, I just had the server send the message to the clients on a 15 second delay, but it could be triggered by whatever you want.

my solution for this: first add "if __name__ == '__main__':" - to the main.py. then import main.py into the websocket module. e.g. (import main as MainApp) . it is now possible to call a function in 'main.py' from within the ws.py/WebSocketHandler-function. - inside the Handler pass the message like so:
MainApp.function(message)
i dunno if this is the opposite of elegant but it works for me.
..plus create and import a custom 'config.py' (thats looks like: someVar = int(0) ) into the 'mainApp.py' .. like so: import config as cfg --> now you can alter variables with cfg.someVar = newValue from inside the function in 'main.py' that once was called by the Handler from 'ws.py'.

How do I map incoming "path" requests when using HTTPServer?

I'm fairly new to coding in python. I created a local web server that says "Hello World" and displays the current time.
Is there a way to create a path, without creating a file, on the server program so that when I type in "/time" after 127.0.0.1 in the browser bar, it will display the current time? Likewise if I type "/date" it will give me the current date.
This is what I have so far:
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer
import datetime
port = 80
class myHandler(BaseHTTPRequestHandler):
#Handler for the GET requests
def do_GET(self):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
# Send the html message
self.wfile.write("<b> Hello World !</b>"
+ "<br><br>Current time and date: " + str(datetime.datetime.now()))
server = HTTPServer(('', port), myHandler)
print 'Started httpserver on port ', port
#Wait forever for incoming http requests
server.serve_forever()

Very simple URL handler:
def do_GET(self):
if self.path == '/time':
do_time(self)
elif self.path == '/date':
do_date(self)
def do_time(self):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
# Send the html message
self.wfile.write("<b> Hello World !</b>"
+ "<br><br>Current time: " + str(datetime.datetime.now()))