We Keep Coding

How to make an HTTP server listen on multiple ports? [duplicate]

I'm writing a small web server in Python, using BaseHTTPServer and a custom subclass of BaseHTTPServer.BaseHTTPRequestHandler. Is it possible to make this listen on more than one port?
What I'm doing now:
class MyRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def doGET
[...]
class ThreadingHTTPServer(ThreadingMixIn, HTTPServer):
pass
server = ThreadingHTTPServer(('localhost', 80), MyRequestHandler)
server.serve_forever()

Sure; just start two different servers on two different ports in two different threads that each use the same handler. Here's a complete, working example that I just wrote and tested. If you run this code then you'll be able to get a Hello World webpage at both http://localhost:1111/ and http://localhost:2222/
from threading import Thread
from SocketServer import ThreadingMixIn
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "text/plain")
self.end_headers()
self.wfile.write("Hello World!")
class ThreadingHTTPServer(ThreadingMixIn, HTTPServer):
daemon_threads = True
def serve_on_port(port):
server = ThreadingHTTPServer(("localhost",port), Handler)
server.serve_forever()
Thread(target=serve_on_port, args=[1111]).start()
serve_on_port(2222)
update:
This also works with Python 3 but three lines need to be slightly changed:
from socketserver import ThreadingMixIn
from http.server import HTTPServer, BaseHTTPRequestHandler
and
self.wfile.write(bytes("Hello World!", "utf-8"))

Not easily. You could have two ThreadingHTTPServer instances, write your own serve_forever() function (don't worry it's not a complicated function).
The existing function:
def serve_forever(self, poll_interval=0.5):
"""Handle one request at a time until shutdown.
Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks, do them in
another thread.
"""
self.__serving = True
self.__is_shut_down.clear()
while self.__serving:
# XXX: Consider using another file descriptor or
# connecting to the socket to wake this up instead of
# polling. Polling reduces our responsiveness to a
# shutdown request and wastes cpu at all other times.
r, w, e = select.select([self], [], [], poll_interval)
if r:
self._handle_request_noblock()
self.__is_shut_down.set()
So our replacement would be something like:
def serve_forever(server1,server2):
while True:
r,w,e = select.select([server1,server2],[],[],0)
if server1 in r:
server1.handle_request()
if server2 in r:
server2.handle_request()

I would say that threading for something this simple is overkill. You're better off using some form of asynchronous programming.
Here is an example using Twisted:
from twisted.internet import reactor
from twisted.web import resource, server
class MyResource(resource.Resource):
isLeaf = True
def render_GET(self, request):
return 'gotten'
site = server.Site(MyResource())
reactor.listenTCP(8000, site)
reactor.listenTCP(8001, site)
reactor.run()
I also thinks it looks a lot cleaner to have each port be handled in the same way, instead of having the main thread handle one port and an additional thread handle the other. Arguably that can be fixed in the thread example, but then you're using three threads.

global name 'WebServer' is not defined

Please find below code as it is throwing the error - global name 'webserver' is not defined.
import SimpleHTTPServer
import SocketServer
import os
from threading import Thread
import threading
class WebServer(Thread):
def __init__(self, stream_path):
"""
Class to create a Web server and add given folder to the web server which is passed as an argument.
In this case it creates the web server to the incoming streams pushed by VLC to temp folder
"""
Thread.__init__(self)
self.stream_path = stream_path
def run(self):
global WebServer
"""
This method is built in Thread object method invoked by start()
and the code which is under run() will be executed.
"""
os.chdir(self.stream_path)
PORT = 8000
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler)
print "serving at port", PORT
print "function thread", threading.currentThread()
httpd.serve_forever()
print "test1"
def create_web_server(self,stream_path):
global WebServer
"""
This method is to create the web server to a given path
"""
obj1 = WebServer(self,stream_path)
obj1.start()
print "server created"
def stop_web_server(self):
cmd='tskill python /A'
os.system(cmd)
if __name__ == '__main__':
create_web_server("","C:\\\\QED")

You don't need the two global WebServer lines in your code. The global keyword is used to grant write access to a global variable, and you don't have a global variable named WebServer.
Your code should look like the following to resolve your error, though in its state, it will still throw errors because your code has multiple problems, including calling the create_web_server method by itself, as it's a method that needs to be run on a WebServer object.
I've included a working prototype example of your code at the bottom of my post.
import SimpleHTTPServer
import SocketServer
import os
from threading import Thread
import threading
class WebServer(Thread):
def __init__(self, stream_path):
"""
Class to create a Web server and add given folder to the web server which is passed as an argument.
In this case it creates the web server to the incoming streams pushed by VLC to temp folder
"""
Thread.__init__(self)
self.stream_path = stream_path
def run(self):
"""
This method is built in Thread object method invoked by start()
and the code which is under run() will be executed.
"""
os.chdir(self.stream_path)
PORT = 8000
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler)
print "serving at port", PORT
print "function thread", threading.currentThread()
httpd.serve_forever()
print "test1"
def create_web_server(self,stream_path):
"""
This method is to create the web server to a given path
"""
obj1 = WebServer(self,stream_path)
obj1.start()
print "server created"
def stop_web_server(self):
cmd='tskill python /A'
os.system(cmd)
if __name__ == '__main__':
create_web_server("","C:\\\\QED")
Here is a simplified version of your code that should do the same thing (serve a directory with a basic SimpleHTTPRequestHandler).
It runs the thread in daemon mode so that it can be interrupted with Ctrl-C. I removed several methods as they didn't seem to serve a purpose in your original code, and to show you a basic framework of what your WebServer class would probably look like.
import threading
import os
import time
import SimpleHTTPServer
import SocketServer
class WebServer(threading.Thread):
# Let's specify additional constructor arguments for host/port
# so you don't have to hardcode those values.
# We can specify default values for them as well.
def __init__(self, stream_path, host='localhost', port=8000):
super(WebServer, self).__init__()
self.stream_path = stream_path
self.host = host
self.port = port
# This is the method that will be called when you call
# .start() on a WebServer object.
def run(self):
os.chdir(self.stream_path)
handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer((self.host, self.port), handler)
httpd.serve_forever()
if __name__ == '__main__':
# Create a WebServer thread object, set it to daemonize
# and start it
ws = WebServer(r'C:\QED')
ws.daemon = True
ws.start()
# Since the main thread doesn't do anything after daemonizing
# the WebServer, it would exit immediately.
# This forever loop just ensures that the main thread runs
while True: time.sleep(1)
And to take it a step further, you don't need to create a Thread subclass. It's allowed because it feels more 'natural' to developers who are familiar with Java, where subclassing Thread or implementing Runnable is standard practice.
You can just do something like:
import threading
import os
import time
import SimpleHTTPServer
import SocketServer
def run_http_server(serve_dir, host='localhost', port=8000):
os.chdir(serve_dir)
handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer((host, port), handler)
httpd.serve_forever()
if __name__ == '__main__':
thread = threading.Thread(target=run_http_server, args=(r'C:\QED',))
thread.daemon = True
thread.start()
while True: time.sleep(1)

createResolver() in twisted not working

I have a simple proxy script where I use installResolver function to choose where the script reads its hosts file:
from twisted.internet import reactor
from twisted.web import proxy, server
from twisted.python import log
from twisted.names import client
def createResolver(servers=None, resolvconf=None, hosts=None):
if platform.getType() == 'posix':
if resolvconf is None:
resolvconf = b'/etc/resolv.conf'
if hosts is None:
hosts = b'/root/example'
theResolver = Resolver(resolvconf, servers)
hostResolver = hostsModule.Resolver(hosts)
else:
if hosts is None:
hosts = r'c:\windows\hosts'
from twisted.internet import reactor
bootstrap = _ThreadedResolverImpl(reactor)
hostResolver = hostsModule.Resolver(hosts)
theResolver = root.bootstrap(bootstrap, resolverFactory=Resolver)
L = [hostResolver, cache.CacheResolver(), theResolver]
return resolve.ResolverChain(L)
site = server.Site(proxy.ReverseProxyResource('www.example.com', 80, ''.encode("utf-8")))
reactor.listenTCP(80, site)
reactor.run()
Whenever I run this script, It just times out and doesn't work. Could anybody tell me what it is I'm doing wrong? thanks!!

createResolver() in twisted not working
You need to pass in the hosts file you want it to read and you need to make sure that it's readable ie where it is /root/example means you need to be running as root to read it. You don't need to override the method. I'll install it tomorrow and try it.
Create a file /tmp/example with an entry in it ie I used
127.0.0.6 twistfoo.com
This is working code
from twisted.names import client
from twisted.internet import reactor
def do_lookup(domain):
hosts = "/tmp/example"
resolver = client.createResolver(servers=None, resolvconf=None, hosts="/tmp/example")
d = resolver.getHostByName(domain)
d.addBoth(lookup_done)
def lookup_done(result):
print(result)
reactor.stop()
domain = b'twistfoo.com'
reactor.callLater(0, do_lookup, domain)
reactor.run()
Run this on command line and you will see the output
127.0.0.6

Python: Twisted server and values

I'd like access to the same value in my protocol and in my factory so I made a test with this code:
import time
from multiprocessing import Process
from twisted.internet import reactor, protocol
class MyServer(protocol.Protocol):
def connectionMade(self):
self.factory.clients.append("client")
print self.factory.clients
class MyServerFactory(protocol.Factory):
def __init__(self):
self.protocol = MyServer
self.clients = []
def printClient(self):
print self.clients
if __name__ == '__main__':
factory = MyServerFactory()
reactor.listenTCP(4433, factory)
processTwisted = Process(target=reactor.run)
processTwisted.start()
time.sleep(10)
factory.printClient()
During the sleep I connect client to the server.
This is the console's log :
['client']
[]
And I expected:
['client']
['client']
How can do it ?

Twisted and multiprocessing don't work together like this. Also, lists and multiprocessing don't work together like this.
If you remove the use of multiprocessing you'll get the behavior you want.

twisted - get OS-chosen listen port

I am writing a twisted P2P client using the application framework. The listen port for incoming connections will be on a random (OS-determined) port. However, I need a way to determine what that port is after creating it:
import twisted... etc.
application = service.Application('vmesh')
peerservice = MyPeerService()
servicecollection = service.IServiceCollection(application)
factory = MyPeerFactory(peerservice)
server = internet.TCPServer(0, factory) # listen on random port
listen_port = server.getHost().port # ??? doesn't work...
server.setServiceParent(servicecollection)
I can't find anything in the docs about querying the port created by internet.TCPServer() or by reactor.listenTCP() which it forwards to. I can't simply wait for a connection to occur since the client has to announce its port in order for those connections to ever happen.

listenTCP returns an IListeningPort, which has a getHost() method that gives back an object with a port. For example:
>>> from twisted.internet import reactor
>>> from twisted.internet.protocol import Factory
>>> port = reactor.listenTCP(0, Factory())
>>> port.getHost().port
55791
However, TCPServer doesn't call listenTCP until it is started with privilegedStartService. Plus, the IListeningPort isn't actually exposed via a public API. So, you will need to write your own Service. Luckily, it's quite easy to do this; TCPServer doesn't do very much. You just need to write one that reports its port somewhere as soon as it starts listening. Here's an example:
from twisted.internet import reactor
from twisted.application.service import Service
class PortReporter(Service, object):
def __init__(self, factory, reportPort):
self.factory = factory
self.reportPort = reportPort
def privilegedStartService(self):
self.listeningPort = reactor.listenTCP(0, self.factory)
self.reportPort(self.listeningPort.getHost().port)
return super(PortReporter, self).privilegedStartService()
def stopService(self):
self.listeningPort.stopListening()
return super(PortReporter, self).stopService()
You can then use this in a tac file, like so:
from twisted.internet.protocol import Factory
from twisted.application.service import Application
application = Application("test")
def showPortNumber(n):
print("The port number is: %d" % (n,))
PortReporter(Factory(), showPortNumber).setServiceParent(application)

FWIW if you need to do this with endpoints here is my implementation with a slight tweak for my local setup (the callback option would work well here too):
class PortReporter(StreamServerEndpointService, object):
def __init__(self, endpoint, factory):
StreamServerEndpointService.__init__(self, endpoint, factory)
self._reportedPort = None
def privilegedStartService(self):
r = super(PortReporter, self).privilegedStartService()
self._waitingForPort.addCallback(self.port_cb)
return r
def port_cb(self, port):
self._reportedPort = port.getHost().port
return port
def getReportedPort(self):
return self._reportedPort

You can access the port bind to your server like so if you didn't start the server yet (didn't call startService yet):
>>> serv._getPort()._realPortNumber
Else you can also do:
>>> serv._port._realPortNumber