I am serving a local directory with Twisted http server as:
from twisted.web.server import Site
from twisted.web.static import File
from twisted.internet import reactor, endpoints
resource = File('./')
factory = Site(resource)
reactor.listenTCP(8888,factory)
reactor.run()
The server serves the specified directory properly. But, when there is any index file (index, index.html) in a directory, then the server displays that index file, instead of serving the local directory.
So, how do I tell Twisted server to ignore the index files and continue to serve the local directory?
twisted.web.static.File uses self.indexNames to determine which files are considered index files. You can override this value to change the behavior.
from twisted.web.server import Site
from twisted.web.static import File
from twisted.internet import reactor, endpoints
resource = File('./')
resource.indexNames = []
factory = Site(resource)
reactor.listenTCP(8888,factory)
reactor.run()
Related
I'm running a cherrypy based app on an openshift gear. Recently I've been getting a "503 service temporarily unavailable" error whenever I try to go to the site. Inspecting the logs, I see I'm getting an ImportError where I try to import CherryPy. This is strange - CherryPy is listed as a dependency in my requirements.txt and used to be imported just fine. I double checked to make sure I'm getting the right path to the openshift activate_this.py and it seems to be correct. I'm not quite sure where to look next; any help would be appreciated. Thanks!
The failed import is at line 14 of app.py:
import os
import files
virtenv = os.path.join(os.environ['OPENSHIFT_PYTHON_DIR'], 'virtenv')
virtualenv = os.path.join(virtenv, 'bin', 'activate_this.py')
conf = os.path.join(files.get_root(), "conf", "server.conf")
try:
execfile(virtualenv, dict(__file__=virtualenv))
print virtualenv
except IOError:
pass
import cherrypy
import wsgi
def mount():
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = os.environ['OPENSHIFT_APP_DNS']
cherrypy.config.update({"tools.staticdir.root": files.get_root()})
cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
cherrypy.tree.mount(wsgi.application(), "/", conf)
def start():
cherrypy.engine.start()
def end():
cherrypy.engine.exit()
if __name__ == "__main__":
mount()
start()
UPDATE
I eventually saw (when pushing to the openshift repo using git bash CLI) that the dependency installation from requirements.txt was failing with some exceptions I haven't bothered to look into yet. It then goes on to try to install dependencies in setup.py, and that works just fine.
Regarding the port in use issue...I have no idea. I changed my startup from tree.mount and engine.start to quickstart, and everything worked when I pushed to openshift. Just for kicks (and because I need it to run my tests), I switched back to cherrypy.tree.mount, pushed it, and it worked just fine.
Go figure.
I use the app.py entry point for Openshift. Here are several examples on how I start my server using the pyramid framework on Openshift. I use waitress as the server but I have also used the cherrypy wsgi server. Just comment out the code you don't want.
app.py
#Openshift entry point
import os
from pyramid.paster import get_app
from pyramid.paster import get_appsettings
if __name__ == '__main__':
here = os.path.dirname(os.path.abspath(__file__))
if 'OPENSHIFT_APP_NAME' in os.environ: #are we on OPENSHIFT?
ip = os.environ['OPENSHIFT_PYTHON_IP']
port = int(os.environ['OPENSHIFT_PYTHON_PORT'])
config = os.path.join(here, 'production.ini')
else:
ip = '0.0.0.0' #localhost
port = 6543
config = os.path.join(here, 'development.ini')
app = get_app(config, 'main') #find 'main' method in __init__.py. That is our wsgi app
settings = get_appsettings(config, 'main') #don't really need this but is an example on how to get settings from the '.ini' files
# Waitress (remember to include the waitress server in "install_requires" in the setup.py)
from waitress import serve
print("Starting Waitress.")
serve(app, host=ip, port=port, threads=50)
# Cherrypy server (remember to include the cherrypy server in "install_requires" in the setup.py)
# from cherrypy import wsgiserver
# print("Starting Cherrypy Server on http://{0}:{1}".format(ip, port))
# server = wsgiserver.CherryPyWSGIServer((ip, port), app, server_name='Server')
# server.start()
#Simple Server
# from wsgiref.simple_server import make_server
# print("Starting Simple Server on http://{0}:{1}".format(ip, port))
# server = make_server(ip, port, app)
# server.serve_forever()
#Running 'production.ini' method manually. I find this method the least compatible with Openshift since you can't
#easily start/stop/restart your app with the 'rhc' commands. Mabye somebody can suggest a better way :)
# #Don't forget to set the Host IP in 'production.ini'. Use 8080 for the port for Openshift
# You will need to use the 'pre_build' action hook(pkill python) so it stops the existing running instance of the server on OS
# You also will have to set up another custom action hook so rhc app-restart, stop works.
# See Openshifts Origin User's Guide ( I have not tried this yet)
#Method #1
# print('Running pserve production.ini')
# os.system("pserve production.ini &")
#Method #2
#import subprocess
#subprocess.Popen(['pserve', 'production.ini &'])
If I have for example this simple TCP server:
from twisted.internet import reactor
from twisted.web.resource import Resource
from twisted.web.server import Site
from resources import SomeResource
logging.info("Starting server...")
root = Resource()
root.putChild("test", SomeResource())
reactor.listenTCP(8080, Site(root))
reactor.run()
With SomeResource which has the render_GET and render_POST methods for example.
Then I know I can just send a POST/GET to hostname:8080/test
But now I want to make it more complicated, I would like to do something like hostname:8080/test/status
Could that be defined inside SomeResource() as a method? or do I have to define a new resource for every different url?
If you want everything that goes to /test/.... to get to the render (render_GET/render_POST) method of SomeResource, just define it as a leaf:
class SomeResource(Resource):
isLeaf = True
If you want to look at the part after "test/", request.postpath will include that.
I wanna be able to run multiple twisted proxy servers on different directories on the same port simultaneously, and I figured I might use flask.
so here's my code:
from flask import Flask
from twisted.internet import reactor
from twisted.web import proxy, server
app = Flask(__name__)
#app.route('/example')
def index():
site = server.Site(proxy.ReverseProxyResource('www.example.com', 80, ''.encode("utf-8")))
reactor.listenTCP(80, site)
reactor.run()
app.run(port=80, host='My_IP')
But whenever I run this script, I get an Internal Server Error, I'm assuming because when app.run is called on port 80, the reactor.run can't be listening on port 80 as well. I wondering if there is some kind of work around to this, or what it is I'm doing wrong. Any help is greatly appreciated, Thanks!!
You can use the WSGIResource from Twisted istead of a ReverseProxy.
UPDATE: Added a more complex example that sets up a WSGIResource at /my_flask and a ReverseProxy at /example
from flask import Flask
from twisted.internet import reactor
from twisted.web.proxy import ReverseProxyResource
from twisted.web.resource import Resource
from twisted.web.server import Site
from twisted.web.wsgi import WSGIResource
app = Flask(__name__)
#app.route('/example')
def index():
return 'My Twisted Flask'
flask_site = WSGIResource(reactor, reactor.getThreadPool(), app)
root = Resource()
root.putChild('my_flask', flask_site)
site_example = ReverseProxyResource('www.example.com', 80, '/')
root.putChild('example', site_example)
reactor.listenTCP(8081, Site(root))
reactor.run()
Try running the above in your localhost and then visiting localhost:8081/my_flask/example or localhost:8081/example
You should give klein a try. It's made and used by most of the twisted core devs. The syntax is very much like flask so you won't have to rewrite much if you already have a working flask app. So something like the following should work:
from twisted.internet import reactor
from twisted.web import proxy, server
from klein import Klein
app = Klein()
#app.route('/example')
def home(request):
site = server.Site(proxy.ReverseProxyResource('www.example.com', 80, ''.encode("utf-8")))
reactor.listenTCP(80, site)
app.run('localhost', 8000) # start the klein app on port 8000 and reactor event loop
Links
Klein Docs
Klein Github
The accepted answer does not cover how to run twisted with Flask, and points to a different framework. The answer with an example no longer works either.
Here are two different examples. The first one is the same as the first answer, but fixed to work on Python 3
from flask import Flask
from twisted.internet import reactor
from twisted.web.proxy import ReverseProxyResource
from twisted.web.resource import Resource
from twisted.web.server import Site
from twisted.web.wsgi import WSGIResource
app = Flask(__name__)
#app.route('/example')
def index():
return 'My Twisted Flask'
flask_site = WSGIResource(reactor, reactor.getThreadPool(), app)
root = Resource()
root.putChild(b'my_flask', flask_site)
site_example = ReverseProxyResource('www.example.com', 80, b'/')
root.putChild(b'example', site_example)
reactor.listenTCP(8081, Site(root))
reactor.run()
For this example, run it and open any of these:
localhost:8081/my_flask/example
localhost:8081/example
This other example is recommended, since it sets up two services and provides them through a .tac file to twistd.
Take the base code from here: https://github.com/pika/pika/blob/master/examples/twisted_service.py
"""Modify the bottom of the file to pick the new MultiService"""
# ... all the code from twisted_service.py goes here.
# Scroll to the bottom of the file and
# remove everything below application = service.Application("pikaapplication")
# You should keep the PikaService, PikaProtocol and PikaFactory
# classes, since we need them for this code:
from pika.connection import ConnectionParameters
from pika.credentials import PlainCredentials
from twisted.application import service, strports
from twisted.internet import reactor
from twisted.web.server import Site
from twisted.web.wsgi import WSGIResource
from flask import Flask
# This IServiceCollection will hold Pika and Flask
flask_pika_multiservice = service.MultiService()
# FLASK SERVICE SETUP
app = Flask("demoapp")
#app.route('/')
def hello_world():
return 'Hello, World!'
flask_site = Site(WSGIResource(reactor, reactor.getThreadPool(), app))
# New resources can be added, such as WSGI, or proxies by creating
# a root resource in the place of the flask_site, and adding the
# new resources to the root.
# root = Resource()
# root.putChild(b'my_flask_site', flask_site)
# from twisted.web.proxy import ReverseProxyResource
# site_example = ReverseProxyResource('www.example.com', 80, b'/')
# root.putChild(b'example', site_example)
i = strports.service(f"tcp:8088", flask_site)
i.setServiceParent(flask_pika_multiservice)
# PIKA SERVICE SETUP
ps = PikaService(
ConnectionParameters(
host="localhost",
virtual_host="/",
credentials=PlainCredentials("guest", "guest")))
ps.setServiceParent(flask_pika_multiservice)
# Application setup
application = service.Application('flask_pika')
flask_pika_multiservice.setServiceParent(application)
Now you can run it with:
PYTHONPATH=. twistd -ny twisted_service.py
you can skip the python path if you don't want to import anything from the same path. twisted expects projects to actually be installed, and does not support running them directly from the source folder unless you use that workaround.
This second example establishes two services, on different ports. It's for pika and flask running simultaneously on the same twisted server. The best part is that it shows how to set up flask as a service, that can be part of an IServiceCollection
I want to be able to run twisted servers on multiple different directories (exp: /example1, /example2...etc), So I thought I'd use flask. Here is what I have so far:
from flask import Flask
from twisted.web.server import Site
from twisted.web.wsgi import WSGIResource
from twisted.internet import reactor
from twisted.web.proxy import ReverseProxyResource
from twisted.web.resource import Resource
app = Flask(__name__)
#app.route('/example1')
def index():
return 'My Twisted Flask'
flask_site = WSGIResource(reactor, reactor.getThreadPool(), app)
root = Resource()
root.putChild('my_flask', flask_site)
site_example = ReverseProxyResource('www.example.com', 80, ''.encode('utf-8'))
root.putChild('example1', site_example)
reactor.listenTCP(80, Site(root))
reactor.run()
The only problem is that it doesn't work, I'm not sure what I'm doing wrong. I appreciate any help, thanks!
My personal opinion: running Flask in Twisted's reactor isn't a good idea because Twisted's reactor is blocked when a request is processed by Flask.
I think you might be interested in Klein, which provided API similar to Flask, but works on Twisted out of the box: http://klein.readthedocs.io/en/latest/
Another option: I'd take a look into nginx as a reverse proxy for Flask applications instead of Twisted. nginx runs in a separate process and isn't blocked while a request is processed by Flask.
https://www.nginx.com/resources/admin-guide/reverse-proxy/
You can use twisted web, as documented on the Flask deploy documentation. Here's how I managed to run a server on my machine:
pip3 install twisted[tls]
export PYTHONPATH=${PYTHONPATH}:${PWD} # exports python path
twistd web -n --port tcp:5000 --wsgi path-to-your-app-root --logfile log.txt
Though I've had some issues with the server after it's up and running for my particular scenario, this might work for you
Hypothetical
I've got a Linux VPS server named myserver, mounted at myserver.com.
On this server I have two Flask WSGI apps app-one and app-two.
I'd like to deploy these two apps on myserver, but I want them mounted on the domains app-one.com and app-two.com.
The apps have no external dependencies (no databases, caches, etc). They're stand-alone, single-file apps.
I do not have Apache, NGinX or any other webserver software installed; just Python, Flask and two WSGI apps.
I have Python/Flask experience, but I don't have a lot of experience with WSGI deployment or multiple-domain work. Basic instructions and/or reading material appreciated.
Question / TL;DR
How can I use a server mounted at one domain to deploy two WSGI apps to two domains? Do I need to install software especially for this case, or is it just a matter of pointing the apps at my chosen domains?
Thank you for any and all advice.
Once you have DNS set up to point both app-one.com and app-two.com to myserver.com's IP address then you need to set up something to route requests coming in on port 80 (or 443 if you are going to use SSL) to each of your apps. This is normally done with virtual hosts in Apache or nginx.
If you need to run both applications in the same Python process (whether you are using a non-Python webserver as your application container or not) then you will need to dispatch to each of your apps by hand:
from werkzeug.exceptions import NotImplemented
from werkzeug.wsgi import get_host
class DomainDispatcher(object):
"""Simple domain dispatch"""
def __init__(self, domain_handlers, default_handler=None):
self.domain_handlers = domain_handlers
self.default_handler = domain_handlers.get("default", default_handler)
if self.default_handler is None:
self.default_handler = NotImplemented()
def __call__(self, environ, start_response):
host = get_host(environ)
handler = self.domain_handlers.get(host, self.default_handler)
return handler(environ, start_response)
An example of usage:
from app_one import app as app1
from app_two import app as app2
from domain_dispatcher import DomainDispatcher
dispatcher = DomainDispatcher({'app-one.com': app1, 'app-two.com': app2})
if __name__ == '__main__':
# Wrap dispatcher in a WSGI container
# such as CherryPy