From other SO posts, it's clear how to serve content from a specific directory, and how to map an incoming path to different do_GET handlers.
To expand on the second question in a way relating to the first, how do you map paths to different kinds of handlers? Specifically, I'd like to map one path to do_GET handler, and another to just serving the content from a specific directory.
If it is not possible, what's the easier way to serve the two different kinds of contents? I know the two could be run on the server in two threads each serving a different port, that's not very neat.
I've got an answer by tracking the code from the first reference question answered by Jaymon, and incorporating the code from the second reference question.
The sample follows. It serves content on the local machine from the directory web/ to the URL base path /file/, and handles requests with URL base path /api in the user-supplied method do_GET() itself. Initially the code was derived from a sample on the web by Dr. Axel Rauschmayer.
#!/usr/bin/env python
# https://2ality.com/2014/06/simple-http-server.html
# https://stackoverflow.com/questions/39801718/how-to-run-a-http-server-which-serves-a-specific-path
from SimpleHTTPServer import SimpleHTTPRequestHandler
from BaseHTTPServer import HTTPServer as BaseHTTPServer
import os
PORT = 8000
class HTTPHandler(SimpleHTTPRequestHandler):
"""This handler uses server.base_path instead of always using os.getcwd()"""
def translate_path(self, path):
if path.startswith(self.server.base_url_path):
path = path[len(self.server.base_url_path):]
if path == '':
path = '/'
else:
#path = '/'
return None
path = SimpleHTTPRequestHandler.translate_path(self, path)
relpath = os.path.relpath(path, os.getcwd())
fullpath = os.path.join(self.server.base_local_path, relpath)
return fullpath
def do_GET(self):
path = self.path
if (type(path) is str or type(path) is unicode) and path.startswith('/api'):
# call local handler
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
# Send the html message
self.wfile.write("<b> Hello World !</b>")
self.wfile.close()
return
elif (type(path) is str or type(path) is unicode) and path.startswith(self.server.base_url_path):
return SimpleHTTPRequestHandler.do_GET(self)
elif (type(path) is str or type(path) is unicode) and path.startswith('/'):
self.send_response(441)
self.end_headers()
self.wfile.close()
return
else:
self.send_response(442)
self.end_headers()
self.wfile.close()
return
Handler = HTTPHandler
Handler.extensions_map.update({
'.webapp': 'application/x-web-app-manifest+json',
});
class HTTPServer(BaseHTTPServer):
"""The main server, you pass in base_path which is the path you want to serve requests from"""
def __init__(self, base_local_path, base_url_path, server_address, RequestHandlerClass=HTTPHandler):
self.base_local_path = base_local_path
self.base_url_path = base_url_path
BaseHTTPServer.__init__(self, server_address, RequestHandlerClass)
web_dir = os.path.join(os.path.dirname(__file__), 'web')
httpd = HTTPServer(web_dir, '/file', ("", PORT))
print "Serving at port", PORT
httpd.serve_forever()
Related
I Need to write a server program in Python serving webpages and handling other GET and POST requests to and from client. I'm new to servers in Python, so I looked up some examples and after a while I had a basic Requesthandler running with some routing to my pages as a start. Routing worked in browser and pages were displayed there but I only got text, no styles, no pictures. Then I looked a bit further and realised that I also needed to handle GET requests for these .css, .js,.jpg files. So I did that, and ended up with smth like this:
class Serv(BaseHTTPRequestHandler):
def do_GET(self):
#route incoming path to correct page
if self.path in("","/"):
self.path = "/my_site/index.html"
#TODO do same for every page in the site
if self.path == "/foo":
self.path = "/my_site/fooandstuff.html"
if self.path == "/bar":
self.path = "/my_site/subdir/barfly.html"
try:
sendReply = False
if self.path.endswith(".html"):
mimetype = "text/html"
sendReply = True
if self.path.endswith(".jpg"):
mimetype = "image/jpg"
sendReply = True
if self.path.endswith(".js"):
mimetype = "application/javascript"
sendReply = True
if self.path.endswith(".css"):
mimetype = "text/css"
sendReply = True
if sendReply == True:
f = open(self.path[1:]).read()
self.send_response(200)
self.send_header('Content-type',mimetype)
self.end_headers()
self.wfile.write(f.encode(mimetype))
return
except IOError:
self.send_error(404, "File not found %s" % self.path)
When I run this and request a page, I get the following LookupError:
File "d:/somedir/myfile.py", line 47, in do_GET
self.wfile.write(f.encode(mimetype))
LookupError: unknown encoding: text/html
if I change text/html to utf-8, that seems te "solve" the problem, but then I run into the same Lookuperror but this time for image/jpg, and so on. It seems like wfile.write only accepts utf-8, although , when I look around, I see people passing file.read() just like that to wfile.write
wfile.write(file.read())
and for them it seems to work. Yet, when I do that, what I get is
File "C:\Users\myuser\AppData\Local\Programs\Python\Python37\lib\socketserver.py", line 799, in write
self._sock.sendall(b)
TypeError: a bytes-like object is required, not 'str'
What could cause this to happen?
for server handling with python better lookup flask
sample code will look like
from flask import Flask, render_template, url_for, request, redirect
import csv
#app.route('/')
def my_home():
return render_template('index.html')
#app.route('/<string:page_name>')
def html_page(page_name):
return render_template(page_name)
put all HTML in the same folder as your server.py in a folder called [template]
and all CSS and java in folder called [static] assets and all include. dont forget to change paths in css, java and html
The answer was in the opening of an image file, that needed an extra argument "rb" , like this:
if mimetype != "image/jpg":
f = open(self.path[1:])
else:
f = open(self.path[1:], "rb")
and then also:
if mimetype == "image/jpg":
self.wfile.write(f.read())
else:
self.wfile.write(f.read().encode("utf-8"))
I like to use Python's SimpleHTTPServer for local development of all kinds of web applications which require loading resources via Ajax calls etc.
When I use query strings in my URLs, the server always redirects to the same URL with a slash appended.
For example /folder/?id=1 redirects to /folder/?id=1/ using a HTTP 301 response.
I simply start the server using python -m SimpleHTTPServer.
Any idea how I could get rid of the redirecting behaviour? This is Python 2.7.2.
The right way to do this, to ensure that the query parameters remain as they should, is to make sure you do a request to the filename directly instead of letting SimpleHTTPServer redirect to your index.html
For example http://localhost:8000/?param1=1 does a redirect (301) and changes the url to http://localhost:8000/?param=1/ which messes with the query parameter.
However http://localhost:8000/index.html?param1=1 (making the index file explicit) loads correctly.
So just not letting SimpleHTTPServer do a url redirection solves the problem.
Okay. With the help of Morten I've come up with this, which seems to be all I need: Simply ignoring the query strings if they are there and serving the static files.
import SimpleHTTPServer
import SocketServer
PORT = 8000
class CustomHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
def __init__(self, req, client_addr, server):
SimpleHTTPServer.SimpleHTTPRequestHandler.__init__(self, req, client_addr, server)
def do_GET(self):
# cut off a query string
if '?' in self.path:
self.path = self.path.split('?')[0]
SimpleHTTPServer.SimpleHTTPRequestHandler.do_GET(self)
class MyTCPServer(SocketServer.ThreadingTCPServer):
allow_reuse_address = True
if __name__ == '__main__':
httpd = MyTCPServer(('localhost', PORT), CustomHandler)
httpd.allow_reuse_address = True
print "Serving at port", PORT
httpd.serve_forever()
I'm not sure how the redirect is generated... I've tried implementing a very basic SimpleHTTPServer, and I don't get any redirects when using query string params.
Just do something like self.path.split("/") and process the path before handling the request?
This code does what you want I think:
import SocketServer
import SimpleHTTPServer
import os
class CustomHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
def folder(self):
fid = self.uri[-1].split("?id=")[-1].rstrip()
return "FOLDER ID: %s" % fid
def get_static_content(self):
# set default root to cwd
root = os.getcwd()
# look up routes and set root directory accordingly
for pattern, rootdir in ROUTES:
if path.startswith(pattern):
# found match!
path = path[len(pattern):] # consume path up to pattern len
root = rootdir
break
# normalize path and prepend root directory
path = path.split('?',1)[0]
path = path.split('#',1)[0]
path = posixpath.normpath(urllib.unquote(path))
words = path.split('/')
words = filter(None, words)
path = root
for word in words:
drive, word = os.path.splitdrive(word)
head, word = os.path.split(word)
if word in (os.curdir, os.pardir):
continue
path = os.path.join(path, word)
return path
def do_GET(self):
path = self.path
self.uri = path.split("/")[1:]
actions = {
"folder": self.folder,
}
resource = self.uri[0]
if not resource:
return self.get_static_content()
action = actions.get(resource)
if action:
print "action from looking up '%s' is:" % resource, action
return self.wfile.write(action())
SimpleHTTPServer.SimpleHTTPRequestHandler.do_GET(self)
class MyTCPServer(SocketServer.ThreadingTCPServer):
allow_reuse_address = True
httpd = MyTCPServer(('localhost', 8080), CustomHandler)
httpd.allow_reuse_address = True
print "serving at port", 8080
httpd.serve_forever()
Try it out:
HTTP GET /folder/?id=500x -> "FOLDER ID: 500x"
EDIT:
Okay so if you haven't used the SimpleHTTPServer-stuff before, you basically implement the base request handler, implement do_GET(), do_PUT(), do_POST() etc.
What I usually do then is parse the request string (using re), pattern match and see if I can find a request-handler and if not, handle the request as a request for static content if possible.
You say you want to serve static content if at all possible, then you should flip this pattern matching around, and FIRST see if the request matches the file-store and if not, then match against handlers :)
I'm trying to make a small testing server for Bugzilla so I can test out changes I make before they are deployed to the main Apache based server. I'm most familiar with Python, and I was wanting to make Python's built-in HTTP server run Bugzilla's CGI programs.
Unfortunately, Bugzilla has lots more than CGI apps. It has a bunch of css and other data that is served up directly. This means the handler needs to deal with those as well. I would like to set up a WSGI handler that looks at the request URL and appropriately routes the request to either one of the Bugzilla CGI scripts or pulls the data directly from the filesystem.
Is there a better way to accomplish what I want to do? If there isn't, is there a WSGI app out there already that will set up a CGI environment and call out to a CGI app via Python's subprocess module?
Here is a solution that works, though it's rather ugly and kind of slow. It demands a separate CGIApplication object for each CGI script you want to run. So if you have a directory full of them, you will need to instantiate a different CGIApplication object for each one. When you instantiate it, of course, is up to you. You can choose to instantiate new ones for every request, but you will likely save a bit of time if you somehow avoid this.
import os
import os.path as _osp
import re
import subprocess
import io
import email.parser
env_forward = re.compile('^[A-Z][A-Z0-9_]*$')
header_match = re.compile(b'^(.*?\\n[ \\t\\r]*\\n)(.*)$', re.M | re.S)
env_whitelist = frozenset(('AUTH_TYPE', 'CONTENT_LENGTH', 'CONTENT_TYPE',
'DOCUMENT_ROOT', 'QUERY_STRING', 'PATH_INFO',
'PATH_TRANSLATED', 'REMOTE_ADDR', 'REMOTE_PORT',
'REMOTE_IDENT', 'REMOTE_USER', 'REQUEST_METHOD',
'REQUEST_URI', 'SCRIPT_NAME',
'SERVER_ADDR', 'SERVER_ADMIN', 'SERVER_NAME',
'SERVER_PORT', 'SERVER_PROTOCOL',
'SERVER_SIGNATURE', 'SERVER_SOFTWARE'))
class CGIApplication(object):
def __init__(self, appfname):
self._appfname = _osp.abspath(appfname)
def __call__(self, environ, start_respose):
appenv = {item[0]: item[1] \
for item in environ.items() \
if ((item[0] in env_whitelist) or
item[0].startswith('HTTP_'))}
appenv['GATEWAY_INTERFACE'] = 'CGI/1.1'
appenv['PATH'] = '/usr/local/bin:/usr/bin:/bin'
appenv['SCRIPT_FILENAME'] = self._appfname
nbytes_for_cgi = appenv.get('CONTENT_LENGTH', '')
nbytes_for_cgi = (int(nbytes_for_cgi) if nbytes_for_cgi != '' else 0)
args = [self._appfname]
query = environ.get('QUERY_STRING', None)
query = query.replace('+', ' ')
if '=' not in query:
args.append(query)
proc = subprocess.Popen(args,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env = appenv,
cwd = _osp.dirname(self._appfname))
bytes_read = 0
data_for_cgi = io.BytesIO()
while bytes_read < nbytes_for_cgi:
data = environ['wsgi.input'].read(nbytes_for_cgi - bytes_read)
bytes_read += len(data)
data_for_cgi.write(data)
data = None
data_for_cgi = data_for_cgi.getvalue()
output, errdata = proc.communicate(data_for_cgi)
data_for_cgi = None
proc.stdin.close()
proc.stdout.close()
proc.stderr.close()
try:
errdata = errdata.decode('utf-8')
except UnicodeDecodeError:
errdata = errdata.decode('iso8859-1')
environ['wsgi.errors'].write(errdata)
errdata = None
if proc.returncode != 0:
start_respose('500 Internal Server Error',
[('Content-Type', 'text/plain')])
return (b"CGI application died with non-zero return code.\n",)
else:
output_hdr = header_match.match(output)
output_hdr, output = output_hdr.groups()
parser = email.parser.HeaderParser()
headers = parser.parsestr(output_hdr.decode('iso8859-1'))
status = headers.get_all('Status', ['200 OK'])[-1]
del headers['Status']
start_respose(status, list(headers.items()))
return (output,)
Are you using pybugz? http://www.liquidx.net/pybugz/
http://code.google.com/p/pybugz/
Also, setting up WSGI through Django and AppEngine is really easy. Pretty quick to install and might be worth using as a work around. That would get you a testing server that should be able to handle bugzilla's cgi, css, etc.
Good Luck
In a normal application i set services
services = {
'users.login': login,
'test': router
}
but I would like to do like this:
services = [
'users.login',
'test'
]
and every request goes to router function. this takes 2 params: service name (can be "users.login" or "test" in this case) and input (that is the object sent by flex to python)
then if the command(service) from flex is named "users.login". I would like to run the router with the params and then this will open the function login in the commands.users.login package. How would I do that? Thanks.
If I understand your question correctly, in order to achieve this, you are going to need to to override getServiceRequest on the Gateway class that you are using:
from pyamf.remoting.gateway.django import DjangoGateway
from pyamf.remoting.gateway import UnknownServiceError
class MyGateway(DjangoGateway):
def __init__(self, router_func, **kwargs):
self.router = router_func
DjangoGateway.__init__(self, **kwargs)
def getServiceRequest(self, request, target):
try:
return DjangoGateway.getServiceRequest(self, request, target)
except UnknownServiceError, e:
pass
# cached service was not found, try to discover it
try:
service_func = self.router(target)
except:
# perhaps some logging here
service_func = None
if not service_func:
# couldn't find a service matching `target`, crap out appropriately
raise e
self.addService(service_func, target)
return DjangoGateway.getServiceRequest(self, request, target)
self.router is a function that you supply to the constructor of the gateway. It takes the string target of the AMF remoting request and returns a matching function. If it returns None or raises an exception, an unknown service response will be returned to the requestor.
Hopefully this goes some way to laying the groundwork for what you require.
Solved last night!
from pyamf.remoting.gateway.google import WebAppGateway
import logging
class TottysGateway(WebAppGateway):
def __init__(self, services_available, root_path, not_found_service, logger, debug):
# override the contructor and then call the super
self.services_available = services_available
self.root_path = root_path
self.not_found_service = not_found_service
WebAppGateway.__init__(self, {}, logger=logging, debug=True)
def getServiceRequest(self, request, target):
# override the original getServiceRequest method
try:
# try looking for the service in the services list
return WebAppGateway.getServiceRequest(self, request, target)
except:
pass
try:
# don't know what it does but is an error for now
service_func = self.router(target)
except:
if(target in self.services_available):
# only if is an available service import it's module
# so it doesn't access services that should be hidden
try:
module_path = self.root_path + '.' + target
paths = target.rsplit('.')
func_name = paths[len(paths) - 1]
import_as = '_'.join(paths) + '_' + func_name
import_string = "from "+module_path+" import "+func_name+' as service_func'
exec import_string
except:
service_func = False
if(not service_func):
# if is not found load the default not found service
module_path = self.rootPath + '.' + self.not_found_service
import_string = "from "+module_path+" import "+func_name+' as service_func'
# add the service loaded above
assign_string = "self.addService(service_func, target)"
exec assign_string
return WebAppGateway.getServiceRequest(self, request, target)
How do I add a custom header to files pylons is serving from public?
a) Let your webserver serve files from /public instead of paster and configure it to pass some special headers.
b) Add a special route and serve the files yourself ala
class FilesController(BaseController):
def download(self, path)
fapp = FileApp( path, headers=self.get_headers(path) )
return fapp(request.environ, self.start_response)
c) maybe there is a way to overwrite headers and i just dont know how.
With a recent version of route, you can use the 'Magic path_info' feature, and follow the documentation from here to write your controller so it calls paster.DirectoryApp.
In my project, I wanted to serve any file in the public directory, including subdirs, and ended with this as controller, to be able to override content_type :
import logging
from paste.fileapp import FileApp
from paste.urlparser import StaticURLParser
from pylons import config
from os.path import basename
class ForceDownloadController(StaticURLParser):
def __init__(self, directory=None, root_directory=None, cache_max_age=None):
if not directory:
directory = config['pylons.paths']['static_files']
StaticURLParser.__init__(self, directory, root_directory, cache_max_age)
def make_app(self, filename):
headers = [('Content-Disposition', 'filename=%s' % (basename(filename)))]
return FileApp(filename, headers, content_type='application/octetstream')
In a standard Pylons setup, the public files are served from a StaticUrlParser. This is typically setup in your config/middleware.py:make_app() function
You need to subclass the StaticUrlParser like Antonin ENFRUN describes, though calling it a Controller is confusing because it's doing a different purpose. Add something like the following to the top of the config/middleware.py:
from paste.fileapp import FileApp
from paste.urlparser import StaticURLParser
class HeaderUrlParser(StaticURLParser):
def make_app(self, filename):
headers = # your headers here
return FileApp(filename, headers, content_type='application/octetstream')
then replace StaticUrlParser in config/middleware.py:make_app() with HeaderUrlParser
static_app = StaticURLParser(config['pylons.paths']['static_files'])
becomes
static_app = HeaderURLParser(config['pylons.paths']['static_files'])
A simpler way to use FileApp for streaming, based on the pylons book. The code below assumes your route provides some_file_identifier, but the other two variables are "magic" (see explanation after code).
class MyFileController(BaseController):
def serve(self, environ, start_response, some_file_identifier):
path = self._convert_id_to_path(some_file_identifier)
app = FileApp(path)
return app(environ, start_response)
Pylons automatically gives you the wsgi environ and start_response variables if you have variables of those names in your method signature. You should not need to set or munge headers otherwise, but if you do you can use the abilities built in to FileApp to achieve this.