Here the simple webserver means a server that deal with simple HTTP request, just like the following one:
import BaseHTTPServer
class WebRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_GET(self):
if self.path == ‘/foo’:
self.send_response(200)
self.do_something()
else:
self.send_error(404)
def do_something(self):
print ‘hello world’
server = BaseHTTPServer.HTTPServer((’127.0.0.1′,8080), WebRequestHandler)
server.serve_forever()
Despite of dealing with request of POST,PUT,DELETE methods, what is the difference between this simple server with Apache Web Server?
Or in other words, if i want to use python to implement a server which can be put into use of business, what also should i do?
It'd be greatly appreciated if the big picture of Apache Server is shown.
Or in other words, if i want to use python to implement a server which can be put into use of business, what also should i do?
There are already python-based web servers, such as CherryPy (which I think is intended to be a web server solution on the same stack level as Apache; it is more python-based though, and Apache has been around a lot longer).
If you wish to write a lightweight extremely simple webserver from scratch, there is probably nothing wrong with using BaseHTTPServer, other than perhaps a few outstanding design issues (I hear race conditions might permanently clog a socket until a thread dies).
Though I would not recommend it (alone) for business, some of the big boys use BaseHTTPServer with a bit of extra machinery:
http://www.cherrypy.org/browser/trunk/cherrypy/_cphttpserver.py?rev=583
To elaborate, Apache is the industry standard. It has a plethora of configuration options, a security team, vulnerability mailing lists I think, etc. It supports modules (e.g. mod_python). Python-based web servers also support python-based modules (maybe they might allow you access to non-python things) via something called a WSGI stack; a WSGI application can run on any python-based web server (and Apache too, which also has a modwsgi); I think they are narrower in scope than Apache modules.
Apache module examples: http://httpd.apache.org/docs/2.0/mod/
WSGI examples (not a valid comparison): http://wsgi.org/wsgi/Middleware_and_Utilities
I might code my own webserver if I'm doing something extremely lightweight, or if I needed massive control over webserver internals that the module interfaces could not provide, or if I was doing a personal project. I would not code my own server for a business unless I had significant experience with how real-world web servers worked. This is especially important from a security vulnerability point-of-view.
For example, I once wrote a web-based music player. I used a BaseHTTPServer to serve music out of a sandbox I had written to ensure that people couldn't access arbitrary files. Threading was a nightmare. (I recall a bug where you needed to pass special arguments to Popen since threading caused an implicit fork which would cause hangs on dangling file descriptors.) There were other various issues. The code needed to be refactored a lot. It can be very worthwhile for a personal project, but is a significant undertaking and not worth it for a business that just needs a website.
I know two startups who have been content using Pylons (using Paste) or Turbogears (using CherryPy) in the past, if you're looking for a lightweight python web server stack. Their default template systems are lacking though. The choice between Apache and a leaner more python-based web server may also depend on the skillset of your co-developers.
Apache is written in C and designed to be scalable while BaseHTTPServer is meant for local/testing/debugging environments.
So you shouldn't use BaseHTTPServer for any production sites.
Apache web server knows and supports the entire HTTP protocol, so it can deal with all the complications having to do with headers, keeping connections open, caching content, all the different HTTP response codes and their proper treatment, etc.
You'd have to understand the entire HTTP protocol and express it in code to go beyond your simple HTTP server.
Related
I am new at Server side,
but I have gotten a chance to design and implement a server that will cover around 2000~3000 client.
And I am thinking that I will use Python and Websocket, though I don't know this choice is appropriate.
In this point, I am curious on how to design the server.
I think there must be some architecture normally in use depending on capacity that server handles.
Otherwise, Could I use a Websocket server offered by some python package like Tornado or Django?
I hope that I can get any information on this.
Any advice?
I've had good experiences using haproxy in front of sockjs-tornado. Depending on how complex your server-side logic, routing, and persistence requirements are, you could write all your server endpoints using tornado and use SQLAlchemy to handle writes to a relational database or use a non SQL data store like Redis.
If your main requirement is real-time interactivity it might be worth investigating meteor as well.
One of solutions could be Pyramid, sockjs, gunicorn, and gevent. Nginx probably better suits to be a frontend than Apache, but of course if you do not have any lengthy processing on the backend, any decent asynchronous Python server with websocket and sockjs support (not sure about socket.io as an alternative) will work for you out of the box.
Lenghty processing should be offloaded to some queue workers anyway, so asynchronous server will fit the bill.
Just check whether all used datastore/database adapters are compatible with your server solution be it asynchronous or multi-threading.
I'm writing a service (daemon) which provides web-unrelated features to my users.
I'd like to implement a minimal web server in that service such that a user can connect on say http://localhost:5000 and get an overview of the service current status. I've read a lot on embedding/extending Python and on how the latter seems to be recommended. However I can't decide how to design this: my service entry point has to be on the C++ side (it's a system daemon and one might not want to compile it with the web-server feature).
I'd like to use something like Django to be able to handle the requests (routing, security, whatever) on the Python side where many, many things already exist. I already have a minimal HTTP server in C++ (mongoose) that can process simple requests but I'd really like to delegate the actual processing to Python, pretty much the way WSGI does.
What is a good approach here ?
I'm struggling with some architectural choices for a scalable internet-of-things application.
I've chosen to base my project on Twisted augmented with the Cyclone framework to provide many Tornado convenances (websockets, auth-decorators, secure-cookies, etc)
Using a Twisted core has worked beautifully for me. I have numerous IP protocol and hardware interfaces all of which turned out to have great library support inside of twisted (and adding new protocols and interfaces to my application are the most-likely angles I'll have project scope creep), all with Twisted needing very low CPU and providing for very high connection counts.
My problems are with second-order webapp functionality.
I pulled in Cyclone thinking that with it's auth goodies (OpenID, oauth, user-auth decorators and secure-cookies) it wouldn't take much to implement user/session/admin functionality in my webapp. After the 500+ lines of abstracting my database (via txmongo) and just building user logins it became clear I both:
Didn't understand how little Cyclone/Tornado bring in the user/session/admin space, and
Didn't understand the amount of code it takes to fill in the gaps if your trying to build a multi-user auth webapp
A friend pointed me at Flask, which initially I thought was completely redundant, until I found flask plugins. The combination of Flask-Login and Flask-Admin would completely cover my user, session and user-admin needs, negating me writing what I would guess to be about 2k lines of code. Unfortunately, the flask plugins are all rife with blocking code and calls to blocking libraries. I don't see them as compatible with my project even if WSGI containers are used given that the user/session functionality happens with every page load (additionally I don't see any short cuts that would allow me to port them to async world without work roughly equal to that of rewriting them)
My question is:
In the python async space (... hopefully in the Twisted space, given my protocol needs), are there any plugins or alternate frameworks that provide ready-to-go user/login/admin functionality similar to what is in Flask-Login and Flask-Admin?
P.S. I've looked at Klein as the obvious Twisted version of Flask, but it doesn't seem to have a plugin ecosystem, and I'm not finding any strong user/session/admin there.
P.P.S. By the time I wrote this question I had already written my own (crappy) user-login-session system. So what I'm really after is the "Admin" capability (automated CRUD functions on user-style records, including web UI rendering, all designed in a Twisted/async way). I asked about user/login in the question in case it turn out there is an already-integraded solution (such as flask-login and flask-admin) in which case I would happily drop my code and switch to that.
Do you really need everything async? Consider async WebSockets but sync page renders. If you must, add an async downstream proxy or load balancer which will virtually eliminate app server's IO overhead.
I want to develop an extremely lightweight web service with a RESTful JSON API. I will do all the session management on the server side. The solution will receive several 100k (or more) API calls an hour and return (compressed) JSON as response, it should be able to scale effortlessly.
Security is naturally important, but I want to avoid heavy weight frameworks like Django etc, and preferably will use a webserver like nginx or lighttpd instead of Apache.
At the server end, this is all I need:
user session management
security (protection against atleast the more common attacks such as cross site request forgery etc)
url routing
http utilities (e.g. compression)
I am aware of web2py, but its deployment options seems 'not well though out' at best - so far, I have been unable to get it to work with Apache, despite following the user manuals.
Can anyone suggest a python framework (and web server) best suited for this task?
if you want to go really lightweight, you might try wsgi itself without a framework, or maybe Flask. I understand wsgi runs on lighttpd, you'll get some hits on a Google search.
Try Pyramid. It's fast, lightweight and with a lot of options to configure your enviroment as you like...
Every time I read either WSGI or CGI I cringe. I've tried reading on it before but nothing really has stuck.
What is it really in plain English?
Does it just pipe requests to a terminal and redirect the output?
From a totally step-back point of view, Blankman, here is my "Intro Page" for Web Server Gateway Interface:
PART ONE: WEB SERVERS
Web servers serve up responses. They sit around, waiting patiently, and then with no warning at all, suddenly:
a client process sends a request. The client process could be a web server, a bot, a mobile app, whatever. It is simply "the client"
the web server receives this request
deliberate mumble various things happen (see below)
The web server sends back something to the client
web server sits around again
Web servers (at least, the better ones) are VERY good at this. They scale up and down processing depending on demand, they reliably hold conversations with the flakiest of clients over really cruddy networks, and we never really have to worry about it. They just keep on serving.
This is my point: web servers are just that: servers. They know nothing about content, nothing about users, nothing in fact other than how to wait a lot and reply reliably.
Your choice of web server should reflect your delivery preference, not your software. Your web server should be in charge of serving, not processing or logical stuff.
PART TWO: (PYTHON) SOFTWARE
Software does not sit around. Software only exists at execution time. Software is not terribly accommodating when it comes to unexpected changes in its environment (files not being where it expects, parameters being renamed etc). Although optimisation should be a central tenet of your design (of course), software itself does not optimise. Developers optimise. Software executes. Software does all the stuff in the 'deliberate mumble' section above. Could be anything.
Your choice or design of software should reflect your application, your choice of functionality, and not your choice of web server.
This is where the traditional method of "compiling in" languages to web servers becomes painful. You end up putting code in your application to cope with the physical server environment or, at least, being forced to choose an appropriate 'wrapper' library to include at runtime, to give the illusion of uniformity across web servers.
SO WHAT IS WSGI?
So, at last, what is WSGI? WSGI is a set of rules, written in two halves. They are written in such a way that they can be integrated into any environment that welcomes integration.
The first part, written for the web server side, says "OK, if you want to deal with a WSGI application, here's how the software will be thinking when it loads. Here are the things you must make available to the application, and here is the interface (layout) that you can expect every application to have. Moreover, if anything goes wrong, here's how the app will be thinking and how you can expect it to behave."
The second part, written for the Python application software, says "OK, if you want to deal with a WSGI server, here's how the server will be thinking when it contacts you. Here are the things you must make available to the server, and here is the interface (layout) that you can expect every server to have. Moreover, if anything goes wrong, here's how you should behave and here's what you should tell the server."
So there you have it - servers will be servers and software will be software, and here's a way they can get along just great without one having to make any allowances for the specifics of the other. This is WSGI.
mod_wsgi, on the other hand, is a plugin for Apache that lets it talk to WSGI-compliant software, in other words, mod_wsgi is an implementation - in Apache - of the rules of part one of the rule book above.
As for CGI.... ask someone else :-)
WSGI runs the Python interpreter on web server start, either as part of the web server process (embedded mode) or as a separate process (daemon mode), and loads the script into it. Each request results in a specific function in the script being called, with the request environment passed as arguments to the function.
CGI runs the script as a separate process each request and uses environment variables, stdin, and stdout to "communicate" with it.
Both CGI and WSGI define standard interfaces that programs can use to handle web requests. The CGI interface is at a lower level than WSGI, and involves the server setting up environment variables containing the data from the HTTP request, with the program returning something formatted pretty much like a bare HTTP server response.
WSGI, on the other hand, is a Python-specific, slightly higher-level interface that allows programmers to write applications that are server-agnostic and which can be wrapped in other WSGI applications (middleware).
If you are unclear on all the terms in this space, and let's face it, it's a confusing acronym-laden one, there's also a good background reader in the form of an official python HOWTO which discusses CGI vs. FastCGI vs. WSGI and so on. I wish I'd read it first.