We Keep Coding

High performance WSGI application

We would like to develop high performance WSGI application in python. This application should serve html or JSON both depening on requirement.There will not be any backend database.
As per my understanding Django not suits well to my requirements. It seems me using Django is overburden because we don't need many of its features. What we basically need is router+controller+view.
Also performance is our big concern. I am going though different banchmarkings and found that Djnago is really slow.
https://falconframework.org/
Keeping all this in view it seems me that we should write our own framework by using some fastest router libraries but this will still be reinventing the wheels.
Please note I have developed database centric web application in Django and it worth to use Django there. But when there is no database and performance really matters then based on different banchmark and own experience it not seems good choice.
Is there any other better minimal web framework that we can use?
Or if suggestion is to write new one then it will be great if you provide few examples.
Below are few links:
https://docs.pylonsproject.org/projects/webob/en/stable/do-it-yourself.html
http://funwithlinux.net/2015/11/how-to-write-a-python-web-framework-from-scratch/
Any expert guidance is really helpful!
Thanks in advance

Rails or Pylons -- Any good framework for Python?

I have no background in web applications, but have a fairly experience background in C++, and a quick learner.
I have spent some time learning Python and reading through SQLAlchemy. I kind of like the idea of coding in pure Python OO, and then use a nice SQLAlchemy mapper to persist everything. I like this decoupled approach (using pure Python classes along a mapper function to talk to DB) better than the ActiveRecord idea of Rails. I think eventually I would have more control over connecting the DB to the app. (I need to work with a DB that is updated by a background process. Something like a web crawler that fills the DB.)
At the same time, some stuff makes me think again about Rails. Like streamlined Email and Ajax handling in Rails.
Am I thinking the right way, that Rails is less flexible for Form Validation Manipulations, and working with DB? And is it harder in Pylons to handle Email (notifications), RSS, Ajax?
What would you suggest?
Thanks

Have a look at Django. It sounds like this exactely what you are looking for :-).

Have a look at these Python frameworks:
Django: Probably the single most popular Python framework, but for better (and worse) is very-much a full-stack solution.
Pylons: As reaction to the Django One Way of working, Pylons, for better (and worse), uses a much looser binding of the modules that make up your framework.
TurboGears: As an attempt at a happy medium between Django and Pylons, TurboGears is based on Pylons, but comes ready made with certain component choices, and the glue to hold it together.
Zope: Zope is more of an application server and framework than a "web framework". It just happens to be web based.
The first three are all inspired by Ruby on Rails, but each has it's own ideas for improvement. Zope predates Rails, and is it's own world.
I've used TurboGears to develop a few small apps. Kinda nice. At the time, their docs were kinda bad. I hope that's changed.
I've also directly used Python Paste a few times. Paste is the HTTP server base upon which Pylons, and in turn TurboGears, is based. Again very nice.
Also: When given the choice, I've always used SQLAlchemy as an ORM. It's truly an impressive piece of software that I've used for even non-web projects.
Hope this helps. Let us know what path you take. :-)

Rails is written in Ruby, not Python. If you have your heart set on Python, then go with Django. But please give Rails a fair shake; ActiveRecord is not the only ORM available either. I use DataMapper for some apps too. I may be biased, but I'm inclined to think the Rails ecosystem is bigger than that of Django too.

If you are looking for a project similar to rails, you should check out Masonite, A modern and developer-centric web framework in Python

You should also checkout web2py instead of Django. Just an alternative you might consider.
Here's two "A vs. B" articles and discussions regarding the two:
Django vs. web2py
Reddit.com community discussion

I wrote a few websites in Pylons over the years and I like it a lot. The great things about Pylons is that it consists mostly of third party libraries. That means that you're learning many useful libraries that can be used in you other projects, for example SQLAlchemy, WebOb, FormEncode, Beaker, Mako and so on ... Especially SQLAlchemy and Beaker are extremely useful in pretty much any context.

You could also have a look to Nagare, another full stack framework.
Some of Nagare based projects already in production can be found on the Nagare web site.

I used Web2Py for many small projects, including many goodies such as the "Workers" & Scheduler concepts, some event-driven updates in web page through the short tornado example in websocket_messaging.py. If you're looking for a small but powerful development framework that includes a small DB and display tables, it's just amazing. You even do not need to write a single HTML line. I do not see any competitor in this area. In my opinion it's far easier and faster than django, but django might provide much freedom in complex apps.

WSGI Authentication: Homegrown, Authkit, OpenID...?

I want basic authentication for a very minimal site, all I personally need is a single superuser. While hard-coding a password and username in one of my source files is awfully tempting, especially since I'm hosting the site on my own server, I feel I'm breaking the law of the internets and I should just use a database (I'm using sqlite for blog posts and such). Which would be the easiest to setup, in terms of time and effort, out of OpenID or AuthKit (repoze just scares me.. it feels like too much overhead for what I'm trying to achieve), or should I roll my own?
Why I brought up OpenID is, it might just solve my spam problem (I'm currently using Akismet), to just require all commentors to login with an OpenID. I have absolutely no idea how to go about integrating OpenID with my WSGI application though (it's probably dead simple, I've never actually looked into it yet).

also look at repose.who
http://static.repoze.org/whodocs/

AuthKit includes a built-in OpenID module, if that helps.
The AuthKit cookbook includes a simple example here... http://wiki.pylonshq.com/display/authkitcookbook/OpenID+Passurl
That said, if you only need a single login (so there's no complex user management going on), why not use Apache's built-in authentication features (AuthUserFile .htpasswd together with Require valid-user)?

Opid is a very small and simple to use WSGI OpenID app: python-opid

You can adapt this.
http://code.activestate.com/recipes/302378/
Or, better, adapt this.
http://devel.almad.net/trac/django-http-digest/
This is quite nice.

Django vs other Python web frameworks?

I've pretty much tried every Python web framework that exists, and it took me a long time to realize there wasn't a silver bullet framework, each had its own advantages and disadvantages. I started out with Snakelets and heartily enjoyed being able to control almost everything at a lower level without much fuss, but then I discovered TurboGears and I have been using it (1.x) ever since. Tools like Catwalk and the web console are invaluable to me.
But with TurboGears 2 coming out which brings WSGI support, and after reading up on the religious debates between the Django and WSGI camps, I'm really torn between "doing it the right way", e.g., learning WSGI, spending valuable time writing functionality that already exists in Django and other full-stack frameworks, as opposed to using Django or some high-level framework that does everything for me. The downsides with the latter that I can see are pretty obvious:
I'm not learning anything in the process
If I ever need to do anything lower level it's going to be a pain
The overhead required for just a basic site which uses authentication is insane. (IMO)
So, I guess my question is, which is the better choice, or is it just a matter of opinion, and should I suck it up and use Django if it achieves what I want with minimal fuss (I want authentication and a CRUD interface to my database)? I tried Werkzeug, Glashammer, and friends, but AuthKit and Repoze scared me off, as well as the number of steps involved to just setup basic authentication. I looked at Pylons, but the documentation seems lacking, and when referencing simple features like authentication or a CRUD interface, various wiki pages and documentation seemed to contradict each other, with different hacks for versions and such.
Thanks to S. Lott for pointing out that I wasn't clear enough. My question is: which of the following is worthwhile in the long run, but not painful in the short (e.g., some sort of middle ground, anyone?) - Learn WSGI, or stick with a "batteries-included" framework? If the latter, I would appreciate a suggestion as to whether I should give Django another try, stick with TurboGears 1.x, or venture into some other framework.
Also, I have tried CherryPy, but couldn't seem to find a good enough CRUD application that I could plop in and use right away.

the religious debates between the Django and WSGI camps
It would seem as though you're a tad bit confused about what WSGI is and what Django is. Saying that Django and WSGI are competing is a bit like saying that C and SQL are competing: you're comparing apples and oranges.
Django is a framework, WSGI is a protocol (which is supported by Django) for how the server interacts with the framework. Most importantly, learning to use WSGI directly is a bit like learning assembly. It's a great learning experience, but it's not really something you should do for production code (nor was it intended to be).
At any rate, my advice is to figure it out for yourself. Most frameworks have a "make a wiki/blog/poll in an hour" type exercise. Spend a little time with each one and figure out which one you like best. After all, how can you decide between different frameworks if you're not willing to try them out?

I'd say you're being a bit too pessimistic about "not learning anything" using Django or a similar full-stack framework, and underestimating the value of documentation and a large community. Even with Django there's still a considerable learning curve; and if it doesn't do everything you want, it's not like the framework code is impenetrable.
Some personal experience: I spent years, on and off, messing around with Twisted/Nevow, TurboGears and a few other Python web frameworks. I never finished anything because the framework code was perpetually unfinished and being rewritten underneath me, the documentation was often nonexistent or wrong and the only viable support was via IRC (where I often got great advice, but felt like I was imposing if I asked too many questions).
By comparison, in the past couple of years I've knocked off a few sites with Django. Unlike my previous experience, they're actually deployed and running. The Django development process may be slow and careful, but it results in much less bitrot and deprecation, and documentation that is actually helpful.
HTTP authentication support for Django finally went in a few weeks ago, if that's what you're referring to in #3.

I suggest taking another look at TG2. I think people have failed to notice some of the strides that have been made since the last version. Aside from the growing WSGI stack of utilities available there are quite a few TG2-specific items to consider. Here are a couple of highlights:
TurboGears Administration System - This CRUD interface to your database is fully customizable using a declarative config class. It is also integrated with Dojo to give you infinitely scrollable tables. Server side validation is also automated. The admin interface uses RESTful urls and HTTP verbs which means it would be easy to connect to programatically using industry standards.
CrudRestController/RestController - TurboGears provides a structured way to handle services in your controller. Providing you the ability to use standardized HTTP verbs simply by extending our RestController. Combine Sprox with CrudRestController, and you can put crud anywhere in your application with fully-customizable autogenerated forms.
TurboGears now supports mime-types as file extensions in the url, so you can have your controller render .json and .xml with the same interface it uses to render html (returning a dictionary from a controller)
If you click the links you will see that we have a new set of documentation built with sphinx which is more extensive than the docs of the past.
With the best web server, ORM, and template system(s) (pick your own) under the hood, it's easy to see why TG makes sense for people who want to get going quickly, and still have scalability as their site grows.
TurboGears is often seen as trying to hit a moving target, but we are consistent about releases, which means you won't have to worry about working out of the trunk to get the latest features you need. Coming to the future: more TurboGears extensions that will allow your application to grow functionality with the ease of paster commands.

Your question seems to be "is it worth learning WSGI and doing everything yourself," or using a "full stack framework that does everything for you."
I'd say that's a false dichotomy and there's an obvious third way. TurboGears 2 tries to provide a smooth path from a "do everything for you" style framework up to an understanding of WSGI middleware, and an ability to customize almost every aspect of the framework to suit your application's needs.
We may not be successful in every place at every level, but particularly if you've already got some TurboGears 1 experience I think the TG2 learning curve will be very, very easy at first and you'll have the ability to go deeper exactly when you need it.
To address your particular issues:
We provide an authorization system out of the box that matches the one you're used to from TG1.
We provide an out of the box "django admin" like interface called the tgext.admin, which works great with dojo to make a fancy spreadsheet like interface the default.
I'd also like to address a couple of the other options that are out there and talk a little bit about the benifits.
CherryPy. I think CherryPy is a great webserver and a nice minimalistic web-framework. It's not based on WSGI internally but has good WSGI support although it will not provide you with the "full stack" experience. But for custom setups that need to be both fast and aren't particularly suited to the defaults provided by Django or TurboGears, it's a great solution.
Django. I think Django is a very nice, tigtly integrated system for developing websites. If your application and style of working fits well within it's standard setup it can be fantastic. If however you need to tune your DB usage, replace the template language, use a different user authorization model or otherwise do things differently you may very likely find yourself fighting the framework.
Pylons Pylons like CherryPy is a great minimalistic web-framework. Unlike CherryPy it's WSGI enabled through the whole system and provides some sane defaults like SQLAlchemy and Mako that can help you scale well. The new official docs are of much better quality than the old wiki docs which are what you seem to have looked at.

Have you taken a look at CherryPy. It is minimalistic, yet efficient and simple. It is low level enough for not it to get in they way, but high enough to hide complexity. If I remember well, TurboGears was built on it.
With CherryPy, you have the choice of much everything. (Template framework, ORM if wanted, back-end, etc.)

Learn WSGI
WSGI is absurdly simple.. It's basically a function that looks like..
def application(environ, start_response) pass
The function is called when an HTTP request is received. environ contains various data (like the request URI etc etc), start_response is a callable function, used to set headers.
The returned value is the body of the website.
def application(environ, start_response):
start_response("200 OK", [])
return "..."
That's all there is to it, really.. It's not a framework, but more a protocol for web-frameworks to use..
For creating sites, using WSGI is not the "right way" - using existing frameworks is.. but, if you are writing a Python web-framework then using WSGI is absolutely the right way..
Which framework you use (CherryPy, Django, TurboGears etc) is basically personal preference.. Play around in each, see which you like the most, then use it.. There is a StackOverflow question (with a great answer) about this, "Recommendation for straight-forward python frameworks"

Have you checked out web2py? After recently evaluating many Python web frameworks recently I've decided to adopt this one. Also check out Google App Engine if you haven't already.

I'd say the correct answer depends on what you actually want and need, as what will be worthwhile in the long run depends on what you'll need in the long run. If your goal is to get applications deployed ASAP then the 'simpler' route, ie. Django, is surely the way to go. The value of a well-tested and well-documented system that exactly what you want can't be underestimated.
On the other hand if you have time to learn a variety of new things which may apply in other domains and want to have the widest scope for customisation then something like Turbogears is superior. Turbogears gives you maximum flexibility but you will have to spend a lot of time reading external docs for things like Repoze, SQLAlchemy, and Genshi to get anything useful done with it. The TG2 docs are deliberately less detailed than the TG1 docs in some cases because it's considered that the external docs are better than they used to be. Whether this sort of thing is an obstacle or an investment depends on your own requirements.

Django is definitely worth learning, and sounds like it will fit your purposes. The admin interface it comes with is easy to get up and running, and it does use authentication.
As for "anything lower level", if you mean sql, it is entirely possible to shove sql into you queries with the extra keyword. Stylistically, you always try to avoid that as much as possible.
As for "not learning anything"...the real question is whether your preference is to be primarily learning something lower-level or higher-level, which is hardly a question anyone here can answer for you.

Pylons seems a great tool for me:
a real web framework (CherryPy is just a web server),
small code base - reuse of other projects,
written entirely with WSGI in mind, based on Paste,
allows you to code the app right away and touch the low level bits if it's necessary,
I've used CherryPy and TurboGears and look at many other frameworks but none of them were so light and productive as Pylons is. Check the presentation at Google.

I'm a TurboGears fan, and this is exactly the reason why: a very nice trade-off between control and doing things right vs. easy.
You'll have to make up your own mind of course. Maybe you'd prefer to learn less, maybe more. Maybe the areas that I like knowledge/control (database for example), you couldn't care less about. And don't misunderstand. I'm not characterizing any frameworks as necessarily hard or wrong. It's just my subjective judgment.
Also I would recommend TurboGears 2 if at all possible. When it comes out, I think it will be much better than 1.0 in terms of what it has selected for defaults (genshi, pylons, SqlAlchemy)

I would suggest for TurboGears 2. They have done a fantastic job of integrating best of Python world.
WSGI: Assuming you are developing moderately complex projects/ business solutions in TG2 or some other framework say Grok. Even though these frameworks supports WSGI does that mean one who is using these frameworks have to learn WSGI? In most cases answer is No. I mean it's good have this knowledge no doubt.
WSGI knowledge is probably is more useful in cases like
you want to use some middleware or some other component which is not provided as part of the standard stack for eg. Authkit with TG or Grok without ZODB.
you are doing some integration.
CherryPy is good but think of handling your database commits/rollbacks at the end of transactions, exposing json, validations in such cases TG, Django like frameworks do it all for you.

Web2py is the secret sauce here. Don't miss checking it out.

How do I use python for web development without relying on a framework?

I know the various frameworks have their benefits, but I personally want my web development in python to be as straight-forward as possible: less writing to the framework, more writing python.
The only thing I have found so far that lets me do this in the most obvious way possible is web.py but I have slight concerns on its performance.
For those of you using nginx(or another flavour)+mod_wsgi+web.py... how's performance? Can it be improved further?
For those of you who have used web.py, liked the idea and went on to write something better or found something better... care to point me to the source?
I'd like to hear about all the conspicuous, minimal yet powerful approaches.

The way to go is wsgi.
WSGI is the Web Server Gateway Interface. It is a specification for web servers and application servers to communicate with web applications (though it can also be used for more than that). It is a Python standard, described in detail in PEP 333.
All current frameworks support wsgi. A lot of webservers support it also (apache included, through mod_wsgi). It is the way to go if you want to write your own framework.
Here is hello world, written to wsgi directly:
def application(environ, start_response):
status = '200 OK'
response_headers = [('Content-type','text/plain')]
start_response(status, response_headers)
return ['Hello world!\n']
Put this in a file.py, point your mod_wsgi apache configuration to it, and it will run. Pure python. No imports. Just a python function.
If you are really writing your own framework, you could check werkzeug. It is not a framework, but a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility modules. It includes a powerful debugger, full featured request and response objects, HTTP utilities to handle entity tags, cache control headers, HTTP dates, cookie handling, file uploads, a powerful URL routing system and a bunch of community contributed addon modules. Takes the boring part out of your hands.

It's hilarious how, even prompted with a question asking how to write without a framework, everyone still piles in to promote their favourite framework. The OP whinges about not wanting a “heavyweight framework”, and the replies mention Twisted, of all things?! Come now, really.
Yes, it is perfectly possible to write straight WSGI apps, and grab bits of wanted functionality from standalone modules, instead of fitting your code into one particular framework's view of the world.
To take this route you'll generally want to be familiar with the basics of HTTP and CGI (since WSGI inherits an awful lot from that earlier specification). It's not necessarily an approach to recommend to beginners, but it's quite doable.
I'd like to hear about all the conspicuous, minimal yet powerful approaches
You won't hear about them, because no-one has a tribal interest in promoting “do it yourself” as a methodology. Me, I use a particular standalone templating package, a particular standalone form-reading package, a particular data access layer, and a few home-brew utility modules. I'm not writing to one particular philosophy I can proselytise about, they're all just boring tools that could be swapped out and replaced with something else just as good.

You could also check cherrypy. The focus of cherrypy is in being a framework that lets you write python. Cherrypy has its own pretty good webserver, but it is wsgi-compatible so you can run cherrypy applications in apache via mod_wsgi. Here is hello world in cherrypy:
import cherrypy
class HelloWorld(object):
def index(self):
return "Hello World!"
index.exposed = True
cherrypy.quickstart(HelloWorld())

+1 to all the answers with WSGI.
Eric Florenzo wrote a great blog post lately you should read: Writing Blazing Fast, Infinitely Scalable, Pure-WSGI Utilities. This will give you a better idea of pure WSGI beyond Hello World. Also pay attention to the comments, especially the first comment by Kevin Dangoor where he recommends at least adding WebOb to your toolset.

For what it's worth, I wrote my website in mod_python without any intervening framework like Django. I didn't really have any reason to complain. (Well maybe a little, mod_python is kind of quirky in a few ways but not in the common use cases) One thing's for sure, it will definitely let you write Python ;-)

Why do you have concerns about web.py's performance? As I mentioned here, we use CherryPy (the web server "built into" web.py) behind nginx to serve most of the HTML at Oyster.com -- nginx splits the traffic across 2 or 3 web servers each running 4 Python processes, and we can easily handle 100s of requests per second.
Oyster.com is a high-volume website averaging 200,000 dynamically-generated pageviews/day, and peaking to much higher numbers than that. However, we do use a content delivery network (CDN) for our static resources like images and CSS.
We definitely care about performance (most of our pages render in less than 25ms), but web.py isn't the bottleneck. Our bottlenecks are template rendering (we use Cheetah, which is fast enough but not other-worldly fast) and database queries (we cache heavily and keep the number of database queries per page to 0 or 1) and accessing our 3rd-party hotel pricing providers (these are accessed when you do a search with dates we don't already have cached).
Remember, premature optimization is the root of all evil -- unless you're serving google.com, web.py will probably work for you.

I've written a few small web applications using mod-python and PSP -- mod-python's equivalent to php.
In one case, I wrote a web page that generates release notes by inspecting our source code repository. I rewrote it into PHP, and was surprised to discover that the PSP version was about 20% faster, as well as being about half as many lines of code.
So, for small problems at least, psp has worked well for me.

I think it depends on the definition of what a framework is and what it should do for you.
As pointed out, a very minimal "framework" would be WSGI as it only defines one small interface for interfacing with a web server. But it's a powerful approach because of the middleware you can put between your app and the server.
If you want more slightly more, like some URL to function mapping, then you have some choices, some of which have been already mentioned.
If you go further you might come to Pylons or Turbogears or Django, after that maybe Zope but it grows bigger and maybe the pain as well as you always buy into the opinions of that framework.
What I recently use more and more (coming from Zope/Plone) is repoze.bfg. It's very small, does not come with a ORM bundled (so you can use SQLAlchemy, Storm or simply go to an object database like the ZODB). What it does is basically handling how you come from a URL to a view which is a function. It supports both URL Mapping (a la Routes) or object traversal, which IMHO is very powerful in some circumstances esp. if you have a not so strict mapping. The good thing is that it directly comes with an ACL based security framework which can use if you want to which IMHO is very practical to have. That way you don't need decorators which seem to be used mostly for such things.
And of course it's based on WSGI. Also look for the repoze subversion repository for quite a lot of middleware and the Paste stuff is also very useful for WSGI related tasks.

What's wrong with Django? It doesn't force you to use it's ORM and controllers are just plain Python functions instead of Rails-like class methods. Also, url routing is done with regular expressions instead of another framework-invented syntax. If django seems like too much for you anyway, i recommend taking a look at Werkzeug

I'm pretty fond of Google AppEngine. I use the ORM and templating system, but otherwise follow a REST-patterned design and just implement Python methods for the corresponding HTTP ones. It makes the raw HTTP interaction central, and optionally gives you other things to use. Plus no more configuring and managing your deployment environment!