I have written a web API in BaseHTTPServer. It is meant to be used only on localhost. It returns JSON objects on GET/POST operations.
http://localhost:8888/operation?param
and code is like
def do_GET(self):
if self.path=="operation":
self.wfile.write("output")
But I am worried about keep-alive mechanisms (read: a webserver that can respawn workers), lack of multi-threading, and PITA-ful maintenance.
And like I said, I am looking at the development and deployment issues for choosing this web framework.
Development
The web interface is currently 250 lines and has very simple features. I am looking for something that lends itself well to clean maintenance and deployment. I dont want the framework's MVC, ORM, templating and other features messing my learning curve. UrL patterns that redirect to appropriate module is nice.
Deployment
It should deploy on a mature server with a WSGI module with minimum fuss. And such a setup has hot-deploy (for want of a better word), installing a new application or updating the code means copying the files to the www-root in the filesystem.
CherryPy and Flask seem interesting. Django and Web2Py seem too comprehensive.
The recommended way of deploying wsgi is as a long-running-process, either embedded or daeomonized, and not as a cgi script. Either way, its going to be a little different than just uploading files like in php, restarting the server/process by touching the config file is normally the closest you get to "hot-deployment" using wsgi.
Needless to say, the framework itself does not impose any kind of deployment restraints if it is wsgi compliant. Take your pick depending on your needs: apache+modwsgi, gunicorn, cherry.py, paste. None of them offer "hot-deployment" (afaik), you will still need to create a wsgi script and reload the processes. The filesystem layout is normally of no concern and that's good. You don't usually get autoreload either. I know werkzeug and cherry.py do, and werkzeug offers some really cool debugging tools too. Please note that tornado/werkzeug* itself offers an autoreload option, but is actually considered for development and not deployment, and not compatible with the wsgi module.
But no matter how painful or painless the deployment is, it is recommended to use something like fabric to automate your deployments, and setting up a wsgi web server isnt that that hard.
Choice of the framework itself is kind of tricky, and depends on what level you want to work in. Tornado, werkzeug are popular low level frameworks, (but also include higher level tools, and many are frameworks+webserver), but you could also work with webob directly and just plugin whatever else you need.
You have the microframeworks like flask or bottle, then the lightweight frameworks, like web2.py, or maybe pyramid (the lines on how heavy a framework are kind of blurry).
Then you have the "full-stack" django, grok, turbogears, etc...
And then you have zope, which has been on a diet but still very heavy.
Note that you can pretty much do anything with all of them (just depends how much you want to bend them), and in many cases you can swap components rather easily. I'd start try out a microframework like bottle or maybe flask (you don't have to use ORM's or templating, but are easily available once you do), but also take a look at webob.
*comment: added werkzeug to the not really autoreload camp.
For what you describe, id go with: Tornado Web Server
This is the hello world:
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
application = tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
It's highly scalable, and I think it may take you 10 minutes to set it up with your code.
Related
I am developing a Python based application (HTTP -- REST or jsonrpc interface) that will be used in a production automated testing environment. This will connect to a Java client that runs all the test scripts. I.e., no need for human access (except for testing the app itself).
We hope to deploy this on Raspberry Pi's, so I want it to be relatively fast and have a small footprint. It probably won't get an enormous number of requests (at max load, maybe a few per second), but it should be able to run and remain stable over a long time period.
I've settled on Bottle as a framework due to its simplicity (one file). This was a tossup vs Flask. Anybody who thinks Flask might be better, let me know why.
I have been a bit unsure about the stability of Bottle's built-in HTTP server, so I'm evaluating these three options:
Use Bottle only -- As http server + App
Use Bottle on top of uwsgi -- Use uwsgi as the HTTP server
Use Bottle with nginx/uwsgi
Questions:
If I am not doing anything but Python/uwsgi, is there any reason to add nginx to the mix?
Would the uwsgi/bottle (or Flask) combination be considered production-ready?
Is it likely that I will gain anything by using a separate HTTP server from Bottle's built-in one?
Flask vs Bottle comes down to a couple of things for me.
How simple is the app. If it is very simple, then bottle is my choice. If not, then I got with Flask. The fact that bottle is a single file makes it incredibly simple to deploy with by just including the file in our source. But the fact that bottle is a single file should be a pretty good indication that it does not implement the full wsgi spec and all of its edge cases.
What does the app do. If it is going to have to render anything other than Python->JSON then I go with Flask for its built in support of Jinja2. If I need to do authentication and/or authorization then Flask has some pretty good extensions already for handling those requirements. If I need to do caching, again, Flask-Cache exists and does a pretty good job with minimal setup. I am not entirely sure what is available for bottle extension-wise, so that may still be worth a look.
The problem with using bottle's built in server is that it will be single process / single thread which means you can only handle processing one request at a time.
To deal with that limitation you can do any of the following in no particular order.
Eventlet's wsgi wrapping the bottle.app (single threaded, non-blocking I/O, single process)
uwsgi or gunicorn (the latter being simpler) which is most ofter set up as single threaded, multi-process (workers)
nginx in front of uwsgi.
3 is most important if you have static assets you want to serve up as you can serve those with nginx directly.
2 is really easy to get going (esp. gunicorn) - though I use uwsgi most of the time because it has more configurability to handle some things that I want.
1 is really simple and performs well... plus there is no external configuration or command line flags to remember.
2017 UPDATE - We now use Falcon instead of Bottle
I still love Bottle, but we reached a point last year where it couldn't scale to meet our performance requirements (100k requests/sec at <100ms). In particular, we hit a performance bottleneck with Bottle's use of thread-local storage. This forced us to switch to Falcon, and we haven't looked back since. Better performance and a nicely designed API.
I like Bottle but I also highly recommend Falcon, especially where performance matters.
I faced a similar choice about a year ago--needed a web microframework for a server tier I was building out. Found these slides (and the accompanying lecture) to be very helpful in sifting through the field of choices: Web micro-framework BATTLE!
I chose Bottle and have been very happy with it. It's simple, lightweight (a plus if you're deploying on Raspberry Pis), easy to use, intuitive, has the features I need, and has been supremely extensible whenever I've needed to add features of my own. Many plugins are available.
Don't use Bottle's built-in HTTP server for anything but dev.
I've run Bottle in production with a lot of success; it's been very stable on Apache/mod_wsgi. nginx/uwsgi "should" work similarly but I don't have experience with it.
I also suggest you look at running bottle via gevent.pywsgi server. It's awesome, super simple to setup, asynchronous, and very fast.
Plus bottle has an adapter built for it already, so even easier.
I love bottle, and this concept that it is not meant for large projects is ridiculous. It's one of the most efficient and well written frameworks, and can be easily molded without a lot of hand wringing.
why? because I have a django project that capture data from user and consume many webservices displaying the results to the user in order to compare information, something like aggregators websites who search flights tickets via airlines webservices and show the result in real time in order to compare tickets.
nowaday im doing this in a "waiting page", where celery hits webservices while jquery is asking every 5 seconds if all results are ready, so when ready redirect to the results page.
what I want to do is not to use this "waiting page", I want to feed the results page in real time as the results are comming, and I want to make it following the best practices, I mean I dont want to jquery get the results each X seconds to feed the table.
I think some coroutine-based python library can help me with this, but I want to learn more about your experience first and see some examples, I am confused because this part of the project was designed to run asynchronously, I mean, consuming webservices with celery-chords, but not designed for dispatching the information in real time through the app server.
actual architecture:
python 2.7, django 1.3, postgresql 9, celery 3 + redis, uwsgi, nginx, all hosted on aws.
thank you in advance.
uWSGI+gevent is a solid combo, while there is currently no-way to run uWSGI with the tornado api (and as uWSGI dropped support in 1.9 for callback based approach, i think that we will never see that combo working).
The problem you need to solve before starting working with gevent, is ensuring that all of your pieces are gevent friendly (redis and celery are ok, you need to check your database adapter). After that simply add --gevent to your uWSGI instance, where is the maximum number of concurrent requests per worker.
I don't know about uWSGI+gevent, but you can use tornado with uWSGI. Tornado basically gives you an inbuilt WSGI support in tornado.wsgi.WSGIContainer module to make it compactable with other WSGI servers like uWSGI and gunicorn.But it depends on your use and I think it's not a good idea to use an Asynchronous framework with a synchronous server(like uWSGI). Tornado has this warning for this.
WSGI is a synchronous interface, while Tornado’s concurrency model is based on single-threaded asynchronous execution. This means that running a WSGI app with Tornado’s WSGIContainer is less scalable than running the same app in a multi-threaded WSGI server like gunicorn or uwsgi. Use WSGIContainer only when there are benefits to combining Tornado and WSGI in the same process that outweigh the reduced scalability.
I've created a simple gstreamer-based python audio application with a GTK+ GUI for picking and playing a webstream from a XML list. Then I connected my PC speakers output to the input of an old stereo receiver with large loudspeakers and presto, I have a pretty good sound system that is heard over most of my home.
Now I'd like to add a web user-interface to remote control the application from a room other than the one with the computer but so far all my attempts have been fruitless.
In particular I wonder if it is possible to create a sort of socket with signals like those of GTK GUIs to run methods that change gstreamer parameters.
Or is there a more realistic/feasible strategy?
Thanks in advance for any help!
You could use Bottle, a very simple micro web-framework.
Bottle is a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no dependencies other than the Python Standard Library.
Hello world:
from bottle import route, run
#route('/hello/:name')
def index(name='World'):
return '<b>Hello %s!</b>' % name
run(host='localhost', port=8080)
The fastest and easiest way would probably be using cgi-scripts. If you want a more sophisticated approach you could consider using a webframework like django, turbogears or the likes.
I would suggest using one of the lighter-weight pure-Python web server options and either write a stand-alone WSGI application or use a micro-framework.
Gevent would be a good option: http://www.gevent.org/servers.html
Here is a sample implementation of a WSGI application using Gevent:
https://bitbucket.org/denis/gevent/src/tip/examples/wsgiserver.py#cl-4
For a micro-framework, I'd suggest using Flask.
I want to create a very simple python web app. I don't need Django or any other web framwwork like that. Isn't there a simpler way to create a web app in python?
Thanks
If you don't need Django, try web.py
http://webpy.org/
import web
urls = (
'/(.*)', 'hello'
)
app = web.application(urls, globals())
class hello:
def GET(self, name):
if not name:
name = 'world'
return 'Hello, ' + name + '!'
if __name__ == "__main__":
app.run()
Sure! For example,
print 'Content-Type: text/plain'
print ''
print 'Hello, world!'
this is a web app -- if you save it into a file in an appropriate directory of a machine running a web server and set the server's configuration properly (depending on the server); the article I pointed to specifically shows how to deploy this web app to Google App Engine, but just about any web server can serve CGI apps, and this is a simple example thereof.
Of course, CGI has its limits, and you can use more sophisticated approaches (still short of a framework!) such as WSGI (also universally supported, if nothing else because it can run on top of CGI -- but in most cases you can also deploy it in more advanced ways) and possibly some of the many excellent utility components you can deploy with WSGI to save you work in coding certain parts of your apps.
WSGI is probably what you are looking for. Though there are several lightweight python web frameworks around which are less monolithic than django.
The truth is that you do need a framework of some sort even if it's extremely minimal. You can use WSGI as a base and at least you're doing a little better. Python is a very powerful, very unspecific programming language so if you decide to do it without a framework you're going to have to rewrite huge amounts of code that you may be taking for granted.
If you do decide to go with something other than Django try this list and maybe you'll find something simple enough that you'll feel good about it. :)
first we have to install the package.
In terminal:
>>> pip install PySimpleGUI
Then go on type in this code...
# Code
import PySimpleGUI as sg
sg.theme('add_your_bgcoloor_here') layout = [[sg.Text('This is for the text')],
[sg.InputText('This is to input the texts')],
[sg.InputCombo(['Option 1', 'Option 2', 'Option 3'])
[sg.Radio('Checkbox', 1)],
[sg.Spin([x for x in range(1, 100)], initial_value=1)],
[sg.Button('go']]
# creating our window window = sg.Window('Window, layout)
# defining the events and the value in the layout using while True statement while True:
event, values = window.read()
# defining our button 'go' (sg.Button('go'))
if event == 'go':
break window.close()
Yep WSGI...
def hello_wsgi(environ, start_response):
start_response('200 OK', [('content-type', 'text/html')])
return ['Hello world!']
If you want to abstract this in terms of request/response to get a little further away from http try webob.
from webob import Request, Response
def hello_wsgi(environ, start_response):
request = Request(environ)
#do something with the request
#return a response
return Response("Hello World!")(environ, start_response)
I use bottle all the time as a minimal web framework.
It is very simple to use.
as a minimum example - taken from the web site :
from bottle import route, run
#route('/hello/:name')
def index(name='World'):
return '<b>Hello %s!</b>' % name
run(host='localhost', port=8080)
you simply associate url (route) to functions. This one even get an optional argument.
It has an optional light templating language, and you can tweak it a lot for our needs. Very powerful.
It is also very easy to instal - as it comes as a single file, standing along your app, and is pure compliant python. It is also very easy to debug, with a nice autoreload on modif while in development mode.
As a final advantages, it runs smoothly under pypy - thus providing a speed boost over other frameworks.
Now a days it is better to use PhusionPassenger Standalone or with NGINX using same technique as PHP by proxy passing it to FastCGI in case of PHP and WSGI for Python.
The URL and all explanation for Passenger can be found: Here
All information about Python running on NGINX over FastCGI, uWSGI or Passenger
About frameworks that wrap Python for easier Web development I do recommend Django if it is a larger application and once you get the hang of it Django isn't that hard actually.
Good Luck!
You can try Appier (https://github.com/hivesolutions/appier). Here's a sample app:
import appier
class HelloApp(appier.App):
#appier.route("/", "GET")
def hello(self):
return "Hello World"
HelloApp().serve()
And here's how you run it:
pip install appier
python hello.py
Disclaimer: This framework is part of the open-source portfolio of my company. We built the framework to make the code for our consulting work as simple and clean as possible (to improve our efficiency). The project is very active, as we use the framework all the time, however, the caveat is that we've only started talking about it publicly recently, so there's no community around it yet. However, for that very reason, we're very open to cooperating closely with early bird developers in improving our documentation and adding new features.
So I'm trying to do more web development in python, and I've picked cherrypy, hosted by lighttpd w/ fastcgi. But my question is a very basic one: why do I need to restart lighttpd (or apache) every time I change my application code, or the code for an underlying library?
I realize this question extends from a basic mis(i.e. poor)understanding of the fastcgi model, so I'm open to any schooling here, but I'm used to just changing a PHP file and it showing up, versus having to bounce the web server.
Any elucidation/useful mockery appreciated.
This is because of performance. For development, autoreloading is helpful. But for production, you don't want to autoreload. This is actually a decently-sized bottleneck in say PHP. Every time you access a PHP webpage, the server has to parse and load each page from scratch. With Python, the script is already loaded and running after the first access.
As has been pointed out, CherryPy has a autoreload setting. I'd recommend using the CherryPy built-in server for development and using lighttpd for production. That will likely save you some time. The tutorial shows you how to do this.
From a system-software-writer's pointer of view: This all depends on how the meta-data about the server process is organized within your daemon (lighttpd or fcgi). Some programs are designed for one time only initialization -- MOSTLY this allows a much simpler and better performing internal programming model.
Often it is very hard to program a server process reload config data in a easy way. You might have to introduce locks and external event objects (signals in UNIX). When you can synchronize the data structures by design -- i.e., only initializing once .... why complicate things by making the data model modifiable multiple times ?