Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
Are there any guidelines to writing Google App Engine Python code that would work without Google's infrastructure on other platforms?
Is there any known attempt to create an open source framework which can run applications designed for Google App Engine on other platforms?
Edit:
To clarify, the question really is:
If I develop an application on Google App Engine now, will I be able to migrate to another platform later, or is it a lock in?
There's a number of components required to make an app fully portable:
The runtime environment itself. This can be ported relatively simply, by setting up a CGI or FastCGI server that emulates the App Engine environment (which itself is basically slightly-enhanced CGI). Most of the code to do this is already in the SDK. Unfortunately, there's no easy pre-packaged toolkit for this yet.
The datastore. By far the most complicated API to port. There are a number of efforts underway: AppScale runs on EC2/Eucalyptus/Xen and uses a HyperTable or HBase backend; it's still very much beta quality and they don't distribute the data connector separately (it's the beginnings of a complete run-your-app-on-your-own-cloud solution). Jens is/was writing an SQLite backend, and there's my own project, BDBDatastore, which uses BDB-JE as the backend, and is fully functional (though beta quality). AppDrop, which others have mentioned, simply uses the development server as a backend, and hence isn't suitable for production use.
The users API needs replacing with something else, such as an OpenID based API. Again, fairly simple, but no premade solutions yet.
The Memcache API needs a backend that uses standard C memcache backends.
The other APIs have perfectly functional backends as part of the SDK, so don't really need porting.
Cron support would also need implementing, as would background processing, XMPP, etc, when they become available.
As you can see, there's a lot of work to be done, but no fundamental barriers to making your App Engine app run outside Google's environment. In fact, if you're interested, you're more than welcome to participate - I and others have plans to combine the solutions for the various pieces into a single 'OpenEngine' solution for hosting your own apps.
Use a high level framework that works on App-Engine. That way you can port your code to other servers when you want.
django has been patched and ported to work in the Appengine patch project and is the most used FW on appengine.
You may want to refer this step by step intro to running a django app on App engine
As far as the parallel infrastructure to run an app engine application is concerned, it is still way far. App Engine itself hasn't got as popular as people believed it to and google wanted it to be. Plus it is harder to develop on the builtin WebApp framework than on django.
Its quite unlikely to see a parallel infrastructure to run app engine application on, atleast for years to come. Rather it is likely to see django and other popular frameworks work out of the box on app engine and the work on this is currently underway in the referred project.
You can build AppEngine applications using the Django python framework (although the supported version is a bit behind the most recent Django release). Where you lose portability (at least right now) is when using GQL/BigTable for persistence. This is Google proprietary database platform. As Hank mentioned this is one of the biggest reasons to actually use AppEngine, but it is also the single largest locking point.
Here are a couple of links to Django support in AppEngine and GQL/BigTable:
http://code.google.com/appengine/articles/django.html
http://code.google.com/appengine/docs/python/datastore/gqlreference.html
So far, I found an experimental host called app-drop which is capable of hosting google app-engine projects. This should mean that it's atleast possible to run app engine projects outside of google's infrastructure.
This is however clearly not yet suitable for production.
The code should be mostly portable (they do a pretty good job of indicating which modules you can't use on AppEngine and which AppEngine-specific code corresponds to which forbidden modules), but the whole point of AppEngine is to get access to Google's infrastructure. There's not much point to writing your code to the AppEngine restrictions if you aren't going to be using their infrastructure.
AppDrop is a proof of concept port of AppEngine to Amazon Web Services / Elastic Computing completed in April of 2008. It uses a flat file instead of BigTable and runs in a single instance, so there are scaling issues; but it's developer says it took him only four days, and perhaps these limitations can be addressed by others.
I did the reverse migration from vanilla Unix to app engine recently very easily by using WHIFF resources. Basically configure anything platform dependant as a resource and then swap/replace the resources on different configurations.
http://piopio.appspot.com/W1000_1000.resources
also see
http://aaron.oirt.rutgers.edu/myapp/docs/W1100_1200.wwiki
for a detailed example of resource swapping/configuration.
(note: links may go away eventually, app is experimental.)
Check out typhoonae. it's in beta, but quite usable – we moved one of our apps to inhouse server running this stack.
AppScale is the most mature open source implementation of Google App Engine. It's been in development since 2008 and currently support all four languages: Python, Java, Go, and PHP. It has users running their application in production today.
The FAQ explains what APIs are supported and what is lacking:
https://github.com/AppScale/appscale/wiki/FAQs
(Disclaimer: I work on the project)
Related
I have been using Google Apps Script for a while and quite like how it can easily be deployed on the web. My question is: is there something similar to Google Apps Script for Python? That is, some where we can create a Python application and run it completely online without having to package or containerize the app.
I am also looking at Docker and Google's App Engine right now but haven't been able to get a clear picture of pros and cons.
Note: the practical application of what I am asking includes
manipulating large databases and let different clients run
manipulation scripts on their own cloud database themselves.
This is a very broad question, and not really the kind that Stackoverflow can help with. Trying reading How to create a Minimal, Complete, and Verifiable example?.
But to attempt to answer your question I need to break it down.
1) I assume that you are using the script editor / IDE for developing / debugging GScript (aka JavaScript) code. And that is the simplest way to develop Google Apps Script (GAS) software. There is no alternative to this for python or any other language. However it is not obligatory to use the IDE to write software that uses the Google Apps Script APIs.
2) This link, for example, is the Google Apps Script API quickstart that demonstrates how to use the GAS APIs from python.
3) However, to look at the big picture... you can regard GAS as an example of Serverless Computing, where the Google App Engine, AWS Lambda, and others (see the link) provide a straightforward environment for developing and deploying software without having to worry about packaging, containers, etc.
Docker might offer some advantages over traditional server deployment, but I would not regard it as an equivalent of serverless, as there is still a container environment to implement and manage.
The Pros and Cons of each are highly subjective, depending upon the nature of the problem your software needs to solve, and your starting point - if your business is already heavily into Azure or AWS then that might be the way to go. But if you require access to Google APIs for Maps, etc. then that might be the way to go, otherwise it probably boils down the specific services on offer, and cost.
I depend a lot on the Webbapp2 Framework to build my web apps and find it to currently be my easiest(favorite) alternative to others like Django & Flask because it integrates seamlessly with the python API for Google app engine.
I am however very worried of the fact that;
the twitter(#webapp2) last tweeted in 2011.
there is no active GitHub repo for webapp2 maintained by Google and yet Google code is turning READ ONLY!.
no new release version since 2011!
Those three reasons above have led me to think that the Project is Abandoned and might be killed eventually.
Since webapp2 (just as its predecessor webapp) was designed primarily/specifically for the GAE Python sandbox I think it's safe to assume it (or a reasonably similar replacement, if any) will remain alive for as long as GAE Python sanbox will.
Should at some point Google will decide to retire or replace it I presume they will also, as usual, provide early official notifications and ample time for migration to an alternate suggested solution before the apps relying on it will stop working.
I'd interpret the lack of activity on its codebase simply a sign of the code's quality and maturity.
WRT the Google Code going read-only it's really just a step in discontinuing Google's service of hosting sw projects, it doesn't mean development of those projects stops altogether. Development will be migrated to better project hosting services:
http://google-opensource.blogspot.ca/2015/03/farewell-to-google-code.html
I don’t want to burden you all with the details, but basically I’m a 2nd year compsci student with no Web dev experience.
Basically I want to create a small “web app” that takes in input from a html form, have a python script perform some calculations, and re-display those results in your browser.
As of right now, I have the form and script built. However, when I try to test the form, instead of running the script, my browser tries to download it. To my understanding, this is a cgi script problem, and that I must create a web server in order to test this script.
And heres were I’m stuck. I know little to nothing about web servers and how to set them up. On top of that I’ve heard that GCI scripting is a thing of the past, and requires major overhead in order to run properly.
This leads to my questions. How do I go about completing my app and testing my cgi script? Do I install apache and mess around with it or should I be looking into something like google app engine? Are there other ways to complete this task without cgi scripts? Where do frameworks like Django fit into this?
Django, while being nice, all-encompassing and well-supported, is sometimes too much for a small web application. Django wants you to play by its rules from the beginning, you'll have to avoid things like the database and admin panels if you don't need them. It's also easier, with Django, to follow its project layout, even when it's too complex for a simple app.
The so-called micro frameworks might suit you better for your small app. They are built upon the opposite principle: use the bare minimum of features now, add more as you need them.
Flask is based on Werkzeug WSGI library and Jinja2 templating (the latter is switchable), is extensively documented (with notes concerning virtualenv and stuff) and well-suited for small and larger apps alike. It comes bundled with an auto-reloading dev server (no need for Apache on your dev machine) and Werkzeug-powered interactive debugger. There are extensions for things like HTML forms and database ORM.
Bottle is as small as a microframework can get, consisting of 1 (one) file, dev server included. Drop it into your project folder and start hacking. The built-in SimpleTemplate templating engine is switchable, but the dev server is flakier in comparison to Flask's. The documentation is less complete, and, in my opinion, the whole thing is less polished and convenient as Flask.
In both cases, you use dev server locally, and then deploy using WSGI, the server interface for Python web apps which both frameworks support. There are many ways to deploy a WSGI app, Apache mod_wsgi being one of the popular ones.
I'd totally go with Flask unless one dependency (Bottle) is better than three (Flask, Jinja2 and Werkzeug).
(There are many other frameworks as well, so wait for their users to come and tell about them. I'd suggest to avoid web.py: it works, but is full of magic, and is inelegant compared to Flask or Bottle.)
One way of getting to working webapp quickly is by first understanding, and then modifying, something like the App Engine "guestbook" example. This has the benefit that much of the otherwise necessary tedium of running a web server and setting up a database server (assuming you need persistence) is done for you. App Engine also provides a fairly flexible development environment. It's certainly not the only way to go, and I'll admit to bias in recommending it, but it's fairly low friction.
GCI scripting is hardly a thing of the past, though it's not what the cool kids are doing. CGI has the benefit, and the curse, of exposing more of the raw plumbing. It forces you to understand a lot about primitive (in the low-level sense) web architecture, but it's also a bit of a large bite to chew on if you have an immediate problem to solve that can solved by simpler means.
It appears most python web development seems to be done by frameworks these days. There are a couple reasons for this:
a plethora of mature tools. Django has built in user auth, built in database management, built in sessions, built in just about everything ORM which lets you seamlessly supports a couple databases.
Built in webservers. The larger python frameworks like django and pylons have built in webservers. Django has a very simple webserver python manage.py startserver (that simple) That makes it extremely easy to create and debug applications. It is single threaded so dropping a debugger into it is painless
Huge communities. If you have a django question it will be answered very quickly the so community is huge.
The django tutorial will introduce you to all the major aspects of development. It is only 4 pages and you will be able to get your app going a lot simpler than having to read, learn and fiddle with an apache setup.
https://docs.djangoproject.com/en/dev/intro/tutorial01/
Although django for right now might be overkill if your app is just going to be 1 form and a script to process it. Because of its seamless testing framework it is quite easy to grow any project. I have never used flask or bottle or the other microframeworks, but I would keep in mind where your project will be in the future.
As for where django fits into this, it is a full stack framework encompassing presentation (templates), data management (server orm), authentication, middleware, forms ... everything necessary to create a completely inclusive web application. Django and almost all other python frameworks implement the wsgi standard. It is an interface that allows for interoperation between webservers. http://en.wikipedia.org/wiki/Web_Server_Gateway_Interface it is pretty dry and you will never have to interface it directly. That is what these frameworks do under the hood.
Why setup and maintain your own webserver if you can use app engine. It has an excellent SDK for testing your code. Here is an example https://developers.google.com/appengine/docs/python/gettingstarted/handlingforms
And Django you will find here : https://developers.google.com/appengine/docs/python/gettingstarted/templates
I prefer to use Jinja for templating.
Django comes with its own server, but in your case i would recommend apache and mod_python since it seems to be a rather simple site you're building.
Setting up Apache is a breeze and a simple search on the web should give you all you need.
You can find more information on mod_python here read up a little bit on it and then google after a tutorial that fits your needs.
I am wondering how to go about implementing a web application with Python.
For example, the html pages would link to python code that would give it increased functionality and allow it to write to a database.
Kind of like how Reddit does it.
If you're looking for server side programming with databases and html templates etc, I think Django is great, along with Pyramid. However, I use Flask ( http://flask.pocoo.org/ ) for this since it is easy to use, learn and deploy even though it may not have as much support as the before mentioned 2 framework since it's just a microframework, using the Jinja2 templating engine, including a development test server with it's own debugger.
On the other hand, if you're going for client-side programming (i.e. in browser implementation ) You can look up .NET Ironpython or even Brython which uses python like javascript.
You might want to check out mod_wsgi or mod_python.
What Is mod_wsgi?
The aim of mod_wsgi is to implement a simple to use
Apache module which can host any Python application which supports the
Python WSGI interface. The module would be suitable for use in hosting
high performance production web sites, as well as your average self
managed personal sites running on web hosting services.
-
Current State of Mod_Python
Currently mod_python is not under active development. This does not
mean that it is "dead" as some people have claimed. It smiply means
that the code and the project are mature enough when very little is
required to maintain it.
This is a good article from the Python website:
http://docs.python.org/howto/webservers.html
Plain CGI is a good starting point to learn about server side scripting, but it is an outdated technology and gets difficult to maintain after certain level of complexity. I would think it is no longer used in industrial-grade web server anymore. Plus you have to setup a web server and then install some module to interpret python script (like Apache with mod_python) just to get started.
I had some experience with Django (https://www.djangoproject.com/) and found it fairly easy to get started with since they come with development test server. All you need to have is a Python interpreter + Django and you can get up-and-running quickly and worry about the deployment setup later. They have pretty good documentation for beginner as well.
We have never used Python for a web site without a framework. In our case that is Django. In other words, we do not use Python for our web sites the way Perl can be used, just having Apache run a Perl script.
The recommendations you have received about Django are sound. If you go the Django route, Graham Dumpleton and the modwsgi Google group were very helpful to me. I could
not have gotten mod_wsgi deployed on Red Hat Enterprise 5 64-bit without Graham's help.
Whether you choose Django or "straight" Python, you will need to become familiar with mod_wsgi.
Good luck in quantum time, which means by now, I hope this all worked out for you.
Basically I have a webfaction space (assume for the purposes of this question that its free).
I am trying to learn python by created some simple web applications on Google App Engine using Eclipse + Pydev for development.
So far I have some basic functionality working in App Engine, though I have had some frustration with some library imports not working and whatnot (this may not be app engine specific).
So, is it worth it to switch now to webfaction, and leave GAE?
I am trying to learn python by created
some simple web applications on Google
App Engine using Eclipse + Pydev for
development.
This seems reasonable. Nothing wrong with using GAE, Eclipse, and Pydev to learn to do Python web dev.
So far I have some basic functionality
working in App Engine, though I have
had some frustration with some library
imports not working and whatnot (this
may not be app engine specific).
So, is it worth it to switch now to
webfaction, and leave GAE?
You haven't provided much of a reason to leave GAE now that you've started there. I think that "some frustration" is normal for learning on any platform.
Beyond that I think this could descend into a general GAE good for learning vs. GAE bad for learning discussion.
So let me get that started...
Since GAE is a 'Platform as a service' (PaaS) it makes deployment and maintenance very simple. You can get right to coding and not worry about the platform. As well, this platform provides some services, such as e-mail and authentication that make those tasks very easy. For example, on first pass you can just make use of their authentication API (supports Google and now openID authentication I think) and leave more complicated authentication options for later.
On the other hand, this platform is kinda' non-standard. The datastore is the main issue and that affects Django. That's a pain because it means that the Django running on GAE is slightly different then the standard Django and the Django docs that you are reading might not apply for GAE, etc.
Here is a current thread about getting started with Django on GAE (which addresses the datastore issue):
http://groups.google.com/group/google-appengine-python/browse_thread/thread/8d1c945d27b6305f
Hope that helps.