I have been using Google Apps Script for a while and quite like how it can easily be deployed on the web. My question is: is there something similar to Google Apps Script for Python? That is, some where we can create a Python application and run it completely online without having to package or containerize the app.
I am also looking at Docker and Google's App Engine right now but haven't been able to get a clear picture of pros and cons.
Note: the practical application of what I am asking includes
manipulating large databases and let different clients run
manipulation scripts on their own cloud database themselves.
This is a very broad question, and not really the kind that Stackoverflow can help with. Trying reading How to create a Minimal, Complete, and Verifiable example?.
But to attempt to answer your question I need to break it down.
1) I assume that you are using the script editor / IDE for developing / debugging GScript (aka JavaScript) code. And that is the simplest way to develop Google Apps Script (GAS) software. There is no alternative to this for python or any other language. However it is not obligatory to use the IDE to write software that uses the Google Apps Script APIs.
2) This link, for example, is the Google Apps Script API quickstart that demonstrates how to use the GAS APIs from python.
3) However, to look at the big picture... you can regard GAS as an example of Serverless Computing, where the Google App Engine, AWS Lambda, and others (see the link) provide a straightforward environment for developing and deploying software without having to worry about packaging, containers, etc.
Docker might offer some advantages over traditional server deployment, but I would not regard it as an equivalent of serverless, as there is still a container environment to implement and manage.
The Pros and Cons of each are highly subjective, depending upon the nature of the problem your software needs to solve, and your starting point - if your business is already heavily into Azure or AWS then that might be the way to go. But if you require access to Google APIs for Maps, etc. then that might be the way to go, otherwise it probably boils down the specific services on offer, and cost.
Related
Let me explain what I'm trying to achieve. In the past while working on Java platform, I used to write Java codes(say, to push or pull data from MySQL database etc.) then create a war file which essentially bundles all the class files, supporting files etc and put it under a servlet container like Tomcat and this becomes a web service and can be invoked from any platform.
In my current scenario, I've majority of work being done in Java, however the Natural Language Processing(NLP)/Machine Learning(ML) part is being done in Python using the NLTK, Scipy, Numpy etc libraries. I'm trying to use the services of this Python engine in existing Java code. Integrating the Python code to Java through something like Jython is not that straight-forward(as Jython does not support calling any python module which has C based extensions, as far as I know), So I thought the next option would be to make it a web service, similar to what I had done with Java web services in the past. Now comes the actual crux of the question, how do I run the ML engine as a web service and call the same from any platform, in my current scenario this happens to be Java. I tried looking in the web, for various options to achieve this and found things like CherryPy, Werkzeug etc but not able to find the right approach or any sample code or anything that shows how to invoke a NLTK-Python script and serve the result through web, and eventually replicating the functionality Java web service provides. In the Python-NLTK code, the ML engine does a data-training on a large corpus(this takes 3-4 minutes) and we don't want the Python code to go through this step every time a method is invoked. If I make it a web service, the data-training will happen only once, when the service starts and then the service is ready to be invoked and use the already trained engine.
Now coming back to the problem, I'm pretty new to this web service things in Python and would appreciate any pointers on how to achieve this .Also, any pointers on achieving the goal of calling NLTK based python scripts from Java, without using web services approach and which can deployed on production servers to give good performance would also be helpful and appreciable. Thanks in advance.
Just for a note, I'm currently running all my code on a Linux machine with Python 2.6, JDK 1.6 installed on it.
One method is to build an XML-RPC server, but you may wish to fork a new process for each connection to prevent the server from seizing up. I have written a detailed tutorial on how to go about this: https://speakerdeck.com/timclicks/case-studies-of-python-in-parallel?slide=68.
NLTK based system tends to be slow at response per request, but good throughput can be achieved given enough RAM.
I don’t want to burden you all with the details, but basically I’m a 2nd year compsci student with no Web dev experience.
Basically I want to create a small “web app” that takes in input from a html form, have a python script perform some calculations, and re-display those results in your browser.
As of right now, I have the form and script built. However, when I try to test the form, instead of running the script, my browser tries to download it. To my understanding, this is a cgi script problem, and that I must create a web server in order to test this script.
And heres were I’m stuck. I know little to nothing about web servers and how to set them up. On top of that I’ve heard that GCI scripting is a thing of the past, and requires major overhead in order to run properly.
This leads to my questions. How do I go about completing my app and testing my cgi script? Do I install apache and mess around with it or should I be looking into something like google app engine? Are there other ways to complete this task without cgi scripts? Where do frameworks like Django fit into this?
Django, while being nice, all-encompassing and well-supported, is sometimes too much for a small web application. Django wants you to play by its rules from the beginning, you'll have to avoid things like the database and admin panels if you don't need them. It's also easier, with Django, to follow its project layout, even when it's too complex for a simple app.
The so-called micro frameworks might suit you better for your small app. They are built upon the opposite principle: use the bare minimum of features now, add more as you need them.
Flask is based on Werkzeug WSGI library and Jinja2 templating (the latter is switchable), is extensively documented (with notes concerning virtualenv and stuff) and well-suited for small and larger apps alike. It comes bundled with an auto-reloading dev server (no need for Apache on your dev machine) and Werkzeug-powered interactive debugger. There are extensions for things like HTML forms and database ORM.
Bottle is as small as a microframework can get, consisting of 1 (one) file, dev server included. Drop it into your project folder and start hacking. The built-in SimpleTemplate templating engine is switchable, but the dev server is flakier in comparison to Flask's. The documentation is less complete, and, in my opinion, the whole thing is less polished and convenient as Flask.
In both cases, you use dev server locally, and then deploy using WSGI, the server interface for Python web apps which both frameworks support. There are many ways to deploy a WSGI app, Apache mod_wsgi being one of the popular ones.
I'd totally go with Flask unless one dependency (Bottle) is better than three (Flask, Jinja2 and Werkzeug).
(There are many other frameworks as well, so wait for their users to come and tell about them. I'd suggest to avoid web.py: it works, but is full of magic, and is inelegant compared to Flask or Bottle.)
One way of getting to working webapp quickly is by first understanding, and then modifying, something like the App Engine "guestbook" example. This has the benefit that much of the otherwise necessary tedium of running a web server and setting up a database server (assuming you need persistence) is done for you. App Engine also provides a fairly flexible development environment. It's certainly not the only way to go, and I'll admit to bias in recommending it, but it's fairly low friction.
GCI scripting is hardly a thing of the past, though it's not what the cool kids are doing. CGI has the benefit, and the curse, of exposing more of the raw plumbing. It forces you to understand a lot about primitive (in the low-level sense) web architecture, but it's also a bit of a large bite to chew on if you have an immediate problem to solve that can solved by simpler means.
It appears most python web development seems to be done by frameworks these days. There are a couple reasons for this:
a plethora of mature tools. Django has built in user auth, built in database management, built in sessions, built in just about everything ORM which lets you seamlessly supports a couple databases.
Built in webservers. The larger python frameworks like django and pylons have built in webservers. Django has a very simple webserver python manage.py startserver (that simple) That makes it extremely easy to create and debug applications. It is single threaded so dropping a debugger into it is painless
Huge communities. If you have a django question it will be answered very quickly the so community is huge.
The django tutorial will introduce you to all the major aspects of development. It is only 4 pages and you will be able to get your app going a lot simpler than having to read, learn and fiddle with an apache setup.
https://docs.djangoproject.com/en/dev/intro/tutorial01/
Although django for right now might be overkill if your app is just going to be 1 form and a script to process it. Because of its seamless testing framework it is quite easy to grow any project. I have never used flask or bottle or the other microframeworks, but I would keep in mind where your project will be in the future.
As for where django fits into this, it is a full stack framework encompassing presentation (templates), data management (server orm), authentication, middleware, forms ... everything necessary to create a completely inclusive web application. Django and almost all other python frameworks implement the wsgi standard. It is an interface that allows for interoperation between webservers. http://en.wikipedia.org/wiki/Web_Server_Gateway_Interface it is pretty dry and you will never have to interface it directly. That is what these frameworks do under the hood.
Why setup and maintain your own webserver if you can use app engine. It has an excellent SDK for testing your code. Here is an example https://developers.google.com/appengine/docs/python/gettingstarted/handlingforms
And Django you will find here : https://developers.google.com/appengine/docs/python/gettingstarted/templates
I prefer to use Jinja for templating.
Django comes with its own server, but in your case i would recommend apache and mod_python since it seems to be a rather simple site you're building.
Setting up Apache is a breeze and a simple search on the web should give you all you need.
You can find more information on mod_python here read up a little bit on it and then google after a tutorial that fits your needs.
I am trying to make a web application to perform scientific and engineering calculations. I am new to web developing and I've been looking for a free framework (with free hosting), that's why I came to Google App Engine, but there is no way to get scipy working, so I decided to switch to Java instead of python (although I found PythonAnywhere and it has numpy, scipy, etcetera, it has no GUI-building support like PyQt, wx, Tkinter...).
I would like to know if there is a way to use COLT or so for Java in the Google App Engine, or if there is some other option. I would rather free options since I'm at college, but cheap-customizable-options are totally welcome :D (even if it means to use another language)
PD: I hope this was understandable since english is not my language.
EDIT:
I TRIED to use apache commons math, and it seems like it's not going to work. The short answer to my question is: NO.
I believe GAE is severely limiting in what it will allow you to run.
I doubt you will find a completely free Java hosting solution.
To clarify the statements in other posts, GAE is incredibly limiting with respect to Python packages with C extensions. Anything pure Python will work fine. Scipy makes heavy use of C extensions, so it falls into this category.
Google recently introduced Python2.7 support, and with it, the ability to use NumPy on App Engine. I'm not sure if this covers your need, but it might be worth checking out.
I only develop with Python for Google App Engine, so I'm afraid I can't comment on the state of Java external dependencies.
GAE will limit lot of things if not all in your case. You might want to try out Heroku, Amazon Web Services within their free quota.
I see no reason not to do this. You can run front-end instances which can use 800MHz of processor and 128MB of RAM - you can run one all the time for free but you need to be able to split your tasks into 10min sections (if you use tasks, or 30 second sections otherwise). A backend is going to be chargeable and you'd probably find it cheaper to run on another system.
Being a web developer (php, symfony, doctrine) for 2 years now, I was recently asked by a friend to come up with a desktop solution. So I developed a project, installed a LAMP on his machine and he is mostly happy using it now. But I'm not. It just doesn't seem right to wait for a server response from a localhost. Obviously php isn't suited for desktop development. So, my question is: what language \ framework would you advice a php programmer if he was going to develop a desktop application (something that you can install, that has it's own gui, but utilizes the similar concepts of web apps: css, javascript, orm).
I would like to bring up Python as a possible answer to my question. Does anyone have an experience of developing a desktop app with Python, utilizing an ORM and(or) HTML-based GUI?
You should definitely use Python. It's great and really easy to get up and running and extremely powerful.
utilizing an ORM and(or) HTML-based GUI
Please don't do this. HTML has its purposes, and it's not in desktop apps. Please don't force your experience into new technologies.
You should look at the various python GUI frameworks. Tkinter, PyQt ( my preference), or GTK
It just doesn't seem right to wait for a server response from a localhost
Exactly! Did you profile your app, both on server side and in browser? There's no reason for a local web app to be slow, except if it is designed or implemented suboptimally. Same applies to a desktop app, which is generally harder to create.
So, fire up your Firebug, do explain plan to every database query your app issues, add whatever profiling your PHP settings allow, and see where the problem lies. Most probably, it's not in the choice of language.
Maybe Adobe AIR is something to take a look at? Check out their sample applications. If you don't like using a "proprietary" technology, check out HTML5 technologies like Web Storage and Web SQL (not in the HTML5) spec.) These technologies are completely client side (but can be extended to work over a network), so no need to wait for a server.
I've no clue how capable/easy to work with the Web SQL route is, but there are some decent AIR applications out there.
I've recently come across a project called Titanium. It's a platform for developing native desktop (and mobile) applications using web technology (html/css, javascript and server-side languages like python, ruby and PHP! That's exactly what I was looking for. However, I haven't found no decent documentation or examples, community is small, so it seems to be underdeveloped right now. It mostly aims at mobile development rather that desktop.
Apart from Titanium, I currently don't see a better way for a web developer to go.
Is the target platform windows? If so, consider C# with WPF. The UI is constructed using XAML which is very similar to HTML/CSS. C# uses the .NET framework, so while it is a much different programming language than PHP, the transition should not be too difficult.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
Are there any guidelines to writing Google App Engine Python code that would work without Google's infrastructure on other platforms?
Is there any known attempt to create an open source framework which can run applications designed for Google App Engine on other platforms?
Edit:
To clarify, the question really is:
If I develop an application on Google App Engine now, will I be able to migrate to another platform later, or is it a lock in?
There's a number of components required to make an app fully portable:
The runtime environment itself. This can be ported relatively simply, by setting up a CGI or FastCGI server that emulates the App Engine environment (which itself is basically slightly-enhanced CGI). Most of the code to do this is already in the SDK. Unfortunately, there's no easy pre-packaged toolkit for this yet.
The datastore. By far the most complicated API to port. There are a number of efforts underway: AppScale runs on EC2/Eucalyptus/Xen and uses a HyperTable or HBase backend; it's still very much beta quality and they don't distribute the data connector separately (it's the beginnings of a complete run-your-app-on-your-own-cloud solution). Jens is/was writing an SQLite backend, and there's my own project, BDBDatastore, which uses BDB-JE as the backend, and is fully functional (though beta quality). AppDrop, which others have mentioned, simply uses the development server as a backend, and hence isn't suitable for production use.
The users API needs replacing with something else, such as an OpenID based API. Again, fairly simple, but no premade solutions yet.
The Memcache API needs a backend that uses standard C memcache backends.
The other APIs have perfectly functional backends as part of the SDK, so don't really need porting.
Cron support would also need implementing, as would background processing, XMPP, etc, when they become available.
As you can see, there's a lot of work to be done, but no fundamental barriers to making your App Engine app run outside Google's environment. In fact, if you're interested, you're more than welcome to participate - I and others have plans to combine the solutions for the various pieces into a single 'OpenEngine' solution for hosting your own apps.
Use a high level framework that works on App-Engine. That way you can port your code to other servers when you want.
django has been patched and ported to work in the Appengine patch project and is the most used FW on appengine.
You may want to refer this step by step intro to running a django app on App engine
As far as the parallel infrastructure to run an app engine application is concerned, it is still way far. App Engine itself hasn't got as popular as people believed it to and google wanted it to be. Plus it is harder to develop on the builtin WebApp framework than on django.
Its quite unlikely to see a parallel infrastructure to run app engine application on, atleast for years to come. Rather it is likely to see django and other popular frameworks work out of the box on app engine and the work on this is currently underway in the referred project.
You can build AppEngine applications using the Django python framework (although the supported version is a bit behind the most recent Django release). Where you lose portability (at least right now) is when using GQL/BigTable for persistence. This is Google proprietary database platform. As Hank mentioned this is one of the biggest reasons to actually use AppEngine, but it is also the single largest locking point.
Here are a couple of links to Django support in AppEngine and GQL/BigTable:
http://code.google.com/appengine/articles/django.html
http://code.google.com/appengine/docs/python/datastore/gqlreference.html
So far, I found an experimental host called app-drop which is capable of hosting google app-engine projects. This should mean that it's atleast possible to run app engine projects outside of google's infrastructure.
This is however clearly not yet suitable for production.
The code should be mostly portable (they do a pretty good job of indicating which modules you can't use on AppEngine and which AppEngine-specific code corresponds to which forbidden modules), but the whole point of AppEngine is to get access to Google's infrastructure. There's not much point to writing your code to the AppEngine restrictions if you aren't going to be using their infrastructure.
AppDrop is a proof of concept port of AppEngine to Amazon Web Services / Elastic Computing completed in April of 2008. It uses a flat file instead of BigTable and runs in a single instance, so there are scaling issues; but it's developer says it took him only four days, and perhaps these limitations can be addressed by others.
I did the reverse migration from vanilla Unix to app engine recently very easily by using WHIFF resources. Basically configure anything platform dependant as a resource and then swap/replace the resources on different configurations.
http://piopio.appspot.com/W1000_1000.resources
also see
http://aaron.oirt.rutgers.edu/myapp/docs/W1100_1200.wwiki
for a detailed example of resource swapping/configuration.
(note: links may go away eventually, app is experimental.)
Check out typhoonae. it's in beta, but quite usable – we moved one of our apps to inhouse server running this stack.
AppScale is the most mature open source implementation of Google App Engine. It's been in development since 2008 and currently support all four languages: Python, Java, Go, and PHP. It has users running their application in production today.
The FAQ explains what APIs are supported and what is lacking:
https://github.com/AppScale/appscale/wiki/FAQs
(Disclaimer: I work on the project)