We Keep Coding

Google App Engine: Correctly configuring a static site with advanced routing

I've spent several shameful hours trying to solve this to no avail...
Problem:
I have a static website I am developing that is 100% pre-processed via Grunt & Assemble (if you are familiar with Jekyll, it is essentially the same concept). It also has a simple static Blog component which houses category directories of various names. As such, I need the catch-all in my app.yaml to route them appropriately.
However, I also would also like to have a custom error page to show in place of the standard GAE page status.
It seems that you cannot accomplish accounting for both scenarios in app.yaml alone because you can only use the catch-all target once.
Here is the logic in my current app.yaml
- url: (.*)/
static_files: dist\1/index.html
upload: dist/index.html
expiration: "15m"
- url: /(.*)
static_files: dist/\1/index.html
upload: dist/(.*)/index.html
expiration: "15m"
This is perfect for my use case because it routes any path to an index file if it exists in the current directory. However, because it uses the catch-all, I cannot again use it for something like the following
- url: /(.*)
static_files: custom_error.html
or depend on
error_handlers:
- file: custom_error.html
because it only renders for paths with no matching url pattern...
Ideas:
My next thoughts were that I may be able to accomplish this with some advanced routing via an external Python script
- url: /.*
script: main.app
but after trying a myriad of various setups I have yet to stumble onto a way to accomplish this.
One example of a trail of breadcrumbs I was on would be
import os
import webapp2
import jinja2
# vars
jinja_environment = jinja2.Environment(loader=jinja2.FileSystemLoader('jinja'), extensions=['jinja2.ext.autoescape'], autoescape=True)
class mainHandler(webapp2.RequestHandler):
def get(self):
if (an-index-file-exists-in-this-directory)
# follow the same static file logic as the app.yaml
# static_files: dist/\1/index.html
# upload: dist/(.*)/index.html
else:
template = jinja_environment.get_template('404/index.html')
context = {
'page_title': '404',
}
self.response.out.write(template.render(context))
self.error(404)
app = webapp2.WSGIApplication([
('/.*', mainHandler)
], debug=False)
I'm not even sure if taking it into an external python file would help solve the issue or not, but this was my awkward stab at it.
Does anyone have any ideas on how you can achieve custom error pages when your catch-all pattern is being used for another important purpose?
Update: Solved
Ok I finally have this figured out, but because Stack Overflow thinks I'm not cool enough to answer my own question (low point threshold?), I've posted the solution here:
https://gist.github.com/dustintheweb/c5e6e4ee1a64d50d7f87
Good luck!

As #Anthuin's answer points out, you can't write (nor modify) those index.html files on disk (nor create new ones dynamically), so it makes little sense to try to read them from disk -- the "disk" available to GAE apps is read-only (and split between the part only available for static serving, and the part readable by the app code itself).
Rather, unless the index.html files are huge (unlikely, I suspect), I'd keep them in the GAE datastore for your app. A really simple model:
from google.appengine.ext import ndb
class Indx(ndb.Model):
body = ndb.TextProperty()
assuming the path is no longer than 500 characters and the body up to a megabyte. Then, your MainHandler becomes pretty simple:
class MainHandler(webapp2.RequestHandler):
def get(self):
key = ndb.Key('Indx', self.request.path)
ent = key.get()
if ent:
self.response.write(ent.body)
else:
# your existing 404-returning code goes here
The app.yaml routing /.* to this script, and your app = code, need no change.
Now, the only thing that remains is, how did you expect to write, or modify, these index.html files (now datastore entities)? I have no idea because as long as they were to be files your app couldn't possibly have written nor modified them. In any case, now that they're in the data store, the writing also becomes very simple:
def write_indx(path, body):
ent = Indx(body=body, id=path)
ent.put()
Note that there is no real "modify" -- to possibly "modify" some index.html's "body", you'll actually read the previous one, make a new body string, and rewrite the entity with the above write_indx.
Potential problems include: bodies of more than 1 MB; and, keys (paths) of more than 500 characters. The former can easily be worked around by using, as #Anhuin suggested, Google Cloud Storage instead of the GAE datastore; the latter may perhaps be a problem because even GCS object names have limits (differently from GCS object lengths) -- specifically, 1024 bytes once the name is converted into utf-8. Is either of these issues likely to be a problem for you? If so, let us know!

I don't think you can make your first solution work like that, because as you said it will match every pattern and the error handler will never match.
And unless you plan to deploy your app each time you have a new blog post to add, I don't think having a static handler is a working solution anyway because you cannot upload content directly in your static app folder.
But you can upload content to GCS (for example) and retrieve it from a WSGIApplication.
Then this WSGIApplication can have a custom error handler:
def handle_404(request, response, exception):
response.write("my custom error 404")
app.error_handlers[404] = handle_404
You just raise a 404 if the template isn't found and this handler will get called.

Authenticating user from SSO + OpenID in django for google apps custom domain

After a lot of reading and researching i found the following library, the most suited to work for my needs: to signup/login a user using a custom google apps domain.
I did the following settings:
GOOGLE_APPS_DOMAIN = 'example.com'
GOOGLE_APPS_CONSUMER_KEY = 'example.com'
GOOGLE_APPS_CONSUMER_SECRET = '*sekret*'
# domain where your application is running
GOOGLE_OPENID_REALM = 'http://*.hudora.biz/'
GOOGLE_OPENID_ENDPOINT = 'https://www.google.com/accounts/o8/ud'
added custom middlewares and other stuff.
But from now, where do i take it further, showing a form and making form posts to urls, how do i ask user information, it is not given in the docs.
Can anybody please help me with it?
How do i give a user the functionality to login/signup?
How do i customize a view?

You are almost there. For the sake of testing it on your localhost following are things you need to do:
You need to register an app at the Google API Console (while registering, put your domain 127.0.0.1:8000 or the exact location where your server is running)
You need to check the scope for your project, eg. calendar or maps etc.
You need to get the secret key.
Once you have got the secret, put them in your settings file, as described above. Change example.com and *.hudora.biz to 127.0.0.1:8000
Import the views from the library views for login and logout and map them to your urls probably login/logout.
This is all that is reqd. Hope it helps!

Optional URL Parameter in Route GAE webapp2

I'm really new to Python and GAE. I'm setting up a basic CRUD app for some test data and am trying to get some routing for the admin pages going. I'd like to use the same page for creating and editing an object. So basically I want:
/admin/edit/<id>
where <id> is optional and /admin/edit will route to the same page. I tried adding <id:\w*> to the route which then allowed me to hit the page without supplying an id, but then when I supplied the id, I received a 404. Then I tried <id:\w+> and got a 404 with and without an id. I'm not having much luck.
Can anyone help me with what regex I need for this?

You can set up a regex to parse IDs out of the URL. Here's a really premitive example using webapp2:
app = webapp2.WSGIApplication([('/', MainPage),
('/property/(.*)', PropertyHandler)],
debug=True)
And you setup your request handler to accept the additional parameter:
class PropertyHandler(webapp2.RequestHandler):
def get(self, propertyId):
For a real-world implementation, you'd want to be a bit more specific on the regex and add validation to the handler incase you get garbage or no ID.

google app engine to make many pages automatically

I am building a web app with google app engine with python as well as HTML and CSS and I have a bunch of pages (around 15) that I want to make that will all somewhat different though will have a similar purpose. Is there any way to only have a few handlers that can create a bunch of pages or do I need a different class in python for every page?
I know this question is somewhat vague without context for the pages but really any information about how to create multiple pages without coding entirely new handlers and HTML for them would be a huge help.

No you don't need a different class for each page.
You can redirect every request to a singe script with app.yaml
handlers:
- url: /.*
script: dispatcher.app
Then, from dispatcher.py you can redirect every request to a single RequestHandler and program all your logic there, like serving a different jinja2 template for different URLs dinamically.
The URL Mappings documentation explains how to redirect multiples URLs to one RequestHandler. For example:
class BrowseHandler(webapp.RequestHandler):
def get(self, category, product_id):
# Display product with given ID in the given category.
# Map URLs like /browse/(category)/(product_id) to BrowseHandler.
application = webapp.WSGIApplication([(r'/browse/(.*)/(.*)', BrowseHandler)
],
debug=True)
def main():
run_wsgi_app(application)
if __name__ == '__main__':
main()

It will really depend on what framework you are using (if at all), but what you are looking for is templating systems. Here is a good list of frameworks and templating systems.

It really depends on what you're trying to do, but if you're using webapp or webapp2, you can use one handler. Check the Request parameter's url field to find out which page the user is requesting.

Google App Engine: Redirect to RequestHandler

I just started with Google App Engine using python and I was following a tutorial and writing my own little app to get familiar the webapp framework. Now I just noticed the tutorial does the following self.redirect('/'). So that got me wondering: is there a way to redirect to a handler instead of a hardcoded path? Thought that might be better so that you can change your urls without breaking your app.

One alternative would be to have a map of symbolic names to URLs, that way you could redirect to the mapped URL - you could then update your URLs with impunity.
Or if you'd rather just execute the code from another handler, I don't know why you couldn't just make a method call - worst case, you could extract a common method from the two handlers and call that.

This isn't a limitation of App Engine so much as the webapp framework. webapp is intended to be a lightweight framework, providing just the essentials.
If you want fancier redirect behavior, try Django.

The accepted answer made me built my own little router before I realized that in webapp2 you can name your routes and then redirect using that name as described in webapp2 Uri routing
app = webapp2.WSGIApplication(
routes=[webapp2.Route('/', handler='RootController', name='root')])
and then redirect to them in the RequestHandler
self.redirect_to('root')
if your path contains placeholders you can supply the values for the placeholders and the webapp2.Router will build the correct uri for you. Again take a look at webapp2 Uri routing for more detailed information.