We Keep Coding

404 from cron job on google app engine django app

So, everything else works... to preface this. But, I haven't really moved outside the admin interface. I'm trying to get data from an API and insert it into the database if there's changes. I've managed to write a script that can do that (in theory... it can do it locally), but I can't get the app in the cloud to recognize its existence. I've followed Google's suggestion of adding it to the app.yaml and cron.yaml to no avail.
Do I need to add this to a urls.py? I haven't mucked with teh handlers at all thus far and I'm not sure what settings.py makes happen, what the yaml files make happen, and how much of this is pixie dust.
here are teh relevant files...
app.yaml
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT mysite.wsgi
threadsafe: yes
beta_settings:
cloud_sql_instances: [redacted]
runtime_config:
python_version: 3
health_check:
enable_health_check: False
handlers:
- url: /static
static_dir: static/
- url: /run/get_data/
script: JSONdownload.app
login: admin
- url: .*
script: mysite.wsgi.application
cron.yaml
cron:
- description: "get data"
url: /run/get_data/
schedule: every 5 minutes
JSONdownload.py
#!/usr/bin/env python
# /var/spool/cron/crontabs
import json
import urllib2
from django.http import HttpResponse
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from .models import Game
from .models import Team
class JSONdownloadHandler(webapp.RequestHandler):
def get(self):
self.response.write('cron')
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.write('yay')
I'm finding great resources for a basic python app... but none for this situation really... anyone know of something better than what I'm doing, PLEASE let me know!

You're mixing up elements of the flexible environment app.yaml with those of the standard environment app.yaml
In particular the script: JSONdownload.app portion is ignored. You need to add the handler for the /run/get_data/ path inside your mysite.wsgi app, maybe from there invoking the JSONdownload.py code.
Somehow related: cron job in google app engine not working.

So... it was that I didn't route to teh location. I was actually able to just set a route (with admin against it) to the URL and then able to call it via the cron.yaml.

App Engine - Python - Open local HTML file from script

When testing a Google App Engine app with the Launcher, the Launcher will start a local server, and open a tab listening to http://localhost:8080/ If the App.yaml file is configured to point to a html file, then the webpage will open. Like if your home page is index.html
app.yaml file
application: myProjectName
version: 2
runtime: python27
threadsafe: false
api_version: 1
handlers:
- url: .*
static_dir: index.html
If the app.yaml is configured to point to a python script in the root url, I don't know how to get the script to load a web page:
app.yaml file
- url: /.*
script: main.py
If my main.py Python script is:
import webbrowser
webbrowser.open_new("README.html")
That will open the README.html file in my browser when I run the code from the Python Shell, but if I launch the app from Google App Engine Launcher, it won't load the html file. How do I get the .py file to open an HTML file after the Launcher has started the app on the localhost:8000 ?
I'm looking at a Google example, and it uses a WSGIApplication webapp I guess. First the Python code goes through an authorization process, then at the end of the script, there is this code:
# Create an WSGI application suitable for running on App Engine
application = webapp.WSGIApplication(
[('/', MainPage), ('/svc', ServiceHandler), ('/about', AboutHandler),
('/user', UserHandler)],
# XXX Set to False in production.
debug=True
)
def main():
"""Main entry point for executing a request with this handler."""
run_wsgi_app(application)
if __name__ == "__main__":
main()
I'd appreciate any feedback from anyone who has any experience with this.

The simplest example that I know is the hello world in the documentation:
# helloworld.py
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Hello, World!')
application = webapp2.WSGIApplication([
('/', MainPage),
], debug=True)
The corresponding app.yaml is:
application: your-app-id
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: helloworld.application
You create a class MainPage -- whenever you make a request to localhost:8080/ (note the slash is unncessary), you will be directed to the main page. the application is responsible for routing the request to the proper class and creating a new instance of that class for each request. It also calls the get or post or whatever HTTP method. Whatever you write into the response is what gets returned to the browser as the web page.
Now a single page isn't all that interesting. Maybe you want localhost:8080/goodbye as well. Then you just add another class and "register" it with the application:
# helloworld.py
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Hello, World!')
class GoodbyePage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Goodbye, World. Time to sleep.')
application = webapp2.WSGIApplication([
('/', MainPage),
('/goodbye', GoodbyePage),
], debug=True)
No changes to app.yaml are necessary at this point.
Life might be kind of frustrating if you needed to keep all your pages in a single file though. We can break this into 2 (or more) files by modifying app.yaml.
application: your-app-id
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /goodbye
script: goodbye.app
- url: /.*
script: helloworld.application
helloworld.py is the same as the first example. goodbye.py looks like this:
# goodbye.py
import webapp2
class GoodbyePage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/html'
self.response.write('<html><head></head><body>Goodbye!</body></html>')
app = webapp2.WSGIApplication([
('/goodbye', GoodbyePage),
], debug=True)
The urls in app.yaml are regular expressions -- you want to order them from most specific to least specific (otherwise, you might handle the request with the wrong handler). Also note that the convention in app.yaml is script: <module_name>.<variable_name> when we set it up this way.

Here is another example without Jinja2. To serve HTML with Python you simply need these 3 simple things.
1) Use the webapp2 Python module.
import webapp2
2) Read your HTML file into a variable:
INDEX_HTML = open('index.html').read()
3) Then write it out when a certain URL is sent:
self.response.out.write(INDEX_HTML)
This is the full Python script:
import cgi
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
INDEX_HTML = open('index.html').read()
self.response.out.write(INDEX_HTML)
class PageTwo(webapp2.RequestHandler):
def post(self):
self.response.write('<html><body>You wrote:<pre>')
self.response.write(cgi.escape(self.request.get('content')))
self.response.write('</pre></body></html>')
application = webapp2.WSGIApplication([
('/', MainPage),
('/sign', PageTwo),
], debug=True)
Note that your app.yaml configuration needs to be correct, and you need an HTML file. The webapp2 module provides a RequestHandler that can route your URL requests. (Display different webpage content) Here is a GitHub repository with the code, manifest app.yaml file and the index.html file.
Serve HTML with Python

This gets the HTML from a file, then writes it to the browser. This is actually more of what I was looking for:
class MainPage(webapp2.RequestHandler):
def get(self):
template = JINJA_ENVIRONMENT.get_template('index.html')
self.response.write(template.render())
This is the entire module:
import cgi
import os
import jinja2
import webapp2
JINJA_ENVIRONMENT = jinja2.Environment(
loader=jinja2.FileSystemLoader(os.path.dirname(__file__)),
extensions=['jinja2.ext.autoescape'],
autoescape=True)
class MainPage(webapp2.RequestHandler):
def get(self):
template = JINJA_ENVIRONMENT.get_template('index.html')
self.response.write(template.render())
class PageTwo(webapp2.RequestHandler):
def post(self):
self.response.write('<html><body>You wrote:<pre>')
self.response.write(cgi.escape(self.request.get('content')))
self.response.write('</pre></body></html>')
application = webapp2.WSGIApplication([
('/', MainPage),
('/sign', PageTwo),
], debug=True)
Here is a link to the Git Hub repository:
Get Template HTML, and write to browser
I saw an interesting variation of the app.yaml configuration file in a code example:
- url: /
static_files: index.html
upload: index.html
Even though the webbapp2 Request Handler is serving a particular HTML file based on the URL, the app.yaml file also controls what happens when different URL's are sent. The app.yaml file can be configured to run the python script AND upload a particular HTML file. If your application has lots of sub-directories, and the main.py script doesn't load until the end of your app.yaml file, configuring the app.yaml file to load static HTML right up front may cause your main page to load faster:
application: YourProjectNameHere
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
#Even though the main.py script also loads the index,
#this is faster if lots of processing
- url: /
static_files: index.html
upload: index.html
- url: /Client_Style
static_dir: Client_Style
- url: /Client_Pages
static_dir: Client_Pages
- url: /Client_Script
static_dir: Client_Script
- url: /Library
static_dir: Library
- url: /apiclient
static_dir: apiclient
- url: /Client_Data
static_dir: Client_Data
- url: /oauth2client
static_dir: oauth2client
- url: /uritemplate
static_dir: uritemplate
- url: /.*
script: main.application
libraries:
- name: webapp2
version: latest

Python ImportError: No module named main in Google app engine project

I have the following app.yaml file
application: gtryapp
version: 1
runtime: python27
api_version: 1
threadsafe: yes
handlers:
- url: /images/(.*\.(gif|png|jpg))
static_files: static/img/\1
upload: static/img/(.*\.(gif|png|jpg))
- url: /css/(.*\.css)
mime_type: text/css
static_files: static/css/\1
upload: static/css/(.*\.css)
- url: /js/(.*\.js)
mime_type: text/javascript
static_files: static/js/\1
upload: static/js/(.*\.js)
- url: /(.*\.html)
mime_type: text/html
static_files: static/\1
upload: static/(.*\.html)
- url: .*
script: main.app
libraries:
- name: webapp2
version: "2.5.2"
And the file app.py:
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
if self.request.url.endswith('/'):
path = '%sindex.html'%self.request.url
else:
path = '%s/index.html'%self.request.url
self.redirect(path)
application = webapp2.WSGIApplication([('/.*', MainPage)],
debug=True)
The files that I should deploy are just html files or js or images, I get the following error after compiling the app:
raise ImportError('%s has no attribute %s' % (handler, name))
ImportError: has no attribute app
Solved: I had to call "app" not "application" !
app = webapp2.WSGIApplication([('/.*', MainPage)],
debug=True)

You've called the file index.py, not main.py. Either rename it, or use index.app in the yaml.

The issue you are having is that your app.yaml file doesn't properly describe your code. Here is the offending bit:
- url: .*
script: main.app
This says that all URLs that were not matched by some previous entry should be handled by the app object of the main module, which should be a WSGI application object (see the WSGI standard).
This doesn't work because your code is set up differently. Your primary module is in index.py (the index module) and its interface with the server is via the CGI standard (though WSGI is used internally).
So, you need to change something. It could either be the app.yaml description of the app, or it could be the organization of your code.
Making your code work as a CGI-style program is easy. Just change app.yaml to point to index.py as the script. The .py part in this case is the file extension, and the file will be run as a script.
If instead you want to go with the newer, WSGI-compatible style (which is probably the best option), the documentation suggests the following format:
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write('Hello, webapp World!')
app = webapp2.WSGIApplication([('/', MainPage)])
Your code is almost like this already. To make it work, get rid of your main function and if __name__ == "__main__" boilerplate. Replace it with:
app = webapp.WSGIApplication([('/.*', IndexHandler)],
debug=False)
This creates an app object at the top level of your module. Now, either rename your index.py file to main.py, or change app.yaml to point to index.app. The .app part of this is different this time. Rather than a file extension, it represents Python member access (in this case, accessing a global variable in a module).

Exclude One URL From Python URL Masking

I have never written a python script in my life, but I have a question that can hopefully be solved pretty quickly...
I'm using Google App Engine and Dropbprox. The script uses a custom domain to point to your public DropBox folder for better DropBox URLs. I'd like to be able to redirect users to my main site (jacob.bearce.me) if they visit my dropbox url (dl.bearce.me).
The problems that I'm having:
I've never used GAE or Python before, so I have no idea where to even begin
Putting a index.html file in my GAE project didn't fix it (I was hoping it'd just default to that if there was no filename specified, like it would on a normal site, but no cigar.)
Just a simple redirect if a users visits the main URL is all I'm after, nothing fancy.
My Python file: http://dl.bearce.me/mirror.py

Here's a main.py that issues a redirect for all requests, using the Python 2.5 runtime environment:
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
class MainHandler(webapp.RequestHandler):
def get(self):
self.redirect('http://jacob.bearce.me/')
application = webapp.WSGIApplication([('/.*', MainHandler)],
debug=True)
def main():
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
And here's the app.yaml file you need to route URLs to this handler:
application: myapp
version: 1
runtime: python
api_version: 1
handlers:
- url: .*
script: main.py
(Replace myapp with your actual app ID.)
For more information about creating and uploading an App Engine app in Python, see the Getting Started tutorial.

How does the warmup service work in python google app engine?

Can someone give an example of how the warmup inbound service works in the python runtime of Google App Engine?
I've read this: http://code.google.com/appengine/docs/python/config/appconfig.html#Inbound_Services, but it doesn't give me much of an example after the GET request is sent (I can't seem to ever pick it up)
My app.yaml looks like this:
application: whatevs
version: 1
runtime: python
api_version: 1
builtins:
- datastore_admin: on
inbound_services:
- warmup
handlers:
- url: /static
static_dir: static
- url: /_ah/warmup
script: main.py
login: admin
- url: /.*
script: main.py
my main.py looks like this:
def main():
application = webapp.WSGIApplication(
[("/", views.LandingPage),
("/_ah/warmup", views.WarmupHandler)
],
debug=True)
run_wsgi_app(application)
WarmupHandler looks like this:
class WarmupHandler(webapp.RequestHandler):
"""
Called on app init
"""
def get(self):
current_user = users.get_current_user()
return
However, WarmupHandler never seems to get called (I have breakpoints and lots of debug code). What am I doing wrong?

App Engine sends warm up requests only if there is some constant traffic on your app. It won't get always called if instances stand mostly idle.