When deploying my app to App Engine Standard's Python 3 runtime, how can I avoid request latency during an update to a new version, or starting new instances? Can I create some type of "warmup request"?
It's possible to configure custom warmup requests for your app. First, add the inbound_services directive and a corresponding handler in your app.yaml file:
inbound_services:
- warmup
handlers:
- url: /_ah/warmup
script: main.py
Then, define a warmup route in your main.py file:
#app.route('/_ah/warmup')
def warmup():
"""Warm up an instance of the app."""
pass # For example, initiate a db connection
See https://cloud.google.com/appengine/docs/standard/python3/configuring-warmup-requests for more details.
Related
So, everything else works... to preface this. But, I haven't really moved outside the admin interface. I'm trying to get data from an API and insert it into the database if there's changes. I've managed to write a script that can do that (in theory... it can do it locally), but I can't get the app in the cloud to recognize its existence. I've followed Google's suggestion of adding it to the app.yaml and cron.yaml to no avail.
Do I need to add this to a urls.py? I haven't mucked with teh handlers at all thus far and I'm not sure what settings.py makes happen, what the yaml files make happen, and how much of this is pixie dust.
here are teh relevant files...
app.yaml
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT mysite.wsgi
threadsafe: yes
beta_settings:
cloud_sql_instances: [redacted]
runtime_config:
python_version: 3
health_check:
enable_health_check: False
handlers:
- url: /static
static_dir: static/
- url: /run/get_data/
script: JSONdownload.app
login: admin
- url: .*
script: mysite.wsgi.application
cron.yaml
cron:
- description: "get data"
url: /run/get_data/
schedule: every 5 minutes
JSONdownload.py
#!/usr/bin/env python
# /var/spool/cron/crontabs
import json
import urllib2
from django.http import HttpResponse
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from .models import Game
from .models import Team
class JSONdownloadHandler(webapp.RequestHandler):
def get(self):
self.response.write('cron')
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.write('yay')
I'm finding great resources for a basic python app... but none for this situation really... anyone know of something better than what I'm doing, PLEASE let me know!
You're mixing up elements of the flexible environment app.yaml with those of the standard environment app.yaml
In particular the script: JSONdownload.app portion is ignored. You need to add the handler for the /run/get_data/ path inside your mysite.wsgi app, maybe from there invoking the JSONdownload.py code.
Somehow related: cron job in google app engine not working.
So... it was that I didn't route to teh location. I was actually able to just set a route (with admin against it) to the URL and then able to call it via the cron.yaml.
I have a warmup request in my Google App Engine project. However, the handler is called multiple times when testing on the local dev server
It is configured properly in the app.yaml file:
- url: /_ah/warmup
script: main.app
login: admin
inbound_services:
- warmup
and is doing it's job...I just can't workout why it is being called multiple times. Is this something to do with the local dev setup or somethign I am doing wrong.
Here is the main.py minus the imports
class WarmUpRequestsHandler(webapp2.RequestHandler):
"""
Warmup requests pre gae instance loading
"""
def get(self):
from google.appengine.api import memcache
from product.lib.lib import ProductCaches
logging.info('warmup request handler')
kwargs = {
'klass': TheLatest(),
'products_qry': 'EMA',
'product_entity': 'ema_latest',
'memcache_key': 'ema-latest',
}
entity = ProductCaches(**kwargs)
entity.get_and_set()
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Warmup successful')
webapp2_config = config.config
app = webapp2.WSGIApplication([("/_ah/warmup", WarmUpRequestsHandler)],debug = os.environ['SERVER_SOFTWARE'].startswith('Dev'), config=webapp2_config)
# Import All Routes
adminroutes.add_routes(app)
authroutes.add_routes(app)
emailroutes.add_routes(app)
hotroutes.add_routes(app)
mediaroutes.add_routes(app)
messageroutes.add_routes(app)
paymentroutes.add_routes(app)
productroutes.add_routes(app)
searchroutes.add_routes(app)
storeroutes.add_routes(app)
routes.add_routes(app)
I have the following in app.yaml
handlers:
- url: /.*
script: app.application
secure: always
login: required
When running tests I'm using this testrunner as suggested by google.
class SearchTest(unittest.TestCase):
def setUp(self):
# Set up app simulator
app = webapp2.WSGIApplication([('/search', search.Search)], debug=True)
self.testapp = webtest.TestApp(app)
# Google testbed
self.testbed = testbed.Testbed()
self.testbed.activate()
self.testbed.init_user_stub()
self.testbed.init_datastore_v3_stub()
self.testbed.init_memcache_stub()
# Disable caching to prevent data from leaking between tests
ndb.get_context().set_cache_policy(False)
def testNotLoggedin(self):
# Test user is redirected to login when not logged in
assert not users.get_current_user()
response = self.testapp.get('/search')
self.assertEqual(response.status_int, 302)
assert response.headers['Location']
The testNotLoggedIn fails with 200 != 302. So it seems like the user is still allowed access even though login is required. Which makes me think that app.yaml isn't recognized in the test?
How can I make sure that app.yaml is recognized and that the user needs to be logged in?
This test code bypasses the app.yaml dispatch. On App Engine (and in a dev server), the HTTP request is routed to the WSGI application instance via app.yaml, and the frontend logic that handles login: required happens before the request handler is invoked. In this test code, you're going directly to the WSGI application: self.testapp.get('/search') is simply calling the WSGI app's own internal URL mapping to get to the search.Search request handler.
The condition you want to test is more like an integration test, and requires either a running dev server or a deployed App Engine test version. It's a fine idea, it's just larger than what testbed et al can do.
I have never written a python script in my life, but I have a question that can hopefully be solved pretty quickly...
I'm using Google App Engine and Dropbprox. The script uses a custom domain to point to your public DropBox folder for better DropBox URLs. I'd like to be able to redirect users to my main site (jacob.bearce.me) if they visit my dropbox url (dl.bearce.me).
The problems that I'm having:
I've never used GAE or Python before, so I have no idea where to even begin
Putting a index.html file in my GAE project didn't fix it (I was hoping it'd just default to that if there was no filename specified, like it would on a normal site, but no cigar.)
Just a simple redirect if a users visits the main URL is all I'm after, nothing fancy.
My Python file: http://dl.bearce.me/mirror.py
Here's a main.py that issues a redirect for all requests, using the Python 2.5 runtime environment:
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
class MainHandler(webapp.RequestHandler):
def get(self):
self.redirect('http://jacob.bearce.me/')
application = webapp.WSGIApplication([('/.*', MainHandler)],
debug=True)
def main():
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
And here's the app.yaml file you need to route URLs to this handler:
application: myapp
version: 1
runtime: python
api_version: 1
handlers:
- url: .*
script: main.py
(Replace myapp with your actual app ID.)
For more information about creating and uploading an App Engine app in Python, see the Getting Started tutorial.
Can someone give an example of how the warmup inbound service works in the python runtime of Google App Engine?
I've read this: http://code.google.com/appengine/docs/python/config/appconfig.html#Inbound_Services, but it doesn't give me much of an example after the GET request is sent (I can't seem to ever pick it up)
My app.yaml looks like this:
application: whatevs
version: 1
runtime: python
api_version: 1
builtins:
- datastore_admin: on
inbound_services:
- warmup
handlers:
- url: /static
static_dir: static
- url: /_ah/warmup
script: main.py
login: admin
- url: /.*
script: main.py
my main.py looks like this:
def main():
application = webapp.WSGIApplication(
[("/", views.LandingPage),
("/_ah/warmup", views.WarmupHandler)
],
debug=True)
run_wsgi_app(application)
WarmupHandler looks like this:
class WarmupHandler(webapp.RequestHandler):
"""
Called on app init
"""
def get(self):
current_user = users.get_current_user()
return
However, WarmupHandler never seems to get called (I have breakpoints and lots of debug code). What am I doing wrong?
App Engine sends warm up requests only if there is some constant traffic on your app. It won't get always called if instances stand mostly idle.