This is very weird. The title says most of it, my code should say the rest. Here's my main.py file:
from google.appengine.api import urlfetch
import webapp2
import jinja2
import json
import os
jinja_environment = jinja2.Environment(
loader=jinja2.FileSystemLoader(os.path.dirname(__file__)))
class MainPage(webapp2.RequestHandler):
def get(self):
response = urlfetch.fetch("http://localhost:8080/api/helloworld?name=totty", method=urlfetch.GET)
if response.status_code == 200:
result = json.loads(response.content)
template_values = {'response': result['msg']}
template = jinja_environment.get_template('index.html')
self.response.out.write(template.render(template_values))
app = webapp2.WSGIApplication(
[('/', MainPage)],
debug=True)
Here's my api.py file:
import webapp2
import json
class HelloWorld(webapp2.RequestHandler):
def get(self):
name = self.request.get('name') or 'world'
msg = "Hello {}!".format(name)
payload = json.dumps({'msg': msg})
# payload = json.dumps({'dir': str(dir(self.request)), 'body': str(self.request.body), 'name': str(self.request.arguments())})
self.response.headers['Content-Type'] = 'application/json'
self.response.write(payload)
app = webapp2.WSGIApplication(
[('/api/helloworld', HelloWorld)],
debug=True)
And in case my app.yaml file would help:
application: stacksort
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /api/.*
script: api.app
- url: /.*
script: main.app
libraries:
- name: webapp2
version: latest
- name: jinja2
version: latest
Nothing changes even if I add deadline=30 to my urlfetch call. I tested the API using httpie and JQuery and it works perfectly fine and returns in under 5 seconds.
I looked at the other questions, but I'm still stumbling into the dark. Help, tips or refactoring would be appreciated.
I plan to add calls to the StackEchange Search API, so I suspect the problem might come in that time also. If there's a better way to do this, please tell. Thanks.
You are trying to fetch a URL to and from your application which is strongly discouraged on Google App Engine.
Locally you won't be able to call the development server because it serves only one request at a time. Multi-threading is not observed.
Note: the new experimental development server is now able to handle multiple requests at a time.
Multithreaded serving for better performance for complex applications and more correct semantics e.g. accessing your own application through urlfetch no longer deadlocks.
On production environment, GAE prevents the fetch service to call the same application.
To prevent an app from causing an endless recursion of requests, a request handler is not allowed to fetch its own URL. It is still possible to cause an endless recursion with other means, so exercise caution if your app can be made to fetch requests for URLs supplied by the user.
While upgrading the SDK, I noticed this addition to the DevServer page:
Note: dev_appserver.py can only serve one request at a time. If your
application makes URL fetch requests to itself while processing a
request, these requests will fail when using the development web
server. (They will not fail when running on App Engine.) To test such
requests, you can run a second instance of dev_appserver.py on a
different port, then code your application to use the other server
when making requests to itself.
So I guess that solves my problem (or at least gives a satisfactory explanation for it).
Related
So I am trying to build a restful API using flask, served up by apache on centos (httpd).
Basic API calls work just fine but I am not making much progress on the more advanced aspects because every time it fails I just get an HTTP 500 response which is completely useless for troubleshooting and I have no server-side logs to look at. I am literally trying to solve this through trial and error and it is making me bang my head against the wall.
In order to make any progress on this project I need to setup some basic error logging, but I do not understand the documentation or existing threads about this. It is completely over my head.
What I want to do is have flask write out all warnings and exceptions generated by my application to a specific file (it can be in the app directory to keep it simple).
I am looking for the simplest, easiest, least mind bendy way of doing this... suggestions?
Here is a very simplified version of my app... it shows the basic structure I am using, so please use that for reference.
from flask import Flask, jsonify, request
from flask_restful import reqparse, abort, Resource, Api
app = Flask(__name__)
api = Api(app)
class fetchTicket(Resource):
def post(self):
request_data = request.get_json(force=True)
r_ticket_id = request_data['ticket_id']
return jsonify(ticket_id=r_ticket_id)
api.add_resource(fetchTicket, '/ticket/fetch')
if __name__ == "__main__":
import logging
from logging.handlers import FileHandler
app.debug = True
file_handler = FileHandler("/var/www/project_folder/error.log")
file_handler.setLevel(logging.DEBUG)
app.logger.addHandler(file_handler)
app.run()
But when I run the above code no error.log file is created. I am not sure what I am doing wrong.
Note: I did set the folder permissions so that the apache user has access to write to the directory where the log file should go, as per http://fideloper.com/user-group-permissions-chmod-apache but it did not help so I don't think it is a permissions issue.
You’ll need to explicitly include the stack trace when logging, using the extra kwarg.
logger.exception('Probably something went wrong', extra={'stack': True})
I have the following in app.yaml
handlers:
- url: /.*
script: app.application
secure: always
login: required
When running tests I'm using this testrunner as suggested by google.
class SearchTest(unittest.TestCase):
def setUp(self):
# Set up app simulator
app = webapp2.WSGIApplication([('/search', search.Search)], debug=True)
self.testapp = webtest.TestApp(app)
# Google testbed
self.testbed = testbed.Testbed()
self.testbed.activate()
self.testbed.init_user_stub()
self.testbed.init_datastore_v3_stub()
self.testbed.init_memcache_stub()
# Disable caching to prevent data from leaking between tests
ndb.get_context().set_cache_policy(False)
def testNotLoggedin(self):
# Test user is redirected to login when not logged in
assert not users.get_current_user()
response = self.testapp.get('/search')
self.assertEqual(response.status_int, 302)
assert response.headers['Location']
The testNotLoggedIn fails with 200 != 302. So it seems like the user is still allowed access even though login is required. Which makes me think that app.yaml isn't recognized in the test?
How can I make sure that app.yaml is recognized and that the user needs to be logged in?
This test code bypasses the app.yaml dispatch. On App Engine (and in a dev server), the HTTP request is routed to the WSGI application instance via app.yaml, and the frontend logic that handles login: required happens before the request handler is invoked. In this test code, you're going directly to the WSGI application: self.testapp.get('/search') is simply calling the WSGI app's own internal URL mapping to get to the search.Search request handler.
The condition you want to test is more like an integration test, and requires either a running dev server or a deployed App Engine test version. It's a fine idea, it's just larger than what testbed et al can do.
I'm getting this error while trying to run my application...
The redirect URI in the request: http://localhost:8080/oauth2callback did not match a registered redirect URI
In google API console i have registered my redirect urls
Redirect URIs: http://localhost:8080/
And in the client_secrets.json also i'm using the same as redirect url
I'm following this tutorial
https://developers.google.com/bigquery/articles/dashboard#addoauth2
Edit:
I just made some changes to the existing code
Now the
redirect URIs in API console is http://localhost:8080/oauth2callback
And here is my app.yaml
application: hellomydashboard
version: 1
runtime: python
api_version: 1
handlers:
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: /oauth2callback
script: oauth2client/appengine.py
- url: .*
script: main.py
Now though its not showing any error but it displays a blank page.
Here is my main.py
from bqclient import BigQueryClient
import httplib2
import os
from google.appengine.api import memcache
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from oauth2client.appengine import oauth2decorator_from_clientsecrets
# Project ID for project to receive bill.
# During limited availability preview, there is no bill.
# The value should be your quoted Client ID number
# which you previously recorded from code.google.com/apis/console
# REPLACE THIS NUMBER WITH YOUR CLIENT ID
PROJECT_ID = "My Project ID" #i just replaced dat
DATASET = "samples"
TABLE = "natality"
# CLIENT_SECRETS, name of a file containing the OAuth 2.0
# information for this application.
CLIENT_SECRETS = os.path.join(os.path.dirname(__file__),
'client_secrets.json')
http = httplib2.Http(memcache)
decorator = oauth2decorator_from_clientsecrets(CLIENT_SECRETS,
'https://www.googleapis.com/auth/bigquery')
bq = BigQueryClient(http, decorator)
class MainHandler(webapp.RequestHandler):
#decorator.oauth_required
def get(self):
self.response.out.write("Hello Dashboard!\n")
application = webapp.WSGIApplication([
('/', MainHandler),
], debug=True)
def main():
run_wsgi_app(application)
if __name__ == '__main__':
main()
So according to main.py if everything is fine it must print Hello Dashboard but it isn't
You will actually need to add the following to your redirect URIs:
http://localhost:8080/oauth2callback
Also, you may need to append a trailing / if the above doesn't match:
http://localhost:8080/oauth2callback/
using google openId I configured this
Redirect URIs: http://domain.com/authenticate/google
on https://code.google.com/apis/console, if you must create a app if you don't have one, note that must match entirely the url
In main.py functions main class add (decorator.callback_path, decorator.callback_handler()), and remove
- url: /oauth2callback
script: oauth2client/appengine.py
from app.yaml.
PS: You might get DownloadError if you have some proxy-configuration/webcontent-filter. If you disable these configurations or deploy it on Google Server, it will work just fine.
seems like google tries to match url with being case-sensitve cause when i tried it with /Authorize and /authorize, it gave me redirect_uri_mismatch error for first one but worked for latter one
someone try and let me know if i m wrong
In main.py file,
in the part where you create a wsgi application
under application = webapp.wsgiapplication(
add a handler
(decorator.callback_path,decorator.callback_handler()),
I have never written a python script in my life, but I have a question that can hopefully be solved pretty quickly...
I'm using Google App Engine and Dropbprox. The script uses a custom domain to point to your public DropBox folder for better DropBox URLs. I'd like to be able to redirect users to my main site (jacob.bearce.me) if they visit my dropbox url (dl.bearce.me).
The problems that I'm having:
I've never used GAE or Python before, so I have no idea where to even begin
Putting a index.html file in my GAE project didn't fix it (I was hoping it'd just default to that if there was no filename specified, like it would on a normal site, but no cigar.)
Just a simple redirect if a users visits the main URL is all I'm after, nothing fancy.
My Python file: http://dl.bearce.me/mirror.py
Here's a main.py that issues a redirect for all requests, using the Python 2.5 runtime environment:
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
class MainHandler(webapp.RequestHandler):
def get(self):
self.redirect('http://jacob.bearce.me/')
application = webapp.WSGIApplication([('/.*', MainHandler)],
debug=True)
def main():
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
And here's the app.yaml file you need to route URLs to this handler:
application: myapp
version: 1
runtime: python
api_version: 1
handlers:
- url: .*
script: main.py
(Replace myapp with your actual app ID.)
For more information about creating and uploading an App Engine app in Python, see the Getting Started tutorial.
I've been trying for a few days now to get Google App Engine to run a cron Python script which will simply execute a script hosted on a server of mine.
It doesn't need to post any data to the page, simply open a connection, wait for it to finish then email me.
The code I've previously written has logged as "successful" but I never got an email, nor did I see any of the logging.info code I added to test things.
Ideas?
The original and wrong code that I originally wrote can be found at Google AppEngine Python Cron job urllib - just so you know I have attempted this before.
Mix of weird things was happening here.
Firstly, app.yaml I had to place my /cron handler before the root was set:
handlers:
- url: /cron
script: assets/backup/main.py
- url: /
static_files: assets/index.html
upload: assets/index.html
Otherwise I'd get crazy errors about not being able to find the file. That bit actually makes sense.
The next bit was the Python code. Not sure what was going on here, but in the end I managed to get it working by doing this:
#!/usr/bin/env python
# import logging
from google.appengine.ext import webapp
from google.appengine.api import mail
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.api import urlfetch
import logging
class CronMailer(webapp.RequestHandler):
def get(self):
logging.info("Backups: Started!")
urlStr = "http://example.com/file.php"
rpc = urlfetch.create_rpc()
urlfetch.make_fetch_call(rpc, urlStr)
mail.send_mail(sender="example#example.com",
to="email#example.co.uk",
subject="Backups complete!",
body="Daily backups have been completed!")
logging.info("Backups: Finished!")
application = webapp.WSGIApplication([('/cron', CronMailer)],debug=True)
def main():
run_wsgi_app(application)
if __name__ == '__main__':
main()
Whatever it was causing the problems, it's now fixed.