We Keep Coding

Python & GAE - StringType write() argument type parsing

When running dev_appserver.py ., I get the following error when trying to access http://localhost:8080:
Traceback (most recent call last):
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 86, in run
self.finish_response()
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 128, in finish_response
self.write(data)
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions 2.7/lib/python2.7/wsgiref/handlers.py", line 204, in write
assert type(data) is StringType,"write() argument must be string"
AssertionError: write() argument must be string
I've searched and it seems to come back to my app.yaml file, per these links:
SO Question for GAE Assertion Error
Russian Question site with
same information
I'm just not sure how to go about debugging it. Below is my app.yaml file and my main.py file. I'm super new to the GAE platform and any help would be appreciated.
app.yaml file:
application: gqtimer
version: 1-56
runtime: python27
api_version: 1
threadsafe: false
handlers:
- url: /favicon.ico
static_files: static/images/favicon.ico
upload: static/images/favicon.ico
- url: /_ah/login_required
script: main.py
- url: /static
static_dir: static
- url: /player.*
script: main.py
login: required
- url: /stat.*
script: main.py
login: required
- url: .*
script: main.py
libraries:
- name: django
version: "1.11"
main.py file:
#!/usr/bin/env python
#
import config
import os
import sys
# Force sys.path to have our own directory first, so we can import from it.
sys.path.insert(0, config.APP_ROOT_DIR)
sys.path.insert(1, os.path.join(config.APP_ROOT_DIR, 'externals'))
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"
import wsgiref.handlers
from google.appengine.ext import webapp
from google.appengine.dist import use_library
use_library('django', '1.2')
from handlers import error, timer, do_openid_login
def main():
application = webapp.WSGIApplication([('/', timer.ExportHandler),
('/_ah/login_required', do_openid_login.OpenIdLoginHandler),
('/player/([-\w]+)', timer.PlayerHandler),
('/player/([-\w]+)/archives', timer.ArchivesHandler),
('/stat/([-\w]+)', timer.StatHandler),
('/stat/([-\w]+)/delete', timer.StatDeleteHandler),
# If we make it this far then the page we are looking
# for does not exist
('/.*', error.Error404Handler),
],
debug=True)
wsgiref.handlers.CGIHandler().run(application)
if __name__ == '__main__':
main()

Indeed, your app.yaml file might not be properly mapping your application code. You need to:
take the application variable outside of the main() function so that it becomes a global variable in the main.py module (and maybe rename it to app as well - just to stay inline with the official convention and documentation examples)
replace the script: main.py statements from your app.yaml's handlers with main.application (or main.app if renamed as mentioned above) - this is the reference to the above-mentioned global variable. From the script row in the Handlers element table:
A script: directive must be a python import path, for example,
package.module.app that points to a WSGI application. The last component of a script: directive using a Python module path is
the name of a global variable in the module: that variable must be a
WSGI app, and is usually called app by convention.
I'd also recommend explicitly passing the app.yaml as argument to dev_appserver.py instead of the app's directory (. in your case) - occasionally the auto-detection doesn't behave as expected. It's also the only way to run multiple services and/or using a dispatch.yaml file for routing, so it's a good habit.

KeyError with CherryPy WSGIServer serving static files

I'm trying to use CherryPy's WSGI server to serve static files, like in Using Flask with CherryPy to serve static files. Option 2 of the accepted answer there looks exactly like what I'd like to do, but I'm getting a KeyError when I try to use the static directory handler.
What I've tried:
>>>> import cherrypy
>>>> from cherrypy import wsgiserver
>>>> import os
>>>> static_handler = cherrypy.tools.staticdir.handler(section='/', dir=os.path.abspath('server_files')
>>>> d = wsgiserver.WSGIPathInfoDispatcher({'/': static_handler})
>>>> server = wsgiserver.CherryPyWSGIServer(('localhost', 12345), d)
>>>> server.start()
Then, when I try to access the server I'm getting a 500 response and the following error in the console:
KeyError('tools',)
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/cherrypy/wsgiserver/wsgiserver2.py", line 1353, in communicate
req.respond()
File "/Library/Python/2.7/site-packages/cherrypy/wsgiserver/wsgiserver2.py", line 868, in respond
self.server.gateway(self).respond()
File "/Library/Python/2.7/site-packages/cherrypy/wsgiserver/wsgiserver2.py", line 2267, in respond
response = self.req.server.wsgi_app(self.env, self.start_response)
File "/Library/Python/2.7/site-packages/cherrypy/wsgiserver/wsgiserver2.py", line 2477, in __call__
return app(environ, start_response)
File "/Library/Python/2.7/site-packages/cherrypy/_cptools.py", line 175, in handle_func
handled = self.callable(*args, **self._merged_args(kwargs))
File "/Library/Python/2.7/site-packages/cherrypy/_cptools.py", line 102, in _merged_args
tm = cherrypy.serving.request.toolmaps[self.namespace]
KeyError: 'tools'
This is displayed twice for each time I try to hit anything that the server should be able to display. When I hooked up a Flask app to the server the Flask app worked as expected, but the static file serving still gave the same error.
What do I need to do to get the staticdir.handler to work?

I've tried various ways of getting this to work and up until today was also hitting the KeyError you have been seeing (among other issues).
I finally managed to get CherryPy to serve static alongside a Django app by adapting the code from this gist (included below).
import os
import cherrypy
from cherrypy import wsgiserver
from my_wsgi_app import wsgi
PATH = os.path.abspath(os.path.join(os.path.dirname(__file__), 'public'))
class Root(object):
pass
def make_static_config(static_dir_name):
"""
All custom static configurations are set here, since most are common, it
makes sense to generate them just once.
"""
static_path = os.path.join('/', static_dir_name)
path = os.path.join(PATH, static_dir_name)
configuration = {static_path: {
'tools.staticdir.on': True,
'tools.staticdir.dir': path}
}
print configuration
return cherrypy.tree.mount(Root(), '/', config=configuration)
# Assuming your app has media on diferent paths, like 'c', 'i' and 'j'
application = wsgiserver.WSGIPathInfoDispatcher({
'/': wsgi.application,
'/c': make_static_config('c'),
'/j': make_static_config('j'),
'/i': make_static_config('i')})
server = wsgiserver.CherryPyWSGIServer(('0.0.0.0', 8070), application,
server_name='www.cherrypy.example')
try:
server.start()
except KeyboardInterrupt:
print "Terminating server..."
server.stop()
Hopefully wrapping a Flask app will be fairly similar.
The key for me was using the cherrypy.tree.mount on a dummy class, rather than trying to use the staticdir.handler directly.
For the curious - I used the code in the gist to customise a version of django-cherrypy's runcpserver management command, although in hindsight it would probably have been easier to create a new command from scratch.
Good luck (and thanks to Alfredo Deza)!

How to use bcrypt on Google App Engine (GAE)? [duplicate]

This question already has answers here:
How to include third party Python libraries in Google App Engine?
(6 answers)
Closed 8 years ago.
I have found a bcrypt library for python that seems to be very easy to use:
bcrypt 1.0.1
After installing it and testing the hello world example in my local machine all seems fine:
>>> import bcrypt
>>> password = b"super secret password"
>>> # Hash a password for the first time, with a certain number of rounds
>>> hashed = bcrypt.hashpw(password, bcrypt.gensalt(10))
>>> # Check that a unhashed password matches one that has previously been
>>> # hashed
>>> if bcrypt.hashpw(password, hashed) == hashed:
... print("It Matches!")
... else:
... print("It Does not Match :(")
However, in my GAE application, when I use import bcrypt I get an error:
Traceback (most recent call last):
File "/home/pedro/google_appengine/google/appengine/runtime/wsgi.py", line 239, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/home/pedro/google_appengine/google/appengine/runtime/wsgi.py", line 298, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/home/pedro/google_appengine/google/appengine/runtime/wsgi.py", line 84, in LoadObject
obj = __import__(path[0])
File "/home/pedro/google_appengine/hw4/blog.py", line 8, in <module>
import bcrypt
ImportError: No module named bcrypt
INFO 2014-05-05 21:17:04,375 module.py:639] default: "GET /blog/signup HTTP/1.1" 500 -
This leads me to believe that I must change the app.yaml file to include this library:
application: calm-grid-571
version: 1
runtime: python27
api_version: 1
threadsafe: False
handlers:
- url: /static
static_dir: static
- url: /.*
script: blog.app
libraries:
- name: jinja2
version: latest
- name: PIL
version: latest
However, when checking the official page for the supported libraries I cannot find anything about bcrypt.
So, how do I use the bcrypt library in GAE? Is it even possible?

You have to include the source of bcrypt (or any other non-embedded library) into your project.
Suggestion is to create a libs folder on the root of your project (same level where app.yaml lives) and place there as many libraries' sources as you need.
For this case, the final result should look like: /libs/bcrypt/
Make sure to include __init__.py blank files on any new folder you want your code to treat this folder as a package. After that, simply import the module: from libs.bcrypt import bcrypt
EDIT: Also note you can just have pure python code on your app engine project. Try py-bcrypt, it is working like a charm for a project hosted on App Engine.

How to load a basic module from a cron job

I am having trouble loading my GAE module.
My cron.yaml:
cron:
- description: call frontend instance to call module
url: /callLoadAndProcess
schedule: every day 01:00
timezone: America/New_York
Then the relevant part of my app.yaml:
- url: /callLoadAndProcess
script: callLoadAndProcess.application
secure: always
login: admin
Now my callLoadAndProcess.py:
import sys
import webapp2
import os
import urllib2
import logging
from google.appengine.api import modules
class callLoadAndProcess(webapp2.RequestHandler):
def get(self):
modules.start_module("loadandprocess","1")
application = webapp2.WSGIApplication([('/callLoadAndProcess', callLoadAndProcess)],debug=True)
For my module, I have a loadandprocess.yaml, which is:
application: [application name]
module: loadandprocess
version: 1
runtime: python27
instance_class: B4_1G
basic_scaling:
max_instances: 1
handlers:
- url: /.*
script: loadAndProcess.application
login: admin
And finally, loadAndProcess.py is the script I want run as a backend module:
class loadAndProcess(webapp2.RequestHandler):
def get(self):
#DO STUFF
application = webapp2.WSGIApplication([('/loadAndProcess', loadAndProcess)],debug=True)
In my development server, when I try to run the cron job via the admin page, I get the following error:
line 138, in _CheckAsyncResult
raise mapped_error()
InvalidVersionError
I feel I set it up correctly... and the version numbers match.. did I miss something? Thanks!

To run the app locally, specify both .yaml files to dev_appserver.py:
dev_appserver.py -A your-app-id application.yaml worker.yaml

mapping values are not allowed here ... in foo.py

I have this GAE python code
In file foo.py
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Hello Foo')
app = webapp2.WSGIApplication([('/', MainPage)], debug = True)
in file app.yaml
application: foo
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: foo.app
I get this error pointing to the third line in file foo.py ( class MainPage(webapp2.RequestHandler): )
Obs. Begin reading from the end of the message
...
line 172, in _HandleEvents
for event in events:
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/yaml_listener.py", line 212, in _GenerateEventParameters
raise yaml_errors.EventListenerYAMLError(e)
google.appengine.api.yaml_errors.EventListenerYAMLError: mapping values are not allowed here
in "foo.py", line 3, column 39
I would appreciate a good help
thanks
Sam

This kind of error occurs if you start the application the wrong way: dev_appserver.py foo.py. You need a directory, e.g., foo with foo/foo.py and foo/app.yaml and then start the program from the parent directory with dev_appserver.py foo/ or in the directory itself with dev_appserver.py .

There is nothing wrong with your code. I copy and pasted both into files and ran them on my Win7 system using App Engine SDK release: "1.7.7" and it served up the page without errors.
You may have issues with the files or your setup.
Have you tried the File->Creating New Application menu option? It will create a new application called engineapp that will display "Hello world!" when browsed on the localhost machine.