Here is my app.yaml:
application: helloworld
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: helloworld.app
libraries:
- name: lxml
version: latest
Here is helloworld.py:
from google.appengine.api import urlfetch
import lxml.html
down = "http://sc.hkex.com.hk/gb/www.hkex.com.hk/chi/market/sec_tradinfo/stockcode/eisdeqty_c.htm"
file = urlfetch.fetch(down)
print file.content
root = lxml.html.parse(file.content)
data = root.xpath('//tr[#class="tr_normal"]/td[position()=1]')
name = [row.text_content() for row in data]
for code in name:
print code
When I run:
google_appengine/dev_appserver.py helloworld/
I open 127.0.0.1:8080 and I can see the website http://sc.hkex.com.hk/gb/www.hkex.com.hk/chi/market/sec_tradinfo/stockcode/eisdeqty_c.htm but there is wrong output from the line print file.content.
May you tell me how to fix this?
lxml is a C module, GAE only supports pure python modules so it's not supported.
Related
When running dev_appserver.py ., I get the following error when trying to access http://localhost:8080:
Traceback (most recent call last):
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 86, in run
self.finish_response()
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 128, in finish_response
self.write(data)
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions 2.7/lib/python2.7/wsgiref/handlers.py", line 204, in write
assert type(data) is StringType,"write() argument must be string"
AssertionError: write() argument must be string
I've searched and it seems to come back to my app.yaml file, per these links:
SO Question for GAE Assertion Error
Russian Question site with
same information
I'm just not sure how to go about debugging it. Below is my app.yaml file and my main.py file. I'm super new to the GAE platform and any help would be appreciated.
app.yaml file:
application: gqtimer
version: 1-56
runtime: python27
api_version: 1
threadsafe: false
handlers:
- url: /favicon.ico
static_files: static/images/favicon.ico
upload: static/images/favicon.ico
- url: /_ah/login_required
script: main.py
- url: /static
static_dir: static
- url: /player.*
script: main.py
login: required
- url: /stat.*
script: main.py
login: required
- url: .*
script: main.py
libraries:
- name: django
version: "1.11"
main.py file:
#!/usr/bin/env python
#
import config
import os
import sys
# Force sys.path to have our own directory first, so we can import from it.
sys.path.insert(0, config.APP_ROOT_DIR)
sys.path.insert(1, os.path.join(config.APP_ROOT_DIR, 'externals'))
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"
import wsgiref.handlers
from google.appengine.ext import webapp
from google.appengine.dist import use_library
use_library('django', '1.2')
from handlers import error, timer, do_openid_login
def main():
application = webapp.WSGIApplication([('/', timer.ExportHandler),
('/_ah/login_required', do_openid_login.OpenIdLoginHandler),
('/player/([-\w]+)', timer.PlayerHandler),
('/player/([-\w]+)/archives', timer.ArchivesHandler),
('/stat/([-\w]+)', timer.StatHandler),
('/stat/([-\w]+)/delete', timer.StatDeleteHandler),
# If we make it this far then the page we are looking
# for does not exist
('/.*', error.Error404Handler),
],
debug=True)
wsgiref.handlers.CGIHandler().run(application)
if __name__ == '__main__':
main()
Indeed, your app.yaml file might not be properly mapping your application code. You need to:
take the application variable outside of the main() function so that it becomes a global variable in the main.py module (and maybe rename it to app as well - just to stay inline with the official convention and documentation examples)
replace the script: main.py statements from your app.yaml's handlers with main.application (or main.app if renamed as mentioned above) - this is the reference to the above-mentioned global variable. From the script row in the Handlers element table:
A script: directive must be a python import path, for example,
package.module.app that points to a WSGI application. The last component of a script: directive using a Python module path is
the name of a global variable in the module: that variable must be a
WSGI app, and is usually called app by convention.
I'd also recommend explicitly passing the app.yaml as argument to dev_appserver.py instead of the app's directory (. in your case) - occasionally the auto-detection doesn't behave as expected. It's also the only way to run multiple services and/or using a dispatch.yaml file for routing, so it's a good habit.
This question already has answers here:
How to include third party Python libraries in Google App Engine?
(6 answers)
Closed 8 years ago.
I have found a bcrypt library for python that seems to be very easy to use:
bcrypt 1.0.1
After installing it and testing the hello world example in my local machine all seems fine:
>>> import bcrypt
>>> password = b"super secret password"
>>> # Hash a password for the first time, with a certain number of rounds
>>> hashed = bcrypt.hashpw(password, bcrypt.gensalt(10))
>>> # Check that a unhashed password matches one that has previously been
>>> # hashed
>>> if bcrypt.hashpw(password, hashed) == hashed:
... print("It Matches!")
... else:
... print("It Does not Match :(")
However, in my GAE application, when I use import bcrypt I get an error:
Traceback (most recent call last):
File "/home/pedro/google_appengine/google/appengine/runtime/wsgi.py", line 239, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/home/pedro/google_appengine/google/appengine/runtime/wsgi.py", line 298, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/home/pedro/google_appengine/google/appengine/runtime/wsgi.py", line 84, in LoadObject
obj = __import__(path[0])
File "/home/pedro/google_appengine/hw4/blog.py", line 8, in <module>
import bcrypt
ImportError: No module named bcrypt
INFO 2014-05-05 21:17:04,375 module.py:639] default: "GET /blog/signup HTTP/1.1" 500 -
This leads me to believe that I must change the app.yaml file to include this library:
application: calm-grid-571
version: 1
runtime: python27
api_version: 1
threadsafe: False
handlers:
- url: /static
static_dir: static
- url: /.*
script: blog.app
libraries:
- name: jinja2
version: latest
- name: PIL
version: latest
However, when checking the official page for the supported libraries I cannot find anything about bcrypt.
So, how do I use the bcrypt library in GAE? Is it even possible?
You have to include the source of bcrypt (or any other non-embedded library) into your project.
Suggestion is to create a libs folder on the root of your project (same level where app.yaml lives) and place there as many libraries' sources as you need.
For this case, the final result should look like: /libs/bcrypt/
Make sure to include __init__.py blank files on any new folder you want your code to treat this folder as a package. After that, simply import the module: from libs.bcrypt import bcrypt
EDIT: Also note you can just have pure python code on your app engine project. Try py-bcrypt, it is working like a charm for a project hosted on App Engine.
I am having trouble loading my GAE module.
My cron.yaml:
cron:
- description: call frontend instance to call module
url: /callLoadAndProcess
schedule: every day 01:00
timezone: America/New_York
Then the relevant part of my app.yaml:
- url: /callLoadAndProcess
script: callLoadAndProcess.application
secure: always
login: admin
Now my callLoadAndProcess.py:
import sys
import webapp2
import os
import urllib2
import logging
from google.appengine.api import modules
class callLoadAndProcess(webapp2.RequestHandler):
def get(self):
modules.start_module("loadandprocess","1")
application = webapp2.WSGIApplication([('/callLoadAndProcess', callLoadAndProcess)],debug=True)
For my module, I have a loadandprocess.yaml, which is:
application: [application name]
module: loadandprocess
version: 1
runtime: python27
instance_class: B4_1G
basic_scaling:
max_instances: 1
handlers:
- url: /.*
script: loadAndProcess.application
login: admin
And finally, loadAndProcess.py is the script I want run as a backend module:
class loadAndProcess(webapp2.RequestHandler):
def get(self):
#DO STUFF
application = webapp2.WSGIApplication([('/loadAndProcess', loadAndProcess)],debug=True)
In my development server, when I try to run the cron job via the admin page, I get the following error:
line 138, in _CheckAsyncResult
raise mapped_error()
InvalidVersionError
I feel I set it up correctly... and the version numbers match.. did I miss something? Thanks!
To run the app locally, specify both .yaml files to dev_appserver.py:
dev_appserver.py -A your-app-id application.yaml worker.yaml
I have this GAE python code
In file foo.py
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Hello Foo')
app = webapp2.WSGIApplication([('/', MainPage)], debug = True)
in file app.yaml
application: foo
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: foo.app
I get this error pointing to the third line in file foo.py ( class MainPage(webapp2.RequestHandler): )
Obs. Begin reading from the end of the message
...
line 172, in _HandleEvents
for event in events:
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/yaml_listener.py", line 212, in _GenerateEventParameters
raise yaml_errors.EventListenerYAMLError(e)
google.appengine.api.yaml_errors.EventListenerYAMLError: mapping values are not allowed here
in "foo.py", line 3, column 39
I would appreciate a good help
thanks
Sam
This kind of error occurs if you start the application the wrong way: dev_appserver.py foo.py. You need a directory, e.g., foo with foo/foo.py and foo/app.yaml and then start the program from the parent directory with dev_appserver.py foo/ or in the directory itself with dev_appserver.py .
There is nothing wrong with your code. I copy and pasted both into files and ran them on my Win7 system using App Engine SDK release: "1.7.7" and it served up the page without errors.
You may have issues with the files or your setup.
Have you tried the File->Creating New Application menu option? It will create a new application called engineapp that will display "Hello world!" when browsed on the localhost machine.
I've split handlers between 2 python files (main.py and main_cms.py). app.yaml defines the URLs that each python file will handle.
When I look at the Appstats, only the handlers from one of the 2 python files are profiled (the ones from main.py).
The 'magic' of webapp_add_wsgi_middleware(app) always used to work just fine, until the split. How can I make Appstats recording apply to all handlers?
appengine_config.py:
def webapp_add_wsgi_middleware(app):
from google.appengine.ext.appstats import recording
app = recording.appstats_wsgi_middleware(app)
return app
app.yaml:
builtins:
- appstats: on
...
- url: /services/.*
script: main.application
- url: /cms.*
script: main_cms.application
main.py and main_cms.py:
application = webapp2.WSGIApplication(url_mapping, config=config)
Running python2.7 / GAE 1.6.3, the dev_appserver.py shows Appstats correct for all handlers. The described problem is only seen in production.