I'm building a basic site with the dev version of Google App Engine(ver 1.9.14.1225) and I can't figure out how to enter the debugger to determine why my template variables are not rendered.
The App Engine documentation says to use: import pdb; pdb.set_trace();
https://cloud.google.com/appengine/docs/python/tools/devserver#Python_Debugging_with_PDB
However, when I inserted pdb into my code, it threw this error:
if self.quitting: raise BdbQuit
How do I enter the pdb debugger?
ERROR 2014-10-30 14:25:16,768 webapp2.py:1552]
Traceback (most recent call last):
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py", line 94, in get
self.response.write(template.render(template_values))
File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py", line 94, in get
self.response.write(template.render(template_values))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/bdb.py", line 49, in trace_dispatch
return self.dispatch_line(frame)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/bdb.py", line 68, in dispatch_line
if self.quitting: raise BdbQuit
osX 10.9.5
It looks like there is a known issue with PDB and Mac. I would "star" it to add more weight and comment that you are still seeing the issue.
Looks like it's caused because PDB uses stdin/stdout for i/o and the MAC dev_server doesn't work with them.
Looks like there's a third party tool that should work.
Here it looks like you can re-direct the I/O, I don't have a mac to test where you can re-direct but might be helpful.
There is also this tool to redirect the I/O to a socket. Not sure it will work but thought I would include it just in case.
If you are using the GUI to run your dev server I don't believe you will have access to pdb.
You should try the command line run server for app engine via:
dev_appserver.py myapp
and possibly even the django-server: django-admin runserver
Related
I'm developing a personal website in Flask and I'm noticing something odd about the way the local server has been working for me.
When I do the following:
$ export FLASK_APP=run.py
$ flask run
This message is displayed:
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger pin code: 262-302-085
That's all well and good. But when I actually go to 127.0.0.1:5000, the server hangs and nothing ever happens. When I say 'hangs', I mean that the page stays blank but the loading icon indicates a successful connection, but the content of the site doesn't load. However, when I CTRL+C to close the script, then the website loads. Before doing so, it says the following:
^C * Serving Flask app "run"
Traceback (most recent call last):
File "/usr/local/bin/flask", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/flask/cli.py", line 507, in main
cli.main(args=args, prog_name=name)
File "/usr/local/lib/python2.7/dist-packages/flask/cli.py", line 374, in main
return AppGroup.main(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/cli.py", line 432, in run_command
use_debugger=debugger, threaded=with_threads)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 708, in run_simple
inner()
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 670, in inner
fd=fd)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 564, in make_server
passthrough_errors, ssl_context, fd=fd)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 474, in __init__
socket.SOCK_STREAM)
socket.error: [Errno 9] Bad file descriptor
I thought this was an error or something, so I CTRL+C and then I get back to my command line. I refresh the page, expecting the website to go down, and it's still there! Not only that, but the terminal has seemingly resurrected the process and is displaying HTTP 2xx status codes in the terminal as I move around the site and refresh pages. Strange. So I killed both chromium and my terminal and the website STILL loaded! So I deleted my cache for the last hour, restarted my laptop, and it was STILL up! It kind of quietly failed after another ten minutes, but I really don't understand what's happening here.
Can anyone show this confused lad some mercy? Because it doesn't seem like I should have to kill the script before the website loads. Something feels off, aside from the obvious. Anyone know what I'm doing wrong? I've been able to reproduce the problem and this has happened more than once now.
Thank you.
EDIT: Fixed a sentence and clarified that the problem is reproducible.
Does google app engine not support umask? I can use it in anaconda on windows, but when I try it in google app engine (running locally), it doesn't work.
This code:
import webapp2
import os
class MainHandler(webapp2.RequestHandler):
def get(self):
mask = os.umask(0o177)
self.response.write(mask)
app = webapp2.WSGIApplication([
('/', MainHandler)
], debug=True)
Gives me the following errors:
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1102, in __call__
return handler.dispatch()
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "C:\Users\Dan\Documents\Apps\test\main.py", line 22, in get
mask = os.umask(0o177)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\devappserver2\python\stubs.py", line 40, in os_error_not_implemented
raise OSError(errno.ENOSYS, 'Function not implemented')
OSError: [Errno 40] Function not implemented
dev_appserver (the program you use to "run locally" a GAE app) does its best to reproduce the limitations you will encounter on the "sandbox" when you deploy and run in production (i.e on appspot.com) -- it IS, after all, a development tool intended to make it easier for you to code and debug applications to deploy to appspot.com.
dev_appserver doesn't do a perfect job of this emulation, but it does a pretty good one -- in particular, forbidding your app from using standard library modules that are not supported in the production sandbox, or specific non-supplied functions in standard library modules that are only partly supported. os.umask is but one example of the latter case.
I have created a backend for my google app that looks like this:
backends:
- name: dbops
options: dynamic
and I've created an admin handler for it:
- url: /backend/.*
script: backend.app
login: admin
Now I understand that admin jobs should be able to run forever and I'm launching this job with a TaskQueue, but for some reason mine is not. My job is simply creating a summary table in datastore from a much larger table. This table holds about 12000 records and it takes several minutes for it to process the job on the development server, but it works fine. When I push the code out to appspot and try to get it to run the same job, I'm getting what looks like datastore timeouts.
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1536, in __call__
rv = self.handle_exception(request, response, e)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1530, in __call__
rv = self.router.dispatch(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~myzencoder/dbops.362541511260492787/backend.py", line 626, in get
for asset in assets:
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 2314, in next
return self.__model_class.from_entity(self.__iterator.next())
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2816, in next
next_batch = self.__batcher.next()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2678, in next
return self.next_batch(self.AT_LEAST_ONE)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2715, in next_batch
batch = self.__next_batch.get_result()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 604, in get_result
return self.__get_result_hook(self)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2452, in __query_result_hook
self._batch_shared.conn.check_rpc_success(rpc)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1224, in check_rpc_success
raise _ToDatastoreError(err)
Timeout: The datastore operation timed out, or the data was temporarily unavailable.
Anyone got any suggestions on how to make this work?
While the backend request can run for a long time, a query can only run for 60 sec. You'll have to loop over your query results with cursors.
Mapreduce will get you a result quicker by doing the queries in parallel.
In production you use the HR datastore and you can run into contention problems. See this article.
https://developers.google.com/appengine/articles/scaling/contention?hl=nl
And have a look at mapreduce for creating a report. Maybe this is a better solution.
I'm trying to run a django project locally on a brand new Mac. It was working up until yesterday, and I have no clue what happened, but all of a sudden I started getting Redis errors.
I can load pages that don't query the database, but once I try to do a search, which relies partly on Redis, it doesn't work.
Any ideas? here is the traceback.
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 85, in run
self.result = application(self.environ, self.start_response)
File "/Library/Python/2.7/site-packages/django/contrib/staticfiles/handlers.py", line 67, in __call__
return self.application(environ, start_response)
File "/Library/Python/2.7/site-packages/django/core/handlers/wsgi.py", line 241, in __call__
response = self.get_response(request)
File "/Library/Python/2.7/site-packages/django/core/handlers/base.py", line 179, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "/Library/Python/2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/Users/dlitwak/mozio/dotcloud/demo/search/views.py", line 391, in results
cache.setDistanceAndDuration(distance, time, request.user.username)
File "/Users/dlitwak/mozio/dotcloud/demo/cache.py", line 305, in setDistanceAndDuration
self.cache.set(key, value, 1800)
File "/Library/Python/2.7/site-packages/redis_cache/cache.py", line 218, in set
result = self._set(key, pickle.dumps(value), int(timeout), client, _add_only)
File "/Library/Python/2.7/site-packages/redis_cache/cache.py", line 199, in _set
return client.setex(key, value, timeout)
File "/Library/Python/2.7/site-packages/redis/client.py", line 1221, in setex
return self.execute_command('SETEX', name, time, value)
File "/Library/Python/2.7/site-packages/redis/client.py", line 338, in execute_command
connection.send_command(*args)
File "/Library/Python/2.7/site-packages/redis/connection.py", line 287, in send_command
self.send_packed_command(self.pack_command(*args))
File "/Library/Python/2.7/site-packages/redis/connection.py", line 269, in send_packed_command
self.connect()
File "/Library/Python/2.7/site-packages/redis/connection.py", line 217, in connect
raise ConnectionError(self._error_message(e))
ConnectionError: Error 2 connecting to unix socket: 127.0.0.1. No such file or directory.
We are running MYSQL. I can access the db through the terminal, so I don't think it's a DB access problem.
Sometimes this is because Django cannot connect to the database, and redis is throwing the error.
And since you can load pages without db, seems like it’s the case.
Ok, so we finally got it to work.
For some reason Redis is no longer being started automatically and is not running in the background.
I'm working on a Mac, and it previously did this automatically when I was running on Ubuntu. The solution was to run "redis-server" in a separate terminal window.
We are still trying to figure out how to make it start automatically, or why it stopped in the first place, but yeah.
If your using a Mac, using the Homebrew package manager is probably the best way to install and manage Redis. Once Homebrew is installed simply, brew install redis on the command line.
After it's installed you can set it up to run automatically on startup:
ln -sfv /usr/local/opts/redis/*.plist ~/Library/LaunchAgents
Then to run it now:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.redis.plist
I am trying to process some data with lxml. It works fine on my development server, but on production the following code:
parser = etree.XMLParser(encoding='cp1251')
throws:
File "parser.pxi", line 1288, in lxml.etree.XMLParser.__init__ (third_party/apphosting/python/lxml/src/lxml/lxml.etree.c:77726)
File "parser.pxi", line 738, in lxml.etree._BaseParser.__init__ (third_party/apphosting/python/lxml/src/lxml/lxml.etree.c:73404)
LookupError: unknown encoding: 'cp1251'
I am using lxml 2.3. The same version seems to be supported by GAE. So why is this error?
Edit:
I specified different encodings to XMLParser, such as cp1252, ISO-8859-5, ISO-8859-2 and it always throwed the same error on GAE, but works on my local machine. These are popular encodings and lxml on GAE must support them. I believe this is something wrong with lxml build on GAE.
I created an issue: http://code.google.com/p/googleappengine/issues/detail?id=7315
Edit2:
Full traceback:
unknown encoding: 'cp1251'
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~my_cool_app_id/1.358126884781269352/main.py", line 29, in get
parser = etree.XMLParser(encoding='cp1251')
File "parser.pxi", line 1288, in lxml.etree.XMLParser.__init__ (third_party/apphosting/python/lxml/src/lxml/lxml.etree.c:77726)
File "parser.pxi", line 738, in lxml.etree._BaseParser.__init__ (third_party/apphosting/python/lxml/src/lxml/lxml.etree.c:73404)
LookupError: unknown encoding: 'cp1251'
There seems to be a bug open about this behavior on OS X where specifying encoding="cp1252" resulted in the error above. The comments also specify other systems as affected: https://bugs.launchpad.net/lxml/+bug/707396
Have you tried specifying other encoding types? (to see if it's just a problem with cp1252)