I have created a backend for my google app that looks like this:
backends:
- name: dbops
options: dynamic
and I've created an admin handler for it:
- url: /backend/.*
script: backend.app
login: admin
Now I understand that admin jobs should be able to run forever and I'm launching this job with a TaskQueue, but for some reason mine is not. My job is simply creating a summary table in datastore from a much larger table. This table holds about 12000 records and it takes several minutes for it to process the job on the development server, but it works fine. When I push the code out to appspot and try to get it to run the same job, I'm getting what looks like datastore timeouts.
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1536, in __call__
rv = self.handle_exception(request, response, e)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1530, in __call__
rv = self.router.dispatch(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~myzencoder/dbops.362541511260492787/backend.py", line 626, in get
for asset in assets:
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 2314, in next
return self.__model_class.from_entity(self.__iterator.next())
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2816, in next
next_batch = self.__batcher.next()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2678, in next
return self.next_batch(self.AT_LEAST_ONE)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2715, in next_batch
batch = self.__next_batch.get_result()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 604, in get_result
return self.__get_result_hook(self)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2452, in __query_result_hook
self._batch_shared.conn.check_rpc_success(rpc)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1224, in check_rpc_success
raise _ToDatastoreError(err)
Timeout: The datastore operation timed out, or the data was temporarily unavailable.
Anyone got any suggestions on how to make this work?
While the backend request can run for a long time, a query can only run for 60 sec. You'll have to loop over your query results with cursors.
Mapreduce will get you a result quicker by doing the queries in parallel.
In production you use the HR datastore and you can run into contention problems. See this article.
https://developers.google.com/appengine/articles/scaling/contention?hl=nl
And have a look at mapreduce for creating a report. Maybe this is a better solution.
Related
I am using this python script to migrate data from one ElastiCache redis instance to another. It uses the redis pipelining to migrate data in chunks.
https://gist.github.com/thomasst/afeda8fe80534a832607
But I am getting this strange error:
Traceback (most recent call last):########### | ETA: 0:00:12
File "migrate-redis.py", line 95, in <module>
migrate()
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 664, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 644, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 837, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 464, in invoke
return callback(*args, **kwargs)
File "migrate-redis.py", line 74, in migrate
results = pipeline.execute(False)
File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 2593, in execute
return execute(conn, stack, raise_on_error)
File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 2446, in _execute_transaction
all_cmds = connection.pack_commands([args for args, _ in cmds])
File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 637, in pack_commands
output.append(SYM_EMPTY.join(pieces))
MemoryError
There are no issues with RAM as node has 6 GB of RAM.
The Memory Profile of source redis is as follows:
used_memory:1483900120
used_memory_human:1.38G
used_memory_rss:1945829376
used_memory_peak:2431795528
used_memory_peak_human:2.26G
used_memory_lua:86016
mem_fragmentation_ratio:1.31
mem_allocator:jemalloc-3.6.0
What can be the possible cause for this ?
From your error log, It has no relation with your redis server. The error happens in your redis client when it pack all commands into a memory buffer.
Maybe you could try to decrease the SCAN count option in your migrate-redis.py to test if it is too large to pack it.
I'm building a basic site with the dev version of Google App Engine(ver 1.9.14.1225) and I can't figure out how to enter the debugger to determine why my template variables are not rendered.
The App Engine documentation says to use: import pdb; pdb.set_trace();
https://cloud.google.com/appengine/docs/python/tools/devserver#Python_Debugging_with_PDB
However, when I inserted pdb into my code, it threw this error:
if self.quitting: raise BdbQuit
How do I enter the pdb debugger?
ERROR 2014-10-30 14:25:16,768 webapp2.py:1552]
Traceback (most recent call last):
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py", line 94, in get
self.response.write(template.render(template_values))
File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py", line 94, in get
self.response.write(template.render(template_values))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/bdb.py", line 49, in trace_dispatch
return self.dispatch_line(frame)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/bdb.py", line 68, in dispatch_line
if self.quitting: raise BdbQuit
osX 10.9.5
It looks like there is a known issue with PDB and Mac. I would "star" it to add more weight and comment that you are still seeing the issue.
Looks like it's caused because PDB uses stdin/stdout for i/o and the MAC dev_server doesn't work with them.
Looks like there's a third party tool that should work.
Here it looks like you can re-direct the I/O, I don't have a mac to test where you can re-direct but might be helpful.
There is also this tool to redirect the I/O to a socket. Not sure it will work but thought I would include it just in case.
If you are using the GUI to run your dev server I don't believe you will have access to pdb.
You should try the command line run server for app engine via:
dev_appserver.py myapp
and possibly even the django-server: django-admin runserver
I'm using imaplib to query Gmail's IMAP, but some requests are taking more than 60 seconds to return. This is already done in a task, so I have a full 10 minutes to do the request, but my tasks are failing due to the 60 second limit on urlfetch.
I've tried setting urlfetch.set_default_fetch_deadline(600), but it doesn't seem to do anything.
Here's a stacktrace:
The API call remote_socket.Receive() took too long to respond and was cancelled.
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/imaplib.py", line 760, in uid
typ, dat = self._simple_command(name, command, *args)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/imaplib.py", line 1070, in _simple_command
return self._command_complete(name, self._command(name, *args))
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/imaplib.py", line 897, in _command_complete
typ, data = self._get_tagged_response(tag)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/imaplib.py", line 999, in _get_tagged_response
self._get_response()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/imaplib.py", line 916, in _get_response
resp = self._get_line()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/imaplib.py", line 1009, in _get_line
line = self.readline()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/imaplib.py", line 1171, in readline
return self.file.readline()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/socket.py", line 445, in readline
data = self._sock.recv(self._rbufsize)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ssl.py", line 301, in recv
return self.read(buflen)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/ssl.py", line 220, in read
return self._sslobj.read(len)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/remote_socket/_remote_socket.py", line 864, in recv
return self.recvfrom(buffersize, flags)[0]
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/remote_socket/_remote_socket.py", line 903, in recvfrom
apiproxy_stub_map.MakeSyncCall('remote_socket', 'Receive', request, reply)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 94, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 328, in MakeSyncCall
rpc.CheckSuccess()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_rpc.py", line 133, in CheckSuccess
raise self.exception
DeadlineExceededError: The API call remote_socket.Receive() took too long to respond and was cancelled.
Which kind of DeadlineExceededError?
There are three kinds of DeadlineExceededError in AppEngine.
https://developers.google.com/appengine/articles/deadlineexceedederrors
google.appengine.runtime.DeadlineExceededError: raised if the overall request times out, typically after 60 seconds, or 10 minutes
for task queue requests.
google.appengine.runtime.apiproxy_errors.DeadlineExceededError: raised if an RPC exceeded its deadline. This is typically 5 seconds,
but it is settable for some APIs using the 'deadline' option.
google.appengine.api.urlfetch_errors.DeadlineExceededError: raised if the URLFetch times out.
As you can see. The 10mins limit of taskqueue only help thegoogle.appengine.runtime.DeadlineExceededError. The type of DeadlineExceededError can be identified via traceback (list below). In this case, it is google.appengine.runtime.apiproxy_errors.DeadlineExceededError. Which will raise in 5secs by default. (I will update the post after figure out how to change it)
TYPE 1. google.appengine.runtime.DeadlineExceededError
The traceback looks like
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 266, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~tagtooadex2/test.371204033771063679/index.py", line 9, in get
pass
DeadlineExceededError
Solution
This exception can be solved by using taskqueue (10mins), backend or manual scaling options.
https://developers.google.com/appengine/docs/python/modules/#Python_Instance_scaling_and_class
TYPE 2. google.appengine.runtime.apiproxy_errors.DeadlineExceededError
The traceback looks like
DeadlineExceededError: The API call remote_socket.Receive() took too long to respond and was cancelled.
TYPE 3. google.appengine.api.urlfetch_errors.DeadlineExceededError
The traceback looks like
DeadlineExceededError: Deadline exceeded while waiting for HTTP response from URL: http://www.sogi.com.tw/newforum/article_list.aspx?topic_ID=6089521
Solution:
This exception can be solved by extend the deadline.
urlfetch.fetch(url, deadline=10*60)
https://developers.google.com/appengine/docs/python/urlfetch/fetchfunction
There's no mentioning of timeout in imaplib sources. So there are several options:
IMAP uses socket library to establish the connection, consider using
socket.setdefaulttimeout(timeoutValue) but if you do so be
aware of side-effects.
The second option is to create your own IMAP4 class child with
tunable timeout, shall we say in open function
From the Google App Engine documentation, it seems like there are many
possible causes for DeadlineExceededError.
In your case, it seems that it may be one of the last two (out of three) types of DeadlineExceededError presented on the page.
We're trying to heavily use MapReduce in our project. Now we have this problem, there are a lot of 'InternalError: internal error.' errors in the log...
One example of it:
"POST /mapreduce/worker_callback HTTP/1.1" 500 0 "http://appname/mapreduce/worker_callback" "AppEngine-Google;
(+http://code.google.com/appengine)" "appname.appspot.com" ms=18856 cpu_ms=15980
queue_name=default task_name=appengine-mrshard-15828822618486744D69C-11-195
instance=00c61b117c47e0cba49bc5e5c7f9d328693e95ce
W 2012-10-24 06:51:27.140
suspended generator _put_tasklet(context.py:274) raised InternalError(internal error.)
W 2012-10-24 06:51:27.153
suspended generator put(context.py:703) raised InternalError(internal error.)
E 2012-10-24 06:51:27.207
internal error.
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~appname/1.362664407983567993/mapreduce/base_handler.py", line 65, in post
self.handle()
File "/base/data/home/apps/s~appname/1.362664407983567993/mapreduce/handlers.py", line 208, in handle
ctx.flush()
File "/base/data/home/apps/s~appname/1.362664407983567993/mapreduce/context.py", line 333, in flush
pool.flush()
File "/base/data/home/apps/s~appname/1.362664407983567993/mapreduce/context.py", line 221, in flush
self.__flush_ndb_puts()
File "/base/data/home/apps/s~appname/1.362664407983567993/mapreduce/context.py", line 239, in __flush_ndb_puts
ndb.put_multi(self.ndb_puts.items, config=self.__create_config())
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3650, in put_multi
for future in put_multi_async(entities, **ctx_options)]
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 325, in get_result
self.check_success()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 368, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 703, in put
key = yield self._put_batcher.add(entity, options)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 368, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 274, in _put_tasklet
keys = yield self._conn.async_put(options, datastore_entities)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 454, in _on_rpc_completion
result = rpc.get_result()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 834, in get_result
result = rpc.get_result()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 604, in get_result
return self.__get_result_hook(self)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1569, in __put_hook
self.check_rpc_success(rpc)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1224, in check_rpc_success
raise _ToDatastoreError(err)
InternalError: internal error.
queue.yaml:
queue:
- name: default
rate: 500/s
bucket_size: 100
max_concurrent_requests: 400
retry_parameters:
min_backoff_seconds: 5
max_backoff_seconds: 120
max_doublings: 2
MapReduce mapper params:
'shard_count': 16,
'processing_rate': 200,
'batch_size': 20
we would like to increase these numbers, since we need more speed in processing, but once we try to increase it increases error rate...
Blobstore Files Count: several ( some of them contain millions of lines )
Frontend Instance Class: F4
Processing flow:
We use only mapper for this particular processing.
We user BlobstoreLineInputReader (blob contains text file).
Each line represents new entry we need to create if it does not exist already(some of them we update).
My questions are:
How can we avoid these errors?
Are there any tips/hints on how we can choose/balance mapper params (shard_count, processing_rate, batch_size) ?
What happens with the job, does it get
retried (if so, how can we control it?) or not ?
BTW, we tried to play with some of the suggestions provided here (control batch_size) but we still see this.
This looks like a timeout error - check you logs to see how long that process is running before that happens.
If it is, you should try reducing the number of items that you're calling put_multi() on (ie reduce your batch size) and adding a timer check so that when your average time per put_multi() call gets close to the process time limit you quit and let another one start.
we're trying to heavily use MapReduce in our project.
Now we have this problem, there is a lots of 'DeadlineExceededError' errors in the log...
One example of it ( traceback differs each time a bit ) :
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 207, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~sba/1.362471299468574812/mapreduce/base_handler.py", line 65, in post
self.handle()
File "/base/data/home/apps/s~sba/1.362471299468574812/mapreduce/handlers.py", line 208, in handle
ctx.flush()
File "/base/data/home/apps/s~sba/1.362471299468574812/mapreduce/context.py", line 333, in flush
pool.flush()
File "/base/data/home/apps/s~sba/1.362471299468574812/mapreduce/context.py", line 221, in flush
self.__flush_ndb_puts()
File "/base/data/home/apps/s~sba/1.362471299468574812/mapreduce/context.py", line 239, in __flush_ndb_puts
ndb.put_multi(self.ndb_puts.items, config=self.__create_config())
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3625, in put_multi
for future in put_multi_async(entities, **ctx_options)]
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 323, in get_result
self.check_success()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 318, in check_success
self.wait()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 302, in wait
if not ev.run1():
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/eventloop.py", line 219, in run1
delay = self.run0()
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/eventloop.py", line 181, in run0
callback(*args, **kwds)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 365, in _help_tasklet_along
value = gen.send(val)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 274, in _put_tasklet
keys = yield self._conn.async_put(options, datastore_entities)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1560, in async_put
for pbs, indexes in pbsgen:
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1350, in __generate_pb_lists
incr_size = pb.lengthString(pb.ByteSize()) + 1
DeadlineExceededError
My questions are:
How can we avoid this Error?
What happens with the job, does it get retried (if so how can we control it?) or not ?
Does it causes data inconsistency in the end ?
Apparently you are doing too many puts than it is possible to insert in one datastore call. You have multiple options here:
If this is a relatively rare event - ignore it. Mapreduce will retry the slice and will lower put pool size. Make sure that your map is idempotent.
Take a look at http://code.google.com/p/appengine-mapreduce/source/browse/trunk/python/src/mapreduce/context.py - in your main.py you can lower DATASTORE_DEADLINE, MAX_ENTITY_COUNT or MAX_POOL_SIZE to lower the size of the pool for the whole mapreduce.
If you're using an InputReader, you might be able to adjust the default batch_size to reduce the number of entities processed by each task.
I believe the task queue will retry tasks, but you probably don't want it to, since it'll likley hit the same DeadlineExceededError.
Data inconsistencies are possible.
See this question as well.
App Engine - Task Queue Retry Count with Mapper API