We Keep Coding

Unable to deploy Python Flask application in Google App Engine Flex Environment

I have searched in multiple places and have tried multiple ways to find a solution to this. But nothing was helpful, can you please help me provide the solution. It is getting extremely difficult to host a python flask application in google app engine flexible environment. Python code: "getresource.py"
Here is the link from google to deploy Python Flask application
Here is the simple code:
from flask import Flask, jsonify
from pymysql import connections, ProgrammingError, DatabaseError, MySQLError, DataError
from os import environ
DB_HOST = environ.get("DB_HOST")
USER = environ.get("USER")
PWD = environ.get("PASSWORD")
DATABASE = environ.get("DATABASE")
app = Flask(__name__)
dbconn = connections.Connection(
host=DB_HOST,
port=3306,
user=USER,
password=PWD,
db=DATABASE
)
#app.route("/getResource/<id>", methods=['GET'])
def getresource(id):
result = ()
select_sql = "SELECT `id`, `firstName`, `middleName`, `lastName`, `listOfTechWorkedOn`, `certifications`, `projects`, `applicationWorkLoadTypes` FROM `resource` WHERE `id`=%s"
cursor = dbconn.cursor()
try:
cursor.execute(select_sql, (id))
result = cursor.fetchone()
response = {}
response['id'] = result[0]
response['firstName'] = result[1]
response['middleName'] = result[2]
response['lastName'] = result[3]
response['listOfTechWorkedOn'] = result[4]
response['certifications'] = result[5]
response['projects'] = result[6]
response['applicationWorkLoadTypes'] = result[7]
return jsonify(response)
except ProgrammingError as p:
return
except DatabaseError as d:
return d
except MySQLError as m:
return m
except DataError as de:
return de
dbconn.close()
if __name__ == '__main__':
app.run(host='127.0.0.1',port=8080, debug=True)
Here is the app.yaml file
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT getresource:app
service: getresource
runtime_config:
python_version: 3
manual_scaling:
instances: 1
env_variables:
DB_HOST: "1.2.3.4"
USER: "root"
PASSWORD: "abcdefg"
DATABASE: "abcd"
requirements.txt file
Flask==1.0.2
gunicorn==19.9.0
PyMySQL==0.9.2
I am getting the following error: "gcloud app deploy"
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred during deployment.
I have tried multiple ways, like changing the python code, putting google cloud libraries in the requirements.txt, chaning port, service name etc. But nothing seems to be working.
And surprisingly, I am not getting the error trace in stackdriver logs as well. And Google does not have proper answer it seems (at least from all the googling that i have done)
gcloud version
Google Cloud SDK 222.0.0
bq 2.0.36
core 2018.10.19
gsutil 4.34
**
EDIT:
** Thanks for all the response. I tried all the options, but getting the same error with following debug message. I cannot make anything out of it
Thanks for the response. I tried with all the suggested options, but getting the same error. Following is the debug log, and I cannot make anything out of it.
Updating service [getresource] (this may take several minutes)...failed.
DEBUG: (gcloud.app.deploy) Error Response: [13] An internal error occurred during deployment.
Traceback (most recent call last):
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\cli.py", line 841, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\backend.py", line 770, in Run
resources = command_instance.Run(args)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\surface\app\deploy.py", line 90, in Run
parallel_build=False)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 620, in RunDeploy
flex_image_build_option=flex_image_build_option)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 422, in Deploy
extra_config_settings)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\appengine_api_client.py", line 207, in DeployService
poller=done_poller)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\operations_util.py", line 315, in WaitForOperation
sleep_ms=retry_interval)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 254, in WaitFor
sleep_ms, _StatusUpdate)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 316, in PollUntilDone
sleep_ms=sleep_ms)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\core\util\retry.py", line 229, in RetryOnResult
if not should_retry(result, state):
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 310, in _IsNotDone
return not poller.IsDone(operation)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\operations_util.py", line 184, in IsDone
encoding.MessageToPyValue(operation.error)))
OperationError: Error Response: [13] An internal error occurred during deployment.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred during deployment.

How to avoid a socket timeout for selenium webdriver?

I am having a complex python-selenium test suite to test a non-public webpage. In that setup I need to get the webdriver like follows:
self.driver = webdriver.Firefox(firefox_profile = profile, log_path = logfile)
with some profile and log path. In most cases this line of code just works fine, but sometimes (5% or the cases) I get a socket timeout error:
File "/root/tests/usecase_tests/tools/basicsuite.py", line 213, in set_driver_firefox
self.driver = webdriver.Firefox(firefox_profile = profile, log_path = logfile)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 158, in __init__
keep_alive=True)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 154, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 243, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 309, in execute
response = self.command_executor.execute(driver_command, params)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/remote_connection.py", line 460, in execute
return self._request(command_info[0], url, body=data)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/remote_connection.py", line 484, in _request
resp = self._conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1136, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 409, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
timeout: timed out
What could be a possible cause of this error? How to debug this error? How to fix it?

The error says it all :
File "/usr/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
timeout: timed out
Sequence of events
Here is the sequence of events that happened :
Initially the error occurs at the following line in basicsuite.py file :
self.driver = webdriver.Firefox(firefox_profile = profile, log_path = logfile)
After a series of checks finally the following method is called which fails :
def readinto(self, b):
"""Read up to len(b) bytes into the writable buffer *b* and return
the number of bytes read. If the socket is non-blocking and no bytes
are available, None is returned.
If *b* is non-empty, a 0 return value indicates that the connection
was shutdown at the other end.
"""
self._checkClosed()
self._checkReadable()
if self._timeout_occurred:
raise OSError("cannot read from timed out object")
while True:
try:
return self._sock.recv_into(b)
except timeout:
self._timeout_occurred = True
raise
except error as e:
if e.args[0] in _blocking_errnos:
return None
raise
The program errors out at :
self._sock.recv(b) # where If *b* is non-empty, a 0 return value indicates that the connection was shutdown at the other end.
Conclusion
The conclusion is that the attempt to make the Connection wasn't successfull which means the initialization of the webdriver instance and further spawning of a new Mozilla Firefox Browser Session was unsuccessful.
Analysis
It won't be possible to conclude the real reason why timeout: timed out occured. But you can follow some of the Best Practices as follows :
Provide the complete name of the logfile along with the logical location of the logfile (from Project Level) as follows :
self.driver = webdriver.Firefox(firefox_profile=profile, log_path='./Log/geckodriver.log')
Always use quit() in the tearDown() method so that the webdriver and the webclient both are properly destroyed.
Before starting your Test Execution, through Task Manager ensure that there are no dangling instances of GeckoDriver or Firefox process within your system.
Ensure that the binary versions you are using JDK, Selenium, GeckoDriver, Mozilla Firefox Browser are compatible. You can find a detailed discussion in this QA Selenium WebDriver 3.4.0 + geckodriver 0.18.0 + Firefox ?? - which combination works?
Clean the Project Workspace from your IDE before and after executing your Test Suite.
Use CCleaner tool regularly to wipe off the OS chores.
If the base version of Firefox Browser is too ancient, uninstall the Firefox Browser through Revo Uninstaller with Moderate Scan and install a recent GA-Released version of Firefox Browser.

This happened to me in my Django app. The problem went away when I set DEBUG = False in my settings.py file.

it's probably not related but
if you are using vs-code on windows and you got this error just try to run from cmd, not from vs-code

Selenium Error: BadStatusLine

Hi I am new in Selenium Webdriver and I am currently creating a test case for a specific function.
I have a current test case that is already working for another function, but when I include a new test case and run it. I receive an error: BadStatusLine.
Also when I run the individual test case it work perfectly fine, but when I run the whole test I will receive an error: BadStatusLine.
ERROR: test_task_xml (__main__.TestActuserLayouts)
Test if the task xml is hidden
----------------------------------------------------------------------
Traceback (most recent call last):
File "acttemplate_layouts.py", line 25, in setUp
driver.find_element_by_name("password").submit()
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 84, in submit
self._execute(Command.SUBMIT_ELEMENT)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 457, in _execute
return self._parent.execute(command, params)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 231, in execute
response = self.command_executor.execute(driver_command, params)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute
return self._request(command_info[0], url, body=data)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/remote_connection.py", line 426, in _request
resp = self._conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1051, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 415, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 379, in _read_status
raise BadStatusLine(line)
BadStatusLine: ''
This is the whole test:
import unittest
from selenium import webdriver
class TestActuserLayouts(unittest.TestCase):
"""Test user functions """
driver = webdriver.Firefox()
driver.implicitly_wait(30)
base_url = "http://samplepage.com"
def setUp(self):
"""Base setting before test
"""
driver = self.driver
# Login
driver.get(self.base_url + "/")
driver.find_element_by_name("user_name").clear()
driver.find_element_by_name("user_name").send_keys("userme")
driver.find_element_by_name("password").clear()
driver.find_element_by_name("password").send_keys("123")
driver.find_element_by_name("password").submit()
self.assertEqual(self.base_url + "/userme/inbox/", driver.current_url)
def test_task_xml(self):
"""Test if the task xml is hidden
"""
driver = self.driver
# Get page by URL
driver.get(self.base_url + "/userme/mission/")
# Get page by URL
driver.get(self.base_url + "/mission/update/0000000a-0000-0000-0000-000000000000/")
self.assertEqual(self.base_url + "/mission/update/0000000a-0000-0000-0000-000000000000/", driver.current_url)
driver.find_element_by_id("discussion-btn").click()
# Scan browser if Task xml is not present
self.assertFalse('Task xml' in self.driver.page_source)
def test_task_list(self):
"""Test if the number of task for each mission is null
"""
driver = self.driver
#Get page url
driver.get(self.base_url + "/userme/mission/")
#Get page url
driver.get(self.base_url + "/acttemplate/list-all/")
#Scan browser if the number of task is null
self.assertEqual(driver.find_element_by_xpath("//tr[3]/td[4]").text != "", True)
def tearDown(self):
"""Clear setting after test
"""
self.driver.close()
# Run unit test directly
if __name__ == "__main__":
unittest.main()
While the other test case is a failure. So, I am not really familiar with selenium and I need help to locate this error. Can someone please help me in this issue. Thanks a lot.

I'm not sure, but I'll make a guess. Somehow, browser dies while you perform such actions like pressing buttons, clicking.
It's an error raised by httplib (low-level python http library), means that server inside browser didn't responded any of known http status codes (200, 300, 500, etc.), and instead, the response was '' (empty string).

Handling Non-SSL Traffic in Python/Tornado

I have a webservice running in python 2.7.10 / Tornado that uses SSL. This service throws an error when a non-SSL call comes through (http://...).
I don't want my service to be accessible when SSL is not used, but I'd like to handle it in a cleaner fashion.
Here is my main code that works great over SSL:
if __name__ == "__main__":
tornado.options.parse_command_line()
#does not work on 2.7.6
ssl_ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_ctx.load_cert_chain("...crt.pem","...key.pem")
ssl_ctx.load_verify_locations("...CA.crt.pem")
http_server = tornado.httpserver.HTTPServer(application, ssl_options=ssl_ctx, decompress_request=True)
http_server.listen(options.port)
mainloop = tornado.ioloop.IOLoop.instance()
print("Main Server started on port XXXX")
mainloop.start()
and here is the error when I hit that server with http://... instead of https://...:
[E 151027 20:45:57 http1connection:700] Uncaught exception
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/http1connection.py", line 691, in _server_request_loop
ret = yield conn.read_response(request_delegate)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 807, in run
value = future.result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 209, in result
raise_exc_info(self._exc_info)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 810, in run
yielded = self.gen.throw(*sys.exc_info())
File "/usr/local/lib/python2.7/dist-packages/tornado/http1connection.py", line 166, in _read_message
quiet_exceptions=iostream.StreamClosedError)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 807, in run
value = future.result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 209, in result
raise_exc_info(self._exc_info)
File "<string>", line 3, in raise_exc_info
SSLError: [SSL: HTTP_REQUEST] http request (_ssl.c:590)
Any ideas how I should handle that exception?
And what the standard-conform return value would be when I catch a non-SSL call to an SSL-only API?
UPDATE
This API runs on a specific port e.g. https://example.com:1234/. I want to inform a user who is trying to connect without SSL, e.g. http://example.com:1234/ that what they are doing is incorrect by returning an error message or status code. As it is the uncaught exception returns a 500, which they could interpret as a programming error on my part. Any ideas?

There's an excelent discussion in this Tornado issue about that, where Tornado maintainer says:
If you have both HTTP and HTTPS in the same tornado process, you must be running two separate HTTPServers (of course such a feature should not be tied to whether SSL is handled at the tornado level, since you could be terminating SSL in a proxy, but since your question stipulated that SSL was enabled in tornado let's focus on this case first). You could simply give the HTTP server a different Application, one that just does this redirect.
So, the best solution it's to HTTPServer that listens on port 80 and doesn't has the ssl_options parameter setted.
UPDATE
A request to https://example.com/some/path will go to port 443, where you must have an HTTPServer configured to handle https traffic; while a request to http://example.com/some/path will go to port 80, where you must have another instance of HTTPServer without ssl options, and this is where you must return the custom response code you want. That shouldn't raise any error.

How am I getting 'InternalError: table "dev~guestbook!!Entities" already exists' when I just created datastore?

I'm trying to figure out how to setup Test Driven Development for GAE.
I start the tests with:
nosetests -v --with-gae
I keep getting the error:
InternalError: table "dev~guestbook!!Entities" already exists
The datastore doesn't exist until I create it in the setUp(), but I'm still getting an error that the entities already exists?
I'm using the code from the GAE tutorial.
Here is my testing code in functional_tests.py:
import sys, os, subprocess, time, unittest, shlex
sys.path.append("/usr/local/google_appengine")
sys.path.append("/usr/local/google_appengine/lib/yaml/lib")
sys.path.append("/usr/local/google_appengine/lib/webapp2-2.5.2")
sys.path.append("/usr/local/google_appengine/lib/django-1.5")
sys.path.append("/usr/local/google_appengine/lib/cherrypy")
sys.path.append("/usr/local/google_appengine/lib/concurrent")
sys.path.append("/usr/local/google_appengine/lib/docker")
sys.path.append("/usr/local/google_appengine/lib/requests")
sys.path.append("/usr/local/google_appengine/lib/websocket")
sys.path.append("/usr/local/google_appengine/lib/fancy_urllib")
sys.path.append("/usr/local/google_appengine/lib/antlr3")
from selenium import webdriver
from google.appengine.api import memcache, apiproxy_stub, apiproxy_stub_map
from google.appengine.ext import db
from google.appengine.ext import testbed
from google.appengine.datastore import datastore_stub_util
from google.appengine.tools.devappserver2 import devappserver2
class NewVisitorTest(unittest.TestCase):
def setUp(self):
# Start the dev server
cmd = "/usr/local/bin/dev_appserver.py /Users/Bryan/work/GoogleAppEngine/guestbook/app.yaml --port 8080 --storage_path /tmp/datastore --clear_datastore --skip_sdk_update_check"
self.dev_appserver = subprocess.Popen(shlex.split(cmd),
stdout=subprocess.PIPE)
time.sleep(2) # Important, let dev_appserver start up
self.testbed = testbed.Testbed()
self.testbed.setup_env(app_id='dermal')
self.testbed.activate()
self.testbed.init_user_stub()
# Create a consistency policy that will simulate the High Replication consistency model.
# with a probability of 1, the datastore should be available.
self.policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=1)
# Initialize the datastore stub with this policy.
self.testbed.init_datastore_v3_stub(datastore_file="/tmp/datastore/datastore.db", use_sqlite=True, consistency_policy=self.policy)
self.testbed.init_memcache_stub()
self.datastore_stub = apiproxy_stub_map.apiproxy.GetStub('datastore_v3')
# setup the dev_appserver
APP_CONFIGS = ['app.yaml']
# setup client to make sure
from guestbook import Author, Greeting
if not ( Author.query( Author.email == "bryan#mail.com").get()):
logging.info("create Admin")
client = Author(
email = "bryan#mail.com",
).put()
Assert( Author.query( Author.email == "bryan#mail.com").get() )
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
def tearDown(self):
self.browser.quit()
self.testbed.deactivate()
self.dev_appserver.terminate()
def test_submit_anon_greeting(self):
self.browser.get('http://localhost:8080')
self.browser.find_element_by_name('content').send_keys('Anonymous test post')
self.browser.find_element_by_name('submit').submit()
Assert.assertEquals(driver.getPageSource().contains('Anonymous test post'))
Here is the traceback:
test_submit_anon_greeting (functional_tests.NewVisitorTest) ... INFO 2015-05-11 14:41:40,516 devappserver2.py:745] Skipping SDK update check.
INFO 2015-05-11 14:41:40,594 api_server.py:190] Starting API server at: http://localhost:59656
INFO 2015-05-11 14:41:40,598 dispatcher.py:192] Starting module "default" running at: http://localhost:8080
INFO 2015-05-11 14:41:40,600 admin_server.py:118] Starting admin server at: http://localhost:8000
WARNING 2015-05-11 14:41:45,008 tasklets.py:409] suspended generator _run_to_list(query.py:964) raised InternalError(table "dev~guestbook!!Entities" already exists)
ERROR 2015-05-11 14:41:45,009 webapp2.py:1552] table "dev~guestbook!!Entities" already exists
Traceback (most recent call last):
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/Users/Bryan/work/GoogleAppEngine/guestbook/guestbook.py", line 50, in get
greetings = greetings_query.fetch(10)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/utils.py", line 142, in positional_wrapper
return wrapped(*args, **kwds)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/query.py", line 1187, in fetch
return self.fetch_async(limit, **q_options).get_result()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/tasklets.py", line 325, in get_result
self.check_success()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/tasklets.py", line 368, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/query.py", line 964, in _run_to_list
batch = yield rpc
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/tasklets.py", line 454, in _on_rpc_completion
result = rpc.get_result()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
return self.__get_result_hook(self)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/datastore/datastore_query.py", line 2870, in __query_result_hook
self._batch_shared.conn.check_rpc_success(rpc)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/datastore/datastore_rpc.py", line 1342, in check_rpc_success
raise _ToDatastoreError(err)
InternalError: table "dev~guestbook!!Entities" already exists

It looks like there are a couple of things happening here.
First, it looks like you are using NoseGAE --with-gae. The plugin handles setting up and tearing down your testbed so you don't have to. This means that you do not need any of the self.testbed code and actually it can cause conflicts internally. Either switch to doing it the NoseGAE way, or don't use the --with-gae flag. If you stick with NoseGAE, it has an option --gae-datastore that lets you set the path to the datastore that it will use for your tests. Then inside your test class, set the property nosegae_datastore_v3 = True to have it set up for you:
class NewVisitorTest(unittest.TestCase):
# enable the datastore stub
nosegae_datastore_v3 = True
Second, the way that dev_appserver / sqlite work together, the appserver loads the sqlite db file into memory and works with it there. When the app server exits, it flushes the database contents back to disk. Since you are using the same datastore for your tests as the dev_appserver.py process you are opening for selenium, they may or may not see the fixture data you set up inside your test.
Here is an example from https://github.com/Trii/NoseGAE/blob/master/nosegae.py#L124-L140
class MyTest(unittest.TestCase):
nosegae_datastore_v3 = True
nosegae_datastore_v3_kwargs = {
'datastore_file': '/tmp/nosegae.sqlite3',
'use_sqlite': True
}
def test_something(self):
entity = MyModel(name='NoseGAE')
entity.put()
self.assertNotNone(entity.key.id())

I guess this line might be the faulty one:
self.testbed.init_datastore_v3_stub(datastore_file="/tmp/datastore/datastore.db", use_sqlite=True, consistency_policy=self.policy)
Setting datastore_file="/tmp/datastore/datastore.db" indicate you want to reuse this existing datastore in your tests
The python code documentation says:
The 'datastore_file' argument can be the path to an existing
datastore file, or None (default) to use an in-memory datastore
that is initially empty.
Personaly I use these in my tests:
def setUp(self):
self.testbed = testbed.Testbed()
self.testbed.activate()
self.testbed.init_datastore_v3_stub(
consistency_policy=datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=0)
)
self.testbed.init_memcache_stub()
def tearDown(self):
self.testbed.deactivate()