We Keep Coding

Random NoTransaction in Pyramid

I'm having trouble identifying the source of transaction.interfaces.NoTransaction errors within my Pyramid App. I don't see any patterns to when the error happens, so to me it's quite random.
This app is a (semi-) RESTful API and uses SQLAlchemy and MySQL. I'm currently running within a docker container that connects to an external (bare metal) MySQL instance on the same host OS.
Here's the stack trace for a login attempt within the App. This error happened right after another login attempt that was actually successful.
2020-06-15 03:57:18,982 DEBUG [txn.140501728405248:108][waitress-1] new transaction
2020-06-15 03:57:18,984 INFO [sqlalchemy.engine.base.Engine:730][waitress-1] BEGIN (implicit)
2020-06-15 03:57:18,984 DEBUG [txn.140501728405248:576][waitress-1] abort
2020-06-15 03:57:18,985 ERROR [waitress:357][waitress-1] Exception while serving /auth
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/waitress/channel.py", line 350, in service
task.service()
File "/usr/local/lib/python3.8/site-packages/waitress/task.py", line 171, in service
self.execute()
File "/usr/local/lib/python3.8/site-packages/waitress/task.py", line 441, in execute
app_iter = self.channel.server.application(environ, start_response)
File "/usr/local/lib/python3.8/site-packages/pyramid/router.py", line 270, in __call__
response = self.execution_policy(environ, self)
File "/usr/local/lib/python3.8/site-packages/pyramid_retry/__init__.py", line 127, in retry_policy
response = router.invoke_request(request)
File "/usr/local/lib/python3.8/site-packages/pyramid/router.py", line 249, in invoke_request
response = handle_request(request)
File "/usr/local/lib/python3.8/site-packages/pyramid_tm/__init__.py", line 178, in tm_tween
reraise(*exc_info)
File "/usr/local/lib/python3.8/site-packages/pyramid_tm/compat.py", line 36, in reraise
raise value
File "/usr/local/lib/python3.8/site-packages/pyramid_tm/__init__.py", line 135, in tm_tween
userid = request.authenticated_userid
File "/usr/local/lib/python3.8/site-packages/pyramid/security.py", line 381, in authenticated_userid
return policy.authenticated_userid(self)
File "/opt/REDACTED-api/REDACTED_api/auth/policy.py", line 208, in authenticated_userid
result = self._authenticate(request)
File "/opt/REDACTED-api/REDACTED_api/auth/policy.py", line 199, in _authenticate
session = self._get_session_from_token(token)
File "/opt/REDACTED-api/REDACTED_api/auth/policy.py", line 320, in _get_session_from_token
session = service.get(session_id)
File "/opt/REDACTED-api/REDACTED_api/service/__init__.py", line 122, in get
entity = self.queryset.filter(self.Meta.model.id == entity_id).first()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3375, in first
ret = list(self[0:1])
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3149, in __getitem__
return list(res)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3481, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3502, in _execute_and_instances
conn = self._get_bind_args(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3517, in _get_bind_args
return fn(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3496, in _connection_from_session
conn = self.session.connection(**kw)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1138, in connection
return self._connection_for_bind(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1146, in _connection_for_bind
return self.transaction._connection_for_bind(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 458, in _connection_for_bind
self.session.dispatch.after_begin(self.session, self, conn)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/event/attr.py", line 322, in __call__
fn(*args, **kw)
File "/usr/local/lib/python3.8/site-packages/zope/sqlalchemy/datamanager.py", line 268, in after_begin
join_transaction(
File "/usr/local/lib/python3.8/site-packages/zope/sqlalchemy/datamanager.py", line 233, in join_transaction
DataManager(
File "/usr/local/lib/python3.8/site-packages/zope/sqlalchemy/datamanager.py", line 89, in __init__
transaction_manager.get().join(self)
File "/usr/local/lib/python3.8/site-packages/transaction/_manager.py", line 91, in get
raise NoTransaction()
transaction.interfaces.NoTransaction
The trace shows that the execution eventually reaches my project, but only my custom authentication policy. And it fails right where the database should be queried for the user.
What intrigues me here is the third line on the stack trace. It seems Waitress somehow aborted the transaction it created? Any clue why?
EDIT: Here's the code where that happens: policy.py:320
def _get_session_from_token(self, token) -> UserSession:
try:
session_id, session_secret = self.parse_token(token)
except InvalidToken as e:
raise SessionNotFound(e)
service = AuthService(self.dbsession, None)
try:
session = service.get(session_id) # <---- Service Class called here
except NoResultsFound:
raise SessionNotFound("Invalid session found Request headers. "
"Session id: %s".format(session_id))
if not service.check_session(session, session_secret):
raise SessionNotFound("Session signature does not match")
now = datetime.now(tz=pytz.UTC)
if session.validity < now:
raise SessionNotFound(
"Current session ID {session_id} is expired".format(
session_id=session.id
)
)
return session
And here is an a view on the that service class method:
class AuthService(ModelService):
class Meta:
model = UserSession
queryset = Query(UserSession)
search_fields = []
order_fields = [UserSession.created_at.desc()]
# These below are from the generic ModelClass father class
def __init__(self, dbsession: Session, user_id: str):
self.user_id = user_id
self.dbsession = dbsession
self.Meta.queryset = self.Meta.queryset.with_session(dbsession)
self.logger = logging.getLogger("REDACTED")
#property
def queryset(self):
return self.Meta.queryset
def get(self, entity_id) -> Base:
entity = self.queryset.filter(self.Meta.model.id == entity_id).first()
if not entity:
raise NoResultsFound(f"Could not find requested ID {entity_id}")
As you can see, the there's already some exception treatment. I really don't see what other exception I could try to catch on AuthService.get

I found the solution to be much simpler than tinkering inside Pyramid or SQLAlchemy.
Debugging my Authentication Policy closely, I found out that my it was keeping a sticky reference for the dbsession. It was stored on the first request ever who used it, and never released.
The first request works as expected, the following one fails: My understanding is that the object is still in memory while the app is running, and after the initial transaction is closed. The second request has a new connection, and a new transaction, but the object in memory still points to the previous one, that when used ultimately causes this.
What I don't understand is why the exception didn't happen sometimes. As I mentioned initially, it was seemingly random.
Another thing that I struggled with was in writing a test case to expose the issue. On my tests, the issue never happens because I have (and I've never seen it done differently) a single connection and a single transaction throughout the entire testing session, as opposed of a new connection/transaction per request, so I have not found no way to actually reproduce.
Please let me know if that makes sense, and if you can shed a light on how to expose the bug on a test case.

Django: ModelChoiceField and django.setup()

If I try to run a test via PyCharm, I get this exception
...bin/python /usr/local/pycharm-4.5.1/helpers/pycharm/utrunner.py ...src/foo/foo/tests/FooEditTest.py::FooEditTest::test_issue_add true
Testing started at 15:59 ...
Traceback (most recent call last):
File "/usr/local/pycharm-4.5.1/helpers/pycharm/utrunner.py", line 139, in <module>
module = loadSource(a[0])
File "/usr/local/pycharm-4.5.1/helpers/pycharm/utrunner.py", line 41, in loadSource
module = imp.load_source(moduleName, fileName)
File "...src/foo/foo/tests/FooEditTest.py", line 45, in <module>
from foo.views.issue.forward import forward
File "...src/foo/foo/views/issue/forward.py", line 29, in <module>
class ForwardForm(forms.Form):
File "...src/foo/foo/views/issue/forward.py", line 36, in ForwardForm
group=forms.ModelChoiceField(Group.objects.exclude(groupconfig__no_issue=True).extra(
File "...python2.7/site-packages/django/db/models/manager.py", line 92, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "...python2.7/site-packages/django/db/models/query.py", line 698, in exclude
return self._filter_or_exclude(True, *args, **kwargs)
File "...python2.7/site-packages/django/db/models/query.py", line 707, in _filter_or_exclude
clone.query.add_q(~Q(*args, **kwargs))
File "...python2.7/site-packages/django/db/models/sql/query.py", line 1331, in add_q
clause, require_inner = self._add_q(where_part, self.used_aliases)
File "...python2.7/site-packages/django/db/models/sql/query.py", line 1358, in _add_q
current_negated=current_negated, connector=connector)
File "...python2.7/site-packages/django/db/models/sql/query.py", line 1182, in build_filter
lookups, parts, reffed_aggregate = self.solve_lookup_type(arg)
File "...python2.7/site-packages/django/db/models/sql/query.py", line 1120, in solve_lookup_type
_, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
File "...python2.7/site-packages/django/db/models/sql/query.py", line 1383, in names_to_path
field, model, direct, m2m = opts.get_field_by_name(name)
File "...python2.7/site-packages/django/db/models/options.py", line 416, in get_field_by_name
cache = self.init_name_map()
File "...python2.7/site-packages/django/db/models/options.py", line 445, in init_name_map
for f, model in self.get_all_related_m2m_objects_with_model():
File "...python2.7/site-packages/django/db/models/options.py", line 563, in get_all_related_m2m_objects_with_model
cache = self._fill_related_many_to_many_cache()
File "...python2.7/site-packages/django/db/models/options.py", line 577, in _fill_related_many_to_many_cache
for klass in self.apps.get_models():
File "...python2.7/site-packages/django/utils/lru_cache.py", line 101, in wrapper
result = user_function(*args, **kwds)
File "...python2.7/site-packages/django/apps/registry.py", line 168, in get_models
self.check_models_ready()
File "...python2.7/site-packages/django/apps/registry.py", line 131, in check_models_ready
raise AppRegistryNotReady("Models aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.
Process finished with exit code 1
I use group=forms.ModelChoiceField(Group.objects.exclude(...))
This line gets executed during importing. The call to django.setup() was not done before.
I have no clue how to solve this:
Should I call django.setup()? But where to insert this line?
Avoid using MyModel.objects.filter(...) during import time? This would need a big refactoring, since we have several ModelChoiceFields.

You're currently running the tests using the PyCharm's Python unit test runner pycharm-4.5.1/helpers/pycharm/utrunner.py.
This test runner is great for low-level unit tests (subclassed from unittest.TestCase) that don't touch Django features like the ORM, but
if the tests rely on Django-specific things like the database, then your TestCases probably need to be subclassed from django.test.testcases.TestCase and you'll need to go through PyCharm's Django test manager django_test_manage.py, which takes care of setting up the test database, initialising the model registry, and so on.
Run > Edit Configurations > Add New Configuration (+ button) > Django Tests
Set the target to foo.foo.tests.FooEditTest and make sure to put DJANGO_SETTINGS_MODULE=... in the environment variables if it doesn't find your settings.

I faced the same problem while running test cases and solved it by adding this to test file at the beginning with import statements
import os
import sys
from django.core.handlers.wsgi import WSGIHandler
os.environ['DJANGO_SETTINGS_MODULE'] = 'myapp.settings'
application = WSGIHandler()

Please show your exact code (especially the extra() clause). Calling filter() or exclude() at import time is not bad because querysets are lazy but you could evaluate the queryset here that caused the exception.
Do not evaluate querysets during import time because import statements are executed only once: e.g., if a new group is created, it won't be available as a choice for you ModelChoiceField.

A bit hard to say without seeing your code, but I had a similar issue once. For me it was related to a module that was being imported by my models and that also contained an import of my models.
ModelA --- imports --> service -- imports --> ModelA
I solved this by moving the import in my service to the method that needed the import. So instead of putting it at the top of the service module, I limited the scope of the import to the method that needed this import, thus avoiding importing the models whilst initializing the service module. Fwew, I wish I could draw it for you :)

How am I getting 'InternalError: table "dev~guestbook!!Entities" already exists' when I just created datastore?

I'm trying to figure out how to setup Test Driven Development for GAE.
I start the tests with:
nosetests -v --with-gae
I keep getting the error:
InternalError: table "dev~guestbook!!Entities" already exists
The datastore doesn't exist until I create it in the setUp(), but I'm still getting an error that the entities already exists?
I'm using the code from the GAE tutorial.
Here is my testing code in functional_tests.py:
import sys, os, subprocess, time, unittest, shlex
sys.path.append("/usr/local/google_appengine")
sys.path.append("/usr/local/google_appengine/lib/yaml/lib")
sys.path.append("/usr/local/google_appengine/lib/webapp2-2.5.2")
sys.path.append("/usr/local/google_appengine/lib/django-1.5")
sys.path.append("/usr/local/google_appengine/lib/cherrypy")
sys.path.append("/usr/local/google_appengine/lib/concurrent")
sys.path.append("/usr/local/google_appengine/lib/docker")
sys.path.append("/usr/local/google_appengine/lib/requests")
sys.path.append("/usr/local/google_appengine/lib/websocket")
sys.path.append("/usr/local/google_appengine/lib/fancy_urllib")
sys.path.append("/usr/local/google_appengine/lib/antlr3")
from selenium import webdriver
from google.appengine.api import memcache, apiproxy_stub, apiproxy_stub_map
from google.appengine.ext import db
from google.appengine.ext import testbed
from google.appengine.datastore import datastore_stub_util
from google.appengine.tools.devappserver2 import devappserver2
class NewVisitorTest(unittest.TestCase):
def setUp(self):
# Start the dev server
cmd = "/usr/local/bin/dev_appserver.py /Users/Bryan/work/GoogleAppEngine/guestbook/app.yaml --port 8080 --storage_path /tmp/datastore --clear_datastore --skip_sdk_update_check"
self.dev_appserver = subprocess.Popen(shlex.split(cmd),
stdout=subprocess.PIPE)
time.sleep(2) # Important, let dev_appserver start up
self.testbed = testbed.Testbed()
self.testbed.setup_env(app_id='dermal')
self.testbed.activate()
self.testbed.init_user_stub()
# Create a consistency policy that will simulate the High Replication consistency model.
# with a probability of 1, the datastore should be available.
self.policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=1)
# Initialize the datastore stub with this policy.
self.testbed.init_datastore_v3_stub(datastore_file="/tmp/datastore/datastore.db", use_sqlite=True, consistency_policy=self.policy)
self.testbed.init_memcache_stub()
self.datastore_stub = apiproxy_stub_map.apiproxy.GetStub('datastore_v3')
# setup the dev_appserver
APP_CONFIGS = ['app.yaml']
# setup client to make sure
from guestbook import Author, Greeting
if not ( Author.query( Author.email == "bryan#mail.com").get()):
logging.info("create Admin")
client = Author(
email = "bryan#mail.com",
).put()
Assert( Author.query( Author.email == "bryan#mail.com").get() )
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
def tearDown(self):
self.browser.quit()
self.testbed.deactivate()
self.dev_appserver.terminate()
def test_submit_anon_greeting(self):
self.browser.get('http://localhost:8080')
self.browser.find_element_by_name('content').send_keys('Anonymous test post')
self.browser.find_element_by_name('submit').submit()
Assert.assertEquals(driver.getPageSource().contains('Anonymous test post'))
Here is the traceback:
test_submit_anon_greeting (functional_tests.NewVisitorTest) ... INFO 2015-05-11 14:41:40,516 devappserver2.py:745] Skipping SDK update check.
INFO 2015-05-11 14:41:40,594 api_server.py:190] Starting API server at: http://localhost:59656
INFO 2015-05-11 14:41:40,598 dispatcher.py:192] Starting module "default" running at: http://localhost:8080
INFO 2015-05-11 14:41:40,600 admin_server.py:118] Starting admin server at: http://localhost:8000
WARNING 2015-05-11 14:41:45,008 tasklets.py:409] suspended generator _run_to_list(query.py:964) raised InternalError(table "dev~guestbook!!Entities" already exists)
ERROR 2015-05-11 14:41:45,009 webapp2.py:1552] table "dev~guestbook!!Entities" already exists
Traceback (most recent call last):
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/Users/Bryan/work/GoogleAppEngine/guestbook/guestbook.py", line 50, in get
greetings = greetings_query.fetch(10)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/utils.py", line 142, in positional_wrapper
return wrapped(*args, **kwds)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/query.py", line 1187, in fetch
return self.fetch_async(limit, **q_options).get_result()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/tasklets.py", line 325, in get_result
self.check_success()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/tasklets.py", line 368, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/query.py", line 964, in _run_to_list
batch = yield rpc
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/tasklets.py", line 454, in _on_rpc_completion
result = rpc.get_result()
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
return self.__get_result_hook(self)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/datastore/datastore_query.py", line 2870, in __query_result_hook
self._batch_shared.conn.check_rpc_success(rpc)
File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/datastore/datastore_rpc.py", line 1342, in check_rpc_success
raise _ToDatastoreError(err)
InternalError: table "dev~guestbook!!Entities" already exists

It looks like there are a couple of things happening here.
First, it looks like you are using NoseGAE --with-gae. The plugin handles setting up and tearing down your testbed so you don't have to. This means that you do not need any of the self.testbed code and actually it can cause conflicts internally. Either switch to doing it the NoseGAE way, or don't use the --with-gae flag. If you stick with NoseGAE, it has an option --gae-datastore that lets you set the path to the datastore that it will use for your tests. Then inside your test class, set the property nosegae_datastore_v3 = True to have it set up for you:
class NewVisitorTest(unittest.TestCase):
# enable the datastore stub
nosegae_datastore_v3 = True
Second, the way that dev_appserver / sqlite work together, the appserver loads the sqlite db file into memory and works with it there. When the app server exits, it flushes the database contents back to disk. Since you are using the same datastore for your tests as the dev_appserver.py process you are opening for selenium, they may or may not see the fixture data you set up inside your test.
Here is an example from https://github.com/Trii/NoseGAE/blob/master/nosegae.py#L124-L140
class MyTest(unittest.TestCase):
nosegae_datastore_v3 = True
nosegae_datastore_v3_kwargs = {
'datastore_file': '/tmp/nosegae.sqlite3',
'use_sqlite': True
}
def test_something(self):
entity = MyModel(name='NoseGAE')
entity.put()
self.assertNotNone(entity.key.id())

I guess this line might be the faulty one:
self.testbed.init_datastore_v3_stub(datastore_file="/tmp/datastore/datastore.db", use_sqlite=True, consistency_policy=self.policy)
Setting datastore_file="/tmp/datastore/datastore.db" indicate you want to reuse this existing datastore in your tests
The python code documentation says:
The 'datastore_file' argument can be the path to an existing
datastore file, or None (default) to use an in-memory datastore
that is initially empty.
Personaly I use these in my tests:
def setUp(self):
self.testbed = testbed.Testbed()
self.testbed.activate()
self.testbed.init_datastore_v3_stub(
consistency_policy=datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=0)
)
self.testbed.init_memcache_stub()
def tearDown(self):
self.testbed.deactivate()

How can I get more detail on why a fixture is not loading?

I have a TestCase that doesn't seem to load the fixtures.
I'm seeing this error as the test database is being built:
No fixtures found.
.............................................Problem installing fixture '/Users/Bryan/work/CNPROG/forum/fixtures/forum_fixtures.json': Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/commands/loaddata.py", line 169, in handle
obj.save(using=using)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/models/base.py", line 543, in save_base
created=(not record_exists), raw=raw)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/dispatch/dispatcher.py", line 162, in send
response = receiver(signal=self, sender=sender, **named)
File "/Users/Bryan/work/CNPROG/forum/models.py", line 656, in record_ask_event
activity = Activity(user=instance.author, active_at=instance.added_at, content_object=instance, activity_type=TYPE_ACTIVITY_ASK_QUESTION)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/models/fields/related.py", line 302, in __get__
rel_obj = QuerySet(self.field.rel.to).using(db).get(**params)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/models/query.py", line 341, in get
% self.model._meta.object_name)
DoesNotExist: User matching query does not exist.
class UserProfileTestCases(TestCase):
"""These are tests to verify that refactoring of UserProfile is correct"""
fixtures = ['forum_fixtures.json'] # apparently this needs to be in fixtures/ directory.
def setUp(self):
self.client = Client()
if self.client.login(username='DickCheney', password='test'):
print "client.login DickCheney successful";
else:
print "client.login FAILED"
For some reason the fixtures are not loading.
The fixture is located at:
forum/fixtures/forum_fixtures.json
How can I output the reason that the fixture is not loading?
The Traceback suggests something is happening here:
File "/Users/Bryan/work/CNPROG/forum/models.py", line 656, in record_ask_event
But I can't imagine why that would affect the loading of the fixtures.
When I looked at the code, record_ask_events is called via post_save event.
I was able to successfully manage.py loaddata forum_fixtures so I believe I set them up correctly.

Run tests with greater detail with this command:
python manage.py test --verbosity=2
I fixed this by rearranging the order of the fixtures.
If you have Related objects, you need to make sure the objects that have the most associations ( e.g. auth_user ) is listed first in the fixture because the related objects may be called during a save.
Django v1.2 only looked for fixtures named 'initial_data'. Not once was there are search for 'forum_fixtures.json'
I changed the fixture name to 'initial_data.json' and it now works.

I'm having this exact problem. My app is included in my INSTALLED_APPS tuple in my settings file. So according to django docs, it should search the fixtures directory under the app directory for any filenames I place in the 'fixtures' attribute of my TestCase class. Doesn't work. Don't want to name my fixture 'initial_data.json' because I only want it to be used when I run unit tests.

You'll need to change your post_save signal handler to respect the 'raw' kwarg. Your signal handler would not cause errors during fixture loading if you did something like this with it:
#receiver(post_save, sender=MyModel, dispatch_uid='MyModelUID')
def handler(sender, *args, **kwargs):
if kwargs.get('raw'):
return
else:
...stuff

Dowser: "bad argument to internal function"

I'm trying out Dowser, it's really cool but I'm stuck with a little problem, and I couldn't find anything helpful on Google, so here I am.. ^^;
I'm running a CherryPy+SQLAlchemy application.. It works normally, except that when I enable Dowser (that is, after I call dowser.Root()), now and then I get exceptions like:
SystemError:
../Objects/tupleobject.c:809: bad
argument to internal function
on innocent-looking istructions, such as accessing a SQLA-mapped field.. Relevant part of a traceback:
File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/attributes.py", line 158, in __get__
return self.impl.get(instance_state(instance), instance_dict(instance))
File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/attributes.py", line 377, in get
value = callable_()
File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/strategies.py", line 586, in __call__
result = q.all()
File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/query.py", line 1267, in all
return list(self)
File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/query.py", line 1422, in instances
rows = [process[0](context, row) for row in fetch]
File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/query.py", line 2032, in main
return _instance(row, None)
File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/mapper.py", line 1653, in _instance
identitykey = identity_key(row)
File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/mapper.py", line 1594, in identity_key
return (identity_class, tuple(row[column] for column in pk_cols))
Could this be related to the Dowser thread, accessing the garbage collector? Any hint on what I could check out?
I'm running Python 2.6.2 on Xubuntu Jaunty.
Thanks for your attention!