I have two tables, testInstance and bugzilla that are associated by a third one, bzCheck, like this:
class Instance(Base):
__tablename__ = "testInstance"
id = Column(Integer, primary_key=True)
bz_checks = relation(BZCheck, backref="instance")
class BZCheck(Base):
__tablename__ = "bzCheck"
instance_id = Column(Integer, ForeignKey("testInstance.id"), primary_key=True)
bz_id = Column(Integer, ForeignKey("bugzilla.id"), primary_key=True)
status = Column(String, nullable=False)
bug = relation(Bugzilla, backref="checks")
class Bugzilla(Base):
__tablename__ = "bugzilla"
id = Column(Integer, primary_key=True)
The backend is a postgresql server ; I'm using SQLalchemy 0.5
If I create the Instance, Bugzilla and BZCheck ojects, then do
bzcheck.bug = bugzilla
instance.bz_checks.append(bzcheck)
and then add and commit them ; everything is fine.
But now, let's assume I have an existing instance and an existing bugzilla and want to associate them:
instance = session.query(Instance).filter(Instance.id == 31).one()
bugzilla = session.query(Bugzilla).filter(Bugzilla.id == 19876).one()
check = BZCheck(status="OK")
check.bug = bugzilla
instance.bz_checks.append(check)
It fails:
In [6]: instance.bz_checks.append(check)
2012-01-09 18:43:50,713 INFO sqlalchemy.engine.base.Engine.0x...3bd0 select nextval('"bzCheck_instance_id_seq"')
2012-01-09 18:43:50,713 INFO sqlalchemy.engine.base.Engine.0x...3bd0 None
2012-01-09 18:43:50,713 INFO sqlalchemy.engine.base.Engine.0x...3bd0 ROLLBACK
It tries to get a new ID from an unexisting sequence instead of using the foreign key "testInstance.id"... I don't understand why.
I have had similar problems when trying to modify objects after commiting them ; I should have missed something fundamental but what ?
the part you're missing here is the stack trace. Always look at the stack trace - what is critical here is that it's autoflush, produced by the access of instance.bz_checks:
Traceback (most recent call last):
File "test.py", line 44, in <module>
instance.bz_checks.append(check)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/attributes.py", line 168, in __get__
return self.impl.get(instance_state(instance),dict_)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/attributes.py", line 453, in get
value = self.callable_(state, passive)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/strategies.py", line 563, in _load_for_state
result = q.all()
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/query.py", line 1983, in all
return list(self)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/query.py", line 2092, in __iter__
self.session._autoflush()
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/session.py", line 973, in _autoflush
self.flush()
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/session.py", line 1547, in flush
self._flush(objects)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/session.py", line 1616, in _flush
flush_context.execute()
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/unitofwork.py", line 328, in execute
rec.execute(self)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/unitofwork.py", line 472, in execute
uow
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/mapper.py", line 2291, in _save_obj
execute(statement, params)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 1405, in execute
params)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 1538, in _execute_clauseelement
compiled_sql, distilled_params
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 1646, in _execute_context
context)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 1639, in _execute_context
context)
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/engine/default.py", line 330, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (IntegrityError) null value in column "instance_id" violates not-null constraint
'INSERT INTO "bzCheck" (bz_id, status) VALUES (%(bz_id)s, %(status)s) RETURNING "bzCheck".instance_id' {'status': 'OK', 'bz_id': 19876}
you can see this because the line of code is:
instance.bz_checks.append(check)
then autoflush:
self.session._autoflush()
File "/Users/classic/dev/sqlalchemy/lib/sqlalchemy/orm/session.py", line 973, in _autoflush
Three solutions:
a. temporarily disable autoflush (see http://www.sqlalchemy.org/trac/wiki/UsageRecipes/DisableAutoflush)
b. ensure that the BZCheck association object is always created with it's full state needed before accessing any collections:
BZState(bug=bugzilla, instance=instance)
(this is usually a good idea for association objects - they represent the association between two points so it's most appropriate that they be instantiated with this state)
c. change the cascade rules so that the operation of check.bug = somebug doesn't actually place check into the session just yet. You can do this with cascade_backrefs, described at http://www.sqlalchemy.org/docs/orm/session.html#controlling-cascade-on-backrefs. (but you'd need to be on 0.6 or 0.7)
Related
I'm trying to create an Order model using SQLAlchemy with a Column having an Array of ProductItem Class defined, but it is throwing an exception
Both the classes are defined in the same file
models.py
# Product Item Class
class ProductItem(Base):
__tablename__ = 'product_items'
id = Column(Integer, primary_key=True, index=True)
productName = Column(String,index=True,nullable=False)
productBrand = Column(String, nullable=False)
def __repr__(self):
return "<ProductList(productName = '%s', productCategoryName = '%s')>" % (
self.productName,
self.productCategoryName
)
#Order Class
class Order(Base):
__tablename__ = 'orders'
id= Column(Integer, index=True, primary_key=True)
dateCreated = Column(DateTime,nullable=False)
dateDelivered = Column(DateTime, nullable=True)
orderItem = Column(ARRAY(ProductItem), nullable=False) # This Column
orderStatus = Column(String, nullable=False)
Exception:
>alembic revision --autogenerate -m "Initial db with order"
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.autogenerate.compare] Detected added table 'orders'
INFO [alembic.autogenerate.compare] Detected added index 'ix_orders_businessId' on '['businessId']'
INFO [alembic.autogenerate.compare] Detected added index 'ix_orders_customerId' on '['customerId']'
INFO [alembic.autogenerate.compare] Detected added index 'ix_orders_id' on '['id']'
INFO [alembic.ddl.postgresql] Detected sequence named 'product_categories_id_seq' as owned by integer column 'product_categories(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'business_billing_id_seq' as owned by integer column 'business_billing(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'business_timings_id_seq' as owned by integer column 'business_timings(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'product_items_id_seq' as owned by integer column 'product_items(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'inventory_id_seq' as owned by integer column 'inventory(id)', assuming SERIAL and omitting
Generating D:\app Backend\db-migration\versions\ceff7117b4cb_initial_db_without_order.py ... done
Traceback (most recent call last):
File "c:\users\mahesh\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\mahesh\appdata\local\programs\python\python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "D:\app Backend\env\Scripts\alembic.exe\__main__.py", line 7, in <module>
File "d:\app backend\env\lib\site-packages\alembic\config.py", line 559, in main
CommandLine(prog=prog).main(argv=argv)
File "d:\app backend\env\lib\site-packages\alembic\config.py", line 553, in main
self.run_cmd(cfg, options)
File "d:\app backend\env\lib\site-packages\alembic\config.py", line 530, in run_cmd
fn(
File "d:\app backend\env\lib\site-packages\alembic\command.py", line 219, in revision
scripts = [script for script in revision_context.generate_scripts()]
File "d:\app backend\env\lib\site-packages\alembic\command.py", line 219, in <listcomp>
scripts = [script for script in revision_context.generate_scripts()]
File "d:\app backend\env\lib\site-packages\alembic\autogenerate\api.py", line 533, in generate_scripts
yield self._to_script(generated_revision)
File "d:\app backend\env\lib\site-packages\alembic\autogenerate\api.py", line 449, in _to_script
return self.script_directory.generate_revision(
File "d:\app backend\env\lib\site-packages\alembic\script\base.py", line 658, in generate_revision
script = Script._from_path(self, path)
File "d:\app backend\env\lib\site-packages\alembic\script\base.py", line 851, in _from_path
return cls._from_filename(scriptdir, dir_, filename)
File "d:\app backend\env\lib\site-packages\alembic\script\base.py", line 904, in _from_filename
module = util.load_python_file(dir_, filename)
File "d:\app backend\env\lib\site-packages\alembic\util\pyfiles.py", line 97, in load_python_file
module = load_module_py(module_id, path)
File "d:\app backend\env\lib\site-packages\alembic\util\compat.py", line 182, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 786, in exec_module
File "<frozen importlib._bootstrap_external>", line 923, in get_code
File "<frozen importlib._bootstrap_external>", line 853, in source_to_code
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "D:\App Backend\db-migration\versions\ceff7117b4cb_initial_db_without_order.py", line 25
sa.Column('orderItem', sa.ARRAY(app.models.<app.models.ProductItem object at 0x00000256DF9ED100>), nullable=False),
^
SyntaxError: invalid syntax
Searched a lot but couldn't find any docs describing custom types for the array. I need a way to store an array of ProductItem class in Order Class. Any help would be appreciated. Thank you!
From the PostgreSQL: documentation:
PostgreSQL allows columns of a table to be defined as variable-length multidimensional arrays. Arrays of any built-in or user-defined base type, enum type, composite type, range type, or domain can be created.
Hence, you will not be able to create an ARRAY column of just any Python class, unless you create a corresponding user-defined type in the database, which is most likely against the reason of how one should use SQLAlchemy.
What you need is a way to create a Many To Many relationship which requires an association table.
See also the answer to this question, a similar question although it focused on enforcing ForeignKey constraint from the ARRAY datatype.
I have Postgres DB with a table of pending operations. One column in the operation in an enum with the status of the enum. I used the standard python (2.7) enum, with AutoNumber (myenum.py):
class AutoNumber(enum.Enum):
def __new__(cls):
value = len(cls.__members__) + 1
obj = object.__new__(cls)
obj._value_ = value
return obj
class MyStatus(AutoNumber):
INITIAL = ()
ACCEPTED = ()
DENIED = ()
ACK_PENDING = ()
AUTHORIZED = ()
ACTIVE = ()
END = ()
DELETED = ()
# end enum
The table looks like (also in myenum.py):
Base = declarative_base()
class MyOperation(Base):
__tablename__ = 'operations'
id = Column( Integer, primary_key=True )
status = Column( Enum(MyStatus) )
status_message = Column( String )
status_time = Column( DateTime )
def __repr__(self):
return "<MyOperation(%s, %s, %s, %s)>" % \
( self.id, self.status, self.status_time, self.status_message )
# end class
Generally this works fine. In the SAME FILE that defines MyStatus (myoper.py), I can change the status and save it back to the DB and it works fine:
def checkOper( oper ):
oper.status = MyStatus.DENIED
oper.status_message = "failed check (internal)"
oper.status_time = datetime.datetime.utcnow()
Here's how I call it (within myoper.py)
checkOper( oper )
session.add(oper)
session.commit()
This is all in the same file (myoper.py).
However, if I pass an oper object to an external function, and IT changes the status, then I get a sqlalchemy.exc.StatementError.
Here's the external function (myoper_test.py):
import datetime
from myoper import MyStatus
def extCheckOper( oper ):
oper.status = MyStatus.DENIED
oper.status_message = "failed check (external)"
oper.status_time = datetime.datetime.utcnow()
Here's how I call it (from myoper.py):
from myoper_test import extCheckOper
extCheckOper( oper )
session.add(oper)
session.commit()
Here's the stack trace:
Traceback (most recent call last):
File "./myoper.py", line 120, in <module>
session.commit()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 906, in commit
self.transaction.commit()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 461, in commit
self._prepare_impl()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2177, in flush
self._flush(objects)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2297, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2261, in _flush
flush_context.execute()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
rec.execute(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
uow
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj
mapper, table, update)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 737, in _emit_update_statements
execute(statement, multiparams)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1121, in _execute_context
None, None)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1116, in _execute_context
context = constructor(dialect, self, conn, *args)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 639, in _init_compiled
for key in compiled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 639, in <genexpr>
for key in compiled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/sqltypes.py", line 1446, in process
value = self._db_value_for_elem(value)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/sqltypes.py", line 1354, in _db_value_for_elem
'"%s" is not among the defined enum values' % elem)
sqlalchemy.exc.StatementError: (exceptions.LookupError) "MyStatus.DENIED" is not among the defined enum values [SQL: u'UPDATE operations SET status=%(status)s, status_message=%(status_message)s, status_time=%(status_time)s WHERE operations.id = %(operations_id)s'] [parameters: [{'status': <MyStatus.DENIED: 6>, 'status_time': datetime.datetime(2017, 10, 18, 20, 22, 44, 350035), 'status_message': 'failed check (external)', 'operations_id': 3}]]
I've tried inspecting the type both in the internal file, and external file, but it's ways listed as <enum 'MyStatus'>.
I have found, that if I assign the oper.status to the enum .name, then that DOES work:
def extCheckOper( oper ):
oper.status = MyStatus.AUTHORIZED.name
oper.status_message = "authorized check (external)"
oper.status_time = datetime.datetime.utcnow()
But that's obviously pretty ugly.
So - what am I doing wrong? What is different about MyStatus in the file it's defined in, vs an external file that screws up SQL Alchemy?
I posted this question to the SQL Alchemy mailing list and got an answer. Link to thread
Turns out this one of those "gotcha's" about python and has nothing really to do with SQL Alchemy. Here's a reference: Executing Main Module Twice.
In this particular case, when I executed my script, the MyStatus was created with a particular id (python handle on the type). But when myoper_test imported MyStatus from myoper, it was created AGAIN with a different id.
So when the extCheckOper assigned a MyStatus value to the status field, it was a different MyStatus than SQL Alchemy created the DB mapping with, so when SQL Alchemy tried to save it to the DB, the " is " operator failed, since the (external) MyStatus was different than (original) MyStatus.
There are a couple of different workarounds. One way is to not run the code as main (after moving exiting main code into main() function):
$ python -c "from myoper import main; import sys; main(*sys.argv[1:])" ext_check 1
The better solution is to avoid this problem entirely - move the code that calls out externally to an internal test script. The code in main stays within mainly within the main script (sorry, couldn't resist... :-) ).
based on my model:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
Base = declarative_base()
class Session(Base):
__tablename__ = 'sessions'
id = Column(Integer, primary_key=True)
token = Column(String(200))
user_id = Column(Integer, ForeignKey('app_users.id'))
user = relationship('model.user.User', back_populates='sessions')
I want to instantiate a new session through:
session = Session(token='test-token-123')
But i get:
AttributeError: mapper
The full stacktrace:
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.5/site-packages/falcon/api.py", line 227, in __call__
responder(req, resp, **params)
File "./app_user/register.py", line 13, in on_post
session = Session(token='test-token-123')
File "<string>", line 2, in __init__
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/instrumentation.py", line 347, in _new_state_if_none
state = self._state_constructor(instance, self)
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 764, in __get__
obj.__dict__[self.__name__] = result = self.fget(obj)
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/instrumentation.py", line 177, in _state_constructor
self.dispatch.first_init(self, self.class_)
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/event/attr.py", line 256, in __call__
fn(*args, **kw)
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/mapper.py", line 2976, in _event_on_first_init
configure_mappers()
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/mapper.py", line 2872, in configure_mappers
mapper._post_configure_properties()
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/mapper.py", line 1765, in _post_configure_properties
prop.init()
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/interfaces.py", line 184, in init
self.do_init()
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/relationships.py", line 1653, in do_init
self._process_dependent_arguments()
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/relationships.py", line 1710, in _process_dependent_arguments
self.target = self.mapper.mapped_table
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 850, in __getattr__
return self._fallback_getattr(key)
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 828, in _fallback_getattr
raise AttributeError(key)
I have no idea where this error is coming from and i can not really debug it.. anybody could help me with this issue?
Thanks and Greetings!
Looking at the traceback you can see these lines:
...
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/relationships.py", line 1653, in do_init
self._process_dependent_arguments()
File "/home/ubuntu/.local/lib/python3.5/site-packages/sqlalchemy/orm/relationships.py", line 1710, in _process_dependent_arguments
self.target = self.mapper.mapped_table
...
which narrow your problem down quite a bit. The relationship
user = relationship('model.user.User', back_populates='sessions')
uses a Python evaluable string as the argument, the use of which is further explained in "Configuring Relationships":
Relationships to other classes are done in the usual way, with the added feature that the class specified to relationship() may be a string name. The “class registry” associated with Base is used at mapper compilation time to resolve the name into the actual class object, which is expected to have been defined once the mapper configuration is used
If you've not imported models.user module anywhere before you try to instantiate a Session object for the first time, then the name resolving fails because the class User has not been created yet and does not exist in the registry. In other words for the name resolving to work, all classes must have been defined, which means that their bodies must have been executed.
And if you actually have imported the models.user module, check your other models and that their related model classes have been defined. Using your models for the first time triggers mapper compilation/configuration, so the source of the error could be other models as well.
I'm currently learning SQLAlchemy, and i found this strange thing. I was experimenting with a table which stores a person's name and address, and to get them i use this:
session.query(User)
And to get the first item, i tried:
session.query(User).first()
Which throws a DatabaseError:
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
session.query(User).first()
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2275, in first
ret = list(self[0:1])
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2142, in __getitem__
return list(res)
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2346, in __iter__
return self._execute_and_instances(context)
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2361, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 664, in execute
return meth(self, multiparams, params)
File "build\bdist.win32\egg\sqlalchemy\sql\elements.py", line 272, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 761, in _execute_clauseelement
compiled_sql, distilled_params
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 874, in _execute_context
context)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 1023, in _handle_dbapi_exception
exc_info
File "build\bdist.win32\egg\sqlalchemy\util\compat.py", line 185, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 867, in _execute_context
context)
File "build\bdist.win32\egg\sqlalchemy\engine\default.py", line 376, in do_execute
cursor.execute(statement, parameters)
DatabaseError: (DatabaseError) ORA-01036: illegal variable name/number
'SELECT test_user_uid, test_user_name, test_user_address \nFROM (SELECT test_user."uid" AS test_user_uid, test_user.name AS test_user_name, test_user.address AS test_user_address \nFROM test_user) \nWHERE ROWNUM <= :ROWNUM_1' {'ROWNUM_1': 1}
However, i was able to retrieve what i wanted if i select all the rows, and loop through the query object:
users = [user for user in session.query(User)]
user1 = users[0]
That's all, i thought it's strange. Here's my mapping class:
class User(Base):
__tablename__ = 'test_user'
uid = Column(Integer, primary_key = True)
name = Column(String(50))
address = Column(String(100))
def __repr__(self):
return "<User (%s, %s)"%(self.name, self.address)
My best guess is that Session.query().first() is looking for the first row, with the generated query. However, the working method retrieves all the rows, and select the first one in Python. The problem is clearly from the generated query (invalid query). The main question is, what caused SQLAlchemy to create an invalid query?
Also, i noticed that SQLAlchemy makes things more difficult by making a query with sub-query. Is that behavior intended?
I hope i can get a satisfying answer, thanks!
Well, it didn't take me long to realize this. It turns out that it's a version problem, i was previously using cx_Oracle version 5.0.2 10g, i tried to upgrade it to version 5.1.2 10g, and things works fine.
This is probably an undocumented bug in SQLAlchemy, i can't find a place where they mention it.
Conclusion: If you want to use the latest version of SQLAlchemy (0.9.0b1) with Oracle 10g, you shouldn't use cx_Oracle older than version 5.1.2 10g.
Hope this helps, and thanks for reading the question!
views.py
def add_post(topic, request):
post_form = PostForm(request.POST)
if 'submit' in request.POST and post_form.validate():
post = Post(body=post_form.body.data)
post.user = request.user
post.topic = topic
DBSession.add(post)
request.session.flash(_('Post was added'))
transaction.commit()
raise HTTPFound(location=request.route_url('topic',id=topic.id))
return {'post_form':post_form}
models.py
class Topic(Base):
__tablename__ = 'topics'
id = Column(Integer, primary_key=True)
...
post_count = Column(Integer, default=0)
posts = relationship('Post', primaryjoin="Post.topic_id==Topic.id", backref='topic', lazy='dynamic')
class Post(Base):
__tablename__ = 'posts'
id = Column(Integer, primary_key=True)
...
topic_id = Column(Integer, ForeignKey('topics.id'))
def post_inserted(mapper, conn, post):
topic = post.topic
topic.post_count = topic.posts.count()
event.listen(Post, "after_insert", post_inserted)
I want in my Pyramid app to use SQLAchemy event 'after_insert', to update Topic model with number of posts belong to it. But I get exeption:
Traceback (most recent call last):
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/waitress/channel.py", line 329, in service
task.service()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/waitress/task.py", line 173, in service
self.execute()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/waitress/task.py", line 380, in execute
app_iter = self.channel.server.application(env, start_response)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/pyramid/router.py", line 251, in __call__
response = self.invoke_subrequest(request, use_tweens=True)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/pyramid/router.py", line 227, in invoke_subrequest
response = handle_request(request)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/pyramid_tm/__init__.py", line 107, in tm_tween
return response
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/transaction/_manager.py", line 116, in __exit__
self.commit()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/transaction/_manager.py", line 107, in commit
return self.get().commit()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/transaction/_transaction.py", line 354, in commit
reraise(t, v, tb)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/transaction/_transaction.py", line 345, in commit
self._commitResources()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/transaction/_transaction.py", line 493, in _commitResources
reraise(t, v, tb)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/transaction/_transaction.py", line 465, in _commitResources
rm.tpc_begin(self)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/zope/sqlalchemy/datamanager.py", line 86, in tpc_begin
self.session.flush()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 1583, in flush
self._flush(objects)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 1654, in _flush
flush_context.execute()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/unitofwork.py", line 331, in execute
rec.execute(self)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/unitofwork.py", line 475, in execute
uow
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/persistence.py", line 67, in save_obj
states_to_insert, states_to_update)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/persistence.py", line 702, in _finalize_insert_update_commands
mapper.dispatch.after_insert(mapper, connection, state)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/event.py", line 291, in __call__
fn(*args, **kw)
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/events.py", line 360, in wrap
wrapped_fn(*arg, **kw)
File "/home/user/workspace/myforum/cube_forum/models.py", line 165, in post_saved
topic.post_count = topic.posts.count()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/dynamic.py", line 249, in count
sess = self.__session()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/dynamic.py", line 219, in __session
sess.flush()
File "/home/user/workspace/myforum/env/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 1577, in flush
raise sa_exc.InvalidRequestError("Session is already flushing")
InvalidRequestError: Session is already flushing
How to do it right in Pyramid/SQLalchemy?
EDIT: Question is actually how to use SQLAlchemy's events in Pyramid.
SQLAlchemy event "after_insert" not suitable for this task.
The answer is using Pyramid's custom events as described here http://dannynavarro.net/2011/06/12/using-custom-events-in-pyramid/
I see that there is a problem on the database level - there no guarantee to get valid posts count value. Imagine if two or more posts were added to topic at the same time within multiple transactions - there is no way to tell for sure valid count of associated posts unless you will lock topic table or record every time you are adding new post.
Possible solutions:
1) Add method to Topic class. Always valid number, but it will execute query into db every time.
class Topic(Base):
...
def posts_count(self):
return self.posts.count()
2) Add post to topic and update post_count in your code without using after_insert event. You'll need to update topic.post_count every time when you change topic for post. Also this method won't resolve issue I described when several posts were added to topic at the same time.
def add_post(topic, request):
post_form = PostForm(request.POST)
if 'submit' in request.POST and post_form.validate():
post = Post(body=post_form.body.data)
post.user = request.user
topic.posts.append(post)
topic.post_count = topic.posts.count()
DBSession.add(post)
DBSession.add(topic)
request.session.flash(_('Post was added'))
transaction.commit()
raise HTTPFound(location=request.route_url('topic',id=topic.id))
return {'post_form':post_form}