I'm using Doubles to track time in my database (as Unix timestamps with fractional portions). For that, I've created the following user type (that takes pytz-aware datetimes from the application and converts them to UTC timestamps).
from sqlalchemy.types import TypeDecorator, Float
class DoubleDatetime(TypeDecorator):
impl = Float
def __init__(self):
TypeDecorator.__init__(self, as_decimal=False)
def process_bind_param(self, value, dialect):
return value.timestamp()
def process_result_value(self, value, dialect):
return datetime.datetime.utcfromtimestamp(value)
This works fine for all datetimes in my system. I'm using the following column definition to have the database (in my case, SQLite3) insert the transaction time:
from sqlalchemy import Column
from sqlalchemy.sql.functions import now
column = Column("transaction_begin", DoubleDatetime, server_default=now())
Despite defining the transaction_begin column as a DoubleDatetime, my database still stores the records with a string.
How can I force the server transaction timestamp to use fractional Unix epochs?
What have I tried?
Reading the SQLite3 docs, I know I need to do some Julian Day trickery to do this.
SELECT (julianday('now') - 2440587.5) * 86400.0;
But I can't find julianday in sqlalchemy.sql.functions, to wrap it in a zero-argument lambda (to pass as the server_default arg to Column).
I tried the following, but it doesn't seem to work.
from sqlalchemy import text
server_default = text("SELECT (julianday('now') - 2440587.5) * 86400.0;")
Nor does the following:
from sqlalchemy.sql import select, text
server_default = select([text("(julianday('now') - 2440587.5) * 86400.0;")])
Traceback:
Traceback (most recent call last):
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: near "SELECT": syntax error
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "tasks.py", line 81, in <module>
metadata.create_all(engine)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 4005, in create_all
tables=tables)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1940, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1549, in _run_visitor
**kwargs).traverse_single(element)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
return meth(obj, **kw)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
_is_metadata_operation=True)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
return meth(obj, **kw)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 791, in visit_table
include_foreign_key_constraints=include_foreign_key_constraints
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
compiled
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
raise value.with_traceback(tb)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "SELECT": syntax error [SQL: "\nCREATE TABLE barchartbarmessage (\n\tsymbol VARCHAR NOT NULL, \n\tdate DATE NOT NULL, \n\topen NUMERIC, \n\thigh NUMERIC, \n\tlow NUMERIC, \n\tclose NUMERIC, \n\tvolume INTEGER, \n\tprevious_day_open_interest INTEGER, \n\tprevious_day_volume INTEGER, \n\tmessage_time FLOAT, \n\ttransaction_begin FLOAT DEFAULT SELECT (julianday('now') - 2440587.5) * 86400.0; NOT NULL, \n\ttransaction_end FLOAT, \n\tPRIMARY KEY (symbol, date, transaction_begin)\n)\n\n"] (Background on this error at: http://sqlalche.me/e/e3q8)
I had this same question, and found that this worked. The expression here is specific to PostgreSQL. I originally was calculating this value on my application server, which led to unexpected results. See this post for more information. Note that in PostgreSQL, at least, when setting defaults using expressions you don't include SELECT at the start.
import sqlalchemy as sa
logged_ts_ms = sa.Column(sa.BigInteger, server_default=sa.text("""(extract(epoch from now()) * 1000)::bigint;"""))
Related
i got work to fix error when computing but i still dont have idea how to fix it because i'm still newbie
Odoo Server Error
Traceback (most recent call last): File
"/home/equipAccounting/equip/odoo/addons/base/models/ir_http.py", line
237, in _dispatch
result = request.dispatch() File "/home/equipAccounting/equip/odoo/http.py", line 683, in dispatch
result = self._call_function(**self.params) File "/home/equipAccounting/equip/odoo/http.py", line 359, in
_call_function
return checked_call(self.db, args, *kwargs) File "/home/equipAccounting/equip/odoo/service/model.py", line 94, in
wrapper
return f(dbname, args, *kwargs) File "/home/equipAccounting/equip/odoo/http.py", line 347, in checked_call
result = self.endpoint(*a, **kw) File "/home/equipAccounting/equip/odoo/http.py", line 912, in call
return self.method(*args, **kw) File "/home/equipAccounting/equip/odoo/http.py", line 531, in response_wrap
response = f(*args, **kw) File "/home/equipAccounting/equip/addons/basic/web/controllers/main.py",
line 1393, in call_button
action = self._call_kw(model, method, args, kwargs) File "/home/equipAccounting/equip/addons/basic/web/controllers/main.py",
line 1381, in _call_kw
return call_kw(request.env[model], method, args, kwargs) File "/home/equipAccounting/equip/odoo/api.py", line 396, in call_kw
result = _call_kw_multi(method, model, args, kwargs) File "/home/equipAccounting/equip/odoo/api.py", line 383, in _call_kw_multi
result = method(recs, args, *kwargs) File "/home/equipAccounting/equip/addons/core/treasury_forecast/models/treasury_bank_forecast.py",
line 290, in compute_bank_balances
self.env.cr.execute(main_query) File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in
fun
return caller(func, (extras + args), *kw) File "/home/equipAccounting/equip/odoo/sql_db.py", line 101, in check
return f(self, args, *kwargs) File "/home/equipAccounting/equip/odoo/sql_db.py", line 298, in execute
res = self._obj.execute(query, params) Exception
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File
"/home/equipAccounting/equip/odoo/http.py", line 639, in
_handle_exception
return super(JsonRequest, self)._handle_exception(exception) File "/home/equipAccounting/equip/odoo/http.py", line 315, in
_handle_exception
raise exception.with_traceback(None) from new_cause psycopg2.errors.SyntaxError: syntax error at or near ")" LINE 9:
WHERE abs.journal_id IN ()
and here is the code :
def get_bank_fc_query(self, fc_journal_list, date_start, date_end,company_domain):
query = """
UNION
SELECT CAST('FBK' AS text) AS type, absl.id AS ID, am.date, absl.payment_ref as name, am.company_id, absl.amount_main_currency as amount, absl.cf_forecast, abs.journal_id, NULL as kind FROM account_bank_statement_line absl
LEFT JOIN account_move am ON (absl.move_id = am.id)
LEFT JOIN account_bank_statement abs ON (absl.statement_id = abs.id)
WHERE abs.journal_id IN {}
AND am.date BETWEEN '{}' AND '{}'
AND am.company_id in {} """
.format(str(fc_journal_list), date_start, date_end,company_domain)
return query
def get_acc_move_query(self, date_start, date_end, company_domain):
query = """
UNION
SELECT CAST('FPL' AS text) AS type, aml.id AS ID,aml.treasury_date AS date, am.name AS name, aml.company_id, aml.amount_residual AS amount, NULL AS cf_forecast,
NULL AS journal_id, am.move_type as kind
FROM account_move_line aml
LEFT JOIN account_move am ON (aml.move_id = am.id)
WHERE am.state NOT IN ('draft')
AND aml.treasury_planning AND aml.amount_residual != 0
AND aml.treasury_date BETWEEN '{}' AND '{}'
AND aml.company_id in {} """
.format(date_start, date_end, company_domain)
return query
Thanks in advance
The error has nothing to do with Odoo.
psycopg2.errors.SyntaxError: syntax error at or near ")" LINE 9:
WHERE abs.journal_id IN ()
It's cleary a syntax error in the query itself. You're using the IN operator without having a value list afterwards.
Your fc_journal_list parameter doesn't have values on your call. You should catch an empty list before creating the query.
And then there are atleast 2 big security risks in your code:
never ever use string formatting for querys, the comment under your question already points to variables in SQL queries that's the common mistake to make SQL injections an easy thing...
don't make such security risky methods (here both query returning methods) public to the odoo external API. Just add a _ at the beginning of the method names and you're fine on that part.
Odoo has a very powerful ORM API to do the psql queries. Is there a good reason you use sql instead?
The functions you need are, Read for selecting the fields you use, search and filtered for filtering the results.
I suggest reading the following tutorial.
https://www.odoo.com/documentation/14.0/developer/reference/addons/orm.html#search-read
also look at good examples inside the odoo source, I think the stock module is a good place to see some examples.
https://github.com/odoo/odoo/blob/14.0/addons/stock/models/stock_move.py
To fix the error without removing the query, In the function calling get_bank_fc_query you have to check for empty lists first. In python that is very easy, becouse everything that is empty equals False, so do this:
if not fc_journal_list:
raise exceptions.UserError('fc_journal_list cannot be empty')
query = self.get_bank_fc_query(fc_journal_list, date_start, date_end,company_domain
....
I'm trying to reflect a table, while loading only specific columns using include_columns. However, omitting a column that has a ForeignKeyConstraint causes an exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 3825, in _set_parent
ColumnCollectionConstraint._set_parent(self, table)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 3414, in _set_parent
ColumnCollectionMixin._set_parent(self, table)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 3371, in _set_parent
for col in self._col_expressions(table):
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 3365, in _col_expressions
return [
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 3366, in <listcomp>
table.c[col] if isinstance(col, util.string_types) else col
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/base.py", line 1213, in __getitem__
return self._index[key]
KeyError: 'DiscountDefinition_FK'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File ".../script.py", line 62, in <module>
POSBillItem = Table('POSBillItem', metadata, include_columns=POSBillItem_columns, autoload_with=engine)
File "<string>", line 2, in __new__
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/deprecations.py", line 309, in warned
return fn(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 607, in __new__
metadata._remove_table(name, schema)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 602, in __new__
table._init(name, metadata, *args, **kw)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 677, in _init
self._autoload(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 712, in _autoload
conn_insp.reflect_table(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/reflection.py", line 795, in reflect_table
self._reflect_fk(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/reflection.py", line 997, in _reflect_fk
table.append_constraint(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 913, in append_constraint
constraint._set_parent_with_dispatch(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/base.py", line 1046, in _set_parent_with_dispatch
self._set_parent(parent, **kw)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 3827, in _set_parent
util.raise_(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
sqlalchemy.exc.ArgumentError: Can't create ForeignKeyConstraint on table 'POSBillItem': no column named 'DiscountDefinition_FK' is present.
from sqlalchemy import create_engine, engine
from sqlalchemy import MetaData, Table
conn_string = engine.URL.create(
"mssql+pyodbc",
username,
password,
host,
port,
db,
query={"driver": "ODBC Driver 18 for SQL Server", "TrustServerCertificate": "YES"}
)
engine = create_engine(conn_string)
metadata = MetaData()
Table1_columns = [
"PosBill_FK",
"Quantity",
"UnitPrice",
#"DiscountDefinition_FK",
#"originID"
]
POSBillItem = Table('POSBillItem', metadata, include_columns=Table1_columns, autoload_with=engine)
If i uncomment "DiscountDefinition_FK", exception no longer occurs, though i get a warning:
.../script.py:62: SAWarning: Omitting index key for (originID), key covers omitted columns.
POSBillItem = Table('POSBillItem', metadata, include_columns=POSBillItem_columns, autoload_with=engine)
If i uncomment "originID", warning goes away as well.
Is there a way to avoid loading these columns without manually adding columns?
The include_columns option is not, in fact, honored by foreign key reflection - in a discussion on GitHub it was confirmed to be a bug.
As for warnings on index keys, they would probably be phased out in the future, but right now the options are: either including them or supressing the warning with warnings library.
UPD: Fixed in SQLAlchemy 1.4.38.
I'm getting an error when I try to pass a non-executed query string into a raw query, using the params argument that Django recommends, to use as a subquery:
from apps.bikeshare.models import Station
qs = Station.objects.filter(...)
subquery_string = qs.values('id').order_by().query
raw = Station.objects.raw('SELECT * FROM bikeshare_station WHERE id IN (%s)', [subquery_string])
The query prints the correct SQL (WHERE clause omitted):
<RawQuerySet: SELECT * FROM bikeshare_station WHERE id IN (SELECT "bikeshare_station"."id" FROM "bikeshare_station" WHERE ...)>
However, executing the raw query (raw[0]) gives error:
Traceback (most recent call last):
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2847, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-4-6eada3e31b4e>", line 1, in <module>
raw[0]
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/models/query.py", line 1329, in __getitem__
return list(self)[k]
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/models/query.py", line 1299, in __iter__
query = iter(self.query)
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/models/sql/query.py", line 93, in __iter__
self._execute_query()
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/models/sql/query.py", line 127, in _execute_query
self.cursor.execute(self.sql, params)
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/backends/utils.py", line 100, in execute
return super().execute(sql, params)
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/gbrown/Envs/bikeshare-dev/lib/python3.5/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: can't adapt type 'Query'
I'm aware that this is dangerous as it is SQL injection. However, I would like to support this, to create a QuerySet method (which happens to require raw query) that can be used on the end of a queryset chain where the filtered queryset is passed into the raw query as a subquery.
Would the params parameter mitigate any risk even if the error didn't happen? Or shall I just allow the risk and use basic python string formatting?
I've just released my method doesn't work properly anyway, since the query method doesn't return valid SQL, i.e.
qs = Station.objects.filter(name='Albert Gate, Hyde Park').values('id').order_by()
print(qs.query)
[out] SELECT "bikeshare_station"."id" FROM "bikeshare_station" WHERE "bikeshare_station"."name" = Albert Gate, Hyde Park
blows up because the name string is not quoted.
It seems the only way then is, as Jon Clements points out in his comment, to execute the query and pass in a list of ids which can be achieved:
qs = Station.objects.filter(name='Albert Gate, Hyde Park')
ids = list(qs.values_list('id', flat=True))
You could then pass the list of ids to the raw query rather than a string.
I have Postgres DB with a table of pending operations. One column in the operation in an enum with the status of the enum. I used the standard python (2.7) enum, with AutoNumber (myenum.py):
class AutoNumber(enum.Enum):
def __new__(cls):
value = len(cls.__members__) + 1
obj = object.__new__(cls)
obj._value_ = value
return obj
class MyStatus(AutoNumber):
INITIAL = ()
ACCEPTED = ()
DENIED = ()
ACK_PENDING = ()
AUTHORIZED = ()
ACTIVE = ()
END = ()
DELETED = ()
# end enum
The table looks like (also in myenum.py):
Base = declarative_base()
class MyOperation(Base):
__tablename__ = 'operations'
id = Column( Integer, primary_key=True )
status = Column( Enum(MyStatus) )
status_message = Column( String )
status_time = Column( DateTime )
def __repr__(self):
return "<MyOperation(%s, %s, %s, %s)>" % \
( self.id, self.status, self.status_time, self.status_message )
# end class
Generally this works fine. In the SAME FILE that defines MyStatus (myoper.py), I can change the status and save it back to the DB and it works fine:
def checkOper( oper ):
oper.status = MyStatus.DENIED
oper.status_message = "failed check (internal)"
oper.status_time = datetime.datetime.utcnow()
Here's how I call it (within myoper.py)
checkOper( oper )
session.add(oper)
session.commit()
This is all in the same file (myoper.py).
However, if I pass an oper object to an external function, and IT changes the status, then I get a sqlalchemy.exc.StatementError.
Here's the external function (myoper_test.py):
import datetime
from myoper import MyStatus
def extCheckOper( oper ):
oper.status = MyStatus.DENIED
oper.status_message = "failed check (external)"
oper.status_time = datetime.datetime.utcnow()
Here's how I call it (from myoper.py):
from myoper_test import extCheckOper
extCheckOper( oper )
session.add(oper)
session.commit()
Here's the stack trace:
Traceback (most recent call last):
File "./myoper.py", line 120, in <module>
session.commit()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 906, in commit
self.transaction.commit()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 461, in commit
self._prepare_impl()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2177, in flush
self._flush(objects)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2297, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2261, in _flush
flush_context.execute()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
rec.execute(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
uow
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj
mapper, table, update)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 737, in _emit_update_statements
execute(statement, multiparams)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1121, in _execute_context
None, None)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1116, in _execute_context
context = constructor(dialect, self, conn, *args)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 639, in _init_compiled
for key in compiled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 639, in <genexpr>
for key in compiled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/sqltypes.py", line 1446, in process
value = self._db_value_for_elem(value)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/sqltypes.py", line 1354, in _db_value_for_elem
'"%s" is not among the defined enum values' % elem)
sqlalchemy.exc.StatementError: (exceptions.LookupError) "MyStatus.DENIED" is not among the defined enum values [SQL: u'UPDATE operations SET status=%(status)s, status_message=%(status_message)s, status_time=%(status_time)s WHERE operations.id = %(operations_id)s'] [parameters: [{'status': <MyStatus.DENIED: 6>, 'status_time': datetime.datetime(2017, 10, 18, 20, 22, 44, 350035), 'status_message': 'failed check (external)', 'operations_id': 3}]]
I've tried inspecting the type both in the internal file, and external file, but it's ways listed as <enum 'MyStatus'>.
I have found, that if I assign the oper.status to the enum .name, then that DOES work:
def extCheckOper( oper ):
oper.status = MyStatus.AUTHORIZED.name
oper.status_message = "authorized check (external)"
oper.status_time = datetime.datetime.utcnow()
But that's obviously pretty ugly.
So - what am I doing wrong? What is different about MyStatus in the file it's defined in, vs an external file that screws up SQL Alchemy?
I posted this question to the SQL Alchemy mailing list and got an answer. Link to thread
Turns out this one of those "gotcha's" about python and has nothing really to do with SQL Alchemy. Here's a reference: Executing Main Module Twice.
In this particular case, when I executed my script, the MyStatus was created with a particular id (python handle on the type). But when myoper_test imported MyStatus from myoper, it was created AGAIN with a different id.
So when the extCheckOper assigned a MyStatus value to the status field, it was a different MyStatus than SQL Alchemy created the DB mapping with, so when SQL Alchemy tried to save it to the DB, the " is " operator failed, since the (external) MyStatus was different than (original) MyStatus.
There are a couple of different workarounds. One way is to not run the code as main (after moving exiting main code into main() function):
$ python -c "from myoper import main; import sys; main(*sys.argv[1:])" ext_check 1
The better solution is to avoid this problem entirely - move the code that calls out externally to an internal test script. The code in main stays within mainly within the main script (sorry, couldn't resist... :-) ).
I'm currently learning SQLAlchemy, and i found this strange thing. I was experimenting with a table which stores a person's name and address, and to get them i use this:
session.query(User)
And to get the first item, i tried:
session.query(User).first()
Which throws a DatabaseError:
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
session.query(User).first()
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2275, in first
ret = list(self[0:1])
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2142, in __getitem__
return list(res)
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2346, in __iter__
return self._execute_and_instances(context)
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2361, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 664, in execute
return meth(self, multiparams, params)
File "build\bdist.win32\egg\sqlalchemy\sql\elements.py", line 272, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 761, in _execute_clauseelement
compiled_sql, distilled_params
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 874, in _execute_context
context)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 1023, in _handle_dbapi_exception
exc_info
File "build\bdist.win32\egg\sqlalchemy\util\compat.py", line 185, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 867, in _execute_context
context)
File "build\bdist.win32\egg\sqlalchemy\engine\default.py", line 376, in do_execute
cursor.execute(statement, parameters)
DatabaseError: (DatabaseError) ORA-01036: illegal variable name/number
'SELECT test_user_uid, test_user_name, test_user_address \nFROM (SELECT test_user."uid" AS test_user_uid, test_user.name AS test_user_name, test_user.address AS test_user_address \nFROM test_user) \nWHERE ROWNUM <= :ROWNUM_1' {'ROWNUM_1': 1}
However, i was able to retrieve what i wanted if i select all the rows, and loop through the query object:
users = [user for user in session.query(User)]
user1 = users[0]
That's all, i thought it's strange. Here's my mapping class:
class User(Base):
__tablename__ = 'test_user'
uid = Column(Integer, primary_key = True)
name = Column(String(50))
address = Column(String(100))
def __repr__(self):
return "<User (%s, %s)"%(self.name, self.address)
My best guess is that Session.query().first() is looking for the first row, with the generated query. However, the working method retrieves all the rows, and select the first one in Python. The problem is clearly from the generated query (invalid query). The main question is, what caused SQLAlchemy to create an invalid query?
Also, i noticed that SQLAlchemy makes things more difficult by making a query with sub-query. Is that behavior intended?
I hope i can get a satisfying answer, thanks!
Well, it didn't take me long to realize this. It turns out that it's a version problem, i was previously using cx_Oracle version 5.0.2 10g, i tried to upgrade it to version 5.1.2 10g, and things works fine.
This is probably an undocumented bug in SQLAlchemy, i can't find a place where they mention it.
Conclusion: If you want to use the latest version of SQLAlchemy (0.9.0b1) with Oracle 10g, you shouldn't use cx_Oracle older than version 5.1.2 10g.
Hope this helps, and thanks for reading the question!