We Keep Coding

Odoo 14, issue when trying to computing bank balance in treasury module

i got work to fix error when computing but i still dont have idea how to fix it because i'm still newbie
Odoo Server Error
Traceback (most recent call last): File
"/home/equipAccounting/equip/odoo/addons/base/models/ir_http.py", line
237, in _dispatch
result = request.dispatch() File "/home/equipAccounting/equip/odoo/http.py", line 683, in dispatch
result = self._call_function(**self.params) File "/home/equipAccounting/equip/odoo/http.py", line 359, in
_call_function
return checked_call(self.db, args, *kwargs) File "/home/equipAccounting/equip/odoo/service/model.py", line 94, in
wrapper
return f(dbname, args, *kwargs) File "/home/equipAccounting/equip/odoo/http.py", line 347, in checked_call
result = self.endpoint(*a, **kw) File "/home/equipAccounting/equip/odoo/http.py", line 912, in call
return self.method(*args, **kw) File "/home/equipAccounting/equip/odoo/http.py", line 531, in response_wrap
response = f(*args, **kw) File "/home/equipAccounting/equip/addons/basic/web/controllers/main.py",
line 1393, in call_button
action = self._call_kw(model, method, args, kwargs) File "/home/equipAccounting/equip/addons/basic/web/controllers/main.py",
line 1381, in _call_kw
return call_kw(request.env[model], method, args, kwargs) File "/home/equipAccounting/equip/odoo/api.py", line 396, in call_kw
result = _call_kw_multi(method, model, args, kwargs) File "/home/equipAccounting/equip/odoo/api.py", line 383, in _call_kw_multi
result = method(recs, args, *kwargs) File "/home/equipAccounting/equip/addons/core/treasury_forecast/models/treasury_bank_forecast.py",
line 290, in compute_bank_balances
self.env.cr.execute(main_query) File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in
fun
return caller(func, (extras + args), *kw) File "/home/equipAccounting/equip/odoo/sql_db.py", line 101, in check
return f(self, args, *kwargs) File "/home/equipAccounting/equip/odoo/sql_db.py", line 298, in execute
res = self._obj.execute(query, params) Exception
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File
"/home/equipAccounting/equip/odoo/http.py", line 639, in
_handle_exception
return super(JsonRequest, self)._handle_exception(exception) File "/home/equipAccounting/equip/odoo/http.py", line 315, in
_handle_exception
raise exception.with_traceback(None) from new_cause psycopg2.errors.SyntaxError: syntax error at or near ")" LINE 9:
WHERE abs.journal_id IN ()
and here is the code :
def get_bank_fc_query(self, fc_journal_list, date_start, date_end,company_domain):
query = """
UNION
SELECT CAST('FBK' AS text) AS type, absl.id AS ID, am.date, absl.payment_ref as name, am.company_id, absl.amount_main_currency as amount, absl.cf_forecast, abs.journal_id, NULL as kind FROM account_bank_statement_line absl
LEFT JOIN account_move am ON (absl.move_id = am.id)
LEFT JOIN account_bank_statement abs ON (absl.statement_id = abs.id)
WHERE abs.journal_id IN {}
AND am.date BETWEEN '{}' AND '{}'
AND am.company_id in {} """
.format(str(fc_journal_list), date_start, date_end,company_domain)
return query
def get_acc_move_query(self, date_start, date_end, company_domain):
query = """
UNION
SELECT CAST('FPL' AS text) AS type, aml.id AS ID,aml.treasury_date AS date, am.name AS name, aml.company_id, aml.amount_residual AS amount, NULL AS cf_forecast,
NULL AS journal_id, am.move_type as kind
FROM account_move_line aml
LEFT JOIN account_move am ON (aml.move_id = am.id)
WHERE am.state NOT IN ('draft')
AND aml.treasury_planning AND aml.amount_residual != 0
AND aml.treasury_date BETWEEN '{}' AND '{}'
AND aml.company_id in {} """
.format(date_start, date_end, company_domain)
return query
Thanks in advance

The error has nothing to do with Odoo.
psycopg2.errors.SyntaxError: syntax error at or near ")" LINE 9:
WHERE abs.journal_id IN ()
It's cleary a syntax error in the query itself. You're using the IN operator without having a value list afterwards.
Your fc_journal_list parameter doesn't have values on your call. You should catch an empty list before creating the query.
And then there are atleast 2 big security risks in your code:
never ever use string formatting for querys, the comment under your question already points to variables in SQL queries that's the common mistake to make SQL injections an easy thing...
don't make such security risky methods (here both query returning methods) public to the odoo external API. Just add a _ at the beginning of the method names and you're fine on that part.

Odoo has a very powerful ORM API to do the psql queries. Is there a good reason you use sql instead?
The functions you need are, Read for selecting the fields you use, search and filtered for filtering the results.
I suggest reading the following tutorial.
https://www.odoo.com/documentation/14.0/developer/reference/addons/orm.html#search-read
also look at good examples inside the odoo source, I think the stock module is a good place to see some examples.
https://github.com/odoo/odoo/blob/14.0/addons/stock/models/stock_move.py
To fix the error without removing the query, In the function calling get_bank_fc_query you have to check for empty lists first. In python that is very easy, becouse everything that is empty equals False, so do this:
if not fc_journal_list:
raise exceptions.UserError('fc_journal_list cannot be empty')
query = self.get_bank_fc_query(fc_journal_list, date_start, date_end,company_domain
....

How to access tables from a different schema in oracle 11g using django?

I have a user named mi_abc in oracle 11g.
The user do not have any table in the database but has access to all the tables in another schema sch_abc.
When I run a normal select query from sqldeveloper on the sch_abc schema from mi_abc, it works perfectly fine, but when I use django, I am always getting the error:-
django.db.utils.DatabaseError: ORA-00942: table or view does not exist
I tried to set the db_table = sch_abc.tableName and also set db_table = tableName but both gives me the same error. Any clue how to resolve this?
TRACE:-
Traceback (most recent call last):
File "C:\xxx\xxx\xxx\xxx\xxx\xxx\xxxx\lib\site-packages\django\core\handlers\exception.py", line 41, in inner
response = get_response(request)
File "C:\xxx\xxx\xxxx\xxxx\xxxx\Python\Python37\lib\site-packages\django\utils\deprecation.py", line 142, in __call__
response = self.process_response(request, response)
File "C:\xxxx\xxxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\contrib\sessions\middleware.py", line 58, in process_response
request.session.save()
File "C:\xxx\xxxx\xxxx\xxxx\xxxxx\Python\Python37\lib\site-packages\django\contrib\sessions\backends\db.py", line 81, in save
return self.create()
File "C:\xxxx\xxxxx\xxxx\xxxx\xxxxx\Python\Python37\lib\site-packages\django\contrib\sessions\backends\db.py", line 50, in create
self._session_key = self._get_new_session_key()
File "C:\xxxx\xxxxx\xxxxx\xxxxx\xxxx\Python\Python37\lib\site-packages\django\contrib\sessions\backends\base.py", line 164, in _get_new_session_key
if not self.exists(session_key):
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\contrib\sessions\backends\db.py", line 46, in exists
return self.model.objects.filter(session_key=session_key).exists()
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\models\query.py", line 673, in exists
return self.query.has_results(using=self.db)
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\models\sql\query.py", line 517, in has_results
return compiler.has_results()
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\models\sql\compiler.py", line 858, in has_results
return bool(self.execute_sql(SINGLE))
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\models\sql\compiler.py", line 899, in execute_sql
raise original_exception
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\models\sql\compiler.py", line 889, in execute_sql
cursor.execute(sql, params)
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\backends\utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\backends\utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\utils\six.py", line 685, in reraise
raise value.with_traceback(tb)
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\backends\utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "C:\xxx\xxx\xxx\xxx\xxx\Python\Python37\lib\site-packages\django\db\backends\oracle\base.py", line 497, in execute
return self.cursor.execute(query, self._param_generator(params))
django.db.utils.DatabaseError: ORA-00942: table or view does not exist

Well, I resolved the issue and let me tell you there is no straight way to do it in django. The problem with my application was, I was using the authetication features of django and also session handling. All these tables are created by django directly on the initial migration. So, there is no existence of them in the models.py file that you can simply append the schema name and ask your application to connect to the table of that schema.
What I ended up doing is, I created private synonyms to all the tables of the other schema which actually contained those tables. If you do this, you don't have to change anything in your django code. Your application will simply work because oracle will do the dirty work of actually connecting to the proper table. You will merely call the table in your application as if its your own. In this way when django looks for tables like django_session, auth_user etc, it simply queries it like it always does and oracle redirects it to the actual tables present in another schema.
Hope this helps people who face this issue in the future.

This is by no means officially supported, but this works in Postgres:
class Meta:
managed = False
db_table = 'schema\".\"table'
It took some trial and error for Postgres, but you can probably do something similar for Oracle. This is because the Postgres engine quotes object names, and this fakes the quoting mechanism out.
UPDATE:
After doing some digging, I found this for Oracle (modified for Python 3):
class Meta:
db_table = '"SCHEMA"."TABLE_NAME"'
Source: https://code.djangoproject.com/ticket/14136
I would recommend keeping managed = False unless you really know what you're doing. Good luck!

You can set the required schema, before executing the command. and then go back to public schema once the queryset is processed.
from django.db import connection
connection.set_schema(schema_name)

SQLAlchemy - Convert sqlalchemy.sql.functions.now() to Unix epoch (with fraction)

I'm using Doubles to track time in my database (as Unix timestamps with fractional portions). For that, I've created the following user type (that takes pytz-aware datetimes from the application and converts them to UTC timestamps).
from sqlalchemy.types import TypeDecorator, Float
class DoubleDatetime(TypeDecorator):
impl = Float
def __init__(self):
TypeDecorator.__init__(self, as_decimal=False)
def process_bind_param(self, value, dialect):
return value.timestamp()
def process_result_value(self, value, dialect):
return datetime.datetime.utcfromtimestamp(value)
This works fine for all datetimes in my system. I'm using the following column definition to have the database (in my case, SQLite3) insert the transaction time:
from sqlalchemy import Column
from sqlalchemy.sql.functions import now
column = Column("transaction_begin", DoubleDatetime, server_default=now())
Despite defining the transaction_begin column as a DoubleDatetime, my database still stores the records with a string.
How can I force the server transaction timestamp to use fractional Unix epochs?
What have I tried?
Reading the SQLite3 docs, I know I need to do some Julian Day trickery to do this.
SELECT (julianday('now') - 2440587.5) * 86400.0;
But I can't find julianday in sqlalchemy.sql.functions, to wrap it in a zero-argument lambda (to pass as the server_default arg to Column).
I tried the following, but it doesn't seem to work.
from sqlalchemy import text
server_default = text("SELECT (julianday('now') - 2440587.5) * 86400.0;")
Nor does the following:
from sqlalchemy.sql import select, text
server_default = select([text("(julianday('now') - 2440587.5) * 86400.0;")])
Traceback:
Traceback (most recent call last):
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: near "SELECT": syntax error
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "tasks.py", line 81, in <module>
metadata.create_all(engine)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 4005, in create_all
tables=tables)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1940, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1549, in _run_visitor
**kwargs).traverse_single(element)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
return meth(obj, **kw)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
_is_metadata_operation=True)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
return meth(obj, **kw)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 791, in visit_table
include_foreign_key_constraints=include_foreign_key_constraints
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
compiled
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
raise value.with_traceback(tb)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/home/randm/Libraries/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "SELECT": syntax error [SQL: "\nCREATE TABLE barchartbarmessage (\n\tsymbol VARCHAR NOT NULL, \n\tdate DATE NOT NULL, \n\topen NUMERIC, \n\thigh NUMERIC, \n\tlow NUMERIC, \n\tclose NUMERIC, \n\tvolume INTEGER, \n\tprevious_day_open_interest INTEGER, \n\tprevious_day_volume INTEGER, \n\tmessage_time FLOAT, \n\ttransaction_begin FLOAT DEFAULT SELECT (julianday('now') - 2440587.5) * 86400.0; NOT NULL, \n\ttransaction_end FLOAT, \n\tPRIMARY KEY (symbol, date, transaction_begin)\n)\n\n"] (Background on this error at: http://sqlalche.me/e/e3q8)

I had this same question, and found that this worked. The expression here is specific to PostgreSQL. I originally was calculating this value on my application server, which led to unexpected results. See this post for more information. Note that in PostgreSQL, at least, when setting defaults using expressions you don't include SELECT at the start.
import sqlalchemy as sa
logged_ts_ms = sa.Column(sa.BigInteger, server_default=sa.text("""(extract(epoch from now()) * 1000)::bigint;"""))

Postgres function json_array_elements does not found while django's tests

I have class method in django model which is using json_array_elements function.
In case when it performs by browser it works fine.
But in tests it fails.
python manage.py test
Traceback (most recent call last):
File "path_to_project/dj_server/model_animations/tests.py", line 94, in test_cteating
response_first = model_animations.views.get_animations_list(request, groupid)
File "path_to_project/dj_server/model_animations/views.py", line 37, in get_animations_list
for model_anim in listArray:
File "/Library/Python/2.7/site-packages/django/db/models/query.py", line 1535, in __iter__
query = iter(self.query)
File "/Library/Python/2.7/site-packages/django/db/models/sql/query.py", line 76, in __iter__
self._execute_query()
File "/Library/Python/2.7/site-packages/django/db/models/sql/query.py", line 90, in _execute_query
self.cursor.execute(self.sql, self.params)
File "/Library/Python/2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/Library/Python/2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/Library/Python/2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
ProgrammingError: function json_array_elements(text) does not exist
LINE 1: ...on_name FROM model_animations_model, json_array...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
in models.py
#classmethod
def animations_list(self, group_id):
if group_id:
try:
listArray = ModelAnimationList.objects.raw(('SELECT * FROM model_animations_model, json_array_elements(animated_groups) AS data WHERE \'"%s"\' = data::text' %group_id))
return listArray
except:
pass
return None
in views.py
def get_animations_list(request, group_id):
...
listArray = ModelAnimationList.animations_list(group_id)
if listArray:
for model_anim in listArray:
if model_anim:
anim_dict = { 'a_id' : model_anim.id, 'a_name' : model_anim.animation_name }
result_anim_list.append(anim_dict)
...
in tests.py
request = HttpRequest()
response_first = model_animations.views.get_animations_list(request, groupid)
Installed:
python 2.7
Django 1.7.1
Postgres 9.3.5
psycopg2 2.5.4
Mac 10.10 yosemite

Mistake in query json_array_elements(animated_groups)
Needs change to this:
json_array_elements(animated_groups::json)

Postgres JSON processing functions such as json_array_elements(animated_groups::json), jsonb_array_elements(animated_groups::jsonb) will take json or jsonb values.
Because postgres JSON field(animated_groups) are store as text format or binary format.
so we need to convert the type by using either ::json or ::jsonb it give the json or jsonb format
check here

SQLAlchemy: Unable to get the first item of query

I'm currently learning SQLAlchemy, and i found this strange thing. I was experimenting with a table which stores a person's name and address, and to get them i use this:
session.query(User)
And to get the first item, i tried:
session.query(User).first()
Which throws a DatabaseError:
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
session.query(User).first()
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2275, in first
ret = list(self[0:1])
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2142, in __getitem__
return list(res)
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2346, in __iter__
return self._execute_and_instances(context)
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 2361, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 664, in execute
return meth(self, multiparams, params)
File "build\bdist.win32\egg\sqlalchemy\sql\elements.py", line 272, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 761, in _execute_clauseelement
compiled_sql, distilled_params
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 874, in _execute_context
context)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 1023, in _handle_dbapi_exception
exc_info
File "build\bdist.win32\egg\sqlalchemy\util\compat.py", line 185, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "build\bdist.win32\egg\sqlalchemy\engine\base.py", line 867, in _execute_context
context)
File "build\bdist.win32\egg\sqlalchemy\engine\default.py", line 376, in do_execute
cursor.execute(statement, parameters)
DatabaseError: (DatabaseError) ORA-01036: illegal variable name/number
'SELECT test_user_uid, test_user_name, test_user_address \nFROM (SELECT test_user."uid" AS test_user_uid, test_user.name AS test_user_name, test_user.address AS test_user_address \nFROM test_user) \nWHERE ROWNUM <= :ROWNUM_1' {'ROWNUM_1': 1}
However, i was able to retrieve what i wanted if i select all the rows, and loop through the query object:
users = [user for user in session.query(User)]
user1 = users[0]
That's all, i thought it's strange. Here's my mapping class:
class User(Base):
__tablename__ = 'test_user'
uid = Column(Integer, primary_key = True)
name = Column(String(50))
address = Column(String(100))
def __repr__(self):
return "<User (%s, %s)"%(self.name, self.address)
My best guess is that Session.query().first() is looking for the first row, with the generated query. However, the working method retrieves all the rows, and select the first one in Python. The problem is clearly from the generated query (invalid query). The main question is, what caused SQLAlchemy to create an invalid query?
Also, i noticed that SQLAlchemy makes things more difficult by making a query with sub-query. Is that behavior intended?
I hope i can get a satisfying answer, thanks!

Well, it didn't take me long to realize this. It turns out that it's a version problem, i was previously using cx_Oracle version 5.0.2 10g, i tried to upgrade it to version 5.1.2 10g, and things works fine.
This is probably an undocumented bug in SQLAlchemy, i can't find a place where they mention it.
Conclusion: If you want to use the latest version of SQLAlchemy (0.9.0b1) with Oracle 10g, you shouldn't use cx_Oracle older than version 5.1.2 10g.
Hope this helps, and thanks for reading the question!