I have query which works in SQLite Spy (a tool for viewing SQLite Databases) but I can't get the query to work in python with peewee as a raw query.
The (simplified) query is
WITH tags(name) AS ( VALUES('foo'),('bar') ) SELECT * FROM tags
The (simplified) python code is:
from peewee import *
db = SqliteDatabase(":memory:")
db.execute_sql("WITH tags(name) AS ( VALUES('foo'),('bar') ) SELECT * FROM tags")
I get an error
peewee.OperationalError: near "WITH": syntax error
I also tried the RawQuery function or using SqliteExtDatabase from the PlayHouse extension.
Is this error caused by me or is it a problem of peewee?
CTE was introduced in SQLite as of version 3.8.3
http://www.sqlite.org/releaselog/3_8_3.html
it is very likely that your Python is using a previous version that does not support WITH
Check sqlite3.sqlite_version_info
https://docs.python.org/2/library/sqlite3.html
Related
I am using flask-sqlalchemy to perform a ranking query on a PlayerKillData table, which has 4 columns: ref_id (Primary Key), kill, all_kill and lvl. Specifically, what I want to achieve is to Rank by kill for player on the same lvl and add a Rank column
#Columns define in Create Table SQL statement and resolved by reflection at runtime
class PlayerKillData(db.Model):
__tablename__ = "player_kill_data"
The following python snippet is that I have tried but no matter how I have tried, it always return sqlite3.OperationalError:
subquery = excDB.db.session.query(
PlayerKillData,
excDB.db.func.rank().over(
order_by = PlayerKillData.kill.desc(),
partition_by = PlayerKillData.lvl).label('RANK')
)
subquery.all()
The SQL statement printed by the error message is:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "(": syntax error
[SQL: SELECT player_kill_data.ref_id AS player_kill_data_ref_id, player_kill_data.kill AS player_kill_data_kill, player_kill_data.all_kill AS player_kill_data_all_kill, player_kill_data.lvl AS player_kill_data_lvl, rank() OVER (PARTITION BY player_kill_data.lvl ORDER BY player_kill_data.kill DESC) AS "RANK"
FROM player_kill_data]
I have also tried running the SQL statement directly using Db Browser for SQLite and it returned the table successfully without any error.
I am really puzzled by this and would like any help I can get to solve this.
Your version of SQLite might be smaller than 3.25.0.
As mentionned here and confirmed in the Release History of SQLite, SQLite supports window function (RANK() OVER ...) since version 3.25.0, which was released on 2018-09-15.
To check your version of SQLite in python:
import sqlite3
print(sqlite3.sqlite_version)
# returns '3.22.0' for me
If many people work on the same codebase and you'd like to have a more explicit error message for other developers, you can assert:
assert sqlite3.sqlite_version_info >= (3,25,0), """sqlite3 version must be >= 3.25.0 because that's when it started to support window functions"""
I'm using python 3.6.4 and sqlite3 2.6.0 to query the nearest consecutive dates in my table in a sqlite 3.27.2 file.
I've tried to get the actual sql string with vscode debugger and test it with DB Browser for SQLite. It works as I expect.
Here's the code:
sql = 'WITH \
dates(cast_date) AS (\
SELECT DISTINCT play_date\
FROM TimeTable\
),\
groups AS (\
SELECT\
date(cast_date, \'-\'||(ROW_NUMBER() OVER (ORDER BY cast_date))||\' days\') AS grp,\
cast_date\
FROM dates\
)\
SELECT\
MIN(cast_date) AS date_start,\
MAX(cast_date) AS date_end\
FROM groups GROUP BY grp ORDER BY 2 DESC LIMIT 1'
cursor = conn.cursor()
result = []
try:
cursor.execute(sql)
result = cursor.fetchone()
except sqlite3.OperationalError:
FileLogger.exception('Exception at '+__file__+' '+__name__)
An exception occurs:
cursor.execute(sql)
sqlite3.OperationalError: near "OVER": syntax error
Window functions support was first added to SQLite with release version 3.25.0 (2018-09-15), according to official documentation.
When using Python, you are using Python SQLite3 client library (which is distributed with Python) instead of your system SQLite3 installation. For Python 2.7, the version is 3.11.0, which is below your required version.
You may try using a newer SQLite3 client library, as suggested by these answers.
Python Model:
class SYSLocation(SYSModel):
__tablename__ = 'sys_location'
rank = db.Column(db.Integer)
Call:
db.session.add(model)
It generate mysql script:
INSERT INTO `sys_location` ( rank) VALUES (13000)
sql error :
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'rank)
VALUES (13000)' at line 1
I check this query run error mysql version 8.0.11 because rank is keyword.
But this sql can run mysql version 10.1.25-MariaDB.
How to fix my Sqlalchemy model run all version of mysql?
As always, when dealing with reserved keywords wrap it in backticks like you did for sys_location:
INSERT INTO `sys_location` (`rank`) VALUES (13000)
You can escape everything if you want, but it's often not necessary. Keywords are a case where it might be necessary because these keywords do change, if infrequently.
I use psycopg2 code to open an sql file (postgresql), and I pass the value for LIMIT as a parameter in my code. And it works fine. Here is some of the codes:
.
.
cur.execute(open("sample.sql","r").read(),(lim,))
.
.
my sample.sql creates a table named sample and populates it with some data from another table named test
CREATE TABLE sample(name varchar(500));
INSERT INTO sample(name) SELECT name FROM test LIMIT %s;
%s takes the value lim passed from cur.execute
My question is: how do I translate this into pyodbc code to use it in sql server?
psycopg2 and pyodbc both implement Python's DB API, and that specification defines several parameter styles that can be used. The developers of psycopg2 and pyodbc simply chose different styles:
psycopg2 uses "format" placeholders: ... WHERE city = %s
pyodbc uses "qmark" placeholders: ... WHERE city = ?
I have a python code, in which I make SQL requests in a database. I would like to be able to switch between a postgresql (using module psycopg2) database and a sqlite one (using module sqlite3), without need of adapting my code. This means, I would like to have in my code some fixed SQL request strings, and I want to switch between the engine, only changing the definition of the database connector object, using one of those:
my_db = psycopg2.connect(...)
my_db = sqlite3.connect(...)
For the moment, I don't see any possibilty since:
Everyone knows that one should NOT use string concatenation to pass arguments to a SQL request, but rather use placeholders (from psycopg2 docu :never, NEVER use Python string concatenation ... to pass variables to a SQL query string. Not even at gunpoint. )
The synthax for placeholders are different is the 2 APIs psycopg2 and sqlite3. Even for NON-named placeholders. Psycopg uses "%" and sqlite3 uses "?":
my_cursor.execute("SELECT * FROM table WHERE id= ?", (my_id,)) # for SQLITE3
my_cursor.execute("SELECT * FROM table WHERE id= %", (my_id,)) # for PSYCOPG2
One could in principle use the SQL built-in placeholder synthax ("?"
for postgresql), but this would mean precisely preparing a SQL-string with python string concatenation, and so on... that is forbidden by 1.
I'm lacking ideas...