We Keep Coding

Python's sqlite3 module: get query with escaped parameters [duplicate]

I'm trying to debug a SQL statement generated with sqlite3 python module...
c.execute("SELECT * FROM %s WHERE :column = :value" % Photo.DB_TABLE_NAME, {"column": column, "value": value})
It is returning no rows when I do a fetchall()
When I run this directly on the database
SELECT * FROM photos WHERE album_id = 10
I get the expected results.
Is there a way to see the constructed query to see what the issue is?

To actually answer your question, you can use the set_trace_callback of the connection object to attach the print function; this will make all queries get printed when they are executed. Here is an example in action:
# Import and connect to database
import sqlite3
conn = sqlite3.connect('example.db')
# This attaches the tracer
conn.set_trace_callback(print)
# Get the cursor, execute some statement as an example
c = conn.cursor()
c.execute("CREATE TABLE stocks (symbol text)")
t = ('RHAT',)
c.execute("INSERT INTO stocks VALUES (?)", t)
c.execute('SELECT * FROM stocks WHERE symbol=?', t)
print(c.fetchone())
This produces the output:
CREATE TABLE stocks (symbol text)
BEGIN
INSERT INTO stocks VALUES ('RHAT')
SELECT * FROM stocks WHERE symbol='RHAT'
('RHAT',)

the problem here is that the string values are automatically embraced with single quotes. You can not dynamically insert column names that way.
Concerning your question, I'm not sure about sqlite3, but in MySQLdb you can get the final query as something like (I am currently not at a computer to check):
statement % conn.literal(query_params)

You can only use substitution parameters for row values, not column or table names.
Thus, the :column in SELECT * FROM %s WHERE :column = :value is not allowed.

Error while performing multiple operations on database

I have to loop over all rows in the database and have to update them at some location. I have written this code:
import sqlite3
con=sqlite3.connect("Models.db")
c=con.cursor()
c1=con.cursor()
c.execute("CREATE TABLE ABC(id INTEGER PRIMARY KEY, COLVAL text)")
///inserted a few rows........
for row in c.execute("SELECT * FROM ABC"):
if $(any condition):
with con:
c1.execute("UPDATE ABC SET COLUMN1=(:COLVAL) WHERE id=(:id)",{'COLVAL':'XYZ','id':row[0]})
but I get an error:
OperationalError: database is locked
in the second query, c1.execute("UPDATE ABC SET COLUMN1=(:COLVAL) WHERE id=(:id)",{'COLVAL':'XYZ','id':row[0]})
I do not want to fetch all rows at a time and apply for a loop, as there is a lot of data. Just one row at a time which in my case:
"for row in c.execute("SELECT * FROM ABC")"
line is doing.
Does anyone know the solution for it?

Basically you can loop within a list such as c.execute("SELECT * FROM abc").fetchall()
c.execute("CREATE TABLE IF NOT EXISTS abc(id INT PRIMARY KEY, column1 TEXT)")
///inserted a few rows........
for row in c.execute("SELECT * FROM abc").fetchall():
if $(any condition):
c.execute("UPDATE abc SET column1=(?) WHERE id=(?)",('XYZ',row[0],))
con.commit()
con.close()
Additionally;
add IF NOT EXISTS to the DDL Statement
replace parameter placeholders with question marks as being securer
no need to open more than one cursor
don't miss commiting the DML Statement at the end
extra tip : use executemany as
c.executemany("UPDATE abc SET column1=(?) WHERE id=(?)",[('XYZ',row[0],)])
instead of execute for the DML statement for the multiple data as being more performant. Indeed, you don't need for this current case as only concerns with only one row for each id value

Get data from select statement mysql.connector

How do I store the content a select statement into a variable?
I have a table called testing with 2 fields: ID, and events.
I do SELECT * FROM testing WHERE id=1; to get the row, but how do I get the data in the events column of that row and store it into a variable x? I couldn't find how to do this online.

I suppose you want to do this in python and I suppose you have already connected your database and put it in a variable named conn
sql = "select * from Laptop"
cursor = conn.cursor()
cursor.execute(sql)
x = cursor.fetchall()
and your result will be stored in the x variable

Why this sqlite3 code in PYTHON 3.7 program about employees and their first name,last name and salaries is not getting printed in the output?

I have a sqlite3 program in which I have to connect to a database stored in 'employees.db'. I will enter the first name, last name and salary and then print it. For printing, I will use fetchall(). But when I print the output, what I get is this-->
[]
only 2 square brackets?
What is wrong?
I have started to learn python 2 weeks ago. But my course where I am learning is going too fast. I don't know what I am doing in this program is the correct approach or not?
HERE IS MY CODE--->>>
import sqlite3;
conn=sqlite3.connect('employees.db');
c=conn.cursor();
c.execute("""CREATE TABLE employees(first text,last text,pay integer)""");
c.execute("INSERT INTO employees VALUES('Tendo','Sinha',600000)");
c.execute("INSERT INTO employees VALUES('Krit','Kumar',40000)");
print(c.fetchall());
conn.commit();
conn.close
I don't get any error. The output only shows:-
[]
Only 2 square brackets.

You are inserting rows into the table but not retrieving them at all.
Try adding a SELECT statement:
import sqlite3
conn = sqlite3.connect("employees.db")
c = conn.cursor()
# Create table, insert things...
c.execute(
"""CREATE TABLE IF NOT EXISTS employees(first text,last text,pay integer)"""
)
c.execute("INSERT INTO employees VALUES('Kritin','Sinha',600000)")
c.execute("INSERT INTO employees VALUES('Krit','Kumar',40000)")
# Commit the changes.
conn.commit()
# Retrieve from the table.
c.execute("SELECT * FROM employees")
print(c.fetchall())

you have to run a query for fetching all rows.
add these lines to your code-
data = c.execute("Select * FROM employees").fetchall()
print(data);

DBAPI syntax with pd.read_sql_query() call

I want to read all of the tables contained in a database into pandas data frames. This answer does what I want to accomplish, but I'd like to use the DBAPI syntax with the ? instead of the %s, per the documentation. However, I ran into an error. I thought this answer may address the problem, but I'm now posting my own question because I can't figure it out.
Minimal example
import pandas as pd
import sqlite3
pd.__version__ # 0.19.1
sqlite3.version # 2.6.0
excon = sqlite3.connect('example.db')
c = excon.cursor()
c.execute('''CREATE TABLE stocks
(date text, trans text, symbol text, qty real, price real)''')
c.execute("INSERT INTO stocks VALUES ('2006-01-05', 'BUY', 'RHAT', 100, 35.14)")
c.execute('''CREATE TABLE bonds
(date text, trans text, symbol text, qty real, price real)''')
c.execute("INSERT INTO bonds VALUES ('2015-01-01', 'BUY', 'RSOCK', 90, 23.11)")
data = pd.read_sql_query('SELECT * FROM stocks', excon)
# >>> data
# date trans symbol qty price
# 0 2006-01-05 BUY RHAT 100.0 35.14
But when I include a ? or a (?) as below, I get the error message pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT * FROM (?)': near "?": syntax error.
Problem code
c.execute("SELECT name FROM sqlite_master WHERE type='table';")
tables = c.fetchall()
# >>> tables
# [('stocks',), ('bonds',)]
table = tables[0]
data = pd.read_sql_query("SELECT * FROM ?", excon, params=table)
It's probably something trivial that I'm missing, but I'm not seeing it!

The problem is that you're trying to use parameter substitution for a table name, which is not possible. There's an issue on GitHub that discusses this. The relevant part is at the very end of the thread, in a comment by #jorisvandenbossche:
Parameter substitution is not possible for the table name AFAIK.
The thing is, in sql there is often a difference between string
quoting, and variable quoting (see eg
https://sqlite.org/lang_keywords.html the difference in quoting
between string and identifier). So you are filling in a string, which
is for sql something else as a variable name (in this case a table
name).

Parameter substitution is essential to prevent SQL Injection from unsafe user-entered values.
In this particular example you are sourcing table names directly from the database's own metadata, which is already safe, so it's OK to just use normal string formatting to construct the query, but still good to wrap the table names in quotes.
If you are sourcing user-entered table names, you can also parameterize them first before using them in your normal python string formatting.
e.g.
# assume this is user-entered:
table = '; select * from members; DROP members --'
c.execute("SELECT name FROM sqlite_master WHERE type='table' and name = ?;", excon, params=table )
tables = c.fetchall()
In this case the user has entered some malicious input intended to cause havoc, and the parameterized query will cleanse it and the query will return no rows.
If the user entered a clean table e.g. table = 'stocks' then the above query would return that same name back to you, through the wash, and it is now safe.
Then it is fine to continue with normal python string formatting, in this case using f-string style:
table = tables[0]
data = pd.read_sql_query(f"""SELECT * FROM "{table}" ;""", excon)
Referring back to your original example, my first step above is entirely unnecessary. I just provided it for context. It is unnecessary, because there is no user input so you could just do something like this to get a dictionary of dataframes for every table.
c.execute("SELECT name FROM sqlite_master WHERE type='table';")
tables = c.fetchall()
# >>> tables
# [('stocks',), ('bonds',)]
dfs = dict()
for t in tables:
dfs[t] = pd.read_sql_query(f"""SELECT * FROM "{t}" ;""", excon)
Then you can fetch the dataframe from the dictionary using the tablename as the key.