I am using Sqlite3 and Python. Here is some sample code:
test
-----------------
amount | date
query = "SELECT SUM (column1) FROM test WHERE date BETWEEN '"+blah+"' AND '"+blah+"'"
c.execute(query)
data = c.fetchone()
if not data:
amountsum = 0
else:
amountsum = data[0]
print(amountsum)
The problem is that it only runs else:. If data is NoneType it does not set amountsum to 0 either. How can I make this work?
In this case, data will never be None, due to the aggregating query. SELECT SUM(...) FROM table will always return exactly one row. However, the SUM can be None in SQLite, if there are no rows in the table, so that should be taken into account:
query = "SELECT SUM (column1) FROM test WHERE ..."
c.execute(query)
data = c.fetchone()
amount = data[0] or 0
(A sidenote: you seem to be creating your SQL query using string concatenation, which is a potential SQL injection vulnerability. Consider using parameterized queries instead.)
Related
I'm trying to make a query to a SQLite database from a python script. However, whenever I use parameterization it just returns the first parameter, which is column2. The desired result is for it to return the value held in column2 on the row where column1 is equal to row1.
conn = sqlite3.connect('path/to/database')
c = conn.cursor()
c.execute('SELECT ? from table WHERE column1 = ? ;', ("column2","row1"))
result = c.fetchone()[0]
print(result)
It prints
>>column2
Whenever I run this using concatenated strings, it works fine.
conn = sqlite3.connect('path/to/database')
c = conn.cursor()
c.execute('SELECT ' + column2 + ' from table WHERE column1 = ' + row1 + ';')
result = c.fetchone()[0]
print(result)
And it prints:
>>desired data
Any idea why this is happening?
This behaves as designed.
The mechanism that parameterized queries provide is meant to pass literal values to the query, not meta information such as column names.
One thing to keep in mind is that the database must be able to parse the parameterized query string without having the parameter at hand: obviously, a column name cannot be used as parameter under such assumption.
For your use case, the only possible solution is to concatenate the column name into the query string, as shown in your second example. If the parameter comes from outside your code, be sure to properly validate it before that (for example, by checking it against a fixed list of values).
I have a piece of code that I realized is probably quite inefficient, though I'm not sure how to improve it.
Basically, I have a database table like this:
Example DB table
Any or several of columns A-G might match my search query. If that is the case, I want to query VALUE from that row. I need VALUE not to equal NULL though, so if that's the case, it should keep looking. If my query were abc, I'd want to obtain correct.
Below is my current code, using a database named db with a table table.
cur=db.cursor()
data="123"
fields_to_check=["A","B","C","D","E","F","G"]
for field in fields_to_check:
"SELECT Value FROM table WHERE {}='{}'".format(field,data)
query=cur.fetchone()
if query and query !="NULL":
break
db.close()
I think that the fact that this performs 8 queries is likely very inefficient.
cur=db.cursor()
data="123"
fields_to_check=["A","B","C","D","E","F","G"]
sub_query = ""
for field in fields_to_check:
sub_query = sub_query + "or {}='{}' ".format(field,data)
if sub_query:
query = "SELECT Value FROM table WHERE ("+ str(sub_query[2:]) +") and value IS NOT NULL;"
if query:
cur.execute(query)
rows = cur.fetchall()
if rows:
for row in rows:
print(row)
I'm trying to create an SQL queries for a large list of records (>42 million) to insert into a remote database. Right now I'm building queries in the format INSERT INTO tablename (columnnames) VALUES (values)
tablename, columnnames, and values are all of varying length so I'm generating a number of placeholders equal to the number of values required.
The result is I have a string called sqcommand that looks like INSERT INTO ColName (?,?,?) VALUES (?,?,?); and a list of parameters that looks like ([Name1, Name2, Name3, Val1, Val2, Val3]).
When try to execute the query as db.execute(sqlcommand, params) I get errors indicating I'm trying to insert into columns "#P1", "#P2", "#P3" et cetera. Why aren't the values from my list properly translating? Where is it getting "#P1" from? I know I don't have a column of that name and as far as I can tell I'm not referencing a column of that name yet the execute method is still trying to use it.
UPDATE: As per request, the full code is below, modified to avoid anything that might be private. The end result of this is to move data, row by row, from an sqlite3 db file to an AWS SQL server.
newDB = pyodbc.connect(newDataBase)
oldDB = sqlite3.connect(oldDatabase)
tables = oldDB.execute("SELECT * FROM sqlite_master WHERE type='table';").fetchall()
t0 = datetime.now()
for table in tables:
print('Parsing:', str(table[1]))
t1 = datetime.now()
colInfo = oldDB.execute('PRAGMA table_info('+table[1]+');').fetchall()
cols = list()
cph = ""
i = 0
for col in colInfo:
cph += "?,"
cols.append(str(col[1]))
rowCount = oldDB.execute("SELECT COUNT(*) FROM "+table[1]+" ;").fetchall()
count = 0
while count <= int(rowCount[0][0]):
params = list()
params.append(cols)
count += 1
row = oldDB.execute("SELECT * FROM "+table[1]+" LIMIT 1;").fetchone()
ph = ""
for val in row:
ph += "?,"
params = params.append(str(val))
ph = ph[:-1]
cph = cph[:-1]
print(str(table[1]))
sqlcommand = "INSERT INTO "+str(table[1])+" ("+cph+") VALUES ("+ph+");"
print(sqlcommand)
print(params)
newDB.execute(sqlcommand, params)
sqlcommand = "DELETE FROM ? WHERE ? = ?;"
oldDB.execute(sqlcommand, (str(table[1]), cols[0], vals[0],))
newDB.commit()
Unbeknownst to me, column names can't be passed as parameters. Panagiotis Kanavos answered this in a comment. I guess I'll have to figure out a different way to generate the queries. Thank you all very much, I appreciate it.
Here is a sample of python code:
query = "SELECT * FROM users WHERE id = 17"
cursor.execute(query)
row = cursor.fetchone()
In this example, the query is meant to return at most a single row; but sometimes I want to fetch a single row from a multiple-row result with the same code, for instance:
query = "SELECT * FROM users WHERE name LIKE 'foo%'"
cursor.execute(query)
row = cursor.fetchone()
In term of performance, is it better to explicitly write LIMIT 1?
No, the LIMIT clause is executed after the SELECT on the Result set. if you have much rows with the id = 17 , you must transfer a little bit more from the server to the client if you do not use LIMIT.
I want to know if a row exists already in one of my tables, in this case coll. In order to do this I played around with SQLite in the shell a little and stumbled upon SELECT EXISTS(SELECT 1 FROM coll WHERE ceeb="1234"). In SQLite this works perfectly and it returns either a 0 or a 1-- which is exactly what I wanted. So, with code in hand, I wrote up a quick Python script to see if I could get this to work for me before sticking it into my program. This is what I came up with:
import sqlite3
conn = sqlite3.connect('stu.db')
c = conn.cursor()
sceeb = int(raw_input(":> "))
ceeb_exists = c.execute('SELECT EXISTS(SELECT 1 FROM coll WHERE ceeb="%d" LIMIT 1)' % sceeb)
print ceeb_exists
Instead of assigning ceeb_existsa 1 or a 0 it gives me an output that looks like <sqlite3.Cursor object at 0x01DF6860>. What am I doing wrong here?
The execution of a query always results in 0 or more rows. You'd need to fetch those rows; a SELECT EXISTS query results in 1 row, so you'd need to fetch that row.
Rows always consist of 1 or more columns, here you get one, so you could use tuple assignment (note the , comma after ceeb_exists):
c.execute('SELECT EXISTS(SELECT 1 FROM coll WHERE ceeb="%d" LIMIT 1)' % sceeb)
ceeb_exists, = c.fetchone()
However, using an EXISTS query is a bit redundant here; you could just test if there is any row returned. You should also use query parameters to avoid a SQL injection attack (you are asking a user to give you the ceeb value, so that is easily hijacked):
c.execute('SELECT 1 FROM coll WHERE ceeb=? LIMIT 1', (sceeb,))
ceeb_exists = c.fetchone() is not None
cursor.fetchone() returns None if there is no row available to fetch, the is not None test turns that into True or False.
.executes() returns a cursor object as you can see.
In order to print the results of the query you need to iterate over it:
for result in exists:
print result