My issue came when I decided to make a method that could handle a variation of queries, instead of coding 3 methods. I wanted to do so as to recycle code.
I have this table:
(I created it in the purpose of this question. You can do it by:
create table example (id int(1), ex1 varchar(15), ex2 varchar(15), ex3 varchar(15));
insert into example values(1, 'whatever11', 'whatever12', 'whatever13');
insert into example values(2, 'whatever21', 'whatever22', 'whatever23');
insert into example values(3, 'whatever31', 'whatever32', 'whatever33');
SO: I was trying to parameterize column names. I have done it in the whereclause all the time, but as I mention earlier, I thought it would be cleaner and more optimal to do just one method ( select %s from example where id=%s), instead of 3 different: (select ex1 from etc, select ex2 from etc.
So I tried this:
So the normal method is this:
def getex1(id):
myCursor=mydb.cursor()
query='select ex1 from example where id=%s'
myCursor.execute(query, (id,))
result=myCursor.fetchone()[0]
print(result) #prints 'whatever11 if id=1'
When I searched how to do parameterized queries, I saw that to do various parameters, you can just do something like input=(param1, param2, and then execute by (query, input), so I tried to do so but with the column name:
here, info is 'ex1', 'ex2' or 'ex3':
def getAllFromExample(info, id):
myCursor = mydb.cursor()
query= 'select %s from example where id=%s'
input = (info, id)
myCursor.execute(query, input)
result = myCursor.fetchone()[0]
print(result) #in this case, prints the column names 'ex1', 'ex2', not the content
My guess is that you can't just do the param by columns, because you are not assigning a value (like in a whereor in a group by, you have an assignment: whatever=value).
Any insights on this? I did quite the research but did not find anything. here it is mentioned this.
Anything you see wrong with the question, ask me and I'll make it clearer!
You cannot parametrizied the table names, you only can do it with the column values, so you would have to do:
def getAllFromExample(info, DNI):
myCursor = mydb.cursor()
query= 'select '+info+' from example where id=%s'
input = (id,)
myCursor.execute(query, input)
result = myCursor.fetchone()[0]
print(result) #in this case, prints the column name
Related
Okay so i have a table which has student id and the student id is used as identifier to edit the column but what if the same student lends a book twice then all the student value will b edited which i don't want....i want the last entered data of student id to b edited and using a Sl.No is not a solution here because its practically complicated.I am using python connector. Please help :) Thanks in advance
code i use right now :
con = mysql.connect(host='localhost', user='root',
password='monkey123', database='BOOK')
c = con.cursor()
c.execute(
f"UPDATE library set `status`='Returned',`date returned`='{str(cal.selection_get())}' WHERE `STUDENT ID`='{e_sch.get()}';")
c.execute('commit')
con.close()
messagebox.showinfo(
'Success', 'Book has been returned successfully')
If I followed you correctly, you want to update just one record that matches the where condition. For this to be done in a reliable manner, you need a column to define the ordering of the records. It could be a date, an incrementing id, or else. I assume that such column exists in your table and is called ordering_column.
A simple option is to use ORDER BY and LIMIT in the UPDATE statement, like so:
sql = """
UPDATE library
SET status = 'Returned', date returned = %s
WHERE student_id = %s
ORDER BY ordering_column DESC
LIMIT 1
"""
c = con.cursor()
c.execute(sql, (str(cal.selection_get()), e_sch.get(), )
Note that I modified your code so input values are given as parameters rather than concatenated into the query string. This is an important change, that makes your code safer and more efficient.
This question already has answers here:
How can I get dict from sqlite query?
(16 answers)
Closed 4 years ago.
Issue:
Hi, right now I am making queries to sqlite and assigning the result to variables like this:
Table structure: rowid, name, something
cursor.execute("SELECT * FROM my_table WHERE my_condition = 'ExampleForSO'")
found_record = cursor.fetchone()
record_id = found_record[0]
record_name = found_record[1]
record_something = found_record[2]
print(record_name)
However, it's very possible that someday I have to add a new column to the table. Let's put the example of adding that column:
Table structure: rowid, age, name, something
In that scenario, if we run the same code, name and something will be assigned wrongly and the print will not get me the name but the age, so I have to edit the code manually to fit the current index. However, I am working now with tables of more than 100 fields for a complex UI and doing this is tiresome.
Desired output:
I am wondering if there is a better way to catch results by using dicts or something like this:
Note for lurkers: The next snipped is made up code that does not works, do not use it.
cursor.execute_to(my_dict,
'''SELECT rowid as my_dict["id"],
name as my_dict["name"],
something as my_dict["something"]
FROM my_table WHERE my_condition = "ExampleForSO"''')
print(my_dict['name'])
I am probably wrong with this approach, but that's close to what I want. That way if I don't access the results as an index, and if add a new column, no matter where it's, the output would be the same.
What is the correct way to achieve it? Is there any other alternatives?
You can use namedtuple and then specify connection.row_factory in sqlite. Example:
import sqlite3
from collections import namedtuple
# specify my row structure using namedtuple
MyRecord = namedtuple('MyRecord', 'record_id record_name record_something')
con = sqlite3.connect(":memory:")
con.isolation_level = None
con.row_factory = lambda cursor, row: MyRecord(*row)
cur = con.cursor()
cur.execute("CREATE TABLE my_table (record_id integer PRIMARY KEY, record_name text NOT NULL, record_something text NOT NULL)")
cur.execute("INSERT INTO my_table (record_name, record_something) VALUES (?, ?)", ('Andrej', 'This is something'))
cur.execute("INSERT INTO my_table (record_name, record_something) VALUES (?, ?)", ('Andrej', 'This is something too'))
cur.execute("INSERT INTO my_table (record_name, record_something) VALUES (?, ?)", ('Adrika', 'This is new!'))
for row in cur.execute("SELECT * FROM my_table WHERE record_name LIKE 'A%'"):
print(f'ID={row.record_id} NAME={row.record_name} SOMETHING={row.record_something}')
con.close()
Prints:
ID=1 NAME=Andrej SOMETHING=This is something
ID=2 NAME=Andrej SOMETHING=This is something too
ID=3 NAME=Adrika SOMETHING=This is new!
I have a problem programming a sqlite3 database in Python.
So I made two lists:
idata=[(0,"Ingredient1"),
(1,"Ingredient2")]
This is the first one that holds "Ingredients" and their ID's.
rdata=[(0,"Recipie1",0,1,1)]
And this is the second one that holds "Recipies" their ID's and and three numbers that indicate the ID of the "Ingredients" to be used in this "Recipie".
Then I created two tables that I filled with the data of these lists:
import sqlite3
conn = sqlite3.connect ("Alchemy_Data_Bank.dat")
c = conn.cursor()
c.execute("""
CREATE TABLE IF NOT EXISTS recipie(id, name, iid_1, iid_2, iid_3);
""")
c.executemany("insert into recipie(id, name, iid_1, iid_2, iid_3) values (?,?,?,?,?)", rdata)
c.execute("""
CREATE TABLE IF NOT EXISTS ingredient(id, name);
""")
c.executemany("insert into ingredient(id, name) values (?,?)", idata)
conn.commit()
And now I want to print out the "Recipies" together with their "Ingredients" combined in a table. So I did this:
for p in c.execute("""SELECT DISTINCT recipie.name,
CASE WHEN recipie.iid_1 = ingredient.id THEN ingredient.name end,
CASE WHEN recipie.iid_2 = ingredient.id THEN ingredient.name end,
CASE WHEN recipie.iid_3 = ingredient.id THEN ingredient.name end
FROM recipie, ingredient;"""):
print(p)
c.close()
conn.close()
What I hoped to get as output is somethin like this:
('Recipie1','Ingredient1', 'Ingredient2', 'Ingredient2')
But it printed this:
('Recipie1', None, None, None)
('Recipie1', None, 'Ingredient2', 'Ingredient2')
('Recipie1', 'Ingedient1', None, None)
I think that my problem lies within the CASE WHEN statments as the programm compares recipie.iid_1, recipie.iid_2 and recipie.iid_3 only with one value for ingredient.id at a time.
So far as I've come the solution must be recursive slection in each CASE WHEN statment but I just can't figure out how to do that.
I hope that someone of you can tell me how to do that!
Thanks in advance!!
Cazo0
Try to rewrite a query. e.g.:
qry1 = """select name,
(select name from ingredient where ingredient.id = recipie.iid_1),
(select name from ingredient where ingredient.id = recipie.iid_2),
(select name from ingredient where ingredient.id = recipie.iid_3)
from recipie;"""
rsl = c.execute(qry1)
for r in rsl:
print (r)
see my gist for the whole code:
https://gist.github.com/mh70cz/5cfa595b455e87d7c08da5315b1abd21
I want to read all of the tables contained in a database into pandas data frames. This answer does what I want to accomplish, but I'd like to use the DBAPI syntax with the ? instead of the %s, per the documentation. However, I ran into an error. I thought this answer may address the problem, but I'm now posting my own question because I can't figure it out.
Minimal example
import pandas as pd
import sqlite3
pd.__version__ # 0.19.1
sqlite3.version # 2.6.0
excon = sqlite3.connect('example.db')
c = excon.cursor()
c.execute('''CREATE TABLE stocks
(date text, trans text, symbol text, qty real, price real)''')
c.execute("INSERT INTO stocks VALUES ('2006-01-05', 'BUY', 'RHAT', 100, 35.14)")
c.execute('''CREATE TABLE bonds
(date text, trans text, symbol text, qty real, price real)''')
c.execute("INSERT INTO bonds VALUES ('2015-01-01', 'BUY', 'RSOCK', 90, 23.11)")
data = pd.read_sql_query('SELECT * FROM stocks', excon)
# >>> data
# date trans symbol qty price
# 0 2006-01-05 BUY RHAT 100.0 35.14
But when I include a ? or a (?) as below, I get the error message pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT * FROM (?)': near "?": syntax error.
Problem code
c.execute("SELECT name FROM sqlite_master WHERE type='table';")
tables = c.fetchall()
# >>> tables
# [('stocks',), ('bonds',)]
table = tables[0]
data = pd.read_sql_query("SELECT * FROM ?", excon, params=table)
It's probably something trivial that I'm missing, but I'm not seeing it!
The problem is that you're trying to use parameter substitution for a table name, which is not possible. There's an issue on GitHub that discusses this. The relevant part is at the very end of the thread, in a comment by #jorisvandenbossche:
Parameter substitution is not possible for the table name AFAIK.
The thing is, in sql there is often a difference between string
quoting, and variable quoting (see eg
https://sqlite.org/lang_keywords.html the difference in quoting
between string and identifier). So you are filling in a string, which
is for sql something else as a variable name (in this case a table
name).
Parameter substitution is essential to prevent SQL Injection from unsafe user-entered values.
In this particular example you are sourcing table names directly from the database's own metadata, which is already safe, so it's OK to just use normal string formatting to construct the query, but still good to wrap the table names in quotes.
If you are sourcing user-entered table names, you can also parameterize them first before using them in your normal python string formatting.
e.g.
# assume this is user-entered:
table = '; select * from members; DROP members --'
c.execute("SELECT name FROM sqlite_master WHERE type='table' and name = ?;", excon, params=table )
tables = c.fetchall()
In this case the user has entered some malicious input intended to cause havoc, and the parameterized query will cleanse it and the query will return no rows.
If the user entered a clean table e.g. table = 'stocks' then the above query would return that same name back to you, through the wash, and it is now safe.
Then it is fine to continue with normal python string formatting, in this case using f-string style:
table = tables[0]
data = pd.read_sql_query(f"""SELECT * FROM "{table}" ;""", excon)
Referring back to your original example, my first step above is entirely unnecessary. I just provided it for context. It is unnecessary, because there is no user input so you could just do something like this to get a dictionary of dataframes for every table.
c.execute("SELECT name FROM sqlite_master WHERE type='table';")
tables = c.fetchall()
# >>> tables
# [('stocks',), ('bonds',)]
dfs = dict()
for t in tables:
dfs[t] = pd.read_sql_query(f"""SELECT * FROM "{t}" ;""", excon)
Then you can fetch the dataframe from the dictionary using the tablename as the key.
I'm practicing in SQLite and Python. I'm trying to build a TABLE using only user prompts as database objects. After some extensive searches (official documentation says nothing about this kind of syntax-please correct me!) I found this method:
new_table = raw_input('Enter a table name: ')
column = raw_input('Enter column name: ')
cur.execute(''' CREATE TABLE IF NOT EXISTS {tn} ({col})'''\
.format(tn = new_table, col = column))
It works very nice and I find it intuitive. My problem is with INSERT INTO syntax. While the following code works ok:
cur.execute("INSERT INTO {tn} ({col}) VALUES (?)", ('goodmorning')\
.format(tn=new_table, col=column))
This code below, won't work:
insdata = raw_input('Insert data for column: ')
cur.execute("INSERT INTO {tn} ({col}) VALUES (?)", (insdata,)\
.format(tn=new_table, col=column))
and fails with error: 'tuple' object has no attribute format.
Question is: what is the proper syntax to assign insdata value to SQLite VALUES?
If you write this is a slightly clearer fashion, you'll see what's going on:
cur.execute(
"INSERT INTO {tn} ({col}) VALUES (?)",
(insdata,).format(tn=new_table, col=column)
)
You're not formatting the string, you're formatting the a tuple of arguments. Instead, you want:
cur.execute(
"INSERT INTO {tn} ({col}) VALUES (?)".format(tn=new_table, col=column),
(insdata,)
)
or perhaps a little more clearly :
sql = "INSERT INTO {tn} ({col}) VALUES (?)".format(tn=new_table, col=column)
cur.execute(sql, (insdata,))
In this case your line continuation character is not needed at all (since you're inside a function call) but if it were needed it would make much more sense to position it between arguments rather than between an object and the method invocation on the object.
I think you are invoking format method of tuple (which appears not to have one) instead of a string with SQL query:
cur.execute("INSERT INTO {tn} ({col}) VALUES ({val})".format(tn=new_table,col=column,val='goodmorning'))