I have an SQL database "Garage.db" that has 3 tables:
Customer, Car and MOT
I want to update the field BookedMOT in the MOT table when someone has entered a Registration that is in the Car table. Can someone help me with the SQL query that can do this, thank you.
I am coding this in python 3.6 using tkinter. Here is my attempt,
def Booked(self):
date = Date.get()
month = Month.get()
year = Year.get()
BookedMOT = (date + '/' + month + '/' + year)
Registration = self.RegistrationEnt.get()
with sqlite3.connect('Garage.db') as db:
cursor = db.cursor()
add_date = ('UPDATE MOT SET MOT.BookedMOT = ? FROM Car WHERE Car.Registration = ?')
cursor.execute(add_date,[(BookedMOT), (Registration)])
db.commit()
(this addresses some Python problems I noticed before even realising that the SQL didn't look right, which should probably be fixed first)
Try this:
with sqlite3.connect('Garage.db') as db:
db.execute(add_date, (BookedMOT, Registration))
In general, when you say with ... as x:, you should probably use x inside the with block. After the block finished it did an automatic commit and trying to use db or cursor afterwards is probably incorrect. The with also means that you don't have to db.commit() any more.
sqlite3 connection objects (db in this case) have an execute method:
This is a nonstandard shortcut that creates a cursor object by calling the cursor() method, calls the cursor’s execute() method with the parameters given, and returns the cursor.
Finally, you had some redundant parentheses that could be removed.
Related
As mentioned above I need to query several databases with all the same schema using pandas.read_sql_query. I've tried to create loop over all the dbs and run on the fly the sql statement that would result in something like this:
USE db_test_1; SELECT * from test
That's what I've done so far:
cursor = conn.cursor()
cursor.execute("SHOW DATABASES LIKE '%test_%'")
cursor.close()
dbs = [v.replace("'", "") for (v, ) in cursor]
for db in dbs[:100]:
temp = "USE " + db + ";"
fd = open('my_query.sql')
query = fd.read()
fd.close
sql = temp + query
data = pd.read_sql_query(sql, conn)
print(data)
Gives an error saying that the mysql syntax is wrong. Do you have any idea how to handle it or point me to the error?
Many thanks
Your problem lies with your my_query.sql file.
SELECT (SELECT * from tab1), (SELECT * from tab2)
The above is not valid SQL; a subselect can only return a single column. To do this, you would need to join the two subselects in the FROM clause. Which columns you do this on will be entirely dependent on your schema and the needed relation.
Update:
Okay, so the problem here seems to be more about how you're dealing with the query. Your cursor object is connected to a single database. Not the entire database server.
That means that your cursor object cannot use the use keyword here. You need to create a new connection and cursor object for each database you want to connect to.
Question: Is it possible to use a variable as your table name without having to use string constructors to do so?
Info:
I'm working on a project right now that catalogs data from a star simulation of mine. To do so I'm loading all the data into a sqlite database. It's working pretty well, but I've decided to add a lot more flexibility, efficiency, and usability to my db. I plan on later adding planetoids to the simulation, and wanted to have a table for each star. This way I wouldn't have to query a table of 20m some planetoids for the 1-4k in each solar system.
I've been told using string constructors is bad because it leaves me vulnerable to a SQL injection attack. While that isn't a big deal here as I'm the only person with access to these dbs, I would like to follow best practices. And also this way if I do a project with a similar situation where it is open to the public, I know what to do.
Currently I'm doing this:
cursor.execute("CREATE TABLE StarFrame"+self.name+" (etc etc)")
This works, but I would like to do something more like:
cursor.execute("CREATE TABLE StarFrame(?) (etc etc)",self.name)
though I understand that this would probably be impossible. though I would settle for something like
cursor.execute("CREATE TABLE (?) (etc etc)",self.name)
If this is not at all possible, I'll accept that answer, but if anyone knows a way to do this, do tell. :)
I'm coding in python.
Unfortunately, tables can't be the target of parameter substitution (I didn't find any definitive source, but I have seen it on a few web forums).
If you are worried about injection (you probably should be), you can write a function that cleans the string before passing it. Since you are looking for just a table name, you should be safe just accepting alphanumerics, stripping out all punctuation, such as )(][;, and whitespace. Basically, just keep A-Z a-z 0-9.
def scrub(table_name):
return ''.join( chr for chr in table_name if chr.isalnum() )
scrub('); drop tables --') # returns 'droptables'
For people searching for a way to make the table as a variable, I got this from another reply to same question here:
It said the following and it works. It's all quoted from mhawke:
You can't use parameter substitution for the table name. You need to add the table name to the query string yourself. Something like this:
query = 'SELECT * FROM {}'.format(table)
c.execute(query)
One thing to be mindful of is the source of the value for the table name. If that comes from an untrusted source, e.g. a user, then you need to validate the table name to avoid potential SQL injection attacks. One way might be to construct a parameterised query that looks up the table name from the DB catalogue:
import sqlite3
def exists_table(db, name):
query = "SELECT 1 FROM sqlite_master WHERE type='table' and name = ?"
return db.execute(query, (name,)).fetchone() is not None
I wouldn't separate the data into more than one table. If you create an index on the star column, you won't have any problem efficiently accessing the data.
Try with string formatting:
sql_cmd = '''CREATE TABLE {}(id, column1, column2, column2)'''.format(
'table_name')
db.execute(sql_cmd)
Replace 'table_name' with your desire.
To avoid hard-coding table names, I've used:
table = "sometable"
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS {} (
importantdate DATE,
somename VARCHAR,
)'''.format(table))
c.execute('''INSERT INTO {} VALUES (?, ?)'''.format(table),
(datetime.strftime(datetime.today(), "%Y-%m-%d"),
myname))
As has been said in the other answers, "tables can't be the target of parameter substitution" but if you find yourself in a bind where you have no option, here is a method of testing if the table name supplied is valid.
Note: I have made the table name a real pig in an attempt to cover all of the bases.
import sys
import sqlite3
def delim(s):
delims="\"'`"
use_delim = []
for d in delims:
if d not in s:
use_delim.append(d)
return use_delim
db_name = "some.db"
db = sqlite3.connect(db_name)
mycursor = db.cursor()
table = 'so""m ][ `etable'
delimiters = delim(table)
if len(delimiters) < 1:
print "The name of the database will not allow this!"
sys.exit()
use_delimiter = delimiters[0]
print "Using delimiter ", use_delimiter
mycursor.execute('SELECT name FROM sqlite_master where (name = ?)', [table])
row = mycursor.fetchall()
valid_table = False
if row:
print (table,"table name verified")
valid_table = True
else:
print (table,"Table name not in database", db_name)
if valid_table:
try:
mycursor.execute('insert into ' +use_delimiter+ table +use_delimiter+ ' (my_data,my_column_name) values (?,?) ',(1,"Name"));
db.commit()
except Exception as e:
print "Error:", str(e)
try:
mycursor.execute('UPDATE ' +use_delimiter+ table +use_delimiter+ ' set my_column_name = ? where my_data = ?', ["ReNamed",1])
db.commit()
except Exception as e:
print "Error:", str(e)
db.close()
you can use something like this
conn = sqlite3.connect()
createTable = '''CREATE TABLE %s (# );''' %dateNow)
conn.execute(createTable)
basically, if we want to separate the data into several tables according to the date right now, for example, you want to monitor a system based on the date.
createTable = '''CREATE TABLE %s (# );''' %dateNow) means that you create a table with variable dateNow which according to your coding language, you can define dateNow as a variable to retrieve the current date from your coding language.
You can save your query in a .sql or txt file and use the open().replace() method to use variables in any part of your query. Long time reader but first time poster so I apologize if anything is off here.
```SQL in yoursql.sql```
Sel *
From yourdbschema.tablenm
```SQL to run```
tablenm = 'yourtablename'
cur = connect.cursor()
query = cur.execute(open(file = yoursql.sql).read().replace('tablenm',tablenm))
You can pass a string as the SQL command:
import sqlite3
conn = sqlite3.connect('db.db')
c = conn.cursor()
tablename, field_data = 'some_table','some_data'
query = 'SELECT * FROM '+tablename+' WHERE column1=\"'+field_data+"\""
c.execute(query)
Question: Is it possible to use a variable as your table name without having to use string constructors to do so?
Info:
I'm working on a project right now that catalogs data from a star simulation of mine. To do so I'm loading all the data into a sqlite database. It's working pretty well, but I've decided to add a lot more flexibility, efficiency, and usability to my db. I plan on later adding planetoids to the simulation, and wanted to have a table for each star. This way I wouldn't have to query a table of 20m some planetoids for the 1-4k in each solar system.
I've been told using string constructors is bad because it leaves me vulnerable to a SQL injection attack. While that isn't a big deal here as I'm the only person with access to these dbs, I would like to follow best practices. And also this way if I do a project with a similar situation where it is open to the public, I know what to do.
Currently I'm doing this:
cursor.execute("CREATE TABLE StarFrame"+self.name+" (etc etc)")
This works, but I would like to do something more like:
cursor.execute("CREATE TABLE StarFrame(?) (etc etc)",self.name)
though I understand that this would probably be impossible. though I would settle for something like
cursor.execute("CREATE TABLE (?) (etc etc)",self.name)
If this is not at all possible, I'll accept that answer, but if anyone knows a way to do this, do tell. :)
I'm coding in python.
Unfortunately, tables can't be the target of parameter substitution (I didn't find any definitive source, but I have seen it on a few web forums).
If you are worried about injection (you probably should be), you can write a function that cleans the string before passing it. Since you are looking for just a table name, you should be safe just accepting alphanumerics, stripping out all punctuation, such as )(][;, and whitespace. Basically, just keep A-Z a-z 0-9.
def scrub(table_name):
return ''.join( chr for chr in table_name if chr.isalnum() )
scrub('); drop tables --') # returns 'droptables'
For people searching for a way to make the table as a variable, I got this from another reply to same question here:
It said the following and it works. It's all quoted from mhawke:
You can't use parameter substitution for the table name. You need to add the table name to the query string yourself. Something like this:
query = 'SELECT * FROM {}'.format(table)
c.execute(query)
One thing to be mindful of is the source of the value for the table name. If that comes from an untrusted source, e.g. a user, then you need to validate the table name to avoid potential SQL injection attacks. One way might be to construct a parameterised query that looks up the table name from the DB catalogue:
import sqlite3
def exists_table(db, name):
query = "SELECT 1 FROM sqlite_master WHERE type='table' and name = ?"
return db.execute(query, (name,)).fetchone() is not None
I wouldn't separate the data into more than one table. If you create an index on the star column, you won't have any problem efficiently accessing the data.
Try with string formatting:
sql_cmd = '''CREATE TABLE {}(id, column1, column2, column2)'''.format(
'table_name')
db.execute(sql_cmd)
Replace 'table_name' with your desire.
To avoid hard-coding table names, I've used:
table = "sometable"
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS {} (
importantdate DATE,
somename VARCHAR,
)'''.format(table))
c.execute('''INSERT INTO {} VALUES (?, ?)'''.format(table),
(datetime.strftime(datetime.today(), "%Y-%m-%d"),
myname))
As has been said in the other answers, "tables can't be the target of parameter substitution" but if you find yourself in a bind where you have no option, here is a method of testing if the table name supplied is valid.
Note: I have made the table name a real pig in an attempt to cover all of the bases.
import sys
import sqlite3
def delim(s):
delims="\"'`"
use_delim = []
for d in delims:
if d not in s:
use_delim.append(d)
return use_delim
db_name = "some.db"
db = sqlite3.connect(db_name)
mycursor = db.cursor()
table = 'so""m ][ `etable'
delimiters = delim(table)
if len(delimiters) < 1:
print "The name of the database will not allow this!"
sys.exit()
use_delimiter = delimiters[0]
print "Using delimiter ", use_delimiter
mycursor.execute('SELECT name FROM sqlite_master where (name = ?)', [table])
row = mycursor.fetchall()
valid_table = False
if row:
print (table,"table name verified")
valid_table = True
else:
print (table,"Table name not in database", db_name)
if valid_table:
try:
mycursor.execute('insert into ' +use_delimiter+ table +use_delimiter+ ' (my_data,my_column_name) values (?,?) ',(1,"Name"));
db.commit()
except Exception as e:
print "Error:", str(e)
try:
mycursor.execute('UPDATE ' +use_delimiter+ table +use_delimiter+ ' set my_column_name = ? where my_data = ?', ["ReNamed",1])
db.commit()
except Exception as e:
print "Error:", str(e)
db.close()
you can use something like this
conn = sqlite3.connect()
createTable = '''CREATE TABLE %s (# );''' %dateNow)
conn.execute(createTable)
basically, if we want to separate the data into several tables according to the date right now, for example, you want to monitor a system based on the date.
createTable = '''CREATE TABLE %s (# );''' %dateNow) means that you create a table with variable dateNow which according to your coding language, you can define dateNow as a variable to retrieve the current date from your coding language.
You can save your query in a .sql or txt file and use the open().replace() method to use variables in any part of your query. Long time reader but first time poster so I apologize if anything is off here.
```SQL in yoursql.sql```
Sel *
From yourdbschema.tablenm
```SQL to run```
tablenm = 'yourtablename'
cur = connect.cursor()
query = cur.execute(open(file = yoursql.sql).read().replace('tablenm',tablenm))
You can pass a string as the SQL command:
import sqlite3
conn = sqlite3.connect('db.db')
c = conn.cursor()
tablename, field_data = 'some_table','some_data'
query = 'SELECT * FROM '+tablename+' WHERE column1=\"'+field_data+"\""
c.execute(query)
#Delete suspense window
class dWindow(QtGui.QMainWindow, Ui_dWindow):
def __init__(self, parent = None):
QtGui.QMainWindow.__init__(self, parent)
self.setupUi(self)
for row in cursor.execute("SELECT FIRSTNAME FROM Staff"):
self.comboUser.addItems(row)
con.close()
self.btnDeleteSuspense.clicked.connect(self.btnDeleteSuspense_Clicked)
def btnDeleteSuspense_Clicked(self):
user = self.comboUser.currentText() #finds selected user
date = self.dateEdit.date().toString("M/d/yyyy")
numrecord = cursor.execute() ??
Here is a sample DB and program file to further help me explain
I have created variables to hold the selection of the combobox and the dateEdit box.
The next step (the one I'm struggling with) is to then use those variables in an SQL query that will first find the count of rows with the selected user name and having a date <= than the selected date. That will populate the numrecord variable so that I can display a "This will delete 'x' rows, are you sure?"
If the user selects yes then I will then use the variable in a delete query to delete the selected rows.
I believe if I can figure out how to use the variables I have in a SQL query then the DELETE query should be rather simple.
An example of a possible DELETE query to show what I'm trying to do
cursor.execute("DELETE TO, count(*) FROM Suspense where TO = [user] and DATE = [date]")
I know that is wrong but maybe it will help clarify.
I hope I have explained my question fully and I appreciate any help provided.
Edit: Thanks so much!!
Just before I saw that you had posted this I figured it out.
What I came up with was the following:
qdate = self.dateTimeEdit.dateTime().toPyDateTime() #grabs the raw datetime from the QDateTimeEdit object and converts to python datetime
query = "SELECT DATE FROM Suspense WHERE DATE >= ?" #creates the query using ? as a placeholder for variable
cursor.execute(query, (qdate,)) #executes the query and passes qdate as a tuple to the placeholder
With this knowledge I can recreate my queries to include both variables.
As mentioned in a comment to another answer, you should be using a proper parameterized query, for example:
# assumes that autocommit=False (the default)
crsr = conn.cursor()
sql = "DELETE FROM [Suspense] WHERE [TO]=? AND [DATE]<=?"
user = self.comboUser.currentText() # as before
date = self.dateEdit.date() # Note: no .toString(...) required
params = (user, date)
crsr.execute(sql, params)
msg = "About to delete {} row(s). Proceed?".format(crsr.rowcount)
if my_confirmation_dialog(msg):
conn.commit()
else:
conn.rollback()
What I came up with was the following:
qdate = self.dateTimeEdit.dateTime().toPyDateTime() #grabs the raw datetime from the QDateTimeEdit object and converts to python datetime
query = "SELECT DATE FROM Suspense WHERE DATE >= ?" #creates the query using ? as a placeholder for variable
cursor.execute(query, (qdate,)) #executes the query and passes qdate as a tuple to the plac
With this knowledge I can now add both variables to the query as needed.
Thanks everyone for their help, especially Gord Thompson!
You use the DELETE sql command.
This assumes your DATE field is actually a date field and not a string field.
user = self.comboUser.currentText()
date = self.dateEdit.date().toString("yyyy-MM-dd")
cmd = "DELETE FROM Suspense WHERE TO = '{}' AND DATE >= '{}'".format(user, date)
cursor.execute(cmd)
Also, you may want to look into using an ORM framework (sqlalchemy is probably the most popular, but there are others). It's best to avoid manually constructing sql queries if possible.
I'm having trouble finding any information on how to do error checking on MySQLdb. I have been trying to do a simple update command for a MySQL database and it simply is not working. No matter how I change the terms, or the type of variables I submit to it.
Here are some of my (commented out) attempts:
timeid = twitseek['max_id']
#timeup = "UPDATE `timeid` set `timestamp`='" + str(timeid) + "';"
#print timeup
#c.execute(timeup)
#timeup = "UPDATE timeid SET timestamp=\"" + str(timeid) + "\"";
#timeup = "UPDATE timeid set timestamp = '500';"
timeup = 500
c.execute("""UPDATE timeid SET timestamp = %d;""", timeup)
#c.execute(timeup)
All I want to do is upload the value of timeid to the timestamp column's first value (or any value) in the table timeid.
Nothing I do seems to work and I've been sitting here for literally hours trying countless iterations.
You seem to be missing an obligatory call to .commit() on your connection object to commit your change.
# Your cursor is c
# We don't see your connection object, but assuming it is conn...
c.execute("""UPDATE timeid SET timestamp = %d;""", timeup)
conn.commit()
The above method will produce valid SQL, but you don't get the security benefit of prepared statements this way. The proper method to pass in parameters is to use %s, and pass in a tuple of parameters:
c.execute("UPDATE timeid SET timestamp = %s;", (timeup,))
conn.commit()
From the MySQLdb FAQ:
Starting with 1.2.0, MySQLdb disables autocommit by default, as
required by the DB-API standard (PEP-249). If you are using InnoDB
tables or some other type of transactional table type, you'll need to
do connection.commit() before closing the connection, or else none of
your changes will be written to the database.
Conversely, you can also use connection.rollback() to throw away any
changes you've made since the last commit.
As far as error checking goes, a failed connection or a syntactically invalid query will throw an exception. So you would want to wrap it in a try/except as is common in Python.