I have started a small project yesterday in python and finally managed to make a database selection function that works, but I was wondering if anybody could tell me if the way I wrote it is good or could eventually end up in multiple problems.
The idea was to make a function that I could call to request/update a table where I would store players data for a small online game I want to create via Python, Pygame and MySQL.Connector-Python
So, here is the function code (I have tried to keep it as clean as possible and as intuitive as I could with my current knowledge of Python which is limited currently as I just picked it back up this week.)
The part I am not sure is the select_statement variable where I do not know for sure if the way I used concatenation is okay or if there is a way as simple and efficient.
def db_select(selection, table):
dbc = db_connect()
if dbc:
print("The SQL connection was successful.")
else:
print("The SQL connection could not be established.")
cur = dbc.cursor()
select_statement = "SELECT * FROM " + table + " WHERE id = %(id)s"
cur.execute(select_statement, {'id': selection})
print(cur.fetchone()[1])
dbc.close()
print("The SQL connection was closed successfully.")
Python has the .format()syntax just for these kinds of situations - handling a dynamic value (such as table being inserted into a string (including SQL queries).
query = "SELECT * FROM {} WHERE id = %(id)s".format(table)
It's cleaner and more reliable in my daily use than the prior approaches such as yours.
Related
I'm pretty new to python and to flask, and I've been trying to make a program where people can store their receipts. I've got three tables, users, receipts, and user_receipts. im trying to make a page where it lists all the receipts from the user thats logged in, however im just getting blank tables every time, even if the user has entered a receipt.
def list():
global user_id
con = sql.connect("database.db")
con.row_factory = sql.Row
cur = con.cursor()
cur.execute("select * from receipts WHERE receiptID = (select receipt_id from user_receipts WHERE user_id =?)", (user_id,))
rows = cur.fetchall();
return render_template("view_receipts.html",rows = rows)
I know that my methods for naming my variables are not consistent, however I'm aware of which variables belong to which table and I am 100 percent sure they are right.
Im just not sure why its giving me empty tables every time.
I recommend first checking if it is a problem with flask or SQLite first by doing a simple print statement for debugging.
If its a problem with SQLite I recommend to try to modify the variable names and see if there was an error while making the table
if all else fails try to move to something like sqlalchemy so you can get a more reliable database.
I have an SQL database "Garage.db" that has 3 tables:
Customer, Car and MOT
I want to update the field BookedMOT in the MOT table when someone has entered a Registration that is in the Car table. Can someone help me with the SQL query that can do this, thank you.
I am coding this in python 3.6 using tkinter. Here is my attempt,
def Booked(self):
date = Date.get()
month = Month.get()
year = Year.get()
BookedMOT = (date + '/' + month + '/' + year)
Registration = self.RegistrationEnt.get()
with sqlite3.connect('Garage.db') as db:
cursor = db.cursor()
add_date = ('UPDATE MOT SET MOT.BookedMOT = ? FROM Car WHERE Car.Registration = ?')
cursor.execute(add_date,[(BookedMOT), (Registration)])
db.commit()
(this addresses some Python problems I noticed before even realising that the SQL didn't look right, which should probably be fixed first)
Try this:
with sqlite3.connect('Garage.db') as db:
db.execute(add_date, (BookedMOT, Registration))
In general, when you say with ... as x:, you should probably use x inside the with block. After the block finished it did an automatic commit and trying to use db or cursor afterwards is probably incorrect. The with also means that you don't have to db.commit() any more.
sqlite3 connection objects (db in this case) have an execute method:
This is a nonstandard shortcut that creates a cursor object by calling the cursor() method, calls the cursor’s execute() method with the parameters given, and returns the cursor.
Finally, you had some redundant parentheses that could be removed.
I'm relatively new at Python but have more experience in Java, so I understand most of the concepts. However, I keep having issues with MySQL and passing information back from a function using MySQL
to use in another function later.
I need to make complex MySQL queries with multiple return field, So I don't want to be running multiple SQL queries for each SQL field as this will smash the database.
Saying that the below is a small example of what I'm trying to achieve.
I wanted a function to run an SQL query (def connectory_mysql()) that took parameters from elsewhere, (this part works) then take the output of the SQL query and pass back to the main function to use.
The main function then needs to use the different column results of the SQL query for different parameters.
I can return the result and assign it to a result1 which appears and looks like a dictionary when printed, but I'm unable to split/use the different keys or data from the result1 = connector_mysql(subSerialNum, ldev, today_date)
If i splituse the keys in the dictionary in the SQL function before returning ie ldev_cap = result['ldev_cap']
I can print the individual elements within the SQL function... However, I cant seem to pass the parameters then back to the main function and split them out??
I must have missed something easy or am not understanding something... any assistance or help would be greatly appreciated...
...
result1 = connector_mysql(subSerialNum, ldev, today_date)
print(result1) #this works and appears to be a dictionary, but I can split it
## into its elements like:
ldev_cap = result1['ldev_cap'] ##-> this dosn't work here.....???? if i return it as a dictonary..
#and i'm unsure how to split them when i pass just the paramaters
#back if i split the dictionary in the sql function.
...
def connector_mysql(subSerialNum, ldev, today_date):
import pymysql.cursors
db_server = 'localhost'
db_name = 'CBDB'
db_pass = 'secure_password'
db_user = 'user1'
sql_query = (
"SELECT ldev_cap, ldev_usdcap FROM Ldevs WHERE sub_serial=%(serial)s "
"and ldev_id=%(lun)s and data_date=%(todayD)s")
connection = pymysql.connect(host=db_server,
user=db_user,
password=db_pass,
db=db_name,
cursorclass=pymysql.cursors.DictCursor)
try:
with connection.cursor() as cursor:
cursor.execute(sql_query, {'serial': subSerialNum, 'lun': ldev, 'todayD': today_date})
result = cursor.fetchone()
while result:
ldev_cap = result['ldev_cap'] #here the dictionary acts as
#expected and i can assign a value
ldev_usdcap = result['ldev_usdcap']
print(result)
return ldev_cap, ldev_usdcap #here i can return
finally:
connection.close()
Any help or assistance would be greatly apricated...
Cheers
Graham
First of all, you should get familiar with the Python style guide for writing python code.
Based on your existing code, result1 return as a tuple that contained the value of (ldev_cap, ldef_usdcap) (it is not a directory). You get access to the return result as result1[0] which corresponding to the return value of ldev_cap or result1[1] which corresponding to the return value of ldev_usdcap.
Alternatively, since you are returning two data, you can access each return data by using
ldev_cap, ldev_usdcap = connector_mysql(subSerialNum, ldev, today_date)
This question already has answers here:
Python MySQLdb string substitution without added quotations
(2 answers)
Closed 9 years ago.
In Python I'm passing in two variables to MySQL, and the second variable is referenced as {0} and works correctly.
How can I make the first parameter do the same thing ? If '{0}' is the second variable. What is the First ? dbname is incorrect, I need to show dbname's value in this string ?
def checkTableExists(dbname,tablename):
sql2 = """SELECT COUNT(*) FROM information_schema.tables
WHERE table_schema = " dbname "
AND table_name = '{0}' """.format(tablename.replace('\'', '\'\''))
A better way to organize your function to create the query might be:
def checkTableExists(dbname, tablename):
query_args = {"table_name":tablename, "database_name":dbname}
exist_query = """
SELECT EXISTS (
SELECT *
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME = '%(table_name)s'
AND TABLE_SCHEMA = '%(database_name)s') AS `Exist`
"""%(query_args)
# Do stuff with the `Exist` column of whatever your db connection returns.
# Should be 1 if True. Might be good to just return bool(...).
This is what EXISTS is for, so you don't need to do a "hack" and check yourself if the COUNT within INFORMATION_SCHEMA.TABLES is greater than 0.
As #jgranger commented on the OP, if you are using MySQLdb, you can follow that link and let it do the argument substitution. But there are many other flavors of interaction with pyodbc, so string-formatted queries will never go away (I wish they would).
Even Better
Write a stored procedure in SQL that checks if tables exist as a function of the database name and schema name, and then do nothing but pass arguments to this stored procedure. Then any component of the system (Python or otherwise) can expect that stored procedure's interface for existence checking.
If the existence-check logic needs to change (say in your application it suddenly matters that a table has more than 0 rows rather than merely having been created with a CREATE TABLE command, or vice versa), then you can change the internals of the stored procedure without needing to go to every downstream location that had embedded string SQL queries and change the logic. It helps add modularity and encapsulation to the system's interaction with the databases.
This isn't a question, so much as a pre-emptive answer. (I have gotten lots of help from this website & wanted to give back.)
I was struggling with a large bit of SQL query that was failing when I tried to run it via python using pymssql, but would run fine when directly through MS SQL. (E.g., in my case, I was using MS SQL Server Management Studio to run it outside of python.)
Then I finally discovered the problem: pymssql cannot handle temporary tables. At least not my version, which is still 1.0.1.
As proof, here is a snippet of my code, slightly altered to protect any IP issues:
conn = pymssql.connect(host=sqlServer, user=sqlID, password=sqlPwd, \
database=sqlDB)
cur = conn.cursor()
cur.execute(testQuery)
The above code FAILS (returns no data, to be specific, and spits the error "pymssql.OperationalError: No data available." if you call cur.fetchone() ) if I call it with testQuery defined as below:
testQuery = """
CREATE TABLE #TEST (
[sample_id] varchar (256)
,[blah] varchar (256) )
INSERT INTO #TEST
SELECT DISTINCT
[sample_id]
,[blah]
FROM [myTableOI]
WHERE [Shipment Type] in ('test')
SELECT * FROM #TEST
"""
However, it works fine if testQuery is defined as below.
testQuery = """
SELECT DISTINCT
[sample_id]
,[blah]
FROM [myTableOI]
WHERE [Shipment Type] in ('test')
"""
I did a Google search as well as a search within Stack Overflow, and couldn't find any information regarding the particular issue. I also looked under the pymssql documentation and FAQ, found at http://code.google.com/p/pymssql/wiki/FAQ, and did not see anything mentioning that temporary tables are not allowed. So I thought I'd add this "question".
Update: July 2016
The previously-accepted answer is no longer valid. The second "will NOT work" example does indeed work with pymssql 2.1.1 under Python 2.7.11 (once conn.autocommit(1) is replaced with conn.autocommit(True) to avoid "TypeError: Cannot convert int to bool").
For those who run across this question and might have similar problems, I thought I'd pass on what I'd learned since the original post. It turns out that you CAN use temporary tables in pymssql, but you have to be very careful in how you handle commits.
I'll first explain by example. The following code WILL work:
testQuery = """
CREATE TABLE #TEST (
[name] varchar(256)
,[age] int )
INSERT INTO #TEST
values ('Mike', 12)
,('someone else', 904)
"""
conn = pymssql.connect(host=sqlServer, user=sqlID, password=sqlPwd, \
database=sqlDB) ## obviously setting up proper variables here...
conn.autocommit(1)
cur = conn.cursor()
cur.execute(testQuery)
cur.execute("SELECT * FROM #TEST")
tmp = cur.fetchone()
tmp
This will then return the first item (a subsequent fetch will return the other):
('Mike', 12)
But the following will NOT work
testQuery = """
CREATE TABLE #TEST (
[name] varchar(256)
,[age] int )
INSERT INTO #TEST
values ('Mike', 12)
,('someone else', 904)
SELECT * FROM #TEST
"""
conn = pymssql.connect(host=sqlServer, user=sqlID, password=sqlPwd, \
database=sqlDB) ## obviously setting up proper variables here...
conn.autocommit(1)
cur = conn.cursor()
cur.execute(testQuery)
tmp = cur.fetchone()
tmp
This will fail saying "pymssql.OperationalError: No data available." The reason, as best I can tell, is that whether you have autocommit on or not, and whether you specifically make a commit yourself or not, all tables must explicitly be created AND COMMITTED before trying to read from them.
In the first case, you'll notice that there are two "cur.execute(...)" calls. The first one creates the temporary table. Upon finishing the "cur.execute()", since autocommit is turned on, the SQL script is committed, the temporary table is made. Then another cur.execute() is called to read from that table. In the second case, I attempt to create & read from the table "simultaneously" (at least in the mind of pymssql... it works fine in MS SQL Server Management Studio). Since the table has not previously been made & committed, I cannot query into it.
Wow... that was a hassle to discover, and it will be a hassle to adjust my code (developed on MS SQL Server Management Studio at first) so that it will work within a script. Oh well...