This question already has answers here:
Python MySQLdb string substitution without added quotations
(2 answers)
Closed 9 years ago.
In Python I'm passing in two variables to MySQL, and the second variable is referenced as {0} and works correctly.
How can I make the first parameter do the same thing ? If '{0}' is the second variable. What is the First ? dbname is incorrect, I need to show dbname's value in this string ?
def checkTableExists(dbname,tablename):
sql2 = """SELECT COUNT(*) FROM information_schema.tables
WHERE table_schema = " dbname "
AND table_name = '{0}' """.format(tablename.replace('\'', '\'\''))
A better way to organize your function to create the query might be:
def checkTableExists(dbname, tablename):
query_args = {"table_name":tablename, "database_name":dbname}
exist_query = """
SELECT EXISTS (
SELECT *
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME = '%(table_name)s'
AND TABLE_SCHEMA = '%(database_name)s') AS `Exist`
"""%(query_args)
# Do stuff with the `Exist` column of whatever your db connection returns.
# Should be 1 if True. Might be good to just return bool(...).
This is what EXISTS is for, so you don't need to do a "hack" and check yourself if the COUNT within INFORMATION_SCHEMA.TABLES is greater than 0.
As #jgranger commented on the OP, if you are using MySQLdb, you can follow that link and let it do the argument substitution. But there are many other flavors of interaction with pyodbc, so string-formatted queries will never go away (I wish they would).
Even Better
Write a stored procedure in SQL that checks if tables exist as a function of the database name and schema name, and then do nothing but pass arguments to this stored procedure. Then any component of the system (Python or otherwise) can expect that stored procedure's interface for existence checking.
If the existence-check logic needs to change (say in your application it suddenly matters that a table has more than 0 rows rather than merely having been created with a CREATE TABLE command, or vice versa), then you can change the internals of the stored procedure without needing to go to every downstream location that had embedded string SQL queries and change the logic. It helps add modularity and encapsulation to the system's interaction with the databases.
Related
This question already has answers here:
How to use variables in SQL statement in Python?
(5 answers)
Closed 2 months ago.
def update_inv_quant():
new_quant = int(input("Enter the updated quantity in stock: "))
Hello! I'm wondering how to insert a user variable into an sql statement so that a record is updated to said variable. Also, it'd be really helpful if you could also help me figure out how to print records of the database into the actual python console. Thank you!
I tried doing soemthing like ("INSERT INTO Inv(ItemName) Value {user_iname)") but i'm not surprised it didnt work
It would have been more helpful if you specified an actual database.
First method (Bad)
The usual way (which is highly discouraged as Graybeard said in the comments) is using python's f-string. You can google what it is and how to use it more in-depth.
but basically, say you have two variables user_id = 1 and user_name = 'fish', f-string turns something like f"INSERT INTO mytable(id, name) values({user_id},'{user_name}')" into the string INSERT INTO mytable(id,name) values(1,'fish').
As we mentioned before, this causes something called SQL injection. There are many good youtube videos that demonstrate what that is and why it's dangerous.
Second method
The second method is dependent on what database you are using. For example, in Psycopg2 (Driver for PostgreSQL database), the cursor.execute method uses the following syntax to pass variables cur.execute('SELECT id FROM users WHERE cookie_id = %s',(cookieid,)), notice that the variables are passed in a tuple as a second argument.
All databases use similar methods, with minor differences. For example, I believe SQLite3 uses ? instead of psycopg2's %s. That's why I said that specifying the actual database would have been more helpful.
Fetching records
I am most familiar with PostgreSQL and psycopg2, so you will have to read the docs of your database of choice.
To fetch records, you send the query with cursor.execute() like we said before, and then call cursor.fetchone() which returns a single row, or cursor.fetchall() which returns all rows in an iterable that you can directly print.
Execute didn't update the database?
Statements executing from drivers are transactional, which is a whole topic by itself that I am sure will find people on the internet who can explain it better than I can. To keep things short, for the statement to physically change the database, you call connection.commit() after cursor.execute()
So finally to answer both of your questions, read the documentation of the database's driver and look for the execute method.
This is what I do (which is for sqlite3 and would be similar for other SQL type databases):
Assuming that you have connected to the database and the table exists (otherwise you need to create the table). For the purpose of the example, i have used a table called trades.
new_quant = 1000
# insert one record (row)
command = f"""INSERT INTO trades VALUES (
'some_ticker', {new_quant}, other_values, ...
) """
cur.execute(command)
con.commit()
print('trade inserted !!')
You can then wrap the above into your function accordingly.
I have a sql query I'm executing that I'm passing variables into. In the current context I'm passing the parameter values in as f strings, but this query is vulnerable to sql injection. I know there is a method to use a stored procedure and restrict permissions on the user executing the query. But is there a way to avoid having to go the stored procedure route and perhaps modify this function to be secure against SQL Injection?
I have the below query created to execute within a python app.
def sql_gen(tv, kv, join_kv, col_inst, val_inst, val_upd):
sqlstmt = f"""
IF NOT EXISTS (
SELECT *
FROM {tv}
WHERE {kv} = {join_kv}
)
INSERT {tv} (
{col_inst}
)
VALUES (
{val_inst}
)
ELSE
UPDATE {tv}
SET {val_upd}
WHERE {kv} = {join_kv};
"""
engine = create_engine(f"mssql+pymssql://{username}:{password}#{server}/{database}")
connection = engine.raw_connection()
cursor = connection.cursor()
cursor.execute(sqlstmt)
connection.commit()
cursor.close()
Fortunately, most database connectors have query parameters in which you pass the variable instead of giving in the string inside the query yourself for the risks you mentioned.
You can read more on this here: https://realpython.com/prevent-python-sql-injection/#understanding-python-sql-injection
Example:
# Vulnerable
cursor.execute("SELECT admin FROM users WHERE username = '" + username + '");
# Safe
cursor.execute("SELECT admin FROM users WHERE username = %s'", (username, ));
As Amanzer mentions correctly in his reply Python has mechanisms to pass parameters safely.
However, there are other elements in your query (table names and column names) that are not supported as parameters (bind variables) because JDBC does not support those.
If these are from an untrusted source (or may be in the future) you should be sure you validate these elements. This is a good coding practice to do even if you are sure.
There are some options to do this safely:
You should limit your tables and columns based on positive validation - make sure that the only values allowed are the ones that are authorized
If that's not possible (because these are user created?):
You should make sure tables or column names limit the
names to use a "safe" set of characters (alphanumeric & dashes,
underscores...)
You should enquote the table names / column names -
adding double quotes around the objects. If you do this, you need to
be careful to validate there are no quotes in the name, and error out
or escape the quotes. You also need to be aware that adding quotes
will make the name case sensitive.
I have a function that executes many SQL queries with different dates.
What I want is to pass all dates and other query variables as function parameters and then just execute the function. I have figured out how to do this for datetime variables as below. But I also have a query that looks at specific campaign_names in a database and pulls those as strings. I want to be able to pass those strings as function parameters but I haven't figured out the correct syntax for this in the SQL query.
def Camp_eval(start_date,end_1M,camp1,camp2,camp3):
query1 = f"""SELECT CONTACT_NUMBER, OUTCOME_DATE
FROM DATABASE1
where OUTCOME_DATE >= (to_date('{start_date}', 'dd/mm/yyyy'))
and OUTCOME_DATE < (to_date('{end_1M}', 'dd/mm/yyyy'))"""
query2 = """SELECT CONTACT_NUMBER
FROM DATABASE2
WHERE (CAMP_NAME = {camp1} or
CAMP_NAME = {camp2} or
CAMP_NAME = {camp3})"""
Camp_eval('01/04/2022','01/05/2022','Camp_2022_04','Camp_2022_05','Camp_2022_06')
The parameters start_date and end_1M work fine with the {} brackets but the camp variables, which are strings don't return any results even though there are results in the database with those conditions if I were to write them directly in the query.
Any help would be appreciated!!
Please, do not use f-strings for creating SQL queries!
Most likely, any library you use for accessing a database already has a way of creating queries: SQLite docs (check code examples).
Another example: cur.execute("SELECT * FROM tasks WHERE priority = ?", (priority,)).
Not only this way is safer (fixes SQL Injection problem mentioned by #d-malan in comments), but it also eliminates the need to care about how data is represented in SQL - the library will automatically cast dates, strings, etc. in what they need to be casted into. Therefore, your problem can be fixed by using proper instruments.
def search(title="",author="",year="",isbn=""):
con = mysql.connector.connect(host="localhost", user="root", passwd="junai2104", database="book")
cur = con.cursor()
sql_statement = "SELECT * FROM book WHERE title={} or author={} or year={} or isbn={} ".format(title,author,year,isbn)
cur.execute(sql_statement)
rows=cur.fetchall()
con.close()
return rows
print(search(title='test2'))
How can I search a value in MySQL using Python argument?
how to get a values from the argument?
You have a couple of issues with your code:
In your SQL SELECT statement you are looking for values in text columns (TEXT, VARCHAR etc.). To do so you must add single quotes to your search qriteria, since you want to indicate a text literal. So WHERE title={} should be WHERE title='{}' (same goes for the other parameters).
When one or more of your arguments are empty, you will search for rows where the respective value is an empty text. So in your example search(title='test2') will trigger a search for an entry where the title column has the value 'test2' or any of the other three columns (author, year and isbn) has an empty text. If you inted to look for a title 'test2', this will only work if none of the other columns will ever contain an empty text. And even then, because of the three OR operators in your query, performance will be poor. What you should do instead is to evaluate each parameter individually and construct the query only with the parameters that are not empty.
By constructing your query with formatting a string, you will create a massive security issue in case the values of your search parameters come from user input. Your code is wide open for SQL injection, which is one of the simplest and most effective attacks on your system. You should always parametrize your queries to prevent this attack. By general principle, never create SQL queries by formating or concatenating strings with their parameters. Note that with parametrized queries you do not need to add single quotes to your query as wriitten in point 1.
This question already has answers here:
Dynamic SQL WHERE clause generation
(3 answers)
How to use variables in SQL statement in Python?
(5 answers)
Python sqlite3 parameterized drop table
(1 answer)
Closed 4 years ago.
From my Python code (Flask application, actually), I need to execute sqlite query, of the following structure
SELECT some_column FROM My_table WHERE some_column=some_value;
Now, some_column recurs twice in the query, one way to execute it is:
cursor.execute('SELECT ? FROM Users WHERE ?=?;', (some_column, some_column, some_value))
Which is not very nice/Pythonic. Then I came up with:
cursor.execute('SELECT {0} FROM Users WHERE {0}=?;'.format(some_column), (some_value,))
Finally, I ended up using .format() all the way:
cursor.execute('SELECT {0} FROM Users WHERE {0}={1};'.format(some_column, some_value), ())
I am wondering if there is prettier and/or more Pythonic way to pass recurring arguments into sqlite's cursor.execute()?
First syntax is incorrect. In SQL parameterized queries can use parameters for values, not for table or column names.
Third form is bad because it hard codes a value in a query which is forbidden per best practices. It used to be common some times ago and was the cause for SQL injection security problems.
So the only possibility is second form: use string construction for table and column names, and parameters for values.
But anyway, your query is close to non sense: you ask the value of one single column when you fix that column value. For each selected row, the value will be some_value!
So, I assume that this was a simplified example of a more complex question, but without more context, I simply cannot imagine why you are asking this question and what is your real problem.
It's not a big improvement, but since you tagged Python 3, consider the f-string:
f"SELECT {col} FROM Users WHERE {col}={val}"
As Klaus D. points out in a comment, however, it's not best practice to allow values to be format strings, as in some cases it makes your code vulnerable to SQL injection.
It's a little less compact, but you can use a mix of f-string (for column names) and sqlite ? syntax to input values (basically a mashup of your first two examples):
params = (val,)
q = f"SELECT {col} FROM Users WHERE {col} = ?"
cursor.execute(q, params)