I'm making a program that is a user interface for quizes set by teachers in a primary school. I am trying this query which is using data typed in by the user on the previous page. it is looking for people in the database who match the username and quiz number concerned. this is so the teacher can see how well pupils are doing on certain quizes.
Here is my code.
dbDatabase = sqlite3.connect('c:\\xampp\\cgi-bin\\MakingATable.db')
cuDatabase = dbDatabase.cursor()
Fieldstorage = cgi.FieldStorage() #what you typed in on the webpage
Quizno = Fieldstorage.getvalue("quizno")
UserID = Fieldstorage.getvalue("username")
#print (Quizno)
#print (UserID)
cuDatabase.execute ("""
SELECT Result
FROM resultstable
WHERE QuizID = '""" + str(Quizno) + """
AND UserID = '""" + UserID + "'")
for (row) in cuDatabase:
print (row)
dbDatabase.commit()
cuDatabase.close()
Here is the error message i am getting when i run my webpage:
40 FROM resultstable
41 WHERE QuizID = '""" + str(Quizno) + """
=> 42 AND UserID = '""" + UserID + "'")
43
44 for (row) in cuDatabase:
AND undefined, UserID = 'HuPa1'
OperationalError: near "HuPa1": syntax error
args = ('near "HuPa1": syntax error',)
with_traceback = <built-in method with_traceback of OperationalError object>
Also should I use an OR instead of AND so that if the user hasn't done that quiz it will display any quiz the user does. or so that if lots of people have done one Quiz then the teacher will see everyone who, for example, have done quiz 1?
You should use SQL parameters:
cuDatabase.execute ("""
SELECT Result
FROM resultstable
WHERE QuizID = ?
AND UserID = ?""", (Quizno, UserID))
The ? placeholders will be replaced by your values, automatically quoted to prevent SQL injection attacks (and operational errors).
Quoting from the sqlite3 module documentation:
Usually your SQL operations will need to use values from Python variables. You shouldn’t assemble your query using Python’s string operations because doing so is insecure; it makes your program vulnerable to an SQL injection attack (see http://xkcd.com/327/ for humorous example of what can go wrong).
Instead, use the DB-API’s parameter substitution. Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor’s execute() method. (Other database modules may use a different placeholder, such as %s or :1.)
Use a separate query to ask the database for other quizzes if this query doesn't return a result; if you use OR instead of AND otherwise, you will get both quizz results that other users have done, and anything this user has completed.
The best solution is to use prepared statement:
cuDatabase.execute("SELECT Result FROM resultstable WHERE QuizID=? AND UserID=?", Quizno, UserID)
In prepared statement mode all variables are replaced by question marks in query text and are parameters to execute(). But not all databases or db drivers support it. Sometimes instead of question mark you will have to use %s (one of PostgreSQL drivers works this way).
Worse, but working solution is to use Python % operator, but with this solution you will have to use your own quote() function that escapes dangerous characters that may appear in data (prevents from SQL injection):
cuDatabase.execute("SELECT Result FROM resultstable WHERE QuizID='%s' AND UserID='%s'" % (quote(Quizno), quote(UserID)))
Related
I have a sql query I'm executing that I'm passing variables into. In the current context I'm passing the parameter values in as f strings, but this query is vulnerable to sql injection. I know there is a method to use a stored procedure and restrict permissions on the user executing the query. But is there a way to avoid having to go the stored procedure route and perhaps modify this function to be secure against SQL Injection?
I have the below query created to execute within a python app.
def sql_gen(tv, kv, join_kv, col_inst, val_inst, val_upd):
sqlstmt = f"""
IF NOT EXISTS (
SELECT *
FROM {tv}
WHERE {kv} = {join_kv}
)
INSERT {tv} (
{col_inst}
)
VALUES (
{val_inst}
)
ELSE
UPDATE {tv}
SET {val_upd}
WHERE {kv} = {join_kv};
"""
engine = create_engine(f"mssql+pymssql://{username}:{password}#{server}/{database}")
connection = engine.raw_connection()
cursor = connection.cursor()
cursor.execute(sqlstmt)
connection.commit()
cursor.close()
Fortunately, most database connectors have query parameters in which you pass the variable instead of giving in the string inside the query yourself for the risks you mentioned.
You can read more on this here: https://realpython.com/prevent-python-sql-injection/#understanding-python-sql-injection
Example:
# Vulnerable
cursor.execute("SELECT admin FROM users WHERE username = '" + username + '");
# Safe
cursor.execute("SELECT admin FROM users WHERE username = %s'", (username, ));
As Amanzer mentions correctly in his reply Python has mechanisms to pass parameters safely.
However, there are other elements in your query (table names and column names) that are not supported as parameters (bind variables) because JDBC does not support those.
If these are from an untrusted source (or may be in the future) you should be sure you validate these elements. This is a good coding practice to do even if you are sure.
There are some options to do this safely:
You should limit your tables and columns based on positive validation - make sure that the only values allowed are the ones that are authorized
If that's not possible (because these are user created?):
You should make sure tables or column names limit the
names to use a "safe" set of characters (alphanumeric & dashes,
underscores...)
You should enquote the table names / column names -
adding double quotes around the objects. If you do this, you need to
be careful to validate there are no quotes in the name, and error out
or escape the quotes. You also need to be aware that adding quotes
will make the name case sensitive.
I'm trying to execute thi query:
SELECT '23.34.67.0/22' CONCAT(DAY_31, 'hello') DAY_31 FROM Jule
using pymysql. My code is:
cursor.execute("SELECT %s CONCAT(%s, %s) %s FROM Jule", (p, 'DAY_' + _day, as_tmp, 'DAY_' + _day))
But python adds single quote and return a syntax error
"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '('DAY_31', 'hello') 'DAY_31' FROM Jule' at line 1"
DAY_31 is a column of Jule Schema
If memory serves, ? in place of %s might do the trick.
The reason that you end up with quotes is because cursor.execute adds these around values you pass in as arguments. This is totally appropriate for your second argument, because if the value 'hello' was inserted into the query as-is you would end up with a query like this:
SELECT '23.34.67.0/22' CONCAT(DAY_31, hello) DAY_31 FROM Jule
and you would have errors telling you that MySQL can't identify what hello should refer to.
Obviously though, this is not appropriate for situations where you want to pass in field names, or any other part of the query that is not a primitive string value. In these cases, you will need to splice them into your string before executing the query. One way you could do that is with f-strings, but there are other alternatives as well. Here's your cursor.execute line with the field names spliced in using f-strings:
cursor.execute(f"SELECT %s, CONCAT({'DAY_'+_day}, %s) {'DAY_'+_day} FROM Jule", (p, as_tmp))
Notice that I've removed the 'DAY_'+_day from the arguments list as well.
Important note:
While this should work like this (although I think you also needed an extra comma after SELECT '23.34.67.0/22', which I've added in the example above), it's very important that if day has a value that originates from outside of your application (e.g. passed in by a user in a form field) that you make sure it is exactly in the format you want before splicing it in to your query. Checking that the string value is an integer could be one way to do that. The reason that it is important is that without this, your application could be prone to SQL injection, which would potentially allow users to run arbitrary SQL on your database. If the value of day is calculated solely by your application, you shouldn't need to worry about this.
Actually what is happening has nothing to do with cursor.execute. It is just the way you are formatting the string in python.
We should first note that %s %d can be specific to SQL, and that these can also be used in python for string formatting.
So taking this into account, I think you have no need to use %s for string formatting (which on top does not help with readability). And since you are cursor.executeing straight to your database, you could format your string straight away in python.
Small check to see whats happening:
p = "23.34.67.0/22"
as_tmp = "hello"
_day = "3"
print("SELECT %s CONCAT(%s, %s) %s FROM Jule", (p, 'DAY_' + _day, as_tmp, 'DAY_' + _day))
# output
# SELECT %s CONCAT(%s, %s) %s FROM Jule ('23.34.67.0/22', 'DAY_3', 'hello', 'DAY_3')
# ^^^^ so this is what is being sent in cursor.execute (with all the quotes and so on)
If you format with an f-string you will increase readability, and you should get rid of your problem with the quotes
print(f"SELECT '{p}' CONCAT(DAY_{_day}, '{as_tmp}') DAY_{_day} FROM Jule")
# output
# SELECT '23.34.67.0/22' CONCAT(DAY_3, 'hello') DAY_3 FROM Jule
So the solution could be:
cursor.execute(f"SELECT '{p}' CONCAT(DAY_{_day}, '{as_tmp}') DAY_{_day} FROM Jule")
I always failled to insert data to Mysql database from my telegram bot, and always run Exception. Only tanggald allways failed to insert. I thing format of date insert query is wrong. How to write correct format?
tanggald column detail : Data Type = DATE
This is piece of code:
def process_lanjut(message):
try:
chat_id = message.chat.id
qlanjut = message.text
user = user_dict[chat_id]
user.qlanjut=qlanjut
d = datetime.datetime.now().date()
next_monday = next_weekday(d, 4)
user.next_monday = next_monday
print(user.next_monday)
with con.cursor() as cursor:
sql = "INSERT INTO diagnosa(sex, tanggald) VALUES('" + user.sex + "','" +next_monday+ "')"
cursor.execute(sql)
con.commit()
con.close()
msg = bot.send_message(chat_id, 'thanks')
bot.register_next_step_handler(msg, send_end)
except Exception as e:
bot.reply_to(message,'oops lanjut')
On command line output : 2018-04-20 (data that should be INSERT to tanggald)
That's not the proper way to insert data into table. Although your way may work, it is not safe and it lacks data escaping (', ", etc.):
The SQL representation of many data types is often different from their Python string representation. The typical example is with single quotes in strings: in SQL single quotes are used as string literal delimiters, so the ones appearing inside the string itself must be escaped, whereas in Python single quotes can be left unescaped if the string is delimited by double quotes.
Because of the difference, sometime subtle, between the data types representations, a naïve approach to query strings composition, such as using Python strings concatenation, is a recipe for terrible problems.
If the variables containing the data to send to the database come from an untrusted source (such as a form published on a web site) an attacker could easily craft a malformed string, either gaining access to unauthorized data or performing destructive operations on the database. This form of attack is called SQL injection and is known to be one of the most widespread forms of attack to database servers. Before continuing, please print this page as a memo and hang it onto your desk.
So in your case the INSERT statement should look like this:
with con.cursor() as cursor:
sql = 'INSERT INTO diagnosa (sex, tanggald) VALUES (%s, %s)'
data = (user.sex, next_monday)
cursor.execute(sql, data)
Further reading:
MySQL docs and example
Similar article for PostgreSQL (contains a better explanation of potential problems)
I am aware that queries in Python can be parameterized using either ? or %s in execute query here or here
However I have some long query that would use some constant variable defined at the beginning of the query
Set #my_const = 'xyz';
select #my_const;
-- Query that use #my_const 40 times
select ... coalesce(field1, #my_const), case(.. then #my_const)...
I would like to do the least modif possible to the query from Mysql. So that instead of modifying the query to
pd.read_sql(select ... coalesce(field1, %s), case(.. then %s)... , [my_const, my_const, my_const, ..]
,I could write something along the line of the initial query. Upon trying the following, however, I am getting a TypeError: 'NoneType' object is not iterable
query_str = "Set #null_val = \'\'; "\
" select #null_val"
erpur_df = pd.read_sql(query_str, con = db)
Any idea how to use the original variable defined in Mysql query ?
The reason
query_str = "Set #null_val = \'\'; "\
" select #null_val"
erpur_df = pd.read_sql(query_str, con = db)
throws that exception is because all you are doing is setting null_value to '' and then selecting that '' - what exactly would you have expected that to give you? EDIT read_sql only seems to execute one query at a time, and as the first query returns no rows it results in that exception.
If you split them in to two calls to read_sql then it will in fact return you the value of your #null value in the second call. Due to this behaviour read_sql is clearly not a good way to do this. I strongly suggest you use one of my suggestions below.
Why are you wanting to set the variable in the SQL using '#' anyway?
You could try using the .format style of string formatting.
Like so:
query_str = "select ... coalesce(field1, {c}), case(.. then {c})...".format(c=my_const)
pd.read_sql(query_str)
Just remember that if you do it this way and your my_const is a user input then you will need to sanitize it manually to prevent SQL injection.
Another possibility is using a dict of params like so:
query_str = "select ... coalesce(field1, %(my_const)s, case(.. then %(my_const)s)..."
pd.read_sql(query_str, params={'my_const': const_value})
However this is dependent on which database driver you use.
From the pandas.read_sql docs:
Check your database driver documentation for which of the five syntax
styles, described in PEP 249’s paramstyle, is supported. Eg. for
psycopg2, uses %(name)s so use params={‘name’ : ‘value’}
I have been learning sqlite3 in python and I was wondering if I could use string formatting to edit the database or query it.
e.g. - SELECT %s FROM (table_name) where % can be the users input stored in a variable?
I tried it but it doesn't work so can someone please give me a working example.
Any help is appreciated. Thanks
Guys i tried this:
dursor = conn.execute("SELECT id FROM books")
# this helps find the correct id for storing in the database
for i in dursor:
lis.append(i[0])
command = """INSERT INTO books VALUES ({0}, {name}, {author})""".format(lis[-1] + 1, name=client_name, author = client_author)
and then
conn.execute(command)
but it returns no such column (name)
when i tried the same query in khan academy sql it worked why not here?
You can place question mark on your query string and pass the parameters from user input while calling the .execute() as a tuple.
Though i don't believe you are using it in a production. If it is the case than first take the data from user, sanitize it and see if you really want to let the user do what he actually wants to do.
Hope this helps:
param1 = sys.argv[1]
param2 = sys.argv[2]
query = "SELECT ? FROM (table_name) where id = ?"
cursor.execute(query, (param1,param2))
I'm unsure if you can do it in sqlite3 but I'd be looking for any alternative method if I were you. Are you REALLY wanting to allow the user to be able to actually alter your SQL on the fly? That is a potentially huge security hole you'd be creating.
e.g. user can essentially alter...
select ? from innocentTable
...to...
select * from tblUser -- from innocentTable
...and trawl your entire user table, just takes a bit of guess work to come up with the object names.
I'd suggest you read up on SQL Injection Attacks then look for an alternative way to achieve what you've suggested.