I'm writing a quick web.py app and take data from web.input...
import web
urls = (
'/', 'something',
)
app = web.application(urls, globals())
db = web.database(dbn='postgres', db='database', user='username', password='password', host='127.0.0.1')
class something:
def GET(self):
i = web.input()
return db.select('foo.table', where="column=$variable", vars={'variable':i.input, })
if __name__ == "__main__": app.run()
Should I worry about passing i.input to db.select (or query etc.) as I do as part of vars? SQL injection possibilities etc. ?
Edit:
I have been playing around with this myself, trying to get something nasty happenning. Playing with quoting for example, http://localhost:8080/?id=13' or 'x' ='x results in nicely escaped sql being shown in Exceptions:
<sql: 'select * from foo.table where id = "13\' or \'x\'=\'x"'>
I've tried a few other common tests that the internet puts forward and think I'm quite happy web.py is dealing with the sanitisation... Would anyone else be able to comment?
http://webpy.org/cookbook/query says:
To prevent SQL injection attacks, db.query also accepts the "vars"
syntax as described in db.select:
results = db.query("SELECT * FROM users WHERE id=$id", vars={'id':10})
This will escape user input, if you're trusting them for the "id"
variable.
So I guess its as simple as that.
Of course I realise that I still need to validate user input if I'm going to be inserting it places...
The "nicely escaped sql" you see does not matter as it's never sent to the database engine.
In all cases, unless you manually insert the values into the SQL request string, you're safe against an SQL injection. This includes select/insert/update/delete methods as well as the $variable_name substitution style. In both cases the SQL request is not fully assembled as text, but properly converted into a SQL prepared statement and compiled by the DB engine as such. Only after that the parameters are actually substituted for statement execution. So, unless you build the SQL query string and/or parts of it by hand using the untrusted data, you're safe.
Unfortunately I'm unable to provide a link to any source better than the source code of the module as it was my only source of information.
Yes because some one can alter the variable especially if it comes from a url to say something like "selecte * from table where id = [ variable + DELETE TABLE[. This is an example of sql injection. This is especially critical if the user has any idea of what your back end data objects are named.
I'm not exactly certain how Python would handle parametrization of variables for a sql statement but I had to perform similar security fixes for cold fusion sites and ASP.net sites.
Related
I have some code in Python that sets a char(80) value in an sqlite DB.
The string is obtained directly from the user through a text input field and sent back to the server with a POST method in a JSON structure.
On the server side I currently pass the string to a method calling the SQL UPDATE operation.
It works, but I'm aware it is not safe at all.
I expect that the client side is unsafe anyway, so any protection is to be put on the server side. What can I do to secure the UPDATE operation agains SQL injection ?
A function that would "quote" the text so that it can't confuse the SQL parser is what I'm looking for. I expect such function exist but couldn't find it.
Edit:
Here is my current code setting the char field name label:
def setLabel( self, userId, refId, label ):
self._db.cursor().execute( """
UPDATE items SET label = ? WHERE userId IS ? AND refId IS ?""", ( label, userId, refId) )
self._db.commit()
From the documentation:
con.execute("insert into person(firstname) values (?)", ("Joe",))
This escapes "Joe", so what you want is
con.execute("insert into person(firstname) values (?)", (firstname_from_client,))
The DB-API's .execute() supports parameter substitution which will take care of escaping for you, its mentioned near the top of the docs; http://docs.python.org/library/sqlite3.html above Never do this -- insecure.
Noooo... USE BIND VARIABLES! That's what they're there for. See this
Another name for the technique is parameterized sql (I think "bind variables" may be the name used with Oracle specifically).
I have an Airflow DAG which takes an argument from the user for table.
I then use this value in an SQL statement and execute it in BigQuery. I'm worried about exposing myself to SQL Injection.
Here is the code:
sql = f"""
CREATE OR REPLACE TABLE {PROJECT}.{dataset}.{table} PARTITION BY DATE(start_time) as (
//OTHER CODE
)
"""
client = bigquery.Client()
query_job = client.query(sql)
Both dataset and table get passed through via airflow but I'm worried someone could pass through something like: random_table; truncate other_tbl; -- as the table argument.
My fear is that the above will create a table called random_table and then truncate an existing table.
Is there a safer way to process these passed through arguments?
I've looked into parameterized queries in BigQuery but these don't work for table names.
You will have to create a table name validator. I think you can safely validate by using just backticks --> ` at the start and at the end of your table name string. It's not a 100% solution but it worked for some of my test scenarios I try. It should work like this:
# validate should look for ` at the beginning and end of your tablename
table_name = validate(f"`{project}.{dataset}.{table}`")
sql = f"""
CREATE OR REPLACE TABLE {table_name} PARTITION BY DATE(start_time) as (
//OTHER CODE
)
"""
...
Note: I suggest you to check the following post on medium site to check about bigquery sql injection.
I checked the official documentation about Running parameterized queries, and sadly it only covers the parameterization of variables not tables or other string part of your query.
As a final note, I recommend to open a feature request for BigQuery for this particular scenario.
You should probably look into sanitization/validation of user input in general. This is done before passing the input to the BQ query.
With Python, you could look for malicious strings in the user input - like truncate in your example - or use a regex to filter input that for instance contains --. Those are just some quick examples. I recommend you do more research on that topic; you will also find quite a few questions on that topic on SE.
I am new to this and trying to learn python. I wrote a select statement in python where I used a parameter
Select """cln.customer_uid = """[(num_cuid_number)])
TypeError: string indices must be integers
Agree with the others, this doesn't look really like Python by itself.
I will see even without seeing the rest of that code I'll guess the [(num_cuid_number)] value(s) being returned is a string, so you'll want to convert it to integer for the select statement to process.
num_cuid_number is most likely a string in your code; the string indices are the ones in the square brackets. So please first check your data variable to see what you received there. Also, I think that num_cuid_number is a string, while it should be in an integer value.
Let me give you an example for the python code to execute: (Just for the reference: I have used SQLAlchemy with flask)
#app.route('/get_data/')
def get_data():
base_sql="""
SELECT cln.customer_uid='%s' from cln
""" % (num_cuid_number)
data = db.session.execute(base_sql).fetchall()
Pretty sure you are trying to create a select statement with a "where" clause here. There are many ways to do this, for example using raw sql, the query should look similar to this:
query = "SELECT * FROM cln WHERE customer_uid = %s"
parameters = (num_cuid_number,)
separating the parameters from the query is secure. You can then take these 2 variables and execute them with your db engine like
results = db.execute(query, parameters)
This will work, however, especially in Python, it is more common to use a package like SQLAlchemy to make queries more "flexible" (in other words, without manually constructing an actual string as a query string). You can do the same thing using SQLAlchemy core functionality
query = cln.select()
query = query.where(cln.customer_uid == num_cuid_number)
results = db.execute(query)
Note: I simplified "db" in both examples, you'd actually use a cursor, session, engine or similar to execute your queries, but that wasn't your question.
Background: Not much documentation on MySQLdb Connector
Maybe I'm looking in the wrong places, but there's not much documentation about Python's MySQLdb family of connectors. Perhaps PEP249 is meant to do the job. Oracle's MySQL/Python connector seems to have much better docs, but at the moment I'm working with mysqlclient (the 3.x version of MySQLdb, which wraps around the C connector).
Named Parameters in MySQLdb: working for single values
After much searching, I stumbled upon beautiful syntax for binding named parameters, so long as they are a single value. For instance (made-up query to simplify the case):
query = """
SELECT...
WHERE
name = %(name)s AND
gender = %(gender)s
"""
parameters = {'name': name, 'gender': gender}
cursor.execute(query, parameters)
This properly escapes the parameters. Terrific.
Named Parameters in MySQLdb: how to use iterables?
Now I'd like to use a set, list or tuple to build queries with IN. Something like:
query = """
SELECT...
WHERE
gender = %(gender)s AND
name IN %(nameset)s
"""
I found a similar question here but that query doesn't use named parameters (the placeholder is named, but not the iterable).
What am I missing? Would someone know the magic syntax to make this work?
I see in the MySQLdb code that paramstyle is set to format rather than pyformat, but pyformat does work for single values.
To clarify,
I am not interested in an answer that just builds a string like ('sophie', 'jane', 'chloe') and concatenates it to the query. I need bound parameters to guarantee proper escaping.
I am also not interested in concatenating a join that uses db.escape_string(), although I may end up going that route if nothing else works.
What I'm really after is a clean idiom that binds named iterable parameters, if there is one.
Don't love answering my own question, but it's been a day...
Having looked inside the MySQLdb code, it looks like I won't get my wish. The quoting function will always add one set of quotes too many.
This is where I've ended up (the fallback option I had mentioned):
idset = ('Chloe', 'Zoe', "Noe';drip dautobus")
quoted_ids = [mdb.escape_string(identifier).decode('utf-8') for identifier in idset]
sql_idset = "('" + "', '".join(quoted_ids) + "')"
query = """
SELECT ...
FROM ...
JOIN ...
WHERE
someid = %(someid)s AND
namae IN """ + sql_idset
parameters = {'someid': someid}
cursor.execute(query, parameters)
Only one of the parameters is bound. The set in the IN clause is pre-quoted.
Not my favorite thing to have to do, but at least each value is run through the MySQLdb quoting function in order to quote any potentially harmful stuff.
The decode is there because escape_string prepares a byte string, but the query being built with the bound parameter someid is a string. Maybe there is an easier way. If you find one, let me know.
I don't know how to express it specifically in MySQLdb but it just works out of the box with the MySQL connector library (version: 1.4.4, MySQL 5.7.x) as suggested in this other answer:
cursor.execute('SELECT * FROM test WHERE id in %(l)s', {'l': (1,2,3)})
If MySQLdb messes it up then I suggest acquiring direct cursor somehow.
I've browsed some of the other questions about MySQL and Python on SO. There are a few things eluding me, because I'm pretty new to Python.
First, I'm trying to get a simple guestbook app to work. It takes posted variables and puts them into a MySQL database. Take a look:
con = MySQLdb.connect (host = "localhost",
user = "Chat",
passwd = "myPass",
db = "Chatserver")
cursor = con.cursor()
cursor.execute ("INSERT INTO guestbook (name,message) VALUES(%s,%s)",(name,greeting))
Ok, so some of the tutorials and answers on SO have many Quotation marks surrounding the SQL query, and I don't know why that is. I've tried it with 1 quote, I've tried it with 3 quotes, and it just never works. There are no exception callbacks and the code seems to run, but no records are ever entered into the database.
So my two questions are, how many quotation marks do I need when encapsulating the Queries, and why doesn't my script add anything to the database but not report any errors?
Ok, this answer Can't execute an INSERT statement in a Python script via MySQLdb helped me figure it out.
You have to add this at the end of your query.
cursor.execute(...)
con.commit() //this is what makes it actually do the execution?