Related
I have the following Python code:
cursor.execute("INSERT INTO table VALUES var1, var2, var3,")
where var1 is an integer, var2 and var3 are strings.
How can I write the variable names without Python including them as part of the query text?
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
Note that the parameters are passed as a tuple.
The database API does proper escaping and quoting of variables. Be careful not to use the string formatting operator (%), because
It does not do any escaping or quoting.
It is prone to uncontrolled string format attacks e.g. SQL injection.
Different implementations of the Python DB-API are allowed to use different placeholders, so you'll need to find out which one you're using -- it could be (e.g. with MySQLdb):
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
or (e.g. with sqlite3 from the Python standard library):
cursor.execute("INSERT INTO table VALUES (?, ?, ?)", (var1, var2, var3))
or others yet (after VALUES you could have (:1, :2, :3) , or "named styles" (:fee, :fie, :fo) or (%(fee)s, %(fie)s, %(fo)s) where you pass a dict instead of a map as the second argument to execute). Check the paramstyle string constant in the DB API module you're using, and look for paramstyle at http://www.python.org/dev/peps/pep-0249/ to see what all the parameter-passing styles are!
Many ways. DON'T use the most obvious one (%s with %) in real code, it's open to attacks.
Here copy-paste'd from pydoc of sqlite3:
# Never do this -- insecure!
symbol = 'RHAT'
cur.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol)
# Do this instead
t = ('RHAT',)
cur.execute('SELECT * FROM stocks WHERE symbol=?', t)
print(cur.fetchone())
# Larger example that inserts many records at a time
purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
('2006-04-05', 'BUY', 'MSFT', 1000, 72.00),
('2006-04-06', 'SELL', 'IBM', 500, 53.00),
]
cur.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases)
More examples if you need:
# Multiple values single statement/execution
c.execute('SELECT * FROM stocks WHERE symbol=? OR symbol=?', ('RHAT', 'MSO'))
print c.fetchall()
c.execute('SELECT * FROM stocks WHERE symbol IN (?, ?)', ('RHAT', 'MSO'))
print c.fetchall()
# This also works, though ones above are better as a habit as it's inline with syntax of executemany().. but your choice.
c.execute('SELECT * FROM stocks WHERE symbol=? OR symbol=?', 'RHAT', 'MSO')
print c.fetchall()
# Insert a single item
c.execute('INSERT INTO stocks VALUES (?,?,?,?,?)', ('2006-03-28', 'BUY', 'IBM', 1000, 45.00))
http://www.amk.ca/python/writing/DB-API.html
Be careful when you simply append values of variables to your statements:
Imagine a user naming himself ';DROP TABLE Users;' --
That's why you need to use SQL escaping, which Python provides for you when you use cursor.execute in a decent manner. Example in the URL is:
cursor.execute("insert into Attendees values (?, ?, ?)", (name, seminar, paid))
The syntax for providing a single value can be confusing for inexperienced Python users.
Given the query
INSERT INTO mytable (fruit) VALUES (%s)
Generally*, the value passed to cursor.execute must wrapped in an ordered sequence such as a tuple or list even though the value itself is a singleton, so we must provide a single element tuple, like this: (value,).
cursor.execute("""INSERT INTO mytable (fruit) VALUES (%s)""", ('apple',))
Passing a single string
cursor.execute("""INSERT INTO mytable (fruit) VALUES (%s)""", ('apple'))
will result in an error which varies by the DB-API connector, for example
psycopg2:
TypeError: not all arguments converted during string formatting
sqlite3
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 5 supplied
mysql.connector
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax;
* The pymysql connector handles a single string parameter without erroring. However it's better to wrap the string in a tuple even if it's a single because
you won't need to change the code if you switch connector package
you keep a consistent mental model of the query parameters being a sequence of objects rather than a single object.
I have the following Python code:
cursor.execute("INSERT INTO table VALUES var1, var2, var3,")
where var1 is an integer, var2 and var3 are strings.
How can I write the variable names without Python including them as part of the query text?
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
Note that the parameters are passed as a tuple.
The database API does proper escaping and quoting of variables. Be careful not to use the string formatting operator (%), because
It does not do any escaping or quoting.
It is prone to uncontrolled string format attacks e.g. SQL injection.
Different implementations of the Python DB-API are allowed to use different placeholders, so you'll need to find out which one you're using -- it could be (e.g. with MySQLdb):
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
or (e.g. with sqlite3 from the Python standard library):
cursor.execute("INSERT INTO table VALUES (?, ?, ?)", (var1, var2, var3))
or others yet (after VALUES you could have (:1, :2, :3) , or "named styles" (:fee, :fie, :fo) or (%(fee)s, %(fie)s, %(fo)s) where you pass a dict instead of a map as the second argument to execute). Check the paramstyle string constant in the DB API module you're using, and look for paramstyle at http://www.python.org/dev/peps/pep-0249/ to see what all the parameter-passing styles are!
Many ways. DON'T use the most obvious one (%s with %) in real code, it's open to attacks.
Here copy-paste'd from pydoc of sqlite3:
# Never do this -- insecure!
symbol = 'RHAT'
cur.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol)
# Do this instead
t = ('RHAT',)
cur.execute('SELECT * FROM stocks WHERE symbol=?', t)
print(cur.fetchone())
# Larger example that inserts many records at a time
purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
('2006-04-05', 'BUY', 'MSFT', 1000, 72.00),
('2006-04-06', 'SELL', 'IBM', 500, 53.00),
]
cur.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases)
More examples if you need:
# Multiple values single statement/execution
c.execute('SELECT * FROM stocks WHERE symbol=? OR symbol=?', ('RHAT', 'MSO'))
print c.fetchall()
c.execute('SELECT * FROM stocks WHERE symbol IN (?, ?)', ('RHAT', 'MSO'))
print c.fetchall()
# This also works, though ones above are better as a habit as it's inline with syntax of executemany().. but your choice.
c.execute('SELECT * FROM stocks WHERE symbol=? OR symbol=?', 'RHAT', 'MSO')
print c.fetchall()
# Insert a single item
c.execute('INSERT INTO stocks VALUES (?,?,?,?,?)', ('2006-03-28', 'BUY', 'IBM', 1000, 45.00))
http://www.amk.ca/python/writing/DB-API.html
Be careful when you simply append values of variables to your statements:
Imagine a user naming himself ';DROP TABLE Users;' --
That's why you need to use SQL escaping, which Python provides for you when you use cursor.execute in a decent manner. Example in the URL is:
cursor.execute("insert into Attendees values (?, ?, ?)", (name, seminar, paid))
The syntax for providing a single value can be confusing for inexperienced Python users.
Given the query
INSERT INTO mytable (fruit) VALUES (%s)
Generally*, the value passed to cursor.execute must wrapped in an ordered sequence such as a tuple or list even though the value itself is a singleton, so we must provide a single element tuple, like this: (value,).
cursor.execute("""INSERT INTO mytable (fruit) VALUES (%s)""", ('apple',))
Passing a single string
cursor.execute("""INSERT INTO mytable (fruit) VALUES (%s)""", ('apple'))
will result in an error which varies by the DB-API connector, for example
psycopg2:
TypeError: not all arguments converted during string formatting
sqlite3
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 5 supplied
mysql.connector
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax;
* The pymysql connector handles a single string parameter without erroring. However it's better to wrap the string in a tuple even if it's a single because
you won't need to change the code if you switch connector package
you keep a consistent mental model of the query parameters being a sequence of objects rather than a single object.
I have the following Python code:
cursor.execute("INSERT INTO table VALUES var1, var2, var3,")
where var1 is an integer, var2 and var3 are strings.
How can I write the variable names without Python including them as part of the query text?
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
Note that the parameters are passed as a tuple.
The database API does proper escaping and quoting of variables. Be careful not to use the string formatting operator (%), because
It does not do any escaping or quoting.
It is prone to uncontrolled string format attacks e.g. SQL injection.
Different implementations of the Python DB-API are allowed to use different placeholders, so you'll need to find out which one you're using -- it could be (e.g. with MySQLdb):
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
or (e.g. with sqlite3 from the Python standard library):
cursor.execute("INSERT INTO table VALUES (?, ?, ?)", (var1, var2, var3))
or others yet (after VALUES you could have (:1, :2, :3) , or "named styles" (:fee, :fie, :fo) or (%(fee)s, %(fie)s, %(fo)s) where you pass a dict instead of a map as the second argument to execute). Check the paramstyle string constant in the DB API module you're using, and look for paramstyle at http://www.python.org/dev/peps/pep-0249/ to see what all the parameter-passing styles are!
Many ways. DON'T use the most obvious one (%s with %) in real code, it's open to attacks.
Here copy-paste'd from pydoc of sqlite3:
# Never do this -- insecure!
symbol = 'RHAT'
cur.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol)
# Do this instead
t = ('RHAT',)
cur.execute('SELECT * FROM stocks WHERE symbol=?', t)
print(cur.fetchone())
# Larger example that inserts many records at a time
purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
('2006-04-05', 'BUY', 'MSFT', 1000, 72.00),
('2006-04-06', 'SELL', 'IBM', 500, 53.00),
]
cur.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases)
More examples if you need:
# Multiple values single statement/execution
c.execute('SELECT * FROM stocks WHERE symbol=? OR symbol=?', ('RHAT', 'MSO'))
print c.fetchall()
c.execute('SELECT * FROM stocks WHERE symbol IN (?, ?)', ('RHAT', 'MSO'))
print c.fetchall()
# This also works, though ones above are better as a habit as it's inline with syntax of executemany().. but your choice.
c.execute('SELECT * FROM stocks WHERE symbol=? OR symbol=?', 'RHAT', 'MSO')
print c.fetchall()
# Insert a single item
c.execute('INSERT INTO stocks VALUES (?,?,?,?,?)', ('2006-03-28', 'BUY', 'IBM', 1000, 45.00))
http://www.amk.ca/python/writing/DB-API.html
Be careful when you simply append values of variables to your statements:
Imagine a user naming himself ';DROP TABLE Users;' --
That's why you need to use SQL escaping, which Python provides for you when you use cursor.execute in a decent manner. Example in the URL is:
cursor.execute("insert into Attendees values (?, ?, ?)", (name, seminar, paid))
The syntax for providing a single value can be confusing for inexperienced Python users.
Given the query
INSERT INTO mytable (fruit) VALUES (%s)
Generally*, the value passed to cursor.execute must wrapped in an ordered sequence such as a tuple or list even though the value itself is a singleton, so we must provide a single element tuple, like this: (value,).
cursor.execute("""INSERT INTO mytable (fruit) VALUES (%s)""", ('apple',))
Passing a single string
cursor.execute("""INSERT INTO mytable (fruit) VALUES (%s)""", ('apple'))
will result in an error which varies by the DB-API connector, for example
psycopg2:
TypeError: not all arguments converted during string formatting
sqlite3
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 5 supplied
mysql.connector
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax;
* The pymysql connector handles a single string parameter without erroring. However it's better to wrap the string in a tuple even if it's a single because
you won't need to change the code if you switch connector package
you keep a consistent mental model of the query parameters being a sequence of objects rather than a single object.
I can iterate through a python object with te following code, however I would like to be able to use placeholders for the schema and table name, normally I do this with {}.{} ad the .format() methods, but how do you combine the two?
cur.executemany("INSERT INTO schema.table_name (x,y,z) "
"values (%s, %s, %s)", top_sample)
Depends on which python you use you can try use f-string
schema = "schema"
table_name = "table_name"
cur.executemany(f"INSERT INTO {schema}.{table_name} (x,y,z) values (%s, %s, %s)", top_sample)
check PEP 498 -- Literal String Interpolation
another option is a simple format
cur.executemany("INSERT INTO {schema}.{table_name} (x,y,z) values (%s, %s, %s)".format(schema=schema, table_name=table_name), top_sample)
but I find the first option shorter and cleaner
I'm not sure what the issue is. You can very well use format like this:
cur.executemany("INSERT INTO {}.{} (x,y,z) values (%s, %s, %s)".format('hello', 'world'), top_sample)
cur.executemany(
"""INSERT INTO schema.{table_name} (x,y,z) values (%s, %s, %s)""".format(table_name=your_table_name),
top_sample
)
place your table name in place of your_table_name
Following the advice on this thread, I am storing my list as string type in MySQL database, but, I'm facing this error:
_mysql_exceptions.ProgrammingError: (1064, 'You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near \'foo_bar" but I can\\\'t bar.\', u\'high\', 0]]")\' at line 1')
on some list entries like this one:
var1 = 'Name'
var2 = 'Surname'
var3 = 15
var4 = [u'The Meter', [[u'Black', u'foo foo bar bar "foo_bar" but I can\'t bar', u'high', 0]]]
I figured that's because there's a double quote at the beginning of foo_bar and I am using the following code to make entries in the database:
SQL = 'INSERT INTO test_table (col1, col2, col3, col4) VALUES ("{}", "{}", "{}", "{}")'.format(var1, var2, var3, var4)
cursor.execute(SQL)
And the double quotes are not being escaped.
If i remove the double quotes from: ("{}", "{}", "{}", "{}"), I get the same error,
I tried using a different string formatting, (using %s) but that didn't work either.
Any ideas?
Do not use string formatting to interpolate SQL values. Use SQL parameters:
SQL = 'INSERT INTO test_table (col1, col2, col3, col4) VALUES (%s, %s, %s, %s)'
cursor.execute(SQL, (var1, var2, var3, var4))
Here the %s are SQL parameters; the database then takes care of escaping the values (passed in as the 2nd argument to `cursor.execute) for you.
Exactly what syntax you need to use depends on your database adapter; some use %s, others use ? for the placeholders.
You can't otherwise use Python containers, like a list, for these parameters. You'd have to serialise that to a string format first; you could use JSON for that, but then you'd also have to remember to decode the JSON into a Python string again when you query the database. That's what the answers to the other question tried to convey.
For example, if var4 is the list, you could use:
cursor.execute(SQL, (var1, var2, var3, json.dumps(var4)))
SQL = 'INSERT INTO test_table (col1, col2, col3, col4) VALUES ("{!a}", "{!a}", "{!a}", "{!a}")'.format(var1, var2, var3, str(var4))
cursor.execute(SQL)
{!a} applies ascii() and hence escapes non-ASCII characters like quotes and even emoticons.
Check out Python3 docs