WHERE IN Clause in python list [duplicate] - python

This question already has answers here:
imploding a list for use in a python MySQLDB IN clause
(8 answers)
Closed 1 year ago.
I need to pass a batch of parameters to mysql in python. Here is my code:
sql = """ SELECT * from my_table WHERE name IN (%s) AND id=%(Id)s AND puid=%(Puid)s"""
params = {'Id':id,'Puid' : pid}
in_p=', '.join(list(map(lambda x: '%s', names)))
sql = sql %in_p
cursor.execute(sql, names) #todo: add params to sql clause
The problem is I want to pass the name list to sql IN clause, meanwhile I also want to pass the id and puid as parameters to the sql query clause. How do I implement these in python?


Think about the arguments to cursor.execute that you want. You want to ultimately execute
cursor.execute("SELECT * FROM my_table WHERE name IN (%s, %s, %s) AND id = %s AND puid = %s;", ["name1", "name2", "name3", id, pid])
How do you get there? The tricky part is getting the variable number of %ss right in the IN clause. The solution, as you probably saw from this answer is to dynamically build it and %-format it into the string.
in_p = ', '.join(list(map(lambda x: '%s', names)))
sql = "SELECT * FROM my_table WHERE name IN (%s) AND id = %s AND puid = %s;" % in_p
But this doesn't work. You get:
TypeError: not enough arguments for format string
It looks like Python is confused about the second two %ss, which you don't want to replace. The solution is to tell Python to treat those %ss differently by escaping the %:
sql = "SELECT * FROM my_table WHERE name IN (%s) AND id = %%s AND puid = %%s;" % in_p
Finally, to build the arguments and execute the query:
args = names + [id, pid]
cursor.execute(sql, args)


sql = """ SELECT * from my_table WHERE name IN (%s) AND id=%(Id)s AND puid=%(Puid)s""".replace("%s", "%(Clause)s")
print sql%{'Id':"x", 'Puid': "x", 'Clause': "x"}
This can help you.

Related

SQL Injection using Python

I have the following problem: I need a dynamic create statement, depending on what attributes my object has.
its following object:
class Table:
columns = []
def __init__(self, name, columns):
self.columns = columns
self.name = name
def columnsNumber(self) -> int:
return self.columns.__len__()
this is what the insert looks like:
sql = "INSERT INTO tableOverview (tableName, columns, datum) VALUES(%s, %s, CURRENT_TIMESTAMP);"
val = (table.name, table.columns.__len__())
await cursor.execute(sql, (val))
for x in table.columns:
sql = "ALTER TABLE %s ADD COLUMN %s VARCHAR(100) UNIQUE " % (table.name,x)
await cursor.execute(sql)
now I don't know, how to prevent a SQL injection.

For the ALTER TABLE statements you can quote the identifier names with backticks as described here.
for x in table.columns:
sql = "ALTER TABLE `%s` ADD COLUMN `%s` VARCHAR(100) UNIQUE " % (table.name,x)
await cursor.execute(sql)
In the insert statement, the code is already correctly using parameter substitution to ensure the inserted values are correctly quoted.
sql = "INSERT INTO tableOverview (tableName, columns, datum) VALUES(%s, %s, CURRENT_TIMESTAMP);"
val = (table.name, table.columns.len())
await cursor.execute(sql, val)

Parameterized Table Population

I am trying to populate a table(whose name is parameterized). The program runs fine, up until the point where the command gets executed.
Here is the code:
table_name = input("Enter table name: ")
value_name = input("Enter name: ")
sql = "INSERT INTO %s (name) VALUES (%s)" % db.escape_string(table_name), (value_name)
cursor.execute(sql)
I get the following error:
TypeError: not enough arguments for format string
Thanks to anyone who takes the time to help. Have a great rest of the day :)

as an alternative you good go with the new formatting format
sql = f"INSERT INTO {tab} (name) VALUES ({val})".format(tab=db.escape_string(table_name),
val=value_name)
or
sql = f"INSERT INTO {db.escape_string(table_name)} (name) VALUES ({value_name})"

Just wrap the sql formatting like below and try.
sql = "INSERT INTO %s (name) VALUES (%s)" % (db.escape_string(table_name), value_name)

How to Import Big JSON file to MYSQL [duplicate]

I am having a hard time using the MySQLdb module to insert information into my database. I need to insert 6 variables into the table.
cursor.execute ("""
INSERT INTO Songs (SongName, SongArtist, SongAlbum, SongGenre, SongLength, SongLocation)
VALUES
(var1, var2, var3, var4, var5, var6)
""")
Can someone help me with the syntax here?

Beware of using string interpolation for SQL queries, since it won't escape the input parameters correctly and will leave your application open to SQL injection vulnerabilities. The difference might seem trivial, but in reality it's huge.
Incorrect (with security issues)
c.execute("SELECT * FROM foo WHERE bar = %s AND baz = %s" % (param1, param2))
Correct (with escaping)
c.execute("SELECT * FROM foo WHERE bar = %s AND baz = %s", (param1, param2))
It adds to the confusion that the modifiers used to bind parameters in a SQL statement varies between different DB API implementations and that the mysql client library uses printf style syntax instead of the more commonly accepted '?' marker (used by eg. python-sqlite).

You have a few options available. You'll want to get comfortable with python's string iterpolation. Which is a term you might have more success searching for in the future when you want to know stuff like this.
Better for queries:
some_dictionary_with_the_data = {
'name': 'awesome song',
'artist': 'some band',
etc...
}
cursor.execute ("""
INSERT INTO Songs (SongName, SongArtist, SongAlbum, SongGenre, SongLength, SongLocation)
VALUES
(%(name)s, %(artist)s, %(album)s, %(genre)s, %(length)s, %(location)s)
""", some_dictionary_with_the_data)
Considering you probably have all of your data in an object or dictionary already, the second format will suit you better. Also it sucks to have to count "%s" appearances in a string when you have to come back and update this method in a year :)

The linked docs give the following example:
cursor.execute ("""
UPDATE animal SET name = %s
WHERE name = %s
""", ("snake", "turtle"))
print "Number of rows updated: %d" % cursor.rowcount
So you just need to adapt this to your own code - example:
cursor.execute ("""
INSERT INTO Songs (SongName, SongArtist, SongAlbum, SongGenre, SongLength, SongLocation)
VALUES
(%s, %s, %s, %s, %s, %s)
""", (var1, var2, var3, var4, var5, var6))
(If SongLength is numeric, you may need to use %d instead of %s).

Actually, even if your variable (SongLength) is numeric, you will still have to format it with %s in order to bind the parameter correctly. If you try to use %d, you will get an error. Here's a small excerpt from this link http://mysql-python.sourceforge.net/MySQLdb.html:
To perform a query, you first need a cursor, and then you can execute queries on it:
c=db.cursor()
max_price=5
c.execute("""SELECT spam, eggs, sausage FROM breakfast
WHERE price < %s""", (max_price,))
In this example, max_price=5 Why, then, use %s in the string? Because MySQLdb will convert it to a SQL literal value, which is the string '5'. When it's finished, the query will actually say, "...WHERE price < 5".

As an alternative to the chosen answer, and with the same safe semantics of Marcel's, here is a compact way of using a Python dictionary to specify the values. It has the benefit of being easy to modify as you add or remove columns to insert:
meta_cols = ('SongName','SongArtist','SongAlbum','SongGenre')
insert = 'insert into Songs ({0}) values ({1})'.format(
','.join(meta_cols), ','.join( ['%s']*len(meta_cols)))
args = [ meta[i] for i in meta_cols ]
cursor = db.cursor()
cursor.execute(insert,args)
db.commit()
Where meta is the dictionary holding the values to insert. Update can be done in the same way:
meta_cols = ('SongName','SongArtist','SongAlbum','SongGenre')
update='update Songs set {0} where id=%s'.
.format(','.join([ '{0}=%s'.format(c) for c in meta_cols ]))
args = [ meta[i] for i in meta_cols ]
args.append(songid)
cursor=db.cursor()
cursor.execute(update,args)
db.commit()

The first solution works well. I want to add one small detail here. Make sure the variable you are trying to replace/update it will has to be a type str. My mysql type is decimal but I had to make the parameter variable as str to be able to execute the query.
temp = "100"
myCursor.execute("UPDATE testDB.UPS SET netAmount = %s WHERE auditSysNum = '42452'",(temp,))
myCursor.execute(var)

Here is another way to do it. It's documented on the MySQL official website.
https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html
In the spirit, it's using the same mechanic of #Trey Stout's answer. However, I find this one prettier and more readable.
insert_stmt = (
"INSERT INTO employees (emp_no, first_name, last_name, hire_date) "
"VALUES (%s, %s, %s, %s)"
)
data = (2, 'Jane', 'Doe', datetime.date(2012, 3, 23))
cursor.execute(insert_stmt, data)
And to better illustrate any need for variables:
NB: note the escape being done.
employee_id = 2
first_name = "Jane"
last_name = "Doe"
insert_stmt = (
"INSERT INTO employees (emp_no, first_name, last_name, hire_date) "
"VALUES (%s, %s, %s, %s)"
)
data = (employee_id, conn.escape_string(first_name), conn.escape_string(last_name), datetime.date(2012, 3, 23))
cursor.execute(insert_stmt, data)

compose mysql query in python

I want to fetch all rows from MySQL table with
query = "SELECT * FROM %s WHERE last_name=%s"
cursor.execute(query, ("employees","Smith"))
but I'm getting
You have an error in your SQL syntax. When I try
query = "SELECT * FROM employees WHERE last_name=%s"
cursor.execute(query, ("Smith",))
all is fine.
Documentation says
cursor.execute(operation, params=None, multi=False)
The parameters found in the tuple or dictionary params are bound to the variables in the operation.link on docs

The first will generate an SQL like this:
SELECT * FROM 'employees' WHERE last_name='smith'
The parameters are SQL quoted.
If you really need to have a table name as param, you must proceed in 2 steps:
table_name = 'employees'
query_tpl = "SELECT * FROM {} WHERE last_name=%s"
query = query_tpl.format(table_name)
cursor.execute(query, ("Smith",))

you need to add the quote symbol. So the query will be like
SELECT * FROM employees WHERE last_name='Smith'
Change both your query to
query = "SELECT * FROM %s WHERE last_name='%s'"
query = "SELECT * FROM employees WHERE last_name='%s'"

You can't use a parameter for the table name in the execute call.
But you can use Python string interpolation for that:
query = "SELECT * FROM %s WHERE last_name=%s" %("employees","Smith")
cursor.execute(query)

You can't use a table name as a parameter. you are generating invalid sql with your code that is putting quotes around each string. the table name cannot have quotes around it.
sql you are generating
select * from 'employees' where last_name = 'Smith'
What sql you want
select * from employees where last_name = 'Smith'
you would have to format the string first like the example below.
query = "SELECT * from {} wherre last_name ='{}'"
cursor.execute(query.format("employees","Smith"))
using code like this does open up the possibility of SQL injection. so please bear that in mind.

query="SELECT * FROM %s WHERE name=%s",(employees,smith)
cursor.execute(query)
rows = cursor.fetchall()
Try this one. Hopefully it works for you.

Python/pg8000 WHERE IN statement

What is the correct method to have the tuple (names) be available via %s in the SQL statement?
names = ('David', 'Isaac')
sql = 'SELECT * from names WHERE name IN %s'
cur.execute(sql,(names,))
The response in https://stackoverflow.com/a/28117658/5879128 works for psycopg2 but does not work in pg8000.
Thanks!

Generate the proper number of placeholders. In pg8000 the placeholder defaults to %s.
Interpolate placeholders to the query string.
Execute SQL-injection-safe query.
Like so:
sql = 'SELECT * from names WHERE name IN ({})'.format( ','.join(['%s']*len(names)) )
# results in -> 'SELECT * from names WHERE name IN (%s,%s)'
cur.execute(sql,(names,))

Categories