We Keep Coding

How can I (safely) convert a dictionary into an UPDATE statement for sqlite in Python?

Say for example, I have a table of students, and I have a Python dictionary
mydict = {"fname" : "samwise", "lname" : "gamgee", "age" : 13}
How can I safely generate a Python function that can UPDATE this into my student table? (In my use-case I can safely assume that the student already exists in the table, AND I KNOW the id already)
I have created a function that achieves this functionality, but I can't help but think it's a bit crude, and perhaps open to SQL injection attacks
def sqlite_update(conn, table, data, pkeyname, pkeyval):
set_lines = ""
for k,v in data.items():
set_lines += "{} = '{}', ".format(k,v)
set_lines = set_lines[:-2] #remove space and comma from last item
sql = "UPDATE {0} SET {1} WHERE {2} = '{3}'"
statement = sql.format(table, set_lines, pkeyname, pkeyval)
conn.execute(statement)
conn.commit()
And to update I just call
sqlite_update(conn, "student", mydict, "id", 1)

As I assume you are using sqlalchemy. In this case, you can use sqlalchemy.sql.text function which escapes strings if required.
You can try to adjust your function as below.
from sqlalchemy.sql import text
def sqlite_update(conn, table, data, pkeyname, pkeyval):
set_lines = ",".join([f"{k}=:{k}" for k in data.keys()])
statement = text(f"UPDATE {table} SET {set_lines} WHERE {pkeyname} = :pkeyval")
args = dict(data)
args["pkeyval"] = pkeyval
conn.execute(statement, args)
conn.commit()
For more details, refer to sqlalchemy official documentation on text function.
EDIT
As for sqlite3 connection you can do basically the same thing as above, with slight changes.
def sqlite_update(conn, table, data, pkeyname, pkeyval):
set_lines = ",".join([f"{k}=:{k}" for k in data.keys()])
statement = f"UPDATE {table} SET {set_lines} WHERE {pkeyname} = :pkeyval"
args = dict(data)
args["pkeyval"] = pkeyval
conn.execute(statement, args)
conn.commit()
Refer to sqlite3 execute

This is indeed widely opened to SQL injection, because you build the query as a string including its values, instead of using a parameterized query.
Building a parameterized query with Python is easy:
def sqlite_update(conn, table, data, pkeyname, pkeyval):
query = f"UPDATE {table} SET " + ', '.join(
"{}=?".format(k) for k in data.keys()) + f" WHERE {pkeyname}=?"
# uncomment next line for debugging
# print(query, list(data.values()) + [pkeyval])
conn.execute(query, list(data.values()) + [pkeyval])
With your example, the query displayed by the debug print line is:
UPDATE student SET fname=?, lname=?, age=? WHERE id=?
with the following values list: ['samwise', 'gamgee', 13, 1]
But beware, to be fully protected from SQL injection, you should sanitize the table and field names to ensure they contain no dangerous characters like ;

Pass a python variable into an SQL query

I am working on Databricks and I am trying to pass a python variable into a SQL query:
series_name "LogTest"
query = """SELECT * FROM results_table
WHERE name = $series_name
"""
spark.sql(query).toPandas()
I tried with $ but it does not work.
How do I do this?
Regards

In this case, your variable and queries are just strings. You can do:
query = f"""SELECT * FROM results_table
WHERE name = '{series_name}'
"""
... in python 3. Remember that your query string needs the single quotes around the inserted variable. However, for certain variables, you may need to pass the variable directly to the spark module so it can interpret it. I use pyodbc a lot and would do this:
query = f"""SELECT * FROM results_table
WHERE name = ?"""
cursor.execute(query, series_name)

Following will also work,
series_name = "LogTest"
spark.sql("SELECT * FROM results_table WHERE name = " + series_name).toPandas()

we can also try the following.
series_name = "LogTest"
query = "SELECT * FROM results_table WHERE name = {}".format(series_name)
spark.sql(query).toPandas() #

Table name as variable in sql string

can someone tell me how to make the table a variable?
tableX = "test"
sql = "SELECT * FROM tableX WHERE datum = %s and name = %s"
val = (datumX, nameX,)
I just can't get it right. I've tried {}and also s%.
Thanks

You can use format function or f-string(if python 3.6=<)
tableX = "test"
sql = "SELECT * FROM {} WHERE datum = %s and name = %s".format(tableX)

In Python2.x works first answer:
sql = "SELECT * FROM {} WHERE datum = %s and name = %s" .format(tableX)
In Python3 you can use second one with f-string:
sql = f"SELECT * FROM {tableX} WHERE datum = %s and name = %s"
But, beware of SQL injection. You must be absolutely sure about source of that tableX variable (no user input). By direct format of the string you by-pass all escaping of input.

Python/pg8000 WHERE IN statement

What is the correct method to have the tuple (names) be available via %s in the SQL statement?
names = ('David', 'Isaac')
sql = 'SELECT * from names WHERE name IN %s'
cur.execute(sql,(names,))
The response in https://stackoverflow.com/a/28117658/5879128 works for psycopg2 but does not work in pg8000.
Thanks!

Generate the proper number of placeholders. In pg8000 the placeholder defaults to %s.
Interpolate placeholders to the query string.
Execute SQL-injection-safe query.
Like so:
sql = 'SELECT * from names WHERE name IN ({})'.format( ','.join(['%s']*len(names)) )
# results in -> 'SELECT * from names WHERE name IN (%s,%s)'
cur.execute(sql,(names,))

How to string format SQL IN clause with Python

I am trying to create a statement as follows:
SELECT * FROM table WHERE provider IN ('provider1', 'provider2', ...)
However, I'm having some trouble with the string formatting of it from the Django API. Here's what I have so far:
profile = request.user.get_profile()
providers = profile.provider.values_list('provider', flat=True) # [u'provider1', u'provider2']
providers = tuple[str(item) for item in providers] # ('provider1', 'provider2')
SQL = "SELECT * FROM table WHERE provider IN %s"
args = (providers,)
cursor.execute(sql,args)
DatabaseError
(1241, 'Operand should contain 1 column(s)')

MySQLdb has a method to help with this:
Doc
string_literal(...)
string_literal(obj) -- converts object obj into a SQL string literal.
This means, any special SQL characters are escaped, and it is enclosed
within single quotes. In other words, it performs:
"'%s'" % escape_string(str(obj))
Use connection.string_literal(obj), if you use it at all.
_mysql.string_literal(obj) cannot handle character sets.
Usage
# connection: <_mysql.connection open to 'localhost' at 1008b2420>
str_value = connection.string_literal(tuple(provider))
# '(\'provider1\', \'provider2\')'
SQL = "SELECT * FROM table WHERE provider IN %s"
args = (str_value,)
cursor.execute(sql,args)

Another answer that I don't like particularly, but will work for your apparent use-case:
providers = tuple[str(item) for item in providers] # ('provider1', 'provider2')
# rest of stuff...
SQL = 'SELECT * FROM table WHERE provider IN {}'.format(repr(providers))
cursor.execute(SQL)

You should probably do the string replacement before passing it to the cursor object to execute:
sql = "SELECT * FROM table WHERE provider IN (%s)" % \
(','.join(str(x) for x in providers))
cursor.execute(sql)

So, you have string input for ID's required:
some_vals = '1 3 5 76 5 4 2 5 7 8'.split() # convert to suitable type if required
SomeModel.objects.filter(provider__in=some_vals)

"SELECT * FROM table WHERE provider IN ({0},{1},{2})".format(*args) #where args is list or tuple of arguments.

try this.... should work.
SQL = "SELECT * FROM table WHERE provider IN %s"%(providers)
exec 'cursor.execute("%s")'%(SQL)