We Keep Coding

How to avoid SQL Injection in Python for Upsert Query to SQL Server?

I have a sql query I'm executing that I'm passing variables into. In the current context I'm passing the parameter values in as f strings, but this query is vulnerable to sql injection. I know there is a method to use a stored procedure and restrict permissions on the user executing the query. But is there a way to avoid having to go the stored procedure route and perhaps modify this function to be secure against SQL Injection?
I have the below query created to execute within a python app.
def sql_gen(tv, kv, join_kv, col_inst, val_inst, val_upd):
sqlstmt = f"""
IF NOT EXISTS (
SELECT *
FROM {tv}
WHERE {kv} = {join_kv}
)
INSERT {tv} (
{col_inst}
)
VALUES (
{val_inst}
)
ELSE
UPDATE {tv}
SET {val_upd}
WHERE {kv} = {join_kv};
"""
engine = create_engine(f"mssql+pymssql://{username}:{password}#{server}/{database}")
connection = engine.raw_connection()
cursor = connection.cursor()
cursor.execute(sqlstmt)
connection.commit()
cursor.close()

Fortunately, most database connectors have query parameters in which you pass the variable instead of giving in the string inside the query yourself for the risks you mentioned.
You can read more on this here: https://realpython.com/prevent-python-sql-injection/#understanding-python-sql-injection
Example:
# Vulnerable
cursor.execute("SELECT admin FROM users WHERE username = '" + username + '");
# Safe
cursor.execute("SELECT admin FROM users WHERE username = %s'", (username, ));

As Amanzer mentions correctly in his reply Python has mechanisms to pass parameters safely.
However, there are other elements in your query (table names and column names) that are not supported as parameters (bind variables) because JDBC does not support those.
If these are from an untrusted source (or may be in the future) you should be sure you validate these elements. This is a good coding practice to do even if you are sure.
There are some options to do this safely:
You should limit your tables and columns based on positive validation - make sure that the only values allowed are the ones that are authorized
If that's not possible (because these are user created?):
You should make sure tables or column names limit the
names to use a "safe" set of characters (alphanumeric & dashes,
underscores...)
You should enquote the table names / column names -
adding double quotes around the objects. If you do this, you need to
be careful to validate there are no quotes in the name, and error out
or escape the quotes. You also need to be aware that adding quotes
will make the name case sensitive.

Escaping single and double quotes for mysql in python

I am using mysql.connector python library with python 2.7
I have a unicode string which may or may not contain single and double quotes.
Here are the things I tried for my escape function:
def escape(string):
#string.MySQL.escape_string()
#string = string.decode('string_escape')
#string = string.encode('unicode-escape').replace("'", "''")
#string = string.encode('unicode-escape').replace('"', '\"')
#string = string.encode('unicode-escape').replace("'", u"\u2019")
#string = string.encode('unicode-escape').replace('''"''', u"\u201D")
#string = string.encode('unicode-escape').replace('''''', u"\u201D")
return string
Nothing seems to have worked. I tried using this function but still gives mysql syntax error.
I need something within mysql.connector library which escapes the single and double quotes without breaking the unicode as well as mysql query.
Here is an example of SQL query I am using:
"""SELECT * FROM messages WHERE msg_id = '{msg_id}'""".format(**db_dict)
Let me know if any more details needed
EDIT: Example SQL query updated

MySQLdb officially declares to use the format paramstyle, but it also supports the pyformat style*, so if you want to use parameters from a dict, you can use:
db_dict = {'msg_id': "1'2'3", ...}
cursor.execute("SELECT * FROM messages WHERE msg_id = %(msg_id)s", db_dict)
Using string manipulation to create sql queries only leads to sql injection vulnerabilities, so you should never do it.
*... most db connectors that use python string formatting behind the screen do the same, they specify one of format or pyformat as paramstyle but actually support both. The dbapi2 doesnt't allow to specify two values here, but it doesn't forbid to support multiple parmstyles either. If you write code that potentially uses an unknowon dbapi2 connector it's enough that you can query a supported paramstyle, being able to know all would be nice but it's not necessary.

cursor.execute('SELECT * FROM messages WHERE msg_id = %s', (db_dict['msg_id'],)) is what you want to run here. Standard string escapes aren't supported by python's database interface, and, per #bobince's comment, are a security hole to boot.

Can I write a python/SQL code that is independent of the sql engine (PostGres / Sqlite)

I have a python code, in which I make SQL requests in a database. I would like to be able to switch between a postgresql (using module psycopg2) database and a sqlite one (using module sqlite3), without need of adapting my code. This means, I would like to have in my code some fixed SQL request strings, and I want to switch between the engine, only changing the definition of the database connector object, using one of those:
my_db = psycopg2.connect(...)
my_db = sqlite3.connect(...)
For the moment, I don't see any possibilty since:
Everyone knows that one should NOT use string concatenation to pass arguments to a SQL request, but rather use placeholders (from psycopg2 docu :never, NEVER use Python string concatenation ... to pass variables to a SQL query string. Not even at gunpoint. )
The synthax for placeholders are different is the 2 APIs psycopg2 and sqlite3. Even for NON-named placeholders. Psycopg uses "%" and sqlite3 uses "?":
my_cursor.execute("SELECT * FROM table WHERE id= ?", (my_id,)) # for SQLITE3
my_cursor.execute("SELECT * FROM table WHERE id= %", (my_id,)) # for PSYCOPG2
One could in principle use the SQL built-in placeholder synthax ("?"
for postgresql), but this would mean precisely preparing a SQL-string with python string concatenation, and so on... that is forbidden by 1.
I'm lacking ideas...

MySQLdb input where strings contain string delimiters

I'm working on a project in Python with MySQLdb. As part of this, I'm moving user details, including salted passwords from one system that generates them to a new one that simply uses them.
Single, double or triple quotes can delineate your string start and end. However, single and double quotes are part of several hashes in the 4.5k or so users I'm migrating. Both tokens appear in about 450 of those salts.
An edited version of the code is as follows
Django.execute ('INSERT INTO auth_user (password) VALUES ("' + user.password + '")')
I have tried swapping between the quote type used in this database cursor object, as and when either quote type or the other are detected, but this still leaves the 150 or so that contain both.
What work arounds can I use for this?
I have tried triple quotes, but they throw a programming error on the cursor object.
Thanks in advance

Query parameters should provide all the proper escaping, for example:
cursor.execute('INSERT INTO auth_user (password) VALUES (%s)', [password])
From the Django docs at: http://docs.djangoproject.com/en/dev/topics/db/sql/
If you're not familiar with
the Python DB-API, note that the SQL
statement in cursor.execute() uses
placeholders, "%s", rather than adding
parameters directly within the SQL. If
you use this technique, the underlying
database library will automatically
add quotes and escaping to your
parameter(s) as necessary. (Also note
that Django expects the "%s"
placeholder, not the "?" placeholder,
which is used by the SQLite Python
bindings. This is for the sake of
consistency and sanity.)

Python: inserting double or single quotes around a string

Im using python to access a MySQL database and im getting a unknown column in field due to quotes not being around the variable.
code below:
cur = x.cnx.cursor()
cur.execute('insert into tempPDBcode (PDBcode) values (%s);' % (s))
rows = cur.fetchall()
How do i manually insert double or single quotes around the value of s?
I've trying using str() and manually concatenating quotes around s but it still doesn't work.
The sql statement works fine iv double and triple check my sql query.

You shouldn't use Python's string functions to build the SQL statement. You run the risk of leaving an SQL injection vulnerability. You should do this instead:
cur.execute('insert into tempPDBcode (PDBcode) values (%s);', s)
Note the comma.

Python will do this for you automatically, if you use the database API:
cur = x.cnx.cursor()
cur.execute('insert into tempPDBcode (PDBcode) values (%s)',s)
Using the DB API means that python will figure out whether to use quotes or not, and also means that you don't have to worry about SQL-injection attacks, in case your s variable happens to contain, say,
value'); drop database; '

If this were purely a string-handling question, the answer would be tojust put them in the string:
cur.execute('insert into tempPDBcode (PDBcode) values ("%s");' % (s))
That's the classic use case for why Python supports both kinds of quotes.
However as other answers & comments have pointed out, there are SQL-specific concerns that are relevant in this case.