We Keep Coding

Given table and column name, how to test if INSERT needs quotes ('') around the values to be inserted?

I have a dictionary of column name / values, to insert into a table. I have a function that generates the INSERT statement. I'm stuck because the function always puts quotes around the values, and some are integers.
e.g. If column 1 is type integer then the statement should be INSERT INTO myTable (col1) VALUES 5; vs
INSERT INTO myTable (col1) VALUES '5'; second one causes an error saying column 5 does not exist.
EDIT: I found the problem (I think). the value was in double quotes not single, so it was "5".
In Python, given a table and column name, how can I test if the INSERT statement needs to have '' around the VALUES ?

This question was tagged with "psycopg2" -- you can prepare the statement using a format string and have psycopg2 infer types for you in many cases.
cur.execute('INSERT INTO myTable (col1, col2) VALUES (%s, %s);', (5, 'abc'))
psycopg2 will deal with it for you, because Python knows that 5 is an integer and 'abc' is a string.
http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries

You certainly want to use a library function to decide whether or not to quote values you insert. If you are inserting anything input by a user, writing your own quoting function can lead to SQL Injection attacks.
It appears from your tags that you're using psycopg2 - I've found another response that may be able to answer your question, since I'm not familiar with that library. The main gist seems to be that you should use
cursor.execute("query with params %s %s", ("param1", "pa'ram2"))
Which will automatically handle any quoting needed for param1 and param2.

Although I personally don't like the idea, you can use single quotes around integers when you insert in Postgres.
Perhaps your problem is the lack of parentheses:
INSERT INTO myTable(col1)
VALUES('5');
Here is a SQL Fiddle illustrating this code.
As you note in the comments, double quotes do not work in Postgres.

You can put always the single quote (be careful, if the value contents a quote you must double it: insert into example (value_t) values ('O''Hara');
You can decide checking the value that you want to insert regardles of the type of de destination
You can decide checking the type of the target field
As you can see in http://sqlfiddle.com/#!15/8bfbd/3 theres no mater with inserting integers into a text field or string that represents an integer in a numeric field.
To check the field type you can use the information_schema:
select data_type from information_schema.columns
where table_schema='public'
and table_name='example'
and column_name='value_i';
http://sqlfiddle.com/#!15/8bfbd/7

dynamic table mysqldb python string/int issue

I am receiving an error when trying to write data to a database table when using a variable for the table name that I do not get when using a static name. For some reason on the line where I insert, if I insert an integer as the column values the code runs and the table is filled, however, if I try to use a string I get a SQL syntax error
cursor = db.cursor()
cursor.execute('DROP TABLE IF EXISTS %s' %data[1])
sql ="""CREATE TABLE %s (IP TEXT, AVAILIBILITY INT)""" %data[1]
cursor.execute(sql)
for key in data[0]:
cur_ip = key.split(".")[3]
cursor.execute("""INSERT INTO %s VALUES (%s,%s)""" %(data[1],key,data[0][key]))
db.commit()
the problem is where I have %(data[1], key, data[0][key]) any ideas?

It's a little hard to analyse your problem when you don't post the actual error, and since we have to guess what your data actually is. But some general points as advise:
Using a dynamic table name is often not way DB-systems want to be used. Try thinking if the problem could be used by using a static table name and adding an additional key column to your table. Into that field you can put what you did now as a dynamic table name. This way the DB might be able to better optimize your queries, and your queries are less likely to get errors (no need to create extra tables on the fly for once, which is not a cheap thing to do. Also you would not have a need for dynamic DROP TABLE queries, which could be a security risk.
So my advice to solve your problem would be to actually work around it by trying to get rid of dynamic table names altogether.
Another problem you have is that you are using python string formatting and not parameters to the query itself. That is a security problem in itself (SQL-Injections), but also is the problem of your syntax error. When you use numbers, your expression evaluates to
INSERT INTO table_name VALUES (100, 200)
Which is valid SQL. But with strings you get
INSERT INTO table_name VALUES (Some Text, some more text)
which is not valid (since you have no quotes ' around the strings.
To get rid of your syntax problem and of the sql-injection-problem, don't add the values to the string, pass them as a list to execute():
cursor.execute("INSERT INTO table_name VALUES (%s,%s)", (key, data[0][key]))
If you must have a dynamic table name, put that in your query string first (e.g. with % formatting), and give the actual values for your query as parameters as above (since I cannot imagine that execute will accept the table name as a parameter).
To put it in some simple sample code. Right now you are trying to do it like this:
# don't do this, this won't even work!
table_name = 'some_table'
user_name = 'Peter Smith'
user_age = 47
query = "INSERT INTO %s VALUES (%s, %s)" % (table_name, user_name, user_age)
cursor.execute(query)
That creates query
INSERT INTO some_table VALUES (Peter Smith, 100)
Which cannot work, because of the unquoted string. So you needed to do:
# DON'T DO THIS, it's bad!
query = "INSERT INTO %s VALUES ('%s', %s)" % (table_name, user_name, user_age)
That's not a good idea, because you need to know where to put quotes and where not (which you will mess up at some point). Even worse, imagine a user named named Connor O'Neal. You would get a syntax error:
INSERT INTO some_table VALUES ('Connor O'Neal', 100)
(This is also the way sql-injections are used to crush your system / steal your data). So you would also need to take care of escaping the values that are strings. Getting more complicated.
Leave those problems to python and mysql, by passing the date (not the table name) as arguments to execute!
table_name = 'some_table'
user_name = 'Peter Smith'
user_age = 47
query = "INSERT INTO " + table_name + " VALUES (%s, %s)"
cursor.execute(query, (user_name, user_age))
This way you can even pass datetime objects directly. There are other ways to put the data than using %s, take a look at this examples http://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html (that is python3 used there, I don't know which you use - but except of the print statements it should work with python2 as well, I think).

Python SQL statement with variables, using psycopg2 library

cur.execute('UPDATE newslib SET (keyword1) = %s WHERE id= %s' % m_k1,row[0])
I use psycopg2 to execute this SQL statement.
m_k1 and row[0] are two variables I want to pass into the statement.
This triggers the following error message:
TypeError: not enough arguments for format string
I’ve been looking for solutions online, but I can’t fix it.

You need to pass the substitutions as arguments to execute. The way you have it written now you have specified two place holders, but did not provide their values, instead you are parsing them in the query itself.
cur.execute('UPDATE newslib SET (keyword1) = %s WHERE id= %s' , (m_k1,row[0],))

Pass the parameter as a sequence (instead of multiple individual values). And use parameters argument instead of formatting sql yourself.
cur.execute('UPDATE newslib SET (keyword1) = %s WHERE id= %s', [m_k1,row[0]])
# ^^^^^^^^^^^^^^^
See Cursor.execute

Python: inserting double or single quotes around a string

Im using python to access a MySQL database and im getting a unknown column in field due to quotes not being around the variable.
code below:
cur = x.cnx.cursor()
cur.execute('insert into tempPDBcode (PDBcode) values (%s);' % (s))
rows = cur.fetchall()
How do i manually insert double or single quotes around the value of s?
I've trying using str() and manually concatenating quotes around s but it still doesn't work.
The sql statement works fine iv double and triple check my sql query.

You shouldn't use Python's string functions to build the SQL statement. You run the risk of leaving an SQL injection vulnerability. You should do this instead:
cur.execute('insert into tempPDBcode (PDBcode) values (%s);', s)
Note the comma.

Python will do this for you automatically, if you use the database API:
cur = x.cnx.cursor()
cur.execute('insert into tempPDBcode (PDBcode) values (%s)',s)
Using the DB API means that python will figure out whether to use quotes or not, and also means that you don't have to worry about SQL-injection attacks, in case your s variable happens to contain, say,
value'); drop database; '

If this were purely a string-handling question, the answer would be tojust put them in the string:
cur.execute('insert into tempPDBcode (PDBcode) values ("%s");' % (s))
That's the classic use case for why Python supports both kinds of quotes.
However as other answers & comments have pointed out, there are SQL-specific concerns that are relevant in this case.

Escape SQL "LIKE" value for Postgres with psycopg2

Does psycopg2 have a function for escaping the value of a LIKE operand for Postgres?
For example I may want to match strings that start with the string "20% of all", so I want to write something like this:
sql = '... WHERE ... LIKE %(myvalue)s'
cursor.fetchall(sql, { 'myvalue': escape_sql_like('20% of all') + '%' }
Is there an existing escape_sql_like function that I could plug in here?
(Similar question to How to quote a string value explicitly (Python DB API/Psycopg2), but I couldn't find an answer there.)

Yeah, this is a real mess. Both MySQL and PostgreSQL use backslash-escapes for this by default. This is a terrible pain if you're also escaping the string again with backslashes instead of using parameterisation, and it's also incorrect according to ANSI SQL:1992, which says there are by default no extra escape characters on top of normal string escaping, and hence no way to include a literal % or _.
I would presume the simple backslash-replace method also goes wrong if you turn off the backslash-escapes (which are themselves non-compliant with ANSI SQL), using NO_BACKSLASH_ESCAPE sql_mode in MySQL or standard_conforming_strings conf in PostgreSQL (which the PostgreSQL devs have been threatening to do for a couple of versions now).
The only real solution is to use the little-known LIKE...ESCAPE syntax to specify an explicit escape character for the LIKE-pattern. This gets used instead of the backslash-escape in MySQL and PostgreSQL, making them conform to what everyone else does and giving a guaranteed way to include the out-of-band characters. For example with the = sign as an escape:
# look for term anywhere within title
term= term.replace('=', '==').replace('%', '=%').replace('_', '=_')
sql= "SELECT * FROM things WHERE description LIKE %(like)s ESCAPE '='"
cursor.execute(sql, dict(like= '%'+term+'%'))
This works on PostgreSQL, MySQL, and ANSI SQL-compliant databases (modulo the paramstyle of course which changes on different db modules).
There may still be a problem with MS SQL Server/Sybase, which apparently also allows [a-z]-style character groups in LIKE expressions. In this case you would want to also escape the literal [ character with .replace('[', '=['). However according to ANSI SQL escaping a character that doesn't need escaping is invalid! (Argh!) So though it will probably still work across real DBMSs, you'd still not be ANSI-compliant. sigh...

I was able to escape % by using %% in the LIKE operand.
sql_query = "select * from mytable where website like '%%.com'"
cursor.fetchall(sql_query)

If you're using a prepared statement, then the input will be wrapped in '' to prevent sql injection. This is great, but also prevents input + sql concatenation.
The best and safest way around this would be to pass in the %(s) as part of the input.
cursor.execute('SELECT * FROM goats WHERE name LIKE %(name)s', { 'name': '%{}%'.format(name)})

You can also look at this problem from a different angle. What do you want? You want a query that for any string argument executes a LIKE by appending a '%' to the argument. A nice way to express that, without resorting to functions and psycopg2 extensions could be:
sql = "... WHERE ... LIKE %(myvalue)s||'%'"
cursor.execute(sql, { 'myvalue': '20% of all'})

I found a better hack. Just append '%' to your search query_text.
con, queryset_list = psycopg2.connect(**self.config), None
cur = con.cursor(cursor_factory=RealDictCursor)
query = "SELECT * "
query += " FROM questions WHERE body LIKE %s OR title LIKE %s "
query += " ORDER BY questions.created_at"
cur.execute(query, ('%'+self.q+'%', '%'+self.q+'%'))

I wonder if all of the above is really needed. I am using psycopg2 and was simply able to use:
data_dict['like'] = psycopg2.Binary('%'+ match_string +'%')
cursor.execute("SELECT * FROM some_table WHERE description ILIKE %(like)s;", data_dict)

Instead of escaping the percent character, you could instead make use of PostgreSQL's regex implementation.
For example, the following query against the system catalogs will provide a list of active queries which are not from the autovacuuming sub-system:
SELECT procpid, current_query FROM pg_stat_activity
WHERE (CURRENT_TIMESTAMP - query_start) >= '%s minute'::interval
AND current_query !~ '^autovacuum' ORDER BY (CURRENT_TIMESTAMP - query_start) DESC;
Since this query syntax doesn't utilize the 'LIKE' keyword, you're able to do what you want... and not muddy the waters with respect to python and psycopg2.

Having failed to find a built-in function so far, the one I wrote is pretty simple:
def escape_sql_like(s):
return s.replace('\\', '\\\\').replace('%', '\\%').replace('_', '\\_')

You can create a Like class subclassing str and register an adapter for it to have it converted in the right like syntax (e.g. using the escape_sql_like() you wrote).

I made some modifications to the code above to do the following:
def escape_sql_like(SQL):
return SQL.replace("'%", 'PERCENTLEFT').replace("%'", 'PERCENTRIGHT')
def reescape_sql_like(SQL):
return SQL.replace('PERCENTLEFT', "'%").replace('PERCENTRIGHT', "%'")
SQL = "SELECT blah LIKE '%OUCH%' FROM blah_tbl ... "
SQL = escape_sql_like(SQL)
tmpData = (LastDate,)
SQL = cur.mogrify(SQL, tmpData)
SQL = reescape_sql_like(SQL)
cur.execute(SQL)

It just requires to concatenate double % before and after it. Using "ilike" instead of "like" makes it case insensitive.
query = """
select
*
from
table
where
text_field ilike '%%' || %(search_text)s || '%%'
"""

I think it would be simpler and more readable to use f-strings.
query = f'''SELECT * FROM table where column like '%%{my_value}%%' '''
cursor.execute(query)