Python: Iterating through MySQL columns - python

I'm wondering if you can help me. I'm trying to change the value in each column if the text matches a corresponding keyword. This is the loop:
for i in range(0, 20, 1):
cur.execute("UPDATE table SET %s = 1 WHERE text rlike %s") %(column_names[i], search_terms[i])
The MySQL command works fine on its own, but not when I put it in the loop. It's giving an error at the first %s
Does anyone have any insights?
This is the error:
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s = 1 WHERE text rlike %s' at line 1")
Column names looks like
column_names = ["col1","col2","col3"...]
Search terms look like
search_terms = ["'(^| |.|-)word1[;:,. ?-]'","'(^| |.|-)word2[;:,. ?-]'",...]

The right way to do this is to give values to Python, which will quote things correctly.
adapted from voyager's post:
for i in range(0, 20, 1):
cur.execute("UPDATE table SET {} = 1 WHERE text rlike %s".format(column_names[i]),
(search_terms[i],),
)
In this case it's confusing because the column_name isn't a value, it's part of the table structure, so it's inserted using good old string formatting. The search_term is a value, so is passed to cursor.execute() for correct, safe quoting.
(Don't use string manipulation to add the quotes -- you're exposing yourself to SQL injection.)

Missing quotes and wrong parenthesis placement...
for i in range(0, 20, 1):
cur.execute("UPDATE table SET %s = 1 WHERE text rlike '%s'" %(column_names[i], search_terms[i]))
# ^ ^
# (-----------------------------------------------------------------------------------)
Please note, this is not the right way of doing this, if your string may contain quotes by itself...
What about that instead:
for i in range(0, 20, 1):
cur.execute("UPDATE table SET %s = 1 WHERE text rlike ?" % (column_names[i],),
(search_terms[i],))
This uses the % operator to set the column name, but uses an executes parameter to bind the data, letting the DB driver escape all characters that need so.

Related

How to use column name dynamically inside insert query in djnago

cursor.execute("insert into demoapp_site ('name','firstname','lastname') values (%s,%s,%s)",site_data)
in above query I want to pass column name dynamically and %s which is inside values also not predefine that how many times it need to write
so, how can I write my sql query according to my data which comes from html table in 2d array from??
We need to build the query before using it. As follows
field_string = ""
value_string = ""
# This will string in this format "'field1','field2'"
# Assuming array is in format [["field1", "value1"],..]
# Update the for loop statement based on your input, other lines remain same
for field, value in my_field_array:
temp_field_string = "'%s'" % field
field_string = field_string + temp_field_string + ","
temp_value_string = "'%s'" % value
value_string = value_string + temp_value_string + ","
# Removing trailing comma's before passing the values
cmd = "insert into demoapp_site (%s) values (%s)" % (field_string[:-1], value_string[:-1])
cursor.execute(cmd)

How do I use escape caracter in SQL to query with LIKE? (Python)

Trying but failing to use LIKE in my Query.
This works fine, using ? and =:
def read_part_of_database(table_column, user_query):
c_lime.execute("SELECT * FROM lime_database_table"
"WHERE {} = ? ORDER BY time_start".format(table_column), [(user_query)])
for row in c_lime.fetchall():
print(row)
But with this, the user needs to input the exact and full query as it is presented in the database. So instead I want to use LIKE. I have tried the following, but none of it seems to work:
c_lime.execute("SELECT * FROM lime_database_table WHERE {} LIKE %s"
"ORDER BY time_start".format(table_column), ["%" + user_query + "%"])
This gives me error:
c_lime.execute("SELECT * FROM lime_database_table WHERE {} LIKE %s ORDER BY time_start".format(table_column), ["%" + user_query + "%"])
sqlite3.OperationalError: near "%": syntax error
I have tried a few more variations of this, taken from SO and other sources, but none of it seems to work. Is there something I'm doing wrong with the escape %s?
user_query is, when using = ?, a date: 2/14/2015 1:00:00 PM and when using LIKE I take a part of that: 2/14/2015
To use % character in a query with LIKE, enclose it in single quotation marks:
c_lime.execute("""SELECT * FROM lime_database_table WHERE {} LIKE '%{}'
ORDER BY time_start""".format(table_column, user_query))

Trying to insert value from txt file with sqlite in python

def quantity():
i = 0
x = 1
file = open("john.txt", "r")
while i < 5000:
for line in file:
c.execute("INSERT INTO test (playerNAME, playerID) VALUES ("+line+", "+str(x)+")")
conn.commit()
x = random.randint(100,10000000000000000)
i += 1
I try to iterate through the John.txt file and insert each value into a table. The first word in the txt file is "abc123". When I run this code there is an error: sqlite3.OperationalError: no such column: abc123
I can get the code to enter the random numbers into playerID but I can't get the txt file query to work...
You need single quotes around the string.
c.execute("INSERT INTO test (playerNAME, playerID) VALUES ('"+line+"', "+str(x)+")")
Otherwise it tries to interpret it as a sql expression and looks for the named column.
More generally you should use parameters or sanitize the incoming data from the file for safety against sql insertion. Even if you trust this particular file. It's a good habit.
c.execute("INSERT INTO test (playerName, playerID) VALUES (?, ?)", (line, x))
Details are here and here is why it's important.
Formatting sql queries via string concatenation is very bad practice.
Variable bindging should always be used:
c.execute("INSERT INTO test (playerNAME, playerID) VALUES (?, ?)", [line, x])
In your case the line probably contains spaces or any punctuation mark.
The sqlite's error string is misleading, though.

Inserting multi-word string and an empty array with psycopg2

psycopg2 complains when inserting multiple words, empty strings, and empty arrays:
name = "Meal Rounds"
description = ""
sizes = []
cur.execute(""" INSERT INTO items (name, description, sizes) VALUES (%s, %s, %s)""" % (name, description, sizes))
Errors:
# Multi word error
psycopg2.ProgrammingError: syntax error at or near "Rounds"
LINE 1: ... (name, description, sizes) VALUES (Meal Rounds, , ...
^
# Empty string error
psycopg2.ProgrammingError: syntax error at or near ","
LINE 1: ...scription, sizes) VALUES ("Meal Rounds", , [], Fals...
^
# Empty array error
psycopg2.ProgrammingError: syntax error at or near "["
LINE 1: ...n, sizes) VALUES ("Meal Rounds", "None", [], False)...
^
I can get around the multi word error by escaping:
""" INSERT INTO items (name, description, sizes) VALUES (\"%s\", \"%s\", %s)"""
But for tables with 15+ columns, escaping each one is a pain. Does psycopg2 not handle this in an easier fashion? It will still throw errors for empty strings though.
How do I insert multiple words more efficiently, and how to insert empty strings and arrays?
Here is what psql prints out on my columns:
name | character varying(255) |
description | character varying(255) |
sizes | integer[] |
Your call to execute is creating a string with Python string substitution, which is turning out to be invalid SQL. You should be using the parameter substitution provided by the Python DB API:
https://www.python.org/dev/peps/pep-0249/#id15
To call execute using parameter substitution, you pass it two arguments. The first is the query with parameter strings which are database dependent. Psycopg2 uses "pyformat" paramstyle so your query will work as written. The second argument should be the variables you want to substitute into the query. The database driver will handle all the quoting/escaping you need. So your call to execute should be
cur.execute("""INSERT INTO items (name, description, sizes) VALUES (%s, %s, %s)""", (name, description, sizes))

Python database insert

I've been trying to parse a text file (opened with parameter encoding='utf8') and insert extracted values into an mdb database using pyodbc module.
I have tried the code below:
for line in fp:
tokens = line.split('\t')
tokens[4] = tokens[4][:len(tokens[4])-1] #to avoid the \n
tokens[1] = tokens[1][1:] #to remove the 'u' from the beginning of utf8 characters like u'\u0622'
content = conn.execute("INSERT INTO Entries (PForm, WForm, Code, Freq, Pattern) VALUES ("+tokens[0]+","+tokens[1]+","+tokens[2]+","+tokens[3]+","+tokens[4]+")")
conn.commit()
and received the following error:
Error: ('07002', '[07002] [Microsoft][ODBC Microsoft Access Driver] Too few parameters. Expected 4. (-3010) (SQLExecDirectW)')
P.S. the first line of my file is: آ 'A Ab 1 S
And the other lines are of the same format.
Your comments will be appreciated :)
You don't put quotes around the strings which you want to insert. Assuming the "Freq" row is of type INTEGER:
stmt = """
INSERT INTO Entries (PForm, WForm, Code, Freq, Pattern)
VALUES ('%s', '%s', '%s', %s, '%s')
"""
params = tuple(t for t in tokens)
conn.execute(stmt % params)
But anyway, you shouldn't be formatting an INSERT statement like this. Doesn't the library you're using provide a facility to parameterize statements ? Something like this:
conn.execute("INSERT INTO Foo VALUES (?, ?, ?)", (foo, bar, baz))

Categories