am trying to execute the below code using python 2.5.2. The script is establishing the connection and creating the table, but then its failing with the below error.
The script
import pymssql
conn = pymssql.connect(host='10.103.8.75', user='mo', password='the_password', database='SR_WF_MODEL')
cur = conn.cursor()
cur.execute('CREATE TABLE persons(id INT, name VARCHAR(100))')
cur.executemany("INSERT INTO persons VALUES(%d, %s)", \
[ (1, 'John Doe'), (2, 'Jane Doe') ])
conn.commit()
cur.execute("SELECT * FROM persons WHERE salesrep='%s'", 'John Doe')
row = cur.fetchone()
while row:
print "ID=%d, Name=%s" % (row[0], row[1])
row = cur.fetchone()
cur.execute("SELECT * FROM persons WHERE salesrep LIKE 'J%'")
conn.close()
The error
Traceback (most recent call last):
File "connect_to_mssql.py", line 9, in <module>
cur.execute("SELECT * FROM persons WHERE salesrep='%s'", 'John Doe')
File "/var/lib/python-support/python2.5/pymssql.py", line 126, in execute
self.executemany(operation, (params,))
File "/var/lib/python-support/python2.5/pymssql.py", line 152, in executemany
raise DatabaseError, "internal error: %s" % self.__source.errmsg()
pymssql.DatabaseError: internal error: None
any suggestions? plus, how do you read the traceback error, anyone can help me understand the error message? how do you read it? bottom up?
I think you are assuming the regular python string interpolation behavior, ie:
>>> a = "we should never do '%s' when working with dbs"
>>> a % 'this'
"we should never do 'this' when working with dbs"
The % operator within the execute method looks like the normal string formatting operator but that is more of a convenience or mnemonic; your code should read:
cur.execute("SELECT * FROM persons WHERE salesrep=%s", 'John Doe')
without the quotes, and this will work with names like O'Reilly, and help prevent SQL injection per the database adapter design. This is really what the database adapter is there for -- converting the python objects into sql; it will know how to quote a string and properly escape punctuation, etc. It would work if you did:
>>> THING_ONE_SHOULD_NEVER_DO = "select * from table where cond = '%s'"
>>> query = THING_ONE_SHOULD_NEVER_DO % 'john doe'
>>> query
"select * from table where cond = 'john doe'"
>>> cur.execute(query)
but this is bad practice.
Related
The code is the following (I am new to Python/Mysql):
import mysql.connector
conn = mysql.connector.connect(host='localhost',user='user1',password='puser1',db='mm')
cursor = conn.cursor()
string1 = 'test1'
insert_query = """INSERT INTO items_basic_info (item_name) VALUES (%s)""", (string1)
cursor.execute(insert_query)
conn.commit()
When I run this code I get this error:
Traceback (most recent call last)
File "test3.py", line 9, in <module>
cursor.execute(insert_query)
File "C:\Users\Emanuele-PC\AppData\Local\Programs\Python\Python36\lib\site-packages\mysql\connector\cursor.py", line 492, in execute
stmt = operation.encode(self._connection.python_charset)
AttributeError: 'tuple' object has no attribute 'encode'
I have seen different answers to this problem but the cases were quite different from mine and I couldn't really understand where I am making mistakes. Can anyone help me?
For avoid SQL-injections Django documentation fully recommend use placeholders like that:
import mysql.connector
conn = mysql.connector.connect(host='localhost',user='user1',password='puser1',db='mm')
cursor = conn.cursor()
string1 = 'test1'
insert_query = """INSERT INTO items_basic_info (item_name) VALUES (%s)"""
cursor.execute(insert_query, (string1,))
conn.commit()
You have to pass tuple/list params in execute method as second argument. And all should be fine.
Not exactly OP's problem but i got stuck for a while writing multiple variables to MySQL.
Following on from Jefferson Houp's answer, if adding in multiple strings, you must specify the argument 'multi=True' in the 'cursor.execute' function.
import mysql.connector
conn = mysql.connector.connect(host='localhost',user='user1',password='puser1',db='mm')
cursor = conn.cursor()
string1 = 'test1'
string2 = 'test2'
insert_query = """INSERT INTO items_basic_info (item_name) VALUES (%s, %s)"""
cursor.execute(insert_query, (string1, string2), multi=True)
conn.commit()
I am trying to pattern match with the LIKE LOWER('% %') command however I think the fact that I am using a python variable with %s is mucking it up. I can't seem to find any escape characters for the percentage symbol and my program gives me no errors. Is this the problem or is there something else I'm missing. It does work if I just run LIKE %s however I need to be able to search like not equals.
# Ask for the database connection, and get the cursor set up
conn = database_connect()
if(conn is None):
return ERROR_CODE
cur = conn.cursor()
print("search_term: ", search_term)
try:
# Select the bays that match (or are similar) to the search term
sql = """SELECT fp.name AS "Name", fp.size AS "Size", COUNT(*) AS "Number of Fish"
FROM FishPond fp JOIN Fish f ON (fp.pondID = f.livesAt)
WHERE LOWER(fp.name) LIKE LOWER('%%s%') OR LOWER(fp.size) LIKE LOWER('%%s%')
GROUP BY fp.name, fp.size"""
cur.execute(sql, (search_term, ))
rows = cur.fetchall()
cur.close() # Close the cursor
conn.close() # Close the connection to the db
return rows
except:
# If there were any errors, return a NULL row printing an error to the debug
print("Error with Database - Unable to search pond")
cur.close() # Close the cursor
conn.close() # Close the connection to the db
return None
Instead of embedding the ampersands in the query string, you could wrap the search term string in ampersands, and then pass that to cursor.execute():
sql = 'SELECT * from FishPond fp WHERE LOWER(fp.name) LIKE LOWER(%s)'
search_term = 'xyz'
like_pattern = '%{}%'.format(search_term)
cur.execute(sql, (like_pattern,))
The query is simplified for the purpose of example.
This is more flexible because the calling code can pass any valid LIKE pattern to the query.
BTW: In Postgresql you can use ILIKE for case insensitive pattern matching, so the example query could be written as this:
sql = 'SELECT * from FishPond fp WHERE fp.name ILIKE %s'
As noted in the documentation ILIKE is a Postgresql extension, not standard SQL.
You can escape % with another %
>>> test = 'test'
>>> a = 'LIKE %%s%'
>>> a % test
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: incomplete format
>>>
>>> a = 'LIKE %%%s%%'
>>> a % test
'LIKE %test%'
P.S. you also have two placeholders, but you are passing only one argument in execute
I have a series of SQL select statements that I need to run from a Python script using MySQLdb. One of the variables I would like to pass to the select statements is called unit_ids. I've tried treating unit_ids as a string as well as a tuple of strings. Initially, backslashes were being inserted into the string. After looking around online, I've been able to avoid the backslashes, but now extra quotation marks are being inserted instead. Here is my current code:
connection = MySQLdb.connect('localhost', 'root', '*****', 'test')
cur = connection.cursor
unit_ids = ('0A1', '0A2', '0A3', '0A4')
attr = 'sample'
cur.execute("""SELECT COUNT(*) FROM test WHERE attribute = %s AND unit_id IN %r""", (a, tuple(unit_ids)))
Using cur._last_executed, I can see that the actual SQL statement that was performed was:
SELECT COUNT(*) FROM test WHERE attribute = 'sample' AND unit_id IN ("'0A1'", "'0A2'", "'0A3'", "'0A4'")
Any ideas on what I need to change in order to have ('0A1', '0A2', '0A3', '0A4') remain unchanged in the SQL statement?
UPDATE: Here's the exact output I get even when using %s:
>>> cn = MySQLdb.connect('localhost', 'root', '***', '***')
>>> c = cn.cursor()
>>> unit_ids = ('0A1', '0A2', '0A3', '0A4')
>>> a = 'foo'
>>> c.execute("""select count(*) from model_test where attribute = %s and unit_id in %s""", (a, unit_ids))
1L
>>> print(c._last_executed)
select count(*) from model_test where attribute = 'foo' and unit_id in ("'0A1'", "'0A2'", "'0A3'", "'0A4'")
At this point, I think I may just need to create separate variables for each element of unit_ids (eg unit_id1 = '0A1'). I'm using Python 2.7.9 and MySQL Server 5.6, by the way.
UPDATE 2: #thebjorn solved it: my version of MySQLdb was outdated. After upgrading, there were no more extra quotation marks being inserted into the SQL statement.
You don't need any magic, just do it the regular mysql way:
connection = MySQLdb.connect('localhost', 'root', '*****', 'test')
cur = connection.cursor()
unit_ids = ('0A1', '0A2', '0A3', '0A4')
attr = 'sample'
cur.execute("""SELECT COUNT(*) FROM test WHERE attribute = %s AND unit_id IN %s""", (a, unit_ids))
the only wrinkle I can see is if a was not included and unit_ids only had one item, then the tuple syntax could trip you up. If you make unit_ids into a list then the syntax wouldn't be quite as awkward:
unit_ids = ('0A1',)
cur.execute("SELECT COUNT(*) FROM test WHERE unit_id IN %s", (unit_ids,))
which when inlined becomes:
cur.execute("SELECT COUNT(*) FROM test WHERE unit_id IN %s", (('0A1',),))
vs. using lists (one parameter, that parameter is a one item list):
cur.execute("SELECT COUNT(*) FROM test WHERE unit_id IN %s", [['0A1']])
You can use %s for all mysql parameters (a few others can be used too, but not %r -- this isn't string interpolation).
Update: you must be doing something differently from me.. here is the output of cursor._last_executed
>>> cn = MySQLdb.connect('server', 'user', 'password', 'database')
>>> c = cn.cursor()
>>> unit_ids = ('0A1', '0A2', '0A3', '0A4')
>>> c.execute("select count(*) from foo where id in %s", (unit_ids,))
Traceback (most recent call last):
File ...
_mysql_exceptions.ProgrammingError: (1146, "Table 'djangodevelop.foo' doesn't exist")
>>> c._last_executed
"select count(*) from foo where id in ('0A1','0A2','0A3','0A4')"
>>>
Do not use %r in unit_id IN %r. Python-Mysql database API only supports %s as place holder when building SQL queries.
From the docs
In the preceding example, we store the SELECT statement in the variable query. Note that we are using unquoted %s-markers where dates should have been. Connector/Python converts hire_start and hire_end from Python types to a data type that MySQL understands and adds the required quotes. In this case, it replaces the first %s with '1999-01-01', and the second with '1999-12-31'.
You can see similar warning and usage in Python-MySql docs.
cur.execute("""
SELECT COUNT(*) FROM test WHERE attribute = %s AND unit_id IN %s
""", (a, ('0A1', '0A2', '0A3', '0A4')))
I have this small decorator function where I am going to find an unknown number of entries in a table:
def Deco(func):
func
conn = sqlite3.connect('/home/User/vocab_database/vocab.db')
with conn:
cur = conn.cursor()
cur.execute("SELECT name FROM sqlite_master WHERE type='table'")
total = cur.fetchall()
print "You have %d tables " % len(total)
## this line below, is where I wanted to use a formatted string ##
cur.execute("SELECT * FROM %s") % total[0]
entries = cur.fetchall()
print "You have %d entries" % len(entries)
Then I get this error message:
Traceback (most recent call last):
File "./database_trial.py", line 19, in <module>
class Vocab:
File "./database_trial.py", line 25, in Vocab
#Deco
File "./database_trial.py", line 15, in Deco
cur.execute("SELECT * FROM %s") % total[0]
sqlite3.OperationalError: near "%": syntax error
Does sqlite3 only accept ? operators? Or is there something I'm mucking up?
You're trying to replace metadata, so unfortunately a parametrized query won't work. You must use interpolation or the like here, but make sure that the value is sanitized; this is a possible vector for a SQL injection.
cur.execute("SELECT * FROM %s" % (total[0],))
In the line cur.execute("SELECT * FROM %s") % total[0], you are applying the % operator to the result of the cur.execute call. I think you want to do the substitution inside the call, e.g., cur.execute("SELECT * FROM ?", (total[0],)).
I am trying to store some parsed feed contents values in Sqlite database table in python.But facing error.Could anybody help me out of this issue.Infact it is so trivial question to ask!I am newbie!..Anyway thanks in advance!
from sqlite3 import *
import feedparser
data = feedparser.parse("some url")
conn = connect('location.db')
curs = conn.cursor()
curs.execute('''create table location_tr
(id integer primary key, title text ,
updated text)''')
for i in range(len(data['entries'])):
curs.execute("insert into location_tr values\
(NULL, data.entries[i].title,data.feed.updated)")
conn.commit()
curs.execute("select * from location_tr")
for row in curs:
print row
And Error is:
Traceback (most recent call last):
File "F:\JavaWorkspace\Test\src\sqlite_example.py", line 16, in <module>
(NULL, data.entries[i].title,data.feed.updated)")
sqlite3.OperationalError: near "[i]": syntax error
Try
curs.execute("insert into location_tr values\
(NULL, '%s', '%s')" % (data.entries[i].title, data.feed.updated))
the error should be this line
curs.execute("insert into location_tr values\
(NULL, data.entries[i].title,data.feed.updated)")
data.entries[i].title comes from Python. So if you enclose it in double quotes, it becomes a literal string, not a value. It should be something like this:
curs.execute("insert into location_tr values (NULL," + data.entries[i].title +","+data.feed.updated+")")