I try to get the number of movies that include the word "the" in their title, by LIKE clause. in Python connectors MySQL:
word='the'
query = """ SELECT COUNT(title) from movies WHERE title LIKE '%%%s%%' """ % (word,)
cursor.execute(query)
# error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SELECT COUNT(title) from movies WHERE title LIKE '%the%'' at line 1
p = "the"
query = ("SELECT COUNT(title) from movies WHERE title LIKE", ("%" + p + "%",))
cursor.execute(query,(p,))
# AttributeError: 'tuple' object has no attribute 'encode'
You would do:
query = "SELECT COUNT(title) from movies WHERE title LIKE CONCAT('%', %s, '%')";
cursor.execute(query, (word,))
This uses a proper prepared statement to pass the variables to the query, so the code is safe from SQL injection, and more efficient.
You could also do the concatenation of wildcard on application side:
query = "SELECT COUNT(title) from movies WHERE title LIKE %s";
cursor.execute(query, ('%' + word + '%',))
Related
I have a string (sql query) in which I want to pass a variable at one point, then pass another variable at another point (list of variables, but just focusing on one for now).
The expected would be something like this:
sql_query = 'SELECT {{field}} FROM {table} WHERE {{field}} IS NULL'.format(table=table_name)
sql_query should now be: 'SELECT {field} FROM table_name WHERE {field} IS NULL'
Then format for field
sql_query = sql_query.format(field_name)
In theory I want sql_query to now be: 'SELECT field_name FROM table_name WHERE field_name IS NULL'
But the above ignores the .format and I get: 'SELECT {field} FROM table_name WHERE {field} IS NULL'
I have tried combining f-strings and .format in multiple ways and the closest to my goal is:
field = field_name
sql_query = f'SELECT {field} FROM {{0}} WHERE {field} IS NULL'.format(table_name)
# Works and I get sql_query : 'SELECT field_name FROM table_name WHERE field_name IS NULL'
The above works but it happens all in the same place and separating where each one happens is the true goal of mine.
sql = "SELECT {{column}} FROM {table}"
sql = sql.format(table="my_table")
print(sql)
sql = sql.format(column="my_column")
print(sql)
Or...
sql = "SELECT {column} FROM {table}"
sql = sql.format(table="my_table", column="{column}")
print(sql)
sql = sql.format(column="my_column")
print(sql)
That said, I'd recommend not actually passing the string around and doing partial replacements on it.
Instead, pass a dictionary around, add the replacements to the dictionary, and call format just once...
Then you don't need to add arbitrary {} around the token you don't want to replace, or be aware that it exists in order to replace it with itself.
sql = "SELECT {column} FROM {table}"
parts = dict()
parts["table"] = "my_table"
# more code here
parts["column"] = "my_column"
print(sql.format(**parts))
NOTE:
As per other warnings here...
NEVER do this with user supplied text.
Literal values should be supplied using parameterisation / prepared statements.
Only ever do this when you are in complete control of the potential values in the dictionary, such as deriving the columns, etc, from ORM meta-data, or some other white list.
Im using python3 and postgres 11.5.
This is the script :
a = cursor.execute("SELECT tablename FROM pg_catalog.pg_tables limit 5")
for table in a:
cursor.execute("SELECT * FROM pg_prewarm(public.%s)", [table[0]])
a query gets some table names , and the loop query should run table name as the %s.
but for some reason i get the arg table[0] with // /n in the query and its messing it up.
if i print a results i get table names as tuple:
[('sa1591354519',), ('sa1591397719',), ('sa1591397719',)]
so [table[0]] is a string.
the error i get:
1574683839 [16177], ERR, execute ({'Error while connecting to PostgreSQL': SyntaxError('syntax error at or near "\'sa1591440919\'"\nLINE 1: SELECT * FROM pg_prewarm(public.\'sa1591440919\')\n ^\n')},)
what can i do ?
The errors don't have anything to do with the newlines you see, which are just an artifact of the error message. If you were to print out the error, would see:
syntax error at or near "'sa1591440919'"
LINE 1: SELECT * FROM pg_prewarm(public.'sa1591440919')
^
In other words, Postgres doesn't like the table name you're passing because it contains quotes. This is happening because you're trying to treat the table names like a normal query parameter, which causes psycopg to quote them...but that's not what you want in this case.
Just replace your use of query templating with normal Python string substitution:
a = cursor.execute("SELECT tablename FROM pg_catalog.pg_tables limit 5")
for table in a:
cursor.execute("SELECT * FROM pg_prewarm(public.%s)" % (table[0]))
But this won't actually work, because cursor.execute doesn't return a value, so a will be None. You would need to do something like:
cursor.execute("SELECT tablename FROM pg_catalog.pg_tables limit 5")
a = cursor.fetchall()
for table in a:
...
I want to fetch all rows from MySQL table with
query = "SELECT * FROM %s WHERE last_name=%s"
cursor.execute(query, ("employees","Smith"))
but I'm getting
You have an error in your SQL syntax. When I try
query = "SELECT * FROM employees WHERE last_name=%s"
cursor.execute(query, ("Smith",))
all is fine.
Documentation says
cursor.execute(operation, params=None, multi=False)
The parameters found in the tuple or dictionary params are bound to the variables in the operation.link on docs
The first will generate an SQL like this:
SELECT * FROM 'employees' WHERE last_name='smith'
The parameters are SQL quoted.
If you really need to have a table name as param, you must proceed in 2 steps:
table_name = 'employees'
query_tpl = "SELECT * FROM {} WHERE last_name=%s"
query = query_tpl.format(table_name)
cursor.execute(query, ("Smith",))
you need to add the quote symbol. So the query will be like
SELECT * FROM employees WHERE last_name='Smith'
Change both your query to
query = "SELECT * FROM %s WHERE last_name='%s'"
query = "SELECT * FROM employees WHERE last_name='%s'"
You can't use a parameter for the table name in the execute call.
But you can use Python string interpolation for that:
query = "SELECT * FROM %s WHERE last_name=%s" %("employees","Smith")
cursor.execute(query)
You can't use a table name as a parameter. you are generating invalid sql with your code that is putting quotes around each string. the table name cannot have quotes around it.
sql you are generating
select * from 'employees' where last_name = 'Smith'
What sql you want
select * from employees where last_name = 'Smith'
you would have to format the string first like the example below.
query = "SELECT * from {} wherre last_name ='{}'"
cursor.execute(query.format("employees","Smith"))
using code like this does open up the possibility of SQL injection. so please bear that in mind.
query="SELECT * FROM %s WHERE name=%s",(employees,smith)
cursor.execute(query)
rows = cursor.fetchall()
Try this one. Hopefully it works for you.
I am trying to create a statement as follows:
SELECT * FROM table WHERE provider IN ('provider1', 'provider2', ...)
However, I'm having some trouble with the string formatting of it from the Django API. Here's what I have so far:
profile = request.user.get_profile()
providers = profile.provider.values_list('provider', flat=True) # [u'provider1', u'provider2']
providers = tuple[str(item) for item in providers] # ('provider1', 'provider2')
SQL = "SELECT * FROM table WHERE provider IN %s"
args = (providers,)
cursor.execute(sql,args)
DatabaseError
(1241, 'Operand should contain 1 column(s)')
MySQLdb has a method to help with this:
Doc
string_literal(...)
string_literal(obj) -- converts object obj into a SQL string literal.
This means, any special SQL characters are escaped, and it is enclosed
within single quotes. In other words, it performs:
"'%s'" % escape_string(str(obj))
Use connection.string_literal(obj), if you use it at all.
_mysql.string_literal(obj) cannot handle character sets.
Usage
# connection: <_mysql.connection open to 'localhost' at 1008b2420>
str_value = connection.string_literal(tuple(provider))
# '(\'provider1\', \'provider2\')'
SQL = "SELECT * FROM table WHERE provider IN %s"
args = (str_value,)
cursor.execute(sql,args)
Another answer that I don't like particularly, but will work for your apparent use-case:
providers = tuple[str(item) for item in providers] # ('provider1', 'provider2')
# rest of stuff...
SQL = 'SELECT * FROM table WHERE provider IN {}'.format(repr(providers))
cursor.execute(SQL)
You should probably do the string replacement before passing it to the cursor object to execute:
sql = "SELECT * FROM table WHERE provider IN (%s)" % \
(','.join(str(x) for x in providers))
cursor.execute(sql)
So, you have string input for ID's required:
some_vals = '1 3 5 76 5 4 2 5 7 8'.split() # convert to suitable type if required
SomeModel.objects.filter(provider__in=some_vals)
"SELECT * FROM table WHERE provider IN ({0},{1},{2})".format(*args) #where args is list or tuple of arguments.
try this.... should work.
SQL = "SELECT * FROM table WHERE provider IN %s"%(providers)
exec 'cursor.execute("%s")'%(SQL)
I'm having a hard time getting some sql in python to correctly go through MySQLdb. It's pythons string formatting that is killing me.
My sql statement is using the LIKE keyword with wildcards. I've tried a number of different things in Python. The problem is once I get one of them working, there's a line of code in MySQLdb that burps on string format.
Attempt 1:
"SELECT tag.userId, count(user.id) as totalRows FROM user INNER JOIN
tag ON user.id = tag.userId WHERE user.username LIKE '%%s%'" % (query)
This is a no go. I get value error:
ValueError: unsupported format character ''' (0x27) at index 128
Attempt 2:
"SELECT tag.userId, count(user.id) as totalRows FROM user INNER JOIN
tag ON user.id = tag.userId WHERE user.username LIKE '\%%s\%'" %
(query)
I get the same result from attempt 1.
Attempt 3:
like = "LIKE '%" + str(query) + "%'" totalq = "SELECT tag.userId,
count(user.id) as totalRows FROM user INNER JOIN tag ON user.id =
tag.userId WHERE user.username " + like
This correctly creates the totalq variable, but now when I go to run the query I get errors from MySQLdb:
File "build/bdist.macosx-10.6-universal/egg/MySQLdb/cursors.py", line
158, in execute query = query % db.literal(args) TypeError: not enough
arguments for format string
Attempt 4:
like = "LIKE '\%" + str(query) + "\%'" totalq = "SELECT tag.userId,
count(user.id) as totalRows FROM user INNER JOIN tag ON user.id =
tag.userId WHERE user.username " + like
This is the same output as attempt 3.
This all seems really strange. How can I use wildcards in sql statements with python?
Those queries all appear to be vulnerable to SQL injection attacks.
Try something like this instead:
curs.execute("""SELECT tag.userId, count(user.id) as totalRows
FROM user
INNER JOIN tag ON user.id = tag.userId
WHERE user.username LIKE %s""", ('%' + query + '%',))
Where there are two arguments being passed to execute().
It's not about string formatting but the problem is how queries should be executed according to db operations requirements in Python (PEP 249)
try something like this:
sql = "SELECT column FROM table WHERE col1=%s AND col2=%s"
params = (col1_value, col2_value)
cursor.execute(sql, params)
here are some examples for psycog2 where you have some explanations that should also be valid for mysql (mysqldb also follows PEP249 dba api guidance 2.0: here are examples for mysqldb)
To escape ampersands in Python string formatting expressions, double the ampersand:
'%%%s%%' % search_string
Edit: But I definitely agree with another answer. Direct string substitution in SQL queries is almost always a bad idea.
import mysql.connector
mydatabase = mysql.connector.connect(host="localhost", user="root", passwd="1234", database="databaseName")
mycursor = mydatabase.cursor()
user_input =[]
item = str("s%")
user_input.append(item)
mycursor.execute("SELECT * FROM employees WHERE FIRST_NAME LIKE %s ESCAPE ''",user_input )
result = mycursor.fetchall()
for row in enumerate(result):
print(row)
I used the following and it worked:
my_str = 'abc'
query = f"""select * from my_table where column_a like '%%{my_str}%%' """
df=pandas.read_sql_query(query, engine)
We could try escaping the percentage character by doubling them like this:
query_to_get_user_name = """
SELECT tag.userId, count(user.id) as totalRows FROM user INNER JOIN tag
ON user.id = tag.userId
WHERE user.username LIKE '%%%s%%' """ % (user_name,)
cursor.execute(query_to_get_user_name)
So I tried this and I think I have got the answer which is simpler for me to understand , a school student .
In my code the table name is "books" and the column I'm Searching for is "Name".
If you need more xplaination , then feel free to drop a mail at dhruv2003.joshi#gmail.com and I will try my best to answer ASAP
def S():
n=str(input('Enter the name of the book: '))
name='%'+n+'%'
NAME=name
query="select * from books where Name like '"+NAME+"' "
c.execute(query)
ans=c.fetchall()
if len(ans)>0:
print('')
for i in ans:
print(i)
print('')
else:
print('')
print('An error occured')
print('Name you gave does not exist :( ')
print('')
I have a solution to your problem :
You can not use :
"SELECT tag.userId, count(user.id) as totalRows FROM user INNER JOIN tag ON user.id = tag.userId WHERE user.username LIKE '%%s%'" % (query)
you can change it with string template, such as :
import MySQLdb
import string # string module
.......
value = {'user':'your value'}
sql_template = string.Template("""
SELECT tag.userId, count(user.id) as totalRows FROM user INNER JOIN
tag ON user.id = tag.userId WHERE user.username LIKE '%$user%'
""")
sql = sql_template.substitute(value)
try:
cursor.execute(sql)
...........
except:
...........
finally :
db.close()