cur.execute('UPDATE newslib SET (keyword1) = %s WHERE id= %s' % m_k1,row[0])
I use psycopg2 to execute this SQL statement.
m_k1 and row[0] are two variables I want to pass into the statement.
This triggers the following error message:
TypeError: not enough arguments for format string
I’ve been looking for solutions online, but I can’t fix it.
You need to pass the substitutions as arguments to execute. The way you have it written now you have specified two place holders, but did not provide their values, instead you are parsing them in the query itself.
cur.execute('UPDATE newslib SET (keyword1) = %s WHERE id= %s' , (m_k1,row[0],))
Pass the parameter as a sequence (instead of multiple individual values). And use parameters argument instead of formatting sql yourself.
cur.execute('UPDATE newslib SET (keyword1) = %s WHERE id= %s', [m_k1,row[0]])
# ^^^^^^^^^^^^^^^
See Cursor.execute
Related
I'm trying to execute thi query:
SELECT '23.34.67.0/22' CONCAT(DAY_31, 'hello') DAY_31 FROM Jule
using pymysql. My code is:
cursor.execute("SELECT %s CONCAT(%s, %s) %s FROM Jule", (p, 'DAY_' + _day, as_tmp, 'DAY_' + _day))
But python adds single quote and return a syntax error
"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '('DAY_31', 'hello') 'DAY_31' FROM Jule' at line 1"
DAY_31 is a column of Jule Schema
If memory serves, ? in place of %s might do the trick.
The reason that you end up with quotes is because cursor.execute adds these around values you pass in as arguments. This is totally appropriate for your second argument, because if the value 'hello' was inserted into the query as-is you would end up with a query like this:
SELECT '23.34.67.0/22' CONCAT(DAY_31, hello) DAY_31 FROM Jule
and you would have errors telling you that MySQL can't identify what hello should refer to.
Obviously though, this is not appropriate for situations where you want to pass in field names, or any other part of the query that is not a primitive string value. In these cases, you will need to splice them into your string before executing the query. One way you could do that is with f-strings, but there are other alternatives as well. Here's your cursor.execute line with the field names spliced in using f-strings:
cursor.execute(f"SELECT %s, CONCAT({'DAY_'+_day}, %s) {'DAY_'+_day} FROM Jule", (p, as_tmp))
Notice that I've removed the 'DAY_'+_day from the arguments list as well.
Important note:
While this should work like this (although I think you also needed an extra comma after SELECT '23.34.67.0/22', which I've added in the example above), it's very important that if day has a value that originates from outside of your application (e.g. passed in by a user in a form field) that you make sure it is exactly in the format you want before splicing it in to your query. Checking that the string value is an integer could be one way to do that. The reason that it is important is that without this, your application could be prone to SQL injection, which would potentially allow users to run arbitrary SQL on your database. If the value of day is calculated solely by your application, you shouldn't need to worry about this.
Actually what is happening has nothing to do with cursor.execute. It is just the way you are formatting the string in python.
We should first note that %s %d can be specific to SQL, and that these can also be used in python for string formatting.
So taking this into account, I think you have no need to use %s for string formatting (which on top does not help with readability). And since you are cursor.executeing straight to your database, you could format your string straight away in python.
Small check to see whats happening:
p = "23.34.67.0/22"
as_tmp = "hello"
_day = "3"
print("SELECT %s CONCAT(%s, %s) %s FROM Jule", (p, 'DAY_' + _day, as_tmp, 'DAY_' + _day))
# output
# SELECT %s CONCAT(%s, %s) %s FROM Jule ('23.34.67.0/22', 'DAY_3', 'hello', 'DAY_3')
# ^^^^ so this is what is being sent in cursor.execute (with all the quotes and so on)
If you format with an f-string you will increase readability, and you should get rid of your problem with the quotes
print(f"SELECT '{p}' CONCAT(DAY_{_day}, '{as_tmp}') DAY_{_day} FROM Jule")
# output
# SELECT '23.34.67.0/22' CONCAT(DAY_3, 'hello') DAY_3 FROM Jule
So the solution could be:
cursor.execute(f"SELECT '{p}' CONCAT(DAY_{_day}, '{as_tmp}') DAY_{_day} FROM Jule")
I am aware that queries in Python can be parameterized using either ? or %s in execute query here or here
However I have some long query that would use some constant variable defined at the beginning of the query
Set #my_const = 'xyz';
select #my_const;
-- Query that use #my_const 40 times
select ... coalesce(field1, #my_const), case(.. then #my_const)...
I would like to do the least modif possible to the query from Mysql. So that instead of modifying the query to
pd.read_sql(select ... coalesce(field1, %s), case(.. then %s)... , [my_const, my_const, my_const, ..]
,I could write something along the line of the initial query. Upon trying the following, however, I am getting a TypeError: 'NoneType' object is not iterable
query_str = "Set #null_val = \'\'; "\
" select #null_val"
erpur_df = pd.read_sql(query_str, con = db)
Any idea how to use the original variable defined in Mysql query ?
The reason
query_str = "Set #null_val = \'\'; "\
" select #null_val"
erpur_df = pd.read_sql(query_str, con = db)
throws that exception is because all you are doing is setting null_value to '' and then selecting that '' - what exactly would you have expected that to give you? EDIT read_sql only seems to execute one query at a time, and as the first query returns no rows it results in that exception.
If you split them in to two calls to read_sql then it will in fact return you the value of your #null value in the second call. Due to this behaviour read_sql is clearly not a good way to do this. I strongly suggest you use one of my suggestions below.
Why are you wanting to set the variable in the SQL using '#' anyway?
You could try using the .format style of string formatting.
Like so:
query_str = "select ... coalesce(field1, {c}), case(.. then {c})...".format(c=my_const)
pd.read_sql(query_str)
Just remember that if you do it this way and your my_const is a user input then you will need to sanitize it manually to prevent SQL injection.
Another possibility is using a dict of params like so:
query_str = "select ... coalesce(field1, %(my_const)s, case(.. then %(my_const)s)..."
pd.read_sql(query_str, params={'my_const': const_value})
However this is dependent on which database driver you use.
From the pandas.read_sql docs:
Check your database driver documentation for which of the five syntax
styles, described in PEP 249’s paramstyle, is supported. Eg. for
psycopg2, uses %(name)s so use params={‘name’ : ‘value’}
I am running a [Water] Meter Data Management system on Linux and Python 2.7. I have been reading SO posts like this.
What is the proper, safe syntax for constructing a query and not having to use the less safe method involving %s?
rc = test_cur.execute("select m.* from meter m where m.acct_no = '%s' ", acct_no)
I have tried several ways based on the errors I am getting.
The %s here does not represent a Python string formatting placeholder. In this case, this is a MySQL driver query parameterization placeholder. The placeholder(s) are substituted by the database driver with the parameter(s) passed in a separate argument to execute(). This is different from regular string formatting.
Note that you don't need quotes around the %s in this case - the driver will automatically decide if quotes are needed depending on the parameter type:
rc = test_cur.execute("select m.* from meter m where m.acct_no = %s", (acct_no, ))
Also note the acct_no enclosed in a tuple.
You probably have seen it before, but I'm basically trying to avoid MySQL Injection, so I'm formatting my query as follows using Python:
if "username" in form:
username = form["username"].value
else:
success = 0
error = "User Name is Missing"
cur.execute("SELECT COUNT(*) FROM users WHERE screenName=':1'",[username])
results = int(cur.fetchall()[0][0])
This throws an error saying:
<type 'exceptions.TypeError'>: not all arguments converted during string formatting
args = ('not all arguments converted during string formatting',)
message = 'not all arguments converted during string formatting'
Any idea what's wrong?
Thanks
You don't specify the exact library you're using, but assuming it's Python DB API compliant, you probably want to change the execute line to this:
cur.execute("SELECT COUNT(*) FROM users WHERE screenName=%s",[username])
Edit in response to comment from OP:
The use of %s is the current standard in terms of preventing SQL Injection. I'm not sure what the post in the answer you linked is getting at there... A couple things to keep in mind about that are that the thread has been closed as "not constructive", and that answer is also ~7 years old, so it's quite possible that there were issues back then that that answer may have been referencing that no longer apply.
But %s means that the library (in the execute method) handles all the escaping and quoting, and is the way to prevent injection. (As opposed to, say, using regular interpolation, say, via format, which would leave it exposed to injection.)
Note that is very specifically not '%s' and then using foo % bar in the execute call, but an unquoted %s and passing the parameters as the 2nd argument to execute.
For example, I use psycopg2, which is fully DB API compliant, and its current doc describes using %s to prevent injection.
i have a simple database query by psycopg2
but i do not know why it always show errors
here is the code
ip ="127.0.0.1"
sql="select count(*) from radacct where nasipaddress=%s"
cur.execute(sql,ip)
then it will show
TypeError: not all arguments converted during string formatting
and if i try in this way
cur.execute("select count(*) from radacct where nasipaddress=%s" % ip)
it is still not working
how can i pass the parameters to psycopg2 in the correct way.please help me!
The sql arguments you pass to execute must be in a tuple or list, even if there's only one of them. This is noted in the documentation:
For positional variables binding, the second argument must always be a
sequence, even if it contains a single variable. And remember that
Python requires a comma to create a single element tuple:
So you need to do it like this:
ip ="127.0.0.1"
sql="select count(*) from radacct where nasipaddress=%s"
cur.execute(sql, (ip,))